Display system, and head-mounted display system

Abstract

The present disclosure generally relates to techniques for improving the performance and efficiency of display systems, such as laser scanning beam display systems or other types of display systems (e.g., microdisplays) in HMDs or other devices. The display systems of the present disclosure may include polarization compensation optics, such as a spatially varying polarizer, that provides a position-varying phase retardation, resulting in light that is well-suited for the polarization-sensitive optics of the display system, such as waveguide-based optics, pancake optics, birdbath optics, coating-based optics, and the like. The spatially varying polarizer can vary spatially in real time based on the user's gaze position to provide an optimized field of view to areas where the user is known or suspected to be gazing, resulting in improved optical performance.

Description

Technical Field

[0001] The present disclosure generally relates to display systems, and more particularly to improving the efficiency and performance of display systems. [Description of Related Art]

[0002] One current generation of virtual reality (“VR”) experiences is generated using a head-mounted display (“HMD”), which can be tethered to a stationary computer (such as a personal computer (“PC”), laptop, or gaming console), combined with and / or integrated with a smartphone and / or its associated display, or made self-contained. Generally, an HMD is a display device worn on the user's head and has a small display device in front of one eye (monocular HMD) or each eye (binocular HMD). The display unit is typically miniaturized and can include, for example, a CRT, LCD, liquid crystal on silicon (LCoS), OLED device, or laser scan beam display. A binocular HMD may be capable of displaying different images to each eye. This function is used to display stereoscopic images.

[0003] The demand for higher performance displays has increased with the development of smartphones, high-resolution televisions, and other electronic devices. Such demand has further increased with the growing popularity of virtual reality and augmented reality systems, particularly systems using HMDs. A virtual reality system typically completely covers both eyes of the wearer and substitutes a “virtual” reality for the actual visual field or physical visual field (or actual reality) in front of the wearer, while an augmented reality system typically provides a translucent or transparent overlay of one or more screens in front of both eyes of the wearer such that the actual visual field is augmented with additional information, and similarly in a mediated reality system, information combining real-world elements and virtual elements can be presented to the viewer.

Summary of the Invention

[0004] A display system is a pupil relay system arranged to relay a first pupil from a display light source to a second pupil in a viewer's eye, the pupil relay system including a polarization-sensitive optical system and a spatially-varying polarizer having spatially-varying polarization that varies according to position to provide polarization compensation for the polarization-sensitive optical system, and a control circuit operably connected to the spatially-varying polarizer, the control circuit being configured to receive line-of-sight information indicating a current or predicted line-of-sight position of a user from an eye-tracking subsystem and to selectively adjust the varying polarization of the spatially-varying polarizer based at least in part on the received line-of-sight information. The spatially-varying polarizer may include a plurality of layers, each of the plurality of layers being independently controllable by the control circuit to be in an active state or an inactive state, and during operation, the control circuit controls the states of the plurality of layers to selectively adjust the varying polarization of the spatially-varying polarizer. Each of the plurality of layers may have a spatially-varying polarization different from at least one of the others of the plurality of layers. The control circuit may optimize the spatially-varying polarizer for a first region related to the user's line-of-sight position with respect to a second region outside the first region. The control circuit may control the spatially-varying polarizer to provide a relatively high luminance in a first region related to the user's line-of-sight position with respect to a second region outside the first region. The control circuit may control the spatially-varying polarizer to provide a relatively low amount of stray light in a first region related to the user's line-of-sight position with respect to a second region outside the first region. The control circuit may selectively adjust the varying polarization of the spatially-varying polarizer in real time when the received line-of-sight information is updated by the eye-tracking subsystem.

[0005] The display system may further include an eye-tracking subsystem that generates eye line information. The spatially-varying polarizer may include a multi-twist retarder. The phase difference of the spatially-varying polarizer may vary according to a horizontal dimension or a vertical dimension. The phase difference of the spatially-varying polarizer may vary across the entire field of view of the display system. The polarization-sensitive optical system may include a waveguide-based optical system, a pancake optical system, a birdbath optical system, or a coating-based optical system.

[0006] The display light source may include a laser light source, and the display system may further include a scan mirror arranged to receive a light beam from the laser light source and relay the received light toward a pupil relay system.

[0007] The display system may further include a beam-forming optical system arranged between the laser light source and the scan mirror. At least a part of the spatially-varying polarizer may be arranged on, adjacent to, or within the polarization-sensitive optical system. The polarization-sensitive optical system may include a waveguide, and the spatially-varying polarizer may be arranged on, inside the waveguide, or proximate to a port of the waveguide. The display system may be a display system of a head-mounted display system.

[0008] A head-mounted display system includes a support structure and a display system coupled to the support structure. The display system includes a display light source and a pupil relay system arranged to relay a first pupil from the display light source to a second pupil in the viewer's eye. The pupil relay system includes a polarization-sensitive optical system and a spatially-varying polarizer having spatially-varying polarization that varies according to position to provide polarization compensation for the polarization-sensitive optical system. The display system further includes a control circuit operably coupled to the spatially-varying polarizer. The control circuit is configured to receive line-of-sight information indicating the user's current or predicted line-of-sight position from an eye-tracking subsystem and selectively adjust the varying polarization of the spatially-varying polarizer based at least in part on the received line-of-sight information. The spatially-varying polarizer may include a plurality of layers, each of the plurality of layers being independently controllable by the control circuit to be in an active or non-active state. During operation, the control circuit controls the states of the plurality of layers to selectively adjust the varying polarization of the spatially-varying polarizer. Each of the plurality of layers may have a spatially-varying polarization different from at least one of the other of the plurality of layers. The control circuit may optimize the spatially-varying polarizer for a first region related to the user's line-of-sight position with respect to a second region outside the first region. The control circuit may control the spatially-varying polarizer to provide a relatively high luminance in a first region related to the user's line-of-sight position with respect to a second region outside the first region. The control circuit may control the spatially-varying polarizer to provide a relatively low amount of stray light in a first region related to the user's line-of-sight position with respect to a second region outside the first region. The control circuit may selectively adjust the varying polarization of the spatially-varying polarizer in real time when the received line-of-sight information is updated by the eye-tracking subsystem. The spatially-varying polarizer may include a multi-twist retarder. The polarization-sensitive optical system may include a waveguide-based optical system, a pancake optical system, a birdbath optical system, or a coating-based optical system.

[0009] The display light source may include a laser light source, and the display system may further include a scan mirror arranged to receive a light beam from the laser light source and relay the received light toward the pupil relay system.

[0010] The head-mounted display system may further include a beam-forming optical system arranged between the laser light source and the scan mirror. The polarization-sensitive optical system may include a waveguide, and the spatially variable polarizer may be arranged on, inside, or proximate to a port of the waveguide.

[0011] A head-mounted display system includes a support structure and a display system coupled to the support structure. The display system includes a laser light source, a scan mirror arranged to receive a light beam from the laser light source, and a pupil relay system arranged to relay a first pupil received from the scan mirror to a second pupil in the viewer's eye. The pupil relay system includes a polarization-sensitive optical system and a spatially-varying polarizer having a spatially-varying polarization that varies according to position to provide polarization compensation for the polarization-sensitive optical system. The head-mounted display system may be summarized as further including an eye tracking subsystem operable to generate eye line-of-sight information indicative of the user's current or predicted line-of-sight position, and a control circuit operably coupled to the spatially-varying polarizer. The control circuit is configured to receive the eye line-of-sight information from the eye tracking subsystem when the user views the display of the display system and selectively adjust the varying polarization of the spatially-varying polarizer based at least in part on the received eye line-of-sight information. The spatially-varying polarizer may include a plurality of layers, each of the plurality of layers being independently controllable by the control circuit to be in an active state or a non-active state. In operation, the control circuit controls the states of the plurality of layers to selectively adjust the varying polarization of the spatially-varying polarizer. Each of the plurality of layers may have a spatially-varying polarization different from at least one of the other of the plurality of layers. The spatially-varying polarizer may include a multi-twist retarder. The polarization-sensitive optical system may include a waveguide-based optical system, a pancake optical system, a birdbath optical system, or a coating-based optical system. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] In this drawing, like elements or operations are identified by like reference numerals. The sizes and relative positions of the elements in the drawing are not necessarily drawn to scale. For example, the shapes and angles of the various elements are not necessarily drawn to scale, and some of these elements may be arbitrarily enlarged and arranged to improve the visibility of the drawing. Further, a particular shape of a drawn element is not necessarily intended to convey any information regarding the actual shape of a particular element, and may merely be selected to make the drawing recognizable.

[0013]

Figure 1

[0014]

Figure 2

[0015]

Figure 3

[0016]

Figure 4

[0017]

Figure 5

[0018]

Figure 6

[0019]

Figure 7

[0020]

Figure 8

[0021]

Figure 9A

[0022]

Figure 9B

[0023]

Figure 10

[0024]

Figure 11

[0025]

Figure 12

DETAILED DESCRIPTION OF THE INVENTION

[0026] In the following description, specific details are set forth in order to provide a thorough understanding of the various implementations disclosed. However, one of ordinary skill in the art will recognize that these implementations may be practiced without one or more of these specific details, or with other methods, components, materials, etc. In other instances, well-known structures associated with computer systems, server computers, and / or communication networks have not been shown or described in detail to avoid obscuring the description of the implementations.

[0027] Unless the context requires otherwise, throughout this specification and the following claims, the word "comprising" is synonymous with "including" and is inclusive or non-exclusive (i.e., it does not exclude additional, unrecited elements or method acts).

[0028] References throughout this specification to "one implementation" or "an implementation" mean that a particular feature, structure, or characteristic described in connection with that implementation is included in at least one implementation. Thus, appearances of the phrases "in one implementation" or "in an implementation" in various places throughout this specification are not necessarily all referring to the same implementation. Further, in one or more implementations, the particular features, structures, or characteristics may be combined in any suitable manner.

[0029] In this specification and the appended claims, the singular forms "a", "an", and "the" include plural referents unless the context clearly dictates otherwise. Note also that the term "or" is generally used herein in its sense including "and / or" unless the context clearly dictates otherwise.

[0030] Each item and summary provided in this specification is for convenience only and does not interpret the scope or meaning of the implementations.

[0031] The present disclosure generally relates to techniques for improving the performance and efficiency of display systems, such as laser scan beam display systems or other types of display systems (e.g., microdisplays). As will be further described below, at least some embodiments of the present disclosure improve the performance of a display system by providing a spatially variant polarizer that results in a phase delay that varies depending on position. This provides polarization compensation and generates light that is well suited for a polarization-sensitive optical system of a display system, such as a waveguide-based optical system, a pancake optical system, a birdbath optical system, a coating-based optical system, etc. Such techniques significantly improve the intensity of light passing through the polarization-sensitive optical system and also suppress unwanted stray light. Advantageously, the polarization of the spatially variant polarizer can be selectively varied in real time based on the known or inferred line-of-sight position of the user. This may be achieved by providing a plurality of layers of spatially variant polarizers stacked together, each of these layers having a different spatially variant polarization and being independently controllable to be in an active or inactive state. In many types of optical systems, it may be difficult to provide polarization correction across the entire field of view, but it may be possible to provide significant polarization correction in a relatively narrow field of view. Thus, using line-of-sight information, it is possible to optimize the relatively narrow field of view or region that the user is gazing at to provide improved brightness, suppressed stray light, or other characteristics that enhance the user's viewing experience. The line-of-sight information may be obtained in any manner, such as via an eye-tracking subsystem. Alternatively or additionally, the line-of-sight information may be inferred based on information about the content being displayed. For example, if a particular object is the target of an object in the displayed scene, the system may optimize the field of view that includes that object because the user is likely to be gazing in that direction.

[0032] First, with reference to FIGS. 1-4, an exemplary application of the head-mounted display device of the method described in this specification will be described. Next, with reference to FIGS. 5-12, an exemplary implementation of a display system including a spatial and temporal variable polarization compensation optical system will be described. [Exemplary Head-Mounted Display System and Environment]

[0033] FIG. 1 is a schematic diagram of a networked environment 100 including a local media rendering (LMR) system 110 (e.g., a gaming console), which includes a local computing system 120 and a display device 180 (e.g., an HMD device with two display panels) suitable for executing at least some of the methods described in this specification. In the illustrated embodiment of FIG. 1, the local computing system 120 is communicatively connected to the display device 180 via a transmission link 115 (the display device may be wired or tethered, such as via one or more cables (cable 220) as illustrated in FIG. 2, or alternatively wirelessly connected). In other embodiments, the local computing system 120 may provide encoded display image data via a wired or wireless link to a panel display device (e.g., a TV, console, or monitor) in addition to or instead of the HMD device 180, each display device including one or more addressable pixel arrays. In various embodiments, the local computing system 120 may include a general-purpose computing system, a gaming console, a video stream processing device, a mobile computing device (e.g., a mobile phone, PDA, or other mobile device), a VR processing device or an AR processing device, or other computing systems.

[0034] In the illustrated embodiment, the local computing system 120 includes one or more hardware processors (e.g., a central processing unit or "CPU") 125, a memory 130, various I / O ("input / output") hardware components 127 (e.g., a keyboard, a mouse, one or more game controllers, speakers, microphones, IR transmitters and / or receivers, etc.), a video subsystem 140 including one or more dedicated hardware processors (e.g., a graphics processing unit or "GPU") 144 and video memory (VRAM) 148, a computer-readable storage 150, and a network connection 160. Also, in the illustrated embodiment, one embodiment of the eye-tracking subsystem 135 is executed in the memory 130 using the CPU 125 and / or the GPU 144 to perform an automated process that implements at least some of the described techniques for performing at least some of the described techniques, and the memory 130 may optionally further execute one or more other programs 133 (e.g., a game program that generates a displayed video or other image). As part of the automated process for implementing at least some of the techniques described herein, the eye-tracking subsystem 135 and / or the program 133 executed in the memory 130 may store or retrieve various types of data including those within the data structure of an exemplary database of the storage 150. In this example, the data used may include various types of image data information within a database ("DB") 154, various types of application data within the DB 152, various types of configuration data within the DB 157, and may also include additional information such as system data or other information.

[0035] In the illustrated embodiment, the LMR system 110 may also be communicatively connected to an exemplary network-accessible media content provider 190 that may further provide content to the LMR system 110 for display via one or more computer networks 101 and network links 102, whether in addition to or instead of the image generation program 133. The media content provider 190 may include one or more computing systems (not shown) having components similar to those of the local computing system 120, including one or more hardware processors, I / O components, local storage devices, and memory. For simplicity, some details of the network-accessible media content provider are not illustrated.

[0036] The display device 180 is illustrated as being different from and separate from the local computing system 120 in the illustrated embodiment of FIG. 1, but it will be understood that in certain embodiments, some or all of the components of the local media rendering system 110 may be integrated or housed within one device, such as a portable gaming machine, a portable VR entertainment system, an HMD device, etc. In such embodiments, the transmission link 115 may include, for example, one or more system buses and / or video bus architectures.

[0037] As an example involving processing performed locally by the local media rendering system 120, assume that the local computing system is a game computing system that includes one or more game applications in which application data 152 is executed by the CPU 125 using the memory 130, and in which various video frame display data is generated and / or processed by an image generation program 133, such as in cooperation with the GPU 144 of the video subsystem 140. To provide a high-quality gaming experience, a large amount of video frame data (corresponding to a high image resolution for each video frame and a high "frame rate" of about 60 to 180 per second for such video frames) is generated by the local computing system 120 and provided to the display device 180 via a wired or wireless transmission link 115.

[0038] The computing system 120 and the display device 180 are merely examples and are not intended to limit the scope of the present disclosure. The computing system 120 may instead include a plurality of computing systems or devices that interact with each other, and may be connected to other devices not illustrated, such as via one or more networks such as the Internet, via the web, or via a private network (such as a mobile communication network, etc.). More generally, a computing system or other computing node may include any combination of hardware or software capable of interacting with each other to perform the types of functions described, including, but not limited to, desktop or other computers, gaming machines, database servers, network storage devices and other network devices, PDAs, mobile phones, wireless phones, pagers, electronic organizers, Internet appliances, television-based systems (such as using set-top boxes and / or personal / digital video recorders), and various other consumer products including appropriate communication capabilities. The display device 180 may similarly include one or more devices having one or more display panels of various types and forms, and optionally may include various other hardware components and / or software components.

[0039] Although various items are illustrated as being stored in and used from memory or storage, it will be understood that some or all of these items may be transferred between memory and other storage devices for purposes of memory management or data integrity. Thus, in some embodiments, some or all of the described techniques may be performed by one or more processors or other configured hardware circuits or hardware including memory or storage (e.g., by executing software instructions of one or more software programs and / or by storing such software instructions and / or data structures), such as when constituted by one or more software programs and / or data structures. Some or all of the components, systems, and data structures may also be stored on a non-transitory computer-readable storage medium (e.g., as software instructions or structured data), such as a hard disk or flash drive or other non-volatile storage device, volatile memory or non-volatile memory (e.g., RAM), network storage device, or portable media article (e.g., DVD disk, CD disk, optical disk, etc.) readable by an appropriate drive or via an appropriate connection. The systems, components, and data structures may also, in some embodiments, be transmitted in various computer-readable transmission media including wireless-based media and wired / cable-based media as generated data signals (e.g., as part of a carrier wave or other analog or digital propagated signal), and may take various forms (e.g., as part of one or multiplexed analog signals or as multiple individual digital packets or frames). Such computer program products may take other forms in other embodiments. Accordingly, the invention may be practiced using other computer system configurations.

[0040] Figure 2 illustrates an exemplary environment 200 that provides a virtual reality display to a human user 206 using at least some of the techniques described, along with an exemplary HMD device 202 coupled to a video rendering computing system 204 via a tether connection 220 (or a wireless connection in other embodiments). The user wears the HMD device 202 and receives display information of a simulated environment different from the actual physical environment from the computing system 204 via the HMD device, and the computing system acts as an image rendering system that supplies an image of the simulated environment (e.g., an image generated by a game program and / or other software programs running on the computing system) to the HMD device for display to the user. In this example, the user can further move around within the tracking volume 201 of the actual physical environment 200 and may further have one or more I / O ("input / output") devices (including the handheld controllers 208 and 210 in this example) that enable the user to further interact with the simulated environment.

[0041] In the illustrated example, the environment 200 may include one or more base stations 214 (two are shown labeled as base stations 214a and 214b) that may facilitate tracking of the HMD device 202 or the controllers 208 and 210. As the user moves locations or changes the orientation of the HMD device 202, the position of the HMD device is tracked, such as to enable the corresponding portion of the simulated environment to be displayed to the user wearing the HMD device, and the controllers 208 and 210 may use a similar technique for use in tracking the position of the controllers (and optionally, for using information useful in determining or verifying the position of the HMD device). After the tracked position of the HMD device 202 is known, the corresponding information is transmitted to the computing system 204 via the tether 220 or wirelessly. The computing system uses the tracked position information to generate one or more next images of the simulated environment and display them to the user.

[0042] There are a number of different methods for position tracking that can be used in various implementations of the present disclosure, including, but not limited to, acoustic tracking, inertial tracking, magnetic tracking, optical tracking, combinations thereof, and the like.

[0043] In at least some implementations, the HMD device 202 may include one or more light receivers or sensors that can be used to implement the tracking functionality or other aspects of the present disclosure. For example, each base station 214 may sweep an optical signal across the entire tracking volume 201. Depending on the requirements of each particular implementation, each base station 214 may generate more than one optical signal. For example, for six-degree-of-freedom tracking, usually one base station 214 is sufficient, but for some embodiments, multiple base stations (e.g., base stations 214a, 214b) may be necessary or desirable to provide stable room-scale tracking for the HMD device and peripheral devices. In this example, the light receivers are incorporated into the HMD device 202 and / or other objects to be tracked (such as controllers 208 and 210). In at least some implementations, on each device to be tracked, the light receivers may be combined with accelerometers and gyroscope inertial measurement units (「IMUs」) to support low-latency sensor fusion.

[0044] In at least some implementations, each base station 214 includes two rotors that sweep a linear beam along the vertical axis across the entire tracking volume 201. At the start of each sweep cycle, the base station 214 may emit an omnidirectional optical pulse (referred to as a 「synchronization signal」) that is visible to all sensors attached to the object to be tracked. In this way, each sensor calculates its unique angular position in the swept volume by measuring the duration between the synchronization signal and the beam signal. The distance and orientation of the sensor may be determined using multiple sensors attached to one fixed object.

[0045] One or more sensors disposed on the object to be tracked (e.g., HMD device 202, controllers 208 and 210) may include optoelectronic devices capable of detecting modulated light from the rotator. In the case of visible light or near-infrared (NIR) light, silicon photodiodes and suitable amplifier / detector circuits may be used. Since environment 200 may include static signals and time-varying signals (optical noise) having a wavelength similar to that of the base station 214 signal, in at least some implementations, the light from the base station may be modulated in such a way as to facilitate discrimination from any interfering signals and / or to facilitate selection of the sensor from any emission wavelengths other than the wavelength of the base station signal.

[0046] Inside-out tracking is also a type of position tracking that can be used to track the position of the HMD device 202 and / or other objects (e.g., controllers 208 and 210, tablet computer, smartphone). Inside-out tracking differs from outside-in tracking in that it determines the position of the HMD using the position of a camera or other sensor. In inside-out tracking, the camera or sensor is located on the HMD or the object being tracked, whereas in outside-out tracking, the camera or sensor is placed at a fixed position within the environment.

[0047] HMDs that utilize inside-out tracking use one or more "looking-out" cameras to determine how their position is changing relative to the environment. As the HMD moves, the sensors re-align its origin point within the room, and the virtual environment responds in real time accordingly. This type of position tracking can be achieved regardless of the presence of markers placed within the environment. The cameras placed on the HMD observe the characteristics of the surrounding environment. When using markers, the markers are designed to be easily detected by the tracking system and are placed in specific areas. In "markerless" inside-out tracking, the HMD system uses prominent features (e.g., topography) that already exist within the environment to determine position and orientation. The algorithms of the HMD system identify specific images or shapes and use them to calculate the position of the device within the space. Data from accelerometers and gyroscopes can also be used to enhance the accuracy of position tracking.

[0048] FIG. 3 shows information 300 that illustrates a front view of an exemplary HMD device 344 when worn on the head of user 342. The HMD device 344 includes a front structure 343 that supports a front or forward camera 346 and a plurality of sensors 348a-348d (collectively 348) of one or more types. As one example, an optical sensor that detects and uses optical information emitted from some or all of the sensors 348, for example, from one or more external devices (not shown, e.g., base station 214 of FIG. 2), can help determine the position and orientation of device 344 within the space. As shown, the forward camera 346 and sensors 348 are directed forward toward an actual scene or environment (not shown) in which user 342 operates the HMD device 344. The actual physical environment can include, for example, one or more objects (e.g., walls, ceilings, furniture, stairs, vehicles, trees, tracking markers, or any other type of object). A particular number of sensors 348 may be fewer or more than the number of sensors shown. The HMD device 344 may further include one or more additional components that are not attached to the front structure (e.g., are inside the HMD device), such as an IMU (inertial measurement unit) electronic device 347 that measures and reports a particular force, angular velocity, and / or magnetic field surrounding the HMD device 344 (e.g., using a combination of an accelerometer and a gyroscope, and optionally a magnetometer). The HMD device may further include additional components not shown, including one or more display panels and optical lens systems that face the user's eyes (not shown), and optionally one or more built-in motors for adjusting the alignment or other positioning of one or both of the optical lens system and / or display panel within the HMD device. This will be described in more detail below with respect to FIG. 4.

[0049] The illustrated example of the HMD device 344 is supported on the head of the user 342, at least in part, based on one or more straps 345 that are attached to the housing of the HMD device 344 and extend, either wholly or in part, to surround the user's head. Although not illustrated here, the HMD device 344 may further include one or more external motors, such as being attached to one or more of the straps 345, and adjusting such straps using such motors to correct the alignment or other positioning of the HMD device on the user's head may be included in an auto-correction operation. The HMD device may include other support structures (e.g., nose pads, chin straps, etc.), not illustrated here, in addition to or instead of the illustrated straps, and some embodiments include one or more such other support structures to which motors are attached, and similarly, it will be understood that the shape and / or position thereof may be adjusted to correct the alignment or other positioning of the HMD device on the user's head. Other display devices that are not attached to the user's head may similarly be attached to one or more structures, or portions thereof, that affect the positioning of the display device, and such other devices include motors or other mechanical actuators in at least some embodiments, and similarly, the shape and / or position thereof may be modified to correct the alignment or other positioning of the display device with respect to one or more pupils of one or more users of the display device.

[0050] Figure 4 illustrates a simplified plan view 400 of an HMD device 405 including a pair of near-to-eye display systems 402 and 404. The HMD device 405 may be the same as or similar to the HMD devices illustrated in FIGS. 1-3, or it may be a different HMD device, and the HMD device described herein may further be used in the examples further described below. The near-to-eye display systems 402 and 404 of FIG. 4 each include a display panel 406 and 408 (e.g., an OLED microdisplay), and each of the optical lens systems 410 and 412 having one or more optical lenses. The display systems 402 and 404 may be installed in a housing (or frame) 414 or otherwise disposed therein, and the housing includes a front portion 416 (e.g., the same as or similar to the front surface 343 of FIG. 3), a left temple 418, a right temple 420, and an inner surface 421 that contacts or is close to the face of the user 424 who is the wearer when the user wears the HMD device. The two display systems 402 and 404 that can be worn on the head 422 of the user 424 who is the wearer may be fixed to the housing 414 in a glasses configuration, and the left temple 418 and the right temple 420 may be placed on the user's ears 426 and 428, respectively, and the nose pad 492 may be placed on the user's nose 430. In the example of FIG. 4, the HMD device 405 may be supported on the user's head partially or entirely by the nose pad and / or the left and right over-ear type temples, but in some embodiments, as in the embodiments shown in FIGS. 2 and 3, a strap (not shown) or other structure may be used to fix the HMD device to the user's head. The housing 414 may be made in a shape and size such that each of the two optical lens systems 410 and 412 is disposed in front of one of the user's eyes 432 and 434, whereby the target position of each pupil 494 will be centered both vertically and horizontally in front of each optical lens system and / or display panel.The housing 414 is shown in a simplified manner similar to glasses for purposes of illustration, but it should be understood that in practice, more elaborate structures (e.g., goggles, integrated headbands, helmets, straps, etc.) can be used to support the display systems 402 and 404 and position them on the user 424's head 422.

[0051] The HMD device 405 of FIG. 4, and other HMD devices described herein, can present virtual reality displays to a user via corresponding video presented at a display rate such as 30 or 60 or 90 frames (or images) per second. In other embodiments of similar systems, augmented reality displays may be presented to the user. The displays 406 and 408 of FIG. 4 may each generate light that is focused onto the eyes 432 and 434 of the user 424 by their respective optical lens systems 410 and 412 after being carried through those optical lens systems. The aperture of the pupil 494 of each eye allows light to enter the eye, and typically, the pupil size can range from 2 millimeters (mm) in a very bright state to 8 mm in a dark state, and the size of the larger iris that includes the pupil can be about 12 mm. The pupil (and the surrounding iris) can also typically move a few millimeters horizontally and / or vertically within the visible portion of the eye when the eyelid is open, which, when the eyeball rotates about its center (resulting in a 3D volume in which the pupil can move), will also move the pupil to different depths relative to the optical lens of the display or other physical elements at different horizontal and vertical positions. The user 424 sees the light entering the user's pupil as an image and / or video. In some implementation examples, the distance between each of the optical lens systems 410 and 412 and the user's eyes 432 and 434 can be relatively short (e.g., less than 30 mm, less than 20 mm), which, since the weight of the optical lens system and the display system is relatively close to the user's face, is advantageous in that the HMD device will feel lighter to the user and can also provide a larger field of view to the user. Although not illustrated here, some embodiments of such HMD devices may include various additional built-in sensors and / or external sensors.

[0052] In the illustrated embodiment, the HMD device 405 of FIG. 4 further includes hardware sensors and additional components, such as including one or more accelerometers and / or gyroscopes 490 (e.g., as part of one or more IMU units). As described in more detail elsewhere herein, values from the accelerometers and / or gyroscopes may be used to locally determine the orientation of the HMD device. Further, the HMD device 405 may include one or more front cameras, e.g., camera 485 on the outer surface of the front portion 416, and the information may be used as part of the operations of the HMD device, such as for providing an AR function or a positioning function. Further, the HMD device 405 may further include other components 475 (e.g., electronic circuits for controlling the display of images to the display panels 406 and 408, built-in storage, one or more batteries, a position tracking device for communicating with an external base station, etc.), which are described in more detail elsewhere herein. Other embodiments may not include one or more of the components 475, 485, and / or 490. Although not illustrated here, some embodiments of such HMD devices may include various additional built-in sensors and / or external sensors for tracking various other types of movements and positions of the user's body, eyes, controller, etc.

[0053] In the illustrated embodiment, the HMD device 405 of FIG. 4 further includes hardware sensors and additional components that can be used by the disclosed embodiment as part of the described techniques to determine the direction of the user's pupil or line of sight, which may be provided for use by one or more components associated with the HMD device. This is described elsewhere in this specification. The hardware sensors in this example include, for example, for obtaining information about the actual position of the user's pupil 494, separately for each pupil in this example, one or more eye-tracking assemblies 472 located on or near the display panels 406 and 408 and / or on the inner surface 421 near the optical lens systems 410 and 412.

[0054] Each of the eye-tracking assemblies 472 may include one or more light sources (e.g., IR LEDs) and one or more light detectors (e.g., silicon photodiodes). Further, for clarity, although only a total of four eye-tracking assemblies 472 are shown in FIG. 4, it should be understood that in practice, a different number of eye-tracking assemblies may be provided. In some embodiments, a total of eight eye-tracking assemblies 472 are provided, i.e., four eye-tracking assemblies for each eye of the user 424. Further, in at least some implementations, each eye-tracking assembly includes a light source directed at one of the user's eyes 432 and 434, a light detector arranged to receive light reflected by each of the user's eyes, and a polarizer arranged and configured such that light reflected by specular reflection does not reach the light detector.

[0055] As described in more detail elsewhere in this specification, information from the eye tracking assembly 472 may be used to determine and track the direction of the user's line of sight during use of the HMD device 405. Further, in at least some embodiments, the HMD device 405 may include one or more built-in motors 438 (or other movement mechanisms) that can be used to move (e.g., in the vertical, horizontal left-right, and / or horizontal forward-backward directions) the alignment and / or other positioning of one or both of the optical lens systems 410 and 412 and / or the display panels 406 and 408 within the housing of the HMD device 405, which is for personalizing or otherwise adjusting the target pupil positions of one or both of the near-to-eye display systems 402 and 404 corresponding to the actual position of one or both of the pupils 494. Such a motor 438 may be controlled, for example, by the user operating one or more control buttons 437 on the housing 414 and / or by the user operating one or more associated separate I / O controllers (not shown). In other embodiments, the HMD device 405 may control the alignment and / or other positioning of the optical lens systems 410 and 412 and / or the display panels 406 and 408 using an adjustable positioning mechanism (e.g., a screw, slider, ratchet, etc.) that can be manually changed by the user using the control button 437, without using such a motor 438. Further, although FIG. 4 illustrates a motor 438 for only one of the near-to-eye display systems, in some embodiments, each near-to-eye display system may have its own one or more motors, and in some embodiments, one or more motors may be used to control each of the plurality of near-to-eye display systems (e.g., independently).

[0056] The described techniques may be used with a display system similar to that illustrated in some embodiments, but in other embodiments, other types of display systems may be used, including, for example, those having one optical lens and a display device, or those having a plurality of such optical lenses and display devices. Non-exclusive examples of other such devices include cameras, telescopes, microscopes, binoculars, spotting scopes, surveying scopes, and the like. Further, the described techniques may be used with a wide variety of display panels or other display devices that emit light to form an image, and as described elsewhere herein, that image is viewed by one or more users through one or more optical lenses. In other embodiments, the user may view one or more images generated, for example, on a surface that partially or fully reflects light from another light source (e.g., a laser scan beam) in a manner other than a display panel, through one or more optical lenses. [Exemplary Display System]

[0057] FIG. 5 is a schematic block diagram of a display system 500 according to one non-limiting, exemplary implementation. The display system 500 may be a head-mounted display system, such as, for example, the display system of the head-mounted display system described above, or any other type of display system (e.g., a wearable or non-wearable display system). The display system 500 may include a scan beam display system, or another type of display system (e.g., a microdisplay). In at least some implementations, the display system 500 may be one of two substantially identical display systems provided in a device such as a head-mounted display device.

[0058] The display system 500 includes a display light source 502 optically coupled to the pupil relay system 504. The display light source 502 may include a laser scan beam light source, a microdisplay, or any other suitable display light source. The pupil relay system 504 is arranged to relay a first pupil from the display light source 502 to a second pupil at the viewer's eye 506 (or other image plane, surface, or material). The pupil relay system 504 includes a polarization-sensitive optical system 508 (e.g., a post-scan optical system), which may include one or more of a waveguide-based optical system, a pancake optical system, a birdbath optical system, a coating-based optical system, or other optical systems. The polarization-sensitive optical system 508 may include one or more components. The efficiency of the polarization-sensitive optical system 508 may be very sensitive to the polarization of the light passing through it. That is, for different polarizations, the intensity and the stray light emerging from the polarization-sensitive optical system 508 will vary significantly.

[0059] To optimize the polarization of the light provided to the polarization-sensitive optical system 508, the pupil relay system 504 of the display system 500 further includes a spatially variable polarizer 510 having a spatially varying polarization depending on position, which functions to compensate for the change in polarization and provide a uniform polarization, or more generally an optimized polarization, to the polarization-sensitive optical system 508. The spatially variable polarizer 510 may include a single layer or may include multiple layers, e.g., multiple layers that can each be independently controlled to be in an active or inactive state. For example, the polarization-sensitive optical system 508 may be configured to compensate for or "cancel out" any polarization generated by one or more mirrors or other optical systems at the synthetic angle. As will be further described below, the spatially variable polarizer 510 may be selectively adjusted in real time based on the user's line-of-sight position to provide optimized performance in a certain region or field of view.

[0060] The spatially-varying polarizer 510 may include a wavelength retarder formed of a birefringent material. Birefringence is a property of a material in which the refractive index depends on the polarization and propagation direction of light. The wavelength retarder changes the polarization state or phase of light passing through it. The wavelength retarder may have a slow axis (or extraordinary axis) and a fast axis (ordinary axis). When polarized light passes through the wavelength retarder, the light along the fast axis travels faster than the light along the slow axis.

[0061] As described above, the spatially-varying polarizer 510 may provide a phase delay that varies across the entire field of view (e.g., from on-axis to off-axis) depending on the position, e.g., the lateral position, the longitudinal position, the radial position, which enables a more uniform and efficient supply of light from the display light source 502 to the polarization-sensitive optical system 508. The particular manner in which the phase difference of the spatially-varying polarizer 510 varies may depend on the particular configuration and materials of the optical system of the display system 500, e.g., the polarization state of the incident light, the angle of incidence, the materials, the shapes of various components, etc.

[0062] As an example, the spatially-varying polarizer 510 may not provide a phase difference at a first position and may linearly increase the phase difference at a second position of the spatially-varying polarizer to provide a phase difference of λ / 4 (or another value). Generally, the spatially-varying polarizer 510 may provide a phase difference that varies with position by any method, and the amount of the phase difference may be any value (e.g., λ / 20, λ / 10, λ / 4, λ, 2λ). Further, the amount of the phase difference may increase only in one or more directions, decrease only in one or more directions, or both increase and decrease may occur. The amount of the phase difference may vary continuously or may vary in multiple steps. The amount of the phase difference may vary according to any type of function, including, for example, a linear function, a polynomial function, an exponential function, a step function, other types of functions, or a combination thereof.

[0063] In at least some implementations, the spatially-varying polarizer 510 may be formed of a multi-twist retarder (MTR), which is a waveplate-like retardation film that provides an accurate and customizable level of broadband retardation, narrowband retardation, or multi-band retardation in a single thin film. More specifically, the MTR has two or more twisted liquid crystal (LC) layers on a single substrate together with an alignment layer. Since each subsequent LC layer is directly aligned by the previous layer, simple manufacturing is enabled, automatic layer-to-layer alignment is achieved, and a monolithic film with a continuously varying optical axis is provided.

[0064] FIG. 6 is a schematic diagram of a scan beam display system or projector 600 according to one non-limiting, exemplary implementation. The scan beam display system 600 includes a light source 602, which may be a laser light source that emits a beam 604. The light source 602 may include two or more light sources, for example, a red light source, a green light source, and a blue light source. In such an example, a beam combiner may be used to combine the plurality of light sources into a single beam. In at least some implementations, the light source 602 may include one or more color light sources (e.g., red, green, blue) and a light source that emits an invisible beam such as an infrared beam or an ultraviolet beam, which may be used for various purposes such as eye tracking.

[0065] Beam 604 is incident on a scan platform 606 that may include a scanner based on a microelectromechanical systems (MEMS) substrate, and is reflected by a scan mirror 608 of the platform to generate a controlled output beam 610. The scan platform 606 may include a diffraction grating, a movable diffraction grating, a light valve, a rotating mirror, a movable silicon device, a digital light projector device, a flying spot projector, a liquid crystal on silicon (LCoS) device, or other devices for scanning or modulating. The scan platform 606 may be coupled to one or more drive circuits that are selectively controlled by a controller 612 coupled to the scan platform and the light source 602, and this controller may include any suitable control circuit having one or more components. The drive circuit modulates the direction in which the scan mirror 608 deflects the incident beam 604 to generate a scan such as a raster scan on the output beam 610, whereby a display image is generated on an image plane such as a display surface or the viewer's eye 614.

[0066] FIG. 7 is a schematic diagram of a scan beam display system or projector 700 according to one non-limiting exemplary implementation. The scan beam display system 700 may be similar or identical to the scan beam display system 600 of FIG. 6 in many respects. Therefore, similar components are referenced by the same numerals, and for simplicity, the description of such components will not be repeated here.

[0067] The scan beam display system 700 includes a post - scan correction optical system 702 (the "post - scan optical system") and a collimation optical system 704. The post - scan optical system 702 may include one or more optical systems disposed in the optical path after the scan platform 606, which may generally be referred to herein as "post - scan". The post - scan correction optical system 702 may be designed and configured to correct or adjust one or more distortion artifacts in the projected image. Examples of such distortions may include smile distortion, barrel distortion, pincushion distortion, off - axis projection - based distortion, etc. It should be understood that these are only non - limiting exemplary types of distortions that the post - scan correction optical system 702 may correct.

[0068] The post - scan optical system 702 may include one or more of a waveguide - based optical system, a pancake optical system, a birdbath optical system, a coating - based optical system, etc. The post - scan optical system may include one or more components. The efficiency of the post - scan optical system 702 may be very sensitive to the polarization of the passing light. That is, for different polarizations, the intensity and the stray light exiting the post - scan optical system 702 will vary significantly. In at least some implementations, the display system 700 may also include a collimating optical system or a beam - forming optical system 704 that can be used to at least partially recover the loss of infinite focus caused by the post - scan optical system 702.

[0069] The display system 700 further includes a polarization compensation optical system in the form of a spatially variant polarizer 706 disposed between the scan mirror 608 and the post-scan optical system 702, providing polarization compensation for the post-scan optical system 702 which may be very sensitive to polarization as described above. The spatially variant polarizer 706 may be disposed, for example, on, adjacent to, or within the post-scan optical system 702. In other implementations, the spatially variant polarizer 706 is disposed at some other location in the optical path between the light source 602 and the displayed image (e.g., pre-scan, post-scan, adjacent to the light source, etc.).

[0070] In at least some implementations, the controller 612 may be operably coupled to the spatially variant polarizer 706 to selectively vary the spatially dependent phase retardation of the spatially variant polarizer for any desired configuration. In such implementations, one or more thin film transistor layers may be provided to enable the spatially dependent phase retardation of the spatially variant polarizer 706 to be selectively controlled by the controller 612. As another example, the spatially variant polarizer 706 may include a plurality of independently controllable layers, each layer being controllable by the controller 612 to be in an active or inactive state. The controller 612 can control the phase retardation at any desired frequency, e.g., once, periodically, at a frequency equal to or a fraction of the frame rate of the display system 700, etc.

[0071] In at least some implementations, the controller 612 may be operable to receive gaze tracking information 708, which may be used to selectively adjust the spatially dependent phase retardation of the spatially variant polarizer 706 optimized in the optimization region where the user is currently considered to be gazing. Such gaze tracking information may be received from an eye tracking subsystem, from information about the displayed content (e.g., the object of focus), or from any other information source providing information related to where on the display the user is likely to be gazing.

[0072] As described above, the efficiency of the post-scan optical system 702 may be very sensitive to the polarization of the light passing through. That is, for different polarizations, the intensity and the stray light emerging from the post-scan optical system 702 will vary significantly. Optimizing the spatially variant polarizer 706 in a relatively small area or field of view (e.g., 0 to 10°, 0 to 30°, other areas) can be advantageous as it can provide significantly better performance than optimizing the entire field of view. Using the gaze tracking information, the system can optimize a certain area for various characteristics (e.g., intensity, stray light) and configure the areas outside that area to have low performance (e.g., low intensity, high stray light). Since the user's vision is not designed to recognize in the peripheral area, the user may not even notice that the performance of the display is suppressed outside the area where the user is gazing.

[0073] FIG. 8 is a schematic diagram of a display system 800 that includes a waveguide-based optical system and a polarization compensation optical system in the form of a spatially variant polarizer for use in a head-mounted display system, according to one non-limiting exemplary implementation. The display system 800 may include a lens or support structure 802 (e.g., a prescription eyeglass lens or a non-prescription eyeglass lens). The planar waveguide structure 804 may be at least partially incorporated into the structure 802 or disposed adjacent to this structure (e.g., in front of or behind the structure). The waveguide 804 may be a rectangular (or other shape) prism structure formed of a material having a refractive index that is sufficiently different from the refractive index of the surrounding structure (e.g., structure 802) to provide total internal reflection within the waveguide.

[0074] To enable light to enter the waveguide 804, the display system 800 includes an incoupler 806 physically coupled to the first portion of the waveguide. Similarly, to enable light to exit the waveguide 804 and travel towards the viewer's eye 810, the display system 800 includes an outcoupler 808 physically coupled to the second portion of the waveguide. The display light that enters through the incoupler 806 and exits through the outcoupler 808 may originate from a display light source, such as a projector, scanner, laser projector, microdisplay, or other display light source, as described above. By way of non-limiting example, the couplers 806 and 808 may include one or more of a diffraction grating, hologram, holographic optical element, volume diffraction grating, surface relief diffraction grating, and the like. The couplers 806 and 808 may also be reflective couplers or transmissive couplers. As an example, the structure 802 may include a right-eye spectacle lens, the incoupler 806 may be disposed near the edge of the spectacle lens proximate to the display source (e.g., projector), and the outcoupler may be disposed towards the center of the spectacle lens such that the viewer can see the light from the waveguide 804 while looking straight ahead or nearly straight ahead.

[0075] The display system 800 further includes a polarization compensation optical system in the form of a spatially variant polarizer 812, various examples of which are shown in FIG. 8 as spatially variant polarizers 812a - 812e. In the non - limiting exemplary examples shown, the instances 812a - 812e of the spatially variant polarizer 812 are shown as being disposed at some non - limiting exemplary locations of the display system 800. In this application example, there may be only one spatially variant polarizer 812, although in other application examples, more than one spatially variant polarizer 812 may be present. As described elsewhere herein, in at least some implementations, the spatially variant polarizer 812 may include a plurality of independently controllable layers. Specifically, 812a shows a spatially variant polarizer disposed adjacent to the incoupler 806 on the side facing the display source to provide polarization compensation for the light entering the waveguide 804, which may be very sensitive to polarization as described above. The spatially variant polarizer 812b is shown in FIG. 8 as being disposed adjacent to the incoupler 806 on the opposite side thereof, as compared to the exemplary spatially variant polarizer 812a. Similarly, the spatially variant polarizer 812c is shown as being disposed adjacent to the outcoupler 808 on the side facing the user's eye 810, and the spatially variant polarizer 812d is shown as being disposed adjacent to the outcoupler 808 on the opposite side thereof. The spatially variant polarizer 812e is shown as being disposed inside the waveguide 804 at a position between the incoupler 806 and the outcoupler 808. It should be understood that the spatially variant polarizer 812 may be disposed, for example, on, adjacent to, or proximate to, or inside the waveguide 804. In other implementations, the spatially variant polarizer 812 is disposed at some other location in the optical path between the display light source and the viewer's eye to provide polarization compensation.

[0076] FIG. 9A is a schematic diagram of a display 900 illustrating an optimization region 902 centered on the display at a first time (t1), where the spatially-varying polarizer is optimized for this region based on the user's line-of-sight position. As described above, the spatially-varying polarizer may be optimized to provide one or more improved characteristics (e.g., intensity, less stray light) to the optimization region 902 as compared to regions 904 outside the optimization region. FIG. 9B is a schematic diagram of a display 900 illustrating an optimization region 902 at the periphery of the display at a second time (t2), where the spatially-varying polarizer is optimized for this region based on the user's line-of-sight position. In general, the optimization region 902 may move to follow or track the user's known or inferred line-of-sight position.

[0077] The optimization region 902 may be of any size or shape, and its size or shape may be fixed or variable. For example, if the user is thought to be gazing over a wide area of the display, or if there is some uncertainty as to where the user is gazing, the optimization region 902 may be relatively large. Conversely, if the user is thought to be gazing at a narrow area of the display, or if there is minimal uncertainty as to where the user is gazing, the optimization region 902 may be relatively small. In at least some implementations, the characteristics of the spatially-varying polarizer near the boundary between the optimization region 902 and the non-optimization region 904 may be blended (e.g., using a linear or non-linear blending function) to provide a smooth transition between regions 902 and 904. The shape of the optimization region may be circular, rectangular, elliptical, or any other shape. As described elsewhere herein, the spatially-varying polarizer may have any suitable number of controllable layers that provide the desired shape and size of the region that can be optimized.

[0078] FIG. 10 shows an exemplary spatially-varying polarizer or compensating optical system 1010 coupled to a controller 1002 such as controller 612 described above. The spatially-varying polarizer 1010 may be the same as or identical to any of the spatially-varying polarizers described herein. Controller 1002 has a plurality of outputs 1014a, 1014b, 1014c, each coupled to a respective one of a plurality of inputs 1012a, 1012b, 1012c of a plurality of spatially-varying polarizer layers 1011a, 1011b, 1011c of the spatially-varying polarizer 1010. In the example shown, a three-layer spatially-varying polarizer is shown for purposes of illustration, but in applications, fewer layers (e.g., 1 layer, 2 layers) or more layers (e.g., 5 layers, 10 layers, 20 layers, etc.) may be used as desired. Controller 1002 transmits control signals via respective outputs 1014 that are operative to activate or deactivate the spatially-varying polarizer 1011.

[0079] In the deactivated case, each of the spatially-varying polarizers 1011 may be optically transparent or may only perform its inherent optical modification associated with the material composition of the spatially-varying polarizer 1011. Various types of materials have optical properties and are operative to change light. As described herein, each of the spatially-varying polarizer layers 1011 may be formed of, for example, two or more twisted liquid crystal layers. The liquid crystal layers of the spatially-varying polarizer layer 1011 can perform their inherent optical modification even when the power is turned off. When the power is turned on, the spatially-varying polarizer 1011 performs the optical modification it is designed to perform.

[0080] The first, second, and third spatially-variant polarizers 1011a, 1011b, 1011c are formed to perform first, second, and third optical corrections or optical optimizations. During operation, the controller 1002 transmits control signals to the spatially-variant polarizers 1011a, 1011b, 1011c to operate the spatially-variant polarizers 1011a, 1011b, 1011c in active and non-active states. For example, to activate the first optical optimization and deactivate the second and third optical optimizations, the controller 1002 transmits a control signal to activate the first spatially-variant polarizer 1011a to the first spatially-variant polarizer 1011a, and transmits control signals to deactivate the second and third spatially-variant polarizers 1011b, 1011c to the second and third spatially-variant polarizers 1011b, 1011c, respectively. The first optical optimization may be a combination of a desired optical correction and optical compensation. In this optical compensation, compensation can be performed for the inherent optical correction performed by the second and third spatially-variant polarizers 1011b, 1011c that are deactivated. Thus, the light emerging from the first spatially-variant polarizer 1011a and passing through the second and third spatially-variant polarizers 1011b, 1011c that are deactivated is pre-compensated for the inherent optical correction performed by the second and third spatially-variant polarizers 1011b, 1011c that are deactivated.

[0081] As a non-limiting example, the first spatially-varying polarizer 1011a may be designed to provide a first optimization region as shown in FIG. 9A, and the second spatially-varying polarizer 1011b may be designed to provide a second optimization region as shown in FIG. 9B. Similarly, the third spatially-varying polarizer 1011c may be designed to provide a third optimization region (not shown). Next, the controller may selectively control the state of the spatially-varying polarizer 1011 using the gaze information so that the region at which the user is currently gazing is optimized by the spatially-varying polarizer. For example, the controller may activate the first spatially-varying polarizer 1011a at a first time t1 and then activate the second spatially-varying polarizer 1011b at a second time t2. As described above, the number of layers that can be independently controlled may be any suitable number of layers (for example, it is possible to provide 5 to 15 layers and provide 5 to 15 different regions that can be optimized, respectively).

[0082] In another example, the controller 1002 may send a control signal to the first spatially-varying polarizer 1011a to prevent the first spatially-varying polarizer 1011a from being activated, and may also send control signals to the second and third spatially-varying polarizers 1011b and 1011c to activate the second and third spatially-varying polarizers 1011b and 1011c, respectively. The first optical compensation provided by the second and third spatially-varying polarizers 1011b and 1011c is a combination of the second and third optical corrections. The controller 1002 can selectively switch the spatially-varying polarizer 1011 between an active state and a non-active state so that the spatially-varying polarizer 1010 provides optical correction.

[0083] The spatially-varying polarizer of the present disclosure may provide spatially-varying polarization defined by a surface phase map or a combination of two or more surface phase maps defined together and multiplexed. FIGS. 11 and 12 show two non-limiting examples of surface phase maps for a spatially-varying polarizer. In the exemplary surface map 1100 of FIG. 11, the phase varies concentrically from -0.433 wavelengths to +0.433 wavelengths from the center to the outer periphery of the optical system. In the exemplary surface map 1200 of FIG. 12, the phase varies linearly from -1.25E+004, which is the lower limit of the optical system (as shown in the figure), to +1.25E+004, which is the upper limit of the optical system (as shown in the figure), with each unit being a period of 2π radians. In applications, two or more spatially-varying polarizers may be stacked together. For example, the concentric surface phase map 1100 may be multiplexed with the linear phase map 1200 and the like. The phase variations of the surface maps 1100 and 1200 are shown as a plurality of discrete steps for simplicity, but it should be noted that in reality, the phase may vary continuously across the surface of the optical system. Further, the specific phase values in the surface phase maps 1100 and 1200 are provided as examples and should not be considered limiting.

[0084] In at least some implementations, the surface phase map of the spatially-varying polarizer may be designed to cancel or compensate for unwanted polarization generated by at least one other component of the display system (e.g., a display source, a lens, etc.). In such implementations, the phase profile or map of the optical system (e.g., a lens, or a lens and a display source) may be determined first. The determined phase map may then be inverted and applied to the spatially-varying polarizer, such that the spatially-varying polarizer cancels or compensates for the unwanted effects generated by other components of the optical system.

[0085] By using the spatially variant polarizer described herein, optical designers have significantly more freedom to create optical systems with improved performance and efficiency, thereby enabling display systems that provide a better viewing experience, are less costly, smaller or lighter in size, consume less power, and provide other advantages that will be apparent to those skilled in the art.

[0086] The various implementation examples described above may be combined to provide further implementation examples. Based on the above detailed description, these and other changes can be made to each implementation example. In general, in the following claims, the terms used should not be construed as limiting the claims to the specific implementation examples disclosed in this specification and the claims, but rather the claims should be construed to include all possible implementation examples in conjunction with the full scope of equivalents to which such claims are entitled. Accordingly, the claims are not limited by the present disclosure.

Claims

**Claim 1**: A display system comprising a display light source, and a pupil relay system arranged to relay a first pupil from the display light source to a second pupil in the viewer's eye, the pupil relay system comprising a polarization-sensitive optical system, and a spatially-varying polarizer for compensating for unwanted polarization generated by at least one other component of the display system, the spatially-varying polarizer having a polarization that varies spatially according to position to provide polarization compensation to the polarization-sensitive optical system, and a control circuit operably coupled to the spatially-varying polarizer, the control circuit being configured to receive gaze information indicative of a current or predicted gaze position of a user from an eye-tracking subsystem, and selectively adjust the varying polarization of the spatially-varying polarizer based at least in part on the received gaze information. A display system comprising the control circuit. **Claim 2**: The display system according to claim 1, wherein the spatially-varying polarizer comprises a plurality of layers, each of the plurality of layers being independently controllable by the control circuit to be in an active state or a non-active state, and during operation, the control circuit controls the active state or non-active state of the plurality of layers to selectively adjust the varying polarization of the spatially-varying polarizer. **Claim 3**: The display system according to claim 2, wherein each of the plurality of layers has a spatially-varying polarization different from at least one of the others of the plurality of layers. **Claim 4**: The display system according to any one of claims 1 to 3, wherein the control circuit optimizes the spatially-varying polarizer for a first region related to the viewer's gaze position more than for a second region outside the first region. **Claim 5**: The display system according to any one of claims 1 to 4, wherein the control circuit controls the spatially-varying polarizer to provide a relatively high luminance in a first region related to the viewer's gaze position compared to a second region outside the first region. **Claim 6**: The display system according to any one of claims 1 to 5, wherein the control circuit controls the spatially-varying polarizer to provide a relatively low amount of stray light in a first region related to the viewer's gaze position compared to a second region outside the first region. ​ ​ ​ ​ ​ ​ ​

7. The display system according to any one of claims 1 to 6, wherein when the received line-of-sight information is updated by the eye tracking subsystem, the control circuit selectively adjusts the varying polarization of the spatially-varying polarizer in real time.

8. The display system according to any one of claims 1 to 7, further comprising an eye tracking subsystem that generates the line-of-sight information.

9. The display system according to any one of claims 1 to 8, wherein the spatially-varying polarizer includes a multi-twist retarder.

10. The display system according to any one of claims 1 to 9, wherein the phase difference of the spatially-varying polarizer varies according to a horizontal position or a vertical position.

11. The display system according to any one of claims 1 to 9, wherein the phase difference of the spatially-varying polarizer varies across the entire field of view of the display system.

12. The display system according to any one of claims 1 to 11, wherein the polarization-sensitive optical system includes a waveguide-based optical system, a pancake optical system, a birdbath optical system, or a coating-based optical system.

13. The display system according to any one of claims 1 to 12, wherein the display light source has a laser light source, and the display system further includes a scan mirror arranged to receive a light beam from the laser light source and relay the received light beam toward the pupil relay system.

14. The display system according to claim 13, further comprising a beam-forming optical system arranged between the laser light source and the scan mirror.

15. The display system according to any one of claims 1 to 14, wherein at least a part of the spatially-varying polarizer is arranged on, adjacent to, or within the polarization-sensitive optical system.

16. The display system according to any one of claims 1 to 15, wherein the polarization-sensitive optical system includes a waveguide, and the spatially-varying polarizer is arranged on, inside, or adjacent to a port of the waveguide.

17. The display system according to any one of claims 1 to 16, wherein the display system is a display system of a head-mounted display system.

18. A support structure, A display system coupled to the support structure, the display system comprising: a display light source; a pupil relay system arranged to relay a first pupil from the display light source to a second pupil in the viewer's eye, the pupil relay system comprising: a polarization-sensitive optical system; a spatially-varying polarizer for compensating for unwanted polarization generated by at least one other component of the display system, the spatially-varying polarizer having a polarization that varies spatially according to position to provide polarization compensation to the polarization-sensitive optical system; and a pupil relay system; a display system; a control circuit operably coupled to the spatially-varying polarizer, the control circuit: receiving line-of-sight information indicative of a current or predicted line-of-sight position of a user from an eye tracking subsystem; selectively adjusting the varying polarization of the spatially-varying polarizer based at least in part on the received line-of-sight information; and a control circuit configured to: a head-mounted display system. **Claim 19** The head-mounted display system of claim 18, wherein the spatially-varying polarizer includes a plurality of layers, each of the plurality of layers being independently controllable by the control circuit to be in an active state or a non-active state, and during operation, the control circuit controls the active state or the non-active state of the plurality of layers to selectively adjust the varying polarization of the spatially-varying polarizer. **Claim 20** The head-mounted display system of claim 19, wherein each of the plurality of layers has a spatially-varying polarization that is different from at least one of the others of the plurality of layers. **Claim 21** The head-mounted display system according to any one of claims 18 to 20, wherein the control circuit optimizes the spatially-varying polarizer for a first region related to the user's line-of-sight position more than for a second region outside the first region. **Claim 22** The head-mounted display system according to any one of claims 18 to 21, wherein the control circuit controls the spatially-varying polarizer to provide a relatively high luminance in a first region related to the user's line-of-sight position compared to a second region outside the first region. **Claim 23** The head-mounted display system according to any one of claims 18 to 22, wherein the control circuit controls the spatially-varying polarizer to provide a relatively low amount of stray light to a second region outside the first region in a first region related to the line-of-sight position of the user.

24. The head-mounted display system according to any one of claims 18 to 23, wherein the control circuit selectively adjusts the varying polarization of the spatially-varying polarizer in real time when the received line-of-sight information is updated by the eye-tracking subsystem.

25. The head-mounted display system according to any one of claims 18 to 24, wherein the spatially-varying polarizer includes a multi-twist retarder.

26. The head-mounted display system according to any one of claims 18 to 25, wherein the polarization-sensitive optical system includes a waveguide-based optical system, a pancake optical system, a birdbath optical system, or a coating-based optical system.

27. The head-mounted display system according to any one of claims 18 to 26, wherein the display light source has a laser light source, and the display system further includes a scan mirror arranged to receive a light beam from the laser light source and relay the received light beam toward the pupil relay system.

28. The head-mounted display system according to claim 27, further comprising a beam-forming optical system arranged between the laser light source and the scan mirror.

29. The head-mounted display system according to any one of claims 18 to 28, wherein the polarization-sensitive optical system includes a waveguide, and the spatially-varying polarizer is arranged on, inside, or proximate to a port of the waveguide.

30. A support structure, and A display system coupled to the support structure, the display system comprising A laser light source, and A scan mirror arranged to receive a light beam from the laser light source, and A pupil relay system arranged to relay a first pupil received from the scan mirror to a second pupil in the viewer's eye, the pupil relay system comprising A polarization-sensitive optical system, and A spatial-variation polarizer for compensating for unwanted polarization generated by at least one other component of the display system, the spatial-variation polarizer having polarization that varies spatially according to position in order to provide polarization compensation to the polarization-sensitive optical system including a pupil relay system a display system having an eye-tracking subsystem operable to generate eye-tracking information indicative of a current or predicted line-of-sight position of a user a control circuit operably coupled to the spatial-variation polarizer, the control circuit receiving eye-tracking information from the eye-tracking subsystem when the user views the display of the display system selectively adjusting the varying polarization of the spatial-variation polarizer based at least in part on the received eye-tracking information a control circuit configured to A head-mounted display system comprising.

31. The head-mounted display system according to claim 30, wherein the spatial-variation polarizer includes a plurality of layers, each of the plurality of layers being independently controllable by the control circuit to be in an active state or an inactive state, and during operation, the control circuit controls the active state or the inactive state of the plurality of layers to selectively adjust the varying polarization of the spatial-variation polarizer.

32. The head-mounted display system according to claim 31, wherein each of the plurality of layers has a spatial-variation polarization different from that of at least one of the others of the plurality of layers.

33. The head-mounted display system according to any one of claims 30 to 32, wherein the spatial-variation polarizer includes a multi-twist retarder.

34. The head-mounted display system according to any one of claims 30 to 33, wherein the polarization-sensitive optical system includes a waveguide-based optical system, a pancake optical system, a birdbath optical system, or a coating-based optical system.

Images