Correction Polarization Compensation Optical System for Display System

Abstract

The present disclosure generally relates to techniques for improving the performance and efficiency of display systems, such as laser scanning beam display systems or other types of display systems (e.g., microdisplays). The display systems of the present disclosure may include polarization compensation optics, such as a spatially varying polarizer, that provides a position-varying phase retardation, which provides polarization compensation and results in light that is well suited to the polarization-sensitive optics of the display system, such as waveguide-based optics, pancake optics, birdbath optics, coating-based optics, and the like. The display systems of the present disclosure may be components of head-mounted display systems or other types of display systems.

Description

Technical Field

[0001] The present disclosure generally relates to display systems, and more particularly to improving the efficiency and performance of display systems. [Description of Related Art]

[0002] One modern virtual reality (“VR”) experience is generated using a head-mounted display (“HMD”), which can be tethered to a stationary computer (such as a personal computer (“PC”), laptop, or gaming console), combined with and / or integrated with a smartphone and / or its associated display, or made self-contained. Generally, an HMD is a display device worn on the user's head and has a small display device in front of one eye (monocular HMD) or each eye (binocular HMD). The display unit is usually miniaturized and can include, for example, a CRT, LCD, liquid crystal on silicon (LCoS), OLED device, or laser scan beam display. In a binocular HMD, it is possible to display different images for each eye. This function is used to display stereoscopic images.

[0003] The demand for higher performance displays has been increasing with the development of smartphones, high-resolution TVs, and other electronic devices. Such demand is further increasing due to the growing popularity of virtual reality systems and augmented reality systems, particularly systems using HMDs. A virtual reality system typically completely covers both eyes of the wearer and substitutes a “virtual” reality for the actual visual field or physical visual field (or actual reality) in front of the wearer, while an augmented reality system typically provides a translucent or transparent overlay of one or more screens in front of both eyes of the wearer such that the actual visual field is augmented with additional information, and in a mediated reality system as well, information combining real-world elements and virtual elements can be presented to the viewer.

Summary of the Invention

[0004] A display system may be summarized as including a display light source and a pupil relay system arranged to relay a first pupil from the display light source to a second pupil in the viewer's eye. The pupil relay system includes a polarization-sensitive optical system and a spatially-varying polarizer having spatially-varying polarization that varies according to position in order to provide polarization compensation to the polarization-sensitive optical system.

[0005] The spatially-varying polarizer may include a multi-twist retarder. The spatially-varying polarizer may not provide a phase difference at a first position and may provide a quarter-wavelength phase difference at a second position. The phase difference of the spatially-varying polarizer may vary according to a lateral dimension or a longitudinal dimension. The phase difference of the spatially-varying polarizer may vary across the entire field of view of the display system. The polarization-sensitive optical system may include a waveguide-based optical system, a pancake optical system, a birdbath optical system, or a coating-based optical system. The display light source may include a laser light source, and the display system may further include a scan mirror arranged to receive a light beam from the laser light source and relay the received light toward the pupil relay system.

[0006] The display may further include a beam-forming optical system arranged between the laser light source and the scan mirror.

[0007] At least a portion of the spatially-varying polarizer may be disposed on, adjacent to, or within the polarization-sensitive optical system. The polarization-sensitive optical system may include a waveguide, and the spatially-varying polarizer may be disposed on, inside, or proximate to a port of the waveguide. The display system may be a display system of a head-mounted display system.

[0008] The display system may further include a control circuit operably coupled to the spatially-varying polarizer, the control circuit being operable to selectively adjust the phase difference provided by the spatially-varying polarizer.

[0009] The display source may include a microdisplay, and the spatially-varying polarizer may be disposed adjacent to the microdisplay. The spatially-varying polarizer may be adhered to the microdisplay. The spatially-varying polarizer may impart telecentricity to light emitted by the microdisplay. The spatially-varying polarizer may include a surface phase map that provides polarization compensation to at least one of the display source and the polarization-sensitive optical system.

[0010] A head-mounted display system may be summarized as including a support structure and a display system coupled to the support structure, the display system having a display light source and a pupil relay system arranged to relay a first pupil from the display light source to a second pupil in a viewer's eye, the pupil relay system including a polarization-sensitive optical system and a spatially-varying polarizer having spatially-varying polarization depending on position to provide polarization compensation to the polarization-sensitive optical system.

[0011] The spatially-varying polarizer may include a multi-twist retarder. The phase difference of the spatially-varying polarizer may vary according to a lateral dimension, a longitudinal dimension, or a field of view of the display system. The polarization-sensitive optical system may include a waveguide-based optical system, a pancake optical system, a birdbath optical system, or a coating-based optical system.

[0012] A head-mounted display system may be summarized as including a support structure and a display system coupled to the support structure, the display system having a laser light source, a scan mirror arranged to receive a light beam from the laser light source, and a pupil relay system arranged to relay a first pupil received from the scan mirror to a second pupil in a viewer's eye, the pupil relay system including a polarization-sensitive optical system and a spatially-varying polarizer having spatially-varying polarization depending on position to provide polarization compensation to the polarization-sensitive optical system.

[0013] The spatially-varying polarizer may include a multi-twist retarder. The phase difference of the spatially-varying polarizer may vary according to the horizontal dimension, vertical dimension, or field of view of the display system. The polarization-sensitive optical system may include a waveguide-based optical system, a pancake optical system, a birdbath optical system, or a coating-based optical system.

Brief Description of the Drawings

[0014] In this drawing, like elements or operations are identified by like reference numerals. The sizes and relative positions of the elements in the drawing are not necessarily drawn to scale. For example, the shapes and angles of various elements are not necessarily drawn to scale, and some of these elements may be arbitrarily enlarged and arranged to improve the visibility of the drawing. Further, the specific shape of the drawn elements is not necessarily intended to convey any information regarding the actual shape of the specific elements, and may only be selected to make the drawing easier to recognize.

[0015]

Figure 1

[0016]

Figure 2

[0017]

Figure 3

[0018]

Figure 4

[0019]

Figure 5

[0020]

Figure 6

[0021]

Figure 7

[0022]

Figure 8

[0023]

Figure 9

[0024]

Figure 10

[0025]

Figure 11

DETAILED DESCRIPTION OF THE INVENTION

[0026] In the following description, specific details are set forth in order to provide a thorough understanding of the various implementations disclosed. However, one of ordinary skill in the art will recognize that these implementations may be practiced without one or more of these specific details, or with other methods, components, materials, etc. In other instances, well-known structures associated with computer systems, server computers, and / or communication networks have not been shown or described in detail to avoid obscuring aspects of the implementations.

[0027] Unless the context requires otherwise, throughout this specification and the following claims, the word "comprising" is synonymous with "including" and is inclusive or non-exclusive (i.e., it does not exclude additional, unrecited elements or method acts).

[0028] References throughout this specification to "one implementation" or "an implementation" mean that a particular feature, structure, or characteristic described in connection with that implementation is included in at least one implementation. Thus, appearances of the phrases "in one implementation" or "in an implementation" in various places throughout this specification are not necessarily all referring to the same implementation. Furthermore, in one or more implementations, the particular features, structures, or characteristics may be combined in any suitable manner.

[0029] In this specification and the appended claims, the singular forms "a", "an", and "the" include plural referents unless the context clearly dictates otherwise. Note also that the term "or" is generally used herein in its sense including "and / or" unless the context clearly dictates otherwise.

[0030] Each item and summary provided in this specification is for convenience only and does not interpret the scope or meaning of the implementations.

[0031] The present disclosure generally relates to techniques for improving the performance and efficiency of display systems, such as laser scan beam display systems or other types of display systems (e.g., microdisplays). As will be further described below, at least some embodiments of the present disclosure improve the performance of a display system by providing a spatially variant polarizer that results in a phase delay that varies depending on position. This results in polarization compensation and generates light that is well-suited for a polarization-sensitive optical system of a display system, such as a waveguide-based optical system, a pancake optical system, a birdbath optical system, a coating-based optical system, etc. Such techniques significantly improve the intensity of light passing through a polarization-sensitive optical system and also suppress unwanted stray light.

[0032] First, with reference to FIGS. 1-4, an exemplary application of the techniques described herein to a head-mounted display device will be described. Next, with reference to FIGS. 5-8, an exemplary implementation of a display system including a polarization compensation optical system will be described. [Exemplary Head-Mounted Display System and Environment]

[0033] FIG. 1 is a schematic diagram of a networked environment 100 that includes a local media rendering (LMR) system 110 (e.g., a gaming console), and the system includes a local computing system 120 and a display device 180 (e.g., an HMD device with two display panels) suitable for performing at least some of the techniques described herein. In the illustrated embodiment of FIG. 1, the local computing system 120 is communicatively connected to the display device 180 via a transmission link 115 (the display device may be wired or tethered, such as via one or more cables (cable 220) as illustrated in FIG. 2, or alternatively wirelessly connected). In other embodiments, the local computing system 120 may provide encoded display image data via a wired or wireless link to a panel display device (e.g., a TV, console, or monitor), in addition to or instead of the HMD device 180, and each display device includes one or more addressable pixel arrays. In various embodiments, the local computing system 120 may include a general-purpose computing system, a gaming console, a video stream processing device, a mobile computing device (e.g., a mobile phone, PDA, or other mobile device), a VR processing device or an AR processing device, or other computing systems.

[0034] In the illustrated embodiment, the local computing system 120 includes one or more hardware processors (e.g., a central processing unit or "CPU") 125, a memory 130, various I / O ("input / output") hardware components 127 (e.g., a keyboard, a mouse, one or more game controllers, speakers, a microphone, an IR transmitter and / or receiver, etc.), a video subsystem 140 including one or more dedicated hardware processors (e.g., a graphics processing unit or "GPU") 144 and video memory (VRAM) 148, a computer-readable storage 150, and a network connection 160. Also, in the illustrated embodiment, one embodiment of the eye tracking subsystem 135 is executed in the memory 130 using the CPU 125 and / or the GPU 144 to perform automated processing for implementing at least some of the described techniques, and the memory 130 may optionally further execute one or more other programs 133 (e.g., a game program that generates a displayed video or other image). As part of the automated processing for implementing at least some of the techniques described herein, the eye tracking subsystem 135 and / or the program 133 executed in the memory 130 may store or retrieve various types of data including those within the data structure of an exemplary database of the storage 150. In this example, the data used may include various types of image data information within a database ("DB") 154, various types of application data within the DB 152, various types of configuration data within the DB 157, and may also include additional information such as system data or other information.

[0035] In the illustrated embodiment, the LMR system 110 may also be communicatively connected to an exemplary network-accessible media content provider 190 that can further provide content to the LMR system 110 for display via one or more computer networks 101 and network links 102, whether in addition to or instead of the image generation program 133. The media content provider 190 may include one or more computing systems (not shown) having components similar to those of the local computing system 120, including one or more hardware processors, I / O components, local storage devices, and memory. For simplicity, some details of the network-accessible media content provider are not illustrated.

[0036] The display device 180 is illustrated as being different from and separate from the local computing system 120 in the illustrated embodiment of FIG. 1, but in certain embodiments, it will be understood that some or all of the components of the local media rendering system 110 may be integrated or housed within one device, such as a portable gaming machine, a portable VR entertainment system, an HMD device, etc. In such embodiments, the transmission link 115 may include, for example, one or more system buses and / or video bus architectures.

[0037] As an example involving the processing performed locally by the local media rendering system 120, assume that the local computing system is a game computing system that includes one or more game applications in which application data 152 is executed by the CPU 125 using the memory 130, and in which various video frame display data is generated and / or processed by an image generation program 133 in cooperation with the GPU 144 of the video subsystem 140. To provide a high-quality game experience, a large amount of video frame data (corresponding to a high image resolution per video frame and a high "frame rate" of about 60 to 180 per second for such video frames) is generated by the local computing system 120 and provided to the display device 180 via a wired or wireless transmission link 115.

[0038] The computing system 120 and the display device 180 are merely examples and are not intended to limit the scope of the present disclosure. The computing system 120 may instead include a plurality of computing systems or devices that interact with each other, and may be connected to other devices not illustrated, such as via one or more networks such as the Internet, via the web, or via a private network (such as a mobile communication network). More generally, a computing system or other computing node may include any combination of hardware or software that can interact with each other to perform the types of functions described, including, but not limited to, desktop or other computers, gaming machines, database servers, network storage devices and other network devices, PDAs, mobile phones, wireless phones, pagers, electronic organizers, Internet appliances, television-based systems (such as using set-top boxes and / or personal / digital video recorders), and various other consumer products including appropriate communication capabilities. The display device 180 may similarly include one or more devices having one or more display panels of various types and forms, and optionally may include various other hardware components and / or software components.

[0039] Although various items are illustrated as being stored in and used from memory or storage, it will be understood that some or all of these items may be transferred between memory and other storage devices for purposes of memory management or data integrity. Thus, in some embodiments, some or all of the described techniques may be performed by one or more processors or other configured hardware circuits or hardware including memory or storage (e.g., by executing software instructions of one or more software programs and / or by storing such software instructions and / or data structures), such as when constituted by one or more software programs and / or data structures. Some or all of the components, systems, and data structures may also be stored on a non-transitory computer-readable storage medium (e.g., as software instructions or structured data), such as a hard disk or flash drive or other non-volatile storage device, volatile memory or non-volatile memory (e.g., RAM), network storage device, or portable media article (e.g., DVD disk, CD disk, optical disk, etc.) readable by an appropriate drive or via an appropriate connection. The systems, components, and data structures may also, in some embodiments, be transmitted in various computer-readable transmission media including wireless-based media and wired / cable-based media as generated data signals (e.g., as part of a carrier wave or other analog or digital propagated signal), and may take various forms (e.g., as part of one or multiplexed analog signals or as multiple individual digital packets or frames). Such computer program products may take other forms in other embodiments. Accordingly, the present invention may be practiced using other computer system configurations.

[0040] Figure 2 illustrates an exemplary environment 200 that provides virtual reality displays to a human user 206 using at least some of the techniques described, along with an exemplary HMD device 202 coupled to a video rendering computing system 204 via a tether connection 220 (or a wireless connection in other embodiments). The user wears the HMD device 202 and receives display information of a simulated environment different from the actual physical environment from the computing system 204 via the HMD device, and the computing system serves as an image rendering system that supplies an image of the simulated environment (e.g., an image generated by a game program and / or other software programs running on the computing system) to the HMD device for display to the user. In this example, the user can further move around within a tracking volume 201 of the actual physical environment 200 and may further have one or more I / O ("input / output") devices (including the handheld controllers 208 and 210 in this example) that enable the user to further interact with the simulated environment.

[0041] In the illustrated example, the environment 200 may include one or more base stations 214 (two are shown labeled as base stations 214a and 214b) that may facilitate tracking of the HMD device 202 or the controllers 208 and 210. The position of the HMD device is tracked to enable, for example, the corresponding portion of the simulated environment to be displayed to the user wearing the HMD device as the user moves locations or changes the orientation of the HMD device 202, and the controllers 208 and 210 may use a similar technique for use in tracking the position of the controllers (and optionally for using information useful in determining or verifying the position of the HMD device). After the tracked position of the HMD device 202 is known, the corresponding information is transmitted to the computing system 204 via a tether 220 or wirelessly. The computing system uses the tracked position information to generate and display one or more next images of the simulated environment to the user.

[0042] There are a number of different methods for position tracking that can be used in various implementations of the present disclosure, including, but not limited to, acoustic tracking, inertial tracking, magnetic tracking, optical tracking, combinations thereof, and the like.

[0043] In at least some implementations, the HMD device 202 may include one or more light receivers or sensors that can be used to implement the tracking functionality or other aspects of the present disclosure. For example, each base station 214 may sweep a light signal across the entire tracking volume 201. Depending on the requirements of each particular implementation, each base station 214 may generate more than one light signal. For example, for six-degree-of-freedom tracking, usually one base station 214 is sufficient, but for some embodiments, multiple base stations (e.g., base stations 214a, 214b) may be necessary or desirable to provide stable room-scale tracking for the HMD device and peripheral devices. In this example, the light receivers are incorporated into the HMD device 202 and / or other objects to be tracked (such as controllers 208 and 210). In at least some implementations, on each device to be tracked, the light receivers may be combined with an accelerometer and gyroscope inertial measurement unit (an "IMU") to support low-latency sensor fusion.

[0044] In at least some implementations, each base station 214 includes two rotors that sweep a linear beam along the vertical axis across the entire tracking volume 201. At the start of each sweep cycle, the base station 214 may emit an omnidirectional light pulse (referred to as a "synchronization signal") that is visible to all sensors attached to the object to be tracked. In this way, each sensor calculates its unique angular position in the swept volume by measuring the duration between the synchronization signal and the beam signal. The distance and orientation of the sensor may be determined using multiple sensors attached to a single stationary object.

[0045] One or more sensors disposed on a tracking object (e.g., HMD device 202, controllers 208 and 210) may include optoelectronic devices capable of detecting modulated light from a rotor. In the case of visible light or near-infrared (NIR) light, silicon photodiodes and suitable amplifier / detector circuits may be used. Since environment 200 may include static signals and time-varying signals (optical noise) having a wavelength similar to that of the base station 214 signal, in at least some implementations, the light from the base station may be modulated in such a way as to facilitate discrimination from any interfering signals and / or to facilitate screening the sensor from any emission wavelengths other than the wavelength of the base station signal.

[0046] Inside-out tracking is also a type of position tracking that can be used to track the position of HMD device 202 and / or other objects (e.g., controllers 208 and 210, tablet computer, smartphone). Inside-out tracking differs from outside-in tracking in that the position of the camera or other sensor is used to determine the position of the HMD. In inside-out tracking, the camera or sensor is located on the HMD or the object being tracked, while in outside-out tracking, the camera or sensor is placed at a fixed position within the environment.

[0047] HMDs that utilize inside-out tracking use one or more "looking-out" cameras to determine how their position is changing relative to the environment. As the HMD moves, the sensors readjust their origin point within the room, and the virtual environment responds in real time accordingly. This type of position tracking can be achieved regardless of the presence or absence of markers placed within the environment. Cameras placed on the HMD observe the characteristics of the surrounding environment. When using markers, the markers are designed to be easily detected by the tracking system and are placed in specific areas. In "markerless" inside-out tracking, the HMD system uses prominent features (e.g., topography) that already exist within the environment to determine position and orientation. The algorithms of the HMD system identify specific images or shapes and use these to calculate the position of the device within the space. Data from accelerometers and gyroscopes can also be used to enhance the accuracy of position tracking.

[0048] Figure 3 shows information 300 that illustrates a front view of an exemplary HMD device 344 when worn on the head of user 342. The HMD device 344 includes a front structure 343 that supports a front or forward camera 346 and one or more types of a plurality of sensors 348a - 348d (collectively 348). As one example, an optical sensor that detects and uses light information emitted from some or all of the sensors 348, for example, one or more external devices (not shown, but for example, the base station 214 of FIG. 2), can help determine the position and orientation of the device 344 within the space. As shown, the forward camera 346 and sensors 348 are directed forward toward the actual scene or environment (not shown) in which user 342 operates the HMD device 344. The actual physical environment can include, for example, one or more objects (e.g., walls, ceilings, furniture, stairs, cars, trees, tracking markers, or any other type of object). A specific number of sensors 348 may be fewer or more than the number of sensors shown. The HMD device 344 may further include one or more additional components not attached to the front structure (e.g., inside the HMD device), such as an IMU (inertial measurement unit) electronic device 347 that measures and reports a specific force, angular velocity, and / or magnetic field surrounding the HMD device 344 (e.g., using a combination of an accelerometer and a gyroscope, and optionally a magnetometer). The HMD device may further include additional components not shown, which include one or more display panels and optical lens systems that face the user's both eyes (not shown), and optionally have one or more built-in motors for changing the alignment or other positioning of one or both of the optical lens system and / or display panel within the HMD device. This will be described in more detail below with respect to FIG. 4.

[0049] The example of the illustrated HMD device 344 is supported on the head of user 342, at least in part, based on one or more straps 345 attached to the housing of the HMD device 344 and extending wholly or partially around the user's head. Although not illustrated here, the HMD device 344 may further include one or more external motors attached to one or more of the straps 345, etc., and adjusting such a strap using such a motor to correct the alignment or other positioning of the HMD device on the user's head may be included in an auto-correction operation. The HMD device may include other support structures (e.g., nose pads, chin straps, etc.) not illustrated here, in addition to or instead of the illustrated straps, and some embodiments include one or more such other support structures to which motors are attached, and similarly, it will be understood that the shape and / or position thereof may be adjusted to correct the alignment or other positioning of the HMD device on the user's head. Other display devices not attached to the user's head may similarly be attached to one or more structures, or portions thereof, that affect the positioning of the display device, and such other devices include motors or other mechanical actuators in at least some embodiments, and similarly, the shape and / or position thereof may be modified to correct the alignment or other positioning of the display device with respect to one or more pupils of one or more users of the display device.

[0050] Figure 4 illustrates a simplified plan view 400 of an HMD device 405 that includes a pair of near-to-eye display systems 402 and 404. The HMD device 405 may be the same as or similar to the HMD devices illustrated in FIGS. 1-3, or it may be a different HMD device, and the HMD device described herein may be further used in the examples described further below. The near-to-eye display systems 402 and 404 of FIG. 4 each include a display panel 406 and 408 (e.g., an OLED microdisplay), and each of the optical lens systems 410 and 412 having one or more optical lenses. The display systems 402 and 404 may be installed in a housing (or frame) 414 or otherwise disposed therein, and the housing includes a front portion 416 (e.g., the same as or similar to the front surface 343 of FIG. 3), a left temple 418, a right temple 420, and an inner surface 421 that contacts or is proximate to the face of the user 424 who is the wearer when the HMD device is worn. The two display systems 402 and 404 that can be worn on the head 422 of the user 424 who is the wearer may be fixed to the housing 414 in a glasses configuration, the left temple 418 and the right temple 420 may be placed on the user's ears 426 and 428 respectively, and the nose rest 492 may be placed on the user's nose 430. In the example of FIG. 4, the HMD device 405 may be supported on the user's head partially or entirely by the nose rest and / or the left and right over-ear type temples, but in some embodiments, as in the embodiments shown in FIGS. 2 and 3, a strap (not shown) or other structure may be used to fix the HMD device to the user's head. The housing 414 may be made in a shape and size such that each of the two optical lens systems 410 and 412 is disposed in front of one of the user's eyes 432 and 434 respectively, whereby the target position of each pupil 494 will be centered both vertically and horizontally in front of each optical lens system and / or display panel.The housing 414 is shown in a simplified manner similar to glasses for purposes of illustration, but it should be understood that in practice, more sophisticated structures (e.g., goggles, integrated headbands, helmets, straps, etc.) can be used to support the display systems 402 and 404 and position them on the head 422 of the user 424.

[0051] The HMD device 405 of FIG. 4, and the other HMD devices described herein, can present virtual reality displays to the user via corresponding videos presented at a display rate such as 30 or 60 or 90 frames (or images) per second. In other embodiments of similar systems, augmented reality displays may be presented to the user. The displays 406 and 408 of FIG. 4 may each generate light that is focused onto the eyes 432 and 434 of the user 424 by the respective optical lens systems 410 and 412 after being carried through the respective optical lens systems. The aperture of the pupil 494 of each eye allows light to enter the eye, but typically, the size of the pupil ranges from 2 millimeters (mm) in a very bright state to 8 mm in a dark state, and the size of the larger iris that includes the pupil can be approximately 12 mm. The pupil (and the surrounding iris) can also typically move a few millimeters horizontally and / or vertically within the visible portion of the eye when the eyelid is open, which, when the eyeball rotates about its center (resulting in a 3D volume in which the pupil can move), will also move the pupil to different depths relative to the optical lens of the display or other physical elements at different horizontal and vertical positions. The user 424 sees the light entering the user's pupil as an image and / or video. In some implementation examples, the distance between each of the optical lens systems 410 and 412 and the user's eyes 432 and 434 may be relatively short (e.g., less than 30 mm, less than 20 mm), which, since the weight of the optical lens system and the display system is relatively close to the user's face, is advantageous in that the HMD device will feel lighter to the user and can also provide a larger field of view to the user. Although not illustrated here, some embodiments of such HMD devices may include various additional built-in sensors and / or external sensors.

[0052] In the illustrated embodiment, the HMD device 405 of FIG. 4 further includes hardware sensors and additional components, such as including one or more accelerometers and / or gyroscopes 490 (e.g., as part of one or more IMU units). As will be described in more detail elsewhere herein, values from the accelerometers and / or gyroscopes may be used to locally determine the orientation of the HMD device. Further, the HMD device 405 may include one or more front cameras, such as camera 485 on the outer surface of the front portion 416, and the information may be used as part of the operations of the HMD device, such as for providing an AR function or a positioning function. Further, the HMD device 405 may further include other components 475 (e.g., electronic circuits for controlling the display of images to the display panels 406 and 408, built-in storage, one or more batteries, a position tracking device for communicating with an external base station, etc.), which will be described in more detail elsewhere herein. Other embodiments may not include one or more of the components 475, 485, and / or 490. Although not illustrated here, some embodiments of such HMD devices may include various additional built-in sensors and / or external sensors for tracking various other types of movements and positions of the user's body, eyes, controller, etc.

[0053] In the illustrated embodiment, the HMD device 405 of FIG. 4 further includes hardware sensors and additional components that can be used by the disclosed embodiments as part of the described techniques to determine the direction of the user's pupil or line of sight, which may be provided for use by one or more components associated with the HMD device. This is described elsewhere in this specification. The hardware sensors in this example include, for example, one or more eye tracking assemblies 472 of an eye tracking subsystem that are located on or near the display panels 406 and 408 and / or on the inner surface 421 near the optical lens systems 410 and 412 for use in obtaining information about the actual position of the user's pupil 494, for example, separately for each pupil in this example.

[0054] Each of the eye tracking assemblies 472 may include one or more light sources (e.g., IR LEDs) and one or more light detectors (e.g., silicon photodiodes). Further, for clarity, FIG. 4 shows only a total of four eye tracking assemblies 472, but it should be understood that in practice, a different number of eye tracking assemblies may be provided. In some embodiments, a total of eight eye tracking assemblies 472 are provided, i.e., four eye tracking assemblies for each of the user's 424 eyes. Further, in at least some implementations, each eye tracking assembly includes a light source directed at one of the user's 424 eyes 432 and 434, a light detector arranged to receive the light reflected by each of the user's eyes, and a polarizer arranged and configured so that the light reflected by specular reflection does not reach the light detector.

[0055] As described in more detail elsewhere in this specification, information from the eye tracking assembly 472 may be used to determine and track the direction of the user's line of sight during use of the HMD device 405. Further, in at least some embodiments, the HMD device 405 may include one or more built-in motors 438 (or other movement mechanisms) that can be used to move (e.g., in the vertical, horizontal left-right, and / or horizontal forward-backward directions) the alignment and / or other positioning of one or both of the optical lens systems 410 and 412 and / or the display panels 406 and 408 within the housing of the HMD device 405, which is for personalizing or otherwise adjusting the target pupil positions of one or both of the near-to-eye display systems 402 and 404 corresponding to the actual position of one or both of the pupils 494. Such a motor 438 may be controlled, for example, by the user operating one or more control buttons 437 on the housing 414 and / or by the user operating one or more associated separate I / O controllers (not shown). In other embodiments, the HMD device 405 may control the alignment and / or other positioning of the optical lens systems 410 and 412 and / or the display panels 406 and 408 using an adjustable positioning mechanism (e.g., a screw, slider, ratchet, etc.) that can be manually changed by the user using the control buttons 437 without using such a motor 438. Further, although FIG. 4 illustrates a motor 438 for only one of the near-to-eye display systems, in some embodiments, each near-to-eye display system may have its own one or more motors, and in some embodiments, one or more motors may be used to control each of the plurality of near-to-eye display systems (e.g., independently).

[0056] The described techniques may be used with a display system similar to that illustrated in some embodiments, but in other embodiments, other types of display systems may be used, including, for example, those having one optical lens and a display device, or those having a plurality of such optical lenses and display devices. Non-exclusive examples of other such devices include cameras, telescopes, microscopes, binoculars, spotting scopes, surveying scopes, and the like. Further, the described techniques may be used with a wide variety of display panels or other display devices that emit light to form an image, and as described elsewhere herein, that image is viewed by one or more users through one or more optical lenses. In other embodiments, the user may view one or more images generated, for example, on a surface that partially or wholly reflects light from another light source (e.g., a laser scan beam) in a manner other than through a display panel, through one or more optical lenses. [Exemplary Display System]

[0057] FIG. 5 is a schematic block diagram of a display system 500 according to one non-limiting exemplary implementation. The display system 500 may be a head-mounted display system, such as, for example, the display system of the head-mounted display system described above, or any other type of display system (e.g., a wearable or non-wearable display system). The display system 500 may include a scan beam display system, or another type of display system (e.g., a microdisplay). In at least some implementations, the display system 500 may be one of two substantially identical display systems provided in a device such as a head-mounted display device.

[0058] The display system 500 includes a display light source 502 optically coupled to the pupil relay system 504. The display light source 502 may include a laser scan beam source, a microdisplay, or any other suitable display light source. The pupil relay system 504 is arranged to relay a first pupil from the display light source 502 to a second pupil at the viewer's eye 506 (or other image plane, surface, or material). The pupil relay system 504 includes a polarization-sensitive optical system 508 (e.g., a post-scan optical system), which may include one or more of a waveguide-based optical system, a pancake optical system, a birdbath optical system, a coating-based optical system, or other optical systems. The polarization-sensitive optical system 508 may include one or more components. The efficiency of the polarization-sensitive optical system 508 may be very sensitive to the polarization of the light passing through it. That is, for different polarizations, the intensity and the stray light exiting the polarization-sensitive optical system 508 will vary significantly.

[0059] To optimize the polarization of the light provided to the polarization-sensitive optical system 508, the pupil relay system 504 of the display system 500 further includes a spatially-varying polarizer 510 having a spatially-varying polarization that varies depending on position, which functions to compensate for the change in polarization and provide a uniform polarization, or more generally an optimized polarization, to the polarization-sensitive optical system 508. For example, the polarization-sensitive optical system 508 may be configured to compensate for or "cancel out" any polarization generated by one or more mirrors or other optical systems at the synthetic angle.

[0060] The spatially-varying polarizer 510 may include a wavelength retarder formed of a birefringent material. Birefringence is a property of a material in which the refractive index depends on the polarization and propagation direction of light. The wavelength retarder changes the polarization state or phase of the light passing through the wavelength retarder. The wavelength retarder may have a slow axis (or extraordinary axis) and a fast axis (ordinary axis). When polarized light passes through the wavelength retarder, the light along the fast axis travels faster than the light along the slow axis.

[0061] As described above, the spatially-varying polarizer 510 may provide a phase delay that varies across the entire field of view (e.g., from on-axis to off-axis) according to position, such as lateral position, longitudinal position, or radial position, which enables a more uniform and efficient supply of light from the display light source 502 to the polarization-sensitive optical system 508. The particular manner in which the phase difference of the spatially-varying polarizer 510 varies may depend on the particular configuration and materials of the optical system of the display system 500, such as the polarization state of the incident light, the angle of incidence, the materials, the shapes of the various components, and the like.

[0062] As an example, the spatially-varying polarizer 510 may not provide a phase difference at a first position, and may linearly increase the phase difference at a second position of the spatially-varying polarizer to provide a phase difference of λ / 4 (or another value). Generally, the spatially-varying polarizer 510 may provide a phase difference that varies according to position by any method, and the amount of the phase difference may be any value (e.g., λ / 20, λ / 10, λ / 4, λ, 2λ). Further, the amount of the phase difference may increase only in one or more directions, decrease only in one or more directions, or both increase and decrease may occur. The amount of the phase difference may vary continuously or may vary in multiple steps. The amount of the phase difference may vary according to any type of function, including, for example, a linear function, a polynomial function, an exponential function, a step function, other types of functions, or a combination thereof.

[0063] In at least some implementations, the spatially-varying polarizer 510 may be formed of a multi-twist retarder (MTR), which is a waveplate-like phase difference film that provides an accurate and customized level of broadband phase difference, narrowband phase difference, or multiple-band phase difference with a single thin film. More specifically, the MTR has two or more twisted liquid crystal (LC) layers with an alignment layer on one substrate. Since the next LC layer is directly aligned by the previous layer, simple manufacturing is possible, automatic interlayer alignment is achieved, and a monolithic film with a continuously varying optical axis is provided.

[0064] FIG. 6 is a schematic diagram of a scan beam display system or projector 600 according to one non-limiting exemplary implementation. The scan beam display system 600 includes a light source 602, which may be a laser light source that emits a beam 604. The light source 602 may include two or more light sources, for example, a red light source, a green light source, and a blue light source. In such an example, a plurality of light sources may be combined by a beam combiner into a single beam. In at least some implementations, the light source 602 may include one or more color light sources (e.g., red, green, blue) and a light source that emits an invisible beam such as an infrared beam or an ultraviolet beam, which may be used for various purposes such as eye tracking.

[0065] The beam 604 is incident on a scan platform 606 that may include a microelectromechanical systems (MEMS)-based scanner and is reflected by a scan mirror 608 of the platform to generate a controlled output beam 610. The scan platform 606 may include an optical diffraction grating, a movable optical diffraction grating, a light valve, a rotating mirror, a movable silicon device, a digital light projector device, a flying spot projector, a liquid crystal on silicon (LCoS) device, or other scanning or modulating devices. The scan platform 606 may be coupled to one or more drive circuits that are selectively controlled by a controller 612 coupled to the scan platform and the light source 602, and this controller may include any suitable control circuit having one or more components. The drive circuit modulates the direction in which the scan mirror 608 deflects the incident beam 604 to generate a scan such as a raster scan on the output beam 610, whereby a display image is generated on an image plane such as a display surface or the eyes 614 of a viewer.

[0066] FIG. 7 is a schematic diagram of a scan beam display system or projector 700 according to one non-limiting exemplary implementation. The scan beam display system 700 may be similar or identical to the scan beam display system 600 of FIG. 6 in many respects. Therefore, similar components are referenced with the same numbers, and for simplicity, the description of such components will not be repeated here.

[0067] The scan beam display system 700 includes a post-scan correction optical system 702 (“post-scan optical system”) and a collimation optical system 704. The post-scan optical system 702 may include one or more optical systems disposed in the optical path after the scan platform 606, which may generally be referred to herein as “post-scan”. The post-scan correction optical system 702 may be designed and configured to correct or adjust one or more distortion artifacts in the projected image. Examples of such distortions may include smile distortion, pincushion distortion, barrel distortion, off-axis projection-based distortion, and the like. It should be understood that these are only non-limiting exemplary types of distortions that the post-scan correction optical system 702 may correct.

[0068] The post-scan optical system 702 may include one or more of a waveguide-based optical system, a pancake optical system, a birdbath optical system, a coating-based optical system, and the like. The post-scan optical system may include one or more components. The efficiency of the post-scan optical system 702 may be very sensitive to the polarization of the light passing through it. That is, for different polarizations, the intensity and the stray light exiting the post-scan optical system 702 will vary significantly. In at least some implementations, the display system 700 may also include a collimation optical system or beam-forming optical system 704 that can be used to at least partially recover the loss of infinite focus caused by the post-scan optical system 702.

[0069] The display system 700 further includes a polarization compensation optical system in the form of a spatially variant polarizer 706 disposed between the scan mirror 608 and the post - scan optics 702, providing polarization compensation for the post - scan optics 702 which may be very sensitive to polarization as described above. The spatially variant polarizer 706 may be disposed, for example, on, adjacent to, or within the post - scan optics 702. In other implementations, the spatially variant polarizer 706 is disposed at some other location in the optical path between the light source 602 and the displayed image (e.g., pre - scan, post - scan, adjacent to the light source, etc.).

[0070] In at least some implementations, the controller 612 may be operatively coupled to the spatially variant polarizer 706 to selectively vary the spatially - dependent phase retardation of the spatially variant polarizer for any desired configuration. In such implementations, one or more thin - film transistor layers may be provided to enable the spatially - dependent phase retardation of the spatially variant polarizer 706 to be selectively controlled by the controller 612. The controller 612 can control the phase retardation at any desired frequency, e.g., once, periodically, at a frequency equal to or a fraction of the frame rate of the display system 700. As an example, the spatially variant polarizer 706 may include a stack of multiple (e.g., 2, 4, 10, 15) layers, each of which may be selectively controlled independently to be in an active or non - active state. Thus, the controller 612 may then selectively activate one of these layers, or a plurality of layers in combination with each other, to provide a desired spatially - dependent phase retardation.

[0071] FIG. 8 is a schematic diagram of a display system 800 for use in a head-mounted display system, including a waveguide-based optical system and a polarization compensation optical system in the form of a spatially variable polarizer, according to one non-limiting exemplary implementation. The display system 800 may include a lens or support structure 802 (e.g., a prescription eyeglass lens or a non-prescription eyeglass lens). A planar waveguide structure 804 may be at least partially incorporated into the structure 802 or disposed proximate to this structure (e.g., in front of or behind the structure). The waveguide 804 may be a rectangular (or other shaped) prism structure formed of a material having a refractive index sufficiently different from that of the surrounding structure (e.g., structure 802) to provide total internal reflection within the waveguide.

[0072] To enable light to enter the waveguide 804, the display system 800 includes an incoupler 806 physically coupled to a first portion of the waveguide. Similarly, to enable light to exit the waveguide 804 and travel towards the viewer's eye 810, the display system 800 includes an outcoupler 808 physically coupled to a second portion of the waveguide. The display light that enters through the incoupler 806 and in-couples through the outcoupler 808 may out-couple from a display light source, such as a projector, scanner, laser projector, microdisplay, or other display light source, as described above. As non-limiting examples, the couplers 806 and 808 may include one or more of a diffraction grating, hologram, holographic optical element, volume diffraction grating, surface relief diffraction grating, etc. The couplers 806 and 808 may also be reflective couplers or transmissive couplers. As an example, the structure 802 may include a right-eye eyeglass lens, the incoupler 806 may be disposed near the edge of the eyeglass lens proximate to the display source (e.g., projector), and the outcoupler may be disposed towards the center of the eyeglass lens such that the viewer can see the light from the waveguide 804 while looking straight ahead or nearly straight ahead.

[0073] The display system 800 further includes a polarization compensation optical system in the form of a spatially variant polarizer 812, various examples of which are shown in FIG. 8 as spatially variant polarizers 812a - 812e. In the non - limiting exemplary examples shown, instances 812a - 812e of the spatially variant polarizer 812 are shown as being disposed at several non - limiting exemplary locations of the display system 800. Specifically, 812a shows a spatially variant polarizer disposed adjacent to the incoupler 806 on the side facing the display source to provide polarization compensation for the light entering the waveguide 804, which may be very sensitive to polarization as described above. The spatially variant polarizer 812b is shown in FIG. 8 as being disposed adjacent to the incoupler 806 on the opposite side thereof as compared to the exemplary spatially variant polarizer 812a. Similarly, the spatially variant polarizer 812c is shown as being disposed adjacent to the outcoupler 808 on the side facing the user's eye 810, and the spatially variant polarizer 812d is shown as being disposed adjacent to the outcoupler 808 on the opposite side thereof. The spatially variant polarizer 812e is shown as being disposed inside the waveguide 804 at a position between the incoupler 806 and the outcoupler 808. It should be understood that the spatially variant polarizer 812 may be disposed, for example, on, adjacent to, or proximate to, or inside the waveguide 804. In other implementations, the spatially variant polarizer 812 is disposed at some other location in the optical path between the display light source and the viewer's eye to provide polarization compensation.

[0074] FIG. 9 illustrates a side view of the components of an HMD system 900 according to one non-limiting, exemplary implementation. The HMD system 900 includes a display panel 902, such as an OLED, LCD, or other type of display panel, and an optical lens 904 that provides image information from the display to an eyebox 906, where the user's eyes can be positioned to view the image from the display. The lens 904 may include a single lens or multiple lenses. The HMD system 900 may include two display panels and two lenses, one for each of the user's eyes. In operation, light 910 emitted from the display panel 902 may be optically modified (e.g., focused) by the lens 904 and become light 912 directed towards the eyebox 906 for viewing by the user's eyes.

[0075] Part of the cause of polarization-based optical distortion phenomena may be the different angles of incidence of various light rays when passing through a curved optical lens such as the lens 904. Thus, in a set of pixels located along the central region of the display panel, the light emitted from this pixel may pass through the optical lens along the central axis with little or no bending of the various light rays, which may have a certain effect on the polarization of the light. Conversely, light rays passing through the optical lens 904 among the light rays located far from the central portion of the display panel 902 may have a greater degree of curvature of the optical lens at such positions and may have differently affected polarization.

[0076] To compensate for the varying effects on the polarization of light emitted from the display panel 902, in at least some implementations, a spatially varying polarization compensation optical system 908 may be provided. In the illustrated example, the polarization compensation optical system 908 may be disposed adjacent to the front surface of the display panel 902 and may optionally be adhered or laminated to the display panel with a suitable adhesive (e.g., an optically clear adhesive (OCA)). The polarization compensation optical system 908 may be formed from a phase retarder material (e.g., a wave plate), such as a multi-twist retarder, as described elsewhere herein.

[0077] The polarization compensation optical system 908 may provide a spatially varying polarization defined by a phase map. FIGS. 10 and 11 show two non-limiting examples of surface phase maps for the polarization compensation optical system 908. In the exemplary surface map 1000 of FIG. 10, the phase varies concentrically from -0.433 wavelengths to +0.433 wavelengths from the center to the outer periphery of the optical system. In the exemplary surface map 1100 of FIG. 11, the phase varies linearly from -1.25E+004, which is the lower limit of the optical system 908 (as shown in the figure), to +1.25E+004, which is the upper limit of the optical system, with each unit being a period of 2π radians. It should be noted that the phase variations of the surface maps 1000 and 1100 are shown as discrete steps for simplicity, but in reality, the phase may vary continuously across the surface of the optical system.

[0078] In at least some implementations, the surface phase map of the compensation optical system 908 may be designed to cancel or compensate for unwanted polarization generated by at least one of the display panel 902 or the lens 904. In such implementations, the phase profile or map of the optical system (e.g., the lens, or the lens and the display panel) may first be determined. The determined phase map may then be inverted and applied to the compensation optical system 908, such that the compensation optical system cancels out the unwanted effects generated by other components of the optical system.

[0079] In at least some implementations, the polarization compensation optical system 908 can improve the polarization performance of the HMD system 900 for light at low incident angles.

[0080] In addition to or as an alternative to polarization compensation, the compensation optical system 908 may be configured to shape the light from the display panel 902 to be more telecentric, such that the light arrives at the lens 904 at more uniform and telecentrically aligned angles. This feature advantageously results in an improvement in performance across the entire eyebox 906.

[0081] By using the spatially-varying polarizer described herein, there is a significant increase in the freedom for an optical designer to create an optical system with improved performance and efficiency, thereby enabling a display system that provides a better viewing experience, is less costly, smaller in size or weight, consumes less power, and provides other advantages that will be apparent to those skilled in the art.

[0082] The various implementations described above may be combined to provide further implementations. Based on the above detailed description, these and other changes can be made to each implementation. In general, in the following claims, the terms used should not be construed as limiting the claims to the specific implementations disclosed in this specification and the claims, but rather the claims should be construed to include all possible implementations together with the full scope of equivalents to which such claims are entitled. Accordingly, the claims are not limited by the present disclosure.

Claims

1. A display light source, A pupil relay system arranged to relay a first pupil from the display light source to a second pupil in the viewer's eye, A display system comprising: wherein the pupil relay system is a polarization-sensitive optical system, and a spatially-varying polarizer having spatially-varying polarization that varies according to position, for providing polarization compensation to the polarization-sensitive optical system, the spatially-varying polarizer including a stack of a plurality of layers each independently selectively controllable to be in an active state or an inactive state, and a control circuit configured to selectively actuate one or a combination of a plurality of the layers to provide a selected spatially-dependent phase delay, thereby selectively changing the spatially-dependent phase delay of the spatially-varying polarizer. A display system having the above.

2. The display system according to claim 1, wherein the spatially-varying polarizer includes a multi-twist retarder.

3. The display system according to claim 1 or 2, wherein the spatially-varying polarizer provides no phase difference at a first position and provides a quarter-wavelength phase difference at a second position.

4. The display system according to any one of claims 1 to 3, wherein the spatially-varying polarizer provides a phase delay that varies across the entire field of view according to a lateral position, a vertical position, or a radial position of the display system.

5. The display system according to any one of claims 1 to 3, wherein the phase difference of the spatially-varying polarizer varies across the entire field of view of the display system.

6. The display system according to any one of claims 1 to 5, wherein the polarization-sensitive optical system includes a waveguide-based optical system, a pancake optical system, a birdbath optical system, or a coating-based optical system.

7. The display system according to any one of claims 1 to 6, wherein the display light source has a laser light source, and the display system further includes a scan mirror arranged to receive a light beam from the laser light source and relay the received light beam toward the pupil relay system.

8. The display system according to claim 7, further comprising a beam-forming optical system arranged between the laser light source and the scan mirror.

9. The display system according to any one of claims 1 to 8, wherein at least a part of the spatially-varying polarizer is arranged on, adjacent to, or within the polarization-sensitive optical system.

10. The display system according to any one of claims 1 to 9, wherein the polarization-sensitive optical system includes a waveguide, and the spatially-varying polarizer is arranged on, inside the waveguide, or adjacent to a port of the waveguide.

11. The display system according to any one of claims 1 to 10, wherein the display system is a display system of a head-mounted display system.

12. The display system according to any one of claims 1 to 11, wherein the display light source has a microdisplay, and the spatially-varying polarizer is arranged adjacent to the microdisplay.

13. The display system according to claim 12, wherein the spatially-varying polarizer is adhered to the microdisplay.

14. The display system according to claim 12 or 13, wherein the spatially-variant polarizer imparts telecentricity to the light emitted by the microdisplay.

15. The display system according to any one of claims 12 to 14, wherein the spatially-variant polarizer includes a surface phase map that provides polarization compensation for at least one of the display light source and the polarization-sensitive optical system.

16. A support structure, A display system coupled to the support structure, A head-mounted display system comprising: wherein the display system includes a display light source, a pupil relay system arranged to relay a first pupil from the display light source to a second pupil in the viewer's eye, and wherein the pupil relay system includes a polarization-sensitive optical system, and a spatially-variant polarizer having polarization that varies spatially according to position to provide polarization compensation for the polarization-sensitive optical system, the spatially-variant polarizer including a stack of a plurality of layers each independently selectively controllable to be in an active state or an inactive state, and a control circuit configured to selectively vary the spatially-dependent phase delay of the spatially-variant polarizer by selectively activating one or a combination of the plurality of layers to provide a selected spatially-dependent phase delay. A head-mounted display system comprising.

17. The head-mounted display system according to claim 16, wherein the spatially-variant polarizer includes a multi-twist retarder.

18. The head-mounted display system according to claim 16 or 17, wherein the spatially-varying polarizer provides a phase delay that varies across the entire field of view according to a horizontal position, a vertical position, or a radial position of the display system.

19. The head-mounted display system according to any one of claims 16 to 18, wherein the polarization-sensitive optical system includes a waveguide-based optical system, a pancake optical system, a birdbath optical system, or a coating-based optical system.

20. A support structure, A display system connected to the support structure, and A head-mounted display system comprising: wherein the display system includes A laser light source, A scan mirror arranged to receive a light beam from the laser light source, A pupil relay system arranged to relay a first pupil received from the scan mirror to a second pupil in the viewer's eye, and has wherein the pupil relay system includes A polarization-sensitive optical system, A spatially-varying polarizer having a spatially-varying polarization that varies according to position to provide polarization compensation for the polarization-sensitive optical system, and including a stack of a plurality of layers that can be selectively and independently controlled to be in an active state or an inactive state respectively, and A control circuit configured to selectively change the spatially-dependent phase delay of the spatially-varying polarizer by selectively activating one or a combination of a plurality of the plurality of layers to provide a selected spatially-dependent phase delay. A head-mounted display system including.

21. The head-mounted display system according to claim 20, wherein the spatially-varying polarizer includes a multi-twist retarder.

22. The head-mounted display system according to claim 20 or 21, wherein the spatially-varying polarizer provides a phase delay that varies across the entire field of view according to a horizontal position, a vertical position, or a radial position of the display system. **Claim 23** The head-mounted display system according to any one of claims 20 to 22, wherein the polarization-sensitive optical system includes a waveguide-based optical system, a pancake optical system, a birdbath optical system, or a coating-based optical system.

Images