Extended reality systems and methods

Abstract

An extended reality display system includes a display subsystem configured to present an image corresponding to image data to a user. The display subsystem includes an optical component that introduce a non-uniformity to the image, a segmented illumination light source, and a spatial light modulator (SLM) configured to receive light from the segmented illumination light source. The system also includes a display controller configured to control the segmented illumination light source. The display controller includes a memory for storing non-uniformity correction information, and a processor to control the segmented illumination light source based on the non-uniformity correction information. The segmented illumination light source is configured to differentially illuminate first and second portions of the SLM using respective first and second portions of the segmented illumination light source.

Description

EXTENDED REALITY SYSTEMS AND METHODSIncorporation By Reference

[0001] This application claims priority to U.S. Provisional Application Ser. No. 63 / 503,667, filed on May 22, 2023, the contents of which are hereby expressly and fully incorporated by reference in its entirety. This application also expressly and fully incorporates by reference the entirety of each of the following patent applications: U.S. Utility Patent Application Serial No. 14 / 555,585 filed on November 27, 2014 under attorney docket number ML.20011.00 and entitled “VIRTUAL AND AUGMENTED REALITY SYSTEMS AND METHODS”; U.S. Utility Patent Application Serial No. 14 / 738,877 filed on June 13, 2015 under attorney docket number ML.20019.00 and entitled “METHODS AND SYSTEMS FOR CREATING VIRTUAL AND AUGMENTED REALITY”; U.S. Utility Patent Application Serial No. 15 / 683,677 filed on August 22, 2017 under attorney docket number ML-0341 US and entitled “VIRTUAL, AUGMENTED, and MIXED REALITY SYSTEMS AND METHODS”; and U.S. Utility Patent Application Serial No. 16 / 215,477 filed on December 10, 2018 under attorney docket number 101782- 013710US-1347331 and entitled “WAVEGUIDE ILLUMINATOR”.Copyright Notice

[0002] A portion of the disclosure of this patent document contains material that is subject to copyright protection. The copyright owner has no objection to the facsimile reproduction by anyone of the patent document or the patent disclosure as it appears in the Patent and Trademark Office patent file or records, but otherwise reserves all copyright rights whatsoever.Field of the Invention

[0003] The present disclosure relates to extended reality imaging, visualization, and display systems and methods. In particular, the present disclosure relates to segmented illumination light sources for use with display systems.Background

[0004] Modern computing and display technologies have facilitated the development of virtual reality (“VR”), augmented reality (“AR”), mixed reality (“MR”), and extended reality (“XR”) systems. VR systems create a simulated environment for a user to experience. This can be done by presenting computer-generated imagery to the user through a head-mounted display. This imagery creates a sensory experience which immerses the user in the simulated environment. A VR scenario typically involves presentation of only computer-generated imagery rather than also including actual real- world imagery.

[0005] AR systems generally supplement a real-world environment with simulated elements. For example, AR systems may provide a user with a view of the surrounding real-world environment via a head-mounted display. However, computer-generated imagery can also be presented on the display to enhance the real-world environment. This computer-generated imagery can include elements which are contextually-related to the real-world environment. Such elements can include simulated text, images, objects, etc. MR systems also introduce simulated objects into a real-world environment, but these objects typically feature a greater degree of interactivity than in AR systems. The simulated elements can often times be interactive in real time. XR systems generallyfacilitate immersive technologies and includes VR systems, AR systems, and MR systems.

[0006] Figure 1 depicts an example AR / MR / XR scene 1 where a user sees a real- world park setting 6 featuring people, trees, buildings in the background, and a concrete platform 20. In addition to these items, computer-generated imagery is also presented to the user. The computer-generated imagery can include, for example, a robot statue 10 standing upon the real-world platform 20, and a cartoon-like avatar character 12 flying by which seems to be a personification of a bumble bee, even though these elements 12, 10 are not actually present in the real-world environment.

[0007] Various optical systems generate images at various depths for displaying VR, AR, MR, orXR scenarios. Some such optical systems are described in U.S. Utility Patent Application Serial No. 14 / 555,585, the contents of which have been previously incorporated by reference herein. Other such optical systems for displaying MR experiences are described in U.S. Utility Patent Application Serial No. 14 / 738,877, the contents of which have been previously incorporated by reference herein.

[0008] Because the human visual perception system is complex, it is challenging to produce a VR / AR / MR / XR technology that facilitates a comfortable, natural-feeling, rich presentation of virtual image elements amongst other virtual or real-world imagery elements. Improved techniques are needed for processing image data in such systems, including techniques for correcting non-uniform ities over the system field of view (“FOV”) resulting from system component non-uniform ities. VR / AR / MR / XR technology also has size and portability issues, battery life issues, system over-heating issues, and other system and optical challenges. Improved techniques are needed for addressing theseissues, including system temperature management. VR / AR / MR / XR display systems also have a plurality of optical components, which can introduce non-uniform ities to images generated by the VR / AR / MR / XR display systems. The systems and methods described herein are configured to address at least some of these and other challenges.

[0009] What is needed is a technique or techniques to improve over legacy techniques and / or over other considered approaches. Some of the approaches described in this background section are approaches that could be pursued, but not necessarily approaches that have been previously conceived or pursued.Summary

[0010] Embodiments are directed to VR, AR, MR, or XR systems and methods. In particular, the embodiments are directed to VR, AR, MR, or XR systems and methods for correcting non-uniformities using light sources having multiple segments that emit light (instead of a single segment). Such light sources are called “segmented illumination light sources”. Segmented illumination light sources are configured such that the brightness of each segment of the light source can be increased (e.g., up to 100% brightness) or decreased (e.g., down to 0% or completely off) independent of the other segments of the segmented illumination light source.

[0011] In one embodiment, an extended reality display system includes a display subsystem configured to present an image corresponding to image data to a user. The display subsystem includes an optical component that introduces a non-uniform ity to the displayed image, a segmented illumination light source, and a spatial light modulator (SLM) configured to receive light from the segmented illumination light source. Thesystem also includes a display controller configured to control the segmented illumination light source. The display controller includes a memory for storing non-uniform ity correction information, and a processor to control the segmented illumination light source base on the non-uniform ity correction information. The segmented illumination light source is configured to differentially illuminate first and second portions of the SLM using respective first and second portions of the segmented illumination light source.

[0012] In one or more embodiments, the segmented illumination light source includes a light emitting diode (LED) array, and the display controller is configured to control each LED of the LED array. The SLM may be a liquid crystal on silicon (LCDS) display. The system may also include a lens disposed between the segmented illumination light source and the SLM. The segmented illumination light source may be configured to improve a uniformity of illumination of the image corresponding to the image data by differentially / independently illuminating first and second portions of the SLM. The uniformity may be an illumination uniformity and / or a color uniformity. The segmented illumination light source may be configured to reduce power consumption by differentially / independently illuminating first and second portions of the SLM. For instance, power consumption can be reduced by disabling the LEDs in areas where no virtual content will be displayed in a frame. The segmented illumination light source may be configured to improve a contrast of the image corresponding to the image data by differentially illuminating first and second portions of the SLM.

[0013] In one or more embodiments, the segmented illumination light source includes a plurality of sub-light sources, and the display controller is configured to control an amount of voltage or current applied to a sub-light source of the plurality of sub-lightsources. The processor may be configured to calculate the amount of current delivered to the sub-light source using the non-uniform ity correction information stored in the memory. The processor may be configured to calculate the amount of current delivered to the sub-light source using LED and optical stack calibration data, which takes into account temperature and per device optical performance. The display controller may be directly coupled to the segmented illumination light source with only a connection element between the display controller and the segmented illumination light source.

[0014] In one or more embodiments, the display controller is configured to instruct the segmented illumination light source to not illuminate a third portion of the SLM by directing a zero voltage or current to be applied to the third portion of the segmented illumination light source. The display controller may also instruct the segmented illumination light source to not illuminate a third portion of the SLM by directing a reverse bias current to be applied to the third portion of the segmented illumination light source. The display controller may be configured to instruct the segmented illumination light source to increase respective first and second amounts of light generated by the first portion and the second portion of the segmented illumination light source. The display controller may be configured to monitor respective first and second temperatures of the first portion and the second portion of the segmented illumination light source.

[0015] In one or more embodiments, the display controller can control both the SLM and segmented illumination light source independently or synchronously. Independent control of the SLM and segmented illumination light source can be used for static nonuniformity correction. Synchronous control of the SLM and segmented illumination light source can be used to further improve non-uniformity correction, contrast, and powersavings. The segmented illumination (“SGIL”) light source can have an integrated driver ASIC with a processor to control the segmented illumination light source with pulse width modulation and / or pulse-amplitude modulation. The display controller (e.g. processor) can communicate with the SGIL ASIC for coordinated control. Additionally, the SGIL ASIC can be configured to perform some processing and calibration for non-uniform ity correction or the non-uniform ity correction can be controlled by the display controller processor. In addition to non-uniform ity correction, the segmented illumination light source can perform LED calibration for feed forward color balancing control (e.g., spectra / intensity vs temp and current density) and gain maps for non-uniform ity correction. The segmented illumination light source can have more than two portions / zones. For example, segmented illumination light source can include various arrays (e.g., 5x5, 8x8, etc.) of independently controlled segmented illumination pixels / LEDs.

[0016] In another embodiment, a method is for presenting an image corresponding to image data to a user using a display subsystem including an optical component that introduces a non-uniform ity to the image. The method includes a display controller storing non-uniform ity correction information in a memory thereof. The method also includes a processor of the display controller controlling a segmented illumination light source of the display subsystem based on the non-uniform ity correction information. The method further includes the segmented illumination light source differentially illuminating first and second portions of an SLM of the display subsystem using respective first and second portions of the segmented illumination light source under control of the processor.

[0017] In one or more embodiments, the segmented illumination light source includes an LED array, and the method further includes the processor of the display controller controlling each LED of the LED array. Other light sources suitable for segmented illumination systems include, but are not limited to, mini-LEDs, micro-LEDs, organic light emitting diodes, vertical-cavity surface-emitting lasers, super-luminescent diodes, and laser diodes. The SLM may be an LCOS display. A lens may be disposed between the segmented illumination light source and the SLM in the display subsystem. The segmented illumination light source differentially illuminating first and second portions of the SLM may improve a uniformity of illumination of the image corresponding to the image data. The uniformity may be an illumination uniformity and / or a color uniformity. The segmented illumination light source differentially illuminating first and second portions of the SLM may reduce power consumption. The segmented illumination light source differentially illuminating first and second portions of the SLM may improve a contrast of the image corresponding to the image data.

[0018] In one or more embodiments, the segmented illumination light source includes a plurality of sub-light sources, and the method further include the processor controlling an amount of voltage or current applied to a sub-light source of the plurality of sub-light sources. The method may further include the processor calculating the amount of current delivered to the sub-light source using the non-uniform ity correction information stored in the memory. The display controller may be directly coupled to the segmented illumination light source with only a connection element between the display controller and the segmented illumination light source.

[0019] In one or more embodiments, the method further includes the processor instructing the segmented illumination light source to not illuminate a third portion of the SLM by directing a zero voltage or current to be applied to the third portion of the segmented illumination light source. The processor may also instruct the segmented illumination light source to not illuminate a third portion of the SLM by directing a reverse bias current to be applied to the third portion of the segmented illumination light source. The method may further include the processor instructing the segmented illumination light source to increase respective first and second amounts of light generated by the first portion and the second portion of the segmented illumination light source. The method may further include the processor monitoring respective first and second temperatures of the first portion and the second portion of the segmented illumination light source.

[0020] In one or more embodiments, the method further includes the processor obtaining new non -uniformity correction information. The method also includes the display controller storing new non-uniform ity correction information in the memory. Moreover, the method includes the processor controlling the segmented illumination light source of the display subsystem based on the new non-uniform ity correction information. In addition, the method includes the segmented illumination light source differentially illuminating the first and second portions of the SLM of the display subsystem using the respective first and second portions of the segmented illumination light source under control of the processor.

[0021] In still another embodiment, a segmented illumination light source includes an application-specific integrated circuit (ASIC) comprising a bond pad array. The light source also includes a first LED array coupled to a first portion of the bond pad array.The light source further includes a second LED array coupled to a second portion of the bond pad array. Moreover, the light source includes a third LED array coupled to a third portion of the bond pad array. The first, second, and third portions are distinct from each other. The ASIC is configured to control the first LED array, the second LED array, and the third LED array.

[0022] In one or more embodiments, the first LED array is configured to emit red light, the second LED array is configured to emit green light, and the third LED array is configured to em it blue light. The first portion of the bond pad array may include a plurality of portions in which the first LED array can be coupled. The bond pad array may include a plurality of portions in which an LED array can be coupled.

[0023] In yet another embodiment, a method for manufacturing a plurality of segmented illumination light sources includes coupling a first LED array to a first portion of a first bond pad array of a first application-specific integrated circuit (ASIC) to form a first segmented illumination light source. The method also includes coupling a second LED array to a second portion of a second bond pad array of a second ASIC to form a first segmented illumination light source. The first ASIC is identical to the second ASIC. The first portion is located at a first location on the first bond pad array. The second portion is located at a second location on the second bond pad array. The first location is different from the second location.

[0024] In one or more embodiments, the first segmented illumination light source is configured for use with a first optical device having a first optical design, and the second segmented illumination light source is configured for use with a second optical device having a second optical design different from the first optical design. The method mayalso include determining the first portion of the first bond pad array of the first ASIC based on the first optical design of the first optical device, and determining the second portion of the second bond pad array of the second ASIC based on the second optical design of the second optical device.

[0025] The aforementioned and other embodiments of the invention are described in the Detailed Description which follows.Brief Description of the Drawings

[0026] The drawings described below are for illustration purposes only. The drawings are not intended to limit the scope of the present disclosure.

[0027] The drawings illustrate the design and utility of various embodiments of the present disclosure. It should be noted that the figures are not drawn to scale and that elements of similar structures or functions are represented by like reference numerals throughout the figures. In order to better appreciate how to obtain the recited and other advantages and objects of various embodiments of the disclosure, a more detailed description of the present disclosure will be rendered by reference to specific embodiments thereof, which are illustrated in the accompanying drawings. Understanding that these drawings depict only typical embodiments of the disclosure and are not therefore to be considered limiting of its scope, the disclosure will be described and explained with additional specificity and detail through the use of the accompanying drawings.

[0028] Figure 1 depicts a user’s view of an AR / MR / XR scene using an exemplary AR / MR / XR system, according to some embodiments.

[0029] Figure 2 depicts an example of wearable display system, according to some embodiments.

[0030] Figure 3 is a block diagram depicting an AR / MR / XR system, according to some embodiments.

[0031] Figure 4A depicts a conventional display system for simulating three- dimensional imagery for a user.

[0032] Figure 4B depicts aspects of an approach for simulating three-dimensional imagery using multiple depth planes, according to some embodiments.

[0033] Figures 5A-5C depict relationships between radius of curvature and focal radius.

[0034] Figure 6 depicts an example of a waveguide stack for outputting image information to a user, according to some embodiments.

[0035] Figure 7 depicts an example of exit beams outputted by a waveguide, according to some embodiments.

[0036] Figure 8 depicts an example design of a waveguide stack in which each depth plane has three associated waveguides that each output light of a different color, according to some embodiments.

[0037] Figures 9-11 depict systems for presenting images to a user’s eye, and for viewing the world, according to some embodiments.

[0038] Figure 12 depicts a fixed illumination pattern generated by a display system without distortion correction.

[0039] Figures 13A-13C depict brightness uniformity correction by segmented illumination, according to some embodiments.

[0040] Figures 14A-14B depict color uniformity correction by segmented illumination, according to some embodiments.

[0041] Figure 15 depicts a segmented illumination pattern configured to reduce power consumption of a display system, according to some embodiments.

[0042] Figures 16A-16B depict contrast improvement by segmented illumination, according to some embodiments.

[0043] Figure 17 depicts a method for displaying an image with segmented illumination, according to some embodiments.

[0044] Figures 18-20 depicts a segmented illumination light sources, according to some embodiments.

[0045] Figures 21A to 27 schematically depict various light sources, according to some embodiments.

[0046] Figure 28 schematically depicts a display system, according to some embodiments.

[0047] Figure 29 schematically depicts in detail the display system of Figure 28.

[0048] Figure 30 schematically depicts in detail a display system, according to some embodiments.

[0049] Figure 31 depicts a method for manufacturing different optical devices having different optical designs using identical application-specific integrated circuits, according to some embodiments.

[0050] Figure 32 is a block diagram schematically depicting an illustrative computing system, according to some embodiments.Detailed Description

[0051] Various embodiments of the disclosure are directed to systems, methods, and articles of manufacture for VR / AR / MR / XR in a single embodiment or in multiple embodiments. Other objects, features, and advantages of the disclosure are described in the detailed description, figures, and claims.

[0052] Various embodiments will now be described in detail with reference to the drawings, which are provided as illustrative examples so as to enable those skilled in the art to practice the disclosure. Notably, the figures and the examples below are not meant to limit the scope of the present disclosure. Where certain elements of the present disclosure may be partially or fully implemented using known components (or methods or processes), only those portions of such known components (or methods or processes) that are necessary for an understanding of the present disclosure will be described, and the detailed descriptions of other portions of such known components (or methods or processes) will be omitted so as not to obscure the disclosure. Further, various embodiments encompass present and future known equivalents to the components referred to herein by way of illustration.

[0053] Embodiments in accordance with the present disclosure address the problem of implementation of VR / AR / MR / XR systems often rely on combinations of off-the-shelf- components and custom components. In some cases the off-the-shelf components do not possess all of the features or performance characteristics that are needed to implement certain desired aspects of the to-be-deployed VR / AR / MR / XR system. Some embodiments are directed to approaches for adding capabilities and / or repurposing resources to accommodate the desired features or performance characteristics of the to-be-deployed VR / AR / MR / XR system. The accompanying figures and discussions herein present example environments, systems, methods, and computer program products for VR / AR / MR / XR systems.Overview

[0054] VR, AR, MR, and XR systems disclosed herein can include a display which presents computer-generated imagery (video / image data) to a user. In some embodiments, the display systems are wearable, which may advantageously provide a more immersive VR / AR / MR / XR experience. Figure 2 illustrates an example of wearable VR / AR / MR / XR display system 80 (hereinafter referred to as “system 80”). The system 80 includes a display 62, and various mechanical and electronic modules and systems to support the functioning of the display 62. The display 62 may be coupled to a frame 64, which is wearable by a display system user or viewer 60 (hereinafter referred to as “user 60”) and which is configured to position the display 62 in front of the eyes of the user 60. In some embodiments, a speaker 66 is coupled to the frame 64 and positioned adjacent the ear canal of the user 60. In some embodiments, another speaker, not shown, is positioned adjacent the other ear canal of the user 60 to provide for stereo / shapeable sound control. The display 62 is operatively coupled, such as by a wired or wireless connection 68, to a local processing and data module 70 which may be mounted in a variety of configurations, such as fixedly attached to the frame 64, fixedly attached to a helmet or hat worn by the user, embedded in headphones, or otherwise removably attached to the user 60 (e.g., in a backpack-style configuration, in a belt-coupling style configuration, etc.) In some embodiments, the local processing and data module 70 may be integrated into the headset / glasses.

[0055] The local processing and data module 70 may include a processor, as well as digital memory, such as non-volatile memory (e.g., flash memory), both of which may be utilized to assist in the processing and storing of data. This includes data captured from sensors, such as ambient light sensors (“ALS”), image capture devices (e.g., cameras), microphones, inertial measurement units, accelerometers, compasses, GPS units, radio devices, antenna arrays, depth sensors, and / or gyros. An ALS may be configured to provide light data for processors and algorithms for segmented illumination brightness control. The sensors may be operatively coupled to the frame 64 or otherwise attached to the user 60. Alternatively, or additionally, sensor data may be acquired and / or processed using a remote processing module 72 and / or a remote data repository 74, possibly for passage to the display 62 after such processing or retrieval. The local processing and data module 70 may be operatively coupled by communication links (76, 78), such as via a wired or wireless communication links, to the remote processing module 72 and remote data repository 74 such that these remote modules (72, 74) are operatively coupled to each other and available as resources to the local processing and data module 70.

[0056] In some embodiments, the remote processing module 72 may include one or more processors configured to analyze and process data (e.g., sensor data and / or image information). In some embodiments, the remote data repository 74 may include a digital data storage facility, which may be available through the internet or other networking configuration in a “cloud” resource configuration. In some embodiments, all data is stored and all computations are performed in the local processing and data module, allowing fully autonomous use from a remote module.

[0057] In some embodiments, the computer-generated image data provided via the display 62 can create the impression of being three-dimensional. This can be done, for example, by presenting stereoscopic images corresponding to the image data to the user 60. In some conventional systems, such image data can include separate images of a scene or object from slightly different perspectives. The separate images can be presented to the right eye and left eye of the user 60, respectively, thus simulating binocular vision and its associated depth perception.

[0058] Referring now to Figure 3, an exemplary embodiment of an AR or MR system 3300 (hereinafter referred to as “system 3300”) is illustrated. The system 3300 uses stacked light guiding optical element (hereinafter referred to as ““LOEs 3390”). The system 3300 generally includes an image generating processor 3310, a light source 3320, a controller / driver 3330, a spatial light modulator (“SLM”) 3340, and at least one set of stacked LOEs 3390 that functions as a multiple plane focus system. The system 3300 may also include an eye-tracking subsystem 3350. It should be appreciated that other embodiments may have multiple sets of stacked LOEs 3390.

[0059] The image generating processor 3310 is configured to generate virtual content to be displayed to the user. The image generating processor 3310 may convert an image or video associated with the virtual content to a format that can be projected to the user in 3D. For example, in generating 3D content, the virtual content may need to be formatted such that portions of a particular image are displayed at a particular depth plane while others are displayed at other depth planes. In one embodiment, all of the image may be generated at a particular depth plane. In another embodiment, the image generating processor 3310 may be programmed to provide slightly different images to theright and left eyes such that when viewed together, the virtual content appears coherent and comfortable to the user’s eyes.

[0060] The image generating processor 3310 may further include a memory 3312, a GPU 331 , a CPU 3316, and other circuitry for image generation and processing. The image generating processor 3310 may be programmed with the desired virtual content to be presented to the user of the system 3300. It should be appreciated that in some embodiments, the image generating processor 3310 may be housed in the system 3300. In other embodiments, the image generating processor 3310 and other circuitry may be housed in a belt pack that is coupled to the system 3300.

[0061] The image generating processor 3310 is operatively coupled to the light source 3320 which projects light associated with the desired virtual content and one or more spatial light modulators 3340. The image generating processor 3310 and / or the controller / driver 3330 may also calibrate and correct for system non-uniform ities. For example, EP non-uniform ities or segmented illumination non-uniform ities, which result from exposure to high temperatures and aging. The image generating processor 3310 may also warp / transform images with low latencies to account for user motions. This warp / transform can also be applied to the segmented illumination. The light source 3320 is compact and has high resolution.

[0062] The light source 3320 is operatively coupled to the controller / driver 3330. The light source 3320 may be connected to the controller / driver 3330 in various embodiments. In some embodiments, the light source 3320 and the controller / driver 3330 are co-located on the same board. Those versed in the art would appreciate the value of minimizing the effect of cable inductance and capacitance by co-locating light sources to “driver”electronics. Minimizing the impact of coupled inductance and capacitance by co-locating the light source 3320 and the controller / driver 3330 facilitates faster transition (i.e. , “ON” to “OFF” and all the intervening illumination levels) of the light source 3320. In embodiments in which the light source 3320 and the controller / driver 3330 are located farther away from each other, the stray inductance and capacitance resulting from the increased distance between the light source 3320 and the controller / driver 3330 may be managed by proper cable construction, such as that used in a Flex cable, which can provide shielding.

[0063] The light source 3320 is coupled to the controller / driver 3330 electrically to allow the controller / driver 3330 to drive the segmented illumination segments. The controller / driver 3330 can also connect to both the image generating processor 3310 and the SLM 3340 electrically to facilitate coordinated control. The image generating processor 3310 may send the processed illumination array data (gray values for each color) to the controller / driver 3330. The controller / driver 3330 may also include uniformity correction tables / processing capabilities. The SLM 3340 may send timing signals to the controller / driver 3330 to facilitate the segmented illumination at the appropriate time(s). In some embodiments, electronically connecting the controller / driver 3330 to the SLM 3340 enables the light source 3320 to emit the correct LED colors when the SLM 3340 is refreshing a color field for a field sequential display system.

[0064] The light source 3320 may be include color specific light emitting diodes (“LEDs”) disposed in various geometric configurations. Alternatively, the light source 3320 may include LEDs of like color, each one linked to a specific region of the field of view of the display. In some embodiments, the light source 3320 may include m ini LEDs,micro LEDs, laser diodes, phosphor converted LED sources, vertical-cavity surfaceemitting laser (“VSCELs”), and / or super luminescent diodes (“SLDs”). In some embodiments, the light source 3320 may include a mask overlay for segmentation of emission areas and positions. Although the light source 3320 is directly connected to the system 3300 in Figure 3, the light source 3320 may be connected to the system 3300 via optical fibers (not shown). The system 3300 may also include condenser (not shown) configured to collimate the light from the light source 3320.

[0065] The SLM 3340 may be reflective (e.g., an LCDS, an FLCOS, a DLP DMD, or a MEMS mirror system) or transmissive (e.g., an LCD) in various exemplary embodiments. The type of SLM 3340 (e.g., speed, size, etc.) can be selected to improve the creation of the 3D perception. While DLP DMDs operating at higher refresh rates may be easily incorporated into stationary systems 3300, wearable systems 3300 may use DLPs of smaller size and power. The power of the DLP changes how 3D depth planes / focal planes are created. The image generating processor 3310 is operatively coupled to the SLM 3340, which encodes the light from the light source 3320 with the desired virtual content. Light from the light source 3320 may be encoded with the image information when it reflects off of, emits from, or passes through the SLM 3340.

[0066] Light from the SLM 3340 is directed to the LOEs 3390 such that light beams encoded with image data for one depth plane and / or color by the SLM 3340 are effectively propagated along a single LOE 3390 for delivery to an eye of a user. Each LOE 3390 is configured to project an image or sub-image that appears to originate from a desired depth plane or FOV angular position onto a user’s retina. The light source 3320 andLOEs 3390 can therefore selectively project images (synchronously encoded by the SLM3340 under the control of controller / driver 3330) that appear to originate from various depth planes or positions in space. By sequentially projecting images using each of the light source 3320 and LOEs 3390 at a sufficiently high frame rate (e.g., 360 Hz for six depth planes at an effective full-volume frame rate of 60 Hz), the system 3300 can generate a 3D image of virtual objects at various depth planes that appear to exist simultaneously in the 3D image.

[0067] The controller / driver 3330 is in communication with and operatively coupled to the image generating processor 3310, the light source 3320 and the SLM 3340 to coordinate the synchronous display of images by instructing the SLM 3340 to encode the light beams from the light source 3320 with appropriate image information from the image generating processor 3310.

[0068] The system 3300 also includes an optional eye-tracking subsystem 3350 that is configured to track the user’s eyes and determine the user’s focus. In one embodiment, the system 3300 is configured to illuminate a subset of LOEs 3390, based on input from the eye-tracking subsystem 3350 such that the image is generated at a desired depth plane that coincides with the user’s focus / accommodation. For example, if the user’s eyes are parallel to each other, the system 3300 may illuminate the LOE 3390 that is configured to deliver collimated light to the user’s eyes, such that the image appears to originate from optical infinity. In another example, if the eye-tracking subsystem 3350 determines that the user’s focus is at 1 meter away, the LOE 3390 that is configured to focus approximately within that range may be illuminated instead.

[0069] Figure 4A illustrates a conventional display system for simulating threedimensional images corresponding to image data to a user (e.g., the user 60). Twodistinct images 84 and 86, one for each eye 4 and 5, are outputted to the user. The images 84 and 86 are spaced from the eyes 4 and 5 by a distance 12 along an optical or z-axis parallel to the line of sight of the user 60. The images 84 and 86 are flat and the eyes 4 and 5 may focus on the images by assuming a single accommodated state. Such systems rely on the human visual system to combine the images 84 and 86 to provide a perception of depth for the combined image.

[0070] It will be appreciated, however, that the human visual system is more complicated and providing a realistic perception of depth is more challenging. For example, many users of conventional 3D display systems find such systems to be uncomfortable or may not perceive a sense of depth at all. Without being limited by theory, it is believed that users viewing an object may perceive the object as being “three- dimensional” due to a combination of vergence and accommodation. Vergence movements (i.e., rolling movements of the pupils toward or away from each other to converge the lines of sight of the eyes to fixate upon an object) of the two eyes relative to each other are closely associated with focusing (or “accommodation”) of the lenses of the eyes. Under normal conditions, changing the focus of the lenses of the eyes, or accommodating the eyes, to change focus from one object to another object at a different distance will automatically cause a matching change in vergence to the same distance, under a relationship known as the “accommodation-vergence reflex.” Likewise, a change in vergence will trigger a matching change in accommodation, under normal conditions. As noted herein, many stereoscopic display systems display a scene using slightly different presentations (and, so, slightly different images) to each eye such that a threedimensional perspective is perceived by the human visual system. Such systems areuncomfortable for many users since they simply provide different presentations of a scene but with the eyes viewing all the image information at a single accommodated state, and thus work against the accommodation-vergence reflex. Systems that provide a better match between accommodation and vergence may form more realistic and comfortable simulations of three-dimensional images corresponding to image data.

[0071] For example, light field video data can be presented to the user to simulate a three-dimensional view. Light field video data can mimic the rays of light which enter the eyes of the user 60 in a real-world environment. For example, when displaying light field video data, light rays from objects that are simulated to be perceived at a distance are made to be more collimated when entering the eyes of the user, while light rays from objects that are simulated to be perceived nearby are made to be more divergent. Thus, the angles at which light rays from objects in a scene enter the eyes of the user are dependent upon the simulated distance of those objects from the viewer. Light field video data in a VR / AR / MR / XR system can include multiple images of a scene or object from different depth planes. The images may be different for each depth plane (e.g., one depth plane may include imagery corresponding to the foreground of a scene, while another depth plane includes imagery corresponding to the background of the scene) and may be separately focused by the viewer’s eyes, thereby helping to provide the user with a comfortable perception of depth.

[0072] When these multiple depth plane images are presented to the viewer simultaneously or in quick succession, the result is interpreted by the viewer as three- dimensional imagery. When the viewer experiences this type of light field video data, the eyes accommodate to focus the different depth planes in much the same way as theywould do when experiencing a real-world scene. These focal cues can provide for a more realistic simulated three-dimensional environment.

[0073] In some configurations, at each depth plane, a full color image may be formed by overlaying component images that each have a particular component color. For example, red, green, and blue images may each be separately outputted to form each full color depth plane image. As a result, each depth plane may have multiple component color images associated with it.

[0074] Figure 4B illustrates aspects of an approach for simulating three-dimensional imagery using multiple depth planes. Objects at various distances from eyes 4 and 5 on the z-axis are accommodated by the eyes (4, 5) so that those objects are in focus. The eyes 4 and 5 assume particular accommodated states to bring into focus objects at different distances along the z-axis. Consequently, a particular accommodated state may be said to be associated with a particular one of depth planes 14, such that objects or parts of objects in a particular depth plane are in focus when the eye is in the accommodated state for that depth plane. In some embodiments, three-dimensional images corresponding to image data may be simulated by providing different presentations of images corresponding to the image data for each of the eyes (4, 5), and also by providing different presentations of images corresponding to the image data corresponding to each of the depth planes.

[0075] The distance between an object and the eye (4 or 5) can change the amount of divergence of light from that object, as viewed by that eye. Figures 5A-5C illustrate relationships between distance and the divergence of light rays. The distance between the object and the eye 4 is represented by, in order of decreasing distance, R1 , R2, andR3. As shown in Figures 5A-5C, the light rays become more divergent as distance from the eye 4 to the object decreases. As distance from the eye 4 to the object increases, the light rays become more collimated. Stated another way, it may be said that the light field produced by a point (the object or a part of the object) has a spherical wavefront curvature, which is a function of how far away the point is from the eye 4 of the user. The curvature increases with decreasing distance between the object and the eye 4. Consequently, at different depth planes, the degree of divergence of light rays is also different, with the degree of divergence increasing with decreasing distance between depth planes and the eye 4. While only a single eye 4 is illustrated for clarity of illustration in Figures 5A-5C and other figures herein, it will be appreciated that the discussions regarding eye 4 may be applied to both eyes (4 and 6) of a viewer.

[0076] Without being limited by theory, it is believed that the human eye typically can interpret a finite number of depth planes to provide depth perception. Consequently, a highly believable simulation of perceived depth may be achieved by providing, to the eye, different presentations of images corresponding to image data corresponding to each of these limited number of depth planes.

[0077] Figure 6 illustrates an example of a waveguide stack for outputting images corresponding to image data to a user (e.g., the user 60). A display system 1000 includes a stack of waveguides, or stacked waveguide assembly 178, that may be utilized to provide three-dimensional perception to the eye / brain using one or more waveguides (182, 184, 186, 188, 190). In some embodiments, the display system 1000 is the system 80 of Figure 2, with Figure 6 schematically showing some parts of the system 80 in greaterdetail. For example, the stacked waveguide assembly 178 may be integrated into the display 62 of Figure 2.

[0078] The stacked waveguide assembly 178 may also include one or more features (198, 196, 194, 192) between the waveguides. In some embodiments, the features (198, 196, 194, 192) may be lenses. The waveguides (182, 184, 186, 188, 190) and / or the lenses (198, 196, 194, 192) may be configured to send light corresponding to image data to the eye with various levels of wavefront curvature or light ray divergence. Each waveguide level may be associated with a particular depth plane and may be configured to output light corresponding to image data corresponding to that depth plane. Image injection devices (200, 202, 204, 206, 208) may be utilized to inject light corresponding to image data into the waveguides (182, 184, 186, 188, 190), each of which may be configured, as described herein, to distribute incoming light across each respective waveguide, for output toward the eye 4. Light exits an output surface (300, 302, 304, 306, 308) of the image injection devices (200, 202, 204, 206, 208) and is injected into a corresponding input edge (382, 384, 386, 388, 390) of the waveguides (182, 184, 186, 188, 190). In some embodiments, a single beam of light (e.g., a collimated beam) may be injected into each waveguide to output an entire field of cloned collimated beams that are directed toward the eye 4 at particular angles (and amounts of divergence) corresponding to the depth plane associated with a particular waveguide.

[0079] In some embodiments, the image injection devices (200, 202, 204, 206, 208) are discrete displays that each produce light corresponding to image data for injection into a corresponding waveguide (182, 184, 186, 188, 190, respectively). In some other embodiments, the image injection devices (200, 202, 204, 206, 208) are the output endsof a single multiplexed display which may pipe light corresponding to image data via one or more optical conduits (such as fiber optic cables) to each of the image injection devices (200, 202, 204, 206, 208).

[0080] A controller 210 controls the operation of the stacked waveguide assembly 178 and the image injection devices (200, 202, 204, 206, 208). In some embodiments, the controller 210 includes programming (e.g., instructions in a non-transitory computer- readable medium) that regulates the timing and provision of light corresponding to image data to the waveguides (182, 184, 186, 188, 190) according to any of the various schemes disclosed herein. In some embodiments, the controller 210 may be a single integral device, or a distributed system connected by wired or wireless communication channels. The controller 210 may be part of a processing module (e.g., the local processing and data module 70 and / or the remote processing module 72 of Figure 2) in some embodiments.

[0081] The waveguides (182, 184, 186, 188, 190) may be configured to propagate light within each respective waveguide by total internal reflection (TIR) . The waveguides (182, 184, 186, 188, 190) may each be planar or curved, with major top and bottom surfaces and edges extending between those major top and bottom surfaces. In the illustrated configuration, the waveguides (182, 184, 186, 188, 190) may each include light redirecting elements (282, 284, 286, 288, 290) that are configured to redirect light, propagating within each respective waveguide, out of the waveguide to output light corresponding to image data to the eye 4. A beam of light is outputted by the waveguide at locations at which the light propagating in the waveguide strikes a light redirecting element. The light redirecting elements (282, 284, 286, 288, 290) may be reflective and / ordiffractive optical features. While illustrated disposed at the bottom major surfaces of the waveguides (182, 184, 186, 188, 190) for ease of description and drawing clarity, in some embodiments, the light redirecting elements (282, 284, 286, 288, 290) may be disposed at the top and / or bottom major surfaces, and / or may be disposed directly in the volume of the waveguides (182, 184, 186, 188, 190). In some embodiments, the light redirecting elements (282, 284, 286, 288, 290) may be formed in a layer of material that is attached to a transparent substrate to form the waveguides (182, 184, 186, 188, 190). In some other embodiments, the waveguides (182, 184, 186, 188, 190) may be a monolithic piece of material and the light redirecting elements (282, 284, 286, 288, 290) may be formed on a surface and / or in the interior of that piece of material.

[0082] As discussed herein, each waveguide (182, 184, 186, 188, 190) is configured to output light to form an image corresponding to a particular depth plane. For example, the waveguide 182 nearest the eye 4 may be configured to deliver collimated light, as injected into such waveguide 182, to the eye 4. The collimated light may be representative of the optical infinity focal plane. The next waveguide up 184 may be configured to send out collimated light which passes through the first lens 192 (e.g., a negative lens) before it can reach the eye 4. The first lens 192 may be configured to create a slight convex wavefront curvature so that the eye / brain interprets light coming from that next waveguide up 184 as coming from a first focal plane closer inward toward the eye 4 from optical infinity. Similarly, the third waveguide up 186 passes its output light through both the first lens 192 and second lens 194 before reaching the eye 4; the combined optical power of the first lens 192 and second lens 194 may be configured to create another incremental amount of wavefront curvature so that the eye / brain interpretslight coming from the third waveguide 186 as coming from a second focal plane that is even closer inward toward the user from optical infinity than was light from the next waveguide up 184.

[0083] The other waveguides (188, 190) and lenses (196, 198) are similarly configured, with the highest waveguide 190 in the stack sending its output through all of the lenses between it and the eye for an aggregate focal power representative of the closest focal plane to the user. To compensate for the stack of lenses (198, 196, 194, 192) when viewing / interpreting light coming from world 144 on the other side of the stacked waveguide assembly 178, a compensating lens layer 180 may be disposed at the top of the stacked waveguide assembly 178 to compensate for the aggregate power of the lens stack (198, 196, 194, 192) below. Such a configuration provides as many perceived focal planes as there are available waveguide / lens pairings. Both the light redirecting elements of the waveguides and the focusing aspects of the lenses may be static (i.e., not dynamic or electro-active). In some alternative embodiments, they may be dynamic using electro-active features.

[0084] The light redirecting elements (282, 284, 286, 288, 290) may be configured to both redirect light out of their respective waveguides and to output this light with the appropriate amount of divergence or collimation for a particular depth plane associated with the waveguide. As a result, waveguides having different associated depth planes may have different configurations of light redirecting elements (282, 284, 286, 288, 290), which output light with a different amount of divergence depending on the associated depth plane. In some embodiments, as discussed herein, the light redirecting elements(282, 284, 286, 288, 290) may be volumetric or surface features, which may be configuredto output light at specific angles. For example, the light redirecting elements (282, 284, 286, 288, 290) may be volume holograms, surface holograms, and / or diffraction gratings. Light redirecting elements, such as diffraction gratings, are described in U.S. Patent Application No. 14 / 641 ,376, filed March 7, 2015, which is incorporated by reference herein in its entirety. In some embodiments, the features (198, 196, 194, 192) may not be lenses; rather, they may simply be spacers (e.g., cladding layers and / or structures for forming air gaps).

[0085] In some embodiments, the light redirecting elements (282, 284, 286, 288, 290) are diffractive features that form a diffraction pattern, or “diffractive optical element” (also referred to herein as a “DOE”). Preferably, the DOE’s have a relatively low diffraction efficiency so that only a portion of the light of the beam is deflected away toward the eye 4 with each intersection of the DOE, while the rest continues to move through a waveguide via total internal reflection. The light carrying the image data is thus divided into a number of related exit beams that exit the waveguide at a multiplicity of locations and the result is a fairly uniform pattern of exit emission toward the eye 4 for this particular collimated beam reflecting around within a waveguide.

[0086] In some embodiments, one or more DOEs may be switchable between “on” states in which they actively diffract, and “off” states in which they do not significantly diffract. For instance, a switchable DOE may include a layer of polymer dispersed liquid crystal, in which microdroplets include a diffraction pattern in a host medium, and the refractive index of the microdroplets can be switched to substantially match the refractive index of the host material (in which case the pattern does not appreciably diffract incidentlight) or the microdroplet can be switched to an index that does not match that of the host medium (in which case the pattern actively diffracts incident light).

[0087] Figure 7 shows an example of exit beams outputted by a waveguide. One waveguide is illustrated, but it will be appreciated that other waveguides in the stacked waveguide assembly 178 may function similarly. Light 400 is injected into the waveguide 182 at the input edge 382 of the waveguide 182 and propagates within the waveguide 182 by TIR. At points where the light 400 impinges on the DOE 282, a portion of the light exits the waveguide as exit beams 402. The exit beams 402 are illustrated as substantially parallel but, as discussed herein, they may also be redirected to propagate to the eye 4 at an angle (e.g., forming divergent exit beams), depending on the depth plane associated with the waveguide 182. It will be appreciated that substantially parallel exit beams may be indicative of a waveguide that corresponds to a depth plane at a large simulated distance (e.g., optical infinity) from the eye 4. Other waveguides may output an exit beam pattern that is more divergent, which would require the eye 4 to accommodate to focus on a closer simulated distance and would be interpreted by the brain as light from a distance closer to the eye 4 than optical infinity.

[0088] Figure 8 schematically illustrates an example design of a stacked waveguide assembly (e.g., the stacked waveguide assembly 178) in which each depth plane has three associated waveguides that each output light of a different color. A full color image may be formed at each depth plane by overlaying images in each of multiple component colors (e.g., three or more component colors). In some embodiments, the component colors include red, green, and blue. In some other embodiments, other colors, including magenta, yellow, and cyan, may be used in conjunction with or may replace one of red,green, or blue. Each waveguide may be configured to output a particular component color and, consequently, each depth plane may have multiple waveguides associated with it. Each depth plane may have three waveguides associated with it: a first for outputting red light, a second for outputting green light, and a third for outputting blue light.

[0089] Depth planes 14a-14f are shown in Figure 8. In the illustrated embodiment, each depth plane has three component color images associated with it: a first image of a first color, G; a second image of a second color, R; and a third image of a third color, B. The numbers following each of these letters indicate diopters (1 / m), or the reciprocal of the apparent distance of the depth plane from a viewer, and each box in the figures represents an individual component color image. In some embodiments, G is the color green, R is the color red, and B is the color blue. As discussed above, the perceived distance of the depth plane from the user may be established by the light redirecting elements (282, 284, 286, 288, 290) (e.g., diffractive optical element (DOE), and / or by lenses (198, 196, 194, 192)) which cause the light to diverge at an angle associated with the apparent distance.

[0090] In some arrangements, each component color image may be outputted by a different waveguide in a stack of waveguides. For example, each depth plane may have three component color images associated with it: a first waveguide to output a first color, G; a second waveguide to output a second color, R; and a third waveguide to output a third color, B. In arrangements in which waveguides are used to output component color images, each box in Figure 8 may be understood to represent an individual waveguide.

[0091] While the waveguides associated with each depth plane are shown adjacent to one another in this schematic drawing for ease of description, it will be appreciated that, in a physical device, the waveguides may all be arranged in a stack with one waveguide per level. Different depth planes are indicated in the figure by different numbers for diopters following the letters G, R, and B.Definitions and Use of Figures

[0092] Some of the terms used in this description are defined below for reference. The presented terms and their respective definitions are not rigidly restricted to these definitions — a term may be further defined by the term’s use within this disclosure. The term “exemplary” is used herein to mean serving as an example, instance, or illustration. Any aspect or design described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other aspects or designs. Rather, use of the word exemplary is intended to present concepts in a concrete fashion. As used in this application and the appended claims, the term “or” is intended to mean an inclusive “or” rather than an exclusive “or”. That is, unless specified otherwise, or is clear from the context, “X employs A or B” is intended to mean any of the natural inclusive permutations. That is, if X employs A, X employs B, or X employs both A and B, then “X employs A or B” is satisfied under any of the foregoing instances. As used herein, at least one of A or B means at least one of A, or at least one of B, or at least one of both A and B. In other words, this phrase is disjunctive. The articles “a” and “an” as used in this application and the appended claims should generally be construed to mean “one or more” unless specified otherwise or is clear from the context to be directed to a singular form.

[0093] Various embodiments are described herein with reference to the figures. It should be noted that the figures are not necessarily drawn to scale and that elements of similar structures or functions are sometimes represented by like reference characters throughout the figures. It should also be noted that the figures are only intended to facilitate the description of the disclosed embodiments — they are not representative of an exhaustive treatment of all possible embodiments, and they are not intended to impute any limitation as to the scope of the claims. In addition, an illustrated embodiment need not portray all aspects or advantages of usage in any particular environment.

[0094] An aspect or an advantage described in conjunction with a particular embodiment is not necessarily limited to that embodiment and can be practiced in any other embodiments even if not so illustrated. References throughout this specification to “some embodiments” or “other embodiments” refer to a particular feature, structure, material or characteristic described in connection with the embodiments as being included in at least one embodiment. Thus, the appearance of the phrases “in some embodiments” or “in other embodiments” in various places throughout this specification are not necessarily referring to the same embodiment or embodiments. The disclosed embodiments are not intended to be limiting of the claims.Non-Uniform ity CorrectionExemplary Segmented illumination Display Systems

[0095] Figure 9 depicts a system (e.g., an AR display system) 900 for presenting images to the user's eye 902 and for viewing the world 904 according to some embodiments. The system 900 includes a segmented illumination light source 910, a spatial light modulator (SLM) 940, and a waveguide 920 arranged such that light from thelight source 910 illuminates the SLM 940, and light reflected from the SLM 940 is coupled into the waveguide 920 to be directed to the eye 902. The system 900 includes optics 930 disposed to both illuminate at least a portion of the SLM 940 and project an image of the SLM 940. Light from the segmented illumination light source 910, for example, propagates in a first direction through the optics 930 onto at least a portion of the SLM 940 thereby illuminating at least a portion of the SLM 940. Light reflected from the SLM 940 propagates again through the optics 930 in a second direction opposite the first direction and is directed to the waveguide 920 and coupled therein.

[0096] The segmented illumination light source 910 may include an array of independently addressable / operable LEDs or other types of light sources in an array of independently addressable / operable array. In some embodiments, such as the one depicted in Figure 9, the segmented illumination light source 910 is a 10 x 10 array of LEDs. In other embodiments, the segmented illumination light source 910 can be other arrays having other numbers of LEDs in each direction (e.g., any number greater than 0). While the depicted segmented illumination light source 910 is a square shaped array, other shapes are also within the scope of the disclosure. In various embodiments, subgroups of the LEDs in the array forming the segmented illumination light source 910 are independently addressable / operable. The LEDs may be operable in on / off states or operable at various brightness levels from 0% (off) to 100%. The LEDs may be configured to emit light of one color. While the system 900 depicted in Figure 9 includes a single segmented illumination light source 910, systems according to other embodiments include multiple segmented illumination light sources. In some embodiments, the system includes three segmented illumination light sources, one for each of three colors (e.g.,red, green, and blue; purple, cyan, and yellow; or any other colors). In some embodiments, the system includes two segmented illumination light sources, one for a first color or combination of colors (e.g., red) and one for a second color or combination of colors (e.g., green and blue).

[0097] In some embodiments, the multiple segmented illumination light sources are executed on a single panel with separate or partially overlapping portions of the single panel functioning as each segmented illumination light source. In some embodiments, the multiple segmented illumination light sources are executed on a plurality (e.g., two or three) of panels. Figure 18 depicts a segmented illumination light source 1810 executed on three panels according to some embodiments. The segmented illumination light source 1810 includes a red panel 1812, a green panel 1814, and a blue panel 1816. Figure 19 depicts a segmented illumination light source 1910 executed on three panels according to some embodiments. The segmented illumination light source 1910 includes a first red / green / blue (“RGB”) panel 1912, a second RGB panel 1914, and a third RGB panel 1916. Figure 20 depicts a segmented illumination light source 2010 executed on three panels according to some embodiments. The segmented illumination light source 2010 includes a first yellow / magenta / cyan (“YMC”) panel 2012, a second YMC panel 2014, and a third YMC panel 2016. While the RBG and YMC segmented illumination light sources 1810, 1910, 2010 each have three exemplary colors, other embodiments of segmented illumination light sources may have fewer or more than three colors. In other embodiments, the colors of the segmented illumination light sources may be different from RGB and YMC.

[0098] The segmented illumination light source 910 may be a polarized light source, however the segmented illumination light source 910 need not be so limited. In some implementations, a polarizer 915 may be positioned between the segmented illumination light source 910 and the SLM 940. As illustrated, the polarizer 915 is between the segmented illumination light source 910 and the waveguide 920. This polarizer 915 may also be a light recycler, transmitting light of a first polarization and reflecting light of a second polarization back to the segmented illumination light source 910. Such a polarizer 915 may be, for example, a wire grid polarizer. A coupling optic 905, such as a nonimaging optical element (e.g., cone, compound parabolic collector (CPC) or lenses, may be disposed with respect to the segmented illumination light source 910 to receive light output from the segmented illumination light source 910. The coupling optic 905 may collect the light from the segmented illumination light source 910 and may, in some cases, reduce the divergence of light emitted from the segmented illumination light source 910. The coupling optic 905 may, for example, collimate the light output from the segmented illumination light source 910. The coupling optic 905 may collect light that matches the angular spectrum field of view of the system 900. Accordingly, the coupling optic 905 may match an angular spectrum of the light output by the segmented illumination light source 910 with the field of view of the system 900. The coupling optic 905 may have an asymmetric profile to operate on the light emitted from the segmented illumination light source 910 asymmetrically. For example, the coupling optic 905 may reduce the divergence a different amount in orthogonal directions (e.g., x and z directions). Such asymmetry in the coupling optic 905 may address asymmetry in the light emitted from the segmented illumination light source 910 which may include, for example, LEDs that emita wider range of angles of light in one direction (e.g., x or z) as opposed to the orthogonal direction (e.g., z or x, respectively).

[0099] As discussed above, the system 900 includes optics 930 configured to illuminate the SLM 940 that is disposed in an optical path between the segmented illumination light source 910 and the SLM 940. The optics 930 may include transmissive optics that transmits light from the segmented illumination light source 910 to the SLM 940. The optics 930 may also be configured to project an image of the SLM 940 or formed by the SLM 940 into the waveguide 920. An image may be projected into the eye 902 of the user. In some designs, the optics 930 may include one or more lenses or optical elements having optic power. The optic 930 may, for example, have positive optical power. The optics 930 may include one or more refractive optical elements such as refractive lenses. Other types of optical elements may also possibly be used.

[0100] The SLM 940 may be reflective, modulating and reflecting light therefrom. The SLM 940 may be a polarization based SLM configured to modulate polarization. The SLM 940 may, for example, include a liquid crystal (LC) SLM (e.g., a liquid crystal on silicon (LCOS) SLM). The LCOS SLM may, for example, include twisted nematic (TN) liquid crystal. The SLM 940 may, for example, include one or more pixels that are configured to selectively modulate light incident on the pixel depending on the state of the pixel. For some types of SLMs 940, the pixel may, for example, modulate the beam incident thereon by altering the polarization state such as rotating the polarization (e.g., rotating the orientation of linearly polarized light).

[0101] As discussed above, the SLM 940 may be a LCOS SLM 940. In a crosspolarizer configuration, the LCOS SLM 940 may be nominally white. When a pixel is off(e.g., 0 voltage), it has a bright state, and when the pixel is on (e g., voltage above a threshold turn on voltage), it has a dark state. In this cross-polarization configuration, leakage is minimized when a pixel is on and it has a dark state.

[0102] In a parallel-polarizer configuration, the LCOS SLM 940 is nominally black. When a pixel is off (e.g., 0 voltage), it has a dark state, and when the pixel is on (e.g., voltage above a threshold turn on voltage), it has a bright state. In this parallel-polarizer configuration, leakage is minimized when a pixel is off and it has a dark state. The dark state may be (re)optimized using rub direction and compensator angle. Compensator angle may refer to an angle of a compensator (not shown) which may be between the optics 930 and the SLM 940.

[0103] Dynamic range and throughput for parallel-polarizer configurations may be different than that of cross-polarizer configurations. Further, parallel-polarizer configurations may be optimized for contrast differently than cross-polarizer configurations.

[0104] The system 900 includes the waveguide 920 for outputting image information to the eye 902. The waveguide 920 may include substantially transparent material having a refractive index sufficient to guide light therein. As illustrated, the waveguide 920 may include a first side 921 and a second side 923 opposite the first side 921 and corresponding upper and lower major surfaces as well as edges therearound. The first and second major 921 , 923 surface may be sufficiently flat such that image information may be retained upon propagating light from the SLM 940 to the eye 902 such than an image formed by the SLM 940 may be injected into the eye 902. The optics 930 and theSLM 940 may be positioned on the first side 921 of the waveguide 920. The segmentedillumination light source 910 may be disposed on the second side 923 such that light from the segmented illumination light source 910 is incident on the second side 923 prior to passing through the waveguide 920 and through the optics 930 to the SLM 940. Accordingly, the waveguide 920 may be disposed between the segmented illumination light source 910 and the optics 930. Additionally, at least a portion of the waveguide 920 may extend between the segmented illumination light source 910 and the optics 930, whereby light passes through the portion of the waveguide 920 to the optics 930. Light emitted from the segmented illumination light source 910 can therefore be directed through the waveguide 920, into and through the optics 930 and incident on the SLM 940. The SLM 940 reflects the light back through the optics 930 and to the waveguide 920.

[0105] The system 900 also includes an in-coupling optical element 960 for coupling light from the optics 930 into the waveguide 920. The in-coupling optical element 960 may be disposed on a major surface (e.g., an upper major surface 923) of the waveguide 920. In some designs, the in-coupling optical element 960 may be disposed on the lower major surface 921 of the waveguide 920. While illustrated on one side or corner of the waveguide 920, the in-coupling optical element 960 may be disposed in / on other areas of the waveguide 920. For instance, the in-coupling optical element 960 may be disposed in the body of the waveguide 920. The in-coupling optical element 960 may be a diffractive optical element or a reflector. Other structures may be used as the in-coupling optical element 960. The in-coupling optical element 960 may be configured to direct the light incident thereon into the waveguide 920 at a sufficiently large grazing angle (e.g., greater than the critical angle) with respect to the upper and lower major surfaces 923,921 of the waveguide 920 to be guided therein by total internal reflection. Further, the in-coupling optical element 960 may operate on a wide range of wavelengths and thus be configured to couple light of multiple colors into the waveguide 920. For instance, the incoupling optical element 960 may be configured to couple red light, green light, and blue light into the waveguide 920, and the segmented illumination light source 910 may emit red, green, and blue color light at different times.

[0106] The system 900 includes a light distributing element 970 disposed on or in the waveguide 920. In some embodiments, the light distributing element 970 may be an orthogonal pupil expander (OPE). The light distributing element 970 may be configured to spread the light within the waveguide 920 by turning the light propagating in an x direction, for example, toward a z direction. The light distributing element 970 may, thus, be configured to increase dimensions of the eyebox along the z-axis. The light distributing element 970 may, for example, include one or more diffractive optical elements configured to diffract the light propagating within the waveguide 920 incident the diffractive optical elements so as to redirect that light, for example, in a generally orthogonal direction.

[0107] Other configurations are possible. For instance, embodiments with multiple segmented illumination light sources may have multiple waveguides corresponding to each of the segmented illumination light sources. In such embodiments, each of the multiple waveguides may have in-coupling gratings that do not overlap with the incoupling gratings for the other waveguides in order to minimize unintended in-coupling into the other waveguides.

[0108] Figure 10 depicts a system (e.g., an AR display system) 1000 for presenting images to a user’s eye and for viewing the world according to some embodiments. Thesystem 1000 includes a light source 1010, a plurality (e.g., three) of waveguides 1020, an SLM 1040, and optics 1030 disposed to both illuminate at least a portion of the SLM 1040 and project an image of the SLM 1040 to the user’s eye. The system 1000 also includes a coupling optic 1005, such as a non-imaging optical element (e.g., cone, compound parabolic collector (CPC, lenses)), disposed with respect to the light source 1010 to receive light output from the light source 1010. The structure and function of the various optical components of the system 1000 depicted in Figure 10 are similar to those of the corresponding optical components of the system 900 depicted in Figure 9 and described above.

[0109] In embodiments where the light source 1010 consists of three LEDs (e.g., one for each of red, green, and blue colors) forming a Lambertian light source, the coupling optics 1005 (e.g., CPC) adapt the angular distribution of light generated by the light source 101 O to fit system needs. However, the coupling optics 1005 can form a fixed illumination pattern at the SLM 1040 such as the fixed illumination pattern 1200 shown in Figure 12. While the light source 1010 can be controlled to change the light level at the SLM 1040, the fixed illumination pattern 1200 remains at the SLM 1040 regardless of the light level at the SLM 1040. Further, non-uniform ities in the various optical components result in a non-uniform illumination pattern 1300 at the SLM 1040 such as that depicted in Figure 13A. The fixed illumination pattern 1200 and the non-uniform illumination pattern 1300 at the SLM 1040 introduce non-uniform ities, distortions, artifacts, and / or aberrations in the image displayed to the user.

[0110] In other embodiments, the light source comprises one or more LED arrays forming a Lambertian light source. Figure 11 depicts such an embodiment of a system(e.g., an AR display system) 1100 for presenting images to a user’s eye and for viewing the world according to some embodiments. The system 1100 includes a segmented illumination light source 1110, a waveguide 1120, an SLM 1140, and optics 1130 disposed to both illuminate at least a portion of the SLM 1140 and project an image generated using the SLM 1140, through the waveguide 1120, and to the user’s eye. The system 1100 also includes a coupling optic 1105. The structure and function of the various optical components of the system 1100 depicted in Figure 11 are similar to those of the corresponding optical components of the system 900 depicted in Figure 9 and described above.

[0111] The segmented illumination light source 1110 includes an array of independently addressable / operable LEDs. By controlling the voltage / current applied to each LED / sub-light source, the amount of light delivered to corresponding portions 1142 of the SLM 1140 and the waveguide 1120 can be controlled. The amount of light delivered to various portions 1142 of the SLM 1140 and onto the waveguide 1120 can be controlled to minimize / eliminate non-uniform ities, distortions, artifacts, and / or aberrations in the image displayed to the user.Non-Uniform ity Correction

[0112] Figures 13A-13C depict brightness uniformity correction by segmented illumination according to some embodiments. Figure 13A depicts a theoretical non- uniform illumination pattern 1300 at an SLM 1140 and a waveguide 1120 (see Figure 11 ) without non-uniform ity correction. The non-uniform illumination pattern 1300 is non- uniform in terms of brightness, with areas of unintended low, medium, and high illumination 1302, 1304, 1306 thereby introducing brightness artifacts in the imagepresented to the user. Figure 13B depicts a segmented illumination pattern 1310 generated by the segmented illumination light source 1110 overlaid on top of and corresponding to the theoretical non-uniform illumination pattern 1300 at the SLM 1140 and the waveguide 1120. The segmented illumination pattern 1310 includes areas of high illumination 1312, areas of medium illumination 1314, and areas of low illumination 1316, which are also marked in Figure 13B. The areas of high, medium, and low illumination 1312, 1314, 1316 in the segmented illumination pattern 1310 that correspond to respective areas of low, medium, and high brightness in the theoretical non-uniform illumination pattern 1300 at the SLM 1140 and the waveguide 1120. Accordingly, illuminating the SLM 1140 and the waveguide 1120 with the segmented illumination pattern 1310 results in the corrected illumination pattern 1320 depicted in Figure 13C. In the ideal case depicted in Figure 13C, the corrected illumination pattern 1320 the brightness artifacts in theoretical non-uniform illumination pattern 1300 have been eliminated and the brightness of the light is consistent throughout the surface of the SLM 1140 and the waveguide 1120.

[0113] Figures 14A-14B depict color uniformity correction by segmented illumination according to some embodiments. Figure 14A depicts a theoretical non-uniform illumination pattern 1400 at an SLM 1140 and a waveguide 1120 (see Figure 11 ) without optical color uniformity and brightness uniformity correction. The non-uniform illumination pattern 1400 is non-uniform in terms of color, with areas of unintended red, green, and blue color illumination 1402, 1404, 1406, thereby introducing color artifacts in the image presented to the user. The non-uniform illumination pattern 1400 is also non-uniform in terms of brightness. A segmented illumination light source 1110 with color controlcapabilities can generate a segmented illumination pattern (not shown) corresponding to the non-uniform illumination pattern 1400 at the SLM 1140 and the waveguide 1120. The segmented illumination pattern would include areas of differential color illumination with complementary colors that correspond to respective areas of red, green, and blue color 1402, 1404, 1406 illumination in the theoretical non-uniform illumination pattern 1400 at the SLM 1140 and the waveguide 1120. Accordingly, illuminating the SLM 1140 and the waveguide 1120 with a segmented illumination pattern (not shown) results in the corrected illumination pattern (example) 1420 depicted in Figure 14B. In the corrected illumination pattern 1420 depicted in Figure 14B, the color artifacts in theoretical none uniform illumination pattern 1400 have been eliminated and the color of the light is consistent throughout the surface of the SLM 1140 and the waveguide 1120. While the corrected illumination pattern 1420 depicted in Figure 14B includes brightness artifacts, those brightness artifacts can be corrected by applying a segmented illumination pattern 1310 such as the one shown in Figure 13B Reduce Power Consumption

[0114] Figure 15 depicts a segmented illumination pattern 1500 configured to reduce power consumption of a display system (e.g. system 1100 depicted in Figure 11 ) according to some embodiments. The segmented illumination pattern 1500 is overlaid on top of an image 1510 to be displayed with light from the segmented illumination pattern 1500. The segmented illumination pattern 1500 only illuminates areas of the SLM 1140 and the waveguide 1120 configured to generate portions of the image 1510. Accordingly, the segmented illumination pattern 1500 includes illuminated “on” areas 1502 and nonilluminated “off” areas 1504. The illuminated “on” areas 1502 and non-illuminated “off”areas 1504 are generated by respective illuminated “on” areas / portions and nonilluminated “off” areas / portions of a segmented illumination light source 1110. The inclusion non-illuminated “off” areas in the segmented illumination pattern 1500 reduces power consumption for the display system 1100. Areas of the segmented illumination pattern can be generated by controlling the amount of voltage / current corresponding portions of the segmented illumination light source 1110. In some embodiments, instead of binary illuminated / non-illuminated areas, the segmented illumination pattern can include analog grayscale illuminated areas with variable amounts of illumination depending on the image to be displayed.

[0115] If only a small portion (e.g., 10%) of the segmented illumination light source 1110 is illuminated, that portion may be overdriven to emit additional light compared to when a larger portion of the segmented illumination light source 1110 is illuminated. In addition, the illuminated portions of the segmented illumination source 1110 may be monitored for over-temperature issues while being overdriven.Improved ANSI Contrast

[0116] Figures 16A-16B depict contrast improvement by segmented illumination according to some embodiments. Figure 16A depicts an image 1600 presented by a system 1000 (see Figure 10) without segmented illumination. The image 1600 includes white areas 1602 and black areas 1604. Figure 16B depicts an image 1610 presented by a system 1100 (see Figure 11 ) with segmented illumination. In such a system 1100, a segmented illumination pattern is generated by illuminated “on” areas / portions and nonilluminated “off” areas / portions of a segmented illumination light source 1110 corresponding to respective white areas 1612 and black areas 1614 of the image 1610.The inclusion non-illuminated “off” areas in the segmented illumination pattern improves contrast of the image 1610. Areas of the segmented illumination pattern can be generated by controlling the amount of voltage / current corresponding portions of the segmented illumination light source 1110. In some embodiments, instead of binary illuminated / non-illuminated areas, the segmented illumination pattern can include analog grayscale illuminated areas with variable amounts of illumination depending on the image to be displayed.Method For Displaying An Image With Segmented illumination

[0117] Figure 17 depicts a method 1710 for displaying an image with segmented illumination according to some embodiments. At 1712, non-uniform ity correction information is optionally obtained (e.g., by a processor of a display controller). The nonuniformity correction information may be obtained by calibrating / testing an image display system with known image data. The non-uniform ity correction information includes, but is not limited to, information relating to illumination non-uniform ity. At 1714, the nonuniformity correction information is stored in a memory of a display controller. The nonuniformity correction information may be stored in the memory using a lookup table. At 1716, the processor controls a segmented illumination light source using the nonuniformity correction information to provide differentially illuminated light. The amount of voltage / current each LED of the segmented illumination light source may be calculated via an algorithm using the non-uniform ity correction information. At 1718, the differentially illuminated light from the segmented illumination light source differentially illuminates first and second portions of an SLM.

[0118] At 1720, the processor optionally obtains new non-uniform ity correction information. The new non-uniform ity correction information may result from changes in components of the image display system over time during use. Temperature, aging, and other environmental factors may cause changes to various components of the display system resulting in new non-uniform ity correction information. At 1722, the new nonuniformity correction information is optionally stored in the memory. At 1724, the method goes to 1716 to control the segmented illumination light source using the new nonuniformity correction information.

[0119] In some embodiments, the processor may optionally warp the image rendered by compute to account for user motion and latency. This warp can be applied to the SLM and segmented illumination. In some embodiments, the processor may optionally synchronize the segmented illumination light source with the SLM and control both together on a field by field basis. Synchronized control of the SLM gain and segmented illumination gain can achieve the desired light output.Exemplary ASIC Drivers With Oversized Bond Pad Arrays

[0120] Segmented illumination light sources may include one or more applicationspecific integrated circuits (ASIC) each having a bond pad array configured for an LED array to be coupled thereto and for controlling the LED array. For example, Figure 21 A depicts a segmented illumination light source 2110 including two ASICs 2120, 2130 according to some embodiments. The ASICs 2120, 2130 include respective bond pad arrays 2122, 2132, and LED arrays 2124, 2134 coupled to the respective bond pad arrays 2122, 2132. Figure 21 A shows that segmented illumination light sources 2110 having a plurality of ASICs 2120, 2130 have a minimum size in order to fit the plurality of ASICs2120, 2130, including a space required between the plurality of ASICs 2120, 2130. This minimum size constraint increases the minimum size of the corresponding display system.

[0121] Figure 21 B depicts a segmented illumination light source 2150 including a single ASIC 2160 according to some embodiments. The ASIC 2160 includes a bond pad array having first, second, and third portions 2162A, 2162B, 2162C, and respective first, second, and third LED arrays 2164, 2174, 2184 coupled to the respective portions 2162A, 2162B, 2162C of the bond pad array. Figure 21 B shows that segmented illumination light sources 2150 having an oversized bond pad array 2162A, 2162B, 2162C can accommodate a plurality of LED arrays 2164, 2174, 2184. Coupling a plurality of LED arrays 2164, 2174, 2184 to respective portions 2162A, 2162B, 2162C of the oversized bond pad array facilitates independent control of the plurality of LED arrays 2164, 2174, 2184 by the ASIC 2160 through the oversized bond pad array. Coupling a plurality of LED arrays 2164, 2174, 2184 to a single ASIC reduces the size of the segmented illumination light source 2150 and the corresponding display system by at least eliminating the required space between ASICs.

[0122] Figure 22 depicts a segmented illumination light source 2250 including a single ASIC 2260 according to some embodiments. The ASIC 2260 includes a bond pad array having first, second, and third portions 2262A, 2262B, 2262C, and respective first, second, and third LED arrays 2264, 2274, 2284 coupled to the respective portions 2262A, 2262B, 2262C of the bond pad array. Figure 22 shows that each of the portions 2262A, 2262B, 2262C of the bond pad array includes a plurality of positions 2266, 2267, 2268 that can accommodate the respective first, second, and third LED arrays 2264, 2274,2284. ASICs including oversized bond pad with arrays portions 2262A, 2262B, 2262C having respective pluralities of positions 2266, 2267, 2268 for coupling LED arrays 2264, 2274, 2284 provide flexibility during manufacturing for alignment of LED arrays with other optical components (e.g., in-coupling gratings) in display systems. Each of the LED arrays 2264, 2274, 2284 may be configured to emit a specific color (e.g., red, green, blue).

[0123] The amount of movement of LED arrays on the bond pad arrays may be limited by pixel pitch. Pixel data for the LED arrays may be mapped to the corresponding bond pad drivers on the bond pad array. During normal use, the ASIC may disable the unused pixel drivers that are not coupled to the LED arrays. The unused pixel driver bond pads may be masked.

[0124] Figures 23 and 24 schematically depict LED arrays coupled to respective bond pad arrays according to various embodiments. The LED arrays 2314, 2324, 2334, 2414, 2424, 2434 depicted in Figures 23 and 24 are all 8 x 8 LED arrays. In other embodiments, LED arrays may be different sizes. The first, second, and third portions 2312A, 2412A, 2312B, 2412B, 2312C, 2412C of the bond pad arrays depicted in Figures 23 and 24 are identical. In fact, the underlying ASIC corresponding to the bond pad arrays 2312A, 2312B, 2312C, 2412A, 2412B, 2412C depicted in Figures 23 and 24 may be identical. Figures 23 and 24 show that oversized bond pad arrays 2312A, 2312B, 2312C, 2412A, 2412B, 2412C provide flexibility in placement of LED arrays during manufacturing.

[0125] Figure 23 shows first, second, and third LED arrays 2314, 2324, 2334 coupled to upper right, upper right, and upper left corners of the respective first, second, and third portions 2312A, 2312B, 2312C of the bond pad array. Figure 24 shows first, second, and third LED arrays 2414, 2424, 2434 coupled to middle areas of the respective first, second,and third portions 2412A, 2412B, 2412C of the bond pad array. Figures 23 and 24 demonstrate the flexibility in mounting LED arrays to oversized bond pad arrays, which facilitates use of the same ASIC for different optical devices having different optical designs. Such flexibility allows use of a single ASIC design for a plurality of segmented illumination light sources configured for use with different optical devices (see Figures 28 to 31 ).

[0126] Figures 25 to 27 schematically depict LED arrays coupled to respective bond pad arrays according to various embodiments. The LED arrays 2514, 2614, 2714 depicted are 8 x 8 (Figures 25 and 26) and 4 x 4 (Figure 27) LED arrays. In other embodiments, LED arrays may be different sizes. The bond pad arrays 2512, 2612, 2712 depicted in Figures 25 to 27 are identical 14 x 14 bond pad arrays. In fact, the underlying ASIC corresponding to the bond pad arrays 2512, 2612, 2712 depicted in Figures 23 and 24 may be identical. Figures 25 to 27 show that oversized bond pad arrays 2512, 2612, 2712 provide flexibility in placement of LED arrays during manufacturing.

[0127] Figures 28 to 30 schematically depict the use of ASICs having oversized bond pads to align a plurality of LED arrays with other optical components during manufacture. Figure 28 depicts a display system 2810, according to some embodiments. The display system 2810 includes an optical device 2910 coupled to a controller (not shown) with a connector 2850. As shown in Figure 29 in greater detail, the optical device 2910 includes a frame 2912 defining first, second, and third light source openings 2914A, 2914B, 2914C therein. The respective positions of the first, second, and third light source openings 2914A, 2914B, 2914C are configured to direct light to other optical components (not shown) of the optical device 2910. In the embodiment depicted in Figures 28 and 29, thefirst, second, and third LED arrays 2924, 2934, 2944 are coupled to first, second, and third portions 2922A, 2922B, 2922C of the oversized bond pad array of the ASIC. However, the first, second, and third LED arrays 2924, 2934, 2944 are coupled to first, second, and third portions 2922A, 2922B, 2922C such that the first, second, and third LED arrays 2924, 2934, 2944 are not aligned with respective defining first, second, and third light source openings 2914A, 2914B, 2914C.

[0128] Figure 30 schematically depicts an optical device 3010 includes a frame 2912 defining first, second, and third light source openings 2914A, 2914B, 2914C therein. The frame 2912 and the first, second, and third light source openings 2914A, 2914B, 2914C defined therein are identical to the corresponding components in the optical device 2910 depicted in Figure 29. In the embodiment depicted in Figure 30, the first, second, and third LED arrays 2924’, 2934’, 2944’ are coupled to first, second, and third portions of the oversized bond pad array of the ASIC such that the first, second, and third LED arrays 2924’, 2934’, 2944’ are aligned with respective defining first, second, and third light source openings 2914A, 2914B, 2914C. The optical device 3010 has LED arrays 2924’, 2934’, 2944’ aligned with other optical components of the optical device 3010, thereby improving optical efficiency and power consumption / heat generation compared to the optical device 2910 depicted in Figure 29. The optical device 2910 depicted in Figure 29 can be modified by coupling LED arrays to different portions of the oversized bond pad array in identical ASICs during manufacturing to arrive at the more optically efficient optical device 3010 depicted in Figure 30.

[0129] Figure 31 depicts a method 3110 for manufacturing different optical devices (e.g., 2910, 3010) having different optical designs using identical ASICs, according to some embodiments.

[0130] At 3112, a first LED array is coupled to a first portion of a first bond pad of a first ASIC to form a first light source. The first ASIC may have an oversized first bond pad array including a plurality of portions to which LED arrays may be coupled (e.g., Figure 22).

[0131] At 3114, a second LED array is coupled to a second portion of a second bond pad of a second ASIC to form a second light source. The second ASIC may have an oversized second bond pad array including a plurality of portions to which LED arrays may be coupled (e.g., Figure 22). The first and second ASICs may be identical and have identical oversized first and second bond pad arrays. Using identical ASICs streamlines the supply chain during manufacturing of segmented illumination light sources for optical devices. The first portion and the second portion may be different from each other to facilitate flexibility in LED array placement during manufacturing while using identical ASICs (e.g., Figures 29 and 30).

[0132] At 3116, the first light source is optionally used with a first optical device. At 3118, the second light source is optionally used with a second optical device. The first and second optical devices have different optical designs (e.g., due to different optical components and / or optical functions). As such, identical ASICs are used to manufacture different light sources for different optical devices (e.g., 2910, 3010) having different optical designs.System Architecture Overview

[0133] Figure 32 is a block diagram of an illustrative computing system 3200 suitable for implementing an embodiment of the present disclosure. Computer system 3200 includes a bus 3206 or other communication mechanism for communicating information, which interconnects subsystems and devices, such as processor 3207, system memory 3208 (e.g., RAM), static storage device 3209 (e.g., ROM), disk drive 3210 (e.g., magnetic or optical), communication interface 3214 (e.g., modem or Ethernet card), display 3211 (e.g., CRT or LCD), input device 3212 (e.g., keyboard), and cursor control.

[0134] According to one embodiment of the disclosure, computer system 3200 performs specific operations by processor 3207 executing one or more sequences of one or more instructions contained in system memory 3208. Such instructions may be read into system memory 3208 from another computer readable / usable medium, such as static storage device 3209 or disk drive 3210. In alternative embodiments, hard-wired circuitry may be used in place of or in combination with software instructions to implement the disclosure. Thus, embodiments of the disclosure are not limited to any specific combination of hardware circuitry and / or software. In one embodiment, the term “logic” shall mean any combination of software or hardware that is used to implement all or part of the disclosure.

[0135] The term “computer readable medium” or “computer usable medium” as used herein refers to any medium that participates in providing instructions to processor 3207 for execution. Such a medium may take many forms, including but not limited to, nonvolatile media and volatile media. Non-volatile media includes, for example, optical ormagnetic disks, such as disk drive 3210. Volatile media includes dynamic memory, such as system memory 3208.

[0136] Common forms of computer readable media includes, for example, floppy disk, flexible disk, hard disk, magnetic tape, any other magnetic medium, CD-ROM, any other optical medium, punch cards, paper tape, any other physical medium with patterns of holes, RAM, PROM, EPROM, FLASH-EPROM (e.g., NAND flash, NOR flash), any other memory chip or cartridge, or any other medium from which a computer can read.

[0137] In an embodiment of the disclosure, execution of the sequences of instructions to practice the disclosure is performed by a single computer system 3200. According to other embodiments of the disclosure, two or more computer systems 3200 coupled by communication link 3215 (e.g., LAN, PTSN, or wireless network) may perform the sequence of instructions required to practice the disclosure in coordination with one another.

[0138] Computer system 3200 may transmit and receive messages, data, and instructions, including program, i.e., application code, through communication link 3215 and communication interface 3214. Received program code may be executed by processor 3207 as it is received, and / or stored in disk drive 3210, or other non-volatile storage for later execution. Database 3232 in storage medium 3231 may be used to store data accessible by system 3200 via data interface 3233.

[0139] While the segmented illumination systems and methods are describe herein as implemented in various display systems, the segmented illumination systems and methods described herein may be implemented in various other display systems.

[0140] Certain aspects, advantages and features of the disclosure have been described herein. It is to be understood that not necessarily all such advantages may be achieved in accordance with any particular embodiment of the disclosure. Thus, the disclosure may be embodied or carried out in a manner that achieves or optimizes one advantage or group of advantages as taught herein without necessarily achieving other advantages as may be taught or suggested herein.

[0141] Embodiments have been described in connection with the accompanying drawings. However, it should be understood that the figures are not drawn to scale. Distances, angles, etc. are merely illustrative and do not necessarily bear an exact relationship to actual dimensions and layout of the devices illustrated. In addition, the foregoing embodiments have been described at a level of detail to allow one of ordinary skill in the art to make and use the devices, systems, methods, and the like described herein. A wide variety of variation is possible. Components, elements, and / or steps may be altered, added, removed, or rearranged.

[0142] The devices and methods described herein can advantageously be at least partially implemented using, for example, computer software, hardware, firmware, or any combination of software, hardware, and firmware. Software modules can include computer executable code, stored in a computer’s memory, for performing the functions described herein. In some embodiments, computer-executable code is executed by one or more general purpose computers. However, a skilled artisan will appreciate, in light of this disclosure, that any module that can be implemented using software to be executed on a general purpose computer can also be implemented using a different combination of hardware, software, or firmware. For example, such a module can be implementedcompletely in hardware using a combination of integrated circuits. Alternatively or additionally, such a module can be implemented completely or partially using specialized computers designed to perform the particular functions described herein rather than by general purpose computers. In addition, where methods are described that are, or could be, at least in part carried out by computer software, it should be understood that such methods can be provided on non-transitory computer-readable media that, when read by a computer or other processing device, cause it to carry out the method.

[0143] While certain embodiments have been explicitly described, other embodiments will become apparent to those of ordinary skill in the art based on this disclosure.

[0144] The various processors and other electronic components described herein are suitable for use with any optical system for projecting light. The various processors and other electronic components described herein are also suitable for use with any audio system for receiving voice commands.

[0145] Various exemplary embodiments of the disclosure are described herein. Reference is made to these examples in a non-limiting sense. They are provided to illustrate more broadly applicable aspects of the disclosure. Various changes may be made to the disclosure described and equivalents may be substituted without departing from the true spirit and scope of the disclosure. In addition, many modifications may be made to adapt a particular situation, material, composition of matter, process, process act(s) or step(s) to the objective(s), spirit or scope of the present disclosure. Further, as will be appreciated by those with skill in the art, each of the individual variations described and illustrated herein has discrete components and features which may be readily separated from or combined with the features of any of the other several embodimentswithout departing from the scope or spirit of the present disclosure. All such modifications are intended to be within the scope of claims associated with this disclosure.

[0146] The disclosure includes methods that may be performed using the subject devices. The methods may include the act of providing such a suitable device. Such provision may be performed by the end user. In other words, the “providing” act merely requires the end user obtain, access, approach, position, set-up, activate, power-up or otherwise act to provide the requisite device in the subject method. Methods recited herein may be carried out in any order of the recited events which is logically possible, as well as in the recited order of events.

[0147] Exemplary aspects of the disclosure, together with details regarding material selection and manufacture have been set forth above. As for other details of the present disclosure, these may be appreciated in connection with the above-referenced patents and publications as well as generally known or appreciated by those with skill in the art. The same may hold true with respect to method-based aspects of the disclosure in terms of additional acts as commonly or logically employed.

[0148] In addition, though the disclosure has been described in reference to several examples optionally incorporating various features, the disclosure is not to be limited to that which is described or indicated as contemplated with respect to each variation of the disclosure. Various changes may be made to the disclosure described and equivalents (whether recited herein or not included for the sake of some brevity) may be substituted without departing from the true spirit and scope of the disclosure. In addition, where a range of values is provided, it is understood that every intervening value, between theupper and lower limit of that range and any other stated or intervening value in that stated range, is encompassed within the disclosure.

[0149] Also, it is contemplated that any optional feature of the variations described may be set forth and claimed independently, or in combination with any one or more of the features described herein. Reference to a singular item, includes the possibility that there are plural of the same items present. More specifically, as used herein and in claims associated hereto, the singular forms “a,” “an,” “said,” and “the” include plural referents unless the specifically stated otherwise. In other words, use of the articles allow for “at least one” of the subject item in the description above as well as claims associated with this disclosure. It is further noted that such claims may be drafted to exclude any optional element. As such, this statement is intended to serve as antecedent basis for use of such exclusive terminology as “solely,” “only” and the like in connection with the recitation of claim elements, or use of a “negative” limitation.

[0150] Without the use of such exclusive terminology, the term “comprising” in claims associated with this disclosure shall allow for the inclusion of any additional element- irrespective of whether a given number of elements are enumerated in such claims, or the addition of a feature could be regarded as transforming the nature of an element set forth in such claims. Except as specifically defined herein, all technical and scientific terms used herein are to be given as broad a commonly understood meaning as possible while maintaining claim validity.

[0151] The breadth of the present disclosure is not to be limited to the examples provided and / or the subject specification, but rather only by the scope of claim language associated with this disclosure.

[0152] In the foregoing specification, the disclosure has been described with reference to specific embodiments thereof. It will, however, be evident that various modifications and changes may be made thereto without departing from the broader spirit and scope of the disclosure. For example, the above-described process flows are described with reference to a particular ordering of process actions. However, the ordering of many of the described process actions may be changed without affecting the scope or operation of the disclosure. The specification and drawings are, accordingly, to be regarded in an illustrative rather than restrictive sense.

Claims

ClaimsWhat is claimed is:1 . An extended reality display system, comprising: a display subsystem configured to present an image corresponding to image data to a user, the display subsystem comprising an optical component that introduces a non-uniform ity to the image, a segmented illumination light source, and a spatial light modulator (SLM) configured to receive light from the segmented illumination light source; and a display controller configured to control the segmented illumination light source, the display controller comprising: a memory for storing non-uniform ity correction information, and a processor to control the segmented illumination light source based on the non-uniform ity correction information, wherein the segmented illumination light source is configured to differentially illuminate first and second portions of the SLM using respective first and second portions of the segmented illumination light source.

2. The system of claim 1 , wherein the segmented illumination light source comprises a light emitting diode (LED) array, and wherein the display controller is configured to control each LED of the LED array.

3. The system of claim 1 , wherein the SLM is a liquid crystal on silicon (LCOS) display.

4. The system of claim 1 , further comprising a lens disposed between the segmented illumination light source and the SLM.

5. The system of claim 1 , wherein the segmented illumination light source is configured to improve a uniformity of illumination of the image corresponding to the image data by differentially illuminating first and second portions of the SLM.

6. The system of claim 5, wherein the uniformity is an illumination uniformity.

7. The system of claim 5, wherein the uniformity is a color uniformity.

8. The system of claim 1 , wherein the segmented illumination light source is configured to reduce power consumption by differentially illuminating first and second portions of the SLM.

9. The system of claim 1 , wherein the segmented illumination light source is configured to improve a contrast of the image corresponding to the image data by differentially illuminating first and second portions of the SLM.

10. The system of claim 1 , wherein the segmented illumination light source comprises a plurality of sub-light sources, and wherein the display controller is configured to control an amount of voltage or current applied to a sub-light source of the plurality of sub-light sources.11 . The system of claim 10, wherein the processor is configured to calculate the amount of current delivered to the sub-light source using the non-uniform ity correction information stored in the memory.

12. The system of claim 1 , wherein the display controller is directly coupled to the segmented illumination light source with only a connection element between the display controller and the segmented illumination light source.

13. The system of claim 1 , wherein the display controller is configured to instruct the segmented illumination light source to not illuminate a third portion of the SLM by directing a zero voltage or current to be applied to the third portion of the segmented illumination light source.

14. The system of claim 13, wherein the display controller is configured to instruct the segmented illumination light source to increase respective first and second amounts of light generated by the first portion and the second portion of the segmented illumination light source.

15. The system of claim 14, wherein the display controller is configured to monitor respective first and second temperatures of the first portion and the second portion of the segmented illumination light source.

16. A method for presenting an image corresponding to image data to a user using a display subsystem including an optical component that introduces a nonuniformity to the image, the method comprising: a display controller storing non-uniform ity correction information in a memory thereof; a processor of the display controller controlling a segmented illumination light source of the display subsystem based on the non-uniform ity correction information; and the segmented illumination light source differentially illuminating first and second portions of an SLM of the display subsystem using respective first and second portions of the segmented illumination light source under control of the processor.

17. The method of claim 16, wherein the segmented illumination light source comprises an LED array, the method further comprising: the processor of the display controller controlling each LED of the LED array.

18. The method of claim 16, wherein the SLM is an LCDS display.

19. The method of claim 16, wherein a lens is disposed between the segmented illumination light source and the SLM in the display subsystem.

20. The method of claim 16, wherein the segmented illumination light source differentially illuminating first and second portions of the SLM improves a uniformity of illumination of the image corresponding to the image data.21 . The method of claim 20, wherein the uniformity is an illumination uniformity.

22. The method of claim 20, wherein the uniformity is a color uniformity.

23. The method of claim 16, wherein the segmented illumination light source differentially illuminating first and second portions of the SLM reduces power consumption.

24. The method of claim 16, wherein the segmented illumination light source differentially illuminating first and second portions of the SLM improves a contrast of the image corresponding to the image data.

25. The method of claim 16, wherein the segmented illumination light source comprises a plurality of sub-light sources, the method further comprising: the processor controlling an amount of voltage or current applied to a sub-light source of the plurality of sub-light sources.

26. The method of claim 25, further comprising the processor calculating the amount of current delivered to the sub-light source using the non-uniform ity correction information stored in the memory.

27. The method of claim 16, wherein the display controller is directly coupled to the segmented illumination light source with only a connection element between the display controller and the segmented illumination light source.

28. The method of claim 16, further comprising the processor instructing the segmented illumination light source to not illuminate a third portion of the SLM by directing a zero voltage or current to be applied to the third portion of the segmented illumination light source.

29. The method of claim 28, further comprising the processor instructing the segmented illumination light source to increase respective first and second amounts of light generated by the first portion and the second portion of the segmented illumination light source.

30. The method of claim 29, further comprising the processor monitoring respective first and second temperatures of the first portion and the second portion of the segmented illumination light source.31 . The method of claim 16, further comprising:the processor obtaining new non-uniform ity correction information; the display controller storing new non-uniform ity correction information in the memory; the processor controlling the segmented illumination light source of the display subsystem based on the new non-uniform ity correction information; and the segmented illumination light source differentially illuminating the first and second portions of the SLM of the display subsystem using the respective first and second portions of the segmented illumination light source under control of the processor.

32. A segmented illumination light source, comprising: an application-specific integrated circuit (ASIC) comprising a bond pad array; a first LED array coupled to a first portion of the bond pad array; a second LED array coupled to a second portion of the bond pad array; and a third LED array coupled to a third portion of the bond pad array, wherein the first, second, and third portions are distinct from each other, and wherein the ASIC is configured to control the first LED array, the second LED array, and the third LED array.

33. The segmented illumination light source of claim 32, wherein the first LED array is configured to emit red light, wherein the second LED array is configured to emit green light, and wherein the third LED array is configured to emit blue light.

34. The segmented illumination light source of claim 32, wherein the first portion of the bond pad array comprises a plurality of portions in which the first LED array can be coupled.

35. The segmented illumination light source of claim 32, wherein the bond pad array comprises a plurality of portions in which an LED array can be coupled.

36. A method for manufacturing a plurality of segmented illumination light sources, the method comprising: coupling a first LED array to a first portion of a first bond pad array of a first application-specific integrated circuit (ASIC) to form a first segmented illumination light source; and coupling a second LED array to a second portion of a second bond pad array of a second ASIC to form a first segmented illumination light source, wherein the first ASIC is identical to the second ASIC, wherein the first portion is located at a first location on the first bond pad array, wherein the second portion is located at a second location on the second bond pad array, and wherein the first location is different from the second location.

37. The method of claim 36, wherein the first segmented illumination light source is configured for use with a first optical device having a first optical design, andwherein the second segmented illumination light source is configured for use with a second optical device having a second optical design different from the first optical design, the method further comprising: determining the first portion of the first bond pad array of the first ASIC based on the first optical design of the first optical device; and determining the second portion of the second bond pad array of the second ASIC based on the second optical design of the second optical device.

Images