Extended reality system and method

Abstract

An extended reality display system includes a display subsystem configured to present an image to the user that corresponds to image data. The display subsystem includes an optical component that introduces heterogeneity into the image, a segmented illumination source, and a spatial light modulator (SLM) configured to receive light from the segmented illumination source. The system also includes a display controller configured to control the segmented illumination source. The display controller includes a memory for storing heterogeneity correction information and a processor that controls the segmented illumination source based on the heterogeneity correction information. The segmented illumination source is configured to differentially illuminate the first and second parts of the SLM using the first and second parts of the segmented illumination source, respectively.

Description

Technical Field

[0001] Incorporation by Reference This application claims priority to U.S. Provisional Application No. 63 / 503,667, filed May 22, 2023, the contents of which are hereby expressly and fully incorporated by reference in their entirety. This application also expressly and fully incorporates by reference in their entirety each of the following patent applications: U.S. Utility Patent Application No. 14 / 555,585, filed November 27, 2014, entitled "VIRTUAL AND AUGMENTED REALITY SYSTEMS AND METHODS," under Attorney Docket No. ML.20011.00; U.S. Utility Patent Application No. 14 / 738,877, filed June 13, 2015, entitled "METHODS AND SYSTEMS FOR CREATING VIRTUAL AND AUGMENTED REALITY," under Attorney Docket No. ML.20019.00; U.S. Utility Patent Application No. 15 / 683,677, filed August 22, 2017, entitled "VIRTUAL, AUGMENTED, and MIXED REALITY SYSTEMS AND METHODS," under Attorney Docket No. ML-0341US; and U.S. Utility Patent Application No. 16 / 215,477, filed December 10, 2018, entitled "WAVEGUIDE ILLUMINATOR," under Attorney Docket No. 101782-013710US-1347331.

[0002] Copyright Notice Part of the disclosure of this patent document contains material that is subject to copyright protection. The copyright owner has no objection to the facsimile reproduction by anyone of the patent document or the patent disclosure, as it appears in the Patent and Trademark Office patent file or records, but reserves all copyrights otherwise.

[0003] Field of the Invention The present disclosure relates to extended reality imaging, visualization, and display systems and methods. In particular, the present disclosure relates to segmented illumination light sources for use in display systems. [Background technology]

[0004] background Modern computing and display technologies have facilitated the development of virtual reality ("VR"), augmented reality ("AR"), mixed reality ("MR"), and extended reality ("XR") systems. VR systems create simulated environments for users to experience. This can be done by presenting computer-generated images to the user through a head-mounted display. These images create a sensory experience that immerses the user in the simulated environment. VR scenarios typically involve the presentation of computer-generated images only, and do not include images of the actual real world.

[0005] AR systems generally complement the real-world environment with simulated elements. For example, an AR system may provide a user with a view of the surrounding real-world environment via a head-mounted display. However, to enhance the real-world environment, computer-generated images can also be presented on the display. These computer-generated images may include elements that are contextually relevant to the real-world environment. Such elements may include simulated text, images, and objects. MR systems also introduce simulated objects into the real-world environment, but these objects typically feature a greater degree of interactivity than those in AR systems. Simulated elements can often be interactive in real time. XR systems generally facilitate immersive technologies and include VR systems, AR systems, and MR systems.

[0006] Figure 1 illustrates an exemplary AR / MR / XR scene 1 in which the user views a real-world park setting 6 featuring people, trees, buildings, and a concrete platform 20 in the background. In addition to these items, computer-generated images are also presented to the user. The computer-generated images could include, for example, a robot statue 10 standing on the real-world platform 20 and a cartoon-like avatar character 12 flying around, which appears to be a personification of a bumblebee, even if these elements 12, 10 do not actually exist in the real-world environment.

[0007] Various optical systems generate images at various depths to display VR, AR, MR, or XR scenarios. Some such optical systems are described in U.S. Utility Patent Application No. 14 / 555,585, the contents of which are incorporated herein by reference. Other such optical systems for displaying MR experiences are described in U.S. Utility Patent Application No. 14 / 738,877, the contents of which are incorporated herein by reference.

[0008] The complexity of the human visual perception system makes it challenging to create VR / AR / MR / XR technologies that facilitate the rich and comfortable presentation of virtual image elements among other virtual or real-world image elements. Improved techniques for processing image data in such systems are needed, including techniques for compensating for non-uniformity across the system field of view ("FOV") resulting from the non-uniformity of system components. VR / AR / MR / XR technologies also have issues of size and portability, battery life, system overheating, and other system and optical challenges. Improved techniques are needed to address these issues, including system temperature management. VR / AR / MR / XR display systems also have multiple optical components that can introduce non-uniformity into the images produced by the VR / AR / MR / XR display system. The systems and methods described herein are configured to address at least some of these and other challenges.

[0009] What is needed are one or more techniques to improve legacy techniques and / or other considered methods. Some of the techniques described in this background section are techniques that can be pursued, but are not necessarily techniques that have been previously conceived or pursued. [Overview of the Initiative] [Means for solving the problem]

[0010] overview The embodiments cover VR, AR, MR, or XR systems and methods. In particular, the embodiments cover VR, AR, MR, or XR systems and methods for compensating for non-uniformity using a light source having multiple segments that emit light (instead of a single segment). Such a light source is called a “segmented illumination source”. The segmented illumination source is configured such that the luminance of each segment of the light source can be increased (e.g., up to 100% luminance) or decreased (e.g., down to 0%, or completely off) independently of the other segments of the segmented illumination source.

[0011] In one embodiment, an extended reality display system includes a display subsystem configured to present an image to a user that corresponds to image data. The display subsystem includes an optical component that introduces heterogeneity into the displayed image, a segmented illumination source, and a spatial light modulator (SLM) configured to receive light from the segmented illumination source. The system also includes a display controller configured to control the segmented illumination source. The display controller includes a memory for storing heterogeneity correction information and a processor that controls the segmented illumination source based on the heterogeneity correction information. The segmented illumination source is configured to differentially illuminate the first and second parts of the SLM using the first and second parts of the segmented illumination source, respectively.

[0012] In one or more embodiments, the segmented illumination source includes an array of light-emitting diodes (LEDs), and a display controller is configured to control each LED in the LED array. The SLM may be a liquid crystal on silicon (LCOS) display. The system may also include a lens placed between the segmented illumination source and the SLM. The segmented illumination source may be configured to improve the uniformity of illumination of the image corresponding to the image data by differentially / independently illuminating a first portion and a second portion of the SLM. Uniformity may be illumination uniformity and / or color uniformity. The segmented illumination source may be configured to reduce power consumption by differentially / independently illuminating a first portion and a second portion of the SLM. For example, power consumption may be reduced by disabling LEDs in areas where virtual content is not displayed in the frame. The segmented illumination source may be configured to improve the contrast of the image corresponding to the image data by differentially illuminating a first portion and a second portion of the SLM.

[0013] In one or more embodiments, the segmented illumination source includes a plurality of sub-light sources, and the display controller is configured to control the amount of voltage or current applied to one of the sub-light sources. The processor may be configured to calculate the amount of current delivered to the sub-light source using non-uniformity correction information stored in memory. The processor may be configured to calculate the amount of current delivered to the sub-light source using LEDs and optical stack calibration data that takes into account temperature and per-device optical performance. The display controller may be directly coupled to the segmented illumination source using only a connecting element between the display controller and the segmented illumination source.

[0014] In one or more embodiments, the display controller is configured to instruct the segmented illuminator not to illuminate the third portion of the SLM by leading a zero voltage or current applied to the third portion of the SLM. The display controller may also instruct the segmented illuminator not to illuminate the third portion of the SLM by leading a reverse bias current applied to the third portion of the SLM. The display controller may be configured to instruct the segmented illuminator to increase the amount of first and second light, respectively, produced by the first and second portions of the segmented illuminator. The display controller may be configured to monitor the first and second temperatures, respectively, of the first and second portions of the segmented illuminator.

[0015] In one or more embodiments, the display controller can control both the SLM and the segmented illumination light source independently or synchronously. Independent control of the SLM and the segmented illumination light source can be used for static non-uniformity correction. Synchronized control of the SLM and the segmented illumination light source can be used to further improve non-uniformity correction, contrast, and power saving. The segmented illumination ("SGIL") light source may have an integrated driver ASIC with a processor for controlling the segmented illumination light source using pulse width modulation and / or pulse amplitude modulation. The display controller (e.g., processor) can communicate with the SGI LASIC for coordinated control. Furthermore, the SGI LASIC may be configured to perform some processing and calibration for non-uniformity correction, or non-uniformity correction may be controlled by the display controller processor. In addition to non-uniformity correction, the segmented illumination light source may perform feedforward color balance control (e.g., spectral / intensity vs. temperature and current density) as well as LED calibration for gain maps for non-uniformity correction. The segmented illumination light source may have more than two parts / zones. For example, a segmented illumination source can include various arrays of independently controlled segmented illumination pixels / LEDs (e.g., 5x5, 8x8, etc.).

[0016] In another embodiment, the method is for presenting an image corresponding to image data to a user using a display subsystem that includes optical components that introduce heterogeneity into the image. The method includes a display controller storing heterogeneity correction information in its memory. The method also includes a processor of the display controller controlling a segmented illumination source of the display subsystem based on the heterogeneity correction information. The method further includes the segmented illumination source differentially illuminating a first and second portion of the SLM of the display subsystem using a first and second portion of the segmented illumination source, under the control of the processor.

[0017] In one or more embodiments, the segmented illumination source includes an LED array, and the method further includes a display controller processor controlling each LED in the LED array. Other light sources suitable for segmented illumination systems include, but are not limited to, mini-LEDs, micro-LEDs, organic light-emitting diodes, vertical resonant surface-emitting lasers, superluminescent diodes, and laser diodes. The SLM may be an LCOS display. A lens may be placed between the segmented illumination source and the SLM in the display subsystem. A segmented illumination source that differentially illuminates a first and second portion of the SLM may improve the uniformity of illumination of the image corresponding to the image data. Uniformity may be illumination uniformity and / or color uniformity. A segmented illumination source that differentially illuminates a first and second portion of the SLM may reduce power consumption. A segmented illumination source that differentially illuminates a first and second portion of the SLM may improve the contrast of the image corresponding to the image data.

[0018] In one or more embodiments, the segmented illumination source includes a plurality of sub-light sources, and the method further includes a processor controlling the amount of voltage or current applied to one of the plurality of sub-light sources. The method may further include the processor calculating the amount of current delivered to the sub-light source using non-uniformity correction information stored in memory. The display controller may be directly coupled to the segmented illumination source using only a connecting element between the display controller and the segmented illumination source.

[0019] In one or more embodiments, the method further includes the processor instructing the segmented illuminator not to illuminate the third portion of the SLM by leading a zero voltage or current applied to the third portion of the segmented illuminator. The processor may also instruct the segmented illuminator not to illuminate the third portion of the SLM by leading a reverse bias current applied to the third portion of the segmented illuminator. The method may further include the processor instructing the segmented illuminator to increase the amount of first light and second light, respectively, produced by the first and second portions of the segmented illuminator. The method may further include the processor monitoring the first temperature of the first portion and the second temperature of the second portion of the segmented illuminator, respectively.

[0020] In one or more embodiments, the method further includes a processor acquiring new non-uniformity correction information. The method also includes a display controller storing the new non-uniformity correction information in memory. Furthermore, the method includes a processor controlling the segmented illumination source of the display subsystem based on the new non-uniformity correction information. In addition, the method includes the segmented illumination source differentially illuminating the first and second portions of the SLM of the display subsystem using the respective first and second portions of the segmented illumination source under the control of the processor.

[0021] In another embodiment, the segmented illumination source includes an application-specific integrated circuit (ASIC) comprising a bond pad array. The light source also includes a first LED array coupled to a first portion of the bond pad array. The light source further includes a second LED array coupled to a second portion of the bond pad array. Furthermore, the light source includes a third LED array coupled to a third portion of the bond pad array. The first, second, and third portions are distinct from each other. The ASIC is configured to control the first, second, and third LED arrays.

[0022] In one or more embodiments, the first LED array is configured to emit red light, the second LED array is configured to emit green light, and the third LED array is configured to emit blue light. The first portion of the bond pad array may include a plurality of portions capable of bonding the first LED array. The bond pad array may include a plurality of portions capable of bonding the LED array.

[0023] In yet another embodiment, a method for manufacturing a plurality of segmented lighting light sources includes coupling a first LED array to a first portion of a first bond pad array of a first application-specific integrated circuit (ASIC) to form a first segmented lighting light source. The method also includes coupling a second LED array to a second portion of a second bond pad array of a second ASIC to form a first segmented lighting light source. The first ASIC is the same as the second ASIC. The first portion is positioned at a first position on the first bond pad array. The second portion is positioned at a second position on the second bond pad array. The first position is different from the second position.

[0024] In one or more embodiments, the first segmented lighting light source is configured to be used with a first optical device having a first optical design, and the second segmented lighting light source is configured to be used with a second optical device having a second optical design different from the first optical design. The method may also include determining a first portion of a first bond pad array of a first ASIC based on the first optical design of the first optical device and determining a second portion of a second bond pad array of a second ASIC based on the second optical design of the second optical device.

[0025] The foregoing and other embodiments of the present invention are described in the following detailed description.

Brief Description of the Drawings

[0026] The drawings described below are for illustrative purposes only. The drawings are not intended to limit the scope of the present disclosure.

[0027] The drawings illustrate the design and utility of various embodiments of the present disclosure. Note that the drawings are not drawn to scale, and elements of similar structure or function are represented by like reference numerals throughout the drawings. To better understand the described and other advantages and objects of various embodiments of the present disclosure, a more detailed description of the present disclosure is provided by reference to the specific embodiments shown in the accompanying drawings. It is understood that these drawings show only typical embodiments of the present disclosure and should not be considered as limiting its scope. Thus, the present disclosure is described and explained with additional specificity and detail by using the accompanying drawings.

[0028] [Figure 1] FIG. 1 depicts a user's view of an AR / MR / XR scene using an exemplary AR / MR / XR system according to some embodiments.

[0029] [Figure 2] FIG. 2 depicts an example of a wearable display system according to some embodiments.

[0030] [Figure 3] FIG. 3 is a block diagram depicting an AR / MR / XR system according to some embodiments.

[0031] [Figure 4A] FIG. 4A depicts a conventional display system for simulating a three-dimensional image of a user.

[0032] [Figure 4B] FIG. 4B depicts aspects of a technique for simulating a three-dimensional image using multiple depth planes according to some embodiments.

[0033] [Figure 5] Figures 5A to 5C illustrate the relationship between the radius of curvature and the radius of focus.

[0034] [Figure 6] Figure 6 illustrates an example of a waveguide stack for outputting image information to the user, according to several embodiments.

[0035] [Figure 7] Figure 7 illustrates examples of output beams produced by waveguides according to several embodiments.

[0036] [Figure 8] Figure 8 illustrates an exemplary design of a waveguide stack having three associated waveguides, each outputting light of a different color from a different depth plane, according to several embodiments.

[0037] [Figure 9] Figure 9 illustrates several embodiments of a system for presenting images to a user's eyes and viewing the world. [Figure 10] Figure 10 illustrates several embodiments of a system for presenting images to a user's eyes and viewing the world. [Figure 11] Figure 11 illustrates several embodiments of a system for presenting images to a user's eyes and viewing the world.

[0038] [Figure 12] Figure 12 illustrates a fixed lighting pattern generated by the display system without distortion correction.

[0039] [Figure 13] Figures 13A to 13C illustrate brightness uniformity correction using segmented illumination in several embodiments.

[0040] [Figure 14]Figures 14A to 14B illustrate color uniformity correction using segmented illumination in several embodiments.

[0041] [Figure 15] Figure 15 illustrates segmented lighting patterns configured to reduce power consumption of a display system, according to several embodiments.

[0042] [Figure 16] Figures 16A-16B illustrate the improvement of contrast by segmented illumination in several embodiments.

[0043] [Figure 17] Figure 17 illustrates a method for displaying an image using segmented illumination according to several embodiments.

[0044] [Figure 18] Figure 18 illustrates a segmented illumination source according to several embodiments. [Figure 19] Figure 19 illustrates a segmented illumination source according to several embodiments. [Figure 20] Figure 20 illustrates a segmented illumination source according to several embodiments.

[0045] [Figure 21A] Figure 21A schematically illustrates various light sources according to several embodiments. [Figure 21B] Figure 21B schematically illustrates various light sources according to several embodiments. [Figure 22] Figure 22 schematically illustrates various light sources according to several embodiments. [Figure 23] Figure 23 schematically illustrates various light sources according to several embodiments. [Figure 24] Figure 24 schematically illustrates various light sources according to several embodiments. [Figure 25]Figure 25 schematically illustrates various light sources according to several embodiments. [Figure 26] Figure 26 schematically illustrates various light sources according to several embodiments. [Figure 27] Figure 27 schematically illustrates various light sources according to several embodiments.

[0046] [Figure 28] Figure 28 schematically illustrates a display system according to several embodiments.

[0047] [Figure 29] Figure 29 provides a detailed schematic illustration of the display system shown in Figure 28.

[0048] [Figure 30] Figure 30 provides a detailed schematic illustration of a display system according to several embodiments.

[0049] [Figure 31] Figure 31 illustrates, by several embodiments, a method for manufacturing different optical devices having different optical designs using the same application-specific integrated circuit.

[0050] [Figure 32] Figure 32 is a block diagram schematically illustrating an exemplary computing system according to several embodiments. [Modes for carrying out the invention]

[0051] Detailed explanation The various embodiments of this disclosure cover systems, methods, and products for VR / AR / MR / XR in single or multiple embodiments. Other objects, features, and advantages of this disclosure are described in the embodiments, drawings, and claims for carrying out the invention.

[0052] Next, various embodiments will be described in detail with reference to the drawings provided as illustrative examples, so as to enable those skilled in the art to implement the disclosure. In particular, the following drawings and embodiments are not intended to limit the scope of the disclosure. Where certain elements of the disclosure can be partially or completely implemented using known components (or methods or processes), only the portion of such known components (or methods or processes) necessary for understanding the disclosure will be described, and detailed descriptions of other portions of such known components (or methods or processes) will be omitted so as not to obscure the disclosure. Furthermore, the various embodiments will include current and future known equivalents to the components mentioned herein as examples.

[0053] The embodiments described herein address the implementation challenges of VR / AR / MR / XR systems and often rely on a combination of off-the-shelf and custom components. In some cases, off-the-shelf components lack all the features or performance characteristics necessary to implement a particular desired aspect of the VR / AR / MR / XR system to be deployed. Some embodiments cover techniques for adding capabilities and / or reusing resources to accommodate the desired features or performance characteristics of the VR / AR / MR / XR system to be deployed. The accompanying drawings and discussion herein present exemplary environments, systems, methods, and computer program products for VR / AR / MR / XR systems.

[0054] overview The VR, AR, MR, and XR systems disclosed herein may include a display that presents computer-generated images (video / image data) to a user. In some embodiments, the display system is wearable and may advantageously provide a more immersive VR / AR / MR / XR experience. Figure 2 shows an example of a wearable VR / AR / MR / XR display system 80 (hereinafter referred to as "System 80"). System 80 includes a display 62 and various mechanical and electronic modules and systems to support the functionality of the display 62. The display 62 is wearable by a user or viewer 60 of the display system (hereinafter referred to as "User 60") and may be coupled to a frame 64 configured to position the display 62 in front of the User 60's eyes. In some embodiments, a speaker 66 is coupled to the frame 64 and positioned adjacent to the User 60's ear canal. In some embodiments, another speaker (not shown) is positioned adjacent to the User 60's other ear canal to provide stereo / shape-changing audio control. The display 62 is operably coupled to a local processing and data module 70 by a wired or wireless connection 68, and the local processing and data module can be mounted in various configurations, such as being fixedly attached to a frame 64, fixedly attached to a helmet or hat worn by the user, embedded in headphones, or otherwise detachably attached to the user 60 (e.g., a backpack configuration, a belt-mounted configuration, etc.). In some embodiments, the local processing and data module 70 may be integrated into a headset / glasses.

[0055] The local processing and data module 70 may include a processor and digital memory such as non-volatile memory (e.g., flash memory), both of which may be used to assist in the processing and storage of data. This includes data captured from sensors such as an ambient light sensor ("ALS"), an image capture device (e.g., a camera), a microphone, an inertial measurement unit, an accelerometer, a compass, a GPS unit, a wireless device, an antenna array, a depth sensor, and / or a gyroscope. The ALS may be configured to provide optical data for a processor and algorithms for segmented illumination brightness control. The sensors may be operably coupled to the frame 64 or otherwise attached to the user 60. Alternatively or further, sensor data may be retrieved and / or processed using the remote processing module 72 and / or remote data repository 74 for passing to the display 62 after such processing or retrieval, if applicable. The local processing and data module 70 may be operably coupled to the remote processing module 72 and the remote data repository 74 via communication links (76, 78), for example via wired or wireless communication links, so that they are operably coupled to each other and available as resources for the local processing and data module 70.

[0056] In some embodiments, the remote processing module 72 may include one or more processors configured to analyze and process data (e.g., sensor data and / or image information). In some embodiments, the remote data repository 74 may include digital data storage facilities that may be available via the Internet or other networking configurations within a “cloud” resource configuration. In some embodiments, all data is stored and all calculations are performed locally and within the data module to enable fully autonomous use from the remote module.

[0057] In some embodiments, computer-generated image data provided via the display 62 can create the impression of being three-dimensional. This can be done, for example, by presenting the user 60 with a stereoscopic image corresponding to the image data. In some conventional systems, such image data may include separate images of a scene or object from slightly different viewpoints. These separate images can be presented to the user 60's right and left eyes, respectively, thus simulating binocular vision and its associated depth perception.

[0058] Referring here to Figure 3, an exemplary embodiment of an AR or MR system 3300 (hereinafter referred to as "System 3300") is shown. System 3300 uses stacked optical guide elements (hereinafter referred to as "LOE3390"). System 3300 generally includes an image generation processor 3310, a light source 3320, a controller / driver 3330, a spatial light modulator ("SLM") 3340, and at least one set of stacked LOE3390 that function as a multi-plane focus system. System 3300 may also include an eye-tracking subsystem 3350. It should be understood that other embodiments may have multiple sets of stacked LOE3390.

[0059] The image generation processor 3310 is configured to generate virtual content to be displayed to the user. The image generation processor 3310 can convert images or videos associated with the virtual content into a format that can be projected in 3D to the user. For example, when generating 3D content, the virtual content may need to be formatted so that certain parts of the image are displayed in a specific depth plane and other parts are displayed in other depth planes. In one embodiment, the entire image may be generated in a specific depth plane. In another embodiment, the image generation processor 3310 may be programmed to supply slightly different images to the left and right eyes so that the virtual content looks consistently comfortable to the user's eyes when viewed together.

[0060] The image generation processor 3310 may further include memory 3312, a GPU 3314, a CPU 3316, and other circuitry for image generation and processing. The image generation processor 3310 may be programmed with desired virtual content to be presented to the user of system 3300. It should be understood that in some embodiments, the image generation processor 3310 may be housed in system 3300. In other embodiments, the image generation processor 3310 and other circuitry may be housed in a beltpack coupled to system 3300.

[0061] The image generation processor 3310 is operably coupled to a light source 3320 and one or more spatial light modulators 3340 that project light associated with the desired virtual content. The image generation processor 3310 and / or controller / driver 3330 may also calibrate and correct system non-uniformity. For example, EP non-uniformity or segmented illumination non-uniformity can result from exposure to high temperatures and aging. The image generation processor 3310 may also warp / transform images with low latency, taking user motion into consideration. This warp / transformation can also be applied to segmented illumination. The light source 3320 is small and has high resolution.

[0062] The light source 3320 is operably coupled to the controller / driver 3330. In various embodiments, the light source 3320 may be connected to the controller / driver 3330. In some embodiments, the light source 3320 and the controller / driver 3330 are located in the same place on the same board. Those skilled in the art will understand the value of minimizing the effects of cable inductance and capacitance by placing the light source in the same location as the “driver” electronic equipment. Minimizing the effects of coupled inductance and capacitance by placing the light source 3320 and the controller / driver 3330 in the same location facilitates faster transitions of the light source 3320 (i.e., “on” to “off” and all intervening illumination levels). In embodiments where the light source 3320 and the controller / driver 3330 are located further apart from each other, the stray inductance and capacitance resulting from the increased distance between the light source 3320 and the controller / driver 3330 can be managed by appropriate cable structures, such as those used in flexible cables that can provide shielding.

[0063] The light source 3320 is electrically coupled to the controller / driver 3330, enabling the controller / driver 3330 to drive the segmented illumination segments. The controller / driver 3330 can also be electrically connected to both the image generation processor 3310 and the SLM 3340 to facilitate coordinated control. The image generation processor 3310 may transmit processed illumination array data (gray values ​​for each color) to the controller / driver 3330. The controller / driver 3330 may also include a uniformity correction table / processing capability. The SLM 3340 may transmit timing signals to the controller / driver 3330 to facilitate segmented illumination at the appropriate time(s). In some embodiments, electronically connecting the controller / driver 3330 to the SLM 3340 allows the light source 3320 to emit the correct LED color while the SLM 3340 is refreshing the color field of the field sequential display system.

[0064] The light source 3320 may include color-specific light-emitting diodes ("LEDs") arranged in various geometric configurations. Alternatively, the light source 3320 may include LEDs of similar colors, each linked to a specific area of ​​the display's field of view. In some embodiments, the light source 3320 may include mini-LEDs, micro-LEDs, laser diodes, phosphor-converted LED sources, vertical-resonant surface-emitting lasers ("VSCELs"), and / or superluminescent diodes ("SLDs"). In some embodiments, the light source 3320 may include a mask overlay for segmenting the light-emitting area and position. Although the light source 3320 is directly connected to the system 3300 in Figure 3, the light source 3320 may be connected to the system 3300 via optical fibers (not shown). The system 3300 may also include capacitors (not shown) configured to collimate the light from the light source 3320.

[0065] In various exemplary embodiments, the SLM3340 may be reflective (e.g., LCOS, FLCOS, DLP DMD, or MEMS mirror system) or transmissive (e.g., LCD). The type of SLM3340 (e.g., speed, size, etc.) can be selected to improve the creation of 3D perception. A DLP DMD operating at a higher refresh rate may be readily incorporated into the fixed system 3300, although the wearable system 3300 may use a DLP of smaller size and power. The output of the DLP changes how the 3D depth plane / focal plane is created. The image generation processor 3310 is operably coupled to the SLM3340, which encodes light from a light source 3320 having desired virtual content. The light from the light source 3320 may be encoded with image information as it reflects from, radiates from, or passes through the SLM3340.

[0066] Light from the SLM3340 is directed to the LOE3390 such that a light beam encoded by the SLM3340 with image data of one depth plane and / or color propagates effectively along a single LOE3390 to be delivered to the user's eye. Each LOE3390 is configured to project an image or sub-image onto the user's retina that appears to originate from a desired depth plane or FOV angle position. Thus, the light sources 3320 and LOE3390 can selectively project images (synchronously encoded by the SLM3340 under the control of the controller / driver 3330) that appear to originate from various depth planes or positions in space. By sequentially projecting images using each of the light sources 3320 and LOE3390 at a sufficiently high frame rate (e.g., 360Hz for six depth planes with an effective full volume frame rate of 60Hz), the system 3300 can generate 3D images of virtual objects in various depth planes that appear to exist simultaneously in the 3D image.

[0067] The controller / driver 3330 communicates with and is operablely coupled to the image generation processor 3310, the light source 3320, and the SLM 3340, and coordinates the synchronized display of images by instructing the SLM 3340 to encode the light beam from the light source 3320 using appropriate image information from the image generation processor 3310.

[0068] System 3300 also includes an optional eye-tracking subsystem 3350 configured to track the user's eyes and determine the user's focus. In one embodiment, System 3300 is configured to illuminate a subset of LOE 3390 based on input from the eye-tracking subsystem 3350 so that an image is produced in a desired depth plane that matches the user's focus / accommodation. For example, if the user's eyes are parallel to each other, System 3300 may illuminate LOE 3390 configured to deliver collimated light to the user's eyes so that the image appears to originate from optical infinity. In another example, if the eye-tracking subsystem 3350 determines that the user's focus is 1 meter away, LOE 3390 configured to focus approximately within that range may be illuminated instead.

[0069] Figure 4A shows a conventional display system for simulating a three-dimensional image corresponding to image data for a user (e.g., user 60). Two separate images 84 and 86 are output to the user, one for each eye 4 and 5. Images 84 and 86 are spaced a distance 12 from eyes 4 and 5 along the optical axis or z-axis parallel to user 60's line of sight. Images 84 and 86 are flat, and eyes 4 and 5 can focus on the images by exhibiting a single accommodation state. Such a system relies on the human visual system to provide a perception of depth of the combined image of images 84 and 86.

[0070] However, it will be understood that the human visual system is more complex and more difficult to provide a realistic sense of depth. For example, many users of conventional 3D display systems may find such systems uncomfortable or may not perceive any sense of depth at all. Although not limited by theory, it is thought that a user looking at an object may perceive the object as "three-dimensional" due to a combination of convergence-divergence movements and accommodation. The convergence-divergence movements of the two eyes relative to each other (e.g., the rolling of the pupils toward or toward each other to converge the lines of sight of the eyes in order to gaze at an object) are closely related to the focusing of the eye's lens (or "accommodation"). Under normal conditions, changing the focus of the eye's lens to change focus from one object to another at a different distance, or accommodation of the eyes, automatically produces a consistent change in convergence-divergence movements for the same distance under a relationship known as the "accommodation-convergence-divergence reflex". Similarly, changes in convergence-divergence movements cause a consistent change in accommodation under normal conditions. As described herein, many stereoscopic display systems display scenes using slightly different presentations (and therefore slightly different images) for each eye so that the human visual system perceives a three-dimensional viewpoint. While such systems simply provide different presentations of the scene, they are uncomfortable for many users because the eyes see all the image information in a single accommodative state, thus going against the accommodative-convergence-divergence reflex. Systems that provide better coherence between accommodative and convergence-divergence movements can form a more realistic and comfortable simulation of the three-dimensional image corresponding to the image data.

[0071] For example, light field video data can be presented to a user to simulate a three-dimensional view. Light field video data can mimic the light rays that enter the user's eyes in a real-world environment. For instance, when displaying light field video data, light rays from objects simulated to be perceived at a certain distance are made to collimate more when they enter the user's eyes, while light rays from objects simulated to be perceived up close are made to diverge more. Therefore, the angle at which light rays from objects in the scene enter the user's eyes depends on the simulated distance of those objects from the viewer. Light field video data in VR / AR / MR / XR systems can include multiple images of a scene or objects from different depth planes. The images may be different for each depth plane (for example, one depth plane may include an image corresponding to the foreground of the scene, and another depth plane may include an image corresponding to the background of the scene), and may be focused separately by the viewer's eyes, thereby helping to provide the user with a comfortable perception of depth.

[0072] When these multiple depth plane images are presented to the viewer simultaneously or rapidly in sequence, the result is interpreted by the viewer as a three-dimensional image. When a viewer experiences this type of light field video data, the eyes adjust to focus on different depth planes in much the same way as when experiencing a real-world scene. These focus cues can provide a more realistic simulated three-dimensional environment.

[0073] In some configurations, a full-color image can be formed by superimposing component images, each having a specific component color, in each depth plane. For example, red, green, and blue images can be output separately to form each full-color depth plane image. As a result, each depth plane may have multiple associated component color images.

[0074] Figure 4B illustrates an embodiment of a technique for simulating a three-dimensional image using multiple depth planes. Objects at various distances from eyes 4 and 5 on the z-axis are accommodated by eyes (4,5) to focus on those objects. Eyes 4 and 5 exhibit specific accommodative states to focus on objects at different distances along the z-axis. As a result, a specific accommodative state can be said to be associated with a particular depth plane such that an object or part of an object in that particular depth plane is in focus when the eye is accommodating to that depth plane. In some embodiments, the three-dimensional image corresponding to the image data can be simulated by providing different presentations of the image corresponding to the image data for each eye (4,5) and by providing different presentations of the image corresponding to the image data for each depth plane.

[0075] The distance between an object and the eye (4 or 5) can change the amount of light diverging from that object as seen by that eye. Figures 5A to 5C illustrate the relationship between distance and ray divergence. The distance between the object and eye 4 is represented by R1, R2, and R3 in decreasing order of distance. As shown in Figures 5A to 5C, the ray diverges more as the distance from eye 4 to the object decreases. As the distance from eye 4 to the object increases, the ray becomes more collimated. In other words, the light field produced by a point (object or part of an object) can be said to have a spherical wavefront curvature that is a function of how far that point is from the user's eye 4. The curvature increases as the distance between the object and eye 4 decreases. As a result, the degree of ray divergence also differs at different depth planes, and the degree of divergence increases as the distance between the depth plane and eye 4 decreases. Although only monocular eye 4 is shown in Figures 5A–5C and other figures herein for clarity, it will be understood that the discussion concerning eye 4 may also apply to both eyes (4 and 6) of the viewer.

[0076] While not limited by theory, the human eye is typically thought to be able to interpret a finite number of depth planes to provide depth perception. As a result, a highly realistic simulation of perceived depth can be achieved by providing the eye with different image presentations corresponding to image data for each of these limited number of depth planes.

[0077] Figure 6 shows an example of a waveguide stack for outputting an image corresponding to image data to a user (e.g., user 60). The display system 1000 includes a waveguide stack or stacked waveguide assembly 178 that can be used to provide three-dimensional perception to the eyes / brain using one or more waveguides (182, 184, 186, 188, 190). In some embodiments, the display system 1000 is system 80 in Figure 2, and Figure 6 schematically shows some parts of that system 80 in more detail. For example, the stacked waveguide assembly 178 may be integrated with the display 62 in Figure 2.

[0078] The stacked waveguide assembly 178 may also include one or more features (198, 196, 194, 192) between the waveguides. In some embodiments, the features (198, 196, 194, 192) may be lenses. The waveguides (182, 184, 186, 188, 190) and / or lenses (198, 196, 194, 192) may be configured to deliver light corresponding to image data to the eye at various levels of wavefront curvature or ray divergence. Each waveguide level may be associated with a specific depth plane and may be configured to output light corresponding to image data corresponding to that depth plane. Image input devices (200, 202, 204, 206, 208) may be used to input light corresponding to image data into waveguides (182, 184, 186, 188, 190), each of which may be configured to distribute the incident light across its respective waveguide for output toward eye 4, as described herein. The light exits the output surfaces (300, 302, 304, 306, 308) of the image input devices (200, 202, 204, 206, 208) and is input to the corresponding input edges (382, 384, 386, 388, 390) of the waveguides (182, 184, 186, 188, 190). In some embodiments, a single light beam (e.g., a collimated beam) may be fed into each waveguide to output an entire cloned collimated beam field that is directed to eye 4 at a specific angle (and divergence) corresponding to a depth plane associated with a particular waveguide.

[0079] In some embodiments, the image input devices (200, 202, 204, 206, 208) are individual displays, each generating light corresponding to image data for input into the corresponding waveguides (182, 184, 186, 188, 190, respectively). In some other embodiments, the image input devices (200, 202, 204, 206, 208) are output terminals of a single multiplexed display capable of sending light corresponding to image data to each of the image input devices (200, 202, 204, 206, 208) via one or more optical conduits (such as optical fiber cables).

[0080] The controller 210 controls the operation of the stacked waveguide assembly 178 and the image input devices (200, 202, 204, 206, 208). In some embodiments, the controller 210 includes programming (e.g., instructions in a non-transient computer-readable medium) to coordinate the timing and delivery of light corresponding to image data to the waveguides (182, 184, 186, 188, 190) according to one of the various schemes disclosed herein. In some embodiments, the controller 210 may be a single integrated device or a distributed system connected by wired or wireless communication channels. In some embodiments, the controller 210 may be part of a processing module (e.g., local processing and data module 70 and / or remote processing module 72 in Figure 2).

[0081] Waveguides (182, 184, 186, 188, 190) may be configured to propagate light within each waveguide by total internal reflection (TIR). Each waveguide (182, 184, 186, 188, 190) may be planar or curved, and have a main upper surface and a main bottom surface, as well as edges extending between these main upper and main bottom surfaces. In the illustrated configuration, each waveguide (182, 184, 186, 188, 190) may include an optical redirection element (282, 284, 286, 288, 290) configured to redirect light propagating within each waveguide and output light corresponding to image data to eye 4. The optical beam is output by the waveguide at the point where light propagating within the waveguide strikes the optical redirection element. The optical redirection elements (282, 284, 286, 288, 290) may also be reflective and / or diffractive optical features. For the sake of clarity of the illustration and for ease of explanation, they are shown mounted on the bottom main surface of the waveguide (182, 184, 186, 188, 190), but in some embodiments, the optical redirection elements (282, 284, 286, 288, 290) may be mounted on the top main surface and / or the bottom main surface, and / or directly within the volume of the waveguide (182, 184, 186, 188, 190). In some embodiments, the optical redirection elements (282, 284, 286, 288, 290) may be formed within a material layer attached to a transparent substrate to form the waveguide (182, 184, 186, 188, 190). In some other embodiments, the waveguides (182, 184, 186, 188, 190) may be monolithic material pieces, and the optical redirection elements (282, 284, 286, 288, 290) may be formed on the surface and / or inside the material piece.

[0082] As discussed herein, each waveguide (182, 184, 186, 188, 190) is configured to output light to form an image corresponding to a particular depth plane. For example, waveguide 182 closest to eye 4 may be configured to deliver collimated light fed into such waveguide 182 to eye 4. The collimated light may represent the optical infinity focal plane. The next upper waveguide 184 may be configured to emit collimated light that passes through a first lens 192 (e.g., a negative lens) before reaching eye 4. The first lens 192 may be configured to produce a slightly convex wavefront curvature so that the eye / brain interprets the light coming from the next upper waveguide 184 as coming from a first focal plane closer inward from optical infinity toward eye 4. Similarly, a third upper waveguide 186 may pass its output light through both the first lens 192 and the second lens 194 before reaching the eye 4, and the combined refractive power of the first lens 192 and the second lens 194 may be configured to produce another incremental amount of wavefront curvature so that the eye / brain interprets the light coming from the third waveguide 186 as coming from a second focal plane that is even closer inward toward the user from optical infinity than the light from the next upper waveguide 184.

[0083] Other waveguide layers (188, 190) and lenses (196, 198) are configured similarly, with the tallest waveguide 190 in the stack sending its output through all the lenses between that waveguide and the eye for a concentrated focal refractive power representing the focal plane closest to the user. To compensate for the stack of lenses (198, 196, 194, 192) when viewing / interpreting light coming from the other side of the stacked waveguide assembly 178, a compensating lens layer 180 may be placed on top of the stacked waveguide assembly 178 to compensate for the concentrated refractive power of the lower lens stack (198, 196, 194, 192). Such a configuration provides as many perceived focal planes as there are available waveguide / lens pairings. Both the optical redirection elements of the waveguides and the focusing modes of the lenses may be static (i.e., not dynamically or electrically active). In some alternative embodiments, they may be dynamic using electrically active features.

[0084] Optical redirection elements (282, 284, 286, 288, 290) can be configured to redirect light out of their respective waveguides and to output this light with an appropriate amount of divergence or collimation to a specific depth plane associated with the waveguide. As a result, waveguides with different associated depth planes may have different configurations of optical redirection elements (282, 284, 286, 288, 290) that output light with different amounts of divergence depending on the associated depth plane. In some embodiments, as discussed herein, the optical redirection elements (282, 284, 286, 288, 290) may be volumetric or surface features that can be configured to output light at a specific angle. For example, the optical redirection elements (282, 284, 286, 288, 290) may be volumetric holograms, surface holograms, and / or diffraction gratings. Optical redirection elements such as diffraction gratings are described in U.S. Patent Application No. 14 / 641,376, filed on March 7, 2015, which is incorporated herein by reference in its entirety. In some embodiments, the features (198, 196, 194, 192) do not have to be lenses, but rather they may simply be spacers (e.g., cladding layers and / or structures for forming voids).

[0085] In some embodiments, the optical redirection elements (282, 284, 286, 288, 290) are diffraction features that form a diffraction pattern, i.e., “diffractive optical elements” (also referred to herein as “DOEs”). Preferably, the DOEs have sufficiently low diffraction efficiency such that only a portion of the beam light is deflected toward the eye 4 at each intersection of the DOEs, while the remainder continues to travel through the waveguide via total internal reflection. Thus, the light carrying the image data is split into several associated outgoing beams exiting the waveguide at numerous locations, resulting in a fairly uniform pattern of outgoing radiation toward the eye 4 for this particular collimated beam reflected within the waveguide.

[0086] In some embodiments, one or more DOEs may be switchable between an "on" state in which they actively diffract and an "off" state in which they do not diffract significantly. For example, a switchable DOE may include a layer of polymer-dispersed liquid crystal, in which microdroplets contain a diffraction pattern in the host medium, and the refractive index of the microdroplets can be switched to substantially match the refractive index of the host material (in this case, the pattern does not diffract enough to recognize incident light), or the microdroplets can be switched to a refractive index that does not match the refractive index of the host medium (in this case, the pattern actively diffracts incident light).

[0087] Figure 7 shows an example of an outgoing beam output by a waveguide. Although one waveguide is illustrated, it will be understood that other waveguides in the stacked waveguide assembly 178 may function similarly. Light 400 is injected into waveguide 182 at its input edge 382 and propagates through waveguide 182 by TIR. At the point where light 400 collides with DOE 282, a portion of the light exits the waveguide as an outgoing beam 402. The outgoing beam 402 is illustrated as substantially parallel, but may be redirected to propagate to eye 4 at a certain angle (e.g., forming a divergent outgoing beam) depending on the depth plane associated with waveguide 182, as discussed herein. It will be understood that a substantially parallel outgoing beam may represent a waveguide corresponding to a depth plane at a large simulated distance from eye 4 (e.g., optical infinity). Other waveguides may output a more divergent outgoing beam pattern, which requires eye 4 to adjust for a closer simulated distance, and is interpreted by the brain as light from a distance closer to eye 4 than optical infinity.

[0088] Figure 8 schematically illustrates an exemplary design of a stacked waveguide assembly (e.g., stacked waveguide assembly 178) having three associated waveguides, each depth plane outputting light of a different color. A full-color image can be formed on each depth plane by superimposing images of each of several component colors (e.g., three or more component colors). In some embodiments, the component colors include red, green, and blue. In some other embodiments, other colors, including magenta, yellow, and cyan, may be used in combination with or substituted for one of red, green, or blue. Each waveguide may be configured to output a specific component color, and as a result, each depth plane may have multiple associated waveguides. Each depth plane may have three associated waveguides: a first waveguide for outputting red light, a second waveguide for outputting green light, and a third waveguide for outputting blue light.

[0089] Depth planes 14a–14f are shown in Figure 8. In the illustrated embodiments, each depth plane has three component color images associated with it, including a first image of a first color G, a second image of a second color R, and a third image of a third color B. The number following each of these letters indicates the diopter (1 / m), i.e., the reciprocal of the apparent distance of the depth plane from the viewer, and each box in the figure represents an individual component color image. In some embodiments, G is green, R is red, and B is blue. As discussed above, the perceived distance of the depth plane from the user can be established by light redirecting elements (282, 284, 286, 288, 290) (e.g., diffractive optical elements (DOE), and / or lenses (198, 196, 194, 192)) that diverge light at angles related to the apparent distance.

[0090] In some arrays, each component color image may be output by a different waveguide in a stack of waveguides. For example, each depth plane may have three associated component color images: a first waveguide for outputting the first color G, a second waveguide for outputting the second color R, and a third waveguide for outputting the third color B. In an array where waveguides are used to output component color images, each box in Figure 8 can be understood to represent an individual waveguide.

[0091] For the sake of clarity, this schematic diagram shows the waveguides associated with each depth plane adjacent to one another; however, it will be understood that in a physical device, all waveguides can be arranged in a stack with one waveguide per level. Different depth planes are indicated in the diagram by different numbers of diopters following the letters G, R, and B.

[0092] Definition and use of drawings Some of the terms used herein are defined below for reference. The terms presented and their respective definitions are not strictly limited to these definitions, and terms may be further defined by their use in this disclosure. The term “exemplary” is used herein to mean an example, case, or illustration. Any aspect or design described herein as “exemplary” should not necessarily be construed as being preferable or advantageous to any other aspect or design. Rather, the use of the term “exemplary” is intended to concretely present a concept. Where used in this application and the attached claims, the term “or” is intended to mean an inclusive “or” rather than an exclusive “or.” That is, unless otherwise specified or as is evident from the context, “X uses A or B” is intended to mean either of the natural inclusive substitutions. That is, if X uses A, if X uses B, or if X uses both A and B, “X uses A or B” satisfies any of the above cases. As used herein, at least one of A or B means at least one of A, or at least one of B, or at least one of both A and B. In other words, this phrase is disjunctive. As used in this application and the appended claims, the articles “a” and “an” should generally be interpreted as meaning “one or plural,” unless otherwise specified, or unless it is clear from the context that they refer to a singular form.

[0093] Various embodiments are described herein with reference to the drawings. Note that the drawings are not necessarily drawn to scale, and elements of similar structure or function may be represented by the same reference numerals throughout the drawings. Also note that the drawings are intended solely to facilitate the description of the disclosed embodiments and do not represent an exhaustive treatment of all possible embodiments, nor are they intended to impose any limitations on the claims. In addition, the illustrated embodiments do not need to depict all aspects or advantages of use in any particular environment.

[0094] Aspects or advantages described in conjunction with a particular embodiment are not necessarily limited to that embodiment and may be implemented in any other embodiment, even if not so illustrated. Throughout this specification, references to “some embodiments” or “other embodiments” refer to specific features, structures, materials, or properties described in relation to an embodiment as being included in at least one embodiment. Thus, occurrences of the phrases “in some embodiments” or “in other embodiments” in various parts of this specification do not necessarily refer to the same one or more embodiments. The disclosed embodiments are not intended to limit the scope of the claims.

[0095] Non-uniformity correction Exemplary segmented lighting display system Figure 9 illustrates a system (e.g., an augmented reality display system) 900 for presenting an image to a user's eye 902 and viewing the world 904, according to several embodiments. The system 900 includes a segmented illumination light source 910, a spatial light modulator (SLM) 940, and a waveguide 920 arranged such that light from the light source 910 illuminates the SLM 940 and the light reflected from the SLM 940 is coupled to the waveguide 920 and directed toward the eye 902. The system 900 includes an optical system 930 configured to illuminate at least a portion of the SLM 940 and to project an image of the SLM 940. Light from the segmented illumination light source 910 propagates, for example, through the optical system 930 to at least a portion of the SLM 940 in a first direction, thereby illuminating at least a portion of the SLM 940. Light reflected from SLM940 propagates again through the optical system 930 in a second direction opposite to the first direction, is directed to the waveguide 920, and is coupled in the waveguide.

[0096] The segmented illumination source 910 may include an array of independently addressable / operable LEDs or other types of light sources within an array of independently addressable / operable arrays. In some embodiments, such as those depicted in Figure 9, the segmented illumination source 910 is a 10 × 10 array of LEDs. In other embodiments, the segmented illumination source 910 may be another array having a different number of LEDs (e.g., any number greater than 0) in each direction. The depicted segmented illumination source 910 is a square array, but other shapes are also within the scope of this disclosure. In various embodiments, subgroups of LEDs within the array forming the segmented illumination source 910 are independently addressable / operable. The LEDs may be operable in an on / off state, or may be operable at various brightness levels from 0% (off) to 100%. The LEDs may be configured to emit light of one color. The system 900 depicted in Figure 9 includes a single segmented illumination source 910, but systems according to other embodiments include multiple segmented illumination sources. In some embodiments, the system includes three segmented illumination sources, one for each of the three colors (e.g., red, green, and blue; violet, cyan, and yellow; or any other color). In some embodiments, the system includes two segmented illumination sources: one for a first color or color combination (e.g., red) and one for a second color or color combination (e.g., green and blue).

[0097] In some embodiments, multiple segmented illumination sources are implemented on a single panel, with separate or partially overlapping portions of the single panel functioning as each segmented illumination source. In some embodiments, multiple segmented illumination sources are implemented on multiple (e.g., two or three) panels. Figure 18 illustrates a segmented illumination source 1810 implemented on three panels according to some embodiments. The segmented illumination source 1810 includes a red panel 1812, a green panel 1814, and a blue panel 1816. Figure 19 illustrates a segmented illumination source 1910 implemented on three panels according to some embodiments. The segmented illumination source 1910 includes a first red / green / blue ("RGB") panel 1912, a second RGB panel 1914, and a third RGB panel 1916. Figure 20 illustrates a segmented illumination source 2010 implemented on three panels according to some embodiments. The segmented illumination light source 2010 includes a first yellow / magenta / cyan ("YMC") panel 2012, a second YMC panel 2014, and a third YMC panel 2016. While the RGB and YMC segmented illumination light sources 1810, 1910, and 2010 each have three exemplary colors, other embodiments of the segmented illumination light source may have fewer or more colors than three. In other embodiments, the colors of the segmented illumination light source may differ from those of RGB and YMC.

[0098] The segmented illumination source 910 may be a polarizing light source, but is not limited to that. In some implementations, a polarizer 915 may be positioned between the segmented illumination source 910 and the SLM 940. As illustrated, the polarizer 915 is located between the segmented illumination source 910 and the waveguide 920. This polarizer 915 may also be an optical recycler that transmits light of a first polarization and reflects light of a second polarization back to the segmented illumination source 910. Such a polarizer 915 may be, for example, a wire grid polarizer. A non-imaging optical element (e.g., a coupled optical system 905 such as a cone, a composite parabolic condenser (CPC), or a lens) may be placed relative to the segmented illumination source 910 to receive the light output from the segmented illumination source 910. The coupled optical system 905 may collect the light from the segmented illumination source 910 and, if necessary, reduce the divergence of the light emitted from the segmented illumination source 910. The coupled optical system 905 may, for example, collimate the light output from the segmented illumination source 910. The coupled optical system 905 may collect light that matches the angular spectral field of view of the system 900. Therefore, the coupled optical system 905 can... The angular spectrum of the light output by the segmented illumination source 910 can be matched to the field of view of the system 900. The coupled optical system 905 may have an asymmetric profile to operate asymmetrically with respect to the light emitted from the segmented illumination source 910. For example, the coupled optical system 905 may reduce divergence by different amounts in the orthogonal directions (e.g., x and z directions). Such asymmetry in the coupled optical system 905 can address the asymmetry of the light emitted from the segmented illumination source 910, for example, by including an LED that emits light in one direction (e.g., x or z) with respect to a wider angular range with respect to the orthogonal directions (e.g., z or x, respectively).

[0099] As discussed above, system 900 includes an optical system 930 configured to illuminate the SLM 940, which is positioned in the optical path between the segmented illumination source 910 and the SLM 940. The optical system 930 may include a transmission optical system that transmits light from the segmented illumination source 910 to the SLM 940. The optical system 930 may also be configured to project an image of the SLM 940, or an image formed by the SLM 940 onto the waveguide 920. The image may be projected onto the user's eye 902. In some designs, the optical system 930 may include one or more lenses or optical elements having refractive power. The optical system 930 may, for example, have positive refractive power. The optical system 930 may include one or more refractive optical elements, such as refractive lenses. Other types of optical elements may also be used in some cases.

[0100] An SLM940 can reflect, modulate, and reflect light from it. An SLM940 may be a polarization-based SLM configured to modulate polarization. An SLM940 may include, for example, a liquid crystal (LC) SLM (e.g., liquid crystal on silicon (LCOS) SLM). An LCOS SLM may include, for example, a twisted nematic (TN) liquid crystal. An SLM940 may include, for example, one or more pixels configured to selectively modulate light incident on a pixel depending on the state of the pixel. In some types of SLM940s, a pixel can modulate a beam incident on it by changing its polarization state, for example, by rotating the polarization (e.g., rotating the direction of linear polarization).

[0101] As discussed above, the SLM940 may also be an LCOS SLM940. In a cross-polarizer configuration, the LCOS SLM940 can be nominally white. When a pixel is off (e.g., 0 voltage), the pixel has a bright state, and when a pixel is on (e.g., a voltage above the threshold rise voltage), the pixel has a dark state. In this cross-polarization configuration, leakage is minimized when the pixel is on and has a dark state.

[0102] In a parallel polarizer configuration, the LCOS SLM940 is nominally black. When a pixel is off (e.g., 0 voltage), the pixel has a dark state, and when a pixel is on (e.g., a voltage above the threshold rise voltage), the pixel has a bright state. In this parallel polarizer configuration, leakage is minimized when the pixel is off and has a dark state. The dark state can be (re)optimized using the rubbing direction and compensator plate angle. The compensator plate angle may refer to the angle of a compensator plate (not shown) that may be between the optical system 930 and the SLM940.

[0103] The dynamic range and throughput of a parallel polarizer configuration may differ from those of a cross polarizer configuration. Furthermore, unlike the cross polarizer configuration, the parallel polarizer configuration can be optimized for contrast.

[0104] The system 900 includes a waveguide 920 for outputting image information to the eye 902. The waveguide 920 may include a substantially transparent material having a refractive index sufficient to guide light therein. As illustrated, the waveguide 920 may include a first side surface 921, a second side surface 923 opposite the first side surface 921, and corresponding upper and lower main surfaces and edges therebetween. The first main surface 921 and the second main surface 923 may be sufficiently flat so that image information can be retained as light propagates from the SLM 940 to the eye 902, so that the image formed by the SLM 940 can be fed into the eye 902. The optical system 930 and the SLM 940 may be positioned on the first side surface 921 of the waveguide 920. The segmented illumination source 910 may be positioned on the second side 923 such that light from the segmented illumination source 910 passes through the waveguide 920 and into the optical system 930 before being incident on the second side 923 before reaching the SLM 940. Thus, the waveguide 920 may be positioned between the segmented illumination source 910 and the optical system 930. Furthermore, at least a portion of the waveguide 920 may extend between the segmented illumination source 910 and the optical system 930, thereby allowing light to proceed into the optical system 930 through a portion of the waveguide 920. Thus, light emitted from the segmented illumination source 910 can be directed into the optical system 930 through the waveguide 920 and incident on the SLM 940 through the optical system. The SLM 940 reflects the light back to the waveguide 920 through the optical system 930.

[0105] System 900 also includes an input coupling optical element 960 for coupling light from the optical system 930 to the waveguide 920. The input coupling optical element 960 may be mounted on the main surface of the waveguide 920 (e.g., the upper main surface 923). In some designs, the input coupling optical element 960 may be mounted on the lower main surface 921 of the waveguide 920. Although exemplified on one side or corner of the waveguide 920, the input coupling optical element 960 may be mounted in / on other areas of the waveguide 920. For example, the input coupling optical element 960 may be mounted within the body of the waveguide 920. The input coupling optical element 960 may be a diffractive optical element or a reflector. Other structures may be used as the input coupling optical element 960. The input coupling optical element 960 may be configured to direct the light incident therein into the waveguide 920 at a glazing angle that is sufficiently large (e.g., greater than the critical angle) relative to the upper primary surface 923 and lower primary surface 921 of the waveguide 920, where the light is to be guided by total internal reflection. Furthermore, the input coupling optical element 960 may operate over a wide wavelength range and therefore may be configured to couple multiple colors of light into the waveguide 920. For example, the input coupling optical element 960 may be configured to couple red, green, and blue light into the waveguide 920, and the segmented illumination light source 910 may emit red, green, and blue light at different times.

[0106] The system 900 includes an optical distribution element 970 installed on or within the waveguide 920. In some embodiments, the optical distribution element 970 may be an orthogonal pupil expander (OPE). The optical distribution element 970 may be configured to diffuse light within the waveguide 920 by directing light propagating in the x-direction, for example, in the z-direction. Thus, the optical distribution element 970 may be configured to increase the dimensions of the eyebox along the z-axis. The optical distribution element 970 may include, for example, one or more diffractive optical elements configured to diffract light propagating within the waveguide 920 that is incident on the diffractive optical element, and to redirect the light, for example, in a substantially orthogonal direction.

[0107] Other configurations are also possible. For example, an embodiment having multiple segmented illumination sources may have multiple waveguides corresponding to each of the segmented illumination sources. In such an embodiment, each of the multiple waveguides may have an internal coupling grid that does not overlap with the internal coupling grids for the other waveguides in order to minimize unintended internal coupling to the other waveguides.

[0108] Figure 10 illustrates a system (e.g., an AR display system) 1000 for presenting images to a user's eyes and viewing the world, according to several embodiments. System 1000 includes a light source 1010, a plurality (e.g., three) waveguides 1020, an SLM 1040, and an optical system 1030 configured to illuminate at least a portion of the SLM 1040 and project the image of the SLM 1040 onto the user's eyes. System 1000 also includes a coupled optical system 1005, such as a non-imaging optical element (e.g., a cone, a composite parabolic condenser (CPC, lens)), configured to receive light output from the light source 1010. The structure and function of the various optical components of system 1000 depicted in Figure 10 are similar to the structure and function of the corresponding optical components of system 900 depicted in Figure 9 and described above.

[0109] In an embodiment where the light source 1010 consists of three LEDs (e.g., one each for red, green, and blue) forming a Lambertian light source, the coupled optical system 1005 (e.g., CPC) adapts the angular distribution of light produced by the light source 1010 to the needs of the system. However, the coupled optical system 1005 can form a fixed illumination pattern on the SLM 1040, such as the fixed illumination pattern 1200 shown in Figure 12. While the light source 1010 can be controlled to vary the light level in the SLM 1040, the fixed illumination pattern 1200 remains on the SLM 1040 regardless of the light level in the SLM 1040. Furthermore, non-uniformity in various optical components results in a non-uniform illumination pattern 1300 on the SLM 1040, as depicted in Figure 13A. The fixed illumination pattern 1200 and the non-uniform illumination pattern 1300 on the SLM 1040 introduce non-uniformity, distortion, artifacts, and / or aberrations into the image displayed to the user.

[0110] In other embodiments, the light source comprises one or more LED arrays forming a Lambertian light source. Figure 11 illustrates such embodiments of a system (e.g., an AR display system) 1100 for presenting images to a user's eyes and viewing the world, according to several embodiments. System 1100 includes a segmented illumination light source 1110, a waveguide 1120, an SLM 1140, and an optical system 1130 configured to illuminate at least a portion of the SLM 1140 and to project images generated using the SLM 1140 through the waveguide 1120 to the user's eyes. System 1100 also includes a coupled optical system 1105. The structure and function of the various optical components of system 1100 depicted in Figure 11 are similar to the structure and function of the corresponding optical components of system 900 depicted in Figure 9 and described above.

[0111] The segmented illumination source 1110 includes an array of independently addressable / operable LEDs. By controlling the voltage / current applied to each LED / sub-source, the amount of light delivered to the corresponding sections 1142 and waveguides 1120 of the SLM 1140 can be controlled. The amount of light delivered to the various sections 1142 and waveguides 1120 of the SLM 1140 can be controlled to minimize / eliminate non-uniformity, distortion, artifacts, and / or aberrations in the image displayed to the user.

[0112] Non-uniformity correction Figures 13A to 13C illustrate luminance uniformity correction by segmented illumination in several embodiments. Figure 13A illustrates the theoretical non-uniform illumination pattern 1300 in the SLM 1140 and waveguide 1120 (see Figure 11) without non-uniformity correction. The non-uniform illumination pattern 1300 is non-uniform with respect to luminance and has unintended low, medium, and high illumination areas 1302, 1304, and 1306, thereby introducing luminance artifacts into the image presented to the user. Figure 13B illustrates the segmented illumination pattern 1310 generated by the corresponding segmented illumination light source 1110, superimposed on the theoretical non-uniform illumination pattern 1300 in the SLM 1140 and waveguide 1120. The segmented illumination pattern 1310 includes a high-illumination area 1312, a medium-illumination area 1314, and a low-illumination area 1316, which are also shown in Figure 13B. The high, medium, and low illumination areas 1312, 1314, and 1316 in the segmented illumination pattern 1310 correspond to the low, medium, and high-luminance areas in the theoretical non-uniform illumination pattern 1300 on the SLM 1140 and waveguide 1120, respectively. Thus, illuminating the SLM 1140 and waveguide 1120 with the segmented illumination pattern 1310 yields the corrected illumination pattern 1320 depicted in Figure 13C. In the ideal case depicted in Figure 13C, the luminance artifacts in the corrected illumination pattern 1320, the theoretical non-uniform illumination pattern 1300, are eliminated, and the luminance of the light is consistent across the entire surface of the SLM 1140 and waveguide 1120.

[0113] Figures 14A and 14B illustrate color uniformity correction by segmented illumination in several embodiments. Figure 14A illustrates a theoretical non-uniform illumination pattern 1400 in the SLM 1140 and waveguide 1120 (see Figure 11) without optical color uniformity and luminance uniformity correction. The non-uniform illumination pattern 1400 is non-uniform in terms of color and has unintended red, green, and blue illumination areas 1402, 1404, and 1406, thereby introducing color artifacts into the image presented to the user. The non-uniform illumination pattern 1400 is also non-uniform in terms of luminance. A segmented illumination light source 1110 with color control capabilities can generate a segmented illumination pattern (not shown) corresponding to the non-uniform illumination pattern 1400 in the SLM 1140 and waveguide 1120. The segmented illumination pattern includes areas of differential color illumination with complementary colors corresponding to the red, green, and blue illumination areas 1402, 1404, and 1406, respectively, in the theoretical non-uniform illumination pattern 1400 on the SLM 1140 and waveguide 1120. Thus, illuminating the SLM 1140 and waveguide 1120 with the segmented illumination pattern (not shown) yields the corrected illumination pattern (example) 1420 depicted in Figure 14B. In the corrected illumination pattern 1420 depicted in Figure 14B, the color artifacts in the theoretical non-uniform illumination pattern 1400 are eliminated, and the color of the light is consistent across the entire surface of the SLM 1140 and waveguide 1120. The corrected illumination pattern 1420 depicted in Figure 14B contains luminance artifacts, but these luminance artifacts can be corrected by applying a segmented illumination pattern 1310 as shown in Figure 13B.

[0114] Reduced power consumption Figure 15 illustrates a segmented lighting pattern 1500 configured to reduce the power consumption of a display system (e.g., system 1100 depicted in Figure 11) according to several embodiments. The segmented lighting pattern 1500 is superimposed on an image 1510 that is displayed by light from the segmented lighting pattern 1500. The segmented lighting pattern 1500 illuminates only the areas of the SLM 1140 and waveguide 1120 configured to produce a portion of the image 1510. Thus, the segmented lighting pattern 1500 includes an illuminated "on" area 1502 and an unilluminated "off" area 1504. The illuminated "on" area 1502 and the unilluminated "off" area 1504 are generated by the respective illuminated "on" area / part and unilluminated "off" area / part of the segmented lighting source 1110. Including an unilluminated "off" area within the segmented lighting pattern 1500 reduces the power consumption of the display system 1100. The area of ​​the segmented illumination pattern can be generated by controlling the amount of the voltage / current-responsive portion of the segmented illumination light source 1110. In some embodiments, instead of binary illuminated / unilluminated areas, the segmented illumination pattern may include analog grayscale illuminated areas with varying amounts of illumination depending on the displayed image.

[0115] If only a small portion (e.g., 10%) of the segmented illumination source 1110 is illuminated, that portion may be overdriven and emit additional light compared to when a larger portion of the segmented illumination source 1110 is illuminated. In addition, the illuminated portion of the segmented illumination source 1110 may be monitored for overheating issues while overdriven.

[0116] Improved ANSI contrast Figures 16A and 16B illustrate the improvement of contrast by segmented illumination in several embodiments. Figure 16A depicts image 1600 presented by system 1000 (see Figure 10) without segmented illumination. Image 1600 includes a white area 1602 and a black area 1604. Figure 16B depicts image 1610 presented by system 1100 (see Figure 11) with segmented illumination. In such system 1100, the segmented illumination pattern is generated by illuminated "on" areas / parts and unilluminated "off" areas / parts of the segmented illumination light source 1110 corresponding to each of the white area 1612 and black area 1614 of image 1610. Including unilluminated "off" areas within the segmented illumination pattern improves the contrast of image 1610. The areas of the segmented illumination pattern can be generated by controlling the amount of the voltage / current corresponding portion of the segmented illumination light source 1110. In some embodiments, instead of binary illuminated / unilluminated areas, the segmented illumination pattern may include analog grayscale illuminated areas with varying amounts of illumination depending on the displayed image.

[0117] A method for displaying images using segmented lighting. Figure 17 illustrates a method 1710 for displaying an image using segmented illumination, according to several embodiments. In 1712, heterogeneity correction information is acquired as needed (e.g., by a display controller processor). Heterogeneity correction information can be obtained by calibrating / testing the image display system with known image data. Heterogeneity correction information includes, but is not limited to, information about illumination heterogeneity. In 1714, heterogeneity correction information is stored in the memory of the display controller. Heterogeneity correction information may be stored in memory using a lookup table. In 1716, the processor uses the heterogeneity correction information to control the segmented illumination light source to provide differentially illuminated light. The voltage / current of each LED in the segmented illumination light source can be calculated via an algorithm using the heterogeneity correction information. In 1718, the differentially illuminated light from the segmented illumination light source differentially illuminates the first and second parts of the SLM.

[0118] In step 1720, the processor acquires new non-uniformity correction information as needed. This new non-uniformity correction information may arise from changes in the components of the image display system over time during use. Temperature, aging, and other environmental factors can cause changes in various components of the display system, resulting in new non-uniformity correction information. In step 1722, the new non-uniformity correction information is stored in memory as needed. In step 1724, the method proceeds to step 1716, where the new non-uniformity correction information is used to control the segmented illumination light source.

[0119] In some embodiments, the processor may, if necessary, warp the computationally rendered image to account for user motion and latency. This warping can be applied to SLM and segmented illumination. In some embodiments, the processor may, if necessary, synchronize the segmented illumination source with the SLM and control both together field by field. Synchronized control of the SLM gain and segmented illumination gain can achieve the desired light output.

[0120] Exemplary ASIC driver with an oversized bond pad array A segmented illumination source may include one or more application-specific integrated circuits (ASICs), each having a bond pad array configured to control an LED array to which an LED array is coupled. For example, Figure 21A depicts a segmented illumination source 2110 including two ASICs 2120, 2130, according to several embodiments. Each ASIC 2120, 2130 includes a bond pad array 2122, 2132 and an LED array 2124, 2134 coupled to each bond pad array 2122, 2132. Figure 21A shows that a segmented illumination source 2110 having multiple ASICs 2120, 2130 has a minimum size to accommodate multiple ASICs 2120, 2130, including the space required between the multiple ASICs 2120, 2130. This minimum size constraint increases the minimum size of the corresponding display system.

[0121] Figure 21B illustrates a segmented illumination source 2150 including a single ASIC 2160 according to several embodiments. The ASIC 2160 includes a bond pad array having first, second, and third portions 2162A, 2162B, and 2162C, and respective first, second, and third LED arrays 2164, 2174, and 2184 coupled to each portion 2162A, 2162B, and 2162C of the bond pad array. Figure 21B shows that a segmented illumination source 2150 having an oversized bond pad array 2162A, 2162B, and 2162C can accommodate multiple LED arrays 2164, 2174, and 2184. The coupling of multiple LED arrays 2164, 2174, and 2184 to their respective portions 2162A, 2162B, and 2162C of an oversized bond pad array facilitates independent control of the multiple LED arrays 2164, 2174, and 2184 through the oversized bond pad array by the ASIC 2160. The coupling of multiple LED arrays 2164, 2174, and 2184 to a single ASIC reduces the size of the segmented illumination source 2150 and the corresponding display system by at least eliminating the necessary space between ASICs.

[0122] Figure 22 illustrates a segmented illumination source 2250, comprising a single ASIC 2260, according to several embodiments. The ASIC 2260 includes a bond pad array having first, second, and third portions 2262A, 2262B, 2262C, and respective first, second, and third LED arrays 2264, 2274, 2284 coupled to each portion 2262A, 2262B, 2262C of the bond pad array. Figure 22 shows that each portion 2262A, 2262B, 2262C of the bond pad array includes a plurality of positions 2266, 2267, 2268, which can accommodate the respective first, second, and third LED arrays 2264, 2274, 2284. An ASIC including an oversized bond pad having array portions 2262A, 2262B, 2262C, each having multiple positions 2266, 2267, 2268 for bonding LED arrays 2264, 2274, 2284, provides manufacturing flexibility for aligning the LED arrays with other optical components in a display system (e.g., an internal bonding grid). Each of the LED arrays 2264, 2274, 2284 may be configured to emit a specific color (e.g., red, green, blue).

[0123] The amount of movement of the LED array on the bond pad array may be limited by the pixel pitch. Pixel data of the LED array may be mapped to the corresponding bond pad driver on the bond pad array. During normal use, the ASIC may disable unused pixel drivers that are not coupled to the LED array. Unused pixel driver bond pads may be masked.

[0124] Figures 23 and 24 schematically illustrate LED arrays coupled to each bond pad array according to various embodiments. The LED arrays 2314, 2324, 2334, 2414, 2424, and 2434 depicted in Figures 23 and 24 are all 8x8 LED arrays. In other embodiments, the LED arrays may be of different sizes. The first, second, and third portions 2312A, 2412A, 2312B, 2412B, 2312C, and 2412C of the bond pad arrays depicted in Figures 23 and 24 are identical. In fact, the underlying ASICs corresponding to the bond pad arrays 2312A, 2312B, 2312C, 2412A, 2412B, and 2412C depicted in Figures 23 and 24 may be identical. Figures 23 and 24 show that the extra-large bond pad arrays 2312A, 2312B, 2312C, 2412A, 2412B, and 2412C provide flexibility in the placement of LED arrays during manufacturing.

[0125] Figure 23 shows the first, second, and third LED arrays 2314, 2324, and 2334 coupled to the upper right, upper right, and upper left corners of the first, second, and third portions 2312A, 2312B, and 2312C of the bond pad array, respectively. Figure 24 shows the first, second, and third LED arrays 2414, 2424, and 2434 coupled to the central area of ​​the first, second, and third portions 2412A, 2412B, and 2412C of the bond pad array, respectively. Figures 23 and 24 demonstrate the flexibility in mounting LED arrays to an oversized bond pad array, which facilitates the use of the same ASIC for different optical devices with different optical designs. Such flexibility enables the use of a single ASIC design for multiple segmented illumination sources configured for use with different optical devices (see Figures 28–31).

[0126] Figures 25-27 schematically illustrate LED arrays coupled to various bond pad arrays according to different embodiments. The depicted LED arrays 2514, 2614, and 2714 are 8×8 (Figures 25 and 26) and 4×4 (Figure 27) LED arrays. In other embodiments, the LED arrays may be of different sizes. The bond pad arrays 2512, 2612, and 2712 depicted in Figures 25-27 are identical 14×14 bond pad arrays. In fact, the underlying ASICs corresponding to the bond pad arrays 2512, 2612, and 2712 depicted in Figures 23 and 24 may be identical. Figures 25-27 demonstrate that the oversized bond pad arrays 2512, 2612, and 2712 provide flexibility in the placement of LED arrays during manufacturing.

[0127] Figures 28–30 schematically illustrate the use of an ASIC having an oversized bond pad for aligning multiple LED arrays with other optical components during manufacturing. Figure 28 illustrates a display system 2810 according to several embodiments. The display system 2810 includes an optical device 2910 coupled to a controller (not shown) using a connector 2850. As shown in more detail in Figure 29, the optical device 2910 includes a frame 2912 that defines first, second, and third light source apertures 2914A, 2914B, and 2914C therein. The positions of the first, second, and third light source apertures 2914A, 2914B, and 2914C are configured to direct light to other optical components (not shown) of the optical device 2910. In the embodiments depicted in Figures 28 and 29, the first, second, and third LED arrays 2924, 2934, and 2944 are coupled to the first, second, and third portions 2922A, 2922B, and 2922C of the oversized bond pad array of the ASIC. However, the first, second, and third LED arrays 2924, 2934, and 2944 are coupled to the first, second, and third portions 2922A, 2922B, and 2922C such that the first, second, and third LED arrays 2924, 2934, and 2944 do not align with the first, second, and third light source apertures 2914A, 2914B, and 2914C that define them, respectively.

[0128] Figure 30 schematically illustrates an optical device 3010, which includes a frame 2912 defining the first, second, and third light source apertures 2914A, 2914B, and 2914C. The frame 2912 and the first, second, and third light source apertures 2914A, 2914B, and 2914C defined therein are identical to the corresponding components of the optical device 2910 depicted in Figure 29. In the embodiment depicted in Figure 30, the first, second, and third LED arrays 2924', 2934', and 2944' are coupled to the first, second, and third portions of the oversized bond pad array of the ASIC so that the first, second, and third LED arrays 2924', 2934', and 2944' are aligned with the first, second, and third light source apertures 2914A, 2914B, and 2914C that define them, respectively. The optical device 3010 has LED arrays 2924', 2934', and 2944' aligned with the other optical components of the optical device 3010, thereby improving optical efficiency and power consumption / heat generation compared to the optical device 2910 depicted in Figure 29. The optical device 2910 depicted in Figure 29 can be modified during manufacturing by integrating the LED array into different parts of an oversized bond pad array within the same ASIC, in order to arrive at the more optically efficient optical device 3010 depicted in Figure 30.

[0129] Figure 31 illustrates method 3110 for manufacturing different optical devices (e.g., 2910, 3010) having different optical designs using the same ASIC, according to several embodiments.

[0130] In 3112, a first LED array is coupled to a first portion of a first bond pad of a first ASIC to form a first light source. The first ASIC may have an oversized first bond pad array that includes multiple portions (e.g., Figure 22) to which the LED array can be coupled.

[0131] In 3114, a second LED array is coupled to a second portion of the second bond pad of the second ASIC to form a second light source. The second ASIC may have an oversized second bond pad array containing multiple portions (e.g., Figure 22) to which the LED array can be coupled. The first and second ASICs may be identical and may have the same oversized first and second bond pad arrays. Using the same ASIC streamlines the supply chain during the manufacturing of segmented illumination light sources for optical devices. The first and second portions may be different from each other while using the same ASIC to facilitate flexibility in LED array arrangement during manufacturing (e.g., Figures 29 and 30).

[0132] In 3116, the first light source is used together with the first optical device, if necessary. In 3118, the second light source is used together with the second optical device, if necessary. The first and second optical devices have different optical designs (for example, due to different optical components and / or optical functions). Therefore, the same ASIC is used to manufacture different light sources for different optical devices (e.g., 2910, 3010) having different optical designs.

[0133] System Architecture Overview Figure 32 is a block diagram of an exemplary computing system 3200 suitable for implementing one embodiment of the present disclosure. The computer system 3200 includes a processor 3207, system memory 3208 (e.g., RAM), static storage device 3209 (e.g., ROM), disk drive 3210 (e.g., magnetic or optical), communication interface 3214 (e.g., modem or Ethernet® card), display 3211 (e.g., CRT or LCD), input device 3212 (e.g., keyboard), and a bus 3206 or other communication mechanism for communicating information that interconnect subsystems and devices such as cursor control.

[0134] According to one embodiment of the present disclosure, the computer system 3200 performs certain operations by a processor 3207 that executes one or more sequences of one or more instructions contained in system memory 3208. Such instructions may be read into system memory 3208 from another computer-readable / usable medium, such as a static storage device 3209 or a disk drive 3210. In alternative embodiments, hardwired circuits may be used instead of, or in combination with, software instructions to implement the present disclosure. Thus, embodiments of the present disclosure are not limited to any particular combination of hardware circuits and / or software. In one embodiment, the term “logic” means any combination of software or hardware used to implement all or part of the present disclosure.

[0135] As used herein, the terms “computer-readable medium” or “computer-usable medium” refer to any medium involved in providing instructions to the processor 3207 for execution. Such mediums can take many forms, including but not limited to non-volatile and volatile media. Non-volatile media include, for example, optical or magnetic disks such as disk drive 3210. Volatile media include dynamic memory such as system memory 3208.

[0136] Common forms of computer-readable media include, for example, floppy disks, flexible disks, hard disks, magnetic tapes, any other magnetic media, CD-ROMs, any other optical media, punch cards, paper tapes, any other physical media having a pattern of holes, RAM, PROMs, EPROMs, FLASH®-EPROMs (e.g., NAND flash, NOR flash), any other memory chips or cartridges, or any other media that a computer can read.

[0137] In one embodiment of the present disclosure, the execution of a set of instructions for implementing the present disclosure is performed by a single computer system 3200. According to other embodiments of the present disclosure, two or more computer systems 3200 connected by a communication link 3215 (e.g., a LAN, PTSN, or wireless network) may work together to execute the set of instructions necessary to implement the present disclosure.

[0138] The computer system 3200 can transmit and receive messages, data, and instructions, including programs, i.e., application code, through the communication link 3215 and the communication interface 3214. The received program code may be executed by the processor 3207 when it is received, and / or stored in the disk drive 3210 or other non-volatile storage device for later execution. A database 3232 in the storage medium 3231 may be used to store data accessible by the system 3200 via the data interface 3233.

[0139] Although segmented lighting systems and methods are described herein as being implemented in various display systems, the segmented lighting systems and methods described herein may be implemented in various other display systems.

[0140] Specific aspects, advantages, and features of this disclosure are described herein. It should be understood that not all of these advantages are necessarily achieved according to any particular embodiment of this disclosure. Therefore, this disclosure may be embodied or made to achieve or optimize one or more advantages or groups of advantages as taught herein, without necessarily achieving other advantages that may be taught or suggested herein.

[0141] Embodiments are described in reference to the accompanying drawings. However, it should be understood that the drawings are not drawn to scale. Distances, angles, etc., are for illustrative purposes only and do not necessarily have an exact relationship to the actual dimensions and layout of the illustrated devices. In addition, the embodiments described above are described at a detailed level to enable those skilled in the art to manufacture and use the devices, systems, methods, etc. described herein. A wide variety of modifications are possible. Components, elements, and / or steps may be changed, added, removed, or rearranged.

[0142] The devices and methods described herein can, advantageously, be implemented at least partially using, for example, computer software, hardware, firmware, or any combination of software, hardware, and firmware. A software module may include computer executable code stored in the memory of a computer for performing the functions described herein. In some embodiments, the computer executable code is executed by one or more general-purpose computers. However, those skilled in the art will understand that any module that can be implemented using software running on a general-purpose computer can also be implemented using different combinations of hardware, software, or firmware. For example, such a module may be implemented entirely in hardware using a combination of integrated circuits. Alternatively or further, such a module may be implemented entirely or partially using a dedicated computer designed to perform the specific functions described herein, rather than a general-purpose computer. In addition, where a method is described that is performed, or may be performed, at least partially by computer software, it should be understood that such a method may be provided on a non-temporary computer-readable medium that, when read by a computer or other processing device, causes the computer or other processing device to perform the method.

[0143] While specific embodiments have been explicitly described, other embodiments will become apparent to those skilled in the art based on this disclosure.

[0144] The various processors and other electronic components described herein are suitable for use in any optical system for projecting light. The various processors and other electronic components described herein are also suitable for use in any audio system for receiving voice commands.

[0145] Various exemplary embodiments of the Disclosure are described herein. References to these examples are made in a non-limiting sense. They are provided to illustrate broader applicable aspects of the Disclosure. Various modifications may be made to the described Disclosure, and equivalents may be substituted without departing from the true spirit and scope of the Disclosure. In addition, many modifications may be made to adapt specific circumstances, materials, substance compositions, processes, process acts, or steps to the purpose, spirit, or scope of the Disclosure. Furthermore, as will be understood by those skilled in the art, each of the individual modifications described and illustrated herein has individual components and features that can be readily separated or combined with features of any of several other embodiments without departing from the scope or spirit of the Disclosure. All such modifications are intended to fall within the scope of the claims relating to the Disclosure.

[0146] This disclosure includes methods that may be carried out using the subject device. The methods may include the act of providing such a suitable device. Such provision may be performed by an end user. In other words, the act of “providing” simply requires the end user to obtain, access, approach, position, configure, operate, power on, or otherwise act upon the device required in this method. The methods enumerated herein may be performed in any logically possible order of the enumerated events, and in any order of the enumerated events.

[0147] Illustrative aspects of this disclosure, along with details relating to the selection and manufacture of materials, are described above. Further details of this disclosure are understood in connection with the patents and publications referenced above and may be generally known or recognized by those skilled in the art. The same may apply to method-based aspects of this disclosure with respect to additional actions that are generally or logically used.

[0148] In addition, while this disclosure is described with reference to several examples incorporating various features as needed, this disclosure is not limited to those described or shown as intended with respect to each modification of this disclosure. Various modifications may be made to the described disclosure without departing from the spirit and scope of this disclosure, and equivalents (whether listed herein or not included for certain brevity) may be replaced. In addition, where a range of values ​​is provided, it is understood that all intervening values ​​between the upper and lower limits of that range, and any other described or intervening values ​​within that described range, are included in this disclosure.

[0149] Furthermore, any feature of the described variation as necessary is intended to be described and claimed independently or in combination with any one or more of the features described herein. References to singular items include the possibility of multiple instances of the same item. More specifically, as used herein and in the claims associated herein, the singular forms “a,” “an,” “said,” and “the” include multiple referents unless otherwise specified. In other words, the use of articles enables “at least one” of the subject items in the above description, as well as of the claims relating to this disclosure. Furthermore, it should be noted that such claims may be written to exclude elements as necessary. Thus, this statement is intended to serve as an antecedent for the use of exclusive terms such as “alone,” “only,” etc., in relation to the enumeration of elements of the claims or the use of “negative” limitations.

[0150] Without using such exclusive terminology, the term “comprising” in the claims relating to this disclosure shall allow for the inclusion of any additional elements, regardless of whether a given number of elements are enumerated in such claims or whether the addition of features may be considered to transform the nature of the elements described in such claims. Unless specifically defined herein, all technical and scientific terms used herein should be given in a sense that is as broadly and generally understood as possible while maintaining the validity of the claims.

[0151] The scope of this disclosure is not limited to the examples and / or subject matter provided, but rather is limited only by the language of the claims relating to this disclosure.

[0152] The aforementioned specification described the disclosure with reference to specific embodiments thereof. However, it will be apparent that various modifications and changes can be made without departing from the broader spirit and scope of the disclosure. For example, the process flow described above is described with reference to a specific order of process actions. However, many of the orderings of process actions described may be changed without affecting the scope or operation of the disclosure. For this reason, this specification and the drawings should be considered illustrative rather than restrictive.

Claims

1. An extended reality display system, A display subsystem configured to present an image corresponding to image data to a user, wherein the display subsystem is An optical component that introduces heterogeneity into the aforementioned image, Segmented lighting source, A spatial light modulator (SLM) configured to receive light from the segmented illumination light source, A display subsystem comprising, A display controller configured to control the segmented illumination light source, wherein the display controller is A memory for storing non-uniformity correction information, A processor that controls the segmented illumination light source based on the non-uniformity correction information. A display controller equipped with Equipped with, An extended reality display system in which the segmented illumination light source is configured to differentially illuminate the first and second parts of the SLM using the respective first and second parts of the segmented illumination light source.

2. The segmented illumination light source comprises a light-emitting diode (LED) array, The system according to claim 1, wherein the display controller is configured to control each LED of the LED array.

3. The system according to claim 1, wherein the SLM is a liquid crystal on silicon (LCOS) display.

4. The system according to claim 1, further comprising a lens installed between the segmented illumination light source and the SLM.

5. The system according to claim 1, wherein the segmented illumination light source is configured to improve the uniformity of illumination of the image corresponding to the image data by differentially illuminating the first and second portions of the SLM.

6. The system according to claim 5, wherein the uniformity is illumination uniformity.

7. The system according to claim 5, wherein the uniformity is color uniformity.

8. The system according to claim 1, wherein the segmented illumination light source is configured to reduce power consumption by differentially illuminating the first and second portions of the SLM.

9. The system according to claim 1, wherein the segmented illumination light source is configured to improve the contrast of the image corresponding to the image data by differentially illuminating the first and second portions of the SLM.

10. The segmented illumination light source includes a plurality of sub-light sources, The system according to claim 1, wherein the display controller is configured to control the amount of voltage or current applied to one of the sub-light sources among the plurality of sub-light sources.

11. The system according to claim 10, wherein the processor is configured to calculate the amount of current delivered to the sub-light source using the non-uniformity correction information stored in the memory.

12. The system according to claim 1, wherein the display controller is directly coupled to the segmented illumination light source using only a connecting element between the display controller and the segmented illumination light source.

13. The system according to claim 1, wherein the display controller is configured to instruct the segmented illumination light source not to illuminate the third portion of the SLM by leading to a zero voltage or current applied to the third portion of the SLM.

14. The system according to claim 13, wherein the display controller is configured to command the segmented illumination light source to increase the amount of first light and the amount of second light, respectively, generated by the first and second portions of the segmented illumination light source.

15. The system according to claim 14, wherein the display controller is configured to monitor the first and second temperatures of the first and second portions of the segmented illumination light source, respectively.

16. A method for presenting an image corresponding to image data to a user using a display subsystem that includes an optical component that introduces heterogeneity into the image, the method being: The display controller stores non-uniformity correction information in its memory, The processor of the display controller controls the segmented illumination light source of the display subsystem based on the non-uniformity correction information, The segmented illumination light source, under the control of the processor, differentially illuminates the first and second portions of the SLM of the display subsystem using the respective first and second portions of the segmented illumination light source. Methods that include...

17. The segmented illumination light source comprises an LED array, and the method is The method according to claim 16, further comprising the processor of the display controller controlling each LED of the LED array.

18. The method according to claim 16, wherein the SLM is an LCOS display.

19. The method according to claim 16, wherein a lens is installed between the segmented illumination light source and the SLM in the display subsystem.

20. The method according to claim 16, wherein the segmented illumination light source differentially illuminates the first and second portions of the SLM, thereby improving the uniformity of illumination of the image corresponding to the image data.

21. The method according to claim 20, wherein the uniformity is illumination uniformity.

22. The method according to claim 20, wherein the uniformity is color uniformity.

23. The method according to claim 16, wherein the segmented illumination light source differentially illuminates the first and second portions of the SLM, thereby reducing power consumption.

24. The method according to claim 16, wherein the segmented illumination light source differentially illuminates the first and second portions of the SLM, thereby improving the contrast of the image corresponding to the image data.

25. The segmented illumination light source includes a plurality of sub-light sources, and the method is The method according to claim 16, further comprising the processor controlling the amount of voltage or current applied to one of the sub-light sources among the plurality of sub-light sources.

26. The method according to claim 25, further comprising the processor calculating the amount of current delivered to the sub-light source using the non-uniformity correction information stored in the memory.

27. The method according to claim 16, wherein the display controller is directly coupled to the segmented illumination light source using only a connecting element between the display controller and the segmented illumination light source.

28. The method according to claim 16, further comprising the processor instructing the segmented illumination light source not to illuminate the third portion of the SLM by leading to a zero voltage or current applied to the third portion of the SLM.

29. The method according to claim 28, further comprising the processor instructing the segmented illumination light source to increase the amount of first light and the amount of second light, respectively, generated by the first and second portions of the segmented illumination light source.

30. The method according to claim 29, further comprising the processor monitoring the first and second temperatures of the first and second portions of the segmented illumination light source, respectively.

31. The aforementioned processor acquires new non-uniformity correction information, The display controller stores new non-uniformity correction information in the memory, The processor controls the segmented illumination light source of the display subsystem based on the new non-uniformity correction information, The segmented illumination light source, under the control of the processor, differentially illuminates the first and second portions of the SLM of the display subsystem using the respective first and second portions of the segmented illumination light source. The method according to claim 16, further comprising:

32. A segmented illumination light source, Application-specific integrated circuits (ASICs) equipped with bond pad arrays, A first LED array coupled to a first portion of the bond pad array, A second LED array coupled to the second portion of the bond pad array, A third LED array coupled to the third portion of the bond pad array and Equipped with, The first part, the second part, and the third part are separate from each other. The ASIC is configured to control the first LED array, the second LED array, and the third LED array, and is a segmented illumination light source.

33. The first LED array is configured to emit red light, The second LED array is configured to emit green light, The segmented illumination light source according to claim 32, wherein the third LED array is configured to emit blue light.

34. The segmented illumination light source according to claim 32, wherein the first portion of the bond pad array includes a plurality of portions to which the first LED array can be coupled.

35. The segmented illumination light source according to claim 32, wherein the bond pad array includes a plurality of portions to which LED arrays can be coupled.

36. A method for manufacturing multiple segmented illumination light sources, wherein the method is The first LED array is coupled to a first portion of the first bond pad array of a first application-specific integrated circuit (ASIC) to form a first segmented illumination light source, The second LED array is coupled to the second portion of the second bond pad array of the second ASIC to form the first segmented illumination light source. Includes, The first ASIC is identical to the second ASIC. The first portion is positioned at a first location on the first bond pad array, The second portion is positioned at a second location on the second bond pad array, The first position is determined by a method different from that of the second position.

37. The first segmented illumination light source is configured to be used in conjunction with a first optical device having a first optical design, The second segmented illumination light source is configured to be used with a second optical device having a second optical design different from the first optical design. The aforementioned method, Determining the first portion of the first bond pad array of the first ASIC based on the first optical design of the first optical device, Determining the second portion of the second bond pad array of the second ASIC based on the second optical design of the second optical device and The method according to claim 36, further comprising:

Images