Augmented and mixed reality systems and methods

Abstract

A head-mounted display system configured to be worn over eyes of a user includes a display disposed over the eyes of the user and configured to emit light into the eyes of the user, where the display includes a dimmer configured to modify a transparency of the display. The system also includes a light sensor separated from an external environment of the user by the display and configured to measure a transmitted light intensity of light from the external environment of the user transmitted through the dimmer. The system further includes a processor operatively coupled to the display and the light sensor, and configured to control an emitted light intensity of the light emitted by the display based on the transmitted light intensity.

Description

AUGMENTED AND MIXED REALITY SYSTEMS AND METHODSCross-Reference to Related Applications

[0001] This application claims the benefit of priority to U.S. Provisional Patent Application Serial No. 63 / 572,129, entitled “AUGMENTED AND MIXED REALITY SYSTEMS AND METHODS,” and filed on March 29, 2024. The present application is also related to U.S. Utility Patent Application Serial No. 16 / 215,477 filed on December 10, 2018 under attorney docket number 101782-013710US-1347331 and entitled “WAVEGUIDE ILLUMINATOR,” U.S. Utility Patent Application Serial No. 17 / 261 ,854 filed on January 20, 2021 under attorney docket number ML-0676US and entitled “SYSTEMS AND METHODS FOR EXTERNAL LIGHT MANAGEMENT,” U.S. Utility Patent Application Serial No. 17 / 275,663 filed on March 11 , 2021 under attorney docket number ML-0783US and entitled “SYSTEMS AND METHODS FOR EXTERNAL LIGHT MANAGEMENT,” and U.S. Utility Patent Application Serial No. 17 / 096,557 filed on November 12, 2020 under attorney docket number ML-0978US and entitled “AMBIENT LIGHT MANAGEMENT SYSTEMS AND METHODS FOR WEARABLE DEVICES.” The contents of the aforementioned patent applications are hereby expressly and fully incorporated by reference in their entirety, as though set forth in full. Described in the aforementioned incorporated patent applications are various embodiments of augmented and mixed reality systems and methods. Described herein are further embodiments of augmented and mixed reality systems and methods.Copyright Notice

[0002] A portion of the disclosure of this patent document contains material that is subject to copyright protection. The copyright owner has no objection to the facsimile reproduction by anyone of the patent document or the patent disclosure as it appears in the Patent and Trademark Office patent file or records, but otherwise reserves all copyright rights whatsoever.Field of the Invention

[0003] The present disclosure relates to augmented reality and mixed reality imaging, visualization, and display systems and methods. In particular, the present disclosure relates to augmented reality and mixed reality imaging, visualization, and display systems and methods for controlling a display light intensity based on a transmitted light intensity.Background

[0004] Modern computing and display technologies have facilitated the development of virtual reality (VR), augmented reality (AR), and mixed reality (MR) systems. VR systems create a simulated environment for a user to experience. This can be done by presenting computer-generated imagery to the user through a head-mounted display. This imagery creates a sensory experience which immerses the user in the simulated environment. A VR scenario typically involves presentation of only computer-generated imagery rather than also including actual real-world imagery.

[0005] AR systems generally supplement a real-world environment with simulated elements. For example, AR systems may provide a user with a view of the surrounding real-world environment via a head-mounted display. However, computer-generatedimagery can also be presented on the display to enhance the real-world environment. This computer-generated imagery can include elements which are contextually-related to the real-world environment. Such elements can include simulated text, images, objects, etc. MR systems also introduce simulated objects into a real-world environment, but these objects typically feature a greater degree of interactivity than in AR systems. The simulated elements can often times be interactive in real time.

[0006] Figure 1 depicts an example AR / MR scene 2 where a user sees a real-world park setting 6 featuring people, trees, buildings in the background, and a concrete platform 20. In addition to these items, computer-generated imagery is also presented to the user. The computer-generated imagery can include, for example, a robot statue 10 standing upon the real-world platform 20, and a cartoon-like avatar character 12 flying by which seems to be a personification of a bumble bee, even though these elements 12, 10 are not actually present in the real-world environment.

[0007] Various optical systems generate images at various depths for displaying VR, AR, or MR scenarios. Some such optical systems are described in U.S. Utility Patent Application Serial No. 14 / 555,585 filed on November 27, 2014 (under attorney docket number ML.20011 .00), the contents of which are hereby expressly and fully incorporated by reference in their entirety, as though set forth in full. Other such optical systems for displaying MR experiences are described in U.S. Utility Patent Application Serial No. 14 / 738,877 filed on June 13, 2015 (under attorney docket number ML.20019.00), the contents of which are hereby expressly and fully incorporated by reference in their entirety, as though set forth in full.

[0008] Human vision is dominated by perception of contrast. In some AR / MR devices, virtual light used to display virtual objects competes with real-world light as both emit from a “combiner” to produce a mixed image. The brightness / intensity of virtual light is preferably properly balanced with the (highly variable) brightness / intensity of real-world light to produce appropriate contrast. It is desirable to modulate one or both to achieve proper balance between the two, such that the user simultaneously perceives both with acceptable acuity.

[0009] Real-world light brightness / intensity is variable based on changes in the external environment (e.g., clouds and other obscuring objects). Virtual light brightness / intensity is also variable (e.g., modulated by the AR / MR device within its capabilities). A particular balance between the two is desirable and results in appropriate “image solidity”. The virtual light should outweigh the real-world light if the virtual image / object is to be perceived as “solid”.

[0010] A dimming element / dimmer may be incorporated in the light path of the AR / MR device before the combiner to attenuate real-world light before it mixes with the virtual light. In some embodiments, the dimming element / dimmer may be a photochromic dimmer, which is passive in nature and cannot be actively controlled. In such embodiments, it is preferable for the device to modulate virtual light brightness / intensity to match the real-world light brightness / intensity from the external environment, which is subject to dimming from the photochromic dimmer.

[0011] Existing AR / MR systems can provide a user with a view of the surrounding real- world external environment through a head-mounted display while modifying a brightness / intensity of light used to display virtual objects based on real-world lighttransmitted through the head-mounted display where net device transmission is known (e.g., with no dimming) or controlled (e.g., with an actively controlled dimmer). However, existing AR / MR systems cannot accurately modify a brightness / intensity of light used to display virtual objects based on real-world light transmitted through head-mounted displays that include passive dimmers, such as photochromic dimmers with variable dimming / transmission based on ambient ultraviolet light levels.

[0012] In some embodiments, a dimming element / dimmer is configured to modify a brightness / intensity of light transmitted from an external environment of the AR / MR display system to a user, so that a matching virtual content brightness can be achieved with lower intensity of the AR / MR display. Reducing the intensity / brightness of the AR / MR display system reduces power consumption and heat generation among other system benefits. While these features improve the perceived reality / solidity of displayed virtual images / objects, the intensity / brightness of transmitted light from the real-world external environment must be matched with the intensity / brightness of virtual light used display virtual images / objects.

[0013] A current solution to the problem of matching transmitted light and virtual light involves detecting real-world light from the real-world external environment using a light sensor. However, in some embodiments, AR / MR display systems include dimming element / dimmers resulting in a mismatch between detected real-world light intensity / brightness and transmitted light intensity / brightness.

[0014] Improved systems and techniques are needed for matching transmitted light and virtual light. The systems and methods described herein are configured to address these and other challenges. What is needed is systems and techniques to improve overlegacy systems and techniques, and / or over other considered approaches. Some of the approaches described in this background section are approaches that could be pursued, but not necessarily approaches that have been previously conceived or pursued.Summary

[0015] In one embodiment, a head-mounted display system configured to be worn over eyes of a user includes a display disposed over the eyes of the user and configured to emit light into the eyes of the user, where the display includes a dimmer configured to modify a transparency of the display. The system also includes a light sensor separated from an external environment of the user by the display and configured to measure a transmitted light intensity of light from the external environment of the user transmitted through the dimmer. The system further includes a processor operatively coupled to the display and the light sensor, and configured to control an emitted light intensity of the light emitted by the display based on the transmitted light intensity.

[0016] In one or more embodiments, the display is configured to display virtual content to the user. The dimmer may be configured to variably / adjustably modify the transparency of the display. The dimmer may be configured to modify the transparency of the display in response to ultraviolet light in the external environment of the user. The dimmer may include a photochromic film. The dimmer may be configured to modify the transparency of the display in response to visible light in the external environment of the user.

[0017] In another embodiment, an image display system includes a head-mounted display system configured to be worn over eyes of a user. The head-mounted displaysystem includes a display disposed over the eyes of the user and configured to emit light into the eyes of the user. The display includes a display dimmer configured to modify a transparency of the display such that a display transmitted light intensity of light from a display external environment is transmitted through the dimmer. The head-mounted display system also includes a first light sensor assembly disposed in a first external environment apart from the head-mounted display system. The first light sensor assembly includes a first light sensor. The first light sensor assembly also includes a first sensor dimmer separating the first light sensor from the first external environment of the first light sensor. The first light sensor is configured to measure a first sensor transmitted light intensity of light from the first external environment of the first light sensor transmitted through the first sensor dimmer into the first light sensor. The head-mounted display system further includes a processor. When the user and the head-mounted display system are in the first external environment, the processor is operatively coupled to the display and the first light sensor assembly, and the processor is configured to control an emitted light intensity of the light emitted by the display based on the first sensor transmitted light intensity configured to control an emitted light intensity of the light emitted by the display based on the first sensor transmitted light intensity.

[0018] In one or more embodiments, when the user and the head-mounted display system are in the first external environment, the display transmitted light intensity is equal to the first sensor transmitted light intensity. The display may be configured to display virtual content to the user. The display dimmer may be configured to variably / adjustably modify the transparency of the display. The display dimmer may be configured to modify the transparency of the display in response to ultraviolet light in the display externalenvironment of the first light sensor. The display dimmer may include a photochromic film. The display dimmer may be configured to modify the transparency of the display in response to visible light in the display external environment of the first light sensor.

[0019] In one or more embodiments, the first sensor dimmer is configured to variably / adjustably modify the emitted light intensity of the light from the first external environment of the first sensor transmitted through the first sensor dimmer into the first light sensor. The first sensor dimmer may be configured to modify the emitted light intensity in response to ultraviolet light in the first external environment of the first light sensor. The first sensor dimmer may include a photochromic film. The first sensor dimmer may be configured to modify the emitted light intensity in response to visible light in the environment of the user. The first light sensor assembly may be wirelessly coupled to the processor.

[0020] In one or more embodiments, the image display system also includes a second light sensor assembly disposed in a second external environment apart from the headmounted display system and the first external environment. The second light sensor assembly includes a second light sensor. The second light sensor assembly may also include a second sensor dimmer separating the second light sensor from the second external environment of the second light sensor. The second light sensor may be configured to measure a second sensor transmitted light intensity of light from the second external environment of the second light sensor transmitted through the second sensor dimmer into the second light sensor. When the user and the head-mounted display system are in the second external environment, the processor may be operatively coupledto second light sensor assembly and configured to control the emitted light intensity of the light emitted by the display based on the second sensor transmitted light intensity.

[0021] In one or more embodiments, when the user and the head-mounted display system are in the second external environment, the display transmitted light intensity is equal to the second sensor transmitted light intensity. The second sensor dimmer may be configured to variably / adjustably modify the emitted light intensity of the light from the second external environment of the second sensor transmitted through the second sensor dimmer into the second light sensor. The second sensor dimmer may be configured to modify the emitted light intensity in response to ultraviolet light in the second external environment of the second light sensor. The first sensor dimmer may include a photochromic film. The second sensor dimmer may be configured to modify the emitted light intensity in response to visible light in the environment of the user. When the user and the head-mounted display system are in the second external environment, the second light sensor assembly may be wirelessly coupled to the processor.

[0022] In still another embodiment, a method for controlling emitted light intensity of light emitted by a head-mounted display comprising a dimmer and configured to be worn over eyes of the user includes a light sensor separated from an external environment by the display measuring a transmitted light intensity of light from the external environment transmitted through display. The method also includes a processor operatively coupled to the display and the light sensor controlling an emitted light intensity of light emitted by the display based on the transmitted light intensity.

[0023] In one or more embodiments, the method also includes the light sensor communicating the transmitted light intensity to the processor.

[0024] In yet another embodiment, a method for controlling emitted light intensity of light emitted by a head-mounted display comprising a dimmer and configured to be worn over eyes of the user includes when the user and the display are located in a first external environment, a first light sensor separated from the first external environment by a first sensor dimmer measuring a first sensor transmitted light intensity of light from the first external environment transmitted through first sensor dimmer. The method also includes when the user and the display are located in the first external environment, a processor operatively coupled to the display and the first light sensor controlling a first emitted light intensity of light emitted by the display based on the first sensor transmitted light intensity.

[0025] In one or more embodiments, the method also includes when the user and the display are located in a first external environment, the first light sensor communicating the first sensor transmitted light intensity to the processor. When the user and the headmounted display system are in the first external environment, a display transmitted light intensity may be equal to the first sensor transmitted light intensity.

[0026] In one or more embodiments, the method also includes the processor determining that the user and the display have moved into a second external environment. When the user and the display are located in the second external environment, the method includes a second light sensor separated from the second external environment by a second sensor dimmer measuring a second sensor transmitted light intensity of light from the second external environment transmitted through second sensor dimmer, wherein the processor operatively coupled to the second light sensor. When the user and the display are located in the second external environment, the method includes the processor control controlling a second emitted light intensity of lightemitted by the display based on the second sensor transmitted light intensity. The method may also include when the user and the head-mounted display system are in the second external environment, the second light sensor communicating the second sensor transmitted light intensity to the processor. When the user and the head-mounted display system are in the second external environment, the display transmitted light intensity may be equal to the second sensor transmitted light intensity.Brief Description of the Drawings

[0027] The drawings described below are for illustration purposes only. The drawings are not intended to limit the scope of the present disclosure. The drawings illustrate the design and utility of various embodiments of the present disclosure. It should be noted that the figures are not drawn to scale and that elements of similar structures or functions are represented by like reference numerals throughout the figures. In order to better appreciate how to obtain the recited and other advantages and objects of various embodiments of the disclosure, a more detailed description of the present disclosure will be rendered by reference to specific embodiments thereof, which are illustrated in the accompanying drawings. Understanding that these drawings depict only typical embodiments of the disclosure and are not therefore to be considered limiting of its scope, the disclosure will be described and explained with additional specificity and detail through the use of the accompanying drawings.

[0028] Figure 1 illustrates a user’s view of an AR / MR scene using an example AR / MR system.

[0029] Figures 2-5 schematically depict users using AR / MR systems according to some embodiments.

[0030] Figure 6 schematically depicts a user using an AR / MR system according to some embodiments.

[0031] Figure 7 schematically depicts users wearing head-mounted AR / MR systems including passive photochromic dimmers according to some embodiments.

[0032] Figure 8 schematically depicts a user using an AR / MR system according to some embodiments.

[0033] Figure 9 and 10 schematically depicts a user using an AR / MR system according to some other embodiments.

[0034] Figure 11 schematically depicts a user using an AR / MR system according to some other embodiments.

[0035] Figures 12 and 13 schematically depict a user using an AR / MR system according to some embodiments.

[0036] Figures 14 to 17 are flowcharts depicting methods for controlling light intensity of light emitted by a head-mounted display comprising a dimmer according to some embodiments.

[0037] Figure 18 is a block diagram schematically depicting an illustrative computing system according to some embodiments.Detailed Description

[0038] Various embodiments of the disclosure are directed to systems, methods, and articles of manufacture for augmented reality (AR) and / or mixed reality (MR) in a singleembodiment or in multiple embodiments. Other objects, features, and advantages of the disclosure are described in the detailed description, figures, and claims.

[0039] Various embodiments will now be described in detail with reference to the drawings, which are provided as illustrative examples so as to enable those skilled in the art to practice the disclosure. Notably, the figures and the examples below are not meant to limit the scope of the present disclosure. Where certain elements of the present disclosure may be partially or fully implemented using known components (or methods or processes), only those portions of such known components (or methods or processes) that are necessary for an understanding of the present disclosure will be described, and the detailed descriptions of other portions of such known components (or methods or processes) will be omitted so as not to obscure the disclosure. Further, various embodiments encompass present and future known equivalents to the components referred to herein by way of illustration.

[0040] Embodiments in accordance with the present disclosure address the problem of implementation of AR / MR systems often rely on combinations of off-the-shelf- components and custom components. In some cases the off-the-shelf components do not possess all of the features or performance characteristics that are needed to implement certain desired aspects of the to-be-deployed AR / MR system. Some embodiments are directed to approaches for adding capabilities and / or repurposing resources to accommodate the desired features or performance characteristics of the to- be-deployed AR / MR system. The accompanying figures and discussions herein present example environments, systems, methods, and computer program products for AR / MR systems.

[0041] The head-mounted display systems described herein may be implemented independently of AR / MR systems, but some embodiments below are described in relation to AR / MR systems for illustrative purposes only.Summary of Problems and Solutions

[0042] As described above, a dimming element / dimmer may be configured to modify a brightness / intensity of light transmitted from an external environment of the AR / MR display system to a user to improve user comfort and AR / MR experience quality while reducing power consumption and heat generation. While these features improve the perceived reality / solidity of displayed virtual images / objects, the intensity / brightness of transmitted light from the real-world external environment must be matched with the intensity / brightness of virtual light used display virtual images / objects.

[0043] A current solution to the problem of matching transmitted light and virtual light involves detecting real-world light from the real-world external environment using a light sensor. However, in embodiments of AR / MR display systems including dimming element / dimmers using real-world light intensity may result in a mismatch between detected real-world light intensity / brightness and transmitted light intensity / brightness.

[0044] The embodiments described herein include head-mounted display systems having light sensors including and / or disposed adjacent to dimming elements / dimmers. Various embodiments of head-mounted display systems also include a processor configured to control an intensity / brightness of the display based on transmitted light intensity / brightness detected by the light sensor. Such head-mounted display systems improve user comfort by dimming light from the external environment while matchingvirtual light intensity / brightness to improve AR / MR experience quality, thereby addressing many of the above described issues.Illustrative Head-Mounted AR and / or MR Systems with Dimming

[0045] The description that follows pertains to illustrative head-mounted AR and / or MR systems with which embodiments of various virtual image display systems may be practiced. However, it is to be understood that the embodiments also lends themselves to applications in other types of head-mounted display systems (including other types of AR and / or MR systems), and therefore the embodiments are not to be limited to only the illustrative systems disclosed herein.

[0046] AR / MR systems disclosed herein can include a head-mounted display which presents computer-generated (i.e., virtual) imagery (video / image data) to a viewer / wearer / user 50. The head-mounted display systems are wearable, which may advantageously provide a more realistic and immersive AR / MR experience. Various components of head-mounted AR and / or MR virtual image systems 100 are depicted in Figures 2 to 6. The virtual image display system 100 includes a frame structure 102 worn by an end user 50, a display subsystem 110 carried by the frame structure 102, such that the display subsystem 110 is positioned in front of the eyes of the end user 50, and a speaker 106 carried by the frame structure 102, such that the speaker 106 is positioned adjacent the ear canal of the end user 50 (optionally, another speaker (not shown) is positioned adjacent the other ear canal of the end user 50 to provide for stereo / shapeable sound control). The display subsystem 110 is designed to present the eyes of the end user 50 with light patterns that can be comfortably perceived as augmentations to physical reality, with high-levels of image quality and three-dimensional perception, as well asbeing capable of presenting two-dimensional content. The display subsystem 110 presents a sequence of frames at high frequency that provides the perception of a single coherent scene.

[0047] In the illustrated embodiments, the display subsystem 110 employs “optical see-through” display through which the user can directly view transmitted light from real objects via transparent or semi-transparent elements. The transparent / semi-transparent element, often referred to as a “combiner,” superimposes light from the display over the user’s view of the real world. To this end, the display subsystem 110 may include a partially transparent display. In some embodiments, the partially transparent display may be electronically controlled. In some embodiments, the partially transparent display may include dimming to modify the opacity of the partially transparent display or one or more portions thereof. In some embodiments, the partially transparent display may include global dimming to control opacity of the entirety of the partially transparent display. The partially transparent display is positioned in the end user’s 50 field of view between the eyes of the end user 50 and an ambient real-world external environment, such that direct light from the ambient real-world external environment is transmitted through the partially transparent display to the eyes of the end user 50.

[0048] In the illustrated embodiments, an image projection assembly provides light to the partially transparent display, thereby combining with the direct light from the ambient real-world external environment, and being transmitted from the partially transparent display to the eyes of the user 50. The projection subsystem may include various light sources and spatial light modulators, and the partially transparent display may be a waveguide-based display into which the light from the projection subsystem is injected toproduce, e.g., images at a single optical viewing distance closer than infinity (e.g., arm’s length), images at multiple, discrete optical viewing distances or focal planes, and / or image layers stacked at multiple viewing distances or focal planes to represent volumetric 3D objects. These layers in the light field may be stacked closely enough together to appear continuous to the human visual system (i.e., one layer is within the cone of confusion of an adjacent layer). Additionally or alternatively, picture elements (i.e., subimages) may be blended across two or more layers to increase perceived continuity of transition between layers in the light field, even if those layers are more sparsely stacked (i.e., one layer is outside the cone of confusion of an adjacent layer). The display subsystem 110 may be monocular or binocular.

[0049] The virtual image display system 100 may also include one or more sensors (not shown) mounted to the frame structure 102 for detecting the position and movement of the head 54 of the end user 50 and / or the eye position and inter-ocular distance of the end user 50. Such sensors may include image capture devices (such as cameras), microphones, inertial measurement units, accelerometers, compasses, GPS units, radio devices, and / or gyros). Many of these sensors operate on the assumption that the frame 102 on which they are affixed is in turn substantially fixed to the user’s head, eyes, and ears.

[0050] The virtual image display system 100 may further include outwardly-facing cameras 660 to capture images of a field-of-view of a user. The virtual image display system 100 may also include one or more dimmers 692 (e.g., passive photochromic dimmers or active LCD dimmers), which together with the respective LOEs 690, form display subsystems 110.

[0051] The virtual image display system 100 may also include a user orientation detection module. The user orientation module detects the instantaneous position of the head 54 of the end user 50 (e.g., via sensors coupled to the frame 102) and may predict the position of the head 54 of the end user 50 based on position data received from the sensors. Detecting the instantaneous position of the head 54 of the end user 50 facilitates determination of the specific actual object that the end user 50 is looking at, thereby providing an indication of the specific virtual object to be generated in relation to that actual object and further providing an indication of the position in which the virtual object is to be displayed. The user orientation module may also track the eyes of the end user 50 based on the tracking data received from the sensors.

[0052] The virtual image display system 100 may also include a control subsystem that may take any of a large variety of forms. The control subsystem includes a number of controllers, for instance one or more microcontrollers, microprocessors or central processing units (CPUs), digital signal processors, graphics processing units (GPUs), other integrated circuit controllers, such as application specific integrated circuits (ASICs), display bridge chips, display controllers, programmable gate arrays (PGAs), for instance field PGAs (FPGAs), and / or programmable logic controllers (PLUs).

[0053] The control subsystem of virtual image display system 100 may include a central processing unit (CPU), a graphics processing unit (GPU), one or more frame buffers, and a three-dimensional database for storing three-dimensional scene data. The CPU may control overall operation, while the GPU may render frames (i.e., translating a three-dimensional scene into a two-dimensional image) from the three-dimensional data stored in the three-dimensional database and store these frames in the frame buffers.One or more additional integrated circuits may control the reading into and / or reading out of frames from the frame buffers and operation of the image projection assembly of the display subsystem 110.

[0054] The various processing components of the virtual image display system 100 may be physically contained in a distributed subsystem. For example, as illustrated in Figures 2 to 5, the virtual image display system 100 may include a local processing and data module 130 operatively coupled, such as by a wired lead or wireless connectivity 136, to a local display bridge 142, the display subsystem 110, and sensors. The local processing and data module 130 may be mounted in a variety of configurations, such as fixedly attached to the frame structure 102 (Figure 2), fixedly attached to a helmet or hat 56 (Figure 3), removably attached to the torso 58 of the end user 50 (Figure 4), or removably attached to the hip 60 of the end user 50 in a belt-coupling style configuration (Figure 5). The virtual image display system 100 may also include a remote processing module 132 and remote data repository 134 operatively coupled, such as by a wired lead or wireless connectivity 138, 140, to the local processing and data module 130 and the local display bridge 142, such that these remote modules 132, 134 are operatively coupled to each other and available as resources to the local processing and data module 130 and the local display bridge 142.

[0055] The local processing and data module 130 and the local display bridge 142 may each include a power-efficient processor or controller, as well as digital memory, such as flash memory, both of which may be utilized to assist in the processing, caching, and storage of data captured from the sensors and / or acquired and / or processed using the remote processing module 132 and / or remote data repository 134, possibly forpassage to the display subsystem 110 after such processing or retrieval. The remote processing module 132 may include one or more relatively powerful processors or controllers configured to analyze and process data and / or image information. The remote data repository 134 may include a relatively large-scale digital data storage facility, which may be available through the internet or other networking configuration in a “cloud” resource configuration. In some embodiments, all data is stored and all computation is performed in the local processing and data module 130 and the local display bridge 142, allowing fully autonomous use from any remote modules.

[0056] The couplings 136, 138, 140 between the various components described above may include one or more wired interfaces or ports for providing wires or optical communications, or one or more wireless interfaces or ports, such as via RF, microwave, and IR for providing wireless communications. In some implementations, all communications may be wired, while in other implementations all communications may be wireless. In still further implementations, the choice of wired and wireless communications may be different from that illustrated in Figures 2 to 5. Thus, the particular choice of wired or wireless communications should not be considered limiting.

[0057] In some embodiments, the user orientation module is contained in the local processing and data module 130 and / or the local display bridge 142, while CPU and GPU are contained in the remote processing module. In alternative embodiments, the CPU, GPU, or portions thereof may be contained in the local processing and data module 130 and / or the local display bridge 142. The 3D database can be associated with the remote data repository 134 or disposed locally.

[0058] Figure 6 schematically depicts an AR or MR system 600 (hereinafter referred to as “system 600”) according to some embodiments. The system 600 uses a light guiding optical element (hereinafter referred to as “LOE 690”). The system 600 generally includes one or more processors 610, one or more light sources 620, one or more controller / display bridges (DB) 630, one or more spatial light modulators (SLM) 640, and one or more LOE 690 that function as a multiple plane focus system. The system 600 may also include an eye-tracking subsystem 650.

[0059] The processor 610 is configured to generate virtual content to be displayed to the user. The processor 610 may convert an image or video associated with the virtual content to a format that can be projected to the user in 3D. For example, in generating 3D content, the virtual content may need to be formatted such that portions of a particular image are displayed at a particular depth plane while others are displayed at other depth planes. In one embodiment, all of the image may be generated at a particular depth plane. In another embodiment, the processor 610 may be programmed to provide slightly different images to the right and left eyes such that when viewed together, the virtual content appears coherent and comfortable to the user’s eyes.

[0060] The processor 610 may further include a memory 612, a GPU 614, a CPU 616, and other circuitry for image processing. The processor 610 may be programmed with the desired virtual content to be presented to the user of the system 600. It should be appreciated that in some embodiments, the processor 610 may be housed in the system 600. In other embodiments, the processor 610 and other circuitry may be housed in a belt pack that is coupled to the system 600. In some embodiments, the processor 610, or one or more components thereof, may be a part of a local processing and data module(e.g., local processing and data module 130). As mentioned above, the local processing and data module 130 may be mounted in a variety of configurations, such as fixedly attached to the frame structure 102 (Figure 2), fixedly attached to a helmet or hat 56 (Figure 3), removably attached to the torso 58 of the end user 50 (Figure 4), or removably attached to the hip 60 of the end user 50 in a belt-coupling style configuration (Figure 5).

[0061] The processor 610 is operatively coupled to the light source 620 which projects light associated with the desired virtual content and one or more spatial light modulators 640. The light source 620 is compact and has high resolution. The light source 620 is operatively coupled to a controller / DB 630. The light source 620 may include color specific LEDs and lasers disposed in various geometric configurations. Alternatively, the light source 620 may include LEDs or lasers of like color, each one linked to a specific region of the field of view of the display. In another embodiment, the light source 620 may include a broad-area emitter such as an incandescent or fluorescent lamp with a mask overlay for segmentation of emission areas and positions. Although the light source 620 is directly connected to the system 600 in Figure 6, the light source 620 may be connected to the system 600 via optical fibers (not shown). The system 600 may also include condenser (not shown) configured to collimate the light from the light source 620.

[0062] The SLM 640 may be reflective (e.g., a liquid crystal on silicon (LCOS), a ferroelectric liquid crystal on silicon (FLCOS), a digital light processing digital micro-mirror device (DLP DMD), or a micro electro mechanical system (MEMS) mirror system), transmissive (e.g., a liquid crystal display (LCD)) or emissive (e.g. a fiber scanning display (FSD) or an organic light emitting diode (OLED)) in various exemplary embodiments. The type of SLM 640 (e.g., speed, size, etc.) can be selected to improve the creation of the3D perception. While DLP DMDs operating at higher refresh rates may be easily incorporated into stationary systems 600, wearable systems 600 may use DLPs of smaller size and power. The power of the DLP changes how 3D depth planes / focal planes are created. The processor 610 is operatively coupled to the SLM 640, which encodes the light from the light source 620 with the desired virtual content. Light from the light source 620 may be encoded with the image information when it reflects off of, emits from, or passes through the SLM 640.

[0063] Light from the SLM 640 is directed to the LOE 690 such that light beams encoded with image data for one depth plane and / or color by the SLM 640 are effectively propagated along the LOE 690 for delivery to an eye of a user. The light source 620 and LOE 690 can selectively project images (synchronously encoded by the SLM 640 under the control of controller / DB 630) that appear to originate from various depth planes or positions in space. Accordingly, the system 600 can generate a 3D image of virtual objects at various depth planes that appear to exist simultaneously in the 3D image.

[0064] The controller / DB 630 is in communication with and operatively coupled to the processor 610, the light source 620 and the SLM 640 to coordinate the synchronous display of images by instructing the SLM 640 to encode the light beams from the light source 620 with appropriate image information from the processor 610. While the system 600 includes a processor 610, the controller / DB 630, in some embodiments, may also perform at least some image processing including, for example, the processes of the memory 612, the GPU 614, and / or the CPU 616. In some embodiments, the controller / DB 630 may include one or more components shown in the processor 610, such as, for example, the memory 612, the GPU 614, and / or the CPU 616.

[0065] The system 600 also includes a pair of light sensors 680 position and configured to detect light transmitted from an external environment through the dimmer 692. The light sensor 680 may be a photoresistor, a photodiode, a phototransistor, or any other type of known light sensor.

[0066] The processor 610 may be configured to modify / adjust / control an intensity / brightness of the light emitted by the LOE 690 based on an intensity / brightness of light transmitted from an external environment of the user through the dimmer 692 and detected by the light sensor 680. In some embodiments, the processor 610 is configured to control an intensity of the light emitted by the LOE 690 such that the emitted intensity is greater than the transmitted intensity of the light from the external environment. Achieving this balance between the virtual light and real-world light is desirable and results in appropriate “image solidity” such that displayed virtual images objects are to perceived as “solid”.

[0067] The system 600 may also include an optional eye-tracking subsystem 650 that is configured to track the user’s eyes to determine respective lines of sight of the users eyes and / or the user’s point of vergence. In one embodiment, the system 600 is configured to display virtual content at a particular depth through the LOE 690, based on input from the eye-tracking subsystem 650 such that the image is generated at a desired depth plane that coincides with the user’s point of vergence. For example, if the user’s eyes are parallel to each other, the system 600 may display virtual content through the LOE 690 that is configured to deliver collimated light to the user’s eyes, such that the image appears to originate from optical infinity. In another example, if the eye-trackingsubsystem 650 determines that the user’s point of vergence is at 1 meter away, the LOE 690 that is configured to focus approximately within that range may be illuminated instead.

[0068] Figure 7 schematically depicts users wearing head-mounted AR / MR systems 700, 700’ including passive photochromic dimmers according to some embodiments. The passive photochromic dimmers in the head-mounted AR / MR systems 700, 700’ respond to the intensity of ultraviolet light in the external environment of the user by increasing opacity with increased ultraviolet light intensity (e.g., from indoor to cloudy to sunny). Such passive photochromic dimmers increase user comfort by dimming light from the external environment upon transition from indoors to outdoors. However, passive photochromic dimmers have a relatively long response time, sometimes taking more than a minute to increase opacity in response to increased ultraviolet light intensity as shown in Figure 7. As discussed above, matching virtual light intensity to transmitted light intensity would improve AR / MR experience quality. To match the virtual light intensity, the head-mounted AR / MR systems 700, 700’ also includes respective light sensors 682 in respective outer corners of the head-mounted AR / MR systems 700, 700’ and behind the respective dimmers to detect the transmitted light intensity.

[0069] Figure 8 schematically depicts a user using an AR / MR system 600 according to some embodiments. The AR / MR system 600 schematically depicted in detail in Figure 8 is similar to the AR / MR system 600 generally depicted in Figure 6, and similar elements are identified with similar reference numbers. The AR / MR system 600 includes, inter alia, an LOE / display 690, a dimmer 692, a light sensor 682, and a processor 610 operatively coupled to the LOE 690 and the light sensor 682. The dimmer 692 may be a passive dimmer (e.g., a photochromic dimmer). While photochromic dimmer responsive toultraviolet light intensity, the dimmer 692 may be responsive to visible light intensity. In some embodiments, the dimmer 692 may be a photochromic film that forms a part of the LOE / display 690. Because the dimmer 692 is larger than the LOE / display 690, the sensor can be placed behind the dimmer 692 without interfering with the LOE / display 690 (e.g., in a corner of the AR / MR system 700 as shown in Figure 7).

[0070] External light 810 from an external environment of the user and the AR / MR system 600 pass through the dimmer 692 as transmitted light 820, which has reduced intensity compared to the external light 810. The LOE / display 690 has a negligible effect on the transmitted light 820 intensity. The light sensor 682 measures an intensity of the transmitted light 820 because it is separated from the external light 810 from the external environment by the same dimmer 692. The LOE / display 690 has negligible effect on the transmitted light 820 intensity. The light sensor 682 is communicatively coupled to the processor 610 by a wired link 830 configured to allow the light sensor 682 to communicate the measured transmitted light intensity 820 to the processor 610. Similarly, the LOE / display 690 (and / or the controllers 630, light source 620, and / or SLM 640; not shown, see Figure 6) is communicatively coupled to the processor 610 by a wired link 840 configured to allow the processor to control at least in emitted light intensity of the light emitted by the LOE / display 690. Controlling emitted light intensity based on the transmitted light intensity 820 allows the processor 610 to match the emitted light intensity to the transmitted light intensity 820 to render virtual images / objects that appear more real / solid. Matching the emitted light intensity to the transmitted light intensity 820 also minimizes system power usage and heat generation while providing a more realistic AR / MR experience.

[0071] Figures 9 and 10 schematically depict a user using a head-mounted AR / MR system 900 according to some embodiments. The head-mounted AR / MR system 900 includes a display system 910 and a light sensor assembly 920. The display system 910 schematically depicted in detail in Figure 9 is similar to the AR / MR system 600 depicted in Figures 6 and 8, and similar elements are identified with similar reference numbers. The display system 910 includes, inter alia, an LOE / display 690, a display dimmer 692, and a processor 610 operatively coupled to the LOE 690. The display dimmer 692 may be a passive dimmer (e.g., a photochromic dimmer). While photochromic dimmer responsive to ultraviolet light intensity, the dimmer 692 may be responsive to visible light intensity. In some embodiments, the dimmer 692 may be a photochromic film that forms a part of the LOE / display 690.

[0072] The light sensor assembly 920 includes a light sensor 982 and a sensor dimmer 992 separating the light sensor 982 from an external environment of the head-mounted AR / MR system 900. The light sensor 982 is configured to measure a sensor transmitted light intensity 820 of light from the external environment of the light sensor 982 transmitted through the sensor dimmer 992 into the light sensor 982. The sensor dimmer 992 is configured to transmit a portion of light similar / identical to a portion of light transmitted by the display dimmer 692. As described above, the LOE / display 690 is substantially transparent and has negligible effect on the transmitted light 820 intensity. The sensor dimmer 992 may be identical to the display dimmer 692.

[0073] In some embodiments, the light sensor assembly 920 is disposed on the AR / MR system 900, but apart from the display system 910. For instance, the light sensor assembly 920 may be disposed on a different part of the head-mounted AR / MR system900 as shown in Figure 10. The light sensor 982 is communicatively coupled to the processor 610 by a link 932 configured to allow the light sensor 982 to communicate the measured transmitted light intensity 820 to the processor 610. The LOE / display 690 (and / or the controllers 630, light source 620, and / or SLM 640; not shown, see Figure 6) is communicatively coupled to the processor 610 by a wired link 840 configured to allow the processor to control at least in emitted light intensity of the light emitted by the LOE / display 690 based on the sensor transmitted light intensity 820.

[0074] Both of the dimmers 992, 692 are disposed on the head-mounted AR / MR system 900, and both respond equally to ambient conditions. Transmitted light 820 passing through the sensor dimmer 992 would not reach the eye, but would be equally attenuated as the transmitted light 820 that passes through the display dimmer 692, thereby providing a measured transmitted light 820 intensity suitable for brightness control.

[0075] Figure 11 schematically depicts a user using an AR / MR system 1100 according to some embodiments. The AR / MR system 1100 includes a head-mounted display system 1110 and a first light sensor assembly 1120. The head-mounted display system 1110 schematically depicted in detail in Figure 11 is similar to the AR / MR system 900 depicted in Figure 9, and similar elements are identified with similar reference numbers. The head-mounted display system 1110 includes, inter alia, an LOE / display 690, a display dimmer 692, and a processor 610 operatively coupled to the LOE 690. The display dimmer 692 may be a passive dimmer (e.g., a photochromic dimmer). While photochromic dimmer responsive to ultraviolet light intensity, the dimmer 692 may beresponsive to visible light intensity. In some embodiments, the dimmer 692 may be a photochromic film that forms a part of the LOE / display 690.

[0076] The first light sensor assembly 1120 includes a first light sensor 982 and a first sensor dimmer 992 separating the first light sensor 982 from a first external environment. The first light sensor 982 is configured to measure a first sensor transmitted light intensity 820 of light from the first external environment of the first light sensor 982 transmitted through the first sensor dimmer 992 into the first light sensor 982. The first sensor dimmer 992 is configured to transmit a portion of light similar / identical to a portion of light transmitted by the display dimmer 692. As described above, the LOE / display 690 is substantially transparent and has negligible effect on the transmitted light 820 intensity,. The sensor dimmer 992 may be identical to the display dimmer 692.

[0077] Unlike in the head-mounted AR / MR system 900 depicted in Figures 9 and 10, the first light sensor assembly 1120 in Figure 11 is disposed apart from the head-mounted display system 1110. For instance, the first light sensor assembly 1120 may be disposed on a wall of a first room. When the user and the head-mounted display system 1110 enter a first external environment (e.g., the first room), the first external environment becomes a display external environment. When the user and the head-mounted display system 1110 are in the first external environment, the processor 610 is operatively coupled to the first light sensor assembly 1120. When the user and the head-mounted display system 1110 are in the first external environment, the processor 610 is configured to control an emitted light intensity of the light emitted by the LOE / display 690 based on the first sensor transmitted light intensity 820.

[0078] When the user and the head-mounted display system 1110 are in the first external environment, external light 810 from an external environment of the user and the AR / MR system 1100 pass through the dimmer 692 as transmitted light 820, which has reduced intensity compared to the external light 810. The light sensor 982 measures an intensity of the transmitted light 820 because the first sensor dimmer 992 is configured to transmit a portion of light similar / identical to a portion of light transmitted by the display dimmer 692, and the LOE / display 690 has a negligible effect on the transmitted light 820 intensity. The light sensor 982 is communicatively coupled to the processor 610 by a wireless link 1132 configured to allow the light sensor 982 to communicate the measured transmitted light intensity 820 to the processor 610. The LOE / display 690 (and / or the controllers 630, light source 620, and / or SLM 640; not shown, see Figure 6) is communicatively coupled to the processor 610 by a wired link 840 configured to allow the processor to control at least in emitted light intensity of the light emitted by the LOE / display 690. Controlling emitted light intensity based on the transmitted light intensity 820 allows the processor 610 to match the emitted light intensity to the transmitted light intensity 820 to render virtual images / objects that appear more real / solid. Matching the emitted light intensity to the transmitted light intensity 820 also minimizes system power usage and heat generation while providing a more realistic AR / MR experience. Separating the first light sensor assembly 1120 from the head-mounted display system 1110 reduces weight, energy usage, and heat generation of the headmounted display system 1110. Further, the separate first light sensor assembly 1120 can include higher quality and more expensive components, and can communicate with multiple head-mounted display systems wirelessly.

[0079] Figures 12 and 13 schematically depict a user using an AR / MR system 1200 spanning two geographically distinct rooms (i.e. , first room 1250 and second room 1260) according to some embodiments. The AR / MR system 1200 includes a head-mounted display system 1110’, a first light sensor assembly 1120, and a second light sensor assembly 1220. The head-mounted display system 1110’ schematically depicted in Figures 12 and 13 is almost identical to the head-mounted display system 1110 schematically depicted in Figure 11 . The only difference is that the head-mounted display system 1110’ depicted in Figures 12 and 13 includes a processor (not shown) configured to be selectively operatively coupled to and to control an emitted light intensity based on information received from the first and second light sensor assemblies 1120, 1220, respectively. The first and second light sensor assemblies 1120, 1220 schematically depicted in Figures 12 and 13 are identical to the first light sensor assembly 1120 schematically depicted in Figure 11.

[0080] As described above, the first and second light sensor assemblies 1120, 1220 are disposed in first and second rooms 1250, 1260, respectively and apart / separate from the head-mounted display system 1110’, which is configured to be mobile between the first and second rooms 1250, 1260. For instance, the first and second light sensor assemblies 1120, 1220 may be disposed on first and second walls of first and second rooms 1250, 1260, respectively.

[0081] When the user and the head-mounted display system 1110’ enter the first room 1250 (Figure 12), the first room 1250 / first external environment becomes a display external environment. When the user and the head-mounted display system 1110’ are in the first room 1250, the processor in the head-mounted display system 1110’ isoperatively coupled to the first light sensor assembly 1120. When the user and the headmounted display system 1110’ are in the first room 1250, the processor in the headmounted display system 1110’ is configured to control an emitted light intensity of the light emitted by the LOE / display based on the first sensor transmitted light intensity detected by the first light sensor assembly 1120.

[0082] When the user and the head-mounted display system 1110’ enter the second room 1260 (Figure 11 ), the second room 1260 / second external environment becomes a display external environment. When the user and the head-mounted display system 1110’ are in the second room 1260, the processor in the head-mounted display system 1110’ is operatively coupled to the second light sensor assembly 1220. When the user and the head-mounted display system 1110’ are in the second room 1260, the processor in the head-mounted display system 1110’ is configured to control an emitted light intensity of the light emitted by the LOE / display based on the second sensor transmitted light intensity detected by the second light sensor assembly 1220.Illustrative Methods Of Controlling Display Light Intensity In Head-Mounted AR and / or MR Systems with Dimming

[0083] Figure 14 depicts a method 1400 of an intensity / brightness of light emitted by a head-mounted display system according to various embodiments. The head-mounted display system may be any of the head-mounted display system 600, 900, 1000 described herein or other similar head-mounted display systems. The head-mounted display system includes a display configured to emit light toward the eyes of the user, a dimmer (e.g., passive dimmer), a light sensor, and a processor. At 1410, the light sensor detects a transmitted light intensity at the light sensor that is similar / identical to atransmitted light intensity at the eyes of the user. At 1420, the processor controls an emitted light intensity of the light emitted by the display toward the eyes of the user based on the detected transmitted light intensity. Controlling emitted light intensity based on the transmitted light intensity allows the processor to match the emitted light intensity to the transmitted light intensity to render virtual images / objects that appear more real / solid. Matching the emitted light intensity to the transmitted light intensity also minimizes system power usage and heat generation while providing a more realistic AR / MR experience.

[0084] Figure 15 depicts a method 1500 of an intensity / brightness of light emitted by a head-mounted display system according to various embodiments. The head-mounted display system may be any of the head-mounted display system 600, 900, 1000 described herein or other similar head-mounted display systems. The head-mounted display system includes a display configured to emit light toward the eyes of the user, a dimmer (e.g., passive dimmer), a light sensor, and a processor. At 1510, the light sensor detects a transmitted light intensity at the light sensor that is similar / identical to a transmitted light intensity at the eyes of the user. At 1515, the light sensor communicates the transmitted light intensity to the processor (e.g., via a wired or wireless link). At 1520, the processor controls an emitted light intensity of the light emitted by the display toward the eyes of the user based on the detected transmitted light intensity. Controlling emitted light intensity based on the transmitted light intensity allows the processor to match the emitted light intensity to the transmitted light intensity to render virtual images / objects that appear more real / solid. Matching the emitted light intensity to the transmitted light intensity also minimizes system power usage and heat generation while providing a more realistic AR / MR experience.

[0085] Figures 16 and 17 depict a method 1600 of an intensity / brightness of light emitted by a head-mounted display system according to various embodiments. The head-mounted display system may be the head-mounted display system 1000 described herein or other similar multi-room head-mounted display systems. The head-mounted display system includes a display configured to emit light toward the eyes of the user, a dimmer (e.g., passive dimmer), first and second light sensors disposed in first and second rooms (e.g., geographically separate locations), and a processor. When the user and the head-mounted display system is in the first room (i.e., first external environment), the processor is operatively coupled to the first light sensor. At 1610, the first light sensor detects a first transmitted light intensity at the first light sensor that is sim ilar / identical to a transmitted light intensity at the eyes of the user. At 1615, the first light sensor communicates the first transmitted light intensity to the processor (e.g., via a wireless link). At 1620, the processor controls an emitted light intensity of the light emitted by the display toward the eyes of the user based on the detected first transmitted light intensity.

[0086] At 1630, the processor detects (e.g., via known location sensors) that the user and the head-mounted display system has moved into the second room (i.e., second external environment). When the user and the head-mounted display system is in the second room (i.e., second external environment), the processor is operatively coupled to the second light sensor. At 1640, when the user and the head-mounted display system is in the second room (i.e., second external environment), the second light sensor detects a second transmitted light intensity at the second light sensor that is similar / identical to a transmitted light intensity at the eyes of the user. At 1645, the second light sensor communicates the second transmitted light intensity to the processor (e.g., via a wirelesslink). At 1650, the processor controls an emitted light intensity of the light emitted by the display toward the eyes of the user based on the detected second transmitted light intensity.

[0087] Controlling emitted light intensity based on the first and second transmitted light intensities based on a location of the user and the head-mounted display system allows the processor to match the emitted light intensity to the transmitted light intensity in the first and second external environments to render virtual images / objects that appear more real / solid. Matching the emitted light intensity to the transmitted light intensity also minimizes system power usage and heat generation while providing a more realistic AR / MR experience.System Architecture Overview

[0001] Figure 18 is a block diagram of an illustrative computing system 1800 suitable for implementing an embodiment of the present disclosure. Computer system 1800 includes a bus 1806 or other communication mechanism for communicating information, which interconnects subsystems and devices, such as processor 1807, system memory 1808 (e.g., RAM), static storage device 1809 (e.g., ROM), disk drive 1810 (e.g., magnetic or optical), communication interface 1814 (e.g., modem or Ethernet card), display 1811 (e.g., CRT or LCD), input device 1812 (e.g., keyboard), and cursor control.

[0002] According to one embodiment of the disclosure, computer system 1800 performs specific operations by processor 1807 executing one or more sequences of one or more instructions contained in system memory 1808. Such instructions may be read into system memory 1808 from another computer readable / usable medium, such as static storage device 1809 or disk drive 1810. In alternative embodiments, hard-wired circuitrymay be used in place of or in combination with software instructions to implement the disclosure. Thus, embodiments of the disclosure are not limited to any specific combination of hardware circuitry and / or software. In one embodiment, the term “logic” shall mean any combination of software or hardware that is used to implement all or part of the disclosure.

[0003] The term “computer readable medium” or “computer usable medium” as used herein refers to any medium that participates in providing instructions to processor 1807 for execution. Such a medium may take many forms, including but not limited to, nonvolatile media and volatile media. Non-volatile media includes, for example, optical or magnetic disks, such as disk drive 1810. Volatile media includes dynamic memory, such as system memory 1808.

[0004] Common forms of computer readable media includes, for example, floppy disk, flexible disk, hard disk, magnetic tape, any other magnetic medium, CD-ROM, any other optical medium, punch cards, paper tape, any other physical medium with patterns of holes, RAM, PROM, EPROM, FLASH-EPROM (e.g., NAND flash, NOR flash), any other memory chip or cartridge, or any other medium from which a computer can read.

[0005] In an embodiment of the disclosure, execution of the sequences of instructions to practice the disclosure is performed by a single computer system 1800. According to other embodiments of the disclosure, two or more computer systems 1800 coupled by communication link 1815 (e.g., LAN, PTSN, or wireless network) may perform the sequence of instructions required to practice the disclosure in coordination with one another.

[0006] Computer system 1800 may transmit and receive messages, data, and instructions, including program, i.e., application code, through communication link 1815 and communication interface 1814. Received program code may be executed by processor 1807 as it is received, and / or stored in disk drive 1810, or other non-volatile storage for later execution. Database 1832 in storage medium 1831 may be used to store data accessible by system 1800 via data interface 1833.

[0007] While the emitted light intensity control systems and methods described herein include various specific components and steps, the described embodiments do not limit the scope of the claims, which encompass similar systems and methods. While the separated light sensor 920 emitted light intensity control system depicted in Figure 10 include a light sensor assembly 920 in the middle of a frame of a head-mounted AR / MR system 900, the light sensor assembly may be located at other areas of the head-mounted AR / MR system 900. While the multiroom emitted light intensity control systems and methods described herein include various “rooms,” any of these rooms may include an outdoor environment. While the multiroom emitted light intensity control systems and methods described herein include first and second rooms, the described embodiments do not limit the scope of the claims which encompass more than two rooms or outdoor environments.

[0088] While emitted light intensity control systems and methods are describe herein as implemented in various display systems, the emitted light intensity control systems and methods described herein may be implemented in various other display systems. For instance, various display system and method embodiments in which the segmentedillumination I dimming systems and methods described herein may be implemented are depicted and described in Appendix A attached hereto.

[0089] Certain aspects, advantages and features of the disclosure have been described herein. It is to be understood that not necessarily all such advantages may be achieved in accordance with any particular embodiment of the disclosure. Thus, the disclosure may be embodied or carried out in a manner that achieves or optimizes one advantage or group of advantages as taught herein without necessarily achieving other advantages as may be taught or suggested herein.

[0090] Embodiments have been described in connection with the accompanying drawings. However, it should be understood that the figures are not drawn to scale. Distances, angles, etc. are merely illustrative and do not necessarily bear an exact relationship to actual dimensions and layout of the devices illustrated. In addition, the foregoing embodiments have been described at a level of detail to allow one of ordinary skill in the art to make and use the devices, systems, methods, and the like described herein. A wide variety of variation is possible. Components, elements, and / or steps may be altered, added, removed, or rearranged.

[0091] The devices and methods described herein can advantageously be at least partially implemented using, for example, computer software, hardware, firmware, or any combination of software, hardware, and firmware. Software modules can include computer executable code, stored in a computer’s memory, for performing the functions described herein. In some embodiments, computer-executable code is executed by one or more general purpose computers. However, a skilled artisan will appreciate, in light of this disclosure, that any module that can be implemented using software to be executedon a general purpose computer can also be implemented using a different combination of hardware, software, or firmware. For example, such a module can be implemented completely in hardware using a combination of integrated circuits. Alternatively or additionally, such a module can be implemented completely or partially using specialized computers designed to perform the particular functions described herein rather than by general purpose computers. In addition, where methods are described that are, or could be, at least in part carried out by computer software, it should be understood that such methods can be provided on non-transitory computer-readable media that, when read by a computer or other processing device, cause it to carry out the method.

[0092] While certain embodiments have been explicitly described, other embodiments will become apparent to those of ordinary skill in the art based on this disclosure.

[0093] The various processors and other electronic components described herein are suitable for use with any optical system for projecting light. The various processors and other electronic components described herein are also suitable for use with any audio system for receiving voice commands.

[0094] Various exemplary embodiments of the disclosure are described herein. Reference is made to these examples in a non-limiting sense. They are provided to illustrate more broadly applicable aspects of the disclosure. Various changes may be made to the disclosure described and equivalents may be substituted without departing from the true spirit and scope of the disclosure. In addition, many modifications may be made to adapt a particular situation, material, composition of matter, process, process act(s) or step(s) to the objective(s), spirit or scope of the present disclosure. Further, as will be appreciated by those with skill in the art, each of the individual variations describedand illustrated herein has discrete components and features which may be readily separated from or combined with the features of any of the other several embodiments without departing from the scope or spirit of the present disclosure. All such modifications are intended to be within the scope of claims associated with this disclosure.

[0095] The disclosure includes methods that may be performed using the subject devices. The methods may include the act of providing such a suitable device. Such provision may be performed by the end user. In other words, the “providing” act merely requires the end user obtain, access, approach, position, set-up, activate, power-up or otherwise act to provide the requisite device in the subject method. Methods recited herein may be carried out in any order of the recited events which is logically possible, as well as in the recited order of events.

[0096] Exemplary aspects of the disclosure, together with details regarding material selection and manufacture have been set forth above. As for other details of the present disclosure, these may be appreciated in connection with the above-referenced patents and publications as well as generally known or appreciated by those with skill in the art. The same may hold true with respect to method-based aspects of the disclosure in terms of additional acts as commonly or logically employed.

[0097] In addition, though the disclosure has been described in reference to several examples optionally incorporating various features, the disclosure is not to be limited to that which is described or indicated as contemplated with respect to each variation of the disclosure. Various changes may be made to the disclosure described and equivalents (whether recited herein or not included for the sake of some brevity) may be substituted without departing from the true spirit and scope of the disclosure. In addition, where arange of values is provided, it is understood that every intervening value, between the upper and lower limit of that range and any other stated or intervening value in that stated range, is encompassed within the disclosure.

[0098] Also, it is contemplated that any optional feature of the variations described may be set forth and claimed independently, or in combination with any one or more of the features described herein. Reference to a singular item, includes the possibility that there are plural of the same items present. More specifically, as used herein and in claims associated hereto, the singular forms “a,” “an,” “said,” and “the” include plural referents unless the specifically stated otherwise. In other words, use of the articles allow for “at least one” of the subject item in the description above as well as claims associated with this disclosure. It is further noted that such claims may be drafted to exclude any optional element. As such, this statement is intended to serve as antecedent basis for use of such exclusive terminology as “solely,” “only” and the like in connection with the recitation of claim elements, or use of a “negative” limitation.

[0099] Without the use of such exclusive terminology, the term “comprising” in claims associated with this disclosure shall allow for the inclusion of any additional element- irrespective of whether a given number of elements are enumerated in such claims, or the addition of a feature could be regarded as transforming the nature of an element set forth in such claims. Except as specifically defined herein, all technical and scientific terms used herein are to be given as broad a commonly understood meaning as possible while maintaining claim validity.

[0100] The breadth of the present disclosure is not to be limited to the examples provided and / or the subject specification, but rather only by the scope of claim language associated with this disclosure.

[0101] In the foregoing specification, the disclosure has been described with reference to specific embodiments thereof. It will, however, be evident that various modifications and changes may be made thereto without departing from the broader spirit and scope of the disclosure. For example, the above-described process flows are described with reference to a particular ordering of process actions. However, the ordering of many of the described process actions may be changed without affecting the scope or operation of the disclosure. The specification and drawings are, accordingly, to be regarded in an illustrative rather than restrictive sense.

Claims

ClaimsWhat is claimed is:1 . A head-mounted display system configured to be worn over eyes of a user, comprising: a display disposed over the eyes of the user and configured to emit light into the eyes of the user, wherein the display comprises a dimmer configured to modify a transparency of the display; a light sensor separated from an external environment of the user by the display and configured to measure a transmitted light intensity of light from the external environment of the user transmitted through the dimmer; and a processor operatively coupled to the display and the light sensor, and configured to control an emitted light intensity of the light emitted by the display based on the transmitted light intensity.

2. The system of claim 1 , wherein the display is configured to display virtual content to the user.

3. The system of claim 1 , wherein the dimmer is configured to variably / adjustably modify the transparency of the display.

4. The system of claim 3, wherein the dimmer is configured to modify the transparency of the display in response to ultraviolet light in the external environment of the user.

5. The system of claim 4, wherein the dimmer comprises a photochromic film.

6. The system of claim 3, wherein the dimmer is configured to modify the transparency of the display in response to visible light in the external environment of the user.

7. An image display system, comprising: a head-mounted display system configured to be worn over eyes of a user, the head-mounted display system comprising a display disposed over the eyes of the user and configured to emit light into the eyes of the user, and a display dimmer configured to modify a transparency of the display such that a display transmitted light intensity of light from a display external environment is transmitted through the dimmer; a first light sensor assembly disposed in a first external environment apart from the head-mounted display system, the first light sensor assembly comprising a first light sensor, and a first sensor dimmer separating the first light sensor from the first external environment of the first light sensor,wherein the first light sensor is configured to measure a first sensor transmitted light intensity of light from the first external environment of the first light sensor transmitted through the first sensor dimmer into the first light sensor; and a processor, wherein when the user and the head-mounted display system are in the first external environment, the processor is operatively coupled to the display and the first light sensor assembly, and the processor is configured to control an emitted light intensity of the light emitted by the display based on the first sensor transmitted light intensity8. The system of claim 7, wherein when the user and the head-mounted display system are in the first external environment, the display transmitted light intensity is equal to the first sensor transmitted light intensity.

9. The system of claim 7, wherein the display is configured to display virtual content to the user.

10. The system of claim 7, wherein the display dimmer is configured to variably / adjustably modify the transparency of the display.11 . The system of claim 10, wherein the display dimmer is configured to modify the transparency of the display in response to ultraviolet light in the display external environment of the first light sensor.

12. The system of claim 11 , wherein the display dimmer comprises a photochromic film.

13. The system of claim 10, wherein the display dimmer is configured to modify the transparency of the display in response to visible light in the display external environment of the first light sensor.

14. The system of claim 7, wherein the first sensor dimmer is configured to variably / adjustably modify the emitted light intensity of the light from the first external environment of the first sensor transmitted through the first sensor dimmer into the first light sensor.

15. The system of claim 14, wherein the first sensor dimmer is configured to modify the emitted light intensity in response to ultraviolet light in the first external environment of the first light sensor.

16. The system of claim 15, wherein the first sensor dimmer comprises a photochromic film.

17. The system of claim 14, wherein the first sensor dimmer is configured to modify the emitted light intensity in response to visible light in the environment of the user.

18. The system of claim 7, wherein the first light sensor assembly is wirelessly coupled to the processor.

19. The system of claim 7, further comprising a second light sensor assembly disposed in a second external environment apart from the head-mounted display system and the first external environment, the second light sensor assembly comprising a second light sensor, and a second sensor dimmer separating the second light sensor from the second external environment of the second light sensor, wherein the second light sensor is configured to measure a second sensor transmitted light intensity of light from the second external environment of the second light sensor transmitted through the second sensor dimmer into the second light sensor, and wherein when the user and the head-mounted display system are in the second external environment, the processor is operatively coupled to second light sensor assembly, and the processor is configured to control the emitted light intensity of the light emitted by the display based on the second sensor transmitted light intensity.

20. The system of claim 19, wherein when the user and the head-mounted display system are in the second external environment, the display transmitted light intensity is equal to the second sensor transmitted light intensity.

21. The system of claim 19, wherein the second sensor dimmer is configured to variably / adjustably modify the emitted light intensity of the light from the second external environment of the second sensor transmitted through the second sensor dimmer into the second light sensor.

22. The system of claim 21 , wherein the second sensor dimmer is configured to modify the emitted light intensity in response to ultraviolet light in the second external environment of the second light sensor.

23. The system of claim 22, wherein the first sensor dimmer comprises a photochromic film.

24. The system of claim 21 , wherein the first sensor dimmer is configured to modify the emitted light intensity in response to visible light in the environment of the user.

25. The system of claim 19, wherein when the user and the head-mounted display system are in the second external environment, the second light sensor assembly is wirelessly coupled to the processor.

26. A method for controlling emitted light intensity of light emitted by a headmounted display comprising a dimmer and configured to be worn over eyes of the user, the method comprising:a light sensor separated from an external environment by the display measuring a transmitted light intensity of light from the external environment transmitted through display; and a processor operatively coupled to the display and the light sensor controlling an emitted light intensity of light emitted by the display based on the transmitted light intensity.

27. The method of claim 26, further comprising the light sensor communicating the transmitted light intensity to the processor.

28. A method for controlling emitted light intensity of light emitted by a headmounted display comprising a dimmer and configured to be worn over eyes of the user, the method comprising: when the user and the display are located in a first external environment, a first light sensor separated from the first external environment by a first sensor dimmer measuring a first sensor transmitted light intensity of light from the first external environment transmitted through first sensor dimmer; and a processor operatively coupled to the display and the first light sensor controlling a first emitted light intensity of light emitted by the display based on the first sensor transmitted light intensity.

29. The method of claim 28, further comprising when the user and the display are located in a first external environment, the first light sensor communicating the first sensor transmitted light intensity to the processor.

30. The method of claim 28, wherein when the user and the head-mounted display system are in the first external environment, a display transmitted light intensity is equal to the first sensor transmitted light intensity.31 . The method of claim 28, further comprising: the processor determining that the user and the display have moved into a second external environment, when the user and the display are located in the second external environment, a second light sensor separated from the second external environment by a second sensor dimmer measuring a second sensor transmitted light intensity of light from the second external environment transmitted through second sensor dimmer, wherein the processor operatively coupled to the second light sensor; and the processor controlling a second emitted light intensity of light emitted by the display based on the second sensor transmitted light intensity.

32. The method of claim 31 , further comprising when the user and the headmounted display system are in the second external environment, the second light sensor communicating the second sensor transmitted light intensity to the processor.

33. The method of claim 31 , wherein when the user and the head-mounted display system are in the second external environment, the display transmitted light intensity is equal to the second sensor transmitted light intensity.

Images