Modular detachable wearable device for AR / VR / MR

Abstract

The present invention relates to a modular, detachable wearable device for AR / VR / MR. A wearable device for augmenting a media content experience can be formed with a mountable physical structure having a removable mounting location and a component device that can be removably mounted via the removable mounting location. The component device can be specifically selected based on the specific type of content consumption environment in which the wearable device will be operated. After a viewer uses the wearable device comprising the mountable physical structure and the component device in a content consumption session in the specific type of content consumption environment, the mountable physical structure can undergo a device cleaning process that the component device would not undergo, as long as the component device is subsequently removed from the mountable physical structure.

Description

[0001] Information about divisional applications

[0002] This application is a divisional application of the Chinese invention patent application with application number 201811003882.1, application date August 30, 2018, and invention name “Modular detachable wearable device for AR / VR / MR”.

[0003] CROSS-REFERENCE TO RELATED APPLICATIONS

[0004] This application claims priority to U.S. Provisional Application No. 62 / 556,915, filed September 11, 2017, and European Patent Application No. 17205123.7, filed December 4, 2017, both of which are incorporated herein by reference. Technical Field

[0005] The present invention relates generally to display systems, and more particularly to modular, detachable wearable devices for augmented reality (AR), virtual reality (VR), mixed reality (MR), and the like. Background Art

[0006] Wearable devices supporting AR, VR, MR, etc., can be used by viewers to blend virtual objects depicted in images on the device display with physical objects in the physical environment, or blend virtual objects depicted in images on the device display with other virtual objects depicted in other images on different displays (e.g., cinema displays, televisions, etc.). To provide viewers with a natural and smooth user experience, wearable devices can integrate a large number of sophisticated and complex electrical and optical components into an overall system with a wearable form factor at great expense.

[0007] If wearable devices are to be used in mass entertainment venues, such as movie theaters, they will need to adhere to strict laws and industry guidelines governing how they should be cleaned. Because wearable devices contain many sensitive electrical and optical components, the rigorous cleaning processes required by laws and industry guidelines are likely to damage and degrade the wearable devices due to the high pressure and movement of the liquids, chemicals, and harsh movements used during the cleaning process.

[0008] The approaches described in this section could be pursued, but not necessarily previously conceived or pursued. Therefore, unless otherwise indicated, it should not be assumed that any of the approaches described in this section qualify as prior art merely by virtue of their inclusion in this section. Similarly, unless otherwise indicated, it should not be assumed that a problem described with respect to one or more approaches has been identified by any prior art in this section. Summary of the Invention

[0009] One aspect of the present disclosure relates to a wearable device for augmenting a media content experience, comprising: a mountable physical structure having one or more removable mounting locations; one or more component devices removably mounted via the one or more removable mounting locations; wherein the one or more component devices are specifically selected based on a specific type of content consumption environment in which the wearable device will be operated; wherein after a viewer uses the wearable device comprising the mountable physical structure and the one or more component devices in a content consumption session in the specific type of content consumption environment, the mountable physical structure is subjected to a device cleaning process that the one or more component devices are not subjected to, as long as the one or more component devices are then removed from the mountable physical structure after the content consumption session.

[0010] In another aspect of the present disclosure, a method includes: removably mounting one or more component devices into a mountable physical structure to form a wearable device for a specific type of content consumption environment, wherein the one or more component devices are mounted into one or more removable mounting locations of the mountable physical structure and have one or more actual position errors within one or more position error tolerances; calibrating the one or more component devices to produce a calibration offset to compensate for the one or more actual position errors within the one or more position error tolerances; and presenting media content to a viewer of the wearable device based at least in part on the calibration offset.

[0011] In yet another aspect of the present disclosure, one or more non-transitory computer-readable storage media are provided to store software instructions that, when executed by one or more processors, cause the method to be performed. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The present invention is illustrated by way of example and not limitation in the figures of the accompanying drawings and in which like reference numerals refer to similar elements and in which:

[0013] Figure 1 side and perspective views showing example mountable physical structures;

[0014] Figures 2A to 2F An example wearable device is shown;

[0015] Figure 3A and Figure 3B An example process flow for device calibration of component devices in a wearable device is shown;

[0016] Figure 4 shows an example process flow; and

[0017] Figure 5An example hardware platform is described upon which a computer or computing device as described herein may be implemented. DETAILED DESCRIPTION

[0018] Described herein are example embodiments of modular, detachable wearable devices for augmented reality (AR), virtual reality (VR), mixed reality (MR), and the like. In the following description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the present invention. However, it will be apparent that the present invention can be practiced without these specific details. In other instances, well-known structures and devices are not described in detail in order to avoid unnecessarily enclosing, obscuring, or obfuscating the present invention.

[0019] Example embodiments are described herein according to the following outline:

[0020] 1. Overview

[0021] 2. Washable and non-washable components

[0022] 3. Installable physical structure

[0023] 4. Wearable devices that can track from the outside in

[0024] 5. Wearable devices that can track from the inside out

[0025] 6. Wearable devices capable of eye tracking

[0026] 7. Additional Examples

[0027] 8. Device Calibration

[0028] 9. Example Process Flow

[0029] 10. Implementation Agency - Hardware Overview

[0030] 11. Equivalents, Extensions, Alternatives and Miscellaneous

[0031] 1. Overview

[0032] This overview presents a basic description of some aspects of example embodiments of the present invention. It should be noted that this overview does not summarize the various aspects of the example embodiments in depth or in detail. In addition, it should be noted that this overview is not intended to be understood as identifying any particularly significant aspects or elements of the example embodiments, nor is it intended to be understood as describing any scope of the example embodiments (specifically) or the present invention (generally). This overview only presents some concepts related to the example embodiments in a streamlined and simplified format, and should be understood as merely a conceptual preface to a more detailed description of the subsequent example embodiments. It should be noted that although separate embodiments are discussed herein, any combination of the embodiments and / or partial embodiments discussed herein may be combined to form other embodiments.

[0033] Wearable devices as described herein can be formed from modular, detachable component devices that support VR, AR, MR, and the like. Some or all of the component devices can be attached to and removed from the wearable device by means of removable mounting locations on a mountable physical structure used to form the wearable device. Additionally, optionally or alternatively, any, some, or all of the component devices can be used individually, in a specific combination of component devices, or in different wearable devices. Additionally, optionally or alternatively, any, some, or all of the component devices can be cleaned together or individually.

[0034] In some embodiments, the wearable device may be an AR headset that includes an imager that accesses media content that is location-based or available along with other images provided in an overall augmented cinema experience.

[0035] For example, imagers may be used to provide position-dependent or position-independent content that may be complementary to the physical scene.

[0036] Additionally, optionally or alternatively, to enhance the cinema 3D experience, a passive optical system or passive optical stack may be incorporated (e.g., mounted, clamped, built into, etc.) into the wearable device to separate the 3D color image presented on the cinema display. The imager may be used to provide additional media content that may complement the scene depicted in the 3D color image.

[0037] The modularity and detachability of the component devices allow for flexible combinations for different content consumption environments. Rather than making a single, efficient and expensive, monolithic device suitable for all content consumption environments, different wearable devices optimized for a specific content consumption environment or wearable devices with different combinations of component devices can be individually made on-site by viewers or cinema staff. As a result, wearable devices can be made lighter and less wasteful (e.g., by only including component devices that will be used in a specific content consumption environment, etc.).

[0038] The wearable device may be specifically designed so that the components of the wearable device that will come into physical contact with the viewer's hair, skin, or bodily fluids (e.g., significant physical contact with the viewer's hair, skin, or bodily fluids from the perspective of relatively strict laws and regulations governing cleaning) are separate modular, detachable component devices that can be removed for cleaning at the end of a content consumption session. Such separate modular, detachable component devices may include, but are not necessarily limited to, any, some, or all of the following: a physical structure that provides a removable mounting location, optical components that are completely or substantially isolated from physical contact, electronic components that are completely or substantially isolated from physical contact, etc.

[0039] In contrast, electronic and optical components that can be cleaned with a gentle cleaning operation (e.g., a simple wipe) or that do not require relatively rigorous device cleaning can be placed in modular, removable component devices that do not physically contact or significantly contact the viewer's hair, skin, or body fluids. Such separate modular, removable component devices may include, but are not necessarily limited to, any, some, or all of the following: imaging devices, media streaming devices, positioning and mapping devices, electronic devices other than plug-in media storage and audio devices (e.g., flash drives, USB drives, earbuds, headphones, clip-on microphones, etc.), and the like.

[0040] Given the strict laws and guidelines for cleaning devices used in public places such as movie theaters, a single monolithic device would contain multiple electronic and optical components and be too expensive to clean using relatively rigorous device cleaning processes, making such devices commercially unviable in mass entertainment venues. A modular, knock-down design for forming a wearable device as described herein can be used to avoid forming such a monolithic device and the need to clean delicate electronic and optical components, making an augmented cinema experience incorporating an AR display commercially viable.

[0041] AR or VR tracking systems can also be implemented at least in part based on modular, detachable component devices. For example, when used outdoors, an inside-out tracking device, such as a simultaneous localization and mapping (SLAM) device, can be attached to a wearable device. When used indoors where external device tracking (e.g., those based on laser scanning and device image capture) can be performed, the SLAM device can be removed, greatly reducing the size and weight of the wearable device. A simple IR light source or optical retroreflector can be placed on the wearable device for external device tracking (or outside the wearable device when tracking) to acquire images of the light source and retroreflector, determine the position of the light source or retroreflector, and generate 3D positioning and mapping data for use by the wearable device in AR or VR.

[0042] The modularity and flexible design offered by these technologies can make these AR or VR systems commercially viable and operable in many content consumption environments, not necessarily limited to augmented entertainment experiences in movie theaters. Rather than striving for a single effective system that works for all use cases or all content consumption environments, the wearable device can be adapted in a flexible manner by the end user or staff on-site to suit these use cases and content consumption environments. The end user or staff can decide if and when they want to obtain a specific component device for a specific use case or a specific content consumption environment.

[0043] Because component devices can be attached to and removed from the wearable device via removable mounting locations, these component devices can be positioned and / or oriented with a positional error tolerance (or position tolerance) due to slight movements or inaccuracies associated with the removable mounting locations or the actual attachment operation.

[0044] At the beginning of a VR / AR / MR session and / or throughout a VR / AR / MR session, device calibration can be performed to generate non-factory calibration offsets. The non-factory calibration offsets can be used to replace the factory calibration offsets or be combined with the factory calibration offsets to form an overall calibration offset that will be used for actual device operation of the wearable device.

[0045] The process flow of device calibration can be implemented based on a master-agent model or a peer-to-peer model. The role of the device calibration controller can be statically assigned to a specific type of component device (or implemented by a specific type of component device) or dynamically assigned to a specific component device, for example, through device negotiation protocol operation, device selection protocol operation, device discovery protocol operation, chronological order based on device attachment time, numerical order based on device correlation ID, etc.

[0046] The component devices of the wearable device can be calibrated independently or autonomously, for example, simultaneously or in any order. In addition, optionally or alternatively, the component devices can be calibrated sequentially.

[0047] For example, if component devices A and B are both present in a wearable device, and if component device A is dependent on component device B (e.g., the imager is dependent on the gaze tracker for viewing direction, the gaze tracker is dependent on the SLAM device or external device tracker for coordinate values, etc.), then the dependent component device, i.e., component device B in this example, may be calibrated first, and then the other component device that is dependent on it may be calibrated. In some embodiments, the SLAM device or external device tracker may be calibrated before the gaze tracker. In a theater where outside-in device tracking is performed by an external device tracker, the external device tracker may be calibrated and / or aligned first, and then (in addition, optionally or alternatively) all wearable devices may be further calibrated and / or aligned.

[0048] Component devices can be calibrated together collaboratively, such as in a peer-to-peer model. Available device calibration results or default calibration parameters can be exchanged via a controller or peer-to-peer. Calibration offsets generated during device calibration can be cached or stored in memory (e.g., registers, cache memory, RAM, ROM, flash memory, etc.).

[0049] In some example embodiments, an apparatus as described herein forms part of a media processing system including, but not limited to, any of the following: a cloud-based server, a mobile device, a virtual reality system, an augmented reality system, a heads-up display device, a helmet-mounted display device, a cave-like system, a wall-sized display, a video gaming device, a display device, a media player, a media server, a media production system, a camera system, a residential-based system, a communication device, a video processing system, a video codec system, a studio system, a streaming media server, a cloud-based content serving system, a handheld device, a game console, a television, a cinema display, a laptop computer, a netbook, a tablet computer, a cellular wireless telephone, an e-book reader, a point-of-sale terminal, a desktop computer, a computer workstation, a computer server, a computer kiosk, or various other types of terminals and media processing units.

[0050] Various modifications to the preferred embodiments and the general principles and features described herein will be readily apparent to those skilled in the art. Therefore, the present disclosure is not intended to be limited to the embodiments shown, but should be accorded the broadest scope consistent with the principles and features described herein.

[0051] 2. Washable and non-washable components

[0052] Wearable devices as described herein can be used for various display applications related to AR, VR, MR, and the like. Example wearable devices include, but are not necessarily limited to, one or more of the following: an image projector, an AR display, a HoloLens display, a Magic Leap display, a mixed reality (MR) display, a tensor display, a stereoscopic display, a light field (LF) display, an Immy display, a Meta display, a pair of relatively simple AR glasses, and the like. Example wearable devices and device displays can be found in U.S. Provisional Patent Application No. 62 / 484,157 (Attorney Docket No. D17013USP1;60175-0303), entitled “AUGMENTED 3D ENTERTAINMENT SYSTEMS,” filed by Ajit Ninan and Neil Mammen on April 11, 2017, the entire contents of which are incorporated herein by reference as if fully set forth herein.

[0053] Wearable devices can adopt a modular disassembled design, wherein multiple component devices involved in these applications can be incorporated into or removed from a mountable physical structure (e.g., eyeglass frames, etc.) to form a wearable device as a single physically integrated interconnected system. Such wearable devices can be used in some or all of a variety of content consumption environments, such as mass entertainment venues, home environments, etc.

[0054] When worn by a viewer (e.g., a human user, etc.) in a content consumption session, the wearable device may be relatively stationary or fixed in position and / or orientation relative to the viewer (e.g., the viewer's head, etc.). Thus, when the viewer's head makes a movement, the wearable device moves with the viewer's head, with no or minimal movement relative to the viewer's head.

[0055] Wearable devices as described herein can be placed together with a combination of specific component devices in public places for use by multiple viewers at different times, such as movie theaters, museums, classrooms, exhibition halls, broadcast studios, etc. Based on corresponding physical device attachment (or mounting) mechanisms provided on the physical structure of the wearable device, the combination of specific devices can be incorporated into the wearable device. In a mass entertainment environment such as a movie theater, a wearable device (e.g., movie theater glasses with specific component devices installed) can be worn on the head of a first viewer in a first content consumption session at a first time, worn on the head of a second different viewer in a second content consumption session at a second different time, and so on.

[0056] The wearable device includes two types of component devices with respect to device washability. The first type of component device in the wearable device is a component device that has relatively significant physical contact with the body of a viewer wearing the wearable device (e.g., skin, eyebrows, hair, bodily fluids such as sweat and tears, etc.). The second type of component device in the wearable device is a component device that has no physical contact with the body of the viewer or has no significant physical contact with the body of the viewer.

[0057] Component devices that come into significant physical contact with a viewer (e.g., first type component devices, etc.) may be specifically designed to be washable. These component devices may not include any sensitive components (e.g., active electrical components, active optical components, etc.). Additionally, optionally or alternatively, these component devices may include sensitive components that are physically enclosed so as not to be damaged during device cleaning.

[0058] As used herein, the term "washable" may refer to a liquid-resistant component device that can be cleaned using liquids such as water, chemicals such as industrial cleaners, device cleaning (e.g., manual, machine, etc.) methods, pressure and movement applied during device cleaning, etc. Device cleaning processes as described herein may be used to comply with laws and / or guidelines of relevant industry associations (e.g., Digital Cinema Initiatives or DCI, etc.), to prevent communicable diseases such as pink eye, and / or to remove bodily fluids (or bodily fluid residue) left on component devices as described herein after a content consumption session.

[0059] Component devices that are in significant physical contact with a viewer (e.g., a first component device type, etc.) may not include any electrically active components, such as batteries, motorized components, etc. Additionally, optionally or alternatively, component devices may contain electrically active components, but these electrically active components are specifically enclosed or fully sealed during the device cleaning process so that the components are not damaged during the device cleaning process.

[0060] Similarly, component devices that are in significant physical contact with a viewer may not include any optically active components, such as laser sources, active light emitters, etc. Additionally, optionally or alternatively, component devices may contain optically active components, but these optically active components are specifically enclosed or fully sealed during device cleaning so that the components are not damaged during device cleaning.

[0061] Component devices that have no physical contact with the viewer or no significant physical contact with the viewer (e.g., second type component devices, etc.) can be specifically designed to be non-washable or only gently washable. These component devices may include sensitive components (e.g., active electrical components, active optical components, etc.). Additionally, optionally or alternatively, these component devices may include sensitive components that may be damaged when exposed to or subjected to a device cleaning process for cleaning washable component devices.

[0062] As used herein, the terms "non-washable" or "gentle wash" may refer to a component device that may not be liquid-resistant and cannot be cleaned using relatively rigorous device cleaning processes, or can only be cleaned using less rigorous cleaning processes, such as physical wiping with a cleaning cloth for use by a viewer or service provider.

[0063] Component devices that are not in physical contact or significant physical contact with a viewer may include electrically active components, such as batteries, motorized components, etc. Additionally, optionally or alternatively, such component devices may contain electrically active components that may or may not be enclosed or fully sealed when exposed to or subjected to a device cleaning process for cleaning the washable component device.

[0064] Similarly, component devices that are not in physical contact or significant physical contact with a viewer may include optically active components, such as laser sources, active light emitters, etc. Additionally, optionally or alternatively, such component devices may contain optically active components that may or may not be enclosed or fully sealed when exposed to or subjected to a device cleaning process for cleaning the washable component device.

[0065] Each of some or all of the component devices that form a wearable device as described herein may be modular and detachable from a mountable physical structure for mounting the component devices to form a physically integrated interconnected system.

[0066] The technology described herein may eliminate the need to produce an expensive single, monolithic device for each of multiple different types of content consumption environments. According to the technology described herein, viewers or staff (e.g., on-site, at runtime, just before or during an AR / VR / MR session, etc.) can make different combinations of specific component devices to adapt to different content consumption environments by adding / installing or removing / disassembling different component devices from a mountable physical structure, depending on the actual content consumption environment (or actual content consumption environment type), rather than producing and using an expensive single, monolithic device that is applicable to multiple different types of content consumption environments and that may be under-engineered or over-engineered for any particular content consumption environment. As used herein, the term "content consumption environment" refers to a physical environment or entertainment venue in which media content related to AR, VR, MR, etc. can be consumed with the help of wearable devices.

[0067] 3. Installable physical structure

[0068] Figure 1 Side and perspective views of an example mountable physical structure 102 are shown. The mountable physical structure 102 serves as a support member to which one or more component devices can be removably mounted to form a wearable device suitable for a specific display application, a specific content consumption environment, etc. The mountable physical structure 102 can have a regular or irregular shape, a skeletal shape, an interconnected shape, a continuous shape, a shape with one or more area voids or volume voids, a shape with a physical indentation, a physical key pattern, etc.

[0069] In some embodiments, the mountable physical structure 102 can have a shape that is rigidly set when in a content consumption session in which a wearable device formed with the mountable physical structure 102 is used to consume media content. In some embodiments, the mountable physical structure 102 can have a shape that is foldable or otherwise deformable (e.g., at least partially, etc.) when not in such a content consumption session.

[0070] By way of example and not limitation, the mountable physical structure 102 is shown as an eyeglass frame including a frame front 104, temples 106, and the like. The front frame 104 may include two spatial voids 108-1 and 108-2, into which a right view optical stack and a left view optical stack may be permanently mounted or removably mounted. In some embodiments, the spatial voids 108-1 and 108-2 may each be permanently or removably mounted with an electro-optical stack, such as a lens and other electro-optical components. In some embodiments, the spatial voids 108-1 and 108-2 may represent additional removable mounting locations for the mountable physical structure 102.

[0071] The mountable physical structure 102 may include one or more removable mounting locations 110-1 through 110-8, such as Figure 1 These removable mounting locations may be distributed among various components of the mountable physical structure (eg, temples 106 , etc.), and are not necessarily limited to the front frame 104 .

[0072] A removable mounting location refers to a specific portion or part of the mountable physical structure 102 for installation (e.g., entry, etc.) on or removal (e.g., pop-up, etc.) from the mountable physical structure 102, wherein the mountable physical structure 102 is a component device having an attachment member that is mechanically or non-mechanically (e.g., magnetically, etc.) compatible with the removable mounting location for component device installation or removal operations.

[0073] For example, the removable mounting location may be, but is not limited to, a specifically shaped recessed or protruding receptacle for mounting (e.g., fitting, attaching, etc.) one or more types of component devices (e.g., rigidly, securely, neatly, tightly, within a minimum tolerance for attachment and removal, etc.). These component devices may have correspondingly shaped protruding or recessed insert members, each of which mates with the specifically shaped recessed or protruding receptacle of the removable mounting location. Additionally, optionally or alternatively, these component devices may have compatible attachment and detachment members (e.g., a clip-on mechanism, etc.) that are mechanically or non-mechanically (e.g., magnetically, etc.) compatible with the removable mounting location for component device installation or removal operations. Additionally, optionally or alternatively, when used in visual applications such as AR, VR, and MR applications, these component devices may be securely attached to the mountable physical structure 102 at the removable mounting location with relatively high spatial accuracy (e.g., in terms of position and / or orientation, etc.), such as by one or more of a locking mechanism, a magnet, a mating shape, a spring, etc.

[0074] The removable mounting locations described herein can be shaped or configured with their own key patterns (e.g., formed with or embodied with a specific physical shape). For example, a specific key pattern of a removable mounting location described herein can only accept a specific shaped insert member of a specific component device for installation, but reject other shaped insert members (e.g., all, etc.) of other component devices.

[0075] Rather than a right component device, a left component device may be permitted to be inserted into the left side of mountable physical structure 102 by matching a left-specific shaped insert member of the left component device with a left-specific key pattern of the left mountable location of mountable physical structure 102 .

[0076] Conversely, rather than a left component device, a right component device may be permitted to be inserted into the right side of mountable physical structure 102 by matching the right side specific shaped insert member of the right component device with the right side specific key pattern of the right side mountable location of mountable physical structure 102 .

[0077] In some embodiments, the mountable physical structure 102, which does not itself have any devices mounted thereon, may not include any electrical components. In some embodiments, the mountable physical structure 102, which does not itself have any devices mounted thereon, may not include any active electrical components, but may include or have embedded therein passive electrical components, such as metallic (e.g., aluminum, gold, copper, metal alloys, etc.) or non-metallic (e.g., thin film-based, rare earth element-based, graphene-based, nanotube-based, etc.) conductive interconnects, for example, between or among some or all of the removable mounting locations 110-1 through 110-8.

[0078] In some embodiments, the mountable physical structure 102, which does not have any devices mounted on it, may not include any optical components. In some embodiments, the mountable physical structure 102, which does not have any devices mounted on it, may not include active optical components, but may include or have embedded therein passive optical components, such as lenses, mirrors, waveguides, and the like, for example, to receive, redirect, or inject light from, to, or through one or more component devices attached at one or more of the removable mounting locations 110-1 through 110-8.

[0079] In some embodiments, the mountable physical structure 102, which does not itself have any devices mounted thereon, may include one or more active electrical components or active optical components. Some or all of these active components may be permanently enclosed during the device cleaning process and / or during a content consumption session (e.g., fully sealed to prevent contact with cleaning liquids or chemicals during the device cleaning process, etc.). Additionally, optionally or alternatively, some or all of these active components may be temporarily enclosed (e.g., a disposable or removable cover, a disposable or removable housing, etc.) to prevent these components from coming into contact with cleaning liquids or chemicals during the device cleaning process and / or from coming into contact with a viewer's hair, skin, body parts, or bodily fluids during a content consumption session. For example, a cleaning-sensitive component device may be placed in a temporary disposable cover, exposing only the attached portion for insertion into the mountable physical structure 102. The temporary disposable cover may prevent a viewer from making significant physical contact with the component device that would result in the component device being cleaned using a relatively rigorous device cleaning process.

[0080] 4. Wearable devices that can track from the outside in

[0081] Figure 2A Shown can be removably mounted on a mountable physical structure (e.g., Figure 1 102, etc.) on an example wearable device 122. In some embodiments, the wearable device 122 includes a right view optical stack 124-1, a left view optical stack 124-2, a right view imager 126-1, a left view imager 126-2, one or more light sources 134-1, 134-2, 134-3, etc.

[0082] like Figure 2A Some or all of the components / devices depicted in the embodiment may be implemented as software, hardware, a combination of software and hardware, etc., by one or more mechanical components, one or more electro-optical components, one or more computing devices, modules, units, etc. Figure 2A Some or all of the components / devices depicted in the drawings may be communicatively coupled (e.g., wirelessly, inductively, in an ad hoc network, in a network formed using one or more device sensing and / or discovery protocols, using wired connections, etc.) to the devices described above. Figure 2A Some other components / devices depicted in or with Figure 2A Other components / devices not depicted are coupled.

[0083] In some embodiments, a wearable device as described herein is worn or mounted on the head of the viewer 112. The wearable device (e.g., 122, etc.) may represent or include one or more of the following: eyeglass frames, headphones, wearable peripherals for mobile phones, masks, helmets, strap accessories, rings, watch straps, headwear, etc. The eyeglass frames may be personalized for individual viewers, or may have a universal size (e.g., a maximum size, a size suitable for children, etc.) designed to be worn or mounted by a relatively large group of viewers. By way of example and not limitation, the eyeglass frames are used to mount (e.g., in a removable manner, in a non-removable manner, etc.) the right view optical stack 124-1 and the left view optical stack 124-2 in front of the right eye 130-1 and the left eye 130-2, respectively, of the viewer 112.

[0084] Additionally, optionally or alternatively, the eyeglass frame is configured to attach or mount the right view imager 126-1 and the left view imager 126-2 (e.g., removably, non-removably, etc.) to, for example, the top edge of the eyeglass frame. Additionally, optionally or alternatively, the eyeglass frame is configured to attach or mount the SLAM device 128 (e.g., removably, non-removably, etc.) to, for example, the top crossbar of the eyeglass frame.

[0085] The right-view optical stack 124-1 may be used by the viewer 112 of the wearable device 122 to view a right-view image presented on a cinema display or a non-cinema display. The left-view optical stack 124-2 may be used by the viewer 112 of the wearable device 122 to view a left-view image presented on a cinema display or a non-cinema display. The right-view image viewed by the viewer 112 through the right-view optical stack 124-1 and the left-view image viewed by the viewer 112 through the left-view optical stack 124-2 form a stereoscopic image.

[0086] The right imager 126-1 may be used by the viewer 112 to view a right display image presented on a component device display, the component device display being virtually or physically formed by the right imager 126-1 and the left imager 126-2. The left imager 126-2 may be used by the viewer 112 to view a left device display image presented on a component device display. The right display image viewed by the viewer 112 through the right imager 126-1 and the left display image viewed by the viewer 112 through the left imager 126-2 form a stereoscopic (component device) display image that may be complementary to the stereoscopic image viewed by the same viewer 112 through the right optical stack 124-1 and the left optical stack 124-2.

[0087] In some embodiments, the component device display is not a physical display, but rather an image plane or virtual display formed by light rays emitted by the right view imager 126-1 and the left view imager 126-2. More specifically, the right view imager 126-1 emits right view light rays that reach the right eye 130-1 of the viewer 112 to allow the viewer 112 to visually perceive or view the right view display image as if the right view display image were displayed on the component device display. Similarly, the left view imager 126-2 emits left view light rays that reach the left eye 130-2 of the viewer 112 to allow the viewer 112 to visually perceive or view the left view display image as if the left view display image were displayed on the component device display.

[0088] In some embodiments, the component device display 116 may be located at a depth that is different from or the same as the depth of a cinema or non-cinema display with respect to the viewer 112. As used herein, the term "depth" may refer to the (e.g., frontal, etc.) spatial distance between a viewer and an image plane of a display (e.g., a cinema or non-cinema display, a device display, etc.).

[0089] In some embodiments, component device display 116 may display or project the device display image at a single image plane located at a single distance in front of the viewer, or at multiple image planes located at multiple different distances in front of the viewer (e.g., via time division multiplexing, etc.). These distances of the image planes may be fixed or automatically adjustable.

[0090] For example, the right view imager 126-1 and the left view imager 126-2 may be used in conjunction with a lens element (e.g., having a fixed focal length, contained in the right view optical stack 124-1 and the left view optical stack 124-2, etc.) to project a right view (component device) display image and a left view (component device) display image from an image plane (or component device display) at a fixed depth to the viewer 112.

[0091] In another non-limiting example, the right view imager 126-1 and the left view imager 126-2 may be used in conjunction with lens elements (e.g., having a fixed focal length, having a variable focal length, included in the right view optical stack 124-1 and the left view optical stack 124-2, etc.) to project right view and left view (component device) display images from image planes (or component device displays) at multiple fixed depths to the viewer 112.

[0092] In some embodiments, a device image renderer operating with or implemented in the wearable device 122 or the left view imager 126-1 and the right view imager 126-2 attached thereto may generate the right view and left view display images as a set of time-contiguous or time-synchronized 3D images.

[0093] Examples (components) of cinema or non-cinema displays, device displays, image presenters, etc. can be found in: U.S. Provisional Patent Application No. 62 / 414,901, filed on October 31, 2016, entitled "EYEWEAR DEVICES WITH FOCUS TUNABLE LENSES"; U.S. Provisional Patent Application No. 62 / 484,157, filed on April 11, 2017, entitled "AUGMENTED 3D ENTERTAINMENT SYSTEMS", the entire contents of the above two applications are incorporated herein by reference as if fully set forth herein.

[0094] The electro-optical stack as described herein may include one or more optical and / or electro-optical component layers, including but not limited to a combination of one or more of the following: a light-transmitting component layer, a light-reflecting component layer, a filter layer, a light-modulating layer, a microprism layer, a microlens layer, a variable or fixed lens, a beam splitter, a beam combiner, a light engine, a switching element (e.g., based on transistors, etc.) for controlling the degree of light transmission (or transmittance) or the degree of light reflection (reflectivity), and the like.

[0095] The right view optical stack 124-1 represents an electro-optical stack that allows right view rays—from a cinema or non-cinema display—for presenting a right view image on a cinema or non-cinema display to reach (or be transmitted to) the right eye 130-1 of the viewer 112. The left view optical stack 124-2 represents an electro-optical stack that allows left view rays—from a cinema or non-cinema display—for presenting a left view image on a cinema or non-cinema display to reach (or be transmitted to) the left eye 130-2 of the viewer 112. At runtime, when a right view cinema display image is being presented on a cinema or non-cinema display, the right view optical stack 124-1 may be optically transparent for right view rays, and when a left view image is being presented on a cinema or non-cinema display, the left view optical stack 124-2 may be optically transparent for left view rays.

[0096] The techniques described herein may be used to support rendering and viewing of 3D images utilizing various right-eye / left-eye separation techniques, including but not limited to those based on complementary colors, linear polarization, circular polarization, shutter mirrors, spectral spatial separation, etc. Any of the aforementioned right-eye / left-eye separation techniques may be used with the left view optical stack 124-1 and the right view optical stack 124-2 to allow light rays used to render the left view image and the right view image to reach the right eye 130-1 and the left eye 130-2, respectively, of the viewer 112—or to reach ocular visual sweet spots (e.g., foveal vision) that are spatially separated by the interpupillary distance 132.

[0097] In some embodiments, the right view optical stack 124-1 and the left view optical stack 124-2 can implement complementary color 3D technology for viewing right view and left view images presented on a cinema or non-cinema display. The right view optical stack 124-1 and the left view optical stack 124-2 provide right eye / left eye separation by filtering light with two color filters (e.g., red light for presenting one image and cyan light for presenting another image, etc.), such as a red filter and a cyan filter.

[0098] In some embodiments, the right view optical stack 124-1 and the left view optical stack 124-2 can implement linear polarization 3D technology for viewing right view and left view images presented on a cinema or non-cinema display. The right view optical stack 124-1 and the left view optical stack 124-2 provide right eye / left eye separation by filtering linearly polarized light (vertically polarized light for presenting one image and horizontally polarized light for presenting the other image) using two orthogonal linear polarizers, such as a vertical polarizer and a horizontal polarizer.

[0099] In some embodiments, the right view optical stack 124-1 and the left view optical stack 124-2 can implement circular polarization 3D technology for viewing right view and left view images presented on a cinema or non-cinema display. The right view optical stack 124-1 and the left view optical stack 124-2 provide right eye / left eye separation by filtering circularly polarized light (right polarized light for presenting one image and left polarized light for presenting the other image) using two orthogonal circular polarizers, such as a right polarizer and a left polarizer.

[0100] In some embodiments, the right view optical stack 124-1 and the left view optical stack 124-2 may implement shutter mirror 3D technology for viewing right view and left view images presented on a cinema or non-cinema display. The right view optical stack 124-1 and the left view optical stack 124-2 provide right eye / left eye separation by synchronizing the time-multiplexed viewing of the right eye and the left eye with the time-multiplexed presentation of the respective right image and the left side through right eye / left eye shuttering (a first image display time interval for presenting one image and a second image display time interval for presenting the other image).

[0101] In some embodiments, the right view optical stack 124-1 and the left view optical stack 124-2 may implement spectral spatial separation 3D technology for viewing left and right view images presented on a cinema or non-cinema display. The right view optical stack 124-1 and the left view optical stack 124-2 provide right eye / left eye separation by filtering (e.g., a first set of red, green, and blue light for presenting one image and a second set of red, green, and blue light for presenting another image, wherein the first set of red, green, and blue light is spectrally separated from the second set of red, green, and blue light, etc.) using two spectral filters (e.g., a first filter that passes a first set of red, green, and blue light but rejects a second set of red, green, and blue light and a second filter that passes the second set of red, green, and blue light but rejects the first set of red, green, and blue light, etc.).

[0102] In various embodiments, the right view imager 126-1 and the left view imager 126-2 may use the same or different left eye / right eye separation technology for presenting right and left view (component device) images than those left eye / right eye separation technologies used to present right and left view images. In an example, the wearable device 122 may include spatially separated right view imager 126-1 and left view imager 126-2—for example, positioned a distance approximately the interpupillary distance 132 apart—to project right view and left view (component device) display images to the right eye 130-1 and left eye 130-2, respectively. In another example, the wearable device 122 may include a central imager (for example, mounted on the top rail of an eyeglass frame, etc.) to transmit or project right view and left view (component device) display images to the right eye 130-1 and left eye 130-2, respectively.

[0103] Light sources 134-1 to 134-3 may be removably or non-removably attached to one or more rigid components (e.g., a nose bridge, a top crossbar, a frame edge, etc.) to which physical structure 102 may be mounted (e.g., mechanically, etc.). When viewer 112 wears wearable device 122, the spatial positions of light sources 134-1 to 134-3 are stationary relative to wearable device 122, but may not be stationary relative to viewer 112 or the 3D space in which wearable device 122 is located because the viewer's body or head is moving.

[0104] The light sources described herein can have an attachment mechanism, such as an insert tab, a keyed mechanical component, etc., that fits into a receptacle that can be mounted on the physical structure 102. In some embodiments, the attachment mechanism can fit securely into the receptacle and cannot be easily or arbitrarily removed by a viewer (e.g., 112, etc.). In some embodiments, the light sources described herein can be permanently attached to the physical structure 102.

[0105] In various embodiments, none of the light sources 134-1 to 134-3, or some or all of the light sources 134-1 to 134-3 may be light emitters (e.g., LED lights, infrared light emitters, etc.) that emit light that is captured in a device tracking image by a device tracker placed in the physical environment in which the viewer 112 or wearable device 122 is located. Additionally, optionally or alternatively, none of the light sources 134-1 to 134-3, or some or all of the light sources 134-1 to 134-3 may be light reflectors that emit light that is captured in a device tracking image by the device tracker.

[0106] For example, in some embodiments, the light sources 134-1 to 134-3 on the wearable device 122 are all light reflectors. The device tracker may include a laser scanner that emits a scanning laser beam (e.g., having a wavelength of light invisible to human vision, etc.) to scan the light source of the wearable device (e.g., 122, etc.) present in the physical space. The light sources 134-1 to 134-3 on the wearable device 122 may be retroreflectors (e.g., reflecting incident light back to the transmitter or a specific direction, etc.), scattering reflectors, etc., which receive incident laser light of the scanning laser beam from the laser scanner and redirect / reflect these incident light rays toward the laser scanner as reflected light. In some embodiments, the light reflectors 134-1 to 134-3 may include a light conversion material, such as quantum dots, etc., that converts the received laser light into regenerated light. Reflected or regenerated light from the light sources 134-1 through 134-3 may be focused or (optionally or alternatively) scattered onto a laser scanner, one or more light sensors operating with a laser scanner, one or more image capture devices operating with a laser scanner, etc. The reflected or regenerated light from the light sources 134-1 through 134-3 may be captured into a device tracking image, as described herein.

[0107] In addition, optionally or alternatively, one or more radio frequency (RF) tracking ID signals, one or more optical tracking ID signals, etc. may be sent by a separate device (operating with the wearable device 122 and installed at the seating space of the viewer of the wearable device 122) and captured by one or more device ID sensors of the device tracker to determine device ID information related to the wearable device 122; the separate device may be stationary, removably or non-removably attached to the seat, etc.

[0108] In some embodiments, one of the light sources 134-1 to 134-3 on the wearable device 122 is selected or used as a light emitter, while all other light sources are light reflectors. By way of illustration and not limitation, light source 134-1 may be selected as a light emitter, which may include one or more of the following: an LED light, a laser emitter, a light emitter with a light-converting material, etc. The remaining light sources (134-2 and 134-3) may be light reflectors, each of which may include one or more of the following: a retroreflector, a diffuse reflector, etc. Light reflector 134-1 emits light that is focused or (optionally or alternatively) scattered onto one or more tracking image sensors in the device tracker. Simultaneously, light reflectors 134-2 and 134-3 receive incident light from light source 134-1 and redirect / reflect the incident light into reflected light. In some embodiments, light reflectors 134-2 and 134-3 may include light-converting materials, such as quantum dots, that convert received laser light into regenerated light. The reflected / regenerated light from light sources 134-2 and 134-3 can be focused or (or, optionally or alternatively) scattered onto a tracking image sensor in a device tracker. The emitted light, reflected / regenerated light can be captured in a device tracking image, as described herein.

[0109] Additionally, optionally or alternatively, light from a light emitter as described herein may be digitally encoded with device ID information of the wearable device (102); at least a portion of the digitally encoded light from the light emitter or light reflector may be captured by one or more device ID sensors in one or more tracking sensor assemblies (e.g., 124, etc.).

[0110] In some embodiments, the light emitters in the light sources 134-1 to 134-3 of the wearable device 122 can be electrically or optoelectronically coupled to a light emission controller, which can control the light emitters to emit light that logically represents one or more device ID signals encoded with device ID information and, optionally, other information related to the wearable device 122.

[0111] Examples of light sources, device tracking, and the like may be found in U.S. Provisional Patent Application No. 62 / 484,131 (Attorney Docket No. D17011USP1; 60175-0301), entitled “PASSIVE MULTI-WEARABLE-DEVICES TRACKING,” filed by Ajit Ninan and Neil Mammen on April 11, 2017, the entire contents of which are incorporated herein by reference as if fully set forth herein.

[0112] Examples of a device tracker as described herein may include, but are not necessarily limited to, any of the following: an external device tracker, an internal device tracker, an outside-in device tracker, an inside-out device tracker, and the like.

[0113] 5. Wearable devices that can track from the inside out

[0114] Figure 2B The invention shows a device that can be removably mounted on a mountable physical structure (e.g., Figure 1 102, etc.) on an example wearable device 122-1 formed of one or more modular detachable component devices. In some embodiments, the wearable device 122-1 includes a right view optical stack 124-3, a left view optical stack 124-4, a right view imager 126-1, a left view imager 126-2, a simultaneous localization and mapping (SLAM) device 128, etc.

[0115] like Figure 2B Some or all of the components / devices depicted in the embodiment may be implemented as software, hardware, a combination of software and hardware, etc., by one or more mechanical components, one or more electro-optical components, one or more computing devices, modules, units, etc. Figure 2B Some or all of the components / devices depicted in the drawings may be communicatively coupled (e.g., wirelessly, inductively, in an ad hoc network, in a network formed using one or more device sensing and / or discovery protocols, using wired connections, etc.) to the devices described above. Figure 2B Some other components / devices depicted in or with Figure 2B Other components / devices not depicted are coupled.

[0116] By way of example and not limitation, the mountable physical structure 102 may be an eyeglass frame for mounting the right view optical stack 124-3 and the left view optical stack 124-4 in front of the right eye 130-1 and the left eye 130-2, respectively, of the viewer 112 (e.g., removably, non-removably, etc.).

[0117] Additionally, optionally or alternatively, the eyeglass frame is configured to attach or mount the right view imager 126-1 and the left view imager 126-2 (e.g., removably, non-removably, etc.) to, for example, the top edge of the eyeglass frame. Additionally, optionally or alternatively, the eyeglass frame is configured to attach or mount the SLAM device 128 (e.g., removably, non-removably, etc.) to, for example, the top crossbar of the eyeglass frame.

[0118] The right view optical stack 124 - 3 and the left view optical stack 124 - 4 may be used by the viewer 112 of the wearable device 122 to view the physical environment in which the viewer 112 or the wearable device 122 - 1 is located.

[0119] The right imager 126-1 may be used by the viewer 112 to view a right display image presented on a component device display, the component device display being virtually or physically formed by the right imager 126-1 and the left imager 126-2. The left imager 126-2 may be used by the viewer 112 to view a left device display image presented on a component device display. The right display image viewed by the viewer 112 through the right imager 126-1 and the left display image viewed by the viewer 112 through the left imager 126-2 form a stereoscopic (component device) display image that may depict objects and provide complementary information that is spatially aligned or depicted at a specific physical location in the physical space viewed by the same viewer 112 through the right optical stack 124-3 and the left optical stack 124-4.

[0120] The right view imager 126-1 and the left view imager 126-2 can be used in conjunction with lens elements (e.g., having a fixed focal length, contained in the right view optical stack 124-3 and the left view optical stack 124-4, etc.) to project right view and left view (component device) display images from an image plane (or component device display) at a fixed depth to the viewer 112.

[0121] In another non-limiting example, the right view imager 126-1 and the left view imager 126-2 may be used in conjunction with lens elements (e.g., having a fixed focal length, having a variable focal length, included in the right view optical stack 124-3 and the left view optical stack 124-4, etc.) to project right view and left view (component device) display images from image planes (or component device displays) at multiple fixed depths to the viewer 112.

[0122] The right view optical stack 124 - 3 and the left view optical stack 124 - 4 represent electro-optical stacks that are optically transparent to light emanating from physical objects, individuals, etc., present in physical space.

[0123] The SLAM device 128 can be used to construct or update a 3D map of the physical environment (e.g., a content consumption environment, a movie theater, a home entertainment room, a venue, an amusement park, a tourist attraction, a museum, etc.) in which the viewer 112 or the wearable device 122 is located. The SLAM device may include or operate in conjunction with one or more image sensors (e.g., a camera element, etc.), position and / or orientation sensors (e.g., a GPS unit, a motion sensor, etc.), a communication device / interface (e.g., wirelessly, based on a wired connection, Wi-Fi, infrared, Bluetooth, laser-based, LED-based, etc.), and the like. The image sensor can be used by the SLAM device 128 to acquire images of the physical environment. The position and / or orientation sensor can be used by the SLAM device 128 to collect sensor data about the position and / or orientation of the viewer 112 or the wearable device 122 at any given point in time. The communication devices / interfaces may be used by the SLAM device 128 to communicate with other component devices of the wearable device 122, and to communicate with cloud-based or built-in servers (e.g., a device image renderer in a content consumption environment, a media streaming server, a mapping data server, a server executing SLAM algorithms or analysis, etc.) and / or a computing system external to or remote from the wearable device 122. In some embodiments, the SLAM device 128 executes a SLAM algorithm or analysis based on some or all of the following: images of the physical environment, sensor data regarding position and / or orientation, and map information cached in a local data storage device or received from an external server, etc., to simultaneously obtain the position of the viewer 112 or wearable device 122 (which may include both position and orientation) and a map (e.g., a 2D map, a 3D map, etc.) of the physical environment within a strict time budget (e.g., less than one millisecond, one millisecond, five milliseconds, etc.) measured based on the time to acquire the images and sensor data. Additionally, optionally or alternatively, in some embodiments, a server remote from the SLAM device 128 may be used in conjunction with the SLAM device 128 to receive some or all of the image and / or sensor data and perform some or all of the SLAM algorithms or analysis.

[0124] 6. Wearable devices capable of eye tracking

[0125] Figure 2C and Figure 2D The invention shows a device that can be removably mounted on a mountable physical structure (e.g., Figure 1 102, etc.) on one or more modular detachable component devices to form example wearable devices 122-2 and 122-3.

[0126] like Figure 2CAs shown in FIG, the wearable device 122-2 includes component devices, such as a right view optical stack 124-1, a left view optical stack 124-2, a right view imager 126-1, a left view imager 126-2, one or more light sources 134-1, 134-2, 134-3, etc. Figure 2A As shown in ; the wearable device 122-2 further includes a right gaze (or eye) tracker 136-1 and a left gaze (or eye) tracker 136-2.

[0127] like Figure 2D As shown in FIG, the wearable device 122-3 includes a right view optical stack 124-3, a left view optical stack 124-4, a right view imager 126-1, a left view imager 126-2, a simultaneous localization and mapping (SLAM) device 128, etc. Figure 2B As illustrated in FIG; the wearable device 122-4 further includes a right gaze tracker 136-1 and a left gaze tracker 136-2.

[0128] like Figure 2C and Figure 2D Some or all of the components / devices depicted in each of the embodiments may be implemented as software, hardware, a combination of software and hardware, etc., by one or more mechanical components, one or more electro-optical components, one or more computing devices, modules, units, etc. Figure 2C and Figure 2D Some or all of the components / devices depicted in each of the embodiments may be communicatively coupled (e.g., wirelessly, inductively, in an ad hoc network, in a network formed using one or more device sensing and / or discovery protocols, using wired connections, etc.) to the devices. Figure 2C and Figure 2D Each of the depicted some other components / devices or with Figure 2C and Figure 2D Each of the components is coupled to other components / devices not depicted.

[0129] For example Figure 2C and Figure 2DThe gaze (or eye) tracker 136-1 or 136-2 represents a component device for tracking the gaze (or viewing) direction of the viewer 112's eyes at any given time in real time or near real time. To track eye movement and / or gaze direction, the gaze tracker may implement one or more of the following: an embedded mirror method, an embedded magnetic field sensor method, a video nystagmus method, an electrooculogram method, an infrared or near-infrared light-based pupil tracking method, etc. The gaze device may include or operate in conjunction with the following: one or more image or non-image sensors, a sensor data analyzer, a communication device / interface (e.g., wirelessly, based on a wired connection, Wi-Fi, infrared, Bluetooth, laser-based, LED-based, etc.), etc. The image and / or non-image sensors may be used by the eye tracker to obtain image and / or non-image sensor data of the eye or some or all parts of the eye (e.g., the iris, pupil, cornea, retinal blood vessels, etc.). The sensor data analyzer can be used by the eye tracker to analyze image and / or non-image sensor data to determine the gaze (or viewing) direction of the eye, for example, within a strict time budget (e.g., less than one millisecond, one millisecond, five milliseconds, etc.) measured based on the time at which the image and / or sensor data was acquired. A communication device / interface can be used by the gaze tracker to communicate with other component devices of the wearable device (e.g., 122-2, 122-3, etc.) (including, but not limited to, a corresponding gaze tracker that tracks the gaze or viewing direction of the viewer's other eye), and with a cloud-based server or an internal server (e.g., a device image renderer in a content consumption environment, a media streaming server, etc.) and / or a computing system external to or remote from the wearable device (e.g., 122-2, 122-3, etc.). Additionally, optionally or alternatively, in some embodiments, a server remote from the gaze tracker can be used in conjunction with the gaze tracker to receive some or all of the image and / or sensor data and perform some or all of the gaze tracking algorithm or analysis.

[0130] 7. Additional Examples

[0131] In some embodiments, a device image renderer (not shown) as described herein may be implemented as a separate device from the left view imager 126-1 and the right view imager 126-2 or may be implemented as part of the left view imager 126-1 and the right view imager 126-2. A device image presenter is communicatively coupled to some or all component devices of a wearable device (e.g., 122, 122-1, 122-2, etc.) as described herein; receives position and orientation (or orientation) data of the wearable device (e.g., 122, 122-1, 122-2, etc.) tracked / monitored by a device tracker or by a SLAM device with the wearable device (e.g., 122, 122-1, 122-2, etc.); generates one or more (component device) display images of the wearable device (e.g., 122, 122-1, 122-2, etc.) based at least in part on the position and orientation data of the wearable device (e.g., 122, 122-1, 122-2, etc.); presents the display images on the component device displays using the left view imager 126-1 and the right view imager 126-2 of the wearable device (e.g., 122, 122-1, 122-2, etc.); and the like. The device image presenter may utilize the left view imager 126-1 and the right view imager 126-2 of the wearable device (e.g., 122, 122-1, 122-2, etc.) and / or other local or remote devices to communicate control information, status information, position and orientation data, image data, such as display images, metadata, etc., via one or more data connections. Example data connections may include, but are not limited to, a wireless data connection, a wired data connection, a radio frequency-based data connection, a cellular data connection, a Wi-Fi data connection, an infrared-based data connection, a data connection via an HDMI cable, a data connection via an optical cable, a data connection via a high-speed serial interface (HSSI), a high-definition serial digital interface (HD-SDI), 12G-SDI, a USB cable, and the like.

[0132] Examples of (device) image presenters, cinema or non-cinema image presentations, (component device) display image presentations, etc. can be found in: U.S. Provisional Patent Application No. 62 / 484,121 (Agent Case No.: D16152BUSP1; 60175-0298), application name “LAYERED AUGMENTED ENTERTAINMENT EXPERIENCES”, filed by Ajit Ninan, Neil Mammen and Tyrome Brown on April 11, 2017; U.S. Provisional Patent Application No. 62 / 484,148 (Agent Case No.: D17012USP1; 60175-0302), application name “ADAPTING VIDEO IMAGES FOR WEARABLE DEVICES”, filed by Ajit Ninan and Neil Mammen on April 11, 2017, the entire contents of which are incorporated herein by reference as if fully set forth herein.

[0133] Figure 2E Wearable device 122 - 4 is shown with a component device attached to a removable mounting location on another component device of wearable device 122 - 4 .

[0134] As shown, the wearable device 122-4 includes an imager 126 mounted on a first removable mounting location of the wearable device 122-4, such as at a top crossbar of an eyeglass frame serving as a mountable physical structure in the wearable device 122-4.

[0135] In some embodiments, a component device as described herein may include zero, one, or more removable mounting locations. For example, imager 126 may be provided with a second removable mounting location. A SLAM device (e.g., 128, etc.) may be mounted to the second removable mounting location of imager 126 to become part of wearable device 122-4.

[0136] Figure 2F A wearable device is shown in which a component device is sealed into a wearable device-mountable physical structure 102. As shown, the component device can be embedded in the temple 106 of the eyeglass frame. The component device can be, but is not necessarily limited to, a built-in speaker 138. The built-in speaker 138 can be sealed (e.g., completely, hermetically, etc.) by a sealing member 140 located in the temple of the eyeglass frame. The sealing member 140 can prevent the built-in speaker 138 from physical contact with liquids or chemicals while the eyeglasses containing the built-in speaker 138 are being cleaned.

[0137] In some embodiments, the sealing member 140 may be made of an acoustically transmissive material 142, such as a rigid material that can transmit sound at some or all of the frequencies audible to humans. Sound transmitted through the acoustically transmissive material 142 of the sealing member 140 may further propagate through the skull structure of the viewer wearing the wearable device (which is in physical contact with the acoustically transmissive material 142) and into the ears of the viewer.

[0138] Thus, in some embodiments, one or more component devices that are part of a wearable device may be non-removably (or permanently) incorporated as part of a mountable physical structure as described herein. Non-removably incorporated component devices may include built-in speakers, gaze trackers, optical stacks, imagers, SLAM devices, and the like.

[0139] Additionally, optionally or alternatively, a portion of the component device may be non-removably incorporated into the mountable physical structure, while the remainder of the component device may be removably mounted directly or indirectly to the mountable physical structure. For example, an imager's optical waveguide may be configured to receive light generated by the imager's light engine. The imager's optical waveguide may be embedded within the integral physical structure, while the remainder of the imager, including but not limited to the light engine, may be incorporated into a separate, removably mounted component device.

[0140] In addition to the mechanical functionality of being removably mounted to a component device to be incorporated into a wearable device, the removable mounting locations described herein may also incorporate other functionality. In some embodiments, the removable mounting locations may incorporate electrical functionality (e.g., electrical interfaces, etc.), such as data connectors, etc. For example, once a component device is mounted at the removable mounting location, a wired data connection may be achieved through a data connector between the component device and other component devices. Additionally, optionally or alternatively, the removable mounting locations may also incorporate optical functionality (e.g., optical interfaces, etc.), such as optical connectors, etc. For example, once a component device is mounted at the removable mounting location, light may be received and / or transmitted through an optical connector between the component device and other component devices.

[0141] According to the technology described herein, a wearable device can be formed relatively efficiently by an end user (e.g., a viewer in a movie theater, a viewer in a home entertainment room, a viewer walking in any physical environment, etc.) by adding or removing modular detachable component devices to or from the wearable device or the mountable physical structure therein.

[0142] One or more component devices—such as one or more of the following: an imager, a SLAM device, a gaze tracker, an optical stack specifically configured for outdoor use, an optical stack specifically configured for indoor use, an optical stack specifically configured for use in a cinema, an optical stack configured for multiple uses (in a variety of different physical environments), infrared or non-infrared light sources for device tracking, RF component devices, other electronic components, etc.—can be securely attached (e.g., suitably, with relatively high spatial accuracy, etc.) directly or indirectly to a mountable physical structure of a wearable device by means of a mountable physical structure and a removable mounting location on the component device that has been assembled into the wearable device by a viewer.

[0143] Component devices removably mounted on a wearable device may be removed from the wearable device by a viewer if the wearable device does not require the component devices to operate in the target physical environment in which the wearable device will be used.

[0144] Therefore, combinations of specific modular detachable component devices can be assembled and used in different specific target physical environments. Viewers can easily assemble these component devices into wearable devices within a relatively short time interval (e.g., less than one minute, about one minute, several minutes, etc.) to form individually configured wearable devices suitable for different specific target physical environments.

[0145] In some physical environments, such as in movie theaters, indoors, home entertainment scenarios, and the like, outside-in device tracking may be performed by an external device tracker that tracks an image formed by a light source positioned with or on the wearable device. In these environments, a relatively large SLAM device that performs inside-out device tracking (as opposed to outside-in device tracking performed using an external device tracker) may be removed / detached from these wearable devices, or otherwise rendered absent from these wearable devices. When the SLAM device is removed from the wearable device or otherwise rendered absent in these environments, the typically large and electro-optically complex SLAM device is not subjected to (e.g., rigorous, etc.) device cleaning processes, liquids, or chemicals. Furthermore, when the wearable device is tracked via outside-in tracking in these environments, the wearable device does not need to implement inside-out device tracking and the associated complex functionality. The wearable device may incorporate relatively simple components and, together with these simple components, occupy a relatively small or modest form factor, thereby enabling the wearable device to be easily handled, assembled, disassembled, reassembled, and cleaned by viewers or cinema personnel who provide at least some portions of the wearable device to viewers.

[0146] Additionally, optionally or alternatively, the remaining component devices on the wearable device may be attached to the wearable device in such a manner that the component devices that are more sensitive to the device cleaning process (e.g., those component devices in which the active optical and / or electrical components are not fully sealed, etc.) are positioned / oriented so as to have no contact or minimal contact with the viewer's hair, skin, or body fluids, while the component devices that are insensitive to the device cleaning process (e.g., those component devices that have no optical and / or electrical components, have no active optical and / or electrical components, or in which the active optical and / or electrical components are fully sealed, etc.) may be positioned / oriented to be in physical contact with the viewer to a relatively high degree or over a relatively large range.

[0147] Imagine that cinema staff provide imagers that are shared by viewers in the cinema at different viewing times. These imagers may include sensitive electrical and / or optical components (e.g., active electrical components, active optical components, etc.). Laws and industry guidelines may dictate that any device that has relatively close physical contact with the viewer's hair, skin, and body fluids must have a relatively strict device cleaning process. Each imager shared by multiple viewers can be modularized into a single physical housing and removably attached to a mountable physical structure of a wearable device. The imager can be positioned / oriented to not contact or minimally contact the viewer's hair, skin, or body fluids to comply with laws and industry guidelines. Although the portion of the wearable device that has closer physical contact with the viewer must undergo a device cleaning process, the imager may not undergo or may undergo relatively gentle cleaning (e.g., the viewer wipes it with a disposable wet cloth, etc.) without violating laws and / or industry guidelines.

[0148] Additionally, optionally or alternatively, in some embodiments, an imager privately owned by a viewer for use in a non-cinema physical environment may be brought by the viewer into a device-sharing environment, such as a cinema, to acquire augmented 3D content.

[0149] When operating in a content consumption environment (e.g., a movie theater) where an external device tracker can be used to track a viewer's wearable device, rather than a SLAM device that may not be present in the wearable device, the external device tracker can interact with the imager and / or device image renderer to render an image that is spatially correctly aligned to an object or location in the physical or virtual scene perceived by the viewer through the separate optical stacks (e.g., 124-1 and 124-2, etc.).

[0150] When operating in other physical environments where an external device tracker is not available to track the viewer's wearable device, the SLAM device attached to the wearable device can interact with the imager to present images that are spatially correctly aligned to objects or locations in the physical or virtual scene perceived by the viewer through the separate optical stacks (e.g., 124-3 and 124-4, etc.).

[0151] The imager may implement the same imaging engine to communicate with a SLAM device that performs inside-out tracking or with an external device tracker that performs outside-in tracking. The device tracking data may be streamed from the external device tracker or SLAM device to the imager, depending on whether an external device tracker or SLAM device is present to perform device tracking, even if some other data is generated at least in part based on the device tracking data.

[0152] It should be noted that in various embodiments, external or outside-in device tracking can be used in other physical environments, in addition to or in place of theaters. For example, a home entertainment setting can implement external or outside-in device tracking and provide / stream device tracking data to a wearable device or its imager and / or other component devices present in the home entertainment setting. Thus, the techniques described herein can be used to acquire and consume augmented content, such as augmented 3D content, in a variety of physical environments.

[0153] 8. Device Calibration

[0154] Although component devices can be attached to and removed from the wearable device via removable mounting locations on a mountable physical structure and / or already attached component devices, these component devices can be positioned and / or oriented to have positional error tolerances (or position tolerances) relative to a spatial coordinate system that is stationary for the wearable device. For these component devices that can operate with relatively high spatial accuracy (e.g., in terms of position and / or orientation, etc.), a calibration operation can be used to determine the calibration offset required to compensate for these errors.

[0155] For example, an imager may be used in a wearable device as described herein to present a (component device) display image superimposed (or overlaid) with a cinema or non-cinema image viewed through an optical stack separate from the imager. Objects depicted, people depicted, and the like in the display image should be accurately spatially located or aligned with corresponding spatial locations and / or orientations in the cinema or non-cinema image in order to provide a relatively high-quality user experience to a viewer of the wearable device. For example, an apple depicted in the display image should be accurately spatially located or aligned (e.g., in terms of size, depth, geometry, etc.) with, for example, a spatial location and / or orientation corresponding to the spatial location and / or orientation of a table depicted in the cinema or non-cinema image.

[0156] Additionally, optionally or alternatively, in some content consumption scenarios, the imager may be used to present a display image that is superimposed (or overlaid) with physical objects, persons, and the like in the physical environment as viewed through an optical stack separate from the imager. The objects, persons, and the like depicted in the display image should be accurately spatially located or aligned with corresponding spatial locations and / or orientations in the physical environment. For example, an apple depicted in the display image should be accurately spatially located or aligned (e.g., in terms of size, depth, geometry, etc.) with respect to, for example, the spatial location and / or orientation of a table in the physical environment.

[0157] As described herein, an imager can be calibrated using one or more of a variety of AR display device calibration methods. For example, a test image (or synthetic image) can be presented by the imager during a calibration operation or dynamically (e.g., during a VR / AR / MR session, etc.) to determine or measure the calibration offset (e.g., in terms of the number of pixels shifted horizontally and / or vertically, rotation angles, fractional pixel shifts, etc.) required to accurately depict an object, person, etc. in the display image. Additionally, optionally or alternatively, locations or objects with preset (or known) spatial positions and / or orientations in the physical environment (e.g., physical objects, depicted objects, physical fiducial markers, fiducial markers depicted at known locations on the display screen, test image patterns, etc.) can be used to calibrate and align the imager in the physical environment. The results of the calibration and alignment can be used to generate a calibration offset for the imager to compensate for any errors when the imager is attached to the wearable device.

[0158] Gaze (or eye) trackers as described herein can be calibrated using one or more of a variety of gaze tracker calibration methods. For example, in a movie theater, visual stimuli with preset spatial locations can be used to attract viewers to look, while images or sensor data of the viewer's eyes and / or eye movements can be generated and analyzed. The calibration results can be used to generate calibration offsets for each of the gaze trackers to compensate for any errors when the gaze tracker is attached to a wearable device.

[0159] As described herein, a SLAM device can be calibrated using one or more of a variety of SLAM device calibration methods. For example, in a movie theater, a spatial location with a preset coordinate value in the coordinate system of the physical environment can be used by the SLAM device to generate position sensor data, determine / estimate the coordinate value of the spatial location, and compare the estimated coordinate value with the preset coordinate value. The result of the calibration or comparison can be used to generate a calibration offset for the SLAM device to compensate for any error when the SLAM device is attached to the wearable device.

[0160] Component devices of the same type may produce different spatial accuracies during the manufacturing process. In some embodiments, a factory-set calibration offset may be generated by device calibration of the component device performed at the factory. The factory-set calibration offset may be stored using the component device or on a server. The component device may be removably mounted to a wearable device for VR / AR / MR sessions in a content consumption environment. Due to tolerances and slight movements when attaching the component to corresponding removable mounting points on the wearable device, the factory-set calibration offset may be invalid. Spatial device calibration may be performed at the beginning of a VR / AR / MR session and / or throughout a VR / AR / MR session to generate an out-of-factory calibration offset. The out-of-factory calibration offset may be used to replace the factory-set calibration offset or be combined with the factory-set calibration offset to form an effective calibration offset to be used for actual device operation of the wearable device. For example, when an imager is attached to a wearable device, position and / or orientation errors may be introduced into the light projection or light waveguide, causing pixels to shift or rotate from their correct positions. Due to positional and / or orientation errors, the imager may inject or direct light intended for a microfiber (in the optical waveguide) corresponding to a particular pixel location into a different microfiber (in the optical waveguide) corresponding to a different pixel location. An effective calibration offset can be used to correct these positional and / or orientation errors and enable the imager to display an image in which the depicted object is aligned at the correct spatial location.

[0161] In some embodiments, some or all of the component devices of a wearable device as described herein can independently and autonomously perform their respective spatial device calibration operations. For example, each of these component devices can independently and autonomously perform its respective spatial device calibration operations without exchanging status or information related to device calibration operations with other component devices.

[0162] In some embodiments, some or all of the component devices of a wearable device as described herein may jointly and cooperatively perform their individual spatial device calibration operations. These component devices may exchange states or information related to device calibration operations with other component devices. In a non-limiting example, a SLAM device may pass coordinate values ​​of physical objects in a physical environment to other component devices, such as a gaze tracker, for the gaze tracker to generate a calibration offset that will be used to determine or estimate the viewing direction of a viewer's eyes. In another non-limiting example, the imager and the gaze tracker may synchronize their individual calibration offsets in such a way that the viewing direction of the viewer's eyes, when determined by the gaze tracker, can be effectively and accurately mapped to a specific location, such as a pixel location on a device display associated with the imager.

[0163] In some embodiments, a viewer or staff member may attach some or all of the component devices forming the wearable device to a mountable physical structure in any order. Each of some or all of the component devices may include its own processing engine that supports some or all of the spatial device calibration operations as described herein. The individual processing engines of the component devices may be configured to detect whether the component device is currently attached (e.g., securely, operatively, etc.) to the wearable device.

[0164] In the case where a component device implements a device calibration controller, in response to detecting that the component device itself is securely attached to the wearable device, the component device or the device calibration controller therein may identify the types of some or all other component devices currently (e.g., securely, operatively, etc.) attached to the wearable device and select a method for calibrating and / or aligning the attached component devices with the wearable device.

[0165] In the event that a component device does not implement a device calibration controller, in response to detecting that a component device is securely attached to a wearable device, the component device or a processing engine therein may send a device attachment signal to some or all of the other component devices currently attached to the wearable device. The device attachment signal may be sent via a wired data connection (e.g., a data bus, an interconnect, a data connection crossbar, etc.) or via a wireless data connection. The device attachment signal may be sent periodically on a linear or logarithmic time scale until the component device designated as the device calibration controller responds to the device attachment signal.

[0166] There are several methods a wearable device may choose to use to calibrate and / or align attached component devices of the wearable device.

[0167] One or more communication mechanisms for calibration and / or alignment of component devices may also be used for actual operation (for consuming VR / AR / MR content). For example, the viewing direction may be communicated to the imager via the communication mechanism when determined by a gaze tracker. In some embodiments, the imager may present the portion of the image or the depicted object in the direction that the current viewer is viewing with relatively high visual acuity. The viewing direction may also be provided to a SLAM device or an external device tracker to generate 3D mapping information, and the spatial position corresponding to the viewing direction may have a relatively high resolution. The 3D mapping information may be provided to the imager. Based at least in part on the 3D map information, the imager may spatially accurately present and align the objects depicted in the component device display image in the correct spatial position, as if the depicted object were an intrinsic part of the physical environment represented by the 3D mapping information (e.g., the depicted apple is as if it were in a physical environment with the correct size, orientation, geometry).

[0168] The offsets generated when the device is calibrated may be cached or stored in memory (eg, registers, cache memory, RAM, ROM, flash memory, etc.).

[0169] Figure 3A An example process flow for device calibration of component devices in a wearable device is shown.

[0170] The process flow described herein can be implemented based on a master-agent model. The wearable device may include a device calibration controller (implemented via software, hardware, or a combination of software and hardware) that enables some or all of the process flow to be executed. The device calibration controller can be implemented using a single component device (e.g., a master component device, etc.), such as an imager, a gaze tracker, a SLAM device, etc. Other component devices, such as proxy component devices, can implement a device calibration agent (implemented via software, hardware, or a combination of software and hardware) that communicates with and operates with the device calibration controller in the master component device to perform device calibration operations in a cooperatively correct and / or semantically correct manner. In some embodiments, the role of the device calibration controller can be statically assigned to a specific type of component device. For example, an imager can be pre-configured with a device calibration controller. In some embodiments, the device calibration controller can be dynamically assigned to a specific component device, such as through a device negotiation protocol operation, a device selection protocol operation, a device discovery protocol operation, a chronological order based on the time of device attachment, a numerical order based on the device correlation ID, etc.

[0171] Additionally, optionally or alternatively, the process flow described herein can be implemented based on a peer-to-peer model. The wearable device can implement a distributed device calibration controller with some or all of the component devices of the wearable device. The component devices communicate with each other and operate with each other to perform device calibration operations of the component devices in a cooperatively correct and / or semantically correct manner. As used herein, the term "device calibration controller" can refer to a device calibration controller implemented by a single component device or by a group of component devices.

[0172] In block 202 , the wearable device or a device calibration controller therein discovers component devices that are removably mounted to form the wearable device, for example, via a device discovery protocol supported by the component devices.

[0173] In block 204, the wearable device identifies the component device to be calibrated. In some embodiments, the component device to be calibrated may set a calibration indicator, such as a data field value, a specific beacon, a specific data tag, etc., to indicate that the component device is the component device to be calibrated. The device calibration controller may access the calibration indicator.

[0174] For example, when a component device is inserted into a wearable device, the component device or a processing engine therein may enter a device initialization state, wherein the component device sets a calibration indicator to indicate that the wearable device (e.g., just inserted, etc.) needs to be calibrated. In response to receiving the calibration indicator, the device calibration controller identifies the component device as one of one or more component devices in the wearable device to be calibrated.

[0175] In block 206 , the wearable device determines an order in which to calibrate the component devices, and performs calibration of the component devices based on the determined order.

[0176] In some embodiments, each of some or all of the component devices to be calibrated may be calibrated independently or autonomously, for example, simultaneously or in any order. In some embodiments, some or all of these component devices may be calibrated sequentially. For example, if component devices A and B are both present in a wearable device, and if component device A depends on component device B (for example, the imager depends on the gaze tracker in terms of viewing direction, the gaze tracker depends on the SLAM device or external device tracker in terms of coordinate values, etc.), then the dependent component device, i.e., component device B in this example, may be calibrated first, and then the other component device that depends on this component device may be calibrated. For example, in some embodiments, the SLAM device or external device tracker may be calibrated before the gaze tracker. In a cinema where device tracking is performed by an external device tracker, the external device tracker may be calibrated first, and then (in addition, optionally or alternatively) all wearable devices may be further calibrated.

[0177] Additionally, optionally or alternatively, some or all of the component devices to be calibrated may be calibrated together collaboratively, such as in a peer-to-peer model. Available device calibration results or default calibration parameters may be exchanged via a controller or peer-to-peer.

[0178] In block 208 , the wearable device, or a device calibration controller therein, receives and / or extracts zero, one, or more calibration-related parameters for sharing among some or all of the component devices or other devices operating with the wearable device.

[0179] In an example, the image presenter can be implemented by the wearable device or by an external device external to the wearable device. The image presenter can receive a calibration offset and / or calibration-related parameters of an imager of the wearable device, generate a (component device) display image, and transform (e.g., shift, rotate, scale, etc.) the display image based at least in part on the calibration offset and / or calibration-related parameters. The transformed display image can be provided to the imager for presentation.

[0180] In another example, a device calibration controller of a wearable device may receive calibration-related parameters, such as calibration offsets extracted from a SLAM device or an external device tracker. When an imager or another component device receives coordinate values ​​from the SLAM device or external device during device calibration or normal operation, these calibration-related parameters or calibration offsets may be used to adjust the received coordinate values ​​to calibrated coordinate values.

[0181] Figure 3B An example process flow for device calibration of component devices in a wearable device is shown. The process flow as described herein can be implemented based on a master-agent model or based on a peer-to-peer model.

[0182] In block 222, the wearable device detects that the component device is attached to the wearable device. The detection may be performed by a system configuration / reconfiguration controller of the wearable device, a device calibration controller of the wearable device, or the like.

[0183] In block 224, the wearable device identifies zero, one, or more calibration-related parameters from other attached component devices that have already been calibrated in the wearable device. This may be determined at least in part based on the type of component device to be calibrated. Any identified calibration-related parameters required by the component device may be extracted from the other attached component devices and transferred to the component device to be calibrated.

[0184] In block 226 , the wearable device subjects the component devices to calibration.

[0185] In block 228 , the wearable device identifies zero, one, or more calibration-related parameters for sharing from the calibrated component device, extracts the calibration-related parameters from the calibrated component device, and shares the parameters with other component devices of the wearable device.

[0186] 9. Example Process Flow

[0187] Figure 4 An example process flow according to an example embodiment of the present invention is shown. In some example embodiments, one or more computing devices or components may perform at least some of this process flow. In block 402, one or more component devices are removably mounted to a mountable physical structure to form a wearable device for a particular type of content consumption environment. The one or more component devices are mounted to the mountable physical structure in one or more removable mounting locations and have one or more actual position errors within one or more position error tolerances.

[0188] In block 404 , one or more component devices are calibrated to generate a calibration offset to compensate for one or more actual position errors within one or more position error tolerances.

[0189] In block 406 , media content is presented to a viewer of the wearable device based at least in part on the calibration offset.

[0190] In an embodiment, a master-agent model is used to calibrate one or more component devices.

[0191] In an embodiment, a peer model is used to calibrate one or more component devices.

[0192] In an embodiment, at least one of the one or more component devices is autonomously and independently calibrated relative to all other component devices in the one or more component devices.

[0193] In one embodiment, one or more first calibration offsets of a first component device of the one or more component devices are used to generate one or more second calibration offsets of a second, different component device of the one or more component devices.

[0194] In one embodiment, one or more component devices are calibrated in a time sequence.

[0195] In an embodiment, one or more component devices include an imager for presenting a component device display image; one or more calibration offsets of the imager are used to accurately spatially align objects depicted in the component device display image with other objects depicted in other display images generated by a display system other than the wearable device.

[0196] In an embodiment, one or more component devices include an imager for presenting a component device display image; one or more calibration offsets of the imager are used to accurately spatially align objects depicted in the component device display image with physical objects in a physical scene.

[0197] In one embodiment, a wearable device for augmenting a media content experience includes: a mountable physical structure having one or more removable mounting locations; and one or more component devices removably mounted via the one or more removable mounting locations. The one or more component devices are specifically selected based on the specific type of content consumption environment in which the wearable device will be operated. After a viewer uses the wearable device comprising the mountable physical structure and the one or more component devices in a content consumption session in the specific type of content consumption environment, the mountable physical structure undergoes a device cleaning process that the one or more component devices do not undergo, provided that the one or more component devices are subsequently removed from the mountable physical structure after the content consumption session.

[0198] In an embodiment, each component device of the one or more component devices represents a modular device enclosed in an individual physical housing dedicated to each such component device.

[0199] In an embodiment, a mountable physical structure has an exterior surface on which a set of light sources are located; light from the set of light sources is used by an external device tracker present in a particular type of content consumption environment for outside-in device tracking.

[0200] In embodiments, each component device of the one or more component devices is removable from the mountable physical structure and mountable to a different mountable physical structure to form a different wearable device.

[0201] In an embodiment, different wearable devices operate in one of: a specific content consumption environment, one or more content consumption environments different from the specific content consumption environment, and the like.

[0202] In an embodiment, the installable physical structure is provided by staff of the mass entertainment venue; the installable physical structure will be cleaned using an equipment cleaning process specified by law or industry regulations; the equipment cleaning process includes the use of one or more of the following: liquids, chemicals, pressure, movement, etc.

[0203] In embodiments, the mountable physical structure is free of electronic components susceptible to damage and physical contact with liquids during device cleaning.

[0204] In embodiments, the mountable physical structure is free of optical components susceptible to damage and physical contact with liquids during device cleaning.

[0205] In an embodiment, at least one of the one or more component devices includes an electronic component susceptible to damage and physical contact with liquid during a device cleaning process.

[0206] In embodiments, at least one of the one or more component devices includes an optical component susceptible to damage and physical contact with liquid during device cleaning.

[0207] In an embodiment, at least one of the one or more component devices is subjected to a second device cleaning process that is different from the device cleaning process.

[0208] In an embodiment, the device cleaning process is specifically designed to allow the physical structure to be installed for use with a second, different wearable device for a second, different viewer in a specific type of content consumption environment.

[0209] In an embodiment, a component device of the one or more component devices is mounted to a removable mounting location of the one or more removable mounting locations; the removable mounting location does not contain any electrical or optical interface.

[0210] In an embodiment, a component device of the one or more component devices is mounted to a removable mounting location of the one or more removable mounting locations; the removable mounting location includes an optical interface to the component device.

[0211] In an embodiment, one or more component devices include one or more of the following: an optical stack for viewing a physical scene, an optical stack for viewing an image presented by a display system external to the wearable device, an imager for the wearable device to present component device display images, a simultaneous positioning and mapping device, an eye tracking device, and the like.

[0212] In an embodiment, the specific type of content consumption environment represents one of: a cinema-based content consumption environment, a home-based content consumption environment, an out-of-home content consumption environment, and the like.

[0213] In an embodiment, a wearable device includes an optical stack for viewing a three-dimensional image presented by a display system other than the wearable device.

[0214] In an embodiment, the optical stack is built into the wearable device in a non-removable manner.

[0215] In an embodiment, the optical stack is clipped to a wearable device.

[0216] In an embodiment, each of the one or more component devices includes its own power supply.

[0217] In an embodiment, a component device of the one or more component devices generates image presentation light that is optically transmitted to another component device in the wearable device.

[0218] In an embodiment, all of the one or more component devices are interconnected wirelessly.

[0219] In an embodiment, at least two of the one or more component devices are electrically or optically interconnected using an electrical or optical interface provided by one or more removable mounting locations.

[0220] In an embodiment, after a viewer of the wearable device removably mounts the one or more component devices into a mountable physical structure of the wearable device, the one or more component devices are calibrated to generate an out-of-factory calibration offset.

[0221] In an embodiment, the wearable device includes a further component device that is removably mounted to a removable mounting location of a component device among the one or more component devices.

[0222] In an embodiment, the wearable device includes one or more audio speakers that transmit sound through the viewer's bone structure.

[0223] In various example embodiments, an apparatus, system, device, or one or more other computing devices performs any one or a portion of the foregoing methods as described. In an embodiment, a non-transitory computer-readable storage medium stores software instructions that, when executed by one or more processors, perform the methods as described herein.

[0224] It should be noted that although separate embodiments are discussed herein, any combination of embodiments and / or portions of embodiments discussed herein may be combined to form further embodiments.

[0225] 10. Implementation Agency - Hardware Overview

[0226] According to one embodiment, the techniques described herein are implemented by one or more special-purpose computing devices. The special-purpose computing devices may be hard-wired to perform the techniques, or may include digital electronic devices that are continuously programmed to perform the techniques, such as one or more application-specific integrated circuits (ASICs) or field-programmable gate arrays (FPGAs), or may include one or more general-purpose hardware processors that are programmed to perform the techniques according to program instructions in firmware, memory, other storage devices, or a combination. Such special-purpose computing devices may also combine custom hard-wired logic, ASICs, or FPGAs with custom programming to implement the techniques. The special-purpose computing devices may be desktop computer systems, portable computer systems, handheld devices, networked devices, or any other devices that incorporate hard-wired and / or program logic to implement the techniques.

[0227] For example, Figure 5 5 is a block diagram illustrating a computer system 500 on which an example embodiment of the present invention may be implemented. Computer system 500 includes a bus 502 or other communication mechanism for communicating information, and a hardware processor 504 coupled with bus 502 for processing information. Hardware processor 504 may be, for example, a general-purpose microprocessor.

[0228] The computer system 500 also includes a main memory 506, such as a random access memory (RAM) or other dynamic storage device, coupled to the bus 502 to store information and instructions to be executed by the processor 504. The main memory 506 may also be used to store temporary variables or other intermediate information during the execution of instructions to be executed by the processor 504. Such instructions, when stored in a non-transitory storage medium accessible to the processor 504, present the computer system 500 as a special-purpose machine customized to perform the operations specified in the instructions.

[0229] Computing system 500 further includes a read only memory (ROM) 508 or other static storage device coupled to bus 502 for storing static information and instructions for processor 504 .

[0230] A storage device 510, such as a magnetic or optical disk, solid-state RAM, is provided and coupled to the bus 502 for storing information and instructions.

[0231] The computer system 500 may be coupled to a display 512, such as a liquid crystal display, via bus 502 for displaying information to a computer viewer. An input device 514, including alphanumeric and other keys, is coupled to bus 502 for communicating information and command selections to processor 504. Another type of viewer input device is a cursor control 516, such as a mouse, trackball, or cursor direction keys, for communicating directional information and command selections to processor 504 and controlling cursor movement on display 512. This input device typically has two degrees of freedom in two axes, a first axis (e.g., x) and a second axis (e.g., y), allowing the device to specify a position in a plane.

[0232] Computer system 500 can implement the techniques described herein using custom hardwired logic, one or more ASICs or FPGAs, firmware, and / or program logic that, in combination with the computer system, turns computer system 500 into a special-purpose machine or programs computer system 500 into a special-purpose machine. According to one embodiment, the techniques described herein are performed by computer system 500 in response to processor 504 executing one or more sequences of one or more instructions contained in main memory 506. Such instructions may be read into main memory 506 from another storage medium, such as storage device 510. Execution of the sequences of instructions contained in main memory 506 causes processor 504 to perform the process steps described herein. In alternative embodiments, hardwired circuitry may be used in place of or in combination with software instructions.

[0233] As used herein, the term "storage media" refers to any non-transitory medium that stores data and / or instructions that cause a machine to operate in a specific manner. Such storage media may include non-volatile media and / or volatile media. Non-volatile media include, for example, optical or magnetic disks, such as storage device 510. Volatile media include dynamic memory, such as main memory 506. Common forms of storage media include, for example, floppy disks, diskettes, hard disks, solid-state drives, magnetic tape or any other magnetic data storage medium, CD-ROMs, any other optical data storage medium, any physical medium with a pattern of holes, RAM, PROM and EPROM, Flash EPROM, NVRAM, any other memory chip or cartridge.

[0234] Storage media is distinct from, but can be used in conjunction with, transmission media. Transmission media participates in the transfer of information between storage media. For example, transmission media include coaxial cables, copper wire, and optical fiber, including the electrical wiring that comprises bus 502. Transmission media can also take the form of acoustic or light waves, such as those generated during radio wave and infrared data communications.

[0235] Various forms of media may be involved in carrying one or more sequences of one or more instructions to the processor 504 for execution. For example, the instructions may initially be carried on a disk or solid-state drive of a remote computer. The remote computer may load the instructions into its dynamic memory and use a modem to send the instructions over a telephone line. A modem local to the computer system 500 may receive the data over the telephone line and convert the data into an infrared signal using an infrared transmitter. An infrared detector may receive the data carried in the infrared signal, and appropriate circuitry may place the data on the bus 502. The bus 502 carries the data to the main memory 506, from which the processor 504 retrieves the instructions and executes them. The instructions received by the main memory 506 may optionally be stored on the storage device 510 before or after execution by the processor 504.

[0236] The computer system 500 also includes a communication interface 518 coupled to the bus 502. The communication interface 518 provides two-way data communication coupled to a network link 520, which is connected to a local area network 522. For example, the communication interface 518 may be an integrated services digital network (ISDN) card, a cable modem, a satellite modem, or a modem for providing a data communication connection to a corresponding type of telephone line. As another example, the communication interface 518 may be a local area network (LAN) card to provide a data communication connection to a compatible LAN. A wireless link may also be implemented. In any such embodiment, the communication interface 518 sends and receives electrical, electromagnetic, or optical signals that carry digital data streams representing various types of information.

[0237] Network link 520 typically provides data communication through one or more networks to other data devices. For example, network link 520 can provide a connection through local area network 522 to host computer 524 or to data equipment operated by Internet service provider (ISP) 526. ISP 526 then provides data communication services through the global packet data communication network now commonly referred to as the "Internet" 528. Local area network 522 and Internet 528 both use electrical, electromagnetic, or optical signals that carry digital data streams. The signals through the various networks and the signals on network link 520 and through communication interface 518 are example forms of transmission media that carry digital data to and from computer system 500.

[0238] Computer system 500 can send messages and receive data, including program code, through the network, network link 520, and communication interface 518. In the Internet example, server 530 can transmit the requested code for an application program through Internet 528, ISP 526, local area network 522, and communication interface 518.

[0239] The received code may be executed by processor 504 as it is received, and / or stored in storage device 510 or other non-volatile storage for later execution.

[0240] 11. Equivalents, Extensions, Alternatives and Miscellaneous

[0241] In the foregoing specification, example embodiments of the present invention have been described with reference to numerous specific details that may vary from implementation to implementation. The sole and exclusive indicator of what is the invention, and what the applicants regard as the invention, is the set of claims identified in this application, which are the specific form in which such claims have been issued, including any subsequent amendments. Any definitions expressly set forth herein for terms contained in such claims shall govern the meaning of such terms as used in the claims. Therefore, no limitation, element, property, feature, advantage, or attribute that is not expressly recited in a claim should limit the scope of such claim in any way. Accordingly, the specification and drawings should be regarded in an illustrative rather than a restrictive sense.

Claims

1. A method for presenting media content, comprising: Removably mounting one or more component devices to a mountable physical structure to form a wearable device for use in a particular type of content consumption environment, wherein the one or more component devices are mounted to the mountable physical structure in one or more removable mounting locations with one or more actual position errors within one or more position error tolerances; calibrating the one or more component devices based on one or more calibration-related parameters to generate a calibration offset to compensate for the one or more actual position errors within the one or more position error tolerances, wherein the one or more calibration-related parameters are identified from other attached component devices in the wearable device that have been calibrated; Based at least in part on the calibration offset, media content is presented to a viewer of the wearable device. The method of claim 1 , wherein the one or more component devices are calibrated using a master-agent model. The method of claim 1 , wherein the one or more component devices are calibrated using a peer model.

4. One or more non-transitory computer-readable storage media storing software instructions that, when executed by one or more processors, cause the method of claim 1 to be performed.

Images