Methods and apparatus for sensor HUB for modular augmented reality (AR) systems

Abstract

In one or more embodiments, the sensors for a head-mounted display (HMD) system can be configured to be included in a modular sensor array that is separate from the display device (e.g., AR glasses). This allows the modular sensor array to be switched out with a different modular sensor array without having to modify or alter the display device.

Description

METHODS AND APPARATUS FOR SENSOR HUB FOR MODULAR AUGMENTEDREALITY (AR) SYSTEMSCROSS REFERENCE TO RELATED APPLICATION

[0001] This application claims priority to U.S. Patent Application No. 63 / 656,570, filed June 5, 2024 and titled “METHODS AND APPARATUS FOR SENSOR HUB FOR MODULAR AUGMENTED REALITY (AR) SYSTEMS,” the contents of which are incorporated by reference herein in their entirety.FIELD

[0002] The present disclosure relates to the field of display devices such as augmented reality (AR) glasses generally, and more specifically to a sensor hub for a display device such as AR glasses.BACKGROUND

[0003] Modularity is desirable in mission critical and ruggedized hardware including, for example, display devices such as augmented reality (AR) displays (e.g., AR glasses or goggles). Such displays, however, are often configured to be used with different types of sensors in fixed customized arrangements that collectively increase the overall weight of the displays and increase the user’s discomfort. When the sensors and the display are not placed on the same rigid structure, however, they can move with respect to each other. Such relative movement between the sensors and the display can cause discomfort especially if the movements are recurrent such as walking or riding in a vehicle. Further, placing sensors at AR glasses or goggles can increase a weight or heat at the AR glasses or goggles.

[0004] Some known solutions, such as HoloLens®, Magic Leap®, and known integrated visual augmentation systems are built v / ith sensors integrated into the headset. These sensors add weight to the headset, and are often specialized for a specific application, limiting effectiveness for different use cases.

[0005] Thus, a need exists to provide a modular sensor hub that has a universal housing to allow for a diverse range of sensors.SUMMARY

[0006] In one or more embodiments, the sensors for a head-mounted display (HMD) system can be configured to be included in a modular sensor array that is separate from the display device (e.g., AR glasses). This allows the modular sensor array to be switched out with a different modular sensor array without having to modify or alter the display device.BRIEF DESCRIPTION OF THE DRAWINGS

[0007] FIG . l is a block diagram of a system that includes a display device and a modular sensor array, according to an embodiment.

[0008] FIG. 2 is a top view representation of a modular sensor array, according to an embodiment.|0009] FIG. 3 is a front view representation of the modular sensor array of FIG. 2.

[0010] FIG. 4 is a rear view representation of the modular sensor array of FIG. 2.

[0011] FIG. 5 is a top view representation of a modular sensor array, according to another embodiment.

[0012] FIG. 6 is a front view representation of the modular sensor array of FIG. 5.

[0013] FIG. 7 is a top view representation of a modular sensor array, according to yet another embodiment.

[0014] FIG. 8 illustrates a modular architecture for an AR system, according to an embodiment.

[0015] FIG. 9 illustrates another modular architecture for an AR system, according to an embodiment.

[0016] FIG. 10 illustrates two versions of an image, each with a different depth of focus, according to an embodiment.

[0017] FIG. 11 illustrates various communication techniques that can be used, according to an embodiment.

[0018] FIG. 12 illustrates a perspective view of an AR system, according to an embodiment.

[0019] FIG. 13A illustrates a perspective view of a sensor hub when a cover is lifted, according to an embodiment.

[0020] FIG. 13B illustrates a perspective view of the sensor hub when the cover is down, according to an embodiment.

[0021] FIG. 14A illustrates a sensor hub debugging architecture, according to an embodiment.

[0022] FIG. 14B illustrates debugging architecture for a sensor hub and / or back mount, according to an embodiment.

[0023] FIG. 14C illustrates another debugging architecture for a sensor hub and / or back mount, according to an embodiment.

[0024] FIG. 14D illustrates a debugging architecture for a sensor hub, back mount, puck, and / or AR glasses, according to an embodiment.

[0025] FIGS. 15A illustrates an AR system architecture where the sensor hub is on standby, according to an embodiment.

[0026] FIG. 15B illustrates an AR system architecture where the sensor hub is not on standby, according to an embodiment

[0027] FIG. 16 illustrates a flowchart of a method 1700 to generate an output based on sensor data captured by a sensor hub and an HMD, according to an embodiment.DETAILED DESCRIPTION

[0028] As mentioned above, the modularity is often desirable for ruggedized hardware such as augmented reality (AR) displays (e.g., AR glasses or goggles). More specifically, the sensors (also referred to herein as a “modular sensor array” or “sensor array”) can be in a modular form factor that allows the modular sensor array to be changed (switched out or detached / replaced) depending on the use case. To provide such a modular form factor, however, the modular sensor array may not be rigidly coupled to the display device. When the sensors and the display device are not placed on the same rigid structure, they can move with respect to each other. Such relative movement between the sensors and the display can cause discomfort especially if the movements are recurrent such as walking or riding in a vehicle.

[0029] In some implementations, a sensor hub serves as a centralized platform housing one or more different sensors that are connected to an onboard processor for edge processing. The sensor hub is positioned independently from the AR glasses. The AR glasses track the sensor hub and data from the sensor hub is delivered to the AR glasses (e.g., to display the captured sensor data at the AR glasses). By separating the AR device and the sensors at the sensor hub, a weight of the AR device is reduced, enhancing usercomfort and usability. This modular approach further allows for the sensor hub to be customized to be used with AR devices at different applications, since the sensor hub can be equipped with different sets of sensors for different applications (e.g., a sensor hub used for indoor navigation can use LiDAR for depth mapping, IMUs for motion tracking, and RGB cameras for object recognition, while in outdoor navigation, the sensor hub could use GPS / GNSS for geolocation, magnetometers for orientation, and barometers for altitude tracking). Further, sensor upgrades can be implemented by updating the sensor hub and not the AR device itself. In some implementations, “set” refers to one or more.

[0030] FIG. 1 is a block diagram of a system that includes a display device and a modular sensor array, according to an embodiment. As shown in FIG. 1, the system includes a display device 1 10, a modular sensor array 120, a helmet mount hinge 130, and a helmet 140. The modular sensor array 120 is rigidly coupled (rigidly mechanically coupled) to the helmet 140 via the helmet mount hinge 130. The display device 110 is not rigidly connected (not mechanically coupled) to the helmet 140 (or modular sensor array 120 or helmet mount hinge 130), although the display device 1 10 and the helmet 140 can be worn by a common user. The display device 110 and the modular sensor array 120 can be coupled to and in communication with each other, for example, via a cable (not shown in FIG. I).

[0031] The display device 110 can be, for example, small display optic in front of one eye or each eye (collectively display 116), and can be worn on the head of the user. In some implementations, the display 1 16 is a virtual reality (VR) or augmented reality (AR) display to provide a virtual environment to the user. The display device 1 10 can be, for example, in the form of glasses (also referred to as “VR glasses” or “AR glasses”), goggles, headset, etc. The display device 110 can include a camera 118 that communicates with processor 112. In some implementations, the display device 1 10 can optionally include an inertial measurement unit (IMG) (not shown in FIG. 1). The camera 1 18 and / or the IMT can provide data to processor 1 12 to, for example, align the display device 1 10 with the modular sensor array 120.

[0032] The display device 110 further includes a processor 1 12 and memory 1 14. The processor 1 12 can be located, for example, in the display device 1 10 such as in the temple of AR glasses when embodying the display device 1 10. T he processor 1 12 (e.g., a central processing unit (CPU), a graphics processing unit (GPU), neural processing unit (NPU), and / or the like) can be, for example, a hardware-based integrated circuit (IC) or any other suitable processing device configured to run or execute a set of instructions or codes. Thememory 1 14 (e.g., a random-access memory (RAM), a hard drive, a flash drive, and / or the like) can store data, and / or code that includes instructions to cause the processor 112 to perform one or more processes or functions. The communication interface (e.g., a network interface card (NIC), a Wi-Fi® transceiver, a Bluetooth® transceiver, and / or the like) can be a hardware component that facilitates data communication between other components or devices such as camera 1 14 of display device 110 and modular sensor array 120.

[0033] Modular sensor array 120 (sometimes referred to herein as “sensor hub”) can have a housing (not shown in FIG. 1) that allows for a set of sensors 125. Such sensors 125 can include, for example, a combination of two or more sensors with two or more sensor types. For example, sensors 125 can include two of an IMU, an accelerometer, a gyroscope, a camera, a red-green-blue (RGB) camera, a low light camera, a thermal imager, a WiFi® sensor (or a WiFi® transceiver or a WiFi® receiver), a radar sensor, a magnetometer, etc. For example, for one use, the modular sensor array 120 can include one set of sensors from sensors 125; for another use, a different modular sensor array can include a different set of sensors. The modular sensor array 120 can include a conneclor(s) that allow for one modular sensor array to be switched out with another modular sensor array, for example, by disconnecting the connector(s) of the former and removing it, and adding and connecting the connector(s) of the latter. Similar to the camera 118 of display device 1 10, the sensors 125 of modular sensor array 120 can communicate with processor 122 and / or processor 1 12, as discussed in further details below.

[0034] Modular sensor array 120 also includes fiducials 127. Fiducials (also referred to as “fiducial markers”) are objects disposed on the sensor array 120 within the field of view of the camera 1 18 of the display device 110. The fiducials 127 are used as a point of reference in an alignment process to align the display 110 and the modular sensor array 120, as described in further details in U.S. Patent Application Serial No. 63 / 656,565, filed June 5, 2024 and entitled “Methods and Apparatus for Active Alignment Passthrough from Detachable Sensors and Head-Mounted Displays,”, which is incorporated herein by reference. The fiducials 127 can have, for example, a checkerboard type pattern (also referred to as a calibration target or a calibration checkerboard).

[0035] Helmet mount hinge 130 can be any type of mechanical connector that moveably connects the modular sensor array 120 to the helmet 140. For example, the helmet mount hinge 130 can be rotatably coupled (attached) the modular sensor 120 and / or the helmet 140. This allows the user to move the modular sensor array 120 into and out ofposition relative to the user’s eyes / vision. Although the helmet mount hinge 130 is moveably coupled to the modular sensor 120 and / or the helmet 140, the coupling is rigid so that the modular sensor 120, helmet mount hinge and the helmet 140 move together collectively under normal / expected operational conditions.

[0036] Helmet 140 can be any type of protective head covering on to which the helmet mount hinge 130 can be coupled such that the modular sensor array 120 can be positioned as desired by the user. The compute device 150 can be optionally mounted to / coupled to the helmet 140.

[0037] FIG. 2 is a top view representation of a modular sensor array, according to an embodiment. As shown in FIG. 2, the modular sensor array includes a camera 1 and sensors 2 and 3 disposed at least partially within a housing 7. The camera 1 and sensors 2 and 3 each include an optical subsystem (optical train) and can operate in different spectral bands and / or cover different fields of view (FOVs). The modular sensor array can also include a light source 6, such as a light emitting diode (LED) that emits energy in the visible or infrared (IR) spectral bands, disposed at least partially within the housing 7. Sensors 5 can be sensors that do not use an optical subsystem (optical train) and can be used for example to estimate the position of the modular sensor array and track it relative to the display device (not shown in FIG. 2). Sensors 5 can be, for example, two or more of an 1MU, a magnetometer, an accelerometer, and / or a gyroscope. The modular sensor array can also include a processor 4, which can be an on-board processor configured to perform edge processing associated with feeds from the camera and / or sensors (i.e., the feeds from camera 1 and / or sensors 2, 3 and 5). In other words, camera 1 and / or sensors 2, 3 and 5 can send its captured data to processor 4 for processing such as preparing the data for integration into content displayed on the display device (e.g., AR glasses).

[0038] FIG. 3 is a front view representation of the modular sensor array of FIG. 2. As shown in FIG. 3, each of the camera 1, the sensors 2 and 3, and the light source 6 can have an aperture, an outer lens(es) and / or surface(s) that is formed by or at least partially external from the housing 7 on the front face (surface) of the modular sensor array.

[0039] FIG. 4 is a rear view representation of the modular sensor array of FIG. 2. As shown in FIG. 4, the rear view (back side) of the modular sensor array can include fiducials 8. The example of FIG. 4 shows two fiducials 8 on the left side of the back side of the modular sensor array and two fiducials 8 on the right side of the back side of the modular sensor array. These fiducials can be passive and in the form of a checkerboard. In thisexample, the outermost two fiducials 8 are mirror images of each other; the innermost two fiducials 8 are also mirror images of each other. These fiducials 8 can be viewed by cameras disposed on the AR glasses (e.g., cameras 118 of display device 110 of FIG. 1) during a head tracking process.

[0040] FIGS. 5 and 6 are a top view representation and a front view representation, respectively, of a modular sensor array, according to another embodiment. As shown in FIGS. 5 and 6, the modular sensor array includes a low light camera 510 between two thermal sensors 520 and 530 (also referred to as “thermal cameras”). In this example, one thermal sensor (e.g., thermal sensor 520) can have a wide FOV (e.g., a fisheye thermal sensor) and the other thermal sensor (e.g., thermal sensor 530) can have a narrow FOV (e.g., a narrower FOV than the FOV of thermal sensor 520). As best shown in FIG 5, the modular sensor array includes a processor board 540 with one or more processors (e.g., CPUs, GPUs, NPUs, and / or etc.), and a connector 550. The connector 550 can be, for example, a USB or serial connector that can receive a cable (not shown) that can also connect to the display device (e.g., AR glasses such as the display device 110 shown in FIG. 1) to provide data communications between the modular sensor array and the display device. Such data communications can allow, for example, for the data prepared by the processors) of the modular sensor array to be provided to the display device for integration into content to be displayed on the display device. The modularity of the modular sensor array allows for not only different sets of sensors but also different processor boards (e.g., with different processors) to be included in different modular sensor arrays that can be switched into / upgraded for the overall system without the need to alter or modify the display device such as AR glasses.

[0041] FIG. 7 is a top view representation of a modular sensor array, according to yet another embodiment. As shown in FIG. 7, the modular sensor array can have an arrangement with fewer apertures albeit with a thicker dimension. More specifically, the modular sensor array of FIG. 7 has two apertures 710, 715. Aperture 715 is optically coupled to detector 740, and aperture 710 is optically coupled to two detectors 730, 735. The light or energy that enters aperture 715 is directed to detector 740. As the light or energy enters aperture 710, the light / energy is split by beamsplitter 720, for example a 50 / 50 split with approximately half of the light / energy being redirected to detector 730 and approximately half of the light / energy passes through to detector 735. In this way, a singleaperture can be associated with multiple detectors, reducing the number of apertures of the modular sensor array.[0042} In use, the processor of the display device (e.g., AR glasses) receives data from the processor of modular sensor array (e.g., processor 4 shown in FIG. 2) and the images of the fiducials (e.g., fiducials 7 of FIG. 4) received by the cameras of the display device to estimate the position and the orientation of the modular sensor array relative to the display device. This position and orientation can be used to estimate the position of the modular sensor array and track it relative to the display device. At a later time, the modular sensor array can be removed from the overall system and replaced with a different modular sensor array without having to modify or alter the display device.

[0043] FIG. 8 illustrates a modular architecture for an AR system, according to an embodiment. FIG. 8 illustrates an HMD 802, communication hub 810, sensor hub 806, compute device 808 (optional), and battery pack 804 (optional).

[0044] The HMD 802 can be a wearable visual interface designed to provide an immersive viewing experience by positioning one or more display screens in front of the user's eyes. The HMD 802 can include one or more screens enclosed within a headset, along with optics that focus and direct the image to the user’s field of view. The HM D 802 can also include one or more sensors to capture sensor data related to the HMD 802, a user wearing the HMD 802, and / or a scene surrounding the HMD 802, such as an inertial measurement unit, gyroscope, camera, magnetometer, and / or the like. Additional details related to the HMD 802 can be found in U.S. Patent Application No. 19 / 049,754, filed February 10, 2025 and titled “Apparatus and Method to Determine a Pose of a Head Mounted Display,” the contents of which are incorporated by reference herein in their entirety.

[0045] The communication hub 810 serves as the interface for external communication. For example, where the sensor hub 806 or HMD 802 lacks a built-in radio, the sensor hub 806 or HMD 802 rely on the communication hub 810 for wireless connectivity. The communication hub 810 can be configured to transmit and receive data via various communication protocols, such as Wi-Fi®, Bluetooth®, or other wireless technologies.

[0046] The sensor hub 806 can be similar in function and / or structure as sensor hub 120. For example, rather than include a sensor at the HMD 802, which can increase the weight and heat at the HMD 802, the sensor can be included in the sensor hub 806. The sensor hub 806 can communicate with the HMD 802 via the communication hub 810.

[0047] In some implementations, the sensor hub 806 receives data that includes (1) sensor data collected by sensors at the sensor hub 806 and / or (2) sensor data collected by sensors at the HMD 802. The received data can then be processed at the sensor hub 806 to generate an output (e.g., a noise-adjusted image, a color-corrected image, a depth-corrected image, a local ized image, a mapped image, etc.), and the sensor hub 806 can send the output to the HMD 802 via the communication hub 810 (e.g., for display at the HMD 802). By generating the output at the sensor hub 806, rather than performing such processing at the HMD 802, heat at the HMD 802 is reduced.

[0048] In some implementations, the sensor hub 806 is configured to perform eye tracking and depth estimation. Eye tracking systems (e.g., at HMD 802 or sensor hub 806) can capture real-time data about the user's gaze, including metrics like vergence angles and pupillary responses. This information can be used to determine (e.g., at sensor hub 806) the user's intent and the depth plane the user is focusing on. Depth estimation complements this process by using visual data collected from one or more cameras, which is then analyzed by arti ficial intelligence (Al) models to calculate the physical depth of objects in the user's field of view (FOV). By comparing the user’s focal depth, derived from eye tracking data, with the physical depth of these objects (e.g., estimated using the Al models), the system can provide feedback and adjust the rendering (e.g., at the HMD 802) to better match the user's perspective. Additional details are discussed at U.S. Patent Application No. 63 / 787,518, filed April 1 1 , 2025 and titled “Reprojection Using Eye Tracking and Depth Estimation for an Extended Reality Device,” the contents of which are incorporated by reference herein in their entirety.

[0049] In some implementations, the sensor hub 806 is configured to perform localization and mapping using, for example, a main device including a first camera and a secondary or auxiliary device including a second camera, where the first camera has a wider field of view (FOV) than the second camera. The FOV of a camera can also be represented as a frustum of a camera in the context of a virtual three-dimensional (3D) space. As such, the main device can have a wide FOV while the auxiliary device can have a narrow FOV. In some embodiments, the main device can be an HMD (e.g., HMD 802). The auxiliary device can be physically uncoupled and free roaming from the HMD (e.g., sensor hub 806, compute device 808, battery pack 804, communication hub 810), which can make it difficult to estimate the relative pose of the auxiliary device relative to the main device, especially in conditions including low light, where no active illumination or radiation canbe used, with sparse visual features, where line-of-sight between the main device and auxiliary device is not provided, where the main device and the auxiliary device have different application processors, and / or where temporary failure of the main device or the auxiliary device is to be tolerated. In some embodiments, systems, devices, and methods described herein can be configured to perform relative pose estimation of the main device and auxiliary device, which can be used to track and illustrate the orientation (also referred to herein as an angle or a rotation) of the auxiliary device relative to the main device (e.g., the HMD) and / or to visually indicate the region of overlap between the view from the main device and the view from the auxiliary device. Additional details are discussed at U.S. Patent Application No. 19 / 204,418, filed May 9, 2025 and titled “Multi -Camera High Speed Simultaneous Localization and Mapping (Slam) for a Head Mounted Display,” the contents of which are incorporated by reference herein in their entirety.

[0050] In some implementations, the sensor hub 806 is configured to provide, to the HMD, colored image data in low light conditions. In some implementations, the sensor hub 806 receives images captured by a camera of an HMD (e.g., HMD 802). The images could be, for example, video frames captured by the camera of a person wearing the HMD while performing a task. The video frames can be captured at different poses / orientations of the HMD and / or user’s head. Objects at the scene can be identified (e.g., at the sensor hub 806) based on the video frames, and different objects can be assigned different colors such that if a given object appears in multiple video frames, that object has the same color in each of those video frames when displayed at the HMD. For example, if a first and second video frame both include the same desk and chair, the HMD can display the desk using a first color (e.g., pink) in the first and second video frames w'hile the HMD can display the chair using a second color (e.g., purple) in the first and second video frames. As an orientation of the HMD modifies, the user can view' color images at the HMD that reflect the modified fields of view. Additional details are discussed at U.S. Patent Application No. 19 / 072,793, filed March 6, 2025 and titled “Color Imagery in Extremely Low Light Conditions for a Head Mounted Display,” the contents of which are incorporated by reference herein in their entirety.

[0051] In some implementations, agents (e.g., individuals, robots, drones, vehicles, etc.) are operated and navigated in global positioning system (GPS) denied environments. Thus, some implementations use an image encoder and a location encoder to train on a variety of images, including visible and thermal imagery. The image encoder can extractfeatures from these images, while the location encoder maps them to location pairs, creating a shared latent space for improved positioning. A location decoder then predicts a coarse location based on image data, refining positional accuracy. Additionally, a machine learning model, multimodal model, and fine-tuning module can work together to integrate sensor data, improving location precision step by step. The sensor hub 806 connected to the HMD 802 can perform these tasks by gathering visible and thermal images from onboard sensors. The sensor hub 806 processes these images using the image encoder, while additional positional data from motion sensors feed into the location encoder. The sensor hub refines coarse location estimates using multimodal fusion techniques, ultimately improving accuracy. The fine-tuning module further enhances positional tracking, ensuring that the HMD 806 delivers precise augmented reality positioning and navigation in dynamic environments (e.g., GPS denied environments). Additional details are discussed at U.S. Patent Application No. 17 / 178,684, filed April 14, 2025 and titled “Methods and Apparatus for Operation and Navigation of Agents in Global Positioning System (GPS) Denied Environments,” the contents of which are incorporated by reference herein in their entirety.

[0052] The compute device 808 can be an anchor implemented as a portable device, such as, for example, in the form factor of a puck or a mushroom. The compute device 808 can be securely attached to moving transports, such as cars or planes, and aid in tracking and stabilization. The compute device 808 can include an attachment mechanism for rigid mounting. The compute device 808 can include sensors, such as an IMU, accelerometer, gyroscope, and cameras, to measure motion-related data, while light projectors emit patterned illumination on surrounding surfaces to enhance head pose determination for the HMD 802. These light patterns provide visual references in low-light or featureless environments, allowing the HMD 802 to capture and process images efficiently. The compute device 808 can be communicatively coupled to the HMD 802 (e.g., via communication hub 810) and transmit IMU data, enabling real-time head tracking and improved spatial awareness. Additional details related to the compute device 808 can be found in U.S. Patent Application No. 19,049,754, filed February 10, 2025 and titled “Apparatus and Method to Determine a Pose of a Head Mounted Display,” the contents of which are incorporated by reference herein in their entirety.

[0053] The battery pack 804 can be a portable energy storage device designed to supply electrical power to the HMD 802, communication hub 810, sensor hub 806, and compute device 808. In some implementations, the battery pack 804 includes multiple rechargeableand / or non-rechargeable battery cells housed within a protective enclosure. The battery pack 804 can integrate circuitry for voltage regulation, charge management, and safety protections, such as overcurrent and temperature monitoring.

[0054] The HMD 802, communication hub 810, sensor hub 806, compute device 808, and battery pack 804 can be operatively coupled via a bidirectional communication protocol that enables exchange of both data and power. This protocol, which can be e.g., USB 3.2 or USB Power Delivery (USB PD), allows the HMD 802, communication hub 810, sensor hub 806, compute device 808, and battery pack 804 to each function as both a sender and receiver of power and data, improving efficiency and reducing system complexity. The communication protocol can be a standardized / interoperable communication protocol, meaning that any of the HMD 802, communication hub 810, sensor hub 806, compute device 808, and / or battery pack 804 can be swapped out or replaced with a different device without modifying the overall communication structure. As long as the replacement device adheres to the same protocol standards, the replacement device can integrate into the system and continue exchanging power and data without compatibility issues. This universality allows for scalability, future upgrades, and modular design flexibility, ensuring that the system remains adaptable to various use cases or updates.

[0055] In some implementations, the sensor hub 806 serves as the primary host, directly processing data captured by the sensor hub’s 806 onboard sensor(s). This approach reduces latency by handling sensor processing remotely from the HMD 802, which helps to reduce heat generation and weight in the HMD 802. As illustrated at FIG. 8, the HMD 802 is connected to both the communication hub 810 and battery pack 804, while the communication hub 810 is connected to the HMD 802, sensor hub 806, battery pack 804, and compute device 808. The sensor hub 806 is connected to both the battery pack 804 and communication hub 810. The battery pack 804 is connected with the sensor hub 806, communication hub 810, compute device 808, and HMD 802. In some implementations, in the event that the sensor hub 806 becomes unavailable (e.g., disconnected, broken, off, etc.), the compute device 808 is configured to assume the host role.

[0056] In some implementations, the HMD 802, communication hub 810, sensor hub 806, compute device 808, and battery pack 804 are connected using a standardized cable. These cables are designed to support a bidirectional communication protocol (such as USB 3.2 / USB Power Delivery) that facilitates both data transfer and power delivery between components. In some implementations, each cable has a 22-pin connector. Each of the 22pins is assigned to a specific function(s), such as transmitting differential data signals, supplying power, grounding, and handling control or configuration tasks.

[0057] In some implementations, the HMD 802 is physically (mechanically) coupled to the sensor hub 806 via the communication hub 810, and the HMD 802 is communicatively (electrically) coupled to the sensor hub 806 using the communication protocol (e.g., USB 3.2 or PD). In some implementations, the HMD 802, communication hub 810, sensor hub 806, compute device 808, and / or battery pack 804 are physically distinct and separate from each other (e.g., but for the connecting cable). For example, moving the HMD 802 will not cause the communication hub 810 or sensor hub 806 to move (as long as the connecting cable has slack), moving the communication hub 810 will not cause the HMD 802 or sensor hub 806 to move (as long as the connecting cable has slack), and moving the sensor hub 806 will not cause the HMD 802 or the communication hub 810 to move (as long as the connecting cable has slack).

[0058] FIG. 9 illustrates another modular architecture for an AR system, according to an embodiment. FIG. 9 illustrates AR glasses 902, a back mount 904, a sensor hub 906, a body mounted compute device 910, conformal wearable battery (CWB) 912, and controller 908. The AR glasses 902 and back mount 904 can exchange data with each other using USB 3.2, and the back mount 904 can provide power to the AR glasses 902 via a universal serial bus type -C (USB-C) cable. The back mount 904 and sensor hub 906 can exchange data with each other using USB 3.2, and the back mount 904 can provide power to the sensor hub 906 via a USBC cable. The body mounted compute device 910 and back mount 904 can exchange data with each other using USB 3.2, and the body mounted compute device 910 can provide power to the back mount 904 via a USBC cable. The CWB 912 and body mounted compute device 910 can be connected via a system management bus (SMB US), and the CWB 912 can provide power to the body mounted compute device 910. The controller 908 and body mounted compute device 910 can exchange data with each other using USB 3.2, and the body mounted compute device 910 can provide power to the controller 908 via a USBC cable.

[0059] AR glasses 902 can be similar in function and / or structure as the HMD 802 of FIG. 8. The back mount 904 can be similar in function and / or structure as the communication hub 810 of FIG. 8. The sensor hub 906 can be similar in function and / or structure as the sensor hub 806 of FIG. 8. Controller 908 can be similar in function and / orstructure as compute device 808 of FIG. 8. CWB 912 can be similar in function and / or structure as battery pack 804 of FIG. 8.[0060J FIG. 10 illustrates two versions of an image, each with a different depth of focus, according to an embodiment. In some implementations, images are captured (e.g., by a sensor at the sensor hub) and a processor (e.g., at the sensor hub) can process the image to focus the image. Specifically, the processor can focus the image such that the image is focused on the portion of the image that the user is viewing. For example, eye-tracking sensors can detect the user’s gaze direction. An artificial intelligence model can then extract the area of interest and calculate depth information from the image. Based on the gaze direction, the area of interest, and the depth information, the camera’s focus can be adjusted. Additional details related to eye-tracking and determining depth information are discussed in U.S. Patent Application No. 63 / 787,518, filed April 1 1 , 2025 and titled “Reprojection Using Eye Tracking and Depth Estimation for an Extended Reality Device,” the contents of which are incorporated by reference herein in their entirety.

[0061] FIG. 11 illustrates various communication techniques that can be used, according to an embodiment. In some implementations, the communication hub 810 of FIG.8 is configured to communicate using, for example, a local network (e.g., Wifi® 6, Bluetooth® 5.4, etc.), iridium short burst data (SBD), cellular (e.g., private 5G, public 5G, LTE, 3G, etc.), secure communications (e.g., Silvus, PS)., navigation satellites (e.g., dualban L1 / L5 GNSS, GPS, GLONASS, BeiDou, Galileo, QZSS, etc.), and / or the like. Communication hub 810 being configured to use / switch between multiple communication techniques ensures reliable connectivity in areas with weak or no local networks, allowing devices to switch between satellite, cellular, Wi-Fi®, and other systems while maintaining seamless communication in remote or unpredictable environments.

[0062] FIG. 12 illustrates a perspective view of an AR system, according to an embodiment. FIG. 12 includes a sensor hub 1302, HMD 1304, and compute device 1306. The HMD 1304 can be similar in function and / or structure as the HMD 802 of FIG. 8 or the AR glasses 902 of FIG. 9, and configured to be worn by a user at the user’s head. The sensor hub 1302 can be similar in function and / or structure as the sensor hub 806 of FIG. 8 or the sensor hub 906 of FIG. 9, and configured to be worn by the user at a non-head location (e.g., the shoulders, the back, the chest, etc.). The compute device 1306 can be coupled to the sensor hub 1302 and the HMD 1304, and can include or store, for example, the communication hub 810, the compute device 808, the battery pack 804, the back mount904, the body mounted compute device 910, the controller 908, and / or the CWB 912. The compute device 1306 is configured to be worn by the user at a non-head location (e.g., the shoulders, the back, the chest, etc.). The sensor hub 103 can be coupled to the compute device 1306 via the cables 1308, where the cables 1308 can transfer data and power (e.g., using USB 3.2, USB PD, etc.). Although not explicitly shown in FIG. 12, in some implementations, cable 1310 connects compute device 1306 to HMD 1304, where the cable 1310 also can transfer data and power using the same protocol as the cables 1308.

[0063] FIG. 13A illustrates a perspective view of a sensor hub when a cover is lifted, according to an embodiment. FIG. 13A illustrates that the cover 1402 is lifted (opened) such that the sensors 1406 are exposed and / or can be covered by the lenses 1404. FIG. 13B illustrates a perspective view of the sensor hub when the cover is down (closed), according to an embodiment. If, for example, the sensors 1406 include a camera, the cover 1402 can block the camera from capturing images through the cover 1402.

[0064] FIG. 14A illustrates a sensor hub debugging architecture, according to an embodiment. In an example, a power supply (e.g., 9 volt DC source, labeled as “DC 9V”) provides power to a debug board that is communicatively coupled to the sensor hub. The debug board is also connected to a compute device (e.g., PC) via a cable. The debug board acts as an intermediary, facilitating data exchange between the sensor hub and the compute device, enabling real-time debugging by transmitting sensor data and diagnostics to the compute device.

[0065] FIG. 14B illustrates debugging architecture for a sensor hub and / or back mount, according to an embodiment. A sensor hub is connected to a back mount via a virtual reality cable, and the back mount is connected to a compute device and a power supply (e.g., 9 volt DC source). In this debugging setup, the sensor hub communicates with the back mount via a virtual reality cable, allowing sensor data and system diagnostics to be transmitted from the sensor hub. The back mount serves as an intermediary, relaying data to the compute device for analysis and troubleshooting. The power supply ensures stable operation, enabling repeated (e.g., continuous, sporadic, periodic) monitoring and debugging of the sensor hub and AR glasses.

[0066] FIG. 14C illustrates another debugging architecture for a sensor hub and / or back mount, according to an embodiment. The debugging architecture at FIG. 14C is similar to the debugging architecture at FIG. 14B, but without the compute device and power supply.Instead, the back mount can be configured to perform debugging of the sensor hub without transmitting data to a separate compute device.

[0067] FIG. 14D illustrates a debugging architecture for a sensor hub, back mount, puck, and / or AR glasses, according to an embodiment. The sensor hub, glasses, and puck are each communicatively coupled to the back mount, where the back mount can receive data from the sensor hub, glasses, and / or puck and analyze the data (e.g., to perform debugging).

[0068] FIG. 15A illustrates an AR system architecture where the sensor hub is on standby, according to an embodiment. FIG. 15B illustrates an AR system architecture where the sensor hub is not on standby, according to an embodiment. FIGS. 15A and 15B illustrate a puck 1602, compute device 1604, battery pack 1606, sensor hub 1610, USB hub 1608, and AR glasses 1612. At FIG. 15A, the sensor hub 1610 is on standby and the puck 1602 is the USB host. At FIG. 15B, the sensor hub 1610 is the USB host and the puck 1602 is a USB device. The puck 1602 can be similar in function and / or structure as compute device 808 of FIG. 8 or controller 908 of FIG. 9. The compute device 1604 can be similar in function and / or structure as communication hub 810 of FIG. 8, back mount 904 of FIG. 9, and / or the body mounted compute device 910 of FIG. 9. The battery pack 1606 can be similar in function and / or structure as the battery pack 804 of FIG. 8 or the CWB 912 of FIG. 9. The sensor hub 1610 can be similar in function and / or structure as the sensor hub 806 of FIG. 8 or the sensor hub 906 of FIG. 9. The AR glasses 1612 can be similar in function and / or structure as the HMD 802 of FIG. 8 or the AR glasses 902 of FIG. 9. The USB host (the puck 1602 in FIG. 15A or sensor host 1610 in FIG. 15B) can act as the central controller that manages communication with connected devices. The USB hub can initiate data transfers, supplies power, and determine how devices interact within the system, and / or the like. The USB host can assign addresses to peripherals, determine connection parameters, handles protocols like data exchange and power distribution, and / or the like.

[0069] In an embodiment, a system includes an HMD (e.g., HMD 802) that includes a first set of sensors, a communication hub (e.g., communication hub 810) connected to the HMD via a predetermined communication protocol (e.g., USB 3.2, USB PD) configured to transfer data and power, and a sensor hub (e.g., sensor hub 806) connected (1) to the communication hub via the predetermined communication protocol and (2) the HMD via the communication hub. The sensor hub is configured to be a host of the predeterminedcommunication protocol. The sensor hub includes a second set of sensors different from the first set of sensors. The sensor hub is configured to request and receive sensor data collected by the first set of sensors from the HMD via the communication hub and the predetermined communication protocol. The sensor hub is configured to process the sensor data collected by the first set of sensors and the second set of sensors. In some implementations, the system further includes a battery pack (e.g., battery pack 804) connected to the sensor hub via the predetermined communication protocol, the communication hub via the predetermined communication protocol, and the communication hub via the predetermined communication protocol. In some implementations, the system further includes a compute device (e.g., compute device 808) connected to the communication hub via the predetermined communication protocol and the battery pack via the predetermined communication protocol. In some implementations, the sensor hub includes a plurality of light sources arranged in a predefined pattern, and the first set of sensors include a camera configured to capture an image of the plurality of light sources. In some implementations, the communication hub is connected to the HMD via a first 22-pin connector, and the communication hub is connected to the sensor hub via a second 22-pin connector. In some implementations, the HMD includes a cooling unit configured to reduce a temperature at the HMD.

[0070] FIG. 16 illustrates a flowchart of a method 1700 to generate an output based on sensor data captured by a sensor hub and an HMD, according to an embodiment. Tn some implementations, method 1700 is performed by a processor (e.g., processor of sensor hub 806).

[0071] At 1702, first sensor data captured by a first sensor included in a sensor hub (e.g., sensor hub 806) is received. The first sensor data could include, for example, environmental readings associated with the sensor hub and / or surrounding area of the sensor hub, such as temperature, depth mapping, or motion tracking from onboard sensors such as LiDAR, I Mils, or proximity detectors.

[0072] At 1704, second sensor data is received from an HMD (e.g., HMD 802). The HMD is coupled to the sensor hub (1) via a communication hub (e.g., communication hub 810) and (2) using a predetermined communication protocol (e.g., USB 3.2, USB PD, and / or the like) that is configured to transfer data and power. The second sensor data is captured by a second sensor included in the HMD. The sensor hub is a host of thepredetermined communication protocol. The second sensor data can include, for example, camera imagery, eye-tracking data, user interaction inputs, and / or the like.

[0073] At 1706, an output is generated based on the first sensor data and the second sensor data. In some implementations, the sensor hub analyzes the first sensor data and the second sensor data to generate the output. Analyzing can include, for example, overlaying AR visuals onto a live scene, modifying spatial awareness for navigation, detecting user gestures, improving visibility of images (e.g., adding color, changing contrast or brightness, modifying focus or depth, etc.), determining a location of the user / HMD / sensor hub, pose determination, eye-tracking, depth estimation, and / or the like.[0074} At 1708, a representation of the output is sent to the communication hub. In response to receiving the representation of the output, the communication hub can send the representation of the output to the HMD. In response to receiving the representation of the output, the HMD can display the output.

[0075] Some implementations of method 1700 further include determining that the sensor hub at least one of (1) is inactive or (2) will become inactive, and in response, sending a signal to cause a compute device to become the host instead of the sensor hub. The compute device is connected to the communication hub via the predetermined communication protocol.

[0076] In some implementations of method 1700, the sensor is a first sensor and the sensor hub is a first sensor hub. Some implementations of method 1700 further include replacing the first sensor hub with a second sensor hub that includes a second sensor not included in the first sensor hub, the second sensor hub connected to the HMD via the predetermined communication protocol via the communication hub.[0077} Some implementations of method 1700 further include activating a plurality of light sources at the sensor hub. The plurality of light sources can be arranged in a predefined pattern (e.g., constellation).

[0078] In some implementations of method 1700, the communication hub is connected to the HMD via a first 22-pin connector. The communication hub can be connected to the sensor hub via a second 22-pin connector.

[0079] All combinations of the foregoing concepts and additional concepts discussed herewithin (provided such concepts are not mutually inconsistent) are contemplated as being part of the subject matter disclosed herein. The terminology explicitly employedherein that also may appear in any disclosure incorporated by reference should be accorded a meaning most consistent with the particular concepts disclosed herein.

[0080] The drawings are primarily for illustrative purposes, and are not intended to limit the scope of the subject matter described herein. The drawings are not necessarily to scale; in some instances, various aspects of the subject matter disclosed herein may be shown exaggerated or enlarged in the drawings to facilitate an understanding of different features. In the drawings, like reference characters generally refer to like features (e.g., functionally similar and / or structurally similar elements).

[0081] The entirety of this application (including the Cover Page, Title, Headings, Background, Summary, Brief Description of the Drawings, Detailed Description, Embodiments, Abstract, Figures, Appendices, and otherwise) shows, by way of illustration, various embodiments in which the embodiments may be practiced. The advantages and features of the application are of a representative sample of embodiments only, and are not exhaustive and / or exclusive. Rather, they are presented to assist in understanding and teach the embodiments, and are not representative of all embodiments. As such, certain aspects of the disclosure have not been discussed herein. That alternate embodiments may not have been presented for a specific portion of the innovations or that further undescribed alternate embodiments may be available for a portion is not to be considered to exclude such alternate embodiments from the scope of the disclosure. It will be appreciated that many of those undescribed embodiments incorporate the same principles of the innovations and others are equivalent. Thus, it is to be understood that other embodiments may be utilized and functional, logical, operational, organizational, structural and / or topological modifications may be made without departing from the scope and / or spirit of the disclosure. As such, all examples and / or embodiments are deemed to be non-limiting throughout this disclosure.

[0082] Also, no inference should be drawn regarding those embodiments discussed herein relative to those not discussed herein other than it is as such for purposes of reducing space and repetition. For instance, it is to be understood that the logical and / or topological structure of any combination of any program components (a component collection), other components and / or any present feature sets as described in the figures and / or throughout are not limited to a fixed operating order and / or arrangement, but rather, any disclosed order is exemplary and all equivalents, regardless of order, are contemplated by the disclosure.

[0083] The term “automatically” is used herein to modify actions that occur without direct input or prompting by an external source such as a user. Automatically occurringactions can occur periodically, sporadically, in response to a detected event (e.g., a user logging in), or according to a predetermined schedule.[0084} The term “determining” encompasses a wide variety of actions and, therefore, “determining” can include calculating, computing, processing, deriving, investigating, looking up (e.g,, looking up in a table, a database or another data structure), ascertaining and the like. Also, “determining” can include receiving (e.g., receiving information), accessing (e.g., accessing data in a memory) and the like. Also, “determining” can include resolving, selecting, choosing, establishing and the like.

[0085] The phrase “based on” does not mean “based only on,” unless expressly specified otherwise. In other words, the phrase “based on” describes both “based only on” and “based at least on.”

[0086] The term “processor” should be interpreted broadly to encompass a general purpose processor, a central processing unit (CPU), a microprocessor, a digital signal processor (DSP), a controller, a microcontroller, a state machine and so forth. Under some circumstances, a “processor” may refer to an application specific integrated circuit (ASIC), a programmable logic device (PUD), a field programmable gate array (FPGA), etc. The term “processor” may refer to a combination of processing devices, e.g., a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a DSP core or any other such configuration.

[0087] The term “memory” should be interpreted broadly to encompass any electronic component capable of storing electronic information. The term memory may refer to various types of processor-readable media such as random access memory (RAM), readonly memory (ROM), non-volatile random access memory (NVRAM), programmable read-only memory (PROM), erasable programmable read only memory (EPROM), electrically erasable PROM (EEPROM), flash memory, magnetic or optical data storage, registers, etc. Memory is said to be in electronic communication with a processor if the processor can read information from and / or write information to the memory. Memory that is integral to a processor is in electronic communication with the processor.

[0088] The terms “instructions” and “code” should be interpreted broadly to include any type of computer-readable statement(s). For example, the terms “instructions” and “code” may refer to one or more programs, routines, sub-routines, functions, procedures, etc. “Instructions” and “code” may comprise a single computer-readable statement or many computer-readable statements.

[0089] Some embodiments described herein relate to a computer storage product with a non- transitory computer-readable medium (also can be referred to as a non-transitory processor-readable medium) having instructions or computer code thereon for performing various computer-implemented operations. The computer-readable medium (or processor- readable medium) is non-transitory in the sense that it does not include transitory propagating signals per se (e.g., a propagating electromagnetic wave carrying information on a transmission medium such as space or a cable). The media and computer code (also can be referred to as code) may be those designed and constructed for the specific purpose or purposes. Examples of non-transitory computer-readable media include, but are not limited to, magnetic storage media such as hard disks, floppy disks, and magnetic tape; optical storage media such as Compact Disc / Digital Video Discs (CD / DVDs), Compact Disc-Read Only Memories (CD-ROMs), and holographic devices; magneto-optical storage media such as optical disks; carrier wave signal processing modules; and hardware devices that are specially configured to store and execute program code, such as Application- Specific Integrated Circuits (ASICs), Programmable Logic Devices (PLDs), Read-Only Memory (ROM) and Random-Access Memory (RAM) devices. Other embodiments described herein relate to a computer program product, which can include, for example, the instructions and / or computer code discussed herein.

[0090] Some embodiments and / or methods described herein can be performed by software (executed on hardware), hardware, or a combination thereof. Hardware modules may include, for example, a general-purpose processor, a field programmable gate array (FPGA), and / or an application specific integrated circuit (ASIC). Software modules (executed on hardware) can be expressed in a variety of software languages (e.g., computer code), including C, C++, Java™, Ruby, Visual Basic™, and / or other object-oriented, procedural, or other programming language and development tools. Examples of computer code include, but are not limited to, micro-code or micro-instructions, machine instructions, such as produced by a compiler, code used to produce a web service, and files containing higher-level instructions that are executed by a computer using an interpreter. For example, embodiments may be implemented using imperative programming languages (e.g., C, Fortran, etc.), functional programming languages (Haskell, Erlang, etc.), logical programming languages (e.g., Prolog), object-oriented programming languages (e.g., Java, C++, etc.) or other suitable programming languages and / or development tools. Additionalexamples of computer code include, but are not limited to, control signals, encrypted code, and compressed code.[0091J Various concepts may be embodied as one or more methods, of which at least one example has been provided. The acts performed as part of the method may be ordered in any suitable way. Accordingly, embodiments may be constructed in which acts are performed in an order different than illustrated, which may include performing some acts simultaneously, even though shown as sequential acts in illustrative embodiments. Put differently, it is to be understood that such features may not necessarily be limited to a particular order of execution, but rather, any number of threads, processes, services, servers, and / or the like that may execute serially, asynchronously, concurrently, in parallel, simultaneously, synchronously, and / or the like in a manner consistent with the disclosure. As such, some of these features may be mutually contradictory, in that they cannot be simultaneously present in a single embodiment. Similarly, some features are applicable to one aspect of the innovations, and inapplicable to others.

[0092] In addition, the disclosure may include other innovations not presently described. Applicant reserves all rights in such innovations, including the right to embodiment such innovations, file additional applications, continuations, continuations-in- part, divisional, and / or the like thereof. As such, it should be understood that advantages, embodiments, examples, functional, features, logical, operational, organizational, structural, topological, and / or other aspects of the disclosure are not to be considered limitations on the disclosure as defined by the embodiments or limitations on equivalents to the embodiments. Depending on the particular desires and / or characteristics of an individual and / or enterprise user, database configuration and / or relational model, data type, data transmission and / or network framework, syntax structure, and / or the like, various embodiments of the technology disclosed herein may be implemented in a manner that enables a great deal of flexibility and customization as described herein.

[0093] All definitions, as defined and used herein, should be understood to control over dictionary definitions, definitions in documents incorporated by reference, and / or ordinary meanings of the defined terms.

[0094] As used herein, in particular embodiments, the terms “about” or “approximately” when preceding a numerical value indicates the value plus or minus a range of 10%. Where a range of values is provided, it is understood that each intervening value, to the tenth of the unit of the lower limit unless the context clearly dictates otherwise,between the upper and lower limit of that range and any other stated or intervening value in that stated range is encompassed within the disclosure. That the upper and lower limits of these smaller ranges can independently be included in the smaller ranges is also encompassed within the disclosure, subject to any specifically excluded limit in the stated range. Where the stated range includes one or both of the limits, ranges excluding either or both of those included limits are also included in the disclosure.

[0095] The indefinite articles “a” and “an,” as used herein in the specification and in the embodiments, unless clearly indicated to the contrary, should be understood to mean “at least one.”

[0096] The phrase “and / or,” as used herein in the specification and in the embodiments, should be understood to mean “either or both” of the elements so conjoined, i.e., elements that are conjunctively present in some cases and disjunctively present in other cases. Multiple elements listed with “and / or” should be construed in the same fashion, i.e., “one or more” of the elements so conjoined. Other elements may optionally be present other than the elements specifically identified by the “and / or” clause, whether related or unrelated to those elements specifically identified. Thus, as a non-limiting example, a reference to “A and / or B”, when used in conjunction with open-ended language such as “comprising” can refer, in one embodiment, to A only (optionally including elements other than B); in another embodiment, to B only (optionally including elements other than A); in yet another embodiment, to both A and B (optionally including other elements); etc.

[0097] As used herein in the specification and in the embodiments, “or” should be understood to have the same meaning as “and / or” as defined above. For example, when separating items in a list, “or” or “and / or” shall be interpreted as being inclusive, i.e., the inclusion of at least one, but also including more than one, of a number or list of elements, and, optionally, additional unlisted items. Only terms clearly indicated to the contrary, such as “only one of’ or “exactly one of,” or, when used in the embodiments, “consisting of,” will refer to the inclusion of exactly one element of a number or list of elements. In general, the term “or” as used herein shall only be interpreted as indicating exclusive alternatives (i.e. “one or the other but not both”) when preceded by terms of exclusivity, such as “either,” “one of,” “only one of,” or “exactly one of.” “Consisting essentially of,” when used in the embodiments, shall have its ordinary meaning as used i n the field of patent law.

[0098] As used herein in the specification and in the embodiments, the phrase “at least one,” in reference to a list of one or more elements, should be understood to mean at leastone element selected from any one or more of the elements in the list of elements, but not necessarily including at least one of each and every element specifically listed within the list of elements and not excluding any combinations of elements in the list of elements. This definition also allows that elements may optionally be present other than the elements specifically identified within the list of elements to which the phrase “at least one” refers, whether related or unrelated to those elements specifically identified. Thus, as a nonlimiting example, “at least one of A and B” (or, equivalently, “at least one of A or B,” or, equivalently “at least one of A and / or B”) can refer, in one embodiment, to at least one, optionally including more than one, A, with no B present (and optionally including elements other than B); in another embodiment, to at least one, optionally including more than one, B, with no A present (and optionally including elements other than A); in yet another embodiment, to at least one, optionally including more than one, A, and at least one, optionally including more than one, B (and optionally including other elements); etc.

[0099] In the embodiments, as well as in the specification above, all transitional phrases such as “comprising,” “including,” “carrying,” “having,” “containing,” “involving,” “holding,” “composed of,” and the like are to be understood to be open-ended, i.e., to mean including but not limited to. Only the transitional phrases “consisting of’ and “consisting essentially of’ shall be closed or semi-closed transitional phrases, respectively, as set forth in the United States Patent Office Manual of Patent Examining Procedures, Section 21 1 1.03.

Claims

CLAIMSWhat is claimed is:

1. A system, comprising: a head-mounted display (HMD) that includes a first set of sensors; a communication hub connected to the HMD via a predetermined communication protocol that is configured to transfer data and power; and a sensor hub connected to (1 ) the communication hub via the predetermined communication protocol and (2) the HMD via the communication hub, the sensor hub configured to be a host of the predetermined communication protocol, the sensor hub including a second set of sensors different than the first set of sensors, the sensor hub configured io request and receive sensor data collected by the first set of sensors from the HM D via the communication hub and the predetermined communication protocol, and to process the sensor data collected by the first set of sensors and the second set of sensors.

2. The system of claim 1 , further comprising: a battery pack connected to the sensor hub via the predetermined communication protocol, the communication hub via the predetermined communication protocol, and the communication hub via the predetermined communication protocol.

3. The system of claim 2, further comprising: a compute device connected to the communication hub via the predetermined communication protocol and the battery pack via the predetermined communication protocol.

4. The system of claim 1, wherein the predetermined communication protocol includes USB 3.2.

5. The system of claim 1 , wherein the predetermined communication protocol includesUSB power delivery (PD).

6. The system of claim 1 , wherein the sensor hub includes a plurality of light sources arranged in a predefined pattern, and the first set of sensors include a camera configured to capture an image of the plurality of light sources.

7. The system of claim 1, wherein the communication hub is connected to the HMD via a first 22-pin connector, and the communication hub is connected to the sensor hub via a second 22-pin connector.

8. The system of claim 1 , wherein the HMD includes a cooling unit configured to reduce a temperature at the HM D.

9. A method, comprising: receiving, at a processor of a sensor hub, first sensor data captured by a first sensor included in the sensor hub; receiving, by the processor, second sensor data from a head-mounted display (HMD) that is connected to the sensor hub (1 ) via a communication hub and (2) using a communication protocol that is configured to transfer data and power, the second sensor data captured by a second sensor included in the HMD, the sensor hub being a host of the predetermined communication protocol; generating, by the processor, an output based on the first sensor data and the second sensor data; and sending, by the processor, a representation of the output to the communication hub to cause the HMD to display the output.

10. The method of claim 9, further comprising: determining, at the processor, that the sensor hub at least one of (1) is inactive or (2) will become inactive; and in response to determining that the sensor hub at least one of (1) is inactive or (2) will become inactive, sending a signal to cause a compute device to become the host instead ofthe sensor hub, the compute device connected to the communication hub via the predetermined communication protocol.1 1. The method of claim 9, wherein the sensor is a first sensor and the sensor hub is a first sensor hub, the method further comprising: replacing the first sensor hub with a second sensor hub that includes a second sensor not included in the first sensor hub, the second sensor hub connected to the HMD via the predetermined communication protocol via the communication hub.

12. The method of claim 9, wherein the predetermined communication protocol includes USB 3.2.

13. The method of claim 9, wherein the predetermined communication protocol includes USB power delivery (PD).

14. The method of claim 9, further comprising: activating, by the processor, a plurality of light sources at the sensor hub, the plurality of light sources arranged in a predefined pattern.

15. The method of claim 9, wherein the communication hub is connected to the HMD via a first 22 -pin connector, and the communication hub is connected to the sensor hub via a second 22-pin connector.

16. A non-transitory, processor-readable medium storing instructions that when executed by a processor of a sensor hub cause the processor to: receive first sensor data captured by a first sensor included in the sensor hub; receive second sensor data from a head-mounted display (HMD) that is connected to the sensor hub (1) via a communication hub and (2) using a predetermined communication protocol that is configured to transfer data and power, the second sensor data captured by a second sensor included in the HMD, the sensor hub being a host of the predetermined communication protocol; generate an output based on the first sensor data and the second sensor data; andsend a representation of the output to the communication hub to cause the HMD to display the output.

17. The non-transitory, processor-readable medium of claim 16, storing further instructions that cause the processor further to: determine that the sensor hub at least one of (1) is inactive or (2) will become inactive; and in response to determining that the sensor hub at least one of (1) is inactive or (2) will become inactive, send a signal to cause a compute device to become the host instead of the sensor hub.

18. The non-transitory, processor-readable medium of claim 16, wherein the predetermined communication protocol includes USB 3.2.

19. The non-transitory, processor-readable medium of claim 16, wherein the predetermined communication protocol includes USB power delivery (PD).

20. The non-transitory, processor-readable medium of claim 16, storing further instructions that cause the processor further to: activate a plurality of light sources at the sensor hub, the plurality of light sources arranged in a predefined pattern.

Images