An electronic device for tracking objects

Abstract

Systems, methods, and non-transitory media for tracking motion using data received from a wearable device are provided. An example method can include determining a first position of the wearable device in a physical space, receiving location information related to the wearable device from the wearable device, determining a second position of the wearable device based on the received location information, and tracking movement of the wearable device relative to an electronic device based on the first and second positions.

Description

[Technical field]

[0001]

[0001] The present disclosure relates generally to tracking systems, and more particularly to electronic devices for tracking objects using data from wearable devices (e.g., for extended reality operations and / or power conservation). [Background technology]

[0002]

[0002] Extended reality (e.g., augmented reality, virtual reality, etc.) devices, such as smart glasses and head-mounted displays (HMDs), generally implement cameras and sensors to track the position of the extended reality (XR) device and other objects in a physical environment. The XR reality device can use the tracking information to provide a realistic XR experience to a user of the XR device. For example, the XR device can enable a user to experience or interact with an immersive virtual environment or content. To provide a realistic XR experience, the XR technology can integrate virtual content with the physical world, which can involve matching the relative pose and movement of objects and devices. The XR technology can use the tracking information to calculate the relative pose of the device, object, and / or map of the real-world environment to match the relative positions and movements of the device, object, and / or real-world environment and anchor the content to the real-world environment in a convincing manner. The relative pose information can be used to match the virtual content with the user's perceived movement and spatiotemporal state of the device, object, and real-world environment. Summary of the Invention

[0003]

[0003] Systems, methods, and computer-readable media for tracking objects and controlling operation, state, and / or settings of an electronic device are disclosed. For example, the electronic device can use / utilize data from a wearable device to implement tracking, power conservation, and / or various operations (e.g., extended reality operations, etc.). The electronic device can communicate with the wearable device to obtain data from the wearable device. In some examples, the wearable device can assist in tracking, power conservation, and / or other operations / settings of the electronic device. According to at least one example, a method is provided for tracking operations using data received from a wearable device. The method can include determining a first position of the wearable device in a physical space, receiving from the wearable device position information related to the wearable device, determining a second position of the wearable device based on the received position information, and tracking movement of the wearable device relative to the electronic device based on the first position and the second position.

[0004] According to at least one example, a non-transitory computer-readable medium for tracking movement using data received from a wearable device is provided. The non-transitory computer-readable medium may include instructions stored thereon that, when executed by one or more processors, cause the one or more processors to determine a first position of the wearable device in physical space, receive from the wearable device position information related to the wearable device, determine a second position of the wearable device based on the received position information, and track movement of the wearable device relative to an electronic device based on the first position and the second position.

[0005] According to at least one example, an apparatus for tracking movement using data received from a wearable device is provided. The apparatus may include a memory and one or more processors coupled to the memory, where the one or more processors are configured to: determine a first position of the wearable device in a physical space, receive from the wearable device position information related to the wearable device, determine a second position of the wearable device based on the received position information, and track movement of the wearable device relative to an electronic device based on the first position and the second position.

[0006] According to at least one example, another apparatus for tracking motion using data received from a wearable device is provided that may include means for determining a first position of the wearable device in physical space, receiving location information related to the wearable device from the wearable device, determining a second position of the wearable device based on the received location information, and tracking movement of the wearable device relative to an electronic device based on the first and second positions.

[0007]

[0007] In some aspects, the methods, non-transitory computer-readable media, and apparatus described above can determine whether the wearable device is within a field of view (FOV) of and / or visible to one or more image sensors on the electronic device based on the second position of the wearable device and / or the tracked movement of the wearable device. In some aspects, the methods, non-transitory computer-readable media, and apparatus described above can track a location of a hand associated with the wearable device based on the second position of the wearable device and / or the tracked movement of the wearable device. In some examples, the hand can include a hand wearing or holding the wearable device (e.g., on a wrist, on a finger, etc.).

[0008]

[0008] In some examples, determining a first position of the wearable device may include receiving from the wearable device image data from one or more image sensors on the electronic device and / or data related to one or more measurements from one or more sensors on the wearable device, and determining the first position of the wearable device based on the image data from the one or more image sensors and / or data related to the one or more measurements from the one or more sensors.

[0009] In some examples, the data may include a distance of the wearable device relative to one or more objects (e.g., a wall, a door, furniture, a device, a person, an animal, etc.), a velocity vector indicating a speed of movement of the wearable device, touch signals measured by touch sensors from one or more sensors, audio data from audio sensors from one or more sensors, and / or an altitude of the wearable device in physical space. In some cases, the one or more objects may include an electronic device, a body part associated with a user of the wearable device (e.g., a hand, a leg, an arm, a head, a torso, etc.), and / or an input device (e.g., a controller, a keyboard, a remote control, etc.).

[0010] In some examples, the location information may include an attitude of the wearable device. In some cases, the location information may be based on sensor data from one or more sensors on the wearable device. In some cases, the location information may include measurements from an inertial measurement unit from one or more sensors on the wearable device, and / or altitude measured by a pressure sensor from one or more sensors.

[0011] In some cases, the second location can include a location of the wearable device relative to the electronic device. In some cases, the second location can include a location of the wearable device in a coordinate system, such as a coordinate system of the wearable device and / or a coordinate system of the electronic device.

[0012]

[0012] In some aspects, tracking movement of the wearable device may include determining a first position of the wearable device in a first coordinate system of the wearable device, transforming the first coordinate system of the wearable device to a second coordinate system of the electronic device, and determining a second position of the wearable device in the second coordinate system of the electronic device.

[0013]

[0013] In some cases, the wearable device may include a bracelet, a ring, or a glove.

[0014] In some aspects, the methods, non-transitory computer-readable media, and apparatus described above may capture one or more images of a hand via at least one image sensor from one or more image sensors based on a determination that the wearable device is within the FOV of and / or visible to the one or more image sensors, and track a location of the hand based on the one or more images of the hand. In some examples, the location of the hand is tracked relative to a first coordinate system of the wearable device.

[0015] In some aspects, the methods, non-transitory computer-readable media, and apparatus described above can determine, based on the location information, that the wearable device is outside the FOV of one or more image sensors and moving toward an area within the FOV of the one or more image sensors, and initiate one or more imaging operations and / or one or more tracking operations at the electronic device based on determining that the wearable device is outside the FOV of the one or more image sensors and moving toward an area within the FOV of the one or more image sensors. In some examples, the one or more tracking operations can be based at least in part on image data from the one or more imaging operations.

[0016] In some aspects, the methods, non-transitory computer-readable media, and apparatus described above can adjust a first setting of a first image sensor based on a first determination that the wearable device is within a first FOV of a first image sensor on the electronic device and / or a second determination that the wearable device is visible to the first image sensor on the electronic device. In some cases, the first setting can include a power mode of the first image sensor and / or an operational state of the first image sensor. In some aspects, the methods, non-transitory computer-readable media, and apparatus described above can adjust a second setting of a second image sensor based on a third determination that the wearable device is outside a second FOV of a second image sensor on the electronic device and / or a fourth determination that the wearable device is not visible to the second image sensor on the electronic device. In some examples, the second setting can include a power mode of the second image sensor and / or an operational state of the second image sensor.

[0017] In some examples, adjusting the first setting of the first image sensor can include changing a power mode of the first image sensor from the first power mode to a second power mode including a higher power mode than the first power mode and / or changing an operating state of the first image sensor from the first operating state to a second operating state including a higher operating state than the first operating state. In some examples, the second operating state can include a higher frame rate and / or a higher resolution.

[0018] In some examples, adjusting the second setting of the second image sensor may include changing a power mode of the second image sensor from the first power mode to a second power mode including a lower power mode than the first power mode and / or changing an operating state of the second image sensor from the first operating state to a second operating state including a lower operating state than the first operating state. In some cases, the second operating state may include a lower frame rate and / or a lower resolution.

[0019] In some aspects, the methods, non-transitory computer-readable media, and apparatus described above can track the location of the wearable device based on additional position information from the wearable device in response to determining that the wearable device is outside the FOV of one or more image sensors on the electronic device and / or the view of the one or more image sensors to the wearable device is obstructed by one or more objects. In some aspects, the methods, non-transitory computer-readable media, and apparatus described above can track the location of the wearable device based on additional position information from the wearable device in response to determining that the wearable device is within the FOV of one or more image sensors but the view of the one or more image sensors to the wearable device is obstructed.

[0020]

[0020] In some aspects, the methods, non-transitory computer-readable media, and apparatus described above can initialize one or more image sensors in response to a determination that the wearable device is within the FOV of the one or more image sensors but that the view of the one or more image sensors to the wearable device is obstructed.

[0021] In some aspects, the methods, non-transitory computer-readable media, and apparatus described above can receive an input from a wearable device configured to trigger a privacy mode at the electronic device, and adjust an operational state of one or more image sensors at the electronic device to an off state and / or a disabled state based on the input configured to trigger the privacy mode. In some examples, the input can be based on sensor data from one or more sensors on the wearable device. In some cases, the sensor data can indicate a touch signal corresponding to a touch input at the wearable device, a location of the wearable device, and / or a distance between the wearable device and a body part of a user of the wearable device.

[0022] In some aspects, the methods, non-transitory computer-readable media, and apparatus described above can receive an additional input from the wearable device configured to trigger the electronic device to cease privacy mode. In some cases, the additional input can be based on a touch signal corresponding to a touch input at the wearable device, a location of the wearable device corresponding to a location of a body part of a user of the wearable device, and / or sensor data indicative of a proximity between the wearable device and the body part.

[0023] In some aspects, the methods, non-transitory computer-readable media, and apparatus described above can determine one or more extended reality (XR) inputs to an XR application on an electronic device based on data from the wearable device and / or commands from the wearable device. In some examples, the one or more XR inputs can include a modification of a virtual element along multiple dimensions in space, a selection of a virtual element, a navigation event, and / or a request to measure a distance defined by a first position of the wearable device, a second position of the wearable device, and / or a movement of the wearable device.

[0024] In some examples, the virtual element can include a virtual object rendered by the electronic device, a virtual plane in an environment rendered by the electronic device, and / or an environment rendered by the electronic device. In some examples, the navigation event can include scrolling the rendered content and / or moving from a first interface element to a second interface element.

[0025] In some aspects, the methods, non-transitory computer-readable media, and apparatus described above can receive an input from the wearable device configured to trigger an adjustment of one or more XR actions in the electronic device. In some examples, the one or more XR actions can include object detection, object classification, object tracking, pose estimation, and / or shape estimation. In some examples, the one or more sensors can include at least one of an accelerometer, a gyroscope, a pressure sensor, an audio sensor, a touch sensor, and a magnetometer.

[0026] In some aspects, the devices described above may include one or more sensors. In some aspects, the devices described above may include a wearable ring. In some aspects, the devices described above may include a mobile device. In some examples, the devices may include a hand controller, a mobile phone, a wearable device, a display device, a mobile computer, a head-mounted device, and / or a camera.

[0027]

[0027] This Summary is not intended to identify key or essential features of the claimed subject matter, nor is it intended to be used independently to determine the scope of the claimed subject matter. The subject matter should be understood by reference to the entire specification of this patent, any or all drawings, and appropriate portions of each claim.

[0028]

[0028] The foregoing, together with other features and embodiments, will become more apparent from the following specification, claims, and accompanying drawings.

[0029]

[0029] In order to explain the manner in which the various advantages and features of the present disclosure may be obtained, a more particular description of the principles described above will be rendered with reference to specific embodiments thereof as illustrated in the accompanying drawings. With the understanding that these drawings merely illustrate exemplary embodiments of the present disclosure and should not be considered as limiting its scope, the principles herein will be described and explained with additional specificity and detail through the use of the drawings. [Brief description of the drawings]

[0030] [Figure 1]

[0030] A diagram showing an example of an extended reality system and wearable device used for extended reality experiences and functionality, according to some examples of the present disclosure. [Figure 2A]

[0031] 1 illustrates an example of a ring device worn on a user's finger for interacting with an extended reality system, according to some examples of the present disclosure. [Figure 2B] 1 illustrates an example of a ring device worn on a user's finger for interacting with an extended reality system, according to some examples of the present disclosure. [Diagram 3]

[0032] A diagram showing an example process for integrating data from wearable devices for extended reality operations in an extended reality system, according to some examples of the present disclosure. [Figure 4]

[0033] 1 is a flow diagram illustrating an example process for using a wearable device in conjunction with an extended reality system (e.g., for improved tracking, power saving, and privacy features), in accordance with some examples of the present disclosure. [Figure 5A]

[0034] 1 is a flow diagram illustrating an example process for using a wearable device in conjunction with an extended reality device, according to some examples of the present disclosure. [Figure 5B]

[0035] 1 is a flow diagram illustrating an example process for tracking an object, in accordance with certain examples of this disclosure. [Figure 6]

[0036] FIG. 1 illustrates an exemplary computing device architecture, in accordance with some examples of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0031]

[0037] Some aspects and embodiments of the present disclosure are provided below. As will be apparent to those skilled in the art, some of these aspects and embodiments can be applied independently, and some of them can be applied in combination. In the following description, for the purpose of explanation, specific details are set forth to provide a thorough understanding of the embodiments of the present application. However, it will be apparent that various embodiments can be implemented without these specific details. The figures and descriptions are not limiting.

[0032]

[0038] The following description merely provides exemplary embodiments and is not intended to limit the scope, applicability, or configuration of the present disclosure. Instead, the following description of exemplary embodiments provides those skilled in the art with an enabling description for implementing the exemplary embodiments. It should be understood that various changes may be made in the function and arrangement of elements without departing from the spirit and scope of the present application, as set forth in the appended claims.

[0033]

[0039] As mentioned above, extended reality (e.g., augmented reality, virtual reality, etc.) devices, such as smart glasses and head mounted displays (HMDs), can implement cameras and sensors to track the position of the extended reality (XR) device and other objects in a physical environment. The XR reality device can use such tracking information to provide a realistic XR experience to a user of the XR device. For example, the XR device can enable a user to experience or interact with an immersive virtual environment or content. To provide a realistic XR experience, the XR technology can integrate the virtual content with the physical world. The XR technology can use the tracking information to calculate a relative pose of the device, object, and / or map of the real-world environment to match the relative positions and movements of the device, object, and / or real-world environment and anchor the content to the real-world environment in a convincing / realistic manner. The relative pose information can be used to match the virtual content with the user's perceived movement and spatiotemporal state of the device, object, and real-world environment.

[0034]

[0040] In some examples, an XR device may implement a tracking algorithm that uses sensor data to track the position of an object in three-dimensional (3D) space, such as a hand, finger, input device (e.g., controller, stylus, joystick, glove, etc.). For example, the tracking algorithm may use measurements from various sensors, such as image sensors and inertial measurement units (IMUs), the controller's pose (or data / measurements thereof), and / or a motion model for the user's hand(s) to estimate the pose of the hand / controller used by the XR device during the XR experience. In some cases, the tracking algorithm may use sensor data to predict the location of the user's hand(s), the location of the XR device's camera, and / or the state of the XR device. The XR device may use such measurements to measure the location of the XR device, the camera, and / or the user's hand(s) and update the state of the XR device.

[0035]

[0041] Often, an XR device may implement multiple cameras for tracking robustness. For example, an XR device may implement multiple cameras to track the full range of motion (e.g., up, down, left, right, etc.) of a hand (and / or any other object). Together, the multiple cameras can provide a full field of view (FOV) or a wider FOV in 3D space to capture (and track) an object from different relative positions in 3D space. While the multiple cameras implemented by an XR device and the tracking, detection, classification, etc. operations performed by the XR device can significantly aid the tracking, detection, and classification operations, they also use a significant amount of power and computational resources in the XR device. This can adversely affect the performance and more limited battery life of the XR device.

[0036]

[0042] In some cases, the camera used by the XR device may also raise privacy issues. For example, in some situations, a user of the XR device may not want the camera on the XR device to capture images or even be turned on in a particular scene / setting. In such cases, the user may want to turn off or disable the use of the camera on the XR device. However, turning off or disabling the use of the camera on the XR device may significantly limit the ability of the XR device to track / detect objects or even prevent the XR device from tracking / detecting objects, which may significantly affect or prevent the XR functionality / experience on the XR device.

[0037]

[0043] In some aspects, systems, apparatuses, processes (also referred to as methods), and computer-readable media (collectively referred to herein as "systems and techniques") for using a wearable accessory (or wearable device) to interface / interact with an XR device to assist in tracking, provide power savings, and / or increase user privacy during an XR experience are described herein. In some examples, the wearable accessory may be worn by a user of the XR device during an XR experience. The wearable accessory may be worn on the user's finger (or fingers), wrist, ankle, and / or any other body part. The wearable accessory may include built-in sensors configured to obtain measurements of the state (e.g., position, movement, etc.) of the wearable accessory in 3D space, and thus of the body part (e.g., finger, hand, etc.) to which the wearable accessory is worn. The wearable accessory may provide its measurements to the XR device, which may integrate with its tracking system for more robust tracking and accuracy.

[0038]

[0044] A wearable accessory may include, for example, but not limited to, a ring that may be worn on a finger, a sleeve of rings that may be worn on multiple fingers, a bracelet that may be worn on a wrist, a glove that may be worn on a hand, etc. For example, a wearable accessory may include a ring device worn on a user's finger. The ring device may include an embedded sensor that captures tracking measurements (e.g., position measurements, movement measurements, etc.) of the ring device (and thus the position of the user's finger / hand). The ring device may send the tracking measurements to the user's XR device, which may use the tracking measurements (with or without measurements separately obtained by the XR device) to track, detect, and / or classify the user's finger / hand. In some examples, tracking measurements from the ring device can help the XR device track the user's fingers / hands even when the user's fingers / hands are outside (or partially outside) the FOV of one or more cameras in the XR device, when lighting conditions in the scene hinder or limit the ability of the XR device to detect and / or image the user's fingers / hands, when the view of one or more cameras to the user's fingers / hands is obstructed by something, etc.

[0039]

[0045] In some cases, the wearable accessory can help reduce power consumption in the XR device by triggering the XR device to modify the operation of the cameras in the XR device and / or the tracking system used by the XR device. For example, if the wearable accessory is outside the FOV of one or more cameras on the XR device or if the view of one or more cameras to the wearable accessory is obstructed (e.g., by an object, poor lighting conditions, etc.), the XR device can use tracking measurements from the wearable accessory to reduce the processing mode of one or more cameras (e.g., turn off one or more cameras, reduce the frame rate of one or more cameras, reduce operation by one or more cameras, and / or possibly reduce power consumption by one or more cameras) to avoid unnecessary use of the one or more cameras and / or power consumption by the one or more cameras. The XR device can use tracking measurements to track and / or assist in tracking the wearable accessory while the one or more cameras are in the reduced processing mode. In some examples, the XR device can use the tracking measurements to track the wearable accessory, with or without other sensor data, such as, for example, image data from one or more cameras in a reduced processing mode and / or image data from one or more other cameras of the XR device.

[0040]

[0046] To illustrate, in a non-limiting example, if an XR device implements four cameras and a wearable accessory is outside the FOV of three of those cameras, the XR device may turn off three cameras (or possibly reduce the processing mode of three cameras) to reduce power consumption, but such cameras may not be able to capture tracking images of the wearable accessory (and the hand or other body part wearing the ring / wearable accessory). The XR device may use image data from the remaining cameras and tracking measurements from the wearable accessory to track, detect, and / or classify the user's hand / fingers wearing the wearable accessory (or any other body part wearing the wearable accessory) while the other three cameras are turned off (or in a particular reduced processing mode). The tracking measurements from the wearable accessory may help increase the tracking fidelity / accuracy of the XR device and may trigger subsequent processing mode adjustments of any of the cameras of the XR device based on the position and / or movement of the wearable accessory. In the event that all four cameras of the XR device are unable to capture an image of the wearable accessory, the XR device can use tracking measurements from the wearable accessory to keep track of the wearable accessory (and the body part(s) to which it is attached). The number of cameras in this example (e.g., four) is merely one illustrative example provided for purposes of explanation and should not be construed as limiting. Those skilled in the art will recognize from this disclosure that in other examples, the XR device can implement any other number of cameras.

[0041]

[0047] In some examples, the XR device can use tracking measurements from the wearable accessory to determine where the wearable accessory (and the body part(s) wearing the wearable accessory) is and / or will be located, and can reduce (e.g., turn off, reduce frame rate, etc.) the processing mode of the camera(s) when the wearable accessory is outside the coverage area of ​​the camera(s) for a certain period of time. The tracking measurements from the wearable accessory can also trigger reactivation of the camera(s) and / or a higher / full processing mode when the wearable accessory is again within the coverage area of ​​the camera(s).

[0042]

[0048] In some cases, tracking measurements from the wearable accessory can help the XR device track the wearable accessory during difficult imaging conditions, such as poor lighting conditions, cluttered backgrounds, etc., and detect the hand (or any other body part) wearing the wearable accessory. In some examples, tracking measurements from the wearable accessory can help the XR device distinguish different objects in a scene. For example, tracking measurements from a wearable accessory worn on a finger can help the XR device distinguish the hand of the finger wearing the wearable accessory from other hands in the scene. To illustrate, tracking measurements from the wearable accessory can provide the XR device with a reference for which hand to track. The XR device can use this information to track the hand of the finger wearing the wearable accessory, as the XR device can determine which of the various hands in the scene is the hand intended to be tracked.

[0043]

[0049] In some cases, the XR device can use one or more tracking measurements from the wearable accessory to distinguish an object of interest (e.g., a hand, a finger, etc.) associated with the wearable accessory from other objects (e.g., different body parts, devices, people, etc.) to enable the XR device to detect and track the object of interest (or the correct object). In some cases, the wearable accessory can send one or more signals with a frequency pattern to the XR device. The frequency pattern can be associated with the wearable accessory and can be used by the XR device to recognize the wearable accessory and the object of interest associated with the wearable accessory. In some examples, the XR device can use the frequency pattern to distinguish the wearable accessory and the object of interest from other objects in the environment. In some cases, the wearable accessory can include one or more visual patterns that the XR device can detect from one or more images to recognize the wearable accessory in the environment.

[0044]

[0050] In some cases, tracking measurements from a wearable accessory can help the XR device save power by reducing and / or optimizing tracking operations. For example, the wearable accessory can obtain tracking measurements and provide a velocity vector to the hand tracking engine of the XR device to help predict where in an image captured by the camera(s) of the XR device to search for a hand associated with the wearable accessory. The velocity vector can also help reduce power consumed by the XR device in searching for a hand in an image by reducing the search time and / or search field. For example, using tracking measurements from the wearable accessory, the XR device can search for a hand in one or more areas of an image without searching the entire image. By reducing the search, the XR device can also reduce power consumed by the tracking algorithm (and the efficiency of the tracking algorithm) in finding a hand in an image. Thus, the XR device can avoid searching unnecessary areas on the image.

[0045]

[0051] In some examples, tracking measurements from the wearable accessory can reduce power consumption in the XR device by reducing and / or adjusting the tracking workflow implemented by the XR device. For example, tracking measurements from the wearable accessory can indicate that a hand wearing the wearable accessory and tracked by the XR device is outside the FOV of the XR device's cameras. The XR device can stop some tracking operations, such as hand detection and / or hand classification operations, while the wearable accessory and hand are outside the FOV of the cameras. In some examples, the XR device can reduce the number of tracking operations while the wearable accessory and hand are outside the FOV of the cameras. For example, when the wearable accessory and hand are outside the FOV of the cameras, rather than performing hand detection and / or hand classification operations on every image captured by any of the cameras, the XR device can perform hand detection and / or hand classification operations every nth image.

[0046]

[0052] In some cases, the wearable accessory can provide tracking measurements to the XR device to enable the XR device to continue tracking the wearable accessory (and the body part on which the wearable accessory is worn) even if the user of the XR device turns off the camera(s) on the XR device for privacy. Moreover, the wearable accessory can also be used to trigger a privacy mode in the XR device to turn off the camera(s) on the XR device. For example, the user can remove the wearable accessory and / or provide input (e.g., touch input, motion-based input, etc.) to the wearable accessory to trigger the privacy mode in the XR device. The XR device can interpret the removal of the wearable accessory and / or the input to the wearable accessory as an input request to trigger the privacy mode.

[0047]

[0053] The wearable accessory may include one or more sensors to track the state of the wearable accessory, e.g., position, movement, etc. For example, the wearable accessory may include an inertial measurement unit (IMU) that may integrate multi-axis, accelerometers, gyroscopes, and / or other sensors to provide the XR device with an estimate of the orientation of the wearable accessory (and thus the body part on which the wearable accessory is worn) in physical space. The wearable accessory may include one or more sensors, such as an ultrasonic sensor and / or microphone, used for ranging the wearable accessory (and the body part on which the wearable accessory is worn). In some examples, the one or more ultrasonic sensors and / or microphones may help determine whether the wearable device is closer to another object(s), whether the user's hands (or other body parts) are closer or farther apart from each other, whether any of the user's hands (or other body parts) are closer to one or more other objects, etc. In some examples, a barometric air pressure sensor in the wearable accessory may determine a relative altitude change associated with the wearable accessory. The wearable accessory can send measurements from one or more sensors to the XR device, which can use the sensor measurements as further described herein.

[0048]

[0054] The technology will be described in the following disclosure as follows: The description begins with a description of example systems and techniques for using a wearable accessory in conjunction with an XR device for tracking, privacy, and / or power conservation, as shown in Figures 1-4. This is followed by a description of an example process for using a wearable device in conjunction with an XR device, as shown in Figure 5. The description concludes with a description of an example computing device architecture including example hardware components suitable for performing XR and related operations, as shown in Figure 6. The disclosure now turns its attention to Figure 1.

[0049]

[0055] 1 is a diagram illustrating an example of an XR system 100 and a wearable device 150 for an XR experience, according to some examples of the disclosure. The wearable device 150 can represent a wearable accessory used with the XR system 100, as described further herein. The wearable accessory can include, for example, but not limited to, a ring, a bracelet, a ring sleeve, a glove, a watch, or any other wearable device.

[0050]

[0056] The XR system 100 and the wearable device 150 can be communicatively coupled to provide various XR functions described herein. The XR system 100 and the wearable device 150 can include separate devices used for the XR experience. In some examples, the XR system 100 can implement one or more XR applications, such as, but not limited to, a video game application, a robotic application, an autonomous driving or navigation application, a productivity application, a social media application, a communication application, a media application, an e-commerce application, and / or any other XR application.

[0051]

[0057] In some examples, the XR system 100 may include an electronic device configured to use information regarding the relative pose of the XR system 100 and / or the wearable device 150 to provide one or more functions, such as XR functions (e.g., tracking, detection, classification, mapping, content rendering, etc.), gaming functions, autonomous driving or navigation functions, computer vision functions, robotic functions, etc. For example, as described further herein, in some cases, the XR system 100 may be an XR device (e.g., a head-mounted display, a head-up display device, smart glasses, etc.), and the wearable device 150 may provide tracking measurements to the XR system 100 for use by the XR system 100 to assist in tracking operations, reduce power usage, support privacy mode operations, etc.

[0052]

[0058] In the illustrative example shown in FIG. 1, the XR system 100 may include one or more image sensors, such as image sensor 102 and image sensor 104, other sensors 106, and one or more computational components 110. The other sensors 106 may include, for example, but not limited to, an inertial measurement unit (IMU), a radar, a light detection and ranging (lidar) sensor, an audio sensor, a position sensor, a pressure sensor, a gyroscope, an accelerometer, a microphone, and / or any other sensor. In some examples, the XR system 100 may include additional sensors and / or components, such as, for example, a light emitting diode (LED) device, a storage device, a cache, a communication interface, a display, a memory device, etc. An example architecture and example hardware components that may be implemented by the XR system 100 are further described below with respect to FIG. 6.

[0053]

[0059] Moreover, in the illustrative example shown in FIG. 1, wearable device 150 includes an IMU 152, an ultrasonic sensor, a pressure sensor 156 (e.g., a barometric air pressure sensor and / or any other pressure sensor), and a touch sensor 158 (or tactile sensor). The sensor devices shown in FIG. 1 are non-limiting examples provided for purposes of illustration. In other examples, wearable device 150 can include more or fewer sensors (of the same and / or different types) than those shown in FIG. 1. Moreover, in some cases, wearable device 150 can include other devices, such as, for example, a microphone, a display component (e.g., a light emitting diode display component), etc.

[0054]

[0060] 1 for XR system 100 and wearable device 150 are merely illustrative examples provided for purposes of explanation. In other examples, XR system 100 and / or wearable device 150 may include more or fewer components than those shown in FIG.

[0055]

[0061] The XR system 100 may be part of or implemented by a single computing device or multiple computing devices. In some examples, the XR system 100 may be part of an electronic device(s), such as a camera system (e.g., digital camera, IP camera, video camera, security camera, etc.), a telephone system (e.g., smartphone, cellular phone, conferencing system, etc.), a laptop or notebook computer, a tablet computer, a set-top box, a smart television, a display device, a gaming console, an XR device such as an HMD, a drone, a computer in a vehicle, an IoT (Internet of Things) device, a smart wearable device, or any other suitable electronic device(s). In some implementations, the image sensor 102, the image sensor 104, the one or more other sensors 106, and / or the one or more computational components 110 may be part of the same computing device.

[0056]

[0062] For example, in some cases, image sensor 102, image sensor 104, one or more other sensors 106, and / or one or more computing components 110 may be integrated with or into a camera system, a smartphone, a laptop, a tablet computer, a smart wearable device, an XR device such as an HMD, an IoT device, a gaming system, and / or any other computing device, although, in other implementations, image sensor 102, image sensor 104, one or more other sensors 106, and / or one or more computing components 110 may be part of or implemented by two or more separate computing devices.

[0057]

[0063] The one or more computing components 110 of the XR system 100 may include, for example, but not limited to, a central processing unit (CPU) 112, a graphics processing unit (GPU) 114, a digital signal processor (DSP) 116, and / or an image signal processor (ISP) 118. In some examples, the XR system 100 may include other types of processors, such as, for example, a computer vision (CV) processor, a neural network processor (NNP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), etc. The XR system 100 may use the one or more computing components 110 to perform various computing operations, such as, for example, extended reality operations (e.g., tracking, localization, object detection, classification, pose estimation, mapping, content anchoring, content rendering, etc.), image / video processing, graphics rendering, machine learning, data processing, modeling, calculations, and / or any other operations.

[0058]

[0064] In some cases, one or more computing components 110 may include other electronic circuitry or hardware, computer software, firmware, or any combination thereof to perform any of the various operations described herein. In some examples, one or more computing components 110 may include more or fewer computing components than those shown in Figure 1. Moreover, CPU 112, GPU 114, DSP 116, and ISP 118 are merely illustrative examples of computing components provided for purposes of explanation.

[0059]

[0065] Image sensor 102 and image sensor 104 may include any image and / or video sensor or capture device, such as a digital camera sensor, a video camera sensor, a smartphone camera sensor, an image / video capture device on an electronic device such as a television or computer, a camera, etc. In some cases, image sensor 102 and / or image sensor 104 may be part of a camera or computing device, such as a digital camera, a video camera, an IP camera, a smartphone, a smart television, a gaming system, etc. Moreover, in some cases, image sensor 102 and / or image sensor 104 may include multiple image sensors, such as rear and front sensor devices, and may be part of a dual camera or other multi-camera assembly (e.g., including two cameras, three cameras, four cameras, or other number of cameras).

[0060]

[0066] In some examples, image sensor 102 and / or image sensor 104 can capture image data, generate frames based on the image data, and / or provide image data or frames to one or more computational components 110 for processing. A frame can include a video frame of a video sequence or a still image. A frame can include a pixel array representing a scene. For example, a frame can be a Red-Green-Blue (RGB) frame having red, green, and blue color components per pixel, a Luma, Chroma Red, Chroma Blue (YCbCr) frame having a luma component and two chroma (color) components (chroma red and chroma blue) per pixel, or any other suitable type of color or monochrome picture.

[0061]

[0067] In some examples, one or more computing components 110 can perform XR processing operations based on data from image sensor 102, image sensor 104, one or more other sensors 106, and / or wearable device 150. For example, in some cases, one or more computing components 110 can perform tracking, localization, object detection, object classification, pose estimation, shape estimation, mapping, content anchoring, content rendering, image processing, modeling, content generation, gesture detection, gesture recognition, and / or other operations based on data from image sensor 102, image sensor 104, one or more other sensors 106, and / or wearable device 150.

[0062]

[0068] In some examples, the one or more computing components 110 can implement one or more algorithms for tracking and estimating a relative pose between the wearable device 150 and the XR system 100. In some cases, the one or more computing components 110 can receive image data captured by the image sensor 102 and / or the image sensor 104, and perform pose estimation based on the received image data to calculate a relative pose between the wearable device 150 and the XR system 100. Moreover, the one or more computing components 110 can receive sensor data (e.g., data from the IMU 152, the ultrasonic sensor 154, the pressure sensor 156, and / or the touch sensor 158) from the wearable device 150, use such data to track the wearable device 150 (with or without other data from the image sensor 102, the image sensor 104, or the other sensor(s) 106), adjust processing / power operations, etc., as described herein. In some cases, the one or more computation components 110 may implement one or more computer vision models to calculate the relative pose of the wearable device 150 and the XR system 100.

[0063]

[0069] In some cases, the one or more other sensors 106 can detect acceleration by the XR system 100 and generate acceleration measurements based on the detected acceleration. In some cases, the one or more other sensors 106 can additionally or alternatively detect and measure the orientation and angular velocity of the XR system 100. For example, the one or more other sensors 106 can measure the pitch, roll, and yaw of the XR system 100. In some examples, the XR system 100 can use measurements obtained by the one or more other sensors 106 to calculate the relative attitude of the XR system 100. In some cases, the XR system 100 can additionally or alternatively use sensor data from the wearable device 150 to perform tracking, attitude estimation, and / or other operations.

[0064]

[0070] The wearable device 150 can use the IMU 152, the ultrasonic sensor 154, and / or the pressure sensor 156 to obtain tracking measurements for the wearable device 150. The tracking measurements can include, for example, but are not limited to, position measurements, speed / motion measurements, range / distance measurements, altitude measurements, etc. The wearable device 150 can provide the tracking measurements to the XR system 100. In some cases, the wearable device 150 can use the touch sensor 158 to receive user input for the XR system 100, such as, for example, touch input (e.g., tapping, squeezing, pressing, rubbing, touching, etc.). The wearable device 150 can provide one or more detected inputs to the XR system 100 to modify the content, operation, and / or behavior of the XR system 100.

[0065]

[0071] In some examples, the XR system 100 can use measurements obtained by the IMU 152, ultrasonic sensor 154, and / or pressure sensor 156 to calculate (and / or assist in calculating) the relative location, movement, and / or position of the wearable device 150. In some cases, the IMU 152 can detect acceleration by the wearable device 150 and generate acceleration measurements based on the detected acceleration. In some cases, the IMU 152 can additionally or alternatively detect and measure the orientation and angular velocity of the wearable device 150. For example, the IMU 152 can measure the pitch, roll, and yaw of the wearable device 150.

[0066]

[0072] In some examples, the wearable device 150 can use an ultrasonic sensor 154 for ranging. For example, the ultrasonic sensor 154 can measure the distance of the wearable device 150 relative to the XR system 100 and / or another object, such as, for example, another wearable device, a body part (e.g., a hand, etc.), a person, a wall, another device (e.g., a controller, a stylus, a joystick, a mouse, a display, etc.), and / or any other object. The pressure sensor 156 can detect pressure, such as air pressure, and can determine relative altitude changes. The touch sensor 158 can measure physical forces or interactions with the wearable device 150, which can be interpreted as inputs to the XR system 100.

[0067]

[0073] The wearable device 150 may include one or more wireless communication interfaces (not shown) for communicating with the XR system 100. In some examples, the one or more wireless communication interfaces may include a wireless transmitter, a wireless transceiver, or any other means for wireless communication and / or for transmitting data. The one or more wireless communication interfaces may implement any wireless protocol and / or technology for communicating with the XR system 100, such as, for example, a short-range wireless technology (e.g., Bluetooth, etc.). The wearable device 150 may use the one or more wireless communication interfaces to transmit sensor measurements and / or other XR inputs to the XR system 100, as further described herein.

[0068]

[0074] Although the XR system 100 and the wearable device 150 are shown to include several components, one skilled in the art will appreciate that the XR system 100 and the wearable device 150 can include more or less components than those shown in FIG. 1. For example, the XR system 100 and / or the wearable device 150 can also include one or more other memory devices (e.g., RAM, ROM, cache, etc.), one or more networking interfaces (e.g., wired and / or wireless communication interfaces, etc.), one or more display devices, cache, storage devices, and / or other hardware or processing devices in some cases not shown in FIG. 1. Illustrative examples of computing devices and / or hardware components that may be implemented with the XR system 100 and / or the wearable device 150 are described below with respect to FIG. 6.

[0069]

[0075] 2A shows an example of a ring device 200 worn on a finger 210 of a user who interacts with the XR system 100. The ring device 200 may be an example of a wearable device, such as the wearable device 150 shown in FIG. 1. For example, in some cases, the ring device 200 may be the same as the wearable device 150 shown in FIG. 1. In other cases, the ring device 200 may be a wearable device with a different form factor and / or a different type of wearable device than the wearable device 150 shown in FIG. 1.

[0070]

[0076] A user can use the ring device 200 to provide tracking measurements to the XR system 100, such as position measurements, distance measurements, altitude measurements, motion measurements, location measurements, etc. In some cases, a user can use the ring device 200 to interact with the XR system 100 and provide XR input. In some examples, the ring device 200 can collect sensor measurements to track the location, movement, pose, etc. of the ring device 200 in 3D space. In some examples, the location, movement, pose, etc. can be tracked relative to the location, movement, pose, etc. of the XR system 100 in 3D space. In some cases, the location, movement, pose, etc. can be tracked relative to the location, movement, pose, etc. of another object in 3D space, such as another wearable device, a person, a hand, a leg, a wall, an input device (e.g., a stylus, a controller, a mouse, etc.), an animal, a vehicle, etc.

[0071]

[0077] In the example shown in FIG. 2A, the ring device 200 includes a touchpad 204 for receiving touch input, a display 206 for displaying information from the ring device 200 and / or the XR system 100, and a sensor 208. However, in other cases, the ring device 200 can include more or less sensors / devices than those shown in FIG. 2A. In some examples, the sensor 208 can include and / or be the same as the IMU 152, ultrasonic sensor 154, pressure sensor 156, and / or touch sensor 158 shown in FIG. 1. In other examples, the sensor 208 can include one or more sensors not shown in FIG. 1. In some cases, the ring device 200 can include a touch or pressure sensitive surface and / or surface portion for measuring touch input.

[0072]

[0078] The XR system 100 can render content, interfaces, controls, etc. to a user wearing the XR system 100 and the ring device 200. The user can use the ring device 200 to wirelessly provide tracking measurements to the XR system 100 to assist the XR system 100 in tracking, detecting, classifying, mapping, etc., one or more objects associated with the ring device 200, such as a finger 210, a hand of the finger 210, etc. In some examples, the user can use the ring device 200 to wirelessly adjust the operation of the XR system 100, interact with XR content / interface / controls, etc. presented in the XR system 100, and / or provide XR input, such as selection, object / environment manipulation, navigational input (e.g., scrolling, moving, etc.), gestures, etc.

[0073]

[0079] The ring device 200 may be used to provide data to the XR system 100 to improve tracking in the XR system 100, reduce power consumption in the XR system 100 when providing an XR experience, trigger a privacy mode (and exit privacy mode) in the XR system 100, improve XR functionality, and the like, as described further herein. For example, with reference to FIG. 2B, at time T1, the ring device 200 may obtain data 220 based at least in part on sensor data from one or more sensors (e.g., touchpad 204, sensor 208, etc.) on the ring device 200 and send the data 220 to the XR system 100. In some examples, the data 220 may include tracking measurements, such as, for example, position measurements, distance measurements, altitude measurements, motion measurements, location measurements, and the like. In some examples, the data 220 may include attitude information related to the ring device 200. For example, the data 220 may include the orientation of the ring device 200 in physical space, the orientation of the ring device 200 relative to the XR system 100, the orientation of the ring device 200 in the coordinate system of the ring device 200, and / or the orientation of the ring device 200 in the coordinate system of the XR system 100. In some examples, the data 220 may include command and / or input information. For example, in some cases, the data 220 may include one or more commands and / or inputs for triggering or stopping an action in the XR system 100, triggering settings and / or operating states / modes of the XR system 100, providing input to an application on the XR system 100, and / or any other commands and / or inputs.

[0074]

[0080] In some examples, the XR system 100 can use the data 220 to track the ring device 200 at T1. In some cases, the XR system 100 can use the data 220 to estimate the position of the ring device 200 at one or more time steps after T1. In some examples, the XR system 100 can use the data 220 to determine whether the finger 210 (and / or the hand of the finger 210) is visible to one or more image sensors (e.g., image sensor 102, image sensor 104) on the XR system 100. For example, the XR system 100 can use the data 220 to determine whether the finger 210 (and / or the hand of the finger 210) is within the FOV of one or more image sensors (e.g., image sensor 102, image sensor 104) on the XR system 100 and / or whether the view of the one or more image sensors to the finger 210 (and / or the hand of the finger 210) is obstructed by one or more objects. The XR system 100 can use this information to adjust device (e.g., image sensor) settings, XR operation, etc., as further described herein.

[0075]

[0081] Time T n In the example, the ring device 200 may obtain data 230 based at least in part on sensor data from one or more sensors on the ring device 200 and send the data 230 to the XR system 100. The XR system 100 may n The data 230 can be used to track the ring device 200 in the XR system 100. n The data 230 can be used to estimate the position of the ring device 200 at one or more time steps after the position of the finger 210. In some examples, the XR system 100 can use the data 230 to determine whether the finger 210 (and / or the hand of the finger 210) is visible to one or more image sensors on the XR system 100, as previously described.

[0076]

[0082] As mentioned above, the XR system may track objects (e.g., hands, fingers, devices, etc.) based on image data received by a camera on the XR system. However, capturing and / or processing images permanently or periodically may be costly. Moreover, in some cases, capturing and / or processing images may result in unnecessary use of power resources. For example, an object of interest being tracked by the XR system may be occluded from the view of the camera(s) on the XR system, preventing the XR system from capturing the object of interest in images obtained from the camera(s) on the XR system. As another example, there may be periods of time where a user may not intend to use the object of interest to provide any input (or any input at all) to the XR system 100, there may be spaces where camera imaging is not permitted or lighting conditions are insufficient, etc.

[0077]

[0083] In some examples, data from the wearable device 150 may be used to help the XR system 100 track objects of interest, conserve power in the XR system 100, control one or more operations / modes in the XR system 100, etc. For example, the wearable device 150 may receive a signal from the wearable device 150 to inform a tracking engine in the XR system 100. In some cases, the signal may include data from sensors built into the wearable device 150 and / or data based on sensor data from those sensors. The XR system 100 may use the signal from the wearable device 150 to track the body part (e.g., finger, hand, wrist, group of fingers, etc.) wearing the wearable device 150 and correlate the estimated location of the body part with the FOV of any image sensor (e.g., image sensor 102, image sensor 104) on the XR system 100. In some examples, the XR system 100 can use this information to limit imaging to the image sensor(s) that can see the body part at a given time. In some cases, when the body part is not observable by the image sensor (e.g., the body part is outside the FOV of the image sensor, the body part is occluded by one or more objects, the lighting conditions in the environment are insufficient or too dark, etc.), the XR system 100 can maintain tracking of the body part based on sensor signals from the wearable device 150 (e.g., position measurements, movement measurements, pressure differences between the XR system 100 and the wearable device 150, location measurements, distance measurements, etc.).

[0078]

[0084] In some cases, the XR system 100 can use a signal from the wearable device 150 to notify a visual object tracking engine in the XR system 100 when an object of interest begins to move towards an observable FOV associated with one or more image sensors on the XR system 100. For example, the XR system 100 can use a signal indicating the speed of a hand moving into view to pre-seed hand tracking. Additionally, the wearable device 150 and associated signals can help conserve power on the XR system 100 by limiting the use of one or more image sensors for tracking to instances where the object of interest is observable by one or more image sensors. In some examples, the user experience can be improved by providing observable gestures by an object (e.g., a hand, etc.) and / or gestures by an object (e.g., a hand, etc.) that is not in the FOV of the image sensor(s) on the XR system 100. The XR system 100 can improve tracking robustness by combining a signal from the wearable device 150 with image data from one or more image sensors on the XR system 100. In some cases, the signal from the wearable device 150 may also be used to switch the operation / mode of the XR system 100 to a non-imaging mode (e.g., private mode) in private locations (e.g., rest rooms, locker rooms, offices, etc.) and / or during periods when a user desires to maintain privacy by preventing images from being captured.

[0079]

[0085] 3 illustrates an example process 300 for integrating data from a wearable device 150 for XR operation in an XR system. The blocks, steps, and / or operations outlined in the example process 300 are illustrative examples and may be implemented in a different order and / or in any combination, including combinations that exclude, add, or modify some blocks, steps, and / or operations.

[0080]

[0086] In this example, the wearable device 150 can send data 302 to the XR system 100. The data 302 can include data calculated, measured, and / or collected by one or more sensors on the wearable device 150, such as the IMU 152, the ultrasonic sensor 154, the pressure sensor 156, and / or the touch sensor 158. In some cases, the data 302 can additionally or alternatively include data generated based at least in part on data calculated, measured, and / or collected by one or more sensors on the wearable device 150. In some cases, the wearable device 150 can acquire the data 302 while the wearable device 150 is worn by a user. For example, the wearable device 150 can acquire and send the data 302 while the user is wearing the wearable device 150 on a finger, wrist, or other body part. Data 302 may include, for example, but is not limited to, one or more position measurements, distance measurements, altitude measurements, location measurements, motion measurements, commands, inputs, attitude information, and the like.

[0081]

[0087] In block 304, the XR system 100 can use the data 302 to determine camera (e.g., image sensor 102, image sensor 104) operational settings. The XR system 100 can adjust the camera operational settings to balance the power consumption of the image sensor(s) in the XR system 100 with the ability of the image sensor(s) to capture an image(s) (or threshold quality / resolution images) of the wearable device 150 and / or a target associated with the wearable device 150, such as a hand, a finger, etc. For example, XR system 100 can adjust camera operation settings to reduce power consumption by an image sensor in XR system 100 when such an image sensor is unable to capture an image (or an image with a threshold quality) of wearable device 150 and / or a target associated with wearable device 150, and to increase or maintain power consumption by the image sensor when the image sensor is able to capture an image of wearable device 150 and / or a target associated with wearable device 150.

[0082]

[0088] The camera operation settings may include, for example, a power state (e.g., turned off, turned on, higher power state, lower / reduced power state, etc.) of one or more image sensors in the XR system 100, a setting (e.g., frame rate, resolution, etc.) of one or more image sensors, an active / inactive state of one or more image sensors, an XR operation enabled / implemented by one or more image sensors, camera hardware used by one or more image sensors, and / or any other state or configuration of one or more image sensors and / or associated hardware. In some examples, the XR system 100 may determine the camera operation settings based on the position, movement, location, and / or state of the wearable device 150 determined based on the data 302.

[0083]

[0089] For example, if the XR system 100 determines (e.g., based at least in part on the data 302) that the wearable device 150 is outside the FOV of one or more image sensors on the XR system 100 or that the view of the one or more image sensors to the wearable device 150 is obstructed (e.g., by an object, poor lighting conditions, background clutter, etc.), the XR device 100 may turn off / disable one or more image sensors, reduce the frame rate of one or more image sensors, reduce the resolution of one or more image sensors, reduce the operation of one or more image sensors, and / or possibly reduce the power consumption or state of one or more image sensors to avoid unnecessary use of and / or power consumption by the one or more image sensors while the one or more image sensors are unable to capture an image of the wearable device 150 (or an image in which the wearable device 150 may be detected).

[0084]

[0090] As another example, if the XR system 100 determines (e.g., based at least in part on the data 302) that the wearable device 150 is visible to one or more image sensors on the XR system 100 (e.g., based on the FOV of the one or more image sensors, lighting conditions, etc.), the XR system 100 may turn / enable or keep on one or more image sensors, increase or maintain a certain frame rate of one or more image sensors, increase or maintain the resolution of one or more image sensors, increase or maintain some operation of one or more image sensors, and / or possibly increase or maintain the power consumption or state of one or more image sensors. This may enable the XR system 100 to capture image(s), higher quality image(s), a higher rate of consecutive images, etc., for use in one or more XR operations, such as, for example, tracking, object detection, classification, mapping, etc.

[0085]

[0091] In some cases, when all image sensors in the XR system 100 are turned off / disabled (and / or the wearable device 150 is not visible to any of the image sensors in the XR system 100), the XR system 100 can use the data 302 to track the wearable device 150 and / or a target associated with the wearable device 150 until the wearable device 150 and / or the target is visible to one or more image sensors in the XR system 100. In some cases, the XR system 100 can turn on or enable an image sensor when the wearable device 150 and / or the target becomes visible to an image sensor (e.g., as determined based on sensor data from the wearable device 150) or approaches (or is within) a threshold distance from the FOV of the image sensor (e.g., as determined based on sensor data from the wearable device 150).

[0086]

[0092] In some cases, the XR system 100 can use sensor data from the wearable device 150 to determine a trajectory of the wearable device 150 (and / or a target associated with the wearable device 150) relative to the FOV of one or more image sensors on the XR system 100. In some examples, the XR system 100 can use this information to (increasingly) reduce or increase the operational state and / or settings of the image sensor as the wearable device 150 moves closer to or farther from the FOV of the image sensor. For example, the XR system 100 can increasingly reduce the frame rate, resolution, power state, etc. of the image sensor as the wearable device 150 moves farther from the FOV of the image sensor. Similarly, the XR system 100 can increase the frame rate, resolution, power state, etc. of the image sensor as the wearable device 150 moves closer to the FOV of the image sensor.

[0087]

[0093] In some cases, if an image sensor in the XR system 100 is turned on / enabled and the wearable device 150 (and / or a target associated with the wearable device 150) is visible to such image sensor, the XR system 100 can use that image sensor to capture one or more images of the wearable device 150 (and / or a target associated with the wearable device 150), which the XR system 100 can use to track the wearable device 150 (and / or a target associated with the wearable device 150). The XR system 100 can also use the data 302 to aid in tracking the wearable device 150 (and / or a target associated with the wearable device 150), as the data 302 can provide additional and / or augmenting information regarding the position, location, movement, etc. of the wearable device 150 and / or can help increase tracking fidelity / accuracy.

[0088]

[0094] In some examples, depending on the view of each image sensor in the XR system 100 to the wearable device 150 (e.g., determined at least in part based on the data 302), the XR system 100 can configure different image sensors in different operating modes. For example, when the wearable device 150 is within the FOV of one image sensor and outside the FOV of a different image sensor, the XR system 100 can turn off / disable or reduce the settings (e.g., frame rate, resolution, etc.) of the different image sensor that does not have a view to the wearable device 150. The XR system 100 can also keep on / enabled and / or increase the settings (e.g., frame rate, resolution, etc.) of the image sensor that has a view to the wearable device 150.

[0089]

[0095] In block 306, the XR system 100 can use the data 302 to determine XR processing settings to be implemented by the XR system 100. The XR system 100 can adjust the XR processing settings to balance the power consumption in the XR system 100 against the potential performance (e.g., accuracy, etc.) and / or benefit of the XR processing settings. For example, the XR system 100 can adjust some XR processing settings to reduce power consumption, and vice versa, if the XR system 100 determines that the reduction in power consumption resulting from such adjustment outweighs the potential (if any) negative performance impact.

[0090]

[0096] In some examples, the XR processing settings can define a tracking workflow and / or a frequency of one or more operations during the tracking workflow. For example, the XR processing settings can define which tracking, object detection, classification, etc. operations should be performed and / or the frequency of such operations. To illustrate, the data 302 may indicate that a hand (or fingers of a hand) wearing the wearable device 150 is outside the FOV of the image sensor(s) of the XR system 100. Thus, the XR system 100 can stop some tracking operations, such as hand detection and / or hand classification operations, while the wearable device 150 and hand are outside the FOV of the image sensor(s).

[0091]

[0097] In some cases, the XR system 100 may conserve power by reducing the number and / or frequency of tracking operations while the wearable device 150 and hand (or any other target) are outside the FOV of the image sensor(s). For example, when the wearable device 150 and hand are outside the FOV of the image sensor(s), rather than performing hand detection and / or hand classification operations on every image captured by the image sensor(s), the XR system 100 may perform hand detection and / or hand classification operations on every nth image captured by the image sensor(s).

[0092]

[0098] In block 308, the XR system 100 may detect a target (e.g., the wearable device 150, a hand associated with the wearable device 150, a finger associated with the wearable device 150, etc.) in an image(s) captured by one or more image sensors of the XR system 100. The one or more image sensors of the XR system 100 may capture an image when the target is within the FOV of the one or more image sensors as determined at least in part based on the data 302. The XR system 100 may implement an object detection algorithm to detect the target in the image. In some examples, the XR system 100 may implement computer vision and / or machine learning to detect the target in the image. In some cases, the XR system 100 can use the data 302 and / or image data from one or more image sensors on the XR system 100 to detect and / or recognize a target gesture, modify content rendered by the XR system 100 (e.g., virtual content, interfaces, controls, etc.), generate input / interaction with content rendered by the XR system 100, etc.

[0093]

[0099] In some cases, the XR system 100 can use the data 302 to reduce the amount of power used to detect a target in an image. For example, in some cases, the XR system 100 can use the data 302 to predict where in the image to search for a target. The XR system 100 can reduce the power consumed by the XR system 100 in searching for a target in an image by reducing the search time and / or search field. For example, using the data 302, the XR system 100 can search for a target in one or more regions of an image without searching the entire image. The data 302 can provide an indication of the location / position of the target, which the XR system 100 can use to determine which region(s) of the image to search. By reducing the search, the XR system 100 can reduce the power consumed by the tracking algorithm (and the efficiency of the tracking algorithm) in finding a target in an image. Thus, the XR system 100 can avoid searching unnecessary regions of the image.

[0094]

[0100] At block 310, the XR system 100 may estimate the pose and / or shape of the target detected in the image(s). In some examples, the XR system 100 may implement machine learning algorithms to estimate the pose and / or shape of the target detected in the image(s). For example, the XR system 100 may implement one or more neural networks to process the data 302 and / or the image(s) and determine the shape and / or pose of the target. In some cases, the XR system 100 may use the estimated pose and / or shape of the target for one or more XR operations, such as, for example, tracking / localization, mapping, content anchoring, content rendering, etc.

[0095]

[0101] 4 is a flow diagram illustrating an example process 400 for using a wearable device 150 in conjunction with an XR system 100 for enhanced tracking, power saving, and privacy features. The blocks and / or operations outlined in the example process 400 are illustrative examples and may be implemented in a different order and / or in any combination, including combinations that exclude, add, or modify some blocks / operations.

[0096]

[0102] In this example, in block 402, the XR system 100 may capture one or more images of a target associated with the wearable device 150. In some cases, the target may include a body part wearing the wearable device 150 or associated with another body part wearing the wearable device 150, such as, for example, a finger wearing the wearable device, a hand of a finger wearing the wearable device 150 (or a hand wearing the wearable device 150). In some cases, the target may include an object, such as, for example, the wearable device 150, an input device (e.g., a controller, stylus, mouse, etc.), a person, an animal, a separate device (e.g., a robotic device, etc.), a vehicle, or any object.

[0097]

[0103] At block 404, the wearable device 150 can send data to the XR system 100. The data can include data captured / measured by one or more sensors in the wearable device 150, such as, for example, position measurements, motion measurements, distance measurements, location measurements, altitude measurements, etc. In some cases, the data can additionally or alternatively include data generated based at least in part on data captured / measured by one or more sensors in the wearable device 150. In some examples, the data can include location information. For example, the data can include a posture of the wearable device 150. The posture can be, for example, a posture in a physical space (e.g., in a 3D space), a posture relative to the XR system 100, a posture in a coordinate system of the wearable device 150, a posture in a coordinate system of the XR system 100, any combination thereof, and / or any other posture. In some examples, the data can include one or more commands and / or inputs to the XR system 100, such as an application command / input, one or more operating states / modes, one or more settings, a command / input to apply one or more actions, etc.

[0098]

[0104] At block 406, the XR system 100 can use one or more images of the target and the data from the wearable device 150 to track the target. For example, the XR system 100 can detect the location of the target in the one or more images and use the location of the target in the one or more images to estimate the position of the target in 3D space. The XR system 100 can use the data to help detect the target in the one or more images and / or determine the position of the target in 3D space. In some examples, the XR system 100 can implement computer vision algorithms and / or machine learning (e.g., neural networks, etc.) algorithms to process the one or more images and the data to track the target.

[0099]

[0105] In block 408, the wearable device 150 can send additional data to the XR system 100. In block 410, the XR system 100 can use the data to determine the visibility of one or more image sensors in the XR system 100 to the target. For example, the XR system 100 can use the data from the wearable device 150 to determine the position of the wearable device 150 and the target in 3D space. Based on the position of the target in 3D space, the XR system 100 can determine whether the target is within the FOV of any image sensor on the XR system 100. Thus, the XR system 100 can determine whether the target is outside or inside the FOV of any image sensor on the XR system 100.

[0100]

[0106] In block 412, the XR system 100 may adjust operational and / or processing settings in the XR system 100 based on the location of the target relative to the FOV of each image sensor on the XR system 100. The operational and / or processing settings may include, for example, but not limited to, camera device settings (e.g., frame rate, resolution, power mode, resource usage, state, etc.), XR processing or workflow settings, etc. In some examples, the XR processing or workflow settings may define the particular XR operations (e.g., object detection, object classification, pose estimation, shape estimation, mapping, gesture detection, gesture recognition, etc.) to be implemented by the XR system 100, the frequency (e.g., based on units of time, number of frames processed, events, etc.) at which any particular XR operations are implemented (or not), the settings (e.g., full implementation, partial implementation, coarse implementation, sparse implementation, fidelity, etc.) of any particular XR operations to be implemented, and / or any other processing settings.

[0101]

[0107] For example, the XR system 100 may adjust the settings of the image sensor to enable (e.g., turn on, enable, etc.) use of the image sensor and / or increase the performance of the image sensor (e.g., increase frame rate, resolution, power mode, resource usage, etc.) if the target is within the FOV of the image sensor (or is approaching the FOV within a threshold proximity and / or estimated time frame), or disable (e.g., turn off, disable) use of the image sensor and / or decrease the performance and / or power consumption of the image sensor (e.g., decrease frame rate, resolution, power mode, resource usage, etc.) if the target is not within the FOV of the image sensor (or is not approaching the FOV within a threshold proximity and / or estimated time frame). In some cases, the XR system 100 may implement different image sensor settings for some or all of the image sensors on the XR system 100. In other cases, the XR system 100 may implement the same image sensor settings for every image sensor on the XR system 100.

[0102]

[0108] As another example, the XR system 100 may (additionally or alternatively) enable one or more operations (or increase performance / processing settings for one or more operations), such as object detection, object classification, pose estimation, shape estimation, etc., if the target is within the image sensor's FOV (or is approaching the FOV within a threshold proximity and / or estimated time frame), or disable one or more operations (or decrease performance / processing settings for one or more operations), such as object detection, object classification, pose estimation, shape estimation, etc., if the target is not within the image sensor's FOV (or is not approaching the FOV within a threshold proximity and / or estimated time frame).

[0103]

[0109] In block 414, the wearable device 150 can send additional data to the XR system 100. In block 416, the XR system 100 can use the data from the wearable device 150 to track the target. If the target is not within the FOV of any image sensor on the XR system 100 and / or the XR system 100 does not have an image capturing the wearable device 150 and / or the target at a location associated with the data (or if the XR system 100 cannot detect the target from any captured image), the XR system 100 can rely on the data from the wearable device 150 to track the target.

[0104]

[0110] For example, in block 412, if the XR system 100 turns off or disables any image sensors in the XR system 100 (e.g., because the target is not within the FOV of any image sensor or is not approaching the FOV within the threshold proximity and / or estimated time frame), the XR system 100 may not have an image of the target at the target's current location (or captured within the threshold time period). Thus, without an image of the target that the XR system 100 can use to track the target, the XR system 100 can rely on data from the wearable device 150 to track the target until the XR system 100 is able to acquire an image of the target (or tracking is terminated).

[0105]

[0111] On the other hand, if in block 412 the XR system 100 does not turn off or disable any image sensors in the XR system 100 and at least one image sensor is capable of capturing an image of the target, the XR system 100 can use the image of the target as well as the data from the wearable device 150 to track the target. In some examples, the XR system 100 can use the data along with the image of the target to determine the location of the target in 3D space. In some cases, the XR system 100 can use the data to help locate the target in the image. For example, the XR system 100 can use the data to estimate which region(s) of the image contain the target and / or reduce which regions of the image it searches for the target. In some cases, by reducing the area of ​​the image that is searched (e.g., instead of searching the entire image), the XR system 100 can reduce the amount of power consumed in searching for the target in the image and / or increase the efficiency of finding the target in the image.

[0106]

[0112] In block 418, the wearable device 150 can send additional data to the XR system 100. In block 420, the XR system 100 can enable or increase one or more operation / processing settings (e.g., image sensor settings, XR processing or workflow settings, etc.) in the XR system 100. For example, the data can trigger the XR system 100 to enable or increase one or more operation / processing settings that were previously disabled or decreased (e.g., in block 412). The XR system 100 can use the data from the wearable device 150 to determine the location of the target and determine whether to enable or increase any operation / processing settings based on the location of the target relative to the FOV of one or more image sensors on the XR system 100. In some examples, the XR system 100 can use the data to determine the visibility of each image sensor on the XR system 100 to the target and enable or increase any operation / processing settings based on the visibility of each image sensor to the target.

[0107]

[0113] For example, if the XR system 100 previously turned off or disabled an image sensor (e.g., at block 412) and subsequently determines (e.g., at block 420) that a target is within the FOV of the image sensor (or is approaching the FOV within a threshold proximity and / or estimated time frame), the XR system 100 may turn on or enable the image sensor to capture an image(s) of the target using that image sensor. The XR system 100 may use such images of the target to track the target, detect the target, estimate the pose of the target, estimate the shape of the target, etc., as described herein. In some examples, if the XR system 100 previously turned off or disabled an image sensor (e.g., at block 412) and determines (e.g., at block 420) that the target is not (or is still not) within the FOV of the image sensor (or is not approaching the FOV within a threshold proximity and / or estimated time frame), the XR system 100 may keep the image sensor in the off or disabled state to conserve power.

[0108]

[0114] In block 422, the XR system 100 can use the turned on / enabled image sensor or sensors to capture an image(s) of the target. The image sensor or sensors can include any image sensor on the XR system 100 that has visibility to the target. In block 424, the wearable device 150 can also send data to the XR system 100.

[0109]

[0115] In block 426, the XR system 100 can use the image(s) of the target and the data from the wearable device 150 to track the target, as previously described.

[0110]

[0116] In block 428, the wearable device 150 can send additional data to the XR system 100. In block 430, the data from the wearable device 150 can trigger a privacy mode in the XR system 100. The privacy mode can include turning off or disabling each image sensor on the XR system 100.

[0111]

[0117] In some examples, the data that triggers the privacy mode may indicate a certain input provided to the wearable device 150 (e.g., touch input, motion-based input, etc.) and / or a certain state of the wearable device 150 (e.g., wearable device 150 being removed from a finger, hand, or other body part, wearable device 150 being placed on a surface, etc.). The XR system 100 may interpret the particular input and / or state indicated in the data as a request or intent to enter / enable privacy mode.

[0112]

[0118] In some examples, the wearable device 150 can send additional data to the XR system 100 to trigger the XR system 100 to stop the privacy mode. For example, the wearable device 150 can send additional data indicating a certain input (e.g., touch input, motion-based input, etc.) and / or a certain state of the wearable device 150 (e.g., the wearable device 150 is placed on a finger, hand, or other body part), which the XR system 100 can interpret as a request or intent to stop the privacy mode. The XR system 100 can use the data from the wearable device 150 to stop the privacy mode and return to the previous operation / processing settings, or to determine the operation / processing settings to implement after the privacy mode.

[0113]

[0119] In some cases, when the XR system 100 is in a privacy mode, the XR system 100 can use the data provided by the wearable device 100 to provide some level of tracking of the target. For example, the wearable device 150 can send data to the XR system 100 while the XR system 100 is in a privacy mode. The XR system 100 can use the data to track the target, as previously described.

[0114]

[0120] 5A is a flowchart illustrating an example process 500 for using a wearable device (e.g., wearable device 150, wearable ring 200) in conjunction with an XR device (e.g., XR system 100). In some examples, the process 500 enables the wearable device to be used in conjunction with an XR device to improve tracking, power saving, and / or privacy features in the XR device.

[0115]

[0121] At block 502, the process 500 may include establishing a wireless connection between the wearable device and the XR device. The wireless connection may be established using a wireless communication interface (e.g., a wireless transmitter, a wireless transceiver, or any other means for transmitting data) on the wearable device and a wireless communication interface on the XR device. The wireless communication interface on the wearable device may implement any wireless protocol and / or technology for communicating with the XR device, such as, for example, a short-range wireless technology (e.g., Bluetooth, etc.). In some cases, the wearable device may be paired with the XR device for communication / interaction between the wearable device and the XR device.

[0116]

[0122] In some examples, the body part associated with the user may include a hand, a finger, multiple fingers, and / or a wrist. In some cases, the wearable device may include a ring. In other cases, the wearable device may include a bracelet, a glove, a ring sleeve, or any other wearable item.

[0117]

[0123] At block 504, the process 500 may include obtaining one or more tracking measurements at the wearable device. The wearable device may obtain the one or more tracking measurements using one or more sensors on the wearable device (e.g., the IMU 152, the ultrasonic sensor 154, the pressure sensor 156, the touch sensor 158). In some examples, the one or more sensors may include an accelerometer, a gyroscope, a pressure sensor, an audio sensor, a touch sensor, and / or a magnetometer.

[0118]

[0124] In some examples, the one or more tracking measurements may include a position of the structure in physical space (e.g., in 3D space), a movement of the structure, a distance of the structure relative to one or more objects, a speed of the movement of the structure, an altitude of the structure in physical space, and / or a pose of the structure in physical space. In some examples, the one or more objects may include an XR device, different body parts associated with a user (e.g., different fingers, different hands, different wrists, different sets of fingers, etc.), an input device (e.g., a controller, a stylus, a mouse, etc.), a wall, a person, an animal, a separate device, and / or any other object.

[0119]

[0125] In some examples, at least one of the one or more tracking measurements may be relative to a reference coordinate system of the wearable device (and / or one or more sensors of the wearable device).

[0120]

[0126] At block 506, the process 500 may include sending data related to the one or more tracking measurements to the XR device. In some cases, the data may include one or more tracking measurements. In some cases, the data may additionally or alternatively include data generated based at least in part on the one or more tracking measurements. In some examples, the data may include location information (e.g., a posture of the wearable device), one or more commands, one or more inputs, and / or any other data and / or sensor measurements. The XR device may use the data to track a target (e.g., a body part, a wearable device, etc.), adjust one or more device settings and / or operations, trigger a privacy mode, etc. In some cases, the XR device may use the data to verify / augment tracking results obtained by the XR device based on image data captured by the XR device. In some cases, the XR device can use the data to maintain tracking of the body part when the body part cannot be imaged by the image sensors on the XR device (e.g., because the body part is outside the FOV of any image sensors, the XR device is operating in a privacy or non-imaging mode, lighting conditions in the environment are insufficient or too dark, the body part is obstructed from the view of any image sensors on the XR device, etc.).

[0121]

[0127] In some cases, the XR device can use the data to distinguish the body part from other objects (e.g., other body parts, devices, people, etc.) to enable the XR device to detect and track the body part (or the correct body part). In some cases, the wearable device can send one or more signals with a frequency pattern to the XR device. The frequency pattern can be associated with the wearable device and used by the XR device to recognize the wearable device and the body part associated with the wearable device. In some examples, the XR device can use the frequency pattern to distinguish the wearable device and the body part from other objects in the environment. In some cases, the wearable device can include one or more visual patterns that the XR device can detect from one or more images to recognize the wearable device in the environment.

[0122]

[0128] In some aspects, the process 500 may include sending, by the wearable device (e.g., via a wireless communication interface), an input to the XR device configured to trigger a privacy mode at the XR device. In some examples, the privacy mode may include an operating state in which one or more image sensors (e.g., image sensor 102, image sensor 104) at the XR device are turned off and / or disabled. The input may be based on the data acquired at block 504 and / or one or more associated measurements from one or more sensors on the wearable device. In some examples, the data may indicate a touch signal corresponding to a touch input at the wearable device, a location of the wearable device that is different from the location of the body part, and / or a distance between the wearable device and the body part. For example, the data may include an indication and / or one or more measurements that indicate that the wearable device is not worn by the user at the body part (e.g., is at a different location and / or is within a threshold distance).

[0123]

[0129] In some aspects, the process 500 may include sending an additional input to the XR device (e.g., via a wireless communication interface) by the wearable device configured to trigger the XR device to stop the privacy mode. The input may be based on a touch signal corresponding to a touch input at the wearable device, a location of the wearable device corresponding to a location of the body part (e.g., indicating that a user may be wearing the wearable device on the body part), and / or data indicating a proximity between the wearable device and the body part (e.g., indicating that a user may be wearing the wearable device on the body part).

[0124]

[0130] In some aspects, the process 500 may include sending, by the wearable device (e.g., via a wireless communication interface), an XR input to the XR device associated with an XR application at the XR device. The XR input may be based on one or more measurements from one or more sensors on the wearable device.

[0125]

[0131] In some aspects, the process 500 may include sending, by the wearable device (e.g., via a wireless communication interface), an input to the XR device configured to trigger an adjustment of device settings in the XR device and / or one or more XR actions in the XR device. In some examples, the device settings may include a power mode (e.g., off, on, lower power, higher power, etc.) associated with one or more image sensors in the XR device, a frame rate associated with one or more image sensors, a resolution associated with one or more image sensors, etc. In some examples, the one or more XR actions may include object detection, object classification, gesture detection and / or recognition, pose estimation, shape estimation, etc.

[0126]

[0132] 5B is a flowchart illustrating an example process 520 for tracking an object. In some examples, the process 520 can be used with a wearable device (e.g., wearable device 150, wearable ring 200) in conjunction with an electronic device (e.g., XR system 100, etc.) to improve tracking, power saving, and / or privacy features in the electronic device. In some examples, the process 520 can be implemented by an electronic device, such as the XR system 100. In some examples, the electronic device can include a mobile phone, a laptop, a tablet, a head mounted display, smart glasses, a camera system, and / or any other electronic device.

[0127]

[0133] At block 522, process 520 may include determining a first position of the wearable device (e.g., wearable device 150, wearable ring 200) in physical space. In some examples, determining the first position of the wearable device may include receiving from the wearable device image data from one or more image sensors on the electronic device and / or data related to one or more measurements from one or more sensors on the wearable device, and determining the first position of the wearable device based on the image data from the one or more image sensors and / or data related to the one or more measurements from the one or more sensors.

[0128]

[0134] In some examples, the data may include a distance of the wearable device relative to one or more objects (e.g., a wall, a door, furniture, a device, a person, an animal, etc.), a velocity vector indicating a speed of movement of the wearable device, touch signals measured by touch sensors from one or more sensors, audio data from audio sensors from one or more sensors, and / or an altitude of the wearable device in physical space. In some cases, the one or more objects may include an electronic device, a body part associated with a user of the wearable device (e.g., a hand, a leg, an arm, a head, a torso, etc.), and / or an input device (e.g., a controller, a keyboard, a remote control, etc.).

[0129]

[0135] In some cases, the wearable device may include a bracelet, a ring, or a glove.

[0130]

[0136] At block 524, process 520 may include receiving, from the wearable device, location information associated with the wearable device. In some examples, the location information may include an attitude of the wearable device. In some cases, the location information may be based on sensor data from one or more sensors on the wearable device. In some cases, the location information may include measurements from an inertial measurement unit from one or more sensors on the wearable device and / or altitude measured by a pressure sensor from one or more sensors.

[0131]

[0137] At block 526, the process 520 may include determining a second position of the wearable device based on the received position information. In some cases, the second position may include a position of the wearable device relative to the electronic device. In some cases, the second position may include a position of the wearable device within a coordinate system, such as a coordinate system of the wearable device and / or a coordinate system of the electronic device.

[0132]

[0138] At block 528, the process 520 may include tracking movement of the wearable device relative to the electronic device based on the first location and the second location.

[0133]

[0139] In some aspects, tracking movement of the wearable device may include determining a first position of the wearable device in a first coordinate system of the wearable device, transforming the first coordinate system of the wearable device to a second coordinate system of the electronic device, and determining a second position of the wearable device in the second coordinate system of the electronic device.

[0134]

[0140] In some aspects, process 520 may include determining whether the wearable device is within an FOV of and / or visible to one or more image sensors on the electronic device based on the second position of the wearable device and / or the tracked movement of the wearable device. In some aspects, process 520 may include tracking a location of a hand associated with the wearable device based on the second position of the wearable device and / or the tracked movement of the wearable device. In some examples, the hand may include a hand wearing or holding the wearable device (e.g., on a wrist, on a finger, etc.).

[0135]

[0141] In some aspects, process 520 may include capturing one or more images of the hand via at least one image sensor from the one or more image sensors based on a determination that the wearable device is within the FOV of and / or visible to the one or more image sensors, and tracking a location of the hand based on the one or more images of the hand. In some examples, the location of the hand is tracked relative to a first coordinate system of the wearable device.

[0136]

[0142] In some cases, process 520 may include determining, based on the location information, that the wearable device is outside the FOV of one or more image sensors and moving toward an area within the FOV of the one or more image sensors, and initiating one or more imaging operations and / or one or more tracking operations at the electronic device based on determining that the wearable device is outside the FOV of the one or more image sensors and moving toward an area within the FOV of the one or more image sensors. In some examples, the one or more tracking operations may be based at least in part on image data from the one or more imaging operations.

[0137]

[0143] In some embodiments, the process 520 may include adjusting a first setting of the first image sensor based on a first determination that the wearable device is within a first FOV of a first image sensor on the electronic device and / or a second determination that the wearable device is visible to the first image sensor on the electronic device. In some cases, the first setting may include a power mode of the first image sensor and / or an operational state of the first image sensor. In some embodiments, the process 520 may include adjusting a second setting of the second image sensor based on a third determination that the wearable device is outside a second FOV of a second image sensor on the electronic device and / or a fourth determination that the wearable device is not visible to the second image sensor on the electronic device. In some examples, the second setting may include a power mode of the second image sensor and / or an operational state of the second image sensor.

[0138]

[0144] In some examples, adjusting the first setting of the first image sensor may include changing a power mode of the first image sensor from the first power mode to a second power mode comprising a higher power mode than the first power mode and / or changing an operating state of the first image sensor from the first operating state to a second operating state comprising a higher operating state than the first operating state. In some examples, the second operating state may include a higher frame rate and / or a higher resolution.

[0139]

[0145] In some examples, adjusting the second setting of the second image sensor may include changing a power mode of the second image sensor from the first power mode to a second power mode comprising a lower power mode than the first power mode and / or changing an operating state of the second image sensor from the first operating state to a second operating state comprising a lower operating state than the first operating state. In some cases, the second operating state may include a lower frame rate and / or a lower resolution.

[0140]

[0146] In some aspects, process 520 may include tracking a location of the wearable device based on additional position information from the wearable device in response to determining that the wearable device is outside the FOV of one or more image sensors on the electronic device and / or that the view of the one or more image sensors to the wearable device is obstructed by one or more objects. In some aspects, process 520 may include tracking a location of the wearable device based on additional position information from the wearable device in response to determining that the wearable device is within the FOV of one or more image sensors but the view of the one or more image sensors to the wearable device is obstructed.

[0141]

[0147] In some aspects, process 520 may include initializing one or more image sensors in response to a determination that the wearable device is within the FOV of the one or more image sensors but that the view of the one or more image sensors to the wearable device is obstructed.

[0142]

[0148] In some aspects, process 520 may include receiving an input from the wearable device configured to trigger a privacy mode at the electronic device and adjusting an operational state of one or more image sensors at the electronic device to an off state and / or a disabled state based on the input configured to trigger the privacy mode. In some examples, the input may be based on sensor data from one or more sensors on the wearable device. In some cases, the sensor data may indicate a touch signal corresponding to a touch input at the wearable device, a location of the wearable device, and / or a distance between the wearable device and a body part of a user of the wearable device.

[0143]

[0149] In some aspects, process 520 may include receiving an additional input from the wearable device configured to trigger the electronic device to terminate the privacy mode. In some cases, the additional input may be based on a touch signal corresponding to a touch input at the wearable device, a location of the wearable device corresponding to a location of a body part of the user of the wearable device, and / or sensor data indicative of a proximity between the wearable device and the body part.

[0144]

[0150] In some aspects, process 520 may include determining one or more extended reality (XR) inputs to an XR application on the electronic device based on data from the wearable device and / or commands from the wearable device. In some examples, the one or more XR inputs may include a modification of a virtual element along multiple dimensions in space, a selection of a virtual element, a navigation event, and / or a request to measure a distance defined by a first position of the wearable device, a second position of the wearable device, and / or a movement of the wearable device.

[0145]

[0151] In some examples, the virtual element can include a virtual object rendered by the electronic device, a virtual plane in an environment rendered by the electronic device, and / or an environment rendered by the electronic device. In some examples, the navigation event can include scrolling the rendered content and / or moving from a first interface element to a second interface element.

[0146]

[0152] In some aspects, the process 520 may include receiving an input from the wearable device configured to trigger an adjustment of one or more XR actions in the electronic device. In some examples, the one or more XR actions may include object detection, object classification, object tracking, pose estimation, and / or shape estimation.

[0147]

[0153] In some examples, process 500 or process 520 may be performed by one or more computing devices or apparatuses. In one illustrative example, process 500 may be performed by XR system 100 and / or wearable device 150 shown in FIG. 1 and / or one or more computing devices with computing device architecture 600 shown in FIG. 6. In another illustrative example, process 520 may be performed by XR system 100 and / or one or more computing devices with computing device architecture 600 shown in FIG. 6. In some cases, such computing devices or apparatuses may include a processor, microprocessor, microcomputer, or other components of a device configured to perform steps of process 500 or process 520. In some examples, such computing devices or apparatuses may include one or more sensors configured to capture image data and / or other sensor measurements. For example, computing devices may include smartphones, head-mounted displays, mobile devices, or other suitable devices. In some examples, such computing devices or apparatuses may include a camera configured to capture one or more images or videos. In some cases, such computing devices may include a display for displaying images. In some examples, the one or more sensors and / or cameras are separate from the computing device, in which case the computing device receives the sensed data. Such a computing device may further include a network interface configured to communicate the data.

[0148]

[0154] The components of a computing device may be implemented in circuits. For example, the components may include one or more programmable electronic circuits (e.g., a microprocessor, a graphics processing unit (GPU), a digital signal processor (DSP), a central processing unit (CPU), and / or other suitable electronic circuitry) and / or may include and / or be implemented using electronic circuitry or other electronic hardware that may include and / or be implemented using computer software, firmware, or any combination thereof, to perform various operations described herein. The computing device may further include a display (as an example of an output device, or in addition to an output device), a network interface configured to communicate and / or receive data, any combination thereof, and / or other component(s). The network interface may be configured to communicate and / or receive Internet Protocol (IP)-based data or other types of data.

[0149]

[0155] Process 500 and process 520 are illustrated as logical flow diagrams, whose operations represent sequences of operations that may be implemented in hardware, computer instructions, or a combination thereof. In the context of computer instructions, the operations represent computer-executable instructions stored on one or more computer-readable storage media that, when executed by one or more processors, perform the recited operations. Generally, computer-executable instructions include routines, programs, objects, components, data structures, etc. that perform particular functions or implement particular data types. The order in which the operations are described is not to be construed as a limitation, and any number of the described operations may be combined in any order and / or in parallel to implement a process.

[0150]

[0156] Further, process 500 or process 520 may be performed under the control of one or more computer systems configured with executable instructions and may be implemented as code (e.g., executable instructions, one or more computer programs, or one or more applications) that collectively execute on one or more processors, by hardware, or a combination thereof. As mentioned above, the code may be stored in a computer-readable or machine-readable storage medium, for example, in the form of a computer program comprising a plurality of instructions executable by one or more processors. The computer-readable or machine-readable storage medium may be non-transitory.

[0151]

[0157] 6 illustrates an exemplary computing device architecture 600 of an exemplary computing device that can implement various techniques described herein. For example, the computing device architecture 600 can implement at least some portions of the XR system 100 illustrated in FIG. 1. The components of the computing device architecture 600 are shown in electrical communication with each other using a connection 605, such as a bus. The exemplary computing device architecture 600 includes a processing unit (CPU or processor) 610 and a computing device connection 605 that couples various computing device components, including a computing device memory 615, such as a read only memory (ROM) 620 and a random access memory (RAM) 625, to the processor 610.

[0152]

[0158] The computing device architecture 600 may include a cache of high speed memory directly connected to the processor 610, in close proximity to the processor 610, or integrated as part of the processor 610. The computing device architecture 600 may copy data from the memory 615 and / or storage device 630 to the cache 612 for quick access by the processor 610. In this manner, the cache may provide performance improvements that avoid processor 610 delays while waiting for data. These and other modules may control or be configured to control the processor 610 to perform various actions. Other computing device memories 615 may also be available for use. The memory 615 may include multiple different types of memory with different performance characteristics. The processor 610 may include any general purpose processor and hardware or software services stored in the storage device 630 and configured to control the processor 610 as well as special purpose processors, where software instructions are built into the processor design. The processor 610 may be a self-contained system including multiple cores or processors, buses, memory controllers, caches, etc. Multi-core processors can be symmetric or asymmetric.

[0153]

[0159] To enable user interaction with the computing device architecture 600, the input device 645 can represent any number of input mechanisms, such as a microphone for speech, a touch-sensitive screen for gesture or graphical input, a keyboard, a mouse, motion input, speech, etc. The output device 665 can also be one or more of several output mechanisms known to those skilled in the art, such as a display, a projector, a television, a speaker device, etc. In some cases, a multimodal computing device can enable a user to provide multiple types of input to communicate with the computing device architecture 600. The communication interface 640 can generally govern and manage user input and computing device output. There is no restriction to operating on any particular hardware configuration, and thus the basic features herein can be easily substituted with improved hardware or firmware configurations as they are developed.

[0154]

[0160] The storage device 630 is a non-volatile memory and may be a hard disk or other type of computer readable medium capable of storing data that is accessible by a computer, such as a magnetic cassette, a flash memory card, a solid-state memory device, a digital versatile disk, a cartridge, a random access memory (RAM) 625, a read only memory (ROM) 620, and hybrids thereof. The storage device 630 may include software, code, firmware, etc. for controlling the processor 610. Other hardware or software modules are contemplated. The storage device 630 may be connected to a computing device connection 605. In one aspect, a hardware module that performs a particular function may include software components stored in a computer readable medium in association with the necessary hardware components, such as the processor 610, the connection 605, the output device 665, etc., to perform that function.

[0155]

[0161] The term "computer-readable medium" includes, but is not limited to, portable or non-portable storage devices, optical storage devices, and various other media capable of storing, containing, or transporting instruction(s) and / or data. Computer-readable media may include non-transitory media on which data may be stored, which does not include carrier waves and / or transitory electronic signals propagating wirelessly or over wired connections. Examples of non-transitory media may include, but are not limited to, magnetic disks or tapes, optical storage media such as compact disks (CDs) or digital versatile disks (DVDs), flash memory, memory or memory devices. A computer-readable medium may have code and / or machine-executable instructions stored thereon, which may represent a procedure, a function, a subprogram, a program, a routine, a subroutine, a module, a software package, a class, or any combination of instructions, data structures, or program statements. A code segment may be coupled to another code segment or a hardware circuit by passing and / or receiving information, data, arguments, parameters, or memory contents. Information, arguments, parameters, data, etc. may be passed, forwarded, or transmitted via any suitable means including memory sharing, message passing, token passing, network transmission, etc.

[0156]

[0162] In some embodiments, computer-readable storage devices, media, and memories may include cable or wireless signals containing bit streams, etc. However, when stated, non-transitory computer-readable storage media specifically excludes media such as energy, carrier signals, electromagnetic waves, and the signals themselves.

[0157]

[0163] Specific details are provided in the above description to provide a thorough understanding of the embodiments and examples provided herein. However, those skilled in the art will appreciate that the embodiments may be practiced without these specific details. For clarity of explanation, in some cases, the technology may be presented as including individual functional blocks comprising devices, device components, steps or routines in a method implemented in software, or a combination of hardware and software. Additional components other than those shown in the figures and / or described herein may be used. For example, circuits, systems, networks, processes, and other components may be shown as components in block diagram form so as not to obscure the embodiments with unnecessary detail. In other cases, well-known circuits, processes, algorithms, structures, and techniques may be shown without unnecessary detail so as to avoid obscuring the embodiments.

[0158]

[0164] Individual embodiments may be described above as a process or method that is depicted as a flowchart, a flow diagram, a data flow diagram, a structure diagram, or a block diagram. Although the flowchart may describe operations as a sequential process, many of the operations may be performed in parallel or simultaneously. Moreover, the order of operations may be rearranged. A process is terminated when an operation of a process is completed, but may have additional steps not included in the diagram. A process may correspond to a method, a function, a procedure, a subroutine, a subprogram, etc. When a process corresponds to a function, its termination may correspond to a return of the function to the calling function or to a main function.

[0159]

[0165] The processes and methods according to the examples described above may be implemented using computer-executable instructions stored or otherwise available from a computer-readable medium. Such instructions may include, for example, instructions and data that cause or otherwise configure a general purpose computer, a special purpose computer, or a processing device to perform a certain function or group of functions. Portions of the computer resources used may be accessible over a network. The computer-executable instructions may be, for example, binaries, intermediate format instructions such as assembly language, firmware, source code. Examples of computer-readable media that may be used to store instructions, information used, and / or information created during methods according to the described examples include magnetic or optical disks, flash memory, USB devices with non-volatile memory, networked storage devices, and the like.

[0160]

[0166] Devices implementing the processes and methods according to these disclosures may include hardware, software, firmware, middleware, microcode, hardware description languages, or any combination thereof, and may take any of a variety of form factors. When implemented in software, firmware, middleware, or microcode, the program code or code segments (e.g., computer program product) for performing the necessary tasks may be stored in a computer-readable or machine-readable medium. A processor(s) may perform the necessary tasks. Common examples of form factors include laptops, smartphones, mobile phones, tablet devices or other small form factor personal computers, personal digital assistants, rack-mounted devices, standalone devices, etc. The functionality described herein may also be embodied in peripheral devices or add-in cards. Such functionality may also be implemented on a circuit board in different chips or different processes executing in a single device, as further examples.

[0161]

[0167] The instructions, media for carrying such instructions, computing resources for executing them, and other structures for supporting such computing resources are exemplary means for providing the functionality described in this disclosure.

[0162]

[0168] In the above description, aspects of the present application have been described with reference to specific embodiments thereof, but those skilled in the art will recognize that the present application is not limited thereto. Thus, although exemplary embodiments of the present application have been described in detail herein, it should be understood that the inventive concepts may be embodied and employed in various ways, and the appended claims are to be construed to include such variations, except as limited by the prior art. Various features and aspects of the applications described above may be used individually or together. Moreover, the embodiments may be utilized in any number of environments and applications other than those described herein without departing from the broader spirit and scope of the present specification. Thus, the present specification and drawings should be considered illustrative rather than restrictive. For purposes of illustration, the methods have been described in a particular order. It should be appreciated that in alternative embodiments, the methods may be performed in an order different from that described.

[0163]

[0169] Those skilled in the art will appreciate that the less than ("<") and greater than (">") symbols or terminology used herein may be replaced with the less than or equal to ("≦") and greater than or equal to ("≧") symbols, respectively, without departing from the scope of the present specification.

[0164]

[0170] When a component is described as being "configured to" perform some operation, such configuration may be achieved, for example, by designing electronic circuitry or other hardware to perform the operation, by programming a programmable electronic circuit (e.g., a microprocessor or other suitable electronic circuitry) to perform the operation, or any combination thereof.

[0165]

[0171] The phrase "coupled to" refers to any component that is physically connected, either directly or indirectly, to another component, and / or any component that is in communication, either directly or indirectly, with another component (e.g., connected to the other component via a wired or wireless connection, and / or other suitable communications interface).

[0166]

[0172] Claim language or other language in this disclosure reciting "at least one of" a set and / or "one or more" of a set indicates that one member of the set or multiple members of the set (in any combination) satisfy the claim. For example, claim language reciting "at least one of A and B" or "at least one of A or B" means A, B, or A and B. In another example, claim language reciting "at least one of A, B, and C" or "at least one of A, B, or C" means A, B, C, or A and B, or A and C, or B and C, or A and B and C. The language "at least one of" a set and / or "one or more" of a set does not limit the set to the items listed in the set. For example, claim language reciting "at least one of A and B" or "at least one of A or B" can mean A, B, or A and B, and can further include items not recited in the set of A and B.

[0167]

[0173] The various exemplary logic blocks, modules, circuits, and algorithm steps described with respect to the examples disclosed herein may be implemented as electronic hardware, computer software, firmware, or a combination thereof. To clearly illustrate this interchangeability of hardware and software, the various exemplary components, blocks, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends on the particular application and design constraints imposed on the overall system. Those skilled in the art may implement the described functionality in various ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.

[0168]

[0174] The techniques described herein may also be implemented in electronic hardware, computer software, firmware, or any combination thereof. Such techniques may be implemented in any of a variety of devices, such as a general purpose computer, a wireless communication device handset, or an integrated circuit device having multiple uses, including applications in wireless communication device handsets and other devices. Features described as modules or components may be implemented together in an integrated logic device, or separately as separate but interoperable logic devices. If implemented in software, the techniques may be realized at least in part by a computer-readable data storage medium comprising program code including instructions that, when executed, perform one or more of the methods, algorithms, and / or operations described above. The computer-readable data storage medium may form part of a computer program product, which may include packaging materials. The computer-readable medium may comprise a memory or data storage medium, such as a random access memory (RAM), such as a synchronous dynamic random access memory (SDRAM), a read-only memory (ROM), a non-volatile random access memory (NVRAM), an electrically erasable programmable read-only memory (EEPROM), a flash memory, a magnetic or optical data storage medium, or the like. The techniques may additionally or alternatively be realized at least in part by a computer-readable communications medium, such as a propagated signal or radio wave, that carries or communicates program code in the form of instructions or data structures and that can be accessed, read, and / or executed by a computer.

[0169]

[0175] The program code may be executed by a processor, which may include one or more processors, such as one or more digital signal processors (DSPs), general-purpose microprocessors, application specific integrated circuits (ASICs), field programmable logic arrays (FPGAs), or other equivalent integrated or discrete logic circuitry. Such a processor may be configured to perform any of the techniques described in this disclosure. A general-purpose processor may be a microprocessor, but alternatively, the processor may be any conventional processor, controller, microcontroller, or state machine. A processor may also be implemented as a combination of computing devices, e.g., a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors in conjunction with a DSP core, or any other such configuration. Thus, the term "processor" as used herein may refer to any of the above structures, any combination of the above structures, or any other structure or apparatus suitable for implementing the techniques described herein.

[0170]

[0176] Illustrative examples of the present disclosure include the following:

[0171]

[0177] Aspect 1. A wearable device comprising: a structure defining a receiving space configured to receive a body part associated with a user; one or more sensors integrated with the structure, the structure having an engagement surface configured to contact the received body part via the receiving space; and a wireless transmitter configured to transmit at least one of a position of the structure in physical space and a movement of the structure to an extended reality device, the one or more sensors configured to obtain one or more tracking measurements associated with the wearable device, the one or more tracking measurements comprising at least one of a position of the structure in physical space and a movement of the structure.

[0172]

[0178] Aspect 2. A wearable device as described in aspect 1, wherein the one or more tracking measurements further comprise at least one of: a distance of the structure relative to one or more objects; a velocity vector indicating a speed of movement of the structure; and an altitude of the structure in physical space.

[0173]

[0179] Aspect 3. A wearable device as described in any of aspects 1 to 2, wherein the one or more objects comprise at least one of an XR device, a different body part associated with the user, and an input device.

[0174]

[0180] Aspect 4. A wearable device as described in any of aspects 1 to 3, wherein the body part associated with the user comprises at least one of a finger, a hand, and a wrist, and wherein a different body part associated with the user comprises at least one of a different finger, a different hand, and a different wrist.

[0175]

[0181] Aspect 5. A wearable device as described in any of aspects 1 to 4, wherein the wearable device is configured to send an input via the wireless transmitter to the XR device configured to trigger a privacy mode in the XR device, wherein the privacy mode comprises an operating state in which one or more image sensors in the XR device are at least one of turned off and disabled.

[0176]

[0182] Aspect 6. A wearable device as described in any of aspects 1 to 5, wherein the input is based on one or more measurements from one or more sensors, where the one or more measurements indicate at least one of: a touch signal corresponding to a touch input at the wearable device, a first location of the wearable device that is different from a second location of the body part, and a distance between the wearable device and the body part.

[0177]

[0183] Aspect 7. A wearable device as described in any of aspects 1 to 6, wherein the wearable device is configured to send an additional input via the wireless transmitter to the XR device configured to trigger the XR device to cease privacy mode, wherein the input is based on sensor data indicating at least one of a touch signal corresponding to a touch input at the wearable device, a location of the wearable device corresponding to a location of the body part, and a proximity between the wearable device and the body part.

[0178]

[0184] Aspect 8. A wearable device as described in any of aspects 1 to 7, wherein the wearable device is configured to send XR input related to an XR application in the XR device via the wireless transmitter to the XR device, the XR input being based on one or more measurements from one or more sensors.

[0179]

[0185] Aspect 9. A wearable device as described in any of aspects 1 to 8, wherein the wearable device is configured to send an input via the wireless transmitter to the XR device, the input configured to trigger an adjustment of at least one of device settings at the XR device and one or more XR actions at the XR device.

[0180]

[0186] Aspect 10. A wearable device as described in any of aspects 1 to 9, wherein the device settings comprise at least one of a power mode associated with one or more image sensors, a frame rate associated with one or more image sensors, and a resolution associated with one or more image sensors, and wherein the one or more XR actions comprise at least one of object detection, object classification, pose estimation, and shape estimation.

[0181]

[0187] Embodiment 11. A wearable device as described in any of embodiments 1 to 10, wherein the one or more sensors include at least one of an accelerometer, a gyroscope, a pressure sensor, an audio sensor, a touch sensor, and a magnetometer.

[0182]

[0188] Aspect 12. A wearable device according to any of aspects 1 to 11, wherein the wearable device comprises a ring, and wherein the body part comprises a user's finger.

[0183]

[0189] Aspect 13. A wearable device according to any of aspects 1 to 11, wherein the wearable device comprises a bracelet, and wherein the body part comprises the user's wrist.

[0184]

[0190] Aspect 14. A wearable device according to any of aspects 1 to 11, wherein the wearable device comprises a glove, and wherein the body part comprises the user's hand.

[0185]

[0191] Aspect 15. A method for processing tracking data, the method comprising: establishing a wireless connection between a wearable device and an extended reality device; the wearable device comprising a structure defining a receptive space configured to receive a body part associated with a user, the structure comprising a surface configured to contact the received body part via the receptive space; acquiring one or more tracking measurements associated with the wearable device via one or more sensors integrated with a structure associated with the wearable device; and transmitting at least one of a position of the structure in the physical space and a movement of the structure to an XR device via a wireless transmitter of the wearable device, the one or more tracking measurements comprising at least one of a position of the structure in the physical space and a movement of the structure.

[0186]

[0192] Aspect 16. The method of aspect 15, wherein the one or more tracking measurements further comprise at least one of a distance of the structure relative to the one or more objects, a velocity vector indicating a speed of movement of the structure, and an altitude of the structure in physical space.

[0187]

[0193] Aspect 17. A method as described in any of aspects 15 to 16, wherein the one or more objects comprise at least one of an XR device, a second body part associated with the user, and an input device.

[0188]

[0194] Example 18. A method as described in any of examples 15 to 17, wherein the body part associated with the user comprises at least one of a finger, a hand, and a wrist, and wherein the second body part associated with the user comprises at least one of a second finger, a second hand, and a second wrist.

[0189]

[0195] Aspect 19. The method of any of aspects 15 to 18, further comprising sending an input to the XR device via a wireless transmitter of the wearable device, the input being configured to trigger a privacy mode in the XR device, wherein the privacy mode comprises an operating state in which one or more image sensors in the XR device are at least one of turned off and disabled.

[0190]

[0196] Aspect 20. A method according to any of aspects 15 to 19, wherein the input is based on one or more measurements from one or more sensors, where the one or more measurements indicate at least one of a touch signal corresponding to a touch input at the wearable device, a first location of the wearable that is different from a second location of the body part, and a distance between the wearable and the body part.

[0191]

[0197] Aspect 21. The method of any of aspects 15 to 20, further comprising sending an additional input to the XR device via a wireless transmitter of the wearable device, the additional input configured to trigger the XR device to cease a privacy mode, wherein the input is based on sensor data indicating at least one of: a touch signal corresponding to a touch input at the wearable device, a location of the wearable device corresponding to a location of the body part, and a proximity between the wearable device and the body part.

[0192]

[0198] Aspect 22. The method of any of aspects 15 to 21, further comprising sending an XR input related to an XR application on the XR device to the XR device via a wireless transmitter of the wearable device, the XR input being based on one or more measurements from one or more sensors.

[0193]

[0199] Aspect 23. The method of any of aspects 15 to 22, further comprising sending an input to the XR device via a wireless transmitter of the wearable device, the input configured to trigger adjustment of at least one of device settings at the XR device and one or more XR actions at the XR device.

[0194]

[0200] Aspect 24. The method of any of aspects 15 to 23, wherein the device settings comprise at least one of a power mode associated with one or more image sensors, a frame rate associated with one or more image sensors, and a resolution associated with one or more image sensors, and wherein the one or more XR actions comprise at least one of object detection, object classification, pose estimation, and shape estimation.

[0195]

[0201] Aspect 25. The method of any of aspects 15 to 24, wherein the wearable device comprises a ring, and wherein the body part comprises a finger of the user.

[0196]

[0202] Aspect 26. The method of any of aspects 15 to 25, wherein the wearable device comprises a bracelet, and wherein the body part comprises the user's wrist.

[0197]

[0203] Aspect 27. The method of any of aspects 15 to 26, wherein the wearable device comprises a glove and wherein the body part comprises the user's hand.

[0198]

[0204] Embodiment 28. The method of any of embodiments 15 to 27, wherein the one or more sensors comprise at least one of an accelerometer, a gyroscope, a pressure sensor, an audio sensor, a touch sensor, and a magnetometer.

[0199]

[0205] Aspect 29. A non-transitory computer-readable medium storing instructions that, when executed by one or more processors, cause the one or more processors to perform a method according to any of aspects 15 to 27.

[0200]

[0206] Example 30. A wearable device comprising means for performing the method of any of examples 15 to 27.

[0201]

[0207] Embodiment 31. The wearable device of embodiment 30, wherein the one or more sensors comprise at least one of an accelerometer, a gyroscope, a pressure sensor, an audio sensor, a touch sensor, and a magnetometer.

[0202]

[0208] Aspect 32. The wearable device of aspect 30 or 31, wherein the wearable device comprises a glove, and wherein the body part comprises the user's hand.

[0203]

[0209] Aspect 33. The wearable device of aspect 30 or 31, wherein the wearable device comprises a ring, and wherein the body part comprises a user's finger.

[0204]

[0210] Aspect 34. The wearable device of aspect 30 or 31, wherein the wearable device comprises a bracelet, and wherein the body part comprises the user's wrist.

[0205]

[0211] Aspect 35. A method for processing tracking data, the method comprising: establishing a wireless connection between a wearable device and an extended reality (XR) device; receiving from the wearable device one or more tracking measurements calculated by one or more sensors of the wearable device, the wearable device comprising a structure defining a receptive space configured to receive a body part associated with a user, the structure comprising a surface configured to contact the received body part via the receptive space; and translating, by the XR device, the one or more tracking measurements comprising at least one of a position of the structure in physical space and a movement of the structure, the one or more tracking measurements into one or more XR inputs to an XR application at the XR device.

[0206]

[0212] Aspect 36. The method of aspect 35, wherein the one or more tracking measurement results further comprise at least one of a distance of the structure relative to the one or more objects, a velocity vector indicating a speed of movement of the structure, and an altitude of the structure in physical space.

[0207]

[0213] Aspect 37. The method of aspect 36, wherein the one or more objects comprise at least one of an XR device, a second body part associated with the user, and an input device.

[0208]

[0214] Aspect 38. The method of aspect 37, wherein the body part associated with the user comprises at least one of a finger, a hand, and a wrist, and wherein the second body part associated with the user comprises at least one of a second finger, a second hand, and a second wrist.

[0209]

[0215] Aspect 39. A method as described in any of aspects 35 to 37, wherein the one or more XR inputs comprise at least one of a request to measure a distance defined by at least one of a position of a structure in physical space and a movement of the structure, a modification of a virtual element along multiple dimensions in space, a selection of a virtual element, and a navigation event.

[0210]

[0216] Aspect 40. The method of aspect 39, wherein the virtual element comprises at least one of a virtual object rendered by the XR device, a virtual plane in the environment rendered by the XR device, and an environment rendered by the XR device.

[0211]

[0217] Aspect 41. The method of aspect 39, wherein the navigation event comprises at least one of scrolling the rendered content and moving from a first interface element to a second interface element.

[0212]

[0218] Aspect 42. The method of any of aspects 35 to 41, further comprising receiving an input from the wearable device configured to trigger a privacy mode in the XR device, wherein the privacy mode comprises an operating state in which one or more image sensors in the XR device are at least one of turned off and disabled.

[0213]

[0219] Aspect 43. The method of aspect 42, wherein the input is based on one or more measurements from one or more sensors, where the one or more measurements indicate at least one of a touch signal corresponding to a touch input at the wearable device, a first location of the wearable that is different from a second location of the body part, and a distance between the wearable and the body part.

[0214]

[0220] Aspect 44. The method of aspect 42, further comprising receiving an additional input from the wearable device to the XR device configured to trigger the XR device to cease privacy mode, where the input is based on sensor data indicating at least one of a touch signal corresponding to a touch input at the wearable device, a location of the wearable device corresponding to a location of the body part, and a proximity between the wearable device and the body part.

[0215]

[0221] Aspect 45. The method of any of aspects 35 to 44, further comprising receiving an input from the wearable device configured to trigger an adjusted power mode in the XR device, where the adjusted power mode comprises a lower power state relative to a power state prior to the adjusted power mode.

[0216]

[0222] Aspect 46. The method of any of aspects 35 to 45, further comprising receiving an input from the wearable device to the XR device configured to trigger adjustment of at least one of device settings at the XR device and one or more XR actions at the XR device.

[0217]

[0223] Aspect 47. The method of aspect 46, wherein the device settings comprise at least one of a power mode associated with one or more image sensors, a frame rate associated with one or more image sensors, and a resolution associated with one or more image sensors.

[0218]

[0224] Aspect 48. The method of aspect 46 or 47, wherein the one or more XR operations comprise at least one of object detection, object classification, pose estimation, and shape estimation.

[0219]

[0225] Embodiment 49. The method of any of embodiments 35 to 48, wherein the one or more sensors comprise at least one of an accelerometer, a gyroscope, a pressure sensor, an audio sensor, a touch sensor, and a magnetometer.

[0220]

[0226] Aspect 50. The method of any of aspects 35 to 49, wherein the wearable device comprises a bracelet, and wherein the body part comprises the user's wrist.

[0221]

[0227] Embodiment 51. The method of any of embodiments 35 to 49, wherein the wearable device comprises a ring, and wherein the body part comprises a finger of the user.

[0222]

[0228] Embodiment 52. The method of any of embodiments 35 to 49, wherein the wearable device comprises a glove and wherein the body part comprises the user's hand.

[0223]

[0229] Aspect 53. An apparatus for processing tracking data, the apparatus comprising: a memory; and one or more processors coupled to the memory, the one or more processors configured to: establish a wireless connection between a wearable device and an extended reality (XR) device; receive from the wearable device, the wearable device comprising a structure defining a receptive space configured to receive a body part associated with a user, the structure comprising a surface configured to contact the received body part via the receptive space, one or more tracking measurements calculated by one or more sensors of the wearable device; and translate by the XR device the one or more tracking measurements into one or more XR inputs to an XR application at the XR device, the one or more tracking measurements comprising at least one of a position of the structure in physical space and a movement of the structure.

[0224]

[0230] Aspect 54. The apparatus of aspect 53, wherein the one or more tracking measurement results further comprise at least one of a distance of the structure relative to one or more objects, a velocity vector indicating a speed of movement of the structure, and an altitude of the structure in physical space.

[0225]

[0231] Aspect 55. The apparatus of aspect 54, wherein the one or more objects comprise at least one of an XR device, a second body part associated with the user, and an input device.

[0226]

[0232] Aspect 56. The apparatus of aspect 55, wherein the body part associated with the user comprises at least one of a finger, a hand, and a wrist, and wherein the second body part associated with the user comprises at least one of a second finger, a second hand, and a second wrist.

[0227]

[0233] Aspect 57. An apparatus as described in any of aspects 53 to 55, wherein the one or more XR inputs comprise at least one of a request to measure a distance defined by at least one of a position of a structure in physical space and a movement of the structure, modification of a virtual element along multiple dimensions in space, selection of a virtual element, and a navigation event.

[0228]

[0234] Aspect 58. The apparatus of aspect 57, wherein the virtual element comprises at least one of a virtual object rendered by the XR device, a virtual plane in the environment rendered by the XR device, and an environment rendered by the XR device.

[0229]

[0235] Aspect 59. The apparatus of aspect 57, wherein the navigation event comprises at least one of scrolling rendered content and moving from a first interface element to a second interface element.

[0230]

[0236] Aspect 60. An apparatus as described in any of aspects 53 to 59, further comprising receiving an input from the wearable device configured to trigger a privacy mode in the XR device, wherein the privacy mode comprises an operating state in which one or more image sensors in the XR device are at least one of turned off and disabled.

[0231]

[0237] Embodiment 61. The apparatus of embodiment 60, wherein the input is based on one or more measurements from one or more sensors, where the one or more measurements indicate at least one of: a touch signal corresponding to a touch input at the wearable device, a first location of the wearable that is different from a second location of the body part, and a distance between the wearable and the body part.

[0232]

[0238] Aspect 62. The apparatus of aspect 60, further comprising receiving an additional input from the wearable device to the XR device configured to trigger the XR device to cease privacy mode, wherein the input is based on sensor data indicating at least one of a touch signal corresponding to a touch input at the wearable device, a location of the wearable device corresponding to a location of the body part, and a proximity between the wearable device and the body part.

[0233]

[0239] Aspect 63. The apparatus of any of aspects 53 to 62, further comprising receiving an input from the wearable device configured to trigger an adjusted power mode in the XR device, where the adjusted power mode comprises a lower power state relative to a power state prior to the adjusted power mode.

[0234]

[0240] Aspect 64. An apparatus as described in any of aspects 53 to 63, further comprising receiving an input from the wearable device to the XR device configured to trigger adjustment of at least one of device settings at the XR device and one or more XR actions at the XR device.

[0235]

[0241] Aspect 65. The apparatus of aspect 64, wherein the device settings comprise at least one of a power mode associated with the one or more image sensors, a frame rate associated with the one or more image sensors, and a resolution associated with the one or more image sensors.

[0236]

[0242] Example 66. The apparatus of example 64 or 65, wherein the one or more XR operations comprise at least one of object detection, object classification, pose estimation, and shape estimation.

[0237]

[0243] Embodiment 67. The device of any of embodiments 53 to 66, wherein the one or more sensors comprise at least one of an accelerometer, a gyroscope, a pressure sensor, an audio sensor, a touch sensor, and a magnetometer.

[0238]

[0244] Aspect 68. An apparatus described in any of aspects 53 to 67, wherein the wearable device comprises a bracelet, and wherein the body part comprises the user's wrist.

[0239]

[0245] Aspect 69. An apparatus as described in any of aspects 53 to 67, wherein the wearable device comprises a ring, and wherein the body part comprises a user's finger.

[0240]

[0246] Aspect 70. An apparatus as described in any of aspects 53 to 67, wherein the wearable device comprises a glove, and wherein the body part comprises the user's hand.

[0241]

[0247] Embodiment 71. The apparatus of any of embodiments 53 to 67, wherein the apparatus comprises a mobile device.

[0242]

[0248] Embodiment 72. An apparatus described in any of embodiments 53 to 67, wherein the apparatus comprises a camera.

[0243]

[0249] Embodiment 73. An apparatus described in any of embodiments 53 to 67, wherein the apparatus comprises an XR device and a display.

[0244]

[0250] Aspect 74. A non-transitory computer-readable medium storing instructions that, when executed by one or more processors, cause the one or more processors to perform a method according to any of aspects 35 to 52.

[0245]

[0251] Embodiment 75. An apparatus comprising means for carrying out the method according to any one of embodiments 35 to 52.

[0246]

[0252] Aspect 76. An apparatus comprising a memory and one or more processors coupled to the memory, the one or more processors configured to determine a first position of a wearable device in physical space, receive position information related to the wearable device from the wearable device, determine a second position of the wearable device based on the received position information, and track movement of the wearable device relative to the apparatus based on the first position and the second position.

[0247]

[0253] Aspect 77. The apparatus of aspect 76, wherein, to track movement of the wearable device, one or more processors are configured to determine a first position of the wearable device in a first coordinate system of the wearable device, transform the first coordinate system of the wearable device to a second coordinate system of the apparatus, and determine a second position of the wearable device in the second coordinate system of the apparatus.

[0248]

[0254] Aspect 78. A device described in any of aspects 76 to 77, wherein the one or more processors are configured to determine, based on at least one of the second position of the wearable device and the tracked movement of the wearable device, whether the wearable device is at least one of within a field of view (FOV) of one or more image sensors on the device and is visible to one or more image sensors on the device.

[0249]

[0255] Aspect 79. The apparatus of aspect 78, wherein the one or more processors are configured to track a location of a hand associated with the wearable device based on at least one of the second position of the wearable device and the tracked movement of the wearable device.

[0250]

[0256] Aspect 80. The apparatus of aspect 79, wherein the one or more processors are configured to: capture one or more images of the hand via at least one image sensor from the one or more image sensors based on a determination that the wearable device is within the FOV of and visible to the one or more image sensors; track a location of the hand further based on the one or more images of the hand; and the location of the hand is tracked relative to a first coordinate system of the wearable device.

[0251]

[0257] Aspect 81. The device of aspect 78, wherein the one or more processors are configured to: determine, based on the position information, that the wearable device is outside the FOV of one or more image sensors and moving toward an area within the FOV of the one or more image sensors; and, based on determining that the wearable device is outside the FOV of the one or more image sensors and moving toward an area within the FOV of the one or more image sensors, initiate one or more imaging operations and one or more tracking operations in the device, the one or more tracking operations being based at least in part on image data from the one or more imaging operations.

[0252]

[0258] Aspect 82. The apparatus of aspect 78, wherein the one or more processors are configured to: adjust a first setting of a first image sensor based on at least one of a first determination that the wearable device is within a first FOV of a first image sensor on the device and a second determination that the wearable device is visible to the first image sensor on the device, the first setting comprising at least one of a power mode of the first image sensor and an operational state of the first image sensor; adjust a second setting of a second image sensor based on at least one of a third determination that the wearable device is outside a second FOV of a second image sensor on the device and a fourth determination that the wearable device is not visible to the second image sensor on the device, the second setting comprising at least one of a power mode of the second image sensor and an operational state of the second image sensor.

[0253]

[0259] Aspect 83. The apparatus of aspect 82, wherein to adjust a first setting of the first image sensor, the one or more processors are configured to at least one of: change a power mode of the first image sensor from the first power mode to a second power mode having a higher power mode than the first power mode; and change an operating state of the first image sensor from the first operating state to a second operating state having a higher operating state than the first operating state, the second operating state comprising at least one of a higher frame rate and a higher resolution.

[0254]

[0260] Aspect 84. The apparatus of aspect 82, wherein to adjust a second setting of the second image sensor, the one or more processors are configured to at least one of: change a power mode of the second image sensor from a first power mode to a second power mode comprising a power mode lower than the first power mode; and change an operating state of the second image sensor from a first operating state to a second operating state comprising an operating state lower than the first operating state, the second operating state comprising at least one of a lower frame rate and a lower resolution.

[0255]

[0261] Aspect 85. The device of aspect 78, wherein the one or more processors are configured to track a location of the wearable device based on additional location information from the wearable device in response to a determination that the wearable device is not visible to one or more image sensors on the device.

[0256]

[0262] Aspect 86. The apparatus of aspect 78, wherein the one or more processors are configured to track a location of the wearable device based on additional position information from the wearable device in response to a determination that the wearable device is within the FOV of the one or more image sensors and that the view of the one or more image sensors to the wearable device is obstructed.

[0257]

[0263] Aspect 87. The apparatus of aspect 86, wherein the one or more processors are configured to initialize the one or more image sensors in response to a determination that the wearable device is within the FOV of the one or more image sensors and that the view of the one or more image sensors to the wearable device is obstructed.

[0258]

[0264] Aspect 88. An apparatus as described in any of aspects 76 to 87, wherein to determine a first position of the wearable device, one or more processors are configured to receive from the wearable device at least one of image data from one or more image sensors on the apparatus and data related to one or more measurements from one or more sensors on the wearable device, and determine the first position of the wearable device based on at least one of the image data from the one or more image sensors and data related to the one or more measurements from the one or more sensors.

[0259]

[0265] Aspect 89. The device of aspect 88, wherein the data comprises at least one of a distance of the wearable device relative to one or more objects, a velocity vector indicating a speed of movement of the wearable device, a touch signal measured by a touch sensor from one or more sensors, audio data from an audio sensor from one or more sensors, and an altitude of the wearable device in physical space, where the one or more objects comprise at least one of a device, a body part associated with a user of the wearable device, and an input device.

[0260]

[0266] Aspect 90. A device described in any of aspects 76 to 89, wherein one or more processors are configured to receive an input from a wearable device configured to trigger a privacy mode in the device, and adjust an operational state of one or more image sensors in the device to at least one of an off state and a disabled state based on the input configured to trigger the privacy mode.

[0261]

[0267] Aspect 91. The apparatus of aspect 90, wherein the input is based on sensor data from one or more sensors on the wearable device, where the sensor data indicates at least one of a touch signal corresponding to a touch input at the wearable device, a location of the wearable device, and a distance between the wearable device and a body part of a user of the wearable device.

[0262]

[0268] Aspect 92. An apparatus as described in any of aspects 90 to 91, wherein the one or more processors are configured to receive an additional input from the wearable device configured to trigger the apparatus to terminate the privacy mode, where the additional input is based on sensor data indicating at least one of: a touch signal corresponding to a touch input at the wearable device, a location of the wearable device corresponding to a location of a body part of a user of the wearable device, and a proximity between the wearable device and the body part.

[0263]

[0269] Aspect 93. A device described in any of aspects 76 to 92, wherein the one or more processors are configured to determine one or more extended reality (XR) inputs to an XR application on the device based on at least one of data from the wearable device and commands from the wearable device.

[0264]

[0270] Aspect 94. The apparatus of aspect 93, wherein the one or more XR inputs comprise at least one of a modification of a virtual element along multiple dimensions in space, a selection of a virtual element, a navigation event, and a request to measure a distance defined by at least one of a first position of the wearable device, a second position of the wearable device, and a movement of the wearable device.

[0265]

[0271] Aspect 95. The device of aspect 94, wherein the virtual element comprises at least one of a virtual object rendered by the device, a virtual plane in an environment rendered by the device, and an environment rendered by the device.

[0266]

[0272] Aspect 96. The apparatus of any of aspects 94 to 95, wherein the navigation event comprises at least one of scrolling rendered content and moving from a first interface element to a second interface element.

[0267]

[0273] Aspect 97. An apparatus as described in any of aspects 76 to 96, wherein the one or more processors are configured to receive an input from the wearable device configured to trigger adjustment of one or more XR actions in the apparatus, where the one or more XR actions comprise at least one of object detection, object classification, object tracking, pose estimation, and shape estimation.

[0268]

[0274] Aspect 98. An apparatus as described in any of aspects 76 to 97, wherein the wearable device comprises a bracelet, ring, or glove, and wherein the location information comprises at least one of measurements from an inertial measurement unit from one or more sensors on the wearable device and altitude measured by a pressure sensor from one or more sensors.

[0269]

[0275] Aspect 99. The apparatus of any of aspects 76 to 98, wherein the apparatus comprises a mobile device.

[0270]

[0276] Embodiment 100. An apparatus described in any of embodiments 76 to 99, wherein the apparatus comprises a camera.

[0271]

[0277] Embodiment 101. An apparatus according to any of embodiments 76 to 100, wherein the apparatus comprises an XR device and a display.

[0272]

[0278] Aspect 102. A method comprising determining a first position of a wearable device in a physical space, receiving from the wearable device position information related to the wearable device, determining a second position of the wearable device based on the received position information, and tracking movement of the wearable device relative to an electronic device based on the first position and the second position.

[0273]

[0279] Aspect 103. The method of aspect 102, wherein tracking the movement of the wearable device further comprises determining a first position of the wearable device in a first coordinate system of the wearable device, transforming the first coordinate system of the wearable device to a second coordinate system of the electronic device, and determining a second position of the wearable device in the second coordinate system of the electronic device.

[0274]

[0280] Aspect 104. The method of any of aspects 102 to 103, further comprising determining, based on at least one of a second position of the wearable device and tracked movement of the wearable device, whether the wearable device is at least one of within a field of view (FOV) of one or more image sensors on the electronic device and is visible to one or more image sensors on the electronic device.

[0275]

[0281]

[0046] Aspect 105. The method of aspect 104, further comprising tracking a location of a hand associated with the wearable device based on at least one of a second position of the wearable device and the tracked movement of the wearable device.

[0276]

[0282] Aspect 106. The method of aspect 105, further comprising: capturing one or more images of the hand via at least one image sensor from the one or more image sensors based on a determination that the wearable device is within the FOV of and visible to the one or more image sensors; tracking a location of the hand further based on the one or more images of the hand; and the location of the hand is tracked relative to a first coordinate system of the wearable device.

[0277]

[0283] Aspect 107. The method of aspect 104, further comprising: determining, based on the position information, that the wearable device is outside the FOV of one or more image sensors and moving toward an area within the FOV of the one or more image sensors; and initiating one or more imaging operations and one or more tracking operations at the electronic device based on determining that the wearable device is outside the FOV of the one or more image sensors and moving toward an area within the FOV of the one or more image sensors; and the one or more tracking operations are based at least in part on image data from the one or more imaging operations.

[0278]

[0284] Aspect 108. The method of aspect 104, further comprising: adjusting a first setting of the first image sensor based on at least one of a first determination that the wearable device is within a first FOV of a first image sensor on the electronic device and a second determination that the wearable device is visible to the first image sensor on the electronic device, the first setting comprising at least one of a power mode of the first image sensor and an operational state of the first image sensor; adjusting a second setting of the second image sensor based on at least one of a third determination that the wearable device is outside a second FOV of a second image sensor on the electronic device and a fourth determination that the wearable device is not visible to the second image sensor on the electronic device, the second setting comprising at least one of a power mode of the second image sensor and an operational state of the second image sensor.

[0279]

[0285] Aspect 109. The method of aspect 108, wherein adjusting the first setting of the first image sensor further comprises at least one of changing a power mode of the first image sensor from the first power mode to a second power mode comprising a power mode higher than the first power mode, and changing an operating state of the first image sensor from the first operating state to a second operating state comprising a higher operating state than the first operating state, the second operating state comprising at least one of a higher frame rate and a higher resolution.

[0280]

[0286] Aspect 110. The method of aspect 108, wherein adjusting the second setting of the second image sensor further comprises at least one of changing a power mode of the second image sensor from a first power mode to a second power mode comprising a power mode lower than the first power mode, and changing an operating state of the second image sensor from a first operating state to a second operating state comprising an operating state lower than the first operating state, the second operating state comprising at least one of a lower frame rate and a lower resolution.

[0281]

[0287] Aspect 111. The method of aspect 104, further comprising, in response to determining that the wearable device is not visible to one or more image sensors on the electronic device, tracking a location of the wearable device based on additional location information from the wearable device.

[0282]

[0288] Aspect 112. The method of aspect 104, further comprising tracking a location of the wearable device based on additional position information from the wearable device in response to determining that the wearable device is within the FOV of one or more image sensors and that the view of the one or more image sensors to the wearable device is obstructed.

[0283]

[0289] Aspect 113. The method of aspect 112, further comprising initializing the one or more image sensors in response to determining that the wearable device is within the FOV of the one or more image sensors and that the view of the one or more image sensors to the wearable device is obstructed.

[0284]

[0290] Embodiment 114. The method of any of embodiments 102 to 113, wherein determining a first position of the wearable device further comprises receiving from the wearable device at least one of image data from one or more image sensors on the electronic device and data related to one or more measurements from the one or more sensors on the wearable device, and determining the first position of the wearable device based on at least one of the image data from the one or more image sensors and data related to the one or more measurements from the one or more sensors.

[0285]

[0291] Aspect 115. The method of aspect 114, wherein the data comprises at least one of a distance of the wearable device relative to one or more objects, a velocity vector indicating a speed of movement of the wearable device, a touch signal measured by a touch sensor from one or more sensors, audio data from an audio sensor from one or more sensors, and an altitude of the wearable device in physical space, wherein the one or more objects comprise at least one of an electronic device, a body part associated with a user of the wearable device, and an input device.

[0286]

[0292] Aspect 116. A method as described in any of aspects 102 to 115, further comprising receiving an input from the wearable device configured to trigger a privacy mode in the electronic device, and adjusting an operational state of one or more image sensors in the electronic device to at least one of an off state and a disabled state based on the input configured to trigger the privacy mode.

[0287]

[0293] Aspect 117. The method of aspect 116, wherein the input is based on sensor data from one or more sensors on the wearable device, where the sensor data indicates at least one of a touch signal corresponding to a touch input at the wearable device, a location of the wearable device, and a distance between the wearable device and a body part of a user of the wearable device.

[0288]

[0294] Aspect 118. A method as described in any of aspects 116 to 117, further comprising receiving an additional input from the wearable device configured to trigger the electronic device to terminate the privacy mode, where the additional input is based on sensor data indicating at least one of a touch signal corresponding to a touch input at the wearable device, a location of the wearable device corresponding to a location of a body part of the user of the wearable device, and a proximity between the wearable device and the body part.

[0289]

[0295] Aspect 119. The method of any of aspects 102 to 118, further comprising determining one or more extended reality (XR) inputs to an XR application on the electronic device based on at least one of data from the wearable device and commands from the wearable device.

[0290]

[0296] Aspect 120. The method of aspect 119, wherein the one or more XR inputs comprise at least one of a modification of a virtual element along multiple dimensions in space, a selection of a virtual element, a navigation event, and a request to measure a distance defined by at least one of a first position of the wearable device, a second position of the wearable device, and a movement of the wearable device.

[0291]

[0297] Aspect 121. The method of aspect 120, wherein the virtual element comprises at least one of a virtual object rendered by the electronic device, a virtual plane in an environment rendered by the electronic device, and an environment rendered by the electronic device.

[0292]

[0298] Aspect 122. The method of any of aspects 120 to 121, wherein the navigation event comprises at least one of scrolling rendered content and moving from a first interface element to a second interface element.

[0293]

[0299] Aspect 123. The method of any of aspects 102 to 122, further comprising receiving an input from the wearable device configured to trigger adjustment of one or more XR actions at the electronic device, where the one or more XR actions comprise at least one of object detection, object classification, object tracking, pose estimation, and shape estimation.

[0294]

[0300] Aspect 124. The method of any of aspects 102 to 123, wherein the wearable device comprises a bracelet, ring, or glove, and wherein the location information comprises at least one of measurements from an inertial measurement unit from one or more sensors on the wearable device and altitude measured by a pressure sensor from one or more sensors.

[0295]

[0301] Aspect 125. A non-transitory computer-readable medium storing instructions that, when executed by one or more processors, cause the one or more processors to perform a method according to any of aspects 102 to 124.

[0296]

[0302] Embodiment 126. An apparatus comprising means for carrying out the method described in any of embodiments 102 to 124.

[0297]

[0303] Aspect 127. The apparatus of aspect 126, wherein the apparatus comprises a mobile device.

[0298]

[0304] Embodiment 128. An apparatus according to any one of embodiments 126 to 127, wherein the apparatus comprises a camera.

[0299]

[0305] Embodiment 129. An apparatus described in any of embodiments 126 to 128, wherein the apparatus comprises an XR device and a display.

Claims

1. A memory, one or more processors coupled to the memory, wherein the one or more processors are configured to: determine a first position of a wearable device in a physical space, where the first position is determined based on position information measured by one or more sensors on the wearable device; receive additional position information related to the wearable device from the wearable device; determine a second position of the wearable device based on the received additional position information; track movement of the wearable device relative to the apparatus based on the first position and the second position; determine that the wearable device is outside a field of view (FOV) of one or more image sensors on the apparatus and that the movement of the wearable device is towards an area within the FOV of the one or more image sensors, based on at least one of the second position of the wearable device and the movement of the wearable device; initiate, based on the determination that the wearable device is outside the FOV of the one or more image sensors and that the movement of the wearable device is towards the area within the FOV of the one or more image sensors, one or more imaging operations and one or more tracking operations in the apparatus, where the one or more tracking operations are at least partially based on image data from the one or more imaging operations; An apparatus configured to perform the above.

2. To track the movement of the wearable device, the one or more processors are configured to: determine the first position of the wearable device in a first coordinate system of the wearable device; transform the first coordinate system of the wearable device to a second coordinate system of the apparatus; determine the second position of the wearable device in the second coordinate system of the apparatus; The apparatus according to claim 1, configured to perform the above.

3. The one or more processors are configured to: Tracking the location of a hand associated with the wearable device based on at least one of the second position of the wearable device and the tracked movement of the wearable device. The apparatus according to claim 1, configured to perform. **Claim 4** The one or more processors Based on the determination that the wearable device is within the FOV of the one or more image sensors, capturing one or more images of the hand via at least one of the one or more image sensors from the one or more image sensors; Tracking the location of the hand further based on the one or more images of the hand, wherein the location of the hand is tracked relative to a first coordinate system of the wearable device. The apparatus according to claim 3, configured to perform. **Claim 5** The one or more processors Based on a first determination that the wearable device is within a first FOV of a first image sensor on the device, adjusting a first setting of the first image sensor, the first setting comprising at least one of a power mode of the first image sensor and an operating state of the first image sensor; Based on a second determination that the wearable device is outside a second FOV of a second image sensor on the device, adjusting a second setting of the second image sensor, the second setting comprising at least one of a power mode of the second image sensor and an operating state of the second image sensor; The apparatus according to claim 1, configured to perform. **Claim 6** To adjust the first setting of the first image sensor, the one or more processors are configured to change the power mode of the first image sensor from a first power mode to a second power mode having a higher power mode than the first power mode, and the operating state of the first image sensor from a first operating state to a second operating state having a higher operating state than the first operating state, and at least one of the second operating states comprises at least one of a higher frame rate and a higher resolution. The apparatus according to claim 5. **Claim 7** To adjust the second setting of the second image sensor, the one or more processors change the power mode of the second image sensor from a first power mode to a second power mode having a lower power mode than the first power mode, and change the operating state of the second image sensor from a first operating state to a second operating state having a lower operating state than the first operating state, and are configured to perform at least one of the above, wherein the second operating state comprises at least one of a lower frame rate and a lower resolution. The apparatus according to claim 5.

8. The one or more processors In response to a determination that the wearable device is within the FOV of the one or more image sensors and the view of the one or more image sensors to the wearable device is obstructed, tracking the location of the wearable device based on additional location information received from the wearable device. is configured to perform Optionally, the one or more processors In response to the determination that the wearable device is within the FOV of the one or more image sensors and the view of the one or more image sensors to the wearable device is obstructed, initializing the one or more image sensors. The apparatus according to claim 1, which is configured to perform

9. To determine the first position of the wearable device, the one or more processors Receive at least one of image data from one or more image sensors on the device and data related to one or more measurements from one or more sensors on the wearable device. Determine the first position of the wearable device based on at least one of the image data from the one or more image sensors and the data related to the one or more measurements from the one or more sensors. is configured to perform Optionally The data includes at least one of the distance of the wearable device with respect to one or more objects, a velocity vector indicating the velocity of the movement of the wearable device, a touch signal measured by a touch sensor from the one or more sensors, audio data from an audio sensor from the one or more sensors, and the altitude of the wearable device in the physical space, wherein the one or more objects include at least one of the device, a body part related to a user of the wearable device, and an input device. The device according to claim 1.

10. The one or more processors determine one or more XR inputs to an extended reality (XR) application on the device based on at least one of the data from the wearable device and the command from the wearable device. The device according to claim 1, configured to perform the above.

11. The one or more XR inputs include at least one of modification of virtual elements along multiple dimensions in space, selection of the virtual elements, navigation events, and a request to measure a distance defined by at least one of the first position of the wearable device, the second position of the wearable device, and the movement of the wearable device. Optionally, the virtual elements include at least one of a virtual object rendered by the device, a virtual plane in the environment rendered by the device, and the environment rendered by the device, and / or the navigation events include at least one of scrolling the rendered content and moving from a first interface element to a second interface element. The device according to claim 10.

12. The one or more processors Receiving, from the wearable device, an input configured to trigger an adjustment of one or more XR operations in the apparatus, wherein the one or more XR operations comprise at least one of object detection, object classification, object tracking, pose estimation, and shape estimation. The apparatus according to claim 1, configured to perform.

13. The wearable device comprises a bracelet, a ring, or a glove, wherein the position information comprises at least one of measurement results from an inertial measurement unit from one or more sensors on the wearable device and altitude measured by a pressure sensor from the one or more sensors. The apparatus according to claim 1.

14. The apparatus comprises a mobile device. Optionally, the apparatus according to claim 1, comprises an XR device including a camera and / or a display.

15. Determining a first position of the wearable device in the physical space, wherein the first position is determined based on position information measured by one or more sensors on the wearable device. Receiving, from the wearable device, additional position information related to the wearable device. Determining a second position of the wearable device based on the received additional position information. Tracking the movement of the wearable device relative to the electronic device based on the first position and the second position. Based on at least one of the second position of the wearable device and the movement of the wearable device, determining that the wearable device is outside the field of view (FOV) of one or more image sensors on the electronic device and that the wearable device is moving towards an area within the FOV of the one or more image sensors. Based on a determination that the wearable device is outside the FOV of the one or more image sensors and the wearable device is moving towards the area within the FOV of the one or more image sensors, starting, in the electronic device, one or more imaging operations and one or more tracking operations, wherein the one or more tracking operations are at least partially based on image data from the one or more imaging operations. A method comprising.

Images