Power-efficient, performance-efficient, and context-adaptive attitude tracking

Abstract

In some aspects, a gesture tracking device may receive availability information from a sensor system including a plurality of sensors based on a current operating condition associated with the plurality of sensors. The attitude tracking device may select a set of sensor modalities associated with the sensor system based on the availability information. The gesture tracking device may select a gesture tracking model based on the selected set of sensor modalities and one or more key performance indicator (KPI) requirements related to a current context associated with a gesture tracking configuration of the client application. The attitude tracking device may estimate an attitude associated with the object using an attitude tracking model based on sensor inputs associated with one or more sensors selected from the plurality of sensors. Numerous other aspects are described.

Description

[0001] Cross-reference to related applications

[0002] This patent application claims priority to U.S. nonprovisional patent application No. 18 / 325,752, filed May 30, 2023, entitled “POWER-EFFICIENT, PERFORMANCE-EFFICIENT, AND CONTEXT-ADAPTIVE POSE TRACKING,” which is expressly incorporated herein by reference. Technical Field

[0003] Various aspects of this disclosure generally relate to attitude estimation, and, for example, to attitude tracking devices capable of performing power-efficient, performance-efficient, and context-adaptive attitude tracking. Background Technology

[0004] "Pose tracking," also known as attitude estimation, refers to techniques used to infer or estimate the position and / or orientation of a device, person, or object in three-dimensional space relative to a given reference frame. Pose tracking can generally refer to techniques used to estimate the position and / or orientation associated with a tracked object (e.g., a user, user device, or physical real-world object) on one or more axes (e.g., three degrees of freedom (3DoF) on three position axes or three orientation axes, or six degrees of freedom (6DoF) on three position axes and three orientation axes). Additionally or alternatively, attitude tracking may include techniques for estimating one or more velocities of the tracked object (such as the absolute or relative linear velocity or absolute or relative angular velocity of the tracked object). Typically, attitude tracking is performed by analyzing signals from various sensor inputs (e.g., images or videos captured by one or more cameras, positioning coordinates obtained from one or more satellite navigation systems, etc.) to determine the position and / or orientation of the object of interest. Summary of the Invention

[0005] Some aspects described herein relate to a method for power-efficient and performance-efficient context-adaptive attitude tracking. The method may include receiving information from an attitude tracking device, including one or more key performance indicators (KPIs) requirements associated with a current context and an attitude tracking configuration of a client application. The method may include receiving availability information from a sensor system comprising multiple sensors based on one or more parameters associated with the current operating conditions of multiple sensors. The method may include selecting a set of sensor modes comprising one or more sensors from multiple sensors included in the sensor system, based on the current context associated with the attitude tracking configuration of the client application and the availability information associated with the current operating conditions of multiple sensors. The method may include selecting an attitude tracking model based on the set of sensor modes and one or more KPI requirements associated with the current context and the attitude tracking configuration of the client application. The method may include estimating the attitude associated with the tracked object using the attitude tracking model based on sensor inputs associated with the set of sensor modes.

[0006] Some aspects described herein relate to an attitude tracking device for power-efficient and performance-efficient context-adaptive attitude tracking. The attitude tracking device may include one or more memories and one or more processors coupled to the memories. The one or more processors may be configured to receive information including one or more KPI requirements associated with a current context, which is associated with an attitude tracking configuration of a client application. The one or more processors may be configured to receive availability information from a sensor system comprising multiple sensors based on one or more parameters associated with the current operating conditions of multiple sensors. The one or more processors may be configured to select a set of sensor modes comprising one or more sensors from multiple sensors included in the sensor system based on the current context associated with the attitude tracking configuration of the client application and the availability information associated with the current operating conditions of multiple sensors. The one or more processors may be configured to select an attitude tracking model based on the set of sensor modes and one or more KPI requirements associated with the current context associated with the attitude tracking configuration of the client application. The one or more processors may be configured to estimate the attitude associated with the tracked object using the attitude tracking model based on sensor inputs associated with the set of sensor modes.

[0007] Some aspects described herein relate to a non-transitory computer-readable medium storing a set of instructions for power-efficient and performance-efficient context-adaptive attitude tracking by an attitude tracking device. When executed by one or more processors of the attitude tracking device, the set of instructions enables the attitude tracking device to receive information including one or more KPI requirements associated with a current context, which is associated with an attitude tracking configuration of a client application. When executed by one or more processors of the attitude tracking device, the set of instructions enables the attitude tracking device to receive availability information from a sensor system comprising multiple sensors based on one or more parameters related to the current operating conditions associated with multiple sensors. When executed by one or more processors of the attitude tracking device, the set of instructions enables the attitude tracking device to select a set of sensor modes comprising one or more sensors from multiple sensors included in the sensor system based on the current context associated with the attitude tracking configuration of the client application and the availability information related to the current operating conditions associated with the multiple sensors. When executed by one or more processors of the attitude tracking device, the set of instructions enables the attitude tracking device to select an attitude tracking model based on the set of sensor modes and one or more KPI requirements associated with the current context associated with the attitude tracking configuration of the client application. When executed by one or more processors of the attitude tracking device, this set of instructions enables the attitude tracking device to estimate the attitude associated with the tracked object using an attitude tracking model based on sensor inputs associated with a set of sensor modes.

[0008] Some aspects described herein relate to an apparatus for power-efficient and performance-efficient context-adaptive attitude tracking. The apparatus may include components for receiving information including one or more KPI requirements associated with a current context, which is associated with an attitude tracking configuration of a client application. The apparatus may include components for receiving availability information from a sensor system comprising multiple sensors based on one or more parameters associated with a current operating condition and multiple sensors. The apparatus may include components for selecting a set of sensor modes comprising one or more sensors from a set of sensors included in the sensor system based on the current context associated with the attitude tracking configuration of the client application and the availability information associated with the current operating condition and multiple sensors. The apparatus may include components for selecting an attitude tracking model based on the set of sensor modes and one or more KPI requirements associated with the current context and the attitude tracking configuration of the client application. The apparatus may include components for estimating the attitude associated with a tracked object using an attitude tracking model based on sensor inputs associated with the set of sensor modes.

[0009] The terms generally include methods, apparatus, systems, computer program products, non-transitory computer-readable media, user equipment, user gear, electronic equipment, and / or processing systems as described substantially with reference to the accompanying drawings and description and as shown in the drawings and description.

[0010] The features and technical advantages of examples according to this disclosure have been outlined quite extensively above to facilitate a better understanding of the detailed description that follows. Additional features and advantages will be described below. The disclosed concepts and specific examples can be readily used as the basis for modifying or designing other structures for performing the same purpose of this disclosure. Such equivalent constructions do not depart from the scope of the appended claims. The characteristics of the concepts disclosed herein (both their organization and manner of operation) and their associated advantages will be better understood when considered in conjunction with the accompanying drawings, based on the following description. Each drawing is provided for illustrative and descriptive purposes and not as a definition of limitation of the claims. Attached Figure Description

[0011] To gain a more detailed understanding of the features of this disclosure, a more specific description of the brief overview above can be obtained by referring to various aspects, some of which are illustrated in the accompanying drawings. However, it should be noted that the drawings illustrate only certain typical aspects of this disclosure and should therefore not be considered as limiting its scope, as the description may allow for other equivalent aspects. The same reference numerals in different drawings may identify the same or similar elements.

[0012] Figure 1 This is a diagram illustrating an example environment according to this disclosure, in which the power-efficient, performance-efficient, and context-adaptive attitude tracking described herein can be implemented.

[0013] Figure 2 This is a diagram showing example components of the device.

[0014] Figure 3 This is a diagram illustrating examples of power-efficient, performance-efficient, and context-adaptive attitude tracking according to this disclosure.

[0015] Figure 4 This is a flowchart illustrating an example process associated with power-efficient, performance-efficient, and context-adaptive attitude tracking according to this disclosure. Detailed Implementation

[0016] Various aspects of this disclosure are described more fully below with reference to the accompanying drawings. However, this disclosure may be embodied in many different forms and should not be construed as limited to any particular structure or function given throughout this disclosure. Rather, these aspects are provided so that this disclosure will be thorough and complete and will fully convey the scope of this disclosure to those skilled in the art. Those skilled in the art will understand that the scope of this disclosure is intended to cover any aspect of this disclosure disclosed herein, whether implemented independently of or in combination with any other aspect of this disclosure. For example, any number of aspects set forth herein may be used to implement an apparatus or method of practice. Furthermore, the scope of this disclosure is intended to cover such apparatus or methods practiced using structures, functions, or structures and functions other than or different from the aspects of this disclosure set forth herein. It should be understood that any aspect of this disclosure disclosed herein may be embodied by one or more elements of the claims.

[0017] Accurate attitude tracking is a challenging problem with a wide range of applications, including smartphone tracking, wearable device tracking, unmanned aerial vehicle tracking, autonomous vehicle tracking, extended reality (XR) applications and / or package tracking, among others. For example, as described herein, “attitude tracking” can generally refer to a technique for estimating the localization and / or orientation associated with a tracked object (e.g., a user, user device, or physical real-world object) on one or more axes (e.g., three degrees of freedom (DoF) on three localization axes or three orientation axes, or six degrees of freedom (6DoF) on three localization axes and three orientation axes), and an attitude tracking device is any suitable device capable of tracking the attitude (e.g., localization and / or orientation) associated with the tracked object. However, existing attitude tracking solutions suffer from various drawbacks, such as being tailored to the set of available sensors, use cases, and / or power specifications.

[0018] For example, existing attitude tracking solutions typically estimate attitude based on a set of available sensors, which can be unreliable or untrustworthy under certain conditions. In particular, different sensors can be calibrated to obtain accurate information under certain operating conditions, and thus parasitic signals can be injected, leading to inaccurate outputs outside of the calibrated operating conditions. For instance, virtual reality (VR) headsets often use visual inertial odometry (VIO) powered by high-power cameras, which may not work well in situations with insufficient light and / or insufficient features in the scene. In other examples, an ambient light sensor (ALS) may not generate accurate sensor input when the device containing the ALS is in the user's pocket, and a Global Navigation Satellite System (GNSS) receiver may not generate accurate sensor input when the GNSS receiver is indoors. Furthermore, another drawback associated with existing attitude tracking solutions is that the same model is often used for every iteration of a given attitude estimation task, even though the model best suited for the attitude estimation task may depend on the operating conditions. For example, the model best suited for a given attitude estimation task can change when KPIs (such as client demand, battery level, sensor availability, model confidence, and / or model power consumption) change. Therefore, in some cases, using the same model every time a given pose estimation task is performed may result in inaccurate output, excessive resource consumption, excessive power consumption, and / or inaccurate output.

[0019] Furthermore, existing attitude tracking solutions may utilize available hardware resources in a suboptimal manner. For example, the available processor resources for a device that is always plugged in (e.g., with a wall power supply) may include a fast and / or powerful graphics processing unit (GPU) or neural processing unit (NPU) compared to a battery-powered device. However, the attitude tracking model can be configured to use a standard central processing unit (CPU) instead, which can result in the attitude tracking model providing high-latency output and / or preventing other high-priority applications from running on the CPU. In another example, wearable devices typically have low-power islands and low-power processors to save power, but the attitude tracking model used on a wearable device could alternatively use a standard CPU, which could drain the battery in a short period of time. In addition to using available hardware resources in a suboptimal manner, existing attitude tracking solutions may provide suboptimal trade-offs between performance and power over time. For example, the attitude tracking device may continue to run the attitude tracking model even after it has started generating outlier outputs, low-confidence outputs, saturated performance metrics, and / or high power consumption, which may lead to inaccurate outputs, battery depletion, and / or other performance and / or power consumption issues.

[0020] The aspects described in this paper achieve power-efficient, performance-efficient, and context-adaptive attitude tracking, providing a general attitude tracking solution that can utilize multiple combinations of sensor modalities across different device shape factors based on various criteria such as desired accuracy, sensor availability, power constraints, and / or current context (e.g., current device type, current device location, current motion detection state, current activity recognition state, and / or current device placement). For example, in some aspects, the attitude tracking device can be configured to read client application configurations and one or more key performance indicator (KPI) requirements (e.g., requirements related to battery level, processor capability, available memory, latency, and / or accuracy) and can adapt to different sensor contexts (e.g., device type, device shape factor, location, positioning, and / or user activity, and other examples). Therefore, the attitude tracking device can select a set of sensor modalities and / or attitude tracking models to optimally balance performance and power consumption requirements. Furthermore, in some aspects, the attitude tracking device can optimize attitude estimation based on feedback associated with the model output through intelligent sensor selection, model selection, and hardware reconfiguration, which can be continuously monitored to improve performance with respect to changing client requirements. In this way, some aspects described in this paper enable intelligent, power-efficient, performance-efficient, and context-adaptive attitude tracking that utilizes different sensing modalities for various environments, sensor systems, device shape factors, user activities, power levels, available hardware resources, and / or client requirements.

[0021] Figure 1 This is a diagram illustrating an example environment 100 according to this disclosure, in which the power-efficient, performance-efficient, and context-adaptive attitude tracking described herein can be implemented. Figure 1 As shown, environment 100 may include attitude tracking device 110, tracked object 120, network node 130, and network 140. The devices in environment 100 may be interconnected via wired connection, wireless connection, or a combination of wired and wireless connection.

[0022] Attitude tracking device 110 includes one or more devices capable of receiving, generating, storing, processing, and / or providing information related to the estimated attitude, estimated velocity, and / or one or more estimated calibration parameters associated with the tracked object 120. For example, such as... Figure 1As shown, attitude tracking device 110 may include a sensor subsystem and an attitude tracking component, which may be configured to estimate the attitude associated with the tracked object using an attitude tracking model based on sensor inputs associated with a selected set of sensor modes. For example, in some aspects, the estimated attitude of the tracked object 120 may include estimated localization and / or estimated orientation associated with the tracked object 120, such as absolute localization on one or more axes at a specific time, relative localization (e.g., displacement) on one or more axes for a duration on one or more axes, absolute orientation on one or more axes at a specific time and / or relative orientation (e.g., change of orientation) on one or more axes for a duration on one or more axes. Additionally or alternatively, the attitude tracking component may be configured to estimate one or more velocities of the tracked object 120 (such as absolute or relative linear velocity or absolute or relative angular velocity). Additionally or alternatively, the attitude tracking component may estimate one or more parameters to calibrate the sensor subsystem (e.g., based on sensor bias, sensor sensitivity, and / or drift over time or temperature, and other examples).

[0023] In some aspects, the attitude tracking device 110 may include wired and / or wireless communication and / or computing devices, such as user equipment (UE), mobile phones (e.g., smartphones, cordless phones, etc.), laptop computers, tablet computers, handheld computers, desktop computers, gaming devices, wearable communication devices (e.g., smartwatches, smart glasses, etc.). Furthermore, the tracked object 120 may include a person or part of a person, a user equipment, or a physical object whose attitude and / or movement can be tracked by the attitude tracking device 110. For example, in some aspects, the tracked object 120 may include one or more body parts of a user, VR or XR headsets, unmanned aerial vehicles, user equipment, vehicles, and / or physical objects (such as packages), and other examples. In some aspects, the attitude tracking device 110 may be included within the tracked object 120 (e.g., where the tracked object 120 is an XR headset or unmanned aerial vehicle with built-in attitude tracking capabilities). Additionally or alternatively, the posture tracking device 110 may be detached from the tracked object 120 (e.g., where the tracked object 120 is a user or one or more body parts of a user, a physical object to be tracked, or a device otherwise detached from the posture tracking device 110, such as a handheld controller tracked by an XR headset).

[0024] Similar to attitude tracking device 110, network node 130 includes one or more devices capable of receiving, generating, storing, processing, and / or providing information related to the estimated attitude associated with the tracked object 120, the estimated velocity associated with the tracked object 120, and / or one or more estimated calibration parameters. For example, network node 130 may include one or more components of a base station (Node B, gNB, and / or 5G Node B (NB) and other examples), a UE, a relay device, a network controller, an access point, a transmit / receive point (TRP), a device, an apparatus, a computing system, and / or another processing entity configured to perform one or more aspects of the techniques described herein (e.g., attitude tracking device 110 may send one or more sensor inputs and / or other suitable information to network node 130, which can process the sensor inputs and / or other suitable information using an attitude tracking model and return one or more outputs to attitude tracking device 110). In some aspects, network node 130 may be one or more components of a converged base station and / or a decomposed base station (e.g., a central unit, a distributed unit, and / or a radio unit).

[0025] Network 140 includes one or more wired and / or wireless networks. For example, network 140 may include cellular networks (e.g., Long Term Evolution (LTE) networks, Code Division Multiple Access (CDMA) networks, 3G networks, 4G networks, 5G networks, another type of next-generation network, etc.), Public Land Mobile Network (PLMN), Local Area Network (LAN), Wide Area Network (WAN), Metropolitan Area Network (MAN), Telephone Network (e.g., Public Switched Telephone Network (PSTN)), Private Network, Self-organizing Network, Intranet, Internet, Fiber-based Network, Cloud Computing Network, etc., and / or combinations of these or other types of networks.

[0026] Figure 1 The number and arrangement of devices and networks shown are provided as an example. In practice, similar arrangements may exist. Figure 1 The devices and / or networks shown are compared to additional devices and / or networks, fewer devices and / or networks, different devices and / or networks, or devices and / or networks with different arrangements. Furthermore, Figure 1 The two or more devices shown can be implemented within a single device, or Figure 1 The single device shown can be implemented as multiple distributed devices. Additionally or alternatively, a collection of devices in environment 100 (e.g., one or more devices) can perform one or more functions described as being performed by another collection of devices in environment 100.

[0027] Figure 2This is a diagram illustrating example components of device 200 according to the present disclosure. Device 200 may correspond to attitude tracking device 110, tracked object 120, and / or network node 130. In some aspects, attitude tracking device 110, tracked object 120, and / or network node 130 may include one or more devices 200 and / or one or more components of device 200. Figure 2 As shown, device 200 may include bus 205, processor 210, memory 215, storage component 220, input component 225, output component 230, communication interface 235, sensor subsystem 240 and / or attitude tracking component 245.

[0028] Bus 205 includes components that allow communication between components of device 200. Processor 210 is implemented in hardware, firmware, or a combination of hardware and software. Processor 210 is a CPU, GPU, NPU, Accelerated Processing Unit (APU), microprocessor, microcontroller, digital signal processor (DSP), field-programmable gate array (FPGA), application-specific integrated circuit (ASIC), or another type of processing component. In some aspects, processor 210 includes one or more processors that can be programmed to perform functions. Memory 215 includes random access memory (RAM), read-only memory (ROM), and / or another type of dynamic or static storage device (e.g., flash memory, magnetic storage, and / or optical storage) that stores information and / or instructions for use by processor 210.

[0029] Storage component 220 stores information and / or software related to the operation and use of device 200. For example, storage component 220 may include hard disks (e.g., magnetic disks, optical disks, magneto-optical disks, and / or solid-state disks), compact discs (CDs), digital versatile discs (DVDs), floppy disks, cassette tapes, magnetic tapes, and / or another type of non-transitory computer-readable media and corresponding drives.

[0030] Input component 225 includes components that allow device 200 to receive information such as via user input (e.g., touchscreen display, keyboard, keypad, mouse, buttons, switches, and / or microphone). Additionally or alternatively, input component 225 may include components for determining the positioning or location of device 200 (e.g., a Global Positioning System (GPS) component or GNSS component) and / or sensors for sensing information (e.g., an accelerometer, gyroscope, actuator, or another type of positioning or environmental sensor). Output component 230 includes components that provide output information from device 200 (e.g., a display, speaker, haptic feedback component, and / or audio or visual indicators).

[0031] Communication interface 235 includes transceiver-like components (e.g., a transceiver and / or separate receiver and transmitter) that enable device 200 to communicate with other devices, such as via wired connections, wireless connections, or a combination of wired and wireless connections. Communication interface 235 may allow device 200 to receive information from and / or provide information to another device. For example, communication interface 235 may include an Ethernet interface, an optical interface, a coaxial interface, an infrared interface, a radio frequency interface, a universal serial bus (USB) interface, a wireless local area network (WLAN) interface (e.g., a Wi-Fi or wireless local area network (WLAN) interface), and / or a cellular network interface.

[0032] The sensor subsystem 240 includes one or more wired or wireless devices capable of receiving, generating, storing, processing, and / or providing information related to the estimated attitude, estimated velocity, and / or one or more estimated calibration parameters for estimating the attitude and / or velocity associated with the tracked object, as described elsewhere herein. For example, sensor subsystem 240 may include normally open cameras, high-resolution cameras, motion sensors, accelerometers, gyroscopes, proximity sensors, optical sensors (e.g., ALS), noise sensors, pressure sensors, ultrasonic (or ultrasonic waves) sensors, positioning (e.g., GNSS) sensors, time-of-flight (ToF) sensors, radio frequency (RF) sensors (e.g., to detect millimeter wave, WLAN, Bluetooth, and / or other wireless signals), capacitive sensors, timing devices, infrared sensors, active sensors (e.g., sensors that require an external power signal), passive sensors (e.g., sensors that do not require an external power signal), biological or biometric sensors, smoke sensors, gas sensors, chemical sensors, alcohol sensors, temperature sensors, humidity sensors, moisture sensors, magnetometers, electromagnetic sensors, analog sensors, and / or digital sensors, etc. In some aspects, sensor subsystem 240 may sense or detect conditions or information related to the state of device 200, the environment surrounding device 200, and / or objects present in the environment surrounding device 200, and may use wired or wireless communication interfaces to transmit indications of the detected conditions or information to other components of device 200 and / or other devices.

[0033] Attitude tracking component 245 includes one or more devices capable of receiving, generating, storing, transmitting, processing, detecting, and / or providing estimated attitude information, estimated motion information, and / or estimated calibration parameters based on one or more sensor inputs using an attitude tracking model, as described elsewhere herein. For example, in some aspects, attitude tracking component 245 may receive information including one or more KPI requirements related to the current context associated with an attitude tracking configuration of a client application; receive availability information from a sensor system including multiple sensors based on one or more parameters related to the current operating status associated with multiple sensors; select a set of sensor modes including one or more sensors from multiple sensors included in the sensor system based on the current context associated with the attitude tracking configuration of the client application and the availability information related to the current operating status associated with multiple sensors; select an attitude tracking model based on the set of sensor modes and one or more KPI requirements related to the current context associated with the attitude tracking configuration of the client application; and use the attitude tracking model to estimate the attitude associated with the tracked object based on sensor inputs associated with the set of sensor modes. Additionally or alternatively, attitude tracking component 245 may perform one or more other operations described herein.

[0034] Device 200 can perform one or more processes described herein. Device 200 can perform these processes based on software instructions stored in a non-transitory computer-readable medium (such as memory 215 and / or storage component 220) executed by processor 210. Computer-readable medium is defined herein as a non-transitory memory device. A memory device includes memory space within a single physical storage device or memory space distributed across multiple physical storage devices.

[0035] Software instructions may be read into memory 215 and / or storage component 220 via communication interface 235 from another computer-readable medium or from another device. When executed, the software instructions stored in memory 215 and / or storage component 220 may cause processor 210 to perform one or more processes described herein. Additionally or alternatively, hardwired circuitry may be used in place of or in combination with software instructions to perform one or more processes described herein. Therefore, the aspects described herein are not limited to any particular combination of hardware circuitry and software.

[0036] In some aspects, device 200 includes components for performing one or more processes described herein and / or components for performing one or more operations of one or more processes described herein. For example, device 200 may include: components for receiving information including one or more KPI requirements associated with a current context, which is associated with an attitude tracking configuration of a client application; components for receiving availability information from a sensor system including multiple sensors based on one or more parameters associated with a current operating condition and multiple sensors; components for selecting a set of sensor modes including one or more sensors from multiple sensors included in the sensor system based on the current context associated with the attitude tracking configuration of the client application and the availability information associated with the current operating condition and multiple sensors; components for selecting an attitude tracking model based on the set of sensor modes and one or more KPI requirements associated with the current context and the attitude tracking configuration for the client application; and / or components for estimating the attitude associated with a tracked object using the attitude tracking model based on sensor inputs associated with the set of sensor modes. In some aspects, such components may include combinations of Figure 2 One or more components of the described device 200 (such as bus 205, processor 210, memory 215, storage component 220, input component 225, output component 230, communication interface 235, sensor subsystem 240 and / or attitude tracking component 245).

[0037] Figure 2 The number and arrangement of components shown are provided as an example. In practice, device 200 may include components with... Figure 2 The components shown are compared to additional components, fewer components, different components, or components with different arrangements. Additionally or alternatively, a collection of components of device 200 (e.g., one or more components) may perform one or more functions described as being performed by a collection of other components of device 200.

[0038] Figure 3 This is a diagram of an example implementation 300 associated with power-efficient, performance-efficient, and context-adaptive attitude tracking according to this disclosure. Figure 3 As shown, example implementation 300 includes one or more components associated with an attitude tracking device (such as sensor subsystem 310), a sensor selection and configuration component 320 that can select one or more sensor modes and / or determine one or more parameters to configure model selection component 340, and an attitude estimation component 350 that can generate model output 360 and model feedback 370. In some aspects, as further described in detail herein, Figure 3The various components shown enable power-efficient, performance-efficient, and context-adaptive attitude tracking, providing a universal attitude tracking solution that can utilize multiple combinations of sensor modes across different device shape factors based on various criteria such as desired accuracy, sensor availability, power constraints, and / or current context.

[0039] like Figure 3 As shown, sensor subsystem 310 may include sensor scanning component 312, which can scan sensor subsystem 310 to identify multiple sensors available in or otherwise associated with sensor subsystem 310. In some aspects, the sensors identified using sensor scanning component 312 may include any suitable sensor capable of detecting conditions or information related to the attitude or motion state associated with the tracked object. For example, in some aspects, the identified sensors may include normally open cameras, high-resolution cameras, positioning sensors (e.g., GNSS receivers), accelerometers, gyroscopes, pressure sensors, magnetometers, ultrasonic sensors, ToF sensors, proximity sensors, millimeter-wave sensors, Wi-Fi (or WLAN) sensors, Bluetooth (or Wireless Personal Area Network (WPAN)) sensors, temperature sensors, ambient light sensors, and / or other suitable sensors. However, as mentioned above, some sensors may be designed or calibrated to generate reliable or trustworthy sensor information under certain operating conditions, and therefore may be prone to consuming very high power or injecting parasitic signals, which may cause the attitude tracking device to produce inaccurate attitude estimates outside the operating conditions in which the sensors are designed or calibrated.

[0040] Therefore, in some aspects, the sensor subsystem 310 may include a sensor availability component 314, which may use information related to the current context 316 to detect the trustworthiness and / or reliability of each available sensor and generate availability information (e.g., availability score or other suitable information) to indicate the trustworthiness and / or reliability of each available sensor. For example, in some aspects, the current context 316 may include one or more parameters related to the current operating status of the available sensors, such as the device type associated with the available sensors (e.g., indicating whether each sensor is included in a wearable device, XR headset, smartphone, tracker device, etc.). Additionally or alternatively, the current operating status of the available sensors may be related to the sensor's location (e.g., indicating whether the sensor is located indoors or outdoors), the motion state associated with the tracked object (e.g., indicating whether appropriate motion, such as stationary, moving, or other appropriate motion characteristics, is detected for each tracked object), the user's activity state (e.g., indicating whether each tracked object is associated with a motion state indicating that the user is sitting, walking, running, cycling, driving, etc.), and / or the device's placement (e.g., indicating whether each tracked object is on the user's body, such as in the user's pocket or in the user's hand, or away from the user's body, such as in a car mount or sitting on a table).

[0041] Therefore, as described herein, sensor availability component 314 can use information associated with the current context 316 to generate availability information (e.g., availability score or other suitable information) indicating the trustworthiness and / or reliability of each available sensor identified by sensor scanning component 312. For example, in a scenario where a client application requests posture tracking while a user is cycling outdoors with a smartphone in their pocket, the current context 316 may include information such as device type (e.g., smartphone, smartwatch, earphones, etc.), user activity state (e.g., cycling, running, walking, etc.), device placement state (e.g., on the user, away from the user, in a trouser pocket, in the hand, etc.), or device location associated with the sensor (e.g., indoors, outdoors, on land, at sea, or in the air). In this example, placement in a user's pocket may cause the camera and ToF sensor to provide sensor information that is less useful for attitude estimation compared to other sensors, such as inertial sensors and / or GNSS receivers. Therefore, sensor availability component 314 can generate a higher availability score for the inertial sensor and / or GNSS receiver, and a lower availability score for the camera and / or ToF sensor (e.g., because the GNSS receiver requires satellite availability to generate reliable sensor input, which is available in the current context 316, and the camera requires sufficient illumination, sufficient features, and less occlusion to generate reliable sensor input, which is not available in the current context 316). In another example, in a scenario where a client application requests attitude tracking when a user is walking in an indoor garage with a smartphone in their hand and detects motion toward a camera, the current context 316 may include information such as device type (e.g., smartphone), current user activity state (e.g., walking), device placement state (e.g., the user's hand), and device location (e.g., indoors). In this example, the indoor location can cause the GNSS receiver to provide sensor information that is less useful for attitude estimation compared to other sensors such as cameras and / or ToF sensors. As a result, the sensor availability component 314 can generate a higher availability score for the camera and / or ToF sensor and a lower availability score for the GNSS receiver.

[0042] like Figure 3As further shown, availability information generated by sensor availability component 314 can be provided to sensor selection and configuration component 320, which can select a set of sensor modes based on availability information and one or more KPI requirements 330 related to the current context associated with the attitude tracking configuration of the client application. For example, in some aspects, one or more KPI requirements 330 may include power constraints and / or accuracy requirements associated with the attitude tracking configuration, wherein the power constraints and / or accuracy requirements may be based on one or more client requirements 332 and / or one or more parameters related to device configuration 334. For example, in some aspects, one or more client requirements may indicate accuracy requirements, power requirements (e.g., required battery level or wall power), processor requirements (e.g., required CPU, GPU, or NPU resources), memory requirements (e.g., required RAM or available disk storage), latency requirements, and / or other parameters related to attitude or velocity estimation requested by the client application. Furthermore, in some aspects, client requirements 332 may be further based on context 316 related to the current device type, device location, current motion state, current user activity state, and / or device placement. Additionally, device configuration 334 may provide one or more parameters related to available hardware resources that can be used to generate attitude or velocity estimates associated with the tracked object. For example, device configuration 334 may include indications of battery size and / or current battery level, whether wall power is available, and / or available processor resources, available memory resources (e.g., RAM), and / or available storage resources (e.g., disk).

[0043] In some aspects, as described herein, sensor selection and configuration component 320 can select from a variety of sensors available in the sensor subsystem a set of sensor modes, including one or more of the available sensors. For example, as described herein, the set of sensor modes can be selected based on sensor availability information provided by sensor availability component 314 and one or more KPI requirements 330 (e.g., accuracy requirements, power constraints, hardware requirements, etc.) based on client requirements 332 and device configuration 334. Therefore, the selected set of sensor modes can be input to model selection component 340, which can select and optimize an attitude tracking model based on the selected set of sensor modes. Furthermore, in some aspects, sensor selection and configuration component 320 can provide model selection component 340 with information relating to the current context 316 and / or a set of model selection parameters (such as power specifications (e.g., constraints or requirements), accuracy specifications, etc.) (e.g., KPI requirements 330 and / or any suitable combination of context 316, client requirements 332, and device configuration 334 used to determine KPI requirements 330). Therefore, the model selection component 340 can then select and optimize the attitude tracking model best suited for the current attitude estimation task based on the selected set of sensor modes and various other model selection parameters provided by the sensor selection and configuration component 320.

[0044] For example, in some aspects, the model selection component 340 can access a variety of different attitude tracking models and can select the attitude tracking model to be used for the current attitude estimation task based on various inputs provided by the sensor selection and configuration component 320. For example, the various attitude tracking models may include visual inertial ranging (VIO) attitude tracking models, learned inertial ranging (LIO) attitude tracking models, GNSS plus LIO (GLIO) attitude tracking models, high-accuracy attitude tracking models, low-power attitude tracking models, one or more user activity recognition models, etc. Furthermore, each attitude tracking model may be associated with one or more model subtypes. For example, an attitude tracking model may be associated with a subtype that relies solely on machine learning, a subtype that relies solely on Kalman filter propagation, and / or a subtype that relies on a combination of machine learning and Kalman filter propagation. In addition, one or more attitude tracking models may be associated with different measurement types, which may include real measurements (e.g., camera measurements, GNSS measurements, pressure sensor measurements, etc.) and / or virtual measurements. For example, virtual measurements may include motion-based or physics-based measurements (e.g., zero-velocity updates, absolute stationary detection, and / or nonholonomic constraints) and / or learning-based measurements (e.g., multi-rate or single-rate LIO).

[0045] Therefore, model selection component 340 can then select and optimize the attitude tracking model best suited for the current attitude estimation task based on the selected set of sensor modes and various other model selection parameters provided by sensor selection and configuration component 320. For example, in a scenario where client request 332 indicates that a client application is requesting high-accuracy attitude tracking and device configuration 334 indicates that the battery is at or near full capacity, model selection component 340 can select a high-accuracy attitude tracking model (e.g., VIO) based on the selected set of sensor modes indicating that the camera and GNSS receiver are available in the current context 316. In another example, client request 332 may indicate a request for power-efficient attitude tracking, and device configuration 334 may indicate a limited battery level (e.g., below a threshold), and model selection component 340 can select a low-power attitude tracking model (e.g., inertial ranging model, inertial navigation model, tight multi-rate LIO model, etc.) that provides reasonable accuracy.

[0046] In another example, model selection component 340 can select a VIO model in a scenario where the selected sensor modalities include a camera, accelerometer, and gyroscope. Device configuration 334 indicates high power availability, client requirement 332 indicates high accuracy requirements, and context 316 indicates that the XR headset is being used in a bright room with abundant visual features. In another example, model selection component 340 can select a GLIO model in a scenario where the selected sensor modalities include a GNSS receiver, accelerometer, and gyroscope. Device configuration 334 indicates high power availability, and client requirement 332 indicates high accuracy requirements. In yet another example, model selection component 340 can select an LIO model in a scenario where the selected sensor modalities include an accelerometer and gyroscope (e.g., a GNSS receiver is unavailable). Device configuration 334 indicates low power availability, and client requirement 332 indicates medium accuracy requirements.

[0047] In some aspects, in addition to selecting the attitude tracking model best suited for the current attitude estimation task based on the selected set of sensor modalities, the current context, and the KPI requirement 330 based on client requirement 332 and device configuration 334, model selection component 340 may include a model optimizer for configuring the attitude tracking model and continuously rebalancing the trade-offs associated with client requirement 332, device configuration 334, and model feedback 370. In this way, model selection component 340 can use the model optimizer to output an optimized attitude tracking model (e.g., to the attitude estimation component) for a given KPI requirement 330, context 316, and / or selected set of sensor modalities. For example, in some aspects, the model optimizer may be configured to perform one or more hardware optimizations on the selected attitude tracking model (e.g., selecting a low-power island for a low-power model or a CPU or GPU for a high-power model, depending on availability). Additionally or alternatively, the model optimizer may perform one or more software optimizations on the selected attitude tracking model (e.g., quantization, neural network pruning, and / or data compression). Additionally or alternatively, the model optimizer may perform one or more reconfiguration optimizations on the selected attitude tracking model based on model feedback 370 (e.g., using model feedback 370 to tune the attitude tracking model and / or reconfigure one or more associated parameters).

[0048] Therefore, as described herein, the model selection component 340 can use a model optimizer to perform one or more hardware optimizations, one or more software optimizations, and / or one or more feedback-based optimizations on the attitude tracking model selected for a given attitude tracking task. For example, in a scenario where the attitude tracking device is included in augmented reality (AR) glasses worn by a user walking outdoors and then entering an indoor area, or in a scenario where the attitude tracking device is included in a vehicle entering a tunnel or parking lot, the attitude tracking device on the AR glasses can reconfigure one or more parameters of the selected attitude tracking model to reduce dependence on GNSS signals and increase the weights applied to sensor inputs associated with the camera. In another example, where the attitude tracking device is included in an interposer with high battery levels and available GPUs, the model optimizer can perform hardware optimization based on a client application requesting high-performance attitude estimation to select the GPU as the preferred hardware resource. In another example, where the attitude tracking device is included in a wearable device with low battery levels and available low-power islands, the model optimizer can perform hardware optimization based on a client application weighting the continuous model output above performance accuracy to use the low-power islands as the preferred hardware resource. Additionally or alternatively, the model optimizer may perform one or more software optimizations, such as machine learning model pruning, quantization, data compression, and / or data transfer optimization, to save computational resources and / or power. In other examples, the model optimizer may shift processing and / or data burdens from high-power hardware (e.g., GPU) to low-power hardware (e.g., low-power islands or CPUs) based on changes in model feedback 370 and / or client requirements 332, device configuration 334, and / or context 316 (e.g., where model feedback 370 indicates that the selected pose tracking model consumes more power than allowed by client requirements 332 and / or based on changes in client requirements 332 that reduce the priority of pose tracking).

[0049] In some aspects, such as Figure 3As shown, the attitude tracking model selected and optimized by the model selection component can be provided to the attitude estimation component 350, which can use the attitude tracking model to generate a set of sensor inputs based on the set of selected sensor modes to produce a model output 360. For example, in some aspects, the model output 360 may include an estimated attitude associated with a tracked object (e.g., a user, user equipment, or another physical object), wherein the estimated attitude may include the estimated position and / or estimated orientation of the tracked object relative to one or more axes. For example, in some aspects, the estimated position of the tracked object may include the absolute position of the tracked object on one or more axes at a specific time and / or the relative position (e.g., displacement) of the tracked object on one or more axes over a given duration. Similarly, the estimated orientation of the tracked object may include the absolute orientation of the tracked object on one or more axes at a specific time and / or the relative orientation (e.g., change of orientation) of the tracked object on one or more axes over a given duration. Additionally or alternatively, the model output 360 may include one or more parameters related to the motion state of the tracked object, such as linear velocity estimates (e.g., absolute and / or relative linear velocity estimates) and / or angular velocity estimates (e.g., absolute and / or relative angular velocity estimates). Additionally or alternatively, the model output 360 may include one or more parameter calibrations, such as one or more sensor biases, sensor sensitivity, drift over time or temperature, or other suitable parameter calibrations.

[0050] In some aspects, such as Figure 3 As further illustrated, the attitude estimation component 350 can generate model feedback 370, which can be continuously monitored and used to improve the performance of various other components of the attitude tracking device. For example, as... Figure 3 As shown, model feedback can be provided to sensor availability component 314, sensor selection and configuration component 320, and / or model selection component 340. Furthermore, in some aspects, model feedback 370 can be used to update the context 316 input to sensor availability component 314, the context 316 input to sensor selection and configuration component 320, to determine one or more client requests 332, and / or to generate user feedback 380 (e.g., requesting the user to reset the attitude tracking device or manually calibrate one or more sensors). For example, in some aspects, model feedback 370 can include information such as estimated confidence or uncertainty associated with model output 360, performance trends or saturation trends (e.g., Kalman filter innovation, loop closure, outlier rejection, etc.), the need for additional or specific sensor modes, and / or power consumption metrics.

[0051] Therefore, as described herein, model feedback 370 can be used in various ways to improve the performance of various other components of the attitude tracking device. For example, model feedback 370 provided to sensor selection and configuration component 320 may include performance and power consumption metrics associated with the current attitude tracking model, which sensor selection and configuration component 320 may match to client requirements 332. For example, in a scenario where model feedback 370 indicates that the performance of the current inertial measurement unit (IMU) attitude tracking model has saturated with power consumption below limits and accuracy that does not meet client requirements 332, the sensor modes selected by sensor selection and configuration component 320 may include one or more additional sensor modes based on the current context 316. For example, additional sensor modes may include GNSS based on context 316 indicating that the user is cycling outdoors, where the attitude tracking device is included in a smartphone in the user's pocket, or a camera based on context 316 indicating that the user is walking on a flat surface indoors, where the attitude tracking device is included in a smartphone in the user's hand, with motion detection facing the camera. Furthermore, in this case, other available sensors may remain disabled. Typically, the process of updating the selected sensor mode can be initiated by the sensor selection and configuration component 320, which can determine when and / or whether to request a new sensor mode or invoke the sensor scanning component 312 or the sensor availability component 314. Furthermore, the sensor selection and configuration component 320 can select appropriate configurations (e.g., IMU sampling frequency and / or camera frame rate, among others) for each sensor mode selected for a given attitude estimation task.

[0052] Therefore, as described herein, model feedback 370 can be used by sensor selection and configuration component 320 to determine whether to add and / or remove one or more sensor modes (e.g., based on the uncertainty or performance trend of the estimate associated with model output 360, which can indicate the validity or confidence of the currently selected sensor mode). Furthermore, model feedback 370 can be used to improve the performance of model selection component 340. For example, model feedback 370 may include performance metrics and / or power consumption metrics that model selection component 340 can use to determine when and / or whether the attitude tracking model needs to be tuned or reconfigured (e.g., based on the uncertainty or performance trend of the estimate indicating the validity of the current attitude tracking model). Additionally or alternatively, model feedback 370 can be used to generate user feedback 380, which may include one or more outputs requesting user intervention to improve the performance of the attitude tracking device. For example, in some aspects, user feedback 380 may include a request for the user to manually recalibrate the magnetometer (e.g., by performing a figure-eight movement using an attitude tracking device that includes the magnetometer), change environmental conditions (e.g., move to a different location when model uncertainty is too high), charge the battery when the battery level fails to meet a threshold, or replace the battery when it consistently fails to maintain charge, and / or provide permission to enable location tracking, Wi-Fi or WLAN radio, and / or cellular radio. For example, in some aspects, user feedback 380 may request permission from the user to share one or more pieces of information or data related to the user in order to protect the user's privacy.

[0053] In this way, model feedback 370 can be used in various ways to improve the overall performance of the attitude tracking device. For example, in some aspects, model feedback 370 can be used to update one or more decision strategies used by sensor selection and configuration component 320 to select a set of sensor modes, or by model selection component 340 to select the current attitude tracking model. Additionally or alternatively, model feedback 370 can be used to generate user feedback 380, wherein user intervention is requested to calibrate one or more decision strategies used for selecting sensor modes and / or the current attitude tracking model. Furthermore, model feedback 370 can be used to update the context used for selecting sensor modes and / or the current attitude tracking model. For example, a user activity recognition algorithm may face difficulties in distinguishing motion on a car from motion on a train, and model feedback 370 can be used as additional input (e.g., as speed over a specific duration) to update the user activity recognition state or vehicle classification state associated with the current context 316. Additionally or alternatively, the Model Feedback 370 may be shared with one or more external devices (e.g., other devices associated with the same user or different original equipment manufacturer (OEM) devices with the same shape factor and / or following public KPI protocols or the Internet cloud).

[0054] As mentioned above, Figure 3 This is provided as an example. Other examples may be provided in conjunction with [the relevant information]. Figure 3 The examples described are different. Figure 3 The number and arrangement of devices shown are provided as an example. In practice, there may be variations. Figure 3 The equipment shown is compared to additional equipment, fewer devices, different equipment, or equipment with a different arrangement. Furthermore, Figure 3 The two or more devices shown can be implemented within a single device, or Figure 3 The single device shown can be implemented as multiple distributed devices. Additionally or alternatively, Figure 3 The collection of devices shown (e.g., one or more devices) can perform actions described as being performed by Figure 3 The set of other devices shown performs one or more functions.

[0055] Figure 4 This is a flowchart of an example process 400 associated with power-efficient, performance-efficient, and context-adaptive attitude tracking according to this disclosure. In some aspects, Figure 4 One or more processing blocks are performed by the attitude tracking device (e.g., attitude tracking device 110). In some aspects, Figure 4One or more process frames are performed by another device or a group of devices, either separate from or including the attitude tracking device, such as the tracked object (e.g., tracked object 120) and / or network nodes (e.g., network node 130). Additionally or alternatively, Figure 4 One or more process frames may be executed by one or more components of device 200 (e.g., processor 210, memory 215, storage component 220, input component 225, output component 230, communication interface 235, sensor subsystem 240 and / or attitude tracking component 245).

[0056] like Figure 4 As shown, process 400 may include receiving information including one or more KPI requirements associated with the current context, which is associated with the attitude tracking configuration of the client application (box 410). For example, the attitude tracking device may receive information including one or more KPI requirements associated with the current context and the attitude tracking configuration of the client application, as described above.

[0057] like Figure 4 As further shown, process 400 may include receiving availability information from a sensor system comprising multiple sensors based on one or more parameters related to the current operating condition associated with the multiple sensors (box 420). For example, an attitude tracking device may receive availability information from a sensor system comprising multiple sensors based on one or more parameters related to the current operating condition associated with the multiple sensors, as described above.

[0058] like Figure 4 As further shown, process 400 may include selecting a set of sensor modes comprising one or more sensors from a plurality of sensors included in a sensor system based on availability information related to the current context associated with the attitude tracking configuration of the client application and the current operating conditions associated with the plurality of sensors (box 430). For example, the attitude tracking device may select a set of sensor modes comprising one or more sensors from a plurality of sensors included in a sensor system based on the current context associated with the attitude tracking configuration of the client application and availability information related to the current operating conditions associated with the plurality of sensors, as described above.

[0059] like Figure 4As further shown, process 400 may include selecting an attitude tracking model based on a set of sensor modes and one or more KPI requirements related to the current context associated with the attitude tracking configuration of the client application (box 440). For example, the attitude tracking device may select an attitude tracking model based on a set of sensor modes and one or more KPI requirements related to the current context associated with the attitude tracking configuration of the client application, as described above.

[0060] like Figure 4 As further shown, process 400 may include estimating the attitude associated with the tracked object using an attitude tracking model based on sensor inputs associated with a set of sensor modes (box 450). For example, an attitude tracking device may estimate the attitude associated with the tracked object using an attitude tracking model based on sensor inputs associated with a set of sensor modes, as described above.

[0061] Process 400 may include additional aspects, such as those described below and / or any single aspect or any combination of aspects described in conjunction with one or more other process descriptions elsewhere herein.

[0062] In the first aspect, the availability information includes, for each of the plurality of sensors included in the sensor system, a context-based availability score, the context including one or more of the following: device type associated with the sensor, motion state, current user activity state, device placement state, or device location state.

[0063] In the second aspect, either alone or in combination with the first aspect, the attitude tracking model is selected based on a set of inputs, including selected sensor modes, available hardware resources associated with the attitude tracking device, accuracy requirements for attitude estimation, context, or one or more KPI requirements.

[0064] In the third aspect, one or more KPI requirements, individually or in combination with one or more of the first and second aspects, and related to the current context of the attitude tracking configuration, include one or more parameters related to the power consumption requirements for estimating the attitude.

[0065] In the fourth aspect, one or more KPI requirements, individually or in combination with one or more of the first to third aspects, and related to the current context of the attitude tracking configuration, include one or more parameters related to the accuracy requirements for attitude estimation.

[0066] In the fifth aspect, one or more KPI requirements, alone or in combination with one or more of the first to fourth aspects, are based on context, which includes one or more of the following: device type associated with the sensor, motion state, current user activity state, device placement state, or device location state.

[0067] In the sixth aspect, either alone or in combination with one or more of the first to fifth aspects, the estimated pose is related to one or more of the position of the tracked object relative to one or more axes or the orientation of the tracked object relative to one or more axes.

[0068] In the seventh aspect, alone or in combination with one or more of the first to sixth aspects, the positioning of the tracked object includes absolute positioning at a specific time instance or relative positioning or displacement over a specified duration.

[0069] In the eighth aspect, alone or in combination with one or more of the first to seventh aspects, the orientation of the tracked object includes the absolute orientation at a specific time instance or the relative orientation or change of orientation over a specified duration.

[0070] In the ninth aspect, either alone or in combination with one or more of the first to eighth aspects, process 400 includes using an attitude tracking model and sensor inputs associated with a set of sensor modes to estimate one or more velocities or one or more parameters associated with the tracked object to calibrate one or more sensors.

[0071] In the tenth aspect, alone or in combination with one or more of the first to ninth aspects, process 400 includes generating feedback related to the performance of the attitude tracking model in estimating the attitude associated with the tracked object, and updating one or more decision strategies based on the feedback for selecting at least one of the set of sensor modes or attitude tracking models.

[0072] In the eleventh aspect, alone or in combination with one or more of the first to tenth aspects, process 400 includes generating feedback related to the performance of the attitude tracking model in estimating the attitude associated with the tracked object, and generating one or more outputs based on the feedback to request one or more user interactions to calibrate one or more decision strategies for selecting at least one of the set of sensor modes or attitude tracking models.

[0073] In the twelfth aspect, alone or in combination with one or more of the first to eleventh aspects, process 400 includes generating performance-related feedback of the attitude tracking model in estimating the attitude associated with the tracked object, and sharing performance-related feedback of the attitude tracking model with one or more external devices.

[0074] In the thirteenth aspect, alone or in combination with one or more of the first to twelfth aspects, process 400 includes generating feedback related to the performance of the attitude tracking model in estimating the attitude associated with the tracked object, and updating the context for selecting at least one of the set of sensor modes or attitude tracking models based on the feedback.

[0075] although Figure 4 An example box of process 400 is shown, but in some respects, process 400 includes... Figure 4 The boxes depicted in the diagram may be fewer, different, or arranged differently compared to additional boxes. Alternatively, two or more boxes in the process 400 may be executed in parallel.

[0076] The following provides an overview of some aspects of this disclosure:

[0077] Aspect 1: A method for power-efficient and performance-efficient context-adaptive attitude tracking, comprising: receiving by an attitude tracking device information including one or more KPI requirements associated with a current context, the current context being associated with an attitude tracking configuration of a client application; receiving by the attitude tracking device availability information from a sensor system including multiple sensors based on one or more parameters associated with a current operating condition and multiple sensors; selecting by the attitude tracking device a set of sensor modes including one or more sensors from multiple sensors included in the sensor system based on the current context associated with the attitude tracking configuration of the client application and the availability information associated with the current operating condition and multiple sensors; selecting by the attitude tracking device an attitude tracking model based on the set of sensor modes and one or more KPI requirements associated with the current context associated with the attitude tracking configuration of the client application; and estimating the attitude associated with a tracked object using the attitude tracking model based on sensor inputs associated with the set of sensor modes.

[0078] Aspect 2: According to the method of aspect 1, wherein the availability information includes, for each of a plurality of sensors included in the sensor system, a corresponding availability score based on context, the context including one or more of the following: device type associated with the sensor, motion state, current user activity state, device placement state, or device location state.

[0079] Aspect 3: The method according to any one of Aspects 1-2, wherein the attitude tracking model is selected based on a set of inputs including selected sensor modes, available hardware resources associated with the attitude tracking device, accuracy requirements for attitude estimation, context, or one or more of one or more KPI requirements.

[0080] Aspect 4: The method according to any one of Aspects 1-3, wherein one or more KPI requirements associated with the current context and attitude tracking configuration include one or more parameters related to power consumption requirements for attitude estimation.

[0081] Aspect 5: The method according to any one of Aspects 1-4, wherein one or more KPI requirements related to the current context associated with the attitude tracking configuration include one or more parameters related to the accuracy requirements for attitude estimation.

[0082] Aspect 6: The method according to any one of Aspects 1-5, wherein one or more KPI requirements are based on context, which includes one or more of the following: device type associated with the sensor, motion state, current user activity state, device placement state, or device location state.

[0083] Aspect 7: The method according to any one of Aspects 1-6, wherein the estimated pose is related to one or more of the positioning of the tracked object relative to one or more axes or the orientation of the tracked object relative to one or more axes.

[0084] Aspect 8: According to the method of aspect 6, the positioning of the tracked object includes absolute positioning at a specific time instance or relative positioning or displacement over a specified duration.

[0085] Aspect 9: According to the method of aspect 6, the orientation of the tracked object includes an absolute orientation at a specific time instance or a relative orientation or a change in orientation over a specified duration.

[0086] Aspect 10: The method according to any one of Aspects 1-9 further includes: using an attitude tracking model and sensor inputs associated with a set of sensor modes to estimate one or more velocities or one or more parameters associated with the tracked object to calibrate one or more sensors.

[0087] Aspect 11: The method according to any one of Aspects 1-10 further includes: generating feedback related to the performance of the attitude tracking model in estimating the attitude associated with the tracked object; and updating one or more decision strategies based on the feedback for selecting at least one of the set of sensor modes or attitude tracking models.

[0088] Aspect 12: The method according to any one of Aspects 1-11 further includes: generating feedback related to the performance of the attitude tracking model in estimating the attitude associated with the tracked object; and generating one or more outputs based on the feedback to request one or more user interactions to calibrate one or more decision strategies for selecting at least one of a set of sensor modes or an attitude tracking model.

[0089] Aspect 13: The method according to any one of Aspects 1-12 further includes: generating performance-related feedback of the attitude tracking model in estimating the attitude associated with the tracked object; and sharing the performance-related feedback of the attitude tracking model with one or more external devices.

[0090] Aspect 14: The method according to any one of Aspects 1-13 further includes: generating feedback related to the performance of the attitude tracking model in estimating the attitude associated with the tracked object; and updating the context for selecting at least one of the set of sensor modes or attitude tracking models based on the feedback.

[0091] Aspect 15: An attitude tracking device for power-efficient and performance-efficient context-adaptive attitude tracking, comprising: one or more memories; and one or more processors coupled to the one or more memories, the one or more processors being configured to: receive information including one or more KPI requirements associated with a current context, which is associated with an attitude tracking configuration of a client application; receive availability information from a sensor system including multiple sensors based on one or more parameters associated with a current operating condition associated with multiple sensors; select a set of sensor modes including one or more sensors from multiple sensors included in the sensor system based on the current context associated with the attitude tracking configuration of the client application and the availability information associated with the current operating condition associated with the multiple sensors; select an attitude tracking model based on the set of sensor modes and one or more KPI requirements associated with the current context associated with the attitude tracking configuration of the client application; and estimate the attitude associated with a tracked object using the attitude tracking model based on sensor inputs associated with the set of sensor modes.

[0092] Aspect 16: The attitude tracking device according to aspect 15, wherein the availability information includes, for each of a plurality of sensors included in a sensor system, a context-based availability score, the context including one or more of the following: device type associated with the sensor, motion state, current user activity state, device placement state, or device location state.

[0093] Aspect 17: An attitude tracking device according to any one of Aspects 15-16, wherein the attitude tracking model is selected based on a set of inputs including selected sensor modes, available hardware resources associated with the attitude tracking device, accuracy requirements for attitude estimation, context, or one or more of one or more KPI requirements.

[0094] Aspect 18: An attitude tracking device according to any one of Aspects 15-17, wherein one or more KPI requirements associated with the current context and attitude tracking configuration include one or more parameters related to power consumption requirements for estimating attitude.

[0095] Aspect 19: An attitude tracking device according to any one of Aspects 15-18, wherein one or more KPI requirements associated with the current context and attitude tracking configuration include one or more parameters related to accuracy requirements for attitude estimation.

[0096] Aspect 20: An attitude tracking device according to any one of Aspects 15-19, wherein one or more KPI requirements are based on a context that includes one or more of the following: device type associated with the sensor, motion state, current user activity state, device placement state, or device location state.

[0097] Aspect 21: An attitude tracking device according to any one of aspects 15 to 20, wherein the estimated attitude is related to one or more of the positioning of the tracked object relative to one or more axes or the orientation of the tracked object relative to one or more axes.

[0098] Aspect 22: The attitude tracking device according to aspect 20, wherein the positioning of the tracked object includes absolute positioning at a specific time instance or relative positioning or displacement over a specified duration.

[0099] Aspect 23: The attitude tracking device according to aspect 20, wherein the orientation of the tracked object includes an absolute orientation at a specific time instance or a relative orientation or a change in orientation over a specified duration.

[0100] Aspect 24: An attitude tracking device according to any one of Aspects 15-23, wherein one or more processors are further configured to: estimate one or more velocities or one or more parameters associated with the tracked object using an attitude tracking model and sensor inputs associated with a set of sensor modes to calibrate one or more sensors.

[0101] Aspect 25: An attitude tracking device according to any one of Aspects 15-24, wherein one or more processors are further configured to: generate feedback relating to the performance of the attitude tracking model in estimating the attitude associated with the tracked object; and update one or more decision strategies based on the feedback for selecting at least one of the set of sensor modes or the attitude tracking model.

[0102] Aspect 26: An attitude tracking device according to any one of Aspects 15 to 25, wherein one or more processors are further configured to: generate feedback relating to the performance of the attitude tracking model in estimating the attitude associated with the tracked object; and generate one or more outputs based on the feedback to request one or more user interactions to calibrate one or more decision strategies for selecting at least one of a set of sensor modes or an attitude tracking model.

[0103] Aspect 27: An attitude tracking device according to any one of Aspects 15 to 26, wherein one or more processors are further configured to: generate performance-related feedback of the attitude tracking model in estimating the attitude associated with the tracked object; and share the performance-related feedback of the attitude tracking model with one or more external devices.

[0104] Aspect 28: An attitude tracking device according to any one of Aspects 15-27, wherein one or more processors are further configured to: generate feedback relating to the performance of the attitude tracking model in estimating the attitude associated with the tracked object; and update the context for selecting at least one of the set of sensor modes or the attitude tracking model based on the feedback.

[0105] Aspect 29: A non-transitory computer-readable medium storing a set of instructions for power-efficient and performance-efficient context-adaptive attitude tracking, the set of instructions comprising: one or more instructions, when executed by one or more processors of an attitude tracking device, causing the attitude tracking device to: receive information including one or more KPI requirements associated with a current context, which is associated with an attitude tracking configuration of a client application; receive availability information from a sensor system including multiple sensors based on one or more parameters associated with a current operating condition associated with multiple sensors; select a set of sensor modes including one or more sensors from multiple sensors included in the sensor system based on the current context associated with the attitude tracking configuration of the client application and the availability information associated with the current operating condition associated with multiple sensors; select an attitude tracking model based on the set of sensor modes and one or more KPI requirements associated with the current context associated with the attitude tracking configuration of the client application; and estimate the attitude associated with a tracked object using the attitude tracking model based on sensor inputs associated with the set of sensor modes.

[0106] Aspect 30: An apparatus for power-efficient and performance-efficient context-adaptive attitude tracking, comprising: means for receiving information including one or more KPI requirements associated with a current context, the current context being associated with an attitude tracking configuration of a client application; means for receiving availability information from a sensor system including multiple sensors based on one or more parameters associated with a current operating condition and multiple sensors; means for selecting a set of sensor modes including one or more sensors from multiple sensors included in the sensor system based on the current context associated with the attitude tracking configuration of the client application and the availability information associated with the current operating condition and multiple sensors; means for selecting an attitude tracking model based on the set of sensor modes and one or more KPI requirements associated with the current context and attitude tracking configuration for the client application; and means for estimating the attitude associated with a tracked object using the attitude tracking model based on sensor inputs associated with the set of sensor modes.

[0107] Aspect 31: A system configured to perform one or more operations described in one or more aspects of aspects 1-30.

[0108] Aspect 32: An apparatus comprising components for performing one or more of the operations described in one or more of aspects 1-30.

[0109] Aspect 33: A non-transitory computer-readable medium storing a set of instructions, the set of instructions comprising one or more instructions which, when executed by a device, cause the device to perform one or more operations described in one or more of aspects 1 to 30.

[0110] Aspect 34: A computer program product comprising instructions or code for performing one or more of the operations described in one or more of aspects 1-30.

[0111] The foregoing disclosure provides explanations and descriptions, but is not intended to be exhaustive or to limit the parties to the precise forms disclosed. Modifications and variations can be made based on the foregoing disclosure, or from various practices.

[0112] As used herein, the term "component" is intended to be interpreted broadly as hardware and / or a combination of hardware and software. Whether referred to as software, firmware, middleware, microcode, hardware description language, or other terms, "software" should be interpreted broadly as instructions, instruction sets, code, code segments, program code, programs, subroutines, software modules, applications, software applications, software packages, routines, subroutines, objects, executable files, threads of execution, procedures and / or functions, and other examples. As used herein, a "processor" is implemented in hardware and / or a combination of hardware and software. It is clear that the systems and / or methods described herein can be implemented in various forms of hardware and / or combinations of hardware and software. The actual dedicated control hardware or software code used to implement these systems and / or methods does not limit these aspects. Therefore, this document describes the operation and behavior of systems and / or methods without reference to any specific software code, as those skilled in the art will understand that software and hardware can be designed to implement systems and / or methods, at least in part, based on the descriptions herein.

[0113] As used in this article, depending on the context, “meeting the threshold” can refer to a value that is greater than the threshold, greater than or equal to the threshold, less than the threshold, less than or equal to the threshold, equal to the threshold, not equal to the threshold, etc.

[0114] Even if a specific combination of features is described in the claims and / or disclosed in the specification, such combinations are not intended to limit the disclosure of the aspects. Many of these features can be combined in ways not specifically described in the claims and / or disclosed in the specification. The disclosure of the aspects includes combinations of each dependent claim with each other claim in the claim set. As used herein, the phrase “at least one” in the list of items refers to any combination of these items, including a single member. As an example, “at least one of a, b, or c” is intended to cover a, b, c, a+b, a+c, b+c, and a+b+c, as well as any combination with multiple identical elements (e.g., a+a, a+a+a, a+a+b, a+a+c, a+b+b, a+c+c, b+b, b+b+b, b+b+c, c+c, and c+c+c, or any other ordering of a, b, and c).

[0115] Unless explicitly stated otherwise, the elements, actions, or instructions used herein should not be construed as critical or necessary. Furthermore, as used herein, the articles “a” and “an” are intended to include one or more items and are interchangeable with “one or more.” Additionally, as used herein, the article “the” is intended to include one or more items referenced by the article “the” and is interchangeable with “one or more.” Furthermore, as used herein, the terms “set” and “group” are intended to include one or more items and are interchangeable with “one or more.” Figure 1 In the case of a single project, use the phrase “only one” or similar language. Furthermore, as used herein, the terms “has,” “have,” “having,” etc., are intended to be open-ended terms that do not limit the elements they modify (e.g., an element “having” A can also have B). Additionally, unless explicitly stated otherwise, the phrase “based on” is intended to mean “at least partially based on.” Furthermore, as used herein, the term “or” is intended to be inclusive when used in series and can be used interchangeably with “and / or” unless explicitly stated otherwise (e.g., if used in combination with “any” or “only one of”).

Claims

1. A method for power and performance efficient context adaptive pose tracking, comprising: receiving, by a pose tracking device, information comprising one or more key performance indicator (KPI) requirements related to a current context associated with a pose tracking configuration of a client application; receiving, by the pose tracking device, availability information from a sensor system comprising a plurality of sensors based on one or more parameters related to current operating conditions associated with the plurality of sensors; selecting, by the pose tracking device, a set of sensor modalities comprising one or more sensors from the plurality of sensors included in the sensor system based on the current context associated with the pose tracking configuration of the client application and the availability information related to the current operating conditions associated with the plurality of sensors; selecting, by the pose tracking device, a pose tracking model based on the set of sensor modalities and the one or more KPI requirements related to the current context associated with the pose tracking configuration of the client application; and estimating, by the pose tracking device, a pose associated with a tracked object using the pose tracking model based on sensor inputs associated with the set of sensor modalities. the availability information comprises, for each sensor of the plurality of sensors included in the sensor system, a respective availability score based on a context comprising one or more of a device type, a motion state, a current user activity state, a device placement state, or a device location state associated with the sensor.

2. The method of claim 1, wherein, the pose tracking model is selected based on a set of inputs comprising one or more of the selected sensor modalities, available hardware resources associated with the pose tracking device, accuracy requirements for estimating the pose, the context, or the one or more KPI requirements.

3. The method of claim 1, wherein, the one or more KPI requirements related to the current context associated with the pose tracking configuration comprise one or more parameters related to power consumption requirements for estimating the pose.

4. The method of claim 1, wherein, the one or more KPI requirements related to the current context associated with the pose tracking configuration comprise one or more parameters related to accuracy requirements for estimating the pose.

5. The method of claim 1, wherein, the one or more KPI requirements are based on a context comprising one or more of a device type, a motion state, a current user activity state, a device placement state, or a device location state associated with the sensor.

6. The method of claim 1, wherein, the estimated pose relates to one or more of a position of the tracked object relative to one or more axes or an orientation of the tracked object relative to one or more axes.

7. The method of claim 1, wherein, the position of the tracked object comprises an absolute position at a particular time instance or a relative position or displacement over a specified duration of time.

8. The method of claim 7, wherein, the orientation of the tracked object comprises an absolute orientation at a particular time instance or a relative orientation or change in orientation over a specified duration of time.

9. The method of claim 7, wherein, 10. The method of claim 1, further comprising: ​ estimating one or more velocities or one or more parameters associated with a tracked object using a pose tracking model and sensor inputs associated with a set of sensor modalities to calibrate the one or more sensors.

11. The method of claim 1, further comprising: generating feedback related to performance of a pose tracking model in estimating a pose associated with a tracked object; and updating one or more decision policies for selecting at least one of a set of sensor modalities or a pose tracking model based on the feedback.

12. The method of claim 1, further comprising: generating feedback related to performance of a pose tracking model in estimating a pose associated with a tracked object; and generating one or more outputs to request one or more user interactions to calibrate one or more decision policies for selecting at least one of a set of sensor modalities or a pose tracking model based on the feedback.

13. The method of claim 1, further comprising: generating feedback related to performance of a pose tracking model in estimating a pose associated with a tracked object; and sharing the feedback related to performance of the pose tracking model with one or more external devices.

14. The method of claim 1, further comprising: generating feedback related to performance of a pose tracking model in estimating a pose associated with a tracked object; and updating a context for selecting at least one of a set of sensor modalities or a pose tracking model based on the feedback.

15. A pose tracking device for power-efficient and performance-efficient context adaptive pose tracking, comprising: one or more memories; and one or more processors coupled to the one or more memories, the one or more processors configured to: receive information including one or more key performance indicator (KPI) requirements related to a current context associated with a pose tracking configuration of a client application; receive availability information from a sensor system including a plurality of sensors based on one or more parameters related to current operating conditions associated with the plurality of sensors; select a set of sensor modalities including one or more sensors from the plurality of sensors included in the sensor system based on the current context associated with the pose tracking configuration of the client application and the availability information related to the current operating conditions associated with the plurality of sensors; select a pose tracking model based on the set of sensor modalities and the one or more KPI requirements related to the current context associated with the pose tracking configuration of the client application; and estimate a pose associated with a tracked object using the pose tracking model based on sensor inputs associated with the set of sensor modalities. the availability information includes, for each sensor of the plurality of sensors included in the sensor system, a respective availability score based on a context including one or more of a device type, a motion state, a current user activity state, a device placement state, or a device location state associated with the sensor.

16. The pose tracking device of claim 15, wherein, ​ 17. The pose tracking device of claim 15, wherein, The attitude tracking model is selected based on a set of inputs, including selected sensor modes, available hardware resources associated with the attitude tracking device, accuracy requirements for attitude estimation, context, or one or more of the aforementioned KPI requirements.

18. The pose tracking device of claim 15, wherein, The one or more KPI requirements associated with the current context of the attitude tracking configuration include one or more parameters related to the power consumption requirements for attitude estimation.

19. The pose tracking device of claim 15, wherein, The one or more KPI requirements associated with the current context of the attitude tracking configuration include one or more parameters related to the accuracy requirements for attitude estimation.

20. The pose tracking device of claim 15, wherein, The one or more KPI requirements are context-based, and the context includes one or more of the following: device type associated with the sensor, motion state, current user activity, device placement state, or device status location.

21. The pose tracking device of claim 15, wherein, The estimated pose is related to one or more of the tracking object's position relative to one or more axes or the tracking object's orientation relative to one or more axes.

22. The pose tracking device of claim 21, wherein, The location of the tracked object includes absolute location at a specific time instance or relative location or displacement over a specified duration.

23. The pose tracking device of claim 21, wherein, The orientation of the tracked object includes the absolute orientation at a specific time instance or the relative orientation or change of orientation over a specified duration.

24. The pose tracking device of claim 15, wherein, The one or more processors are further configured to: The attitude tracking model and the sensor inputs associated with the set of sensor modes are used to estimate one or more velocities or one or more parameters associated with the tracked object to calibrate the one or more sensors.

25. The pose tracking device of claim 15, wherein, The one or more processors are further configured to: Generate feedback related to the performance of the pose tracking model in estimating the pose associated with the tracked object; as well as One or more decision strategies based on feedback updates are used to select at least one of the set of sensor modes or attitude tracking models.

26. The pose tracking device of claim 15, wherein, The one or more processors are further configured to: Generate feedback related to the performance of the pose tracking model in estimating the pose associated with the tracked object; as well as Based on feedback, one or more outputs are generated to request one or more user interactions to calibrate one or more decision strategies, which are used to select at least one of a set of sensor modes or an attitude tracking model.

27. The pose tracking device of claim 15, wherein, The one or more processors are further configured to: Generate feedback related to the performance of the pose tracking model in estimating the pose associated with the tracked object; as well as It can share feedback related to the performance of the attitude tracking model with one or more external devices.

28. The pose tracking device of claim 15, wherein, The one or more processors are further configured to: Generate feedback related to the performance of the pose tracking model in estimating the pose associated with the tracked object; as well as The context is updated based on feedback to select at least one of the set of sensor modes or attitude tracking models.

29. A non-transitory computer-readable medium storing a set of instructions for power-efficient and performance-efficient context-adaptive attitude tracking, the set of instructions comprising: One or more instructions, when executed by one or more processors of the attitude tracking device, cause the attitude tracking device to: Receive information including one or more key performance indicators (KPIs) requirements related to the current context, which is associated with the pose tracking configuration of the client application; Availability information is received from a sensor system including the multiple sensors based on one or more parameters related to the current operating status associated with the multiple sensors. Based on the current context associated with the attitude tracking configuration of the client application and availability information related to the current operating status associated with the plurality of sensors, a set of sensor modes including one or more sensors is selected from the plurality of sensors included in the sensor system. Based on the set of sensor modes and one or more KPI requirements related to the current context associated with the attitude tracking configuration of the client application, select an attitude tracking model; and Based on sensor inputs associated with a set of sensor modes, an attitude tracking model is used to estimate the attitude associated with the tracked object.

30. An apparatus for power-efficient and performance-efficient context-adaptive attitude tracking, comprising: A component for receiving information including one or more key performance indicators (KPIs) requirements related to the current context, which is associated with the pose tracking configuration of the client application; A component for receiving availability information from a sensor system including the multiple sensors based on one or more parameters related to the current operating status associated with the multiple sensors; A component for selecting a set of sensor modes, including one or more sensors, from the plurality of sensors included in a sensor system based on the current context associated with the attitude tracking configuration of the client application and availability information related to the current operating status associated with the plurality of sensors. Components for selecting the attitude tracking model based on the set of sensor modes and the current context associated with the attitude tracking configuration of the client application; as well as A component used to estimate the attitude associated with a tracked object using an attitude tracking model, based on sensor inputs associated with a set of sensor modes.

Images