Hierarchical motion-based enhanced inertial localization system

Abstract

The hierarchical motion-based enhanced inertial localization system introduces a hierarchical approach to indoor localization that transforms traditional IMU-based tracking from a sensor processing problem into a human motion understanding challenge. The system leverages a multi-level architecture that incorporates biomechanical constraints, activity recognition, and environmental context through a modular framework that adapts to available information. The system processes movement patterns at different levels of abstraction, from basic IMU measurements to full-body pose estimation and high-level activity recognition, the system improves state estimation constraints. The system models uncertainties at each hierarchical level while automatically adapting to available information sources through a flexible loss function framework. The hierarchical motion-based enhanced inertial localization system achieves a reduction in translation error, with significantly improved robustness in challenging real-world conditions where conventional approaches fail. This advancement represents a fundamental shift in indoor localization technology, offering enhanced accuracy and adaptability across diverse deployment scenarios.

Description

HIERARCHICAL MOTION-BASED ENHANCED INERTIAL LOCALIZATION SYSTEMCROSS REFERENCE TO RELATED APPLICATION(S)

[0001] This application claims priority to provisional Patent Application No. 63 / 790,485 entitled “Hierarchical Motion-Based Enhanced Inertial Localization System” and filed on 4 / 17 / 2025, which is included herein in its entirety.BACKGROUND

[0002] Indoor localization for humans has traditionally relied on direct processing of inertial measurements to estimate position and movement. However, this approach faces fundamental limitations - sensor noise accumulates through integration, leading to drift, while the absence of external reference points makes error correction difficult. For example, indoor localization using inertial measurement units (IMUs) has evolved through several distinct paradigms, each attempting to address the fundamental challenges of drift and error accumulation. Understanding this evolution provides crucial context to understand why a need has been recognized for a more accurate localization process.Classical Physics-based Approaches

[0003] The earliest approaches to IMU-based localization relied purely on physical principles.These strap-down inertial navigation systems (SINS) [Savage 1998] operate by doubleintegrating linear acceleration measurements after rotating them from the body frame to the navigation frame using device orientation data. However, these methods suffer from quadratic error propagation due to sensor noise, making them impractical for extended use [Shen et al. 2018], The fundamental limitation of these approaches isthat they treat the problem as pure physics, ignoring the structured nature of human motion.Heuristic-based Methods

[0004] Recognizing the limitations of pure physics-based approaches, researchers developed methods that leverage heuristic assumptions about human movement. Pedestrian Dead Reckoning (PDR) systems [Jimenez et al. 2009; Tian et al. 2015] decompose the problem into discrete components: step detection, step-length estimation, and heading estimation. While these methods showed improved performance over SINS, they rely on fixed parameters and assumptions about regular gait patterns that do not hold true for many real -world scenarios.Learning-based Approaches

[0005] The advent of deep learning led to a new generation of approaches that learn to estimate motion directly from IMU data. RIDI [Yan et al. 2018] introduced a two-stage system that first corrects acceleration measurements before integration. loNet [Chen et al. 2018] and RoNIN [Herath et al. 2020] demonstrated that neural networks could directly regress velocity segments, thus bypassing the need for explicit integration. These methods showed significant improvements for reducing drift, but still treated IMU data processing as an isolated problem.

[0006] Recent work has begun to explore the benefits of fusing multiple types of information.TLIO [Liu et al. 2020] combines displacement estimates with raw IMU measurements using an Extended Kalman Filter. In a similar vein, MobilePoser [Xu et al. 2024] combined IMU data with pose parameters. The importance of fusing information at different semantic levels has been demonstrated in other domains. Visual-inertial odometry systems benefit from combining low-level feature tracking with high-levelsemantic understanding [Campos et al. 2021], Similarly, human pose estimation has shown improved results when incorporating both bottom-up and top-down processing [Pavlakos et al. 2019].

[0007] Inertial Measurement Units (IMUs) are compact and low-cost sensors that measure linear acceleration and angular velocity. They are widely used in computer vision, robotics, and extended reality (XR) for motion tracking, by operating reliably under challenging conditions with a low power profile and lightweight form-factor. Both their small form factor and low power consumption make them a core sensing modality for wearable and mobile devices.

[0008] However, estimating position from IMU signals involves integrating noisy acceleration and angular velocity, which leads to cumulative drift. Visual-inertial odometry (VIO) methods mitigate this drift by fusing IMU signals with image features from cameras. While effective, visual methods degrade in high-speed motion or visually challenging conditions such as low light, motion blur, or lack of visual features. Moreover, cameras introduce power, privacy, and environmental limitations, making IMU-only odometry increasingly attractive for lightweight AR and wearable devices. Others have approached this problem in three main ways. First, visual-inertial odometry systems fuse cameras with IMUs to reduce drift, but these solutions depend on visual features, consume more power, and raise privacy concerns. Second, learning-based inertial odometry methods attempt to regress velocity or displacement directly from a single IMU stream using recurrent, convolutional, or equivariant neural networks. While effective, these systems remain vulnerable to accumulated drift, heading ambiguity, and poor generalization under complex human motion. Third, sparse-IMU motion-capture systems reconstruct body pose from multiple body-worn IMUs, but these typically require more sensors placed across the body and are less practical for lightweight consumer wearables. While some prior work also explores usingindividual auxiliary sensors such as barometers or magnetometers, these sensor types are not used in an integrated human-pose-grounded inertial odometry framework.SUMMARY

[0009] The following presents a simplified summary in order to provide a basic understanding of some aspects of the disclosure. The summary is not an extensive overview of the disclosure. It is neither intended to identify key or critical elements of the disclosure nor to delineate the scope of the disclosure. The following summary presents some concepts of the disclosure in a simplified form as a prelude to the description below.

[0010] The hierarchical motion-based enhanced inertial localization system introduces a hierarchical approach to indoor localization that transforms traditional IMU-based tracking from a sensor processing problem into a human motion understanding challenge and may incorporate one or more inertial odometry frameworks. The system leverages a multi-level architecture that incorporates biomechanical constraints, activity recognition, and environmental context through a modular framework that can adapt to varying levels of available information. By understanding human movement patterns at different levels of abstraction - from basic IMU measurements to full-body pose estimation and high-level activity recognition - the system provides stronger constraints for state estimation than traditional approaches. The key innovation lies in explicitly modeling uncertainties at each hierarchical level while automatically adapting to available information sources through a flexible loss function framework. Empirical studies demonstrate up to a 30% reduction in translation error compared to traditional methods, with significantly improved robustness in challenging real-world conditions where conventional approaches fail. This advancement represents afundamental shift in indoor localization technology, offering enhanced accuracy and adaptability across diverse deployment scenarios.

[0011] Inertial odometry (IO) using only IMUs may provide a lightweight solution for human motion tracking in augmented reality (AR) and / or with wearable devices. Current learning-based IO methods have improved the generalizability of inertial localization via large-scale pretraining on human motion datasets. However, these approaches remain prone to drift and noise because they lack crucial spatio-temporal human motion dynamics. In contrast, the present disclosure describes systems and methods to ground inertial odometry in human kinematics through a learned IMU inferred pose prior that promotes the propagation of physically consistent motion constraints. Integration of the pose prior to existing IO architectures reduces positional drift by up to thirty-six percent in the challenging Nymeria dataset (e.g., a dataset five times larger than was used with prior works). In some cases, improved long-term performance is showcased by developing a sensor-fusion framework that incorporates auxiliary signals from other lightweight sensors such as the magnetometer, barometer, and / or a secondary IMU already available on commercial AR glasses. With this fusion strategy, drift is reduced to forty-two percent, thus improving robustness and generalization across diverse motion conditions. Together, these results establish a new paradigm for inertial and lightweight odometry, unifying human motion kinematics with multimodal sensing, setting a new benchmark for accurate and robust camera-less human tracking.

[0012] IO using only IMUs may enable camera-free head tracking in head-mounted devices such as VR headsets, AR glasses, and earbuds. Prior IO methods operate in a gravity- aligned world frame, which is not yaw-equivariant. Yaw augmentation improves robustness but does not guarantee equivariance; learned equivariant layers achieve it but increase total pipeline FLOPs nearly by four times. While raw body-frame IMU inputs are inherently yaw-invariant, head tilt redistributes both gravity and motionsignals across device axes, making the representation tilt-variant. Body-Frame Heading Decomposition for Neural Inertial Odometry on Head-Mounted Devices (HeadlO) combines a body-frame heading decomposition with a Transformer and Bi- LSTM backbone for head-mounted inertial odometry. The decomposition achieves tilt invariance by separating gravity-aligned and horizontal components using only the tilt component of device orientation. Three head-mounted datasets were evaluated to find that HeadlO achieves 0.205m RTE-5s on Nymeria, the largest benchmark dataset — a 30% improvement over world-frame methods with learned equivariant layers, at nearly four times lower FLOPs. HeadlO achieves the lowest RTE and drift rate across all three datasets compared with prior state-of-the-art methods. This decomposition is lightweight, composes with any backbone, and is suitable for realtime deployment on head-mounted devices.

[0013] The hierarchical motion-based enhanced inertial localization system may integrate one or more inertial odometry systems, such as MARIO and / or HeadlO to perform to indoor localization operations.

[0014] These features, along with many others, are discussed in greater detail below.BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The present disclosure is described by way of example and not limited in the accompanying figures in which like reference numerals indicate similar elements and in which:

[0016] FIG. 1 shows an illustrative image of a hierarchical motion-based inertial localization system according to one or more aspects of the disclosure;

[0017] FIG. 2 shows a representative view of a framework for inertial odometry, according to aspects of the present disclosure;

[0018] FIG. 3 shows an overview of an inertial odometry framework for head-mounted and wearable devices, according to aspects of the present disclosure;

[0019] FIG. 4 shows a graph comparing barometric altitude measurements with ground truth altitude measurements over time, according to aspects of the present disclosure;

[0020] FIG. 5 shows a graph comparing magnetometer yaw measurements with ground truth yaw measurements over time, according to aspects of the present disclosure;

[0021] FIG. 6 shows trajectory visualizations comparing variants of an inertial odometry model on a dataset, according to aspects of the present disclosure;

[0022] FIG. 7 shows a table presenting dataset results comparing different inertial odometry models across sensor configurations, according to aspects of the present disclosure:

[0023] FIG. 8 shows a table presenting results from a dataset using a pretrained model, according to aspects of the present disclosure;

[0024] FIG. 9 shows cumulative distribution function graphs for performance metrics of inertial odometry models, according to aspects of the present disclosure;

[0025] FIG. 10 shows a table presenting results from a dataset evaluation, according to aspects of the present disclosure;

[0026] FIG. 11 shows a table presenting results with ablations comparing models with and without gravity compensation, according to aspects of the present disclosure;

[0027] FIG. 12 shows a table presenting results from fusion strategy ablations on a dataset, according to aspects of the present disclosure;

[0028] FIG. 13 shows a table comparing performance of a pose estimation method against other methods, according to aspects of the present disclosure;

[0029] FIGS. 14A-D show trajectory visualizations comparing model configurations for an inertial odometry architecture, according to aspects of the present disclosure;

[0030] FIG. 15 shows diagrams illustrating yaw invariance and tilt invariance concepts for body-frame heading decomposition, according to aspects of the present disclosure;

[0031] FIG. 16 shows a comparison between an un-tilted configuration and a tilted configuration for a head-mounted device coordinate frame system, according to aspects of the present disclosure;

[0032] FIG. 17 shows a block diagram of a velocity estimation network, according to aspects of the present disclosure;

[0033] FIG. 18 shows a comparison table of results with prior methods across head-mounted datasets, according to aspects of the present disclosure;

[0034] FIG. 20 shows a table presenting ablation results across multiple datasets for evaluating inertial odometry preprocessing configurations, according to aspects of the present disclosure;

[0035] FIG. 21 shows a table presenting ablation results across multiple datasets for evaluating inertial odometry preprocessing configurations, according to aspects of the present disclosure;

[0036] FIG. 22 shows a comparison of acceleration input distribution across three representations on a dataset, according to aspects of the present disclosure;

[0037] FIG. 23 shows a graph of validation mean squared error across different representations using a neural network backbone, according to aspects of the present disclosure;

[0038] FIG. 24 shows trajectory comparison plots of ground truth trajectories versus predicted trajectories from different inertial odometry methods, according to aspects of the present disclosure;

[0039] FIG. 25 shows a table presenting ablation study results for loss components on a dataset, according to aspects of the present disclosure;

[0040] FIG. 26 shows a table presenting window length ablation study results, according to aspects of the present disclosure; and

[0041] FIG. 27 shows an illustrative block diagram of a processor platform implementing at least a portion of the hierarchical motion-based inertial localization system, according to aspects of the present disclosure.DETAILED DESCRIPTION

[0042] In the following description of various illustrative embodiments, reference is made to the accompanying drawings, which form a part hereof, and in which is shown, by way of illustration, various embodiments in which aspects of the disclosure may be practiced. It is to be understood that other embodiments may be utilized, and structural and functional modifications may be made, without departing from the scope of the present disclosure.

[0043] It is noted that various connections between elements are discussed in the following description. It is noted that these connections are general and, unless specifiedotherwise, may be direct or indirect, wired or wireless, and that the specification is not intended to be limiting in this respect.

[0044] As used throughout this disclosure, computer-executable “software and data” can include one or more: algorithms, applications, application program interfaces (APIs), attachments, big data, daemons, emails, encryptions, databases, datasets, drivers, data structures, file systems or distributed file systems, firmware, graphical user interfaces, images, instructions, machine learning (e.g., supervised, semi-supervised, reinforcement, and unsupervised), middleware, modules, objects, operating systems, processes, protocols, programs, scripts, tools, and utilities. The computer-executable software and data is on tangible, computer-readable memory (local, in network- attached storage, or remote), can be stored in volatile or non-volatile memory, and can operate autonomously, on-demand, on a schedule, and / or spontaneously.

[0045] “Computer machines” can include one or more: general-purpose or special-purpose network-accessible administrative computers, clusters, computing devices, computing platforms, desktop computers, distributed systems, enterprise computers, laptop or notebook computers, primary node computers, nodes, personal computers, portable electronic devices, servers, node computers, smart devices, tablets, and / or workstations, which have one or more microprocessors or executors for executing or accessing the computer-executable software and data. References to computer machines and names of devices within this definition are used interchangeably in this specification and are not considered limiting or exclusive to only a specific type of device. Instead, references in this disclosure to computer machines and the like are to be interpreted broadly as understood by skilled artisans. Further, as used in this specification, computer machines also include all hardware and components typically contained therein such as, for example, processors, executors, cores, volatile and non-volatile memories, communication interfaces, etc.

[0046] Computer “networks" can include one or more local area networks (LANs), wide area networks (WANs), the Internet, wireless networks, digital subscriber line (DSL) networks, frame relay networks, asynchronous transfer mode (ATM) networks, virtual private networks (VPN), or any combination of the same. Networks also include associated “network equipment” such as access points, ethernet adaptors (physical and wireless), firewalls, hubs, modems, routers, and / or switches located inside the network and / or on its periphery, and software executing on the foregoing.

[0047] The above -described examples and arrangements are merely some examples of arrangements in which the systems described herein may be used. Various other arrangements employing aspects described herein may be used without departing from the innovative concepts described.

[0048] Humans do not move randomly through space. Instead, their movements are structured by biomechanics, guided by intent, and constrained by their environment. By understanding these higher-order patterns, indoor localization accuracy can be dramatically improved from those made from lower-order state estimation. At the most basic level, consider step detection from an IMU in a phone, in real-time. Traditional approaches look for characteristic acceleration patterns that might indicate a step. However, human locomotion is too complex to simply utilize this to model human movements. For example, the same sensor readings may indicate a step, a hop, or may be due to the user shifting their phone. By first understanding the user's fullbody pose, we can better interpret these acceleration patterns. Empirical studies show that this higher-order understanding of pose reduces translation error by approximately 30%. This improvement comes from the additional context and constraints that pose estimation provides, not just whether an acceleration spike occurred, but whether the user's body was in a particular configuration consistent with taking a step.

[0049] Building on this foundation, improvements can be incorporated from even higher-order understanding such as through activity recognition. Different activities, such as walking, running, climbing stairs, and the like, have characteristic motion patterns that inform how sensor data can be interpreted. For instance, knowing that a user is climbing stairs tells us their motion should have a vertical component and that their steps likely have a different pattern than level walking. This activity-level understanding provides another layer of constraints that further improves the estimation accuracy.

[0050] A crucial aspect of the hierarchical motion-based enhanced inertial localization system is its flexibility when incorporating external context. While the system can infer pose and activity from IMUs, it can also integrate externally provided information such as environmental maps to improve movement estimations. These maps provide known physical constraints, such as walls, doors, corridors, that help validate and / or refine the lower-level estimates. Further, the system is designed to accommodate any other forms of external context that might become available with additional modules, such as crowd dynamics, building usage patterns, and / or real-time occupancy data, to name a few.

[0051] This modularity makes this approach particularly powerful. For example, the system can be trained with any subset of higher-order or higher-dimensional information. Whether the system includes access to IMU-inferred step counts, lower-body pose, fullbody pose, activity recognition, or external environmental maps, or combinations thereof, the system can leverage whatever information is available to improve localization accuracy. This flexibility means the system is adaptable to different sensor configurations and available contextual information while maintaining its hierarchical benefits. The hierarchical motion-based enhanced inertial localization system can recognize and explicitly model the uncertainty at each level within theanalyzed hierarchy. This uncertainty, at the sensor level, includes measurement noise and integration drift. At the pose level, the uncertainty includes ambiguity when reconstructing 3D pose from sparse sensors. At the activity level, the uncertainty is found in classification and transition points. And at the contextual level, the uncertainty includes varying degrees of confidence in the external information. By modeling these uncertainties explicitly and showing how they interact across levels, hierarchical motion-based enhanced inertial localization system can be built as a more robust system that degrades gracefully when faced with challenging conditions.100521 This hierarchical approach represents a fundamental shift from treating localization as a sensor processing problem to viewing it as a problem with understanding human biomechanics and behaviors. Rather than fighting against sensor limitations with better filtering or integration techniques, the hierarchical motion-based enhanced inertial localization system leverages an understanding of human motion and behavior to provide stronger constraints on various estimates. Understanding this evolution provides crucial context for why a hierarchical approach represents a significant advancement in the field. The system's ability to incorporate any available higher- order information, whether inferred from IMU data or provided externally, makes it highly adaptable and future -proof. The result is a system that not only provides better accuracy under ideal conditions but also maintains performance better in challenging real-world scenarios where traditional approaches fail.Distinction of Prior Work

[0053] While each of the existing paradigms discussed above has advanced the field, resulting works have typically focused on improving individual components of the localization pipeline. What is missing is a unified framework that can (1) leverage higher-order motion understanding to constrain lower-level estimation (2) incorporate externalcontext when available, (3) handle varying levels of sensor and contextual information, and (4) model uncertainties across different levels of abstraction

[0054] This gap motivates our hierarchical approach to IMU-based localization, which provides a structured way to integrate information across multiple levels of abstraction while maintaining flexibility in what information is available.

[0055] This gap motivates this hierarchical approach to IMU-based localization, which provides a structured way to integrate information across multiple levels of abstraction while maintaining flexibility in what information is available.

[0056] FIG. 1 illustrates a block diagram of a hierarchical motion-based inertial localization system 100 according to one or more aspects of the present disclosure. The hierarchical motion-based inertial localization system 100 may transform inertial measurement unit (IMU) based tracking from a sensor processing problem into a human motion understanding challenge. The hierarchical motion-based inertial localization system 100 may leverage a multi-level architecture that incorporates biomechanical constraints, activity recognition, and environmental context through a modular framework that adapts to available information.

[0057] The hierarchical motion-based inertial localization system 100 may receive input from user devices 105, environmental sensors 107, and an environmental information repository 109. The user devices 105 may include one or more of a mobile phone, a smart watch, an audio device, an augmented reality headset, earbuds, or other wearable devices. In some cases, each user device of the user devices 105 may be located adjacent a different portion of a user's body. The user devices 105 may provide IMU data comprising acceleration measurements and angular velocity measurements to the hierarchical motion-based inertial localization system 100. In some cases, thehierarchical motion-based inertial localization system 100 may accept IMU measurements recorded over three channels corresponding to x, y, and z axes for a number of timesteps. For variable device configurations, measurements from multiple user devices 105 may be concatenated, and missing devices may be masked with zero values. The hierarchical motion-based inertial localization system 100 may support sampling rates of 100Hz or 200Hz.

[0058] The environmental sensors 107 may provide sensor data and measurements to the hierarchical motion-based inertial localization system 100 for processing. In some cases, the environmental sensors 107 may include barometers, magnetometers, secondary IMUs, or other auxiliary sensors. The environmental information repository 109 may supply contextual environmental data to support localization operations. In some cases, the environmental information repository 109 may provide environmental map information, positioning information, or geolocation information.

[0059] The hierarchical motion-based inertial localization system 100 includes an input processing engine 120 that receives and processes incoming data from the user devices 105, the environmental sensors 107, and the environmental information repository 109. The input processing engine 120 may process IMU measurements and prepare the measurements for downstream processing by other components of the hierarchical motion-based inertial localization system 100.

[0060] A temporal encoder 130 may be coupled to the input processing engine 120 and may encode temporal patterns from the processed input data. The temporal encoder 130 may decompose complex temporal patterns into interpretable seasonal and trend-cycle signals. In some cases, the temporal encoder 130 may output motion features that capture fundamental motion dynamics.

[0061] An environmental context engine 140 may process environmental context information received from the environmental sensors 107 and the environmental information repository 109. The environmental context engine 140 may encode external contexts, such as environmental constraints, as a discrete latent space and a semantic embedding map. In some cases, the environmental context engine 140 may process environmental maps that provide known physical constraints, such as walls, doors, and corridors, that help validate and refine lower-level estimates.

[0062] A decoder 150 may be coupled to the environmental context engine 140 and may decode the encoded representations to generate motion estimates. The decoder 150 may process motion features to classify activities and may employ self-attention across temporal dimensions and cross-attention with pose features to capture activity-pose relationships.

[0063] An activity recognition module 160 may classify user activities based on the processed sensor data. Different activities, such as walking, running, and climbing stairs, have characteristic motion patterns that inform how sensor data may be interpreted. For instance, knowing that a user is climbing stairs indicates that the user's motion should have a vertical component and that the user's steps likely have a different pattern than level walking. The activity recognition module 160 may provide activity -level understanding that provides another layer of constraints for estimation accuracy.

[0064] A full body pose engine 170 may estimate body pose parameters from the input measurements. In some cases, the full body pose engine 170 may estimate body model parameters comprising joint rotation parameters and global translation. The full body pose engine 170 may include a pose decoder that performs joint position estimation to determine three-dimensional joint positions and rotation estimation to map combined features to pose orientations. By understanding the user's full-body pose, thehierarchical motion-based inertial localization system 100 may better interpret acceleration patterns from the IMU data.

[0065] A geolocation module 180 may determine geographic location information based on the processed sensor data and environmental context. In some cases, the geolocation module 180 may utilize environmental map information, positioning information, or geolocation information from the environmental information repository 109.

[0066] An external context engine 190 may process external contextual factors that may influence motion estimation. In some cases, the external context engine 190 may process crowd dynamics, building usage patterns, and real-time occupancy data. The external context engine 190 may integrate externally provided information to improve movement estimations.

[0067] The hierarchical motion-based inertial localization system 100 outputs processed motion localization data to a motion localization consuming device 110. The motion localization consuming device 110 may receive the localization results generated by the hierarchical motion-based inertial localization system 100 and may utilize the motion tracking information for downstream applications.

[0068] The components of the hierarchical motion-based inertial localization system 100 may work together to transform raw sensor measurements into position and motion estimates by leveraging hierarchical processing across multiple levels of abstraction. The hierarchical motion-based inertial localization system 100 may process movement patterns at different levels of abstraction, from basic IMU measurements to full-body pose estimation and high-level activity recognition. In some cases, the hierarchical motion-based inertial localization system 100 may model uncertainties at each hierarchical level while automatically adapting to available information sources. At thesensor level, uncertainty may include measurement noise and integration drift. At the pose level, uncertainty may include ambiguity when reconstructing three-dimensional pose from sparse sensors. At the activity level, uncertainty may be found in classification and transition points. At the contextual level, uncertainty may include varying degrees of confidence in the external information.

[0069] The hierarchical motion-based inertial localization system 100 may be adaptable to different sensor configurations and available contextual information. In some cases, the hierarchical motion-based inertial localization system 100 may be trained with any subset of higher-order or higher-dimensional information. Whether the hierarchical motion-based inertial localization system 100 includes access to IMU-inferred step counts, lower-body pose, full-body pose, activity recognition, or external environmental maps, or combinations thereof, the hierarchical motion-based inertial localization system 100 may leverage whatever information is available to improve localization accuracy.System DescriptionInput Processing

[0070] The system accepts IMU measurements A = {Aa, Ag] G RA(2DxT), where Aa G RA(DxT) and Ag G RA(DxT) represent acceleration and angular velocity respectively, recorded over D = 3 channels (x, y, z axes) for T timesteps. For variable device configurations, measurements are concatenated into A G RA(2DxNxT) where N is the number of active devices (e.g., phones, watches, earbuds, AR headsets, etc.). Missing devices are masked with zero values. The system supports both 100Hz and 200Hz sampling rates, with T=100 or T=200 for 1-second windows.Hierarchical Architecture

[0071] The hierarchical motion-based enhanced inertial localization system employs a temporal encoder to decompose complex temporal patterns into interpretable seasonal and trend-cycle signals. The base encoder outputs motion features that capture fundamental motion dynamics.

[0072] The hierarchical motion-based enhanced inertial localization system estimates body model parameters 0 G RA72 (24 joints x 3 rotation parameters) and global translation t G RA3 which is the main indoor localization output. The pose decoder includes two stages:• Joint position estimation FJoint: M — > J where J G RA(24x3) represents 3D joint positions; and• Rotation estimation F_rot: [M, J] 0 mapping combined features to pose orientations.

[0073] A decoder processes motion features M to classify activities a G RAC where C is the number of activity classes. The decoder employs self-attention across temporal dimensions and cross-attention with pose features to capture activity -pose relationships. External contexts, such as environmental constraints, are encoded as a discrete latent space and a semantic embedding map.Training Strategy

[0074] The hierarchical motion-based enhanced inertial localization system employs a two- stage training approach by pre-training individual components using available ground truth for each level and end-to-end fine-tuning with combined loss.

[0075] An important innovation of this approach is the ability to operate with any subset of the following losses. When certain types of ground truth data or environmental information are unavailable, their corresponding losses are simply set to zero, and thesystem automatically adapts to use whatever information is available. This modularity allows the system to be deployed in various scenarios with different levels of available information while maintaining its hierarchical benefits.Always Present Losses

[0076] The hierarchical motion-based enhanced inertial localization system losses may include one or more of the following losses.Motion Estimation Losses:• Direct Velocity Loss'. Measures how accurately the system estimates instantaneous velocity by comparing predicted velocities with ground truth measurements• Drift Control Loss'. Penalizes the accumulation of position errors over time, with increasing penalties for errors that occur later in a sequence. This helps prevent the common problem of position estimates drifting far from true locations over time• Motion Smoothness Loss'. Ensures that predicted movements are physically plausible by penalizing sudden, unrealistic changes in velocity. This helps produce more natural motion estimatesOptional Losses:• Joint Position Loss: Measures how accurately the system estimates the 3D positions of body joints by comparing them with ground truth positions from motion capture data• Joint Rotation Loss: Evaluates the accuracy of estimated joint rotations, ensuring that limb orientations are correctly predicted. This is crucial for understanding the full body configuration.• Kinematic Chain Loss: Ensures that the predicted pose follows natural human body constraints - for example, maintaining proper bone lengths and joint angle limits• Pose Smoothness Loss'. Penalizes jittery or unrealistic changes in pose over time, ensuring smooth and natural motion of the full body• Activity Classification Loss: Measures how accurately the system identifies the current activity from a predefined set of possible activities (walking, running, climbing stairs, etc.)• Activity Transition Loss: Penalizes physically impossible transitions between activities, such as instant changes from sitting to running. This ensures predicted activity sequences are physically plausible• Activity-Pose Consistency Loss: Ensures that the predicted activities align with the estimated body poses - for example, a "sitting" activity should correspond to a seated pose• Environmental Barrier Loss: Prevents predicted trajectories from passing through walls, furniture, or other physical barriers defined in the environmental map• Environmental Semantic Loss: Ensures that predicted activities and movements make sense for the current location - for example, "desk work" activities should occur in office areas.• Path Plausibility) Loss: Encourages movements to follow common patterns for the given environment - for example, following hallways rather than randomly cutting through rooms.

[0077] In some cases, the motion-based enhanced inertial localization system may incorporate the Motion-Augmented Real-Time Multi-Sensor Inertial Odometry (MARIO) framework. FIG. 2 shows an illustrative representative view of the MARIO framework, which is an inertial odometry framework that builds on a single-IMU model, grounds motion in lower-body pose predicted by a pretrained PoseNet and enhances robustness through sensor fusion with a secondary IMU, a barometer, and amagnetometer. The MARIO framework utilizes head-mounted and wearable devices that improve trajectory estimation by combining two core ideas: (1) a learned humanmotion prior inferred from IMU data, and (2) multimodal fusion of lightweight onboard sensors already available on commercial AR glasses. The system takes as input time- synchronized signals from a primary head-mounted IMU, and when available, a secondary IMU, a magnetometer, and a barometer. The primary IMU provides accelerometer and gyroscope measurements over time. Raw signals are resampled to a common frequency and aligned in a common device-centric coordinate frame. In some cases, gravity is removed from the accelerometer using known or estimated device orientation.10078 J FIG. 3 illustrates an overview of a system 300 for inertial odometry according to one or more aspects of the present disclosure. The system 300 may implement a Motion- Augmented Real-Time Multi-Sensor Inertial Odometry (MARIO) framework that combines a learned human-motion prior with multimodal sensor fusion to estimate device trajectories. The system 300 may receive time-synchronized signals from multiple sensor inputs 310 and may generate a trajectory output 360 representing estimated device motion over time.

[0079] The sensor inputs 310 may include a first inertial measurement unit (IMU 1), a second inertial measurement unit (IMU 2), a barometer, a magnetometer, and a pose estimation module 370. The first IMU may serve as a primary IMU and may provide accelerometer and gyroscope measurements over time. The second IMU may be positioned at a different location on a head-mounted device than the first IMU. In some cases, the second IMU may improve observability of rotational and translational motion and may provide redundancy against sensor noise and bias. The barometer may provide altitude-related information derived from ambient pressure and may improve vertical motion estimation and long-term drift in elevation. In some cases, amoving-average filter may be applied to denoise both altitude and derivative vertical velocity from the barometer, yielding a one -dimensional velocity estimate at each timestamp. The magnetometer may measure a local magnetic field as a three- dimensional vector, which may be used to infer heading relative to magnetic north. The magnetometer may provide an absolute heading cue and may reduce yaw drift when magnetic conditions are sufficiently reliable.

[0080] Each of the sensor inputs 310 may be connected to corresponding encoders 320. The first IMU may connect to a first IMU encoder, the second IMU may connect to a second IMU encoder, the barometer may connect to a barometer encoder, the magnetometer may connect to a magnetometer encoder, and the pose estimation module 370 may connect to a pose encoder. Each encoder of the encoders 320 may comprise a temporal feature extractor, such as a causal temporal convolutional network. In some cases, each auxiliary stream from the second IMU, the magnetometer, and the barometer may be time-aligned to the first IMU before encoding. Each encoder may encode a respective stream to obtain per-step features.

[0081] The outputs from the first IMU encoder and the second IMU encoder may be directed to a concatenation module 330. The concatenation module 330 may combine encoded IMU features from the first IMU encoder and the second IMU encoder. The concatenated IMU features, along with encoded features from the barometer encoder, the magnetometer encoder, and the pose encoder, may be provided to a fusion module 340. The fusion module 340 may integrate multiple encoded feature streams through addition operations to produce a fused latent representation. In some cases, the fusion module 340 may comprise a multilayer perceptron that receives concatenated features and generates the fused latent representation.

[0082] The fused latent representation from the fusion module 340 may be provided to an inertial odometry network 350. The inertial odometry network 350 may process the combined sensor information to estimate motion states such as displacement, velocity, or trajectory increments. The inertial odometry network 350 may generate the trajectory output 360 representing the estimated device trajectory over time. In some cases, the inertial odometry network 350 may be backbone-agnostic and may be integrable with different existing inertial odometry architectures. The inertial odometry network 350 may comprise a preexisting inertial odometry architecture, including but not limited to TLIO, AirlO, EqNIO, or RoNIN-style models.

[0083] The pose estimation module 370 may receive IMU data and orientation information and may predict human body pose parameters. The pose estimation module 370 may comprise a lightweight network that learns a pose prior from a single IMU stream. The pose estimation module 370 may take linear accelerations, angular velocities, and orientations as input and may predict body pose in Skinned Multi-Person Linear (SMPL) format using a six-dimensional rotation representation.

[0084] The pose estimation module 370 may include a feature encoder 380 that processes IMU and orientation inputs. The feature encoder 380 may use a hybrid convolutional- recurrent architecture designed to capture both short-term inertial dynamics and long- range temporal dependencies. IMU signals may be encoded by a temporal convolutional neural network (CNN) that extracts local motion patterns. The feature encoder 380 may connect to a first Gated Recurrent Unit (GRU) network. The first GRU network may output joint positions represented by a skeletal figure. In some cases, the first GRU network may predict three-dimensional joint positions. The joint positions may be processed by a second GRU network to produce joint rotations. In some cases, the pose estimation module 370 may estimate only nine joint angles corresponding to pelvis and lower-limb joints, focusing on locomotion-related jointsthat dominate global motion while reducing model complexity. The pose estimation module 370 may output a 9x6D dimension pose vector at each timestamp, which may be fed into the fusion module 340.

[0085] A velocity decoder may process features from the pose estimation module 370 to generate joint velocities. The pose estimation module 370 may provide kinematic structure and temporal consistency information that serves as a motion prior for the inertial odometry framework. In some cases, the pose estimation module 370 may be trained using ground-truth motion capture or pseudo-ground-truth pose labels and may be supervised with losses on joint positions, joint rotations, and temporal velocity smoothness. The pose estimation module 370 may be trained using an L2 loss on joint positions and six-dimensional joint rotations and a Huber loss on joint-angle velocities to encourage temporal smoothness. In some cases, the pose estimation module 370 may be frozen during training of the inertial odometry network 350.

[0086] The first component of the MARIO framework is a pose-prior network (“PoseNet”).PoseNet receives an IMU sequence and predicts a lower-body human pose representation over time. In our implementation, the predicted pose is represented using SMPL-format lower-body joint rotations in a continuous 6D rotation parameterization. PoseNet is trained using ground-truth motion capture or pseudoground-truth pose labels and supervised with losses on joint positions, joint rotations, and temporal velocity smoothness. The purpose of PoseNet is not necessarily to output final motion capture results, but to provide a kinematically meaningful latent representation that encodes physically plausible human locomotion dynamics.

[0087] The second component of the MARIO framework is a multi-sensor fusion module.Auxiliary streams from the secondary IMU, magnetometer, and barometer are first time-aligned to the primary IMU. Each stream is encoded by a temporal featureextractor, such as a causal temporal convolutional network. The predicted pose features from PoseNet are also encoded and treated as an auxiliary motion prior. These feature streams are concatenated and passed through a learned fusion layer, such as a multilayer perceptron, to generate a fused latent representation. This fused representation is then provided to an inertial odometry backbone to estimate motion states such as displacement, velocity, or trajectory increments.

[0088] In some cases, the odometry backbone can be a preexisting inertial odometry architecture, including but not limited to TLIO, AirlO, EqNIO, or RoNIN-style models. The MARIO framework is backbone-agnostic in that the pose-prior features and auxiliary sensor features can be integrated into multiple existing inertial odometry systems without changing the fundamental objective of those systems.

[0089] The barometer provides altitude-related information derived from ambient pressure and can improve vertical motion estimation and long-term drift in elevation. The magnetometer provides an absolute heading cue relative to magnetic north and can reduce yaw drift when magnetic conditions are sufficiently reliable. The secondary IMU, positioned at a different location on the glasses frame, improves observability of rotational and translational motion and provides redundancy against sensor noise and bias.

[0090] An aspect of MARIO comprises grounding inertial odometry in human kinematics while simultaneously leveraging multiple lightweight non-visual sensors. Unlike conventional IMU-only odometry methods that rely on a single IMU and purely data- driven motion regression, MARIO constrains odometry with a learned human pose prior and enriches the state estimate with complementary heading, altitude, and multipoint inertial cues. This results in improved robustness, lower positional drift, and better generalization across users, activities, and environments.

[0091] The MARIO framework can be implemented in real time on wearable hardware or edge compute platforms and is suitable for camera-free motion tracking in AR / VR, wearable navigation, sports and fitness tracking, rehabilitation, and human-computer interaction systems.

[0092] Recently, learning-based inertial odometry (IO) methods have emerged that mitigate drift through data-driven motion priors learned directly from IMU signals [6, 15, 18, 27]. Despite these advances, most models rely on a single IMU and remain prone to drift and noise, limiting their robustness in tracking scenarios that involve complex human motions and diverse environments. Further, because IMU signals are tightly coupled to device placement and individual motion patterns, learned priors often fail to generalize across users, activities, and / or large motion footprints.

[0093] A MARIO-based system may ground inertial odometry in human kinematics through a learned IMU inferred pose prior that introduces explicit motion constraints and temporal consistency. PoseNet, which predicts full lower-body pose, is used in the SMPL body

[0013] format, from a single head IMU. The learned pose prior serves as a spatiotemporal kinematic anchor, to inject physically meaningful structure into motion estimation and to substantially reduce translation error and drift when integrated into existing IO architectures. To further augment performance, a Multi-Sensor Fusion Module is used to incorporate auxiliary signals from a magnetometer, barometer, and dual IMUs already available on commercial AR glasses such as Meta’s Aria [3]. These complementary cues provide accurate heading and elevation, to improve robustness and generalization across motion patterns.

[0094] The generality of the framework is demonstrated by integrating it with four state-of- the-art inertial odometry methods and conducting comprehensive evaluations on different datasets for comparison, such as the Nymeria (300 hrs, 297.87m avg.trajectory length) dataset

[0015] , the Aria Everyday Activities (7.3 hrs, 29.35m avg. trajectory length) dataset

[0014] , and the TLIO (60 hrs, 119.78m avg. trajectory length) dataset

[0012] , Quantitative and qualitative analyses show that this approach consistently reduces drift and lowers translation error across all benchmarks to confirm confirming its robustness and ability for generalization for human-centric motion tracking.

[0095] Aspects of a MARIO-based system (i) includes a human-pose-grounded inertial odometry (IO) framework that injects a learnt IMU inferred kinematics prior into the odometry model to improve stability and reduce drift, (2) showcases the effects of a sensor-fusion network that integrates barometer, magnetometer, and dual-IMU signals to provide robust odometry to substantially improve vertical and heading cues, and (3) integrates the modules into existing IO architectures (TLIO

[0012] , AirlO

[0018] , RoNIN

[0027] and EQNIO [6]) and benchmarks operation on Nymeria, Aria Everyday, and TLIO datasets to demonstrate consistent improvements across datasets and metrics.Related WorkInertial Odometry

[0096] Inertial odometry (IO) estimates a device’s 6-degree of freedom (DoF) pose (position and orientation) from IMU signals. Orientation is obtained by integrating gyroscope readings, while double integration of linear acceleration yields position. However, noise and bias in low-cost IMUs accumulate during integration, which causes drift over time. Heuristic methods such as pedestrian dead reckoning (PDR) exploit motion regularities (e.g., step detection and stride estimation) but rely on hand-tuned assumptions and degrade in unconstrained motion.

[0097] Learning-based IO replaces such heuristics with data-driven motion priors that infer displacement or velocity directly from IMU sequences. RIDI

[0026] regresses shortterm

[0098] velocity to correct low-frequency drift, while lONet [2] and RoNIN

[0027] integrate learned velocity estimates for accurate 2-dimensional (2D) trajectory reconstruction. RoNIN further introduces a heading-agnostic coordinate frame (HACF) aligning gravity with the vertical axis. TLIO

[0012] extends this by combining a learned displacement estimation with an Extended Kalman Filter (EKF) to jointly refine position, orientation, and bias. IDOL

[0020] explicitly estimates orientation from magnetometer readings and regresses translation with a bi-directional LSTM, improving robustness and reducing long-term drift.

[0099] To improve generalization, recent works incorporate structural and equivariant priors.AirlO

[0018] shows that retaining IMU data in the body frame, rather than transforming it to the global frame, improves drone odometry. RIO [1] leverages rotational equivariance as a self-supervised signal, and EqNIO [6] learns canonical displacement priors in a gravity-aligned frame for rotation consistency. TartanIMU

[0034] extends IO to a foundation model trained on 100+ hours of IMU data across cars, drones, robots, and humans, achieving cross-domain generalization through low-rank fine-tuning and online adaptation. M2EIT

[0011] further introduces a mixture -of-experts framework that fuses spatial, temporal, and frequency features for state-of-the-art robustness across motion domains.

[0100] Compared to prior works that rely on a single IMU, MARIO-based system first introduces a human-pose-guided prior within a multi-stage framework, grounding motion estimation in body dynamics to improve both accuracy and robustness. The MARIO-based system further leverages multi-sensor fusion on AR glasses —combining multiple IMUs and auxiliary sensors (barometer, magnetometer) already available on existing devices [3] — to further mitigate long-term instability and bias drift. Body Motion Capture for 1MU Devices

[0101] Inertial measurement units (IMUs) have become an attractive sensing modality for motion capture due to their compact size, low power consumption, and portability. Commercial systems such as Xsens

[0024] employ many high-grade IMUs (typically 17) for high-fidelity full-body tracking but remain intrusive and impractical for everyday use.

[0102] To improve practicality, recent research has explored reconstructing full-body pose from fewer sensors. SIP

[0022] demonstrated that six Xsens IMUs can reconstruct fullbody motion using offline optimization. Deep Inertial Poser (DIP) [5] introduced a bi-directional LSTM model for pose estimation from six IMUs, while TransPose

[0029] extended this to estimate global translation via a multi-stage architecture. TIP [7] applied Transformers for non-planar motion, and PIP

[0030] , PNP

[0031] , and GlobalPose

[0032] incorporated physics-based optimization for more plausible motion reconstruction.

[0103] Building on these advances, recent work leverages IMUs embedded in everyday devices. IMUPoser

[0017] demonstrated that arbitrary combinations of IMUs in earbuds, smartwatches, and smartphones can reconstruct full-body poses using masking strategies for missing sensors, while MobilePoser

[0025] extended this approach with a multi-stage framework that jointly estimates pose and global translation. DiffusionPoser

[0021] further employed a diffusion-based model to inpaint missing signals during denoising, supporting more flexible sensor placements.

[0104] These works pave the way for reconstructing full-body motion from sparse IMU configurations, but they often rely on multiple on-body IMUs. A general human-poseprior may be learned from a single IMU which, when integrated into inertial odometry models, significantly reduces error and drift.Sensor Fusion for IMU Motion Tracking

[0105] IMUs are compact and low-cost but are sensitive to noise, leading to drift over time

[0036] . To mitigate these limitations, extensive research has explored complementing IMUs with other sensors to improve stability and accuracy. The most common is visual-inertial odometry (VIO), which jointly optimizes motion estimates from IMU and camera observations. LiDAR - inertial odometry (LIO) follows a similar principle, using geometric depth constraints from LiDAR scans to improve robustness in low-light or textureless environments. Comprehensive surveys of

[0106] VIO and LIO methods [4, 8] may be reviewed. Also, MINS [9] proposes a tightly coupled multisensor-aided inertial navigation system that fuses IMU, wheel encoders, cameras, LiDAR, and GNSS to address asynchronous measurements.

[0107] Cameras and LiDARs are power-intensive and often impractical for lightweight platforms such as AR glasses. Recent work instead incorporates non-visual modalities — magnetometers, barometers, ultrasound, and ToF sensors — for efficient motion tracking. MagShield

[0019] uses magnetic cues to detect disturbances and correct orientation errors. BaroPoser

[0033] exploits barometric pressure on wearables to recover global translation on nonflat terrain. UltraPoser

[0010] leverages ultrasound via built-in speakers and microphones to expand body-area sensing, while ToF-IP

[0028] integrates Time-of- Flight depth with sparse IMUs to constrain geometry and reduce drift. However, there is little work exploring the integration of easily accessible sensors, such as barometers and magnetometers, to enhance inertial odometry.Method

[0108] The MARIO system processes the inertial odometry (IO) problem as estimating a device trajectory, T, from a time series of IMU measurements comprising linear acceleration and angular velocity. Existing approaches differ in how they reconstruct motion from these signals.

[0109] By explicitly introducing a human motion prior — learned from IMU signals — and integrating complementary sensing modalities, inertial tracking can be made more stable, accurate, and generalizable. As illustrated in FIG. 3, the MARIO framework includes two main components: (1) PoseNet, which learns a human motion prior from a single IMU to provide kinematic structure and temporal consistency; and (2) a MultiSensor Fusion Module, which combines magnetometer, barometer, and secondary-IMU signals through a learned mid-level fusion strategy to enhance robustness and long-term stability.

[0110] To validate the approach, the modules were integrated into four representative IO architectures. AirlO

[0018] predicts body-frame velocity and transforms the body-frame velocity to global coordinates using gyroscope-derived rotations before integration. TLIO

[0012] instead directly regresses 3D positional displacements over fixed 1 -second windows using a ResNet architecture with gravity-aligned input. EqNIO[6] extends this displacement-based approach by learning in a canonical 0(2) equivariant frame for improved rotation consistency. Finally, RoNIN-LSTM

[0027] uses bi-directional LSTM for 2D velocity, which was modified to output 3D velocity vectors for fair comparison.PoseNet

[0111] PoseNet is a lightweight network that learns pose prior from a single IMU stream. The network takes linear accelerations, angular velocities, and orientations as input and predicts body pose in SMPL format using a 6D rotation representation

[0035] .

[0112] Following AirIO

[0018] architecture to extract IMU feature information, PoseNet uses a hybrid convolutional - recurrent architecture designed to capture both short-term inertial dynamics and long-range temporal dependencies. IMU signals are first encoded by a temporal CNN that extracts local motion patterns, followed by stacked GRUs that sequentially predict joint positions and then rotations in the SMPL format. Only nine joint angles corresponding to the pelvis and lower limbs are estimated, focusing on locomotion-related joints that dominate global motion while reducing model complexity. Estimating arm or hand motion from head-mounted IMUs is inherently under-constrained and thus excluded. Therefore, PoseNet will give a 9 X 6D dimension pose vector each timestamp, which will be fed into the fusion module describe in next subsection. The pose is trained prior using anloss on joint positions and 6D joint rotations, and a Huber loss on joint-angle velocities to encourage temporal smoothness. Let J and J denote the ground-truth and predicted joint positions, and 9 and 9 the corresponding 6D rotations:£e=|| j-J Hi, Lang=11 e-e Hi

[0113] Given ground-truth velocity v and predicted velocity v, the velocity loss defined by a Huber loss function is computed:if | v — v |< Sotherwise

[0114] with 6 = 0.005. The total training objective is~ £pos- pos T ^-ang ang + ^vel^vel

[0115] where Xpos= 0.05, X = 0.05, and Xvel= l.

[0116] PoseNet is trained on the Nymeria dataset

[0015] , which contains over 300 hours of synchronized IMU and ground-truth pose data collected from head-mounted devices across diverse activities and environments.Multi-Sensor Fusion Module

[0117] Predicting motion from IMU signals is inherently prone to drift due to sensor noise and bias. To address this limitation, a Multi-Sensor Fusion Module combines complementary signals from a barometer, a magnetometer, and a secondary IMU that are already available on AR glasses [3]. These additional modalities supply grounding for elevation and heading and provide data redundancy that improves robustness and long-term stability.

[0118] FIG. 3 shows an illustrative overview of the MARIO inertial odometry framework.Here, a PoseNet is learned that takes IMU measurements and orientations as input and predicts human pose (represented by joint rotations). We encode multiple sensor signals — secondary IMU, barometer, magnetometer, and the pose estimated by the frozen PoseNet — are encoded to concatenate their features through a fusion layer, and then the fused representation is fed to an existing inertial odometry model to predict the trajectory.

[0119] Barometer A barometer measures ambient air pressure as a scalar time series in pascals.Pressure p (Pa) is converted to altitude h (m) using the hypsometric relation under the International Standard Atmosphere (ISA) model, assuming sea-level pressure Po = 101,325 Pa, temperature To, and lapse rate L. The altitude may be computed according to:

[0120] where R is the specific gas constant for dry air and g0is standard gravity. The altitude signal is generally smooth but may drift slowly due to weather or indoor pressure changes. To reduce noise, a moving-average filter is applied to denoise both the altitude and its derivative vertical velocity, yielding a ID velocity estimate at each timestamp that is later used in the fusion module. FIG. 4 shows that barometric altitude closely follows ground-truth, indicating that pressure provides a strong cue for estimating altitude changes. For example, FIG. 4 shows an illustrative visualization of altitude from the barometer compared with the ground-troth altitude. This demonstrates that the barometer provides valuable altitude information.

[0121] Magnetometer A magnetometer measures the local magnetic field as a 3D vector, which can be used to infer heading relative to magnetic north. Despite susceptibility to magnetic disturbances indoors, it provides a valuable absolute orientation cue when calibrated.

[0122] The yaw angle is computed by combining the magnetometer reading m with gravity g estimated from the IMU. After normalization, m = m / ||m|| and g = g / ||g||, tilt compensation may remove a vertical component, tilt compensation removes the vertical component:

[0123] and the yaw is computed as

[0124] This ID yaw signal serves as the magnetometer input to the fusion module. For example, FIG. 5 compares the derived yaw with ground-truth, showing alignment anddemonstrating that the magnetometer provides a useful heading cue. FIG. 5 shows an illustrative visualization of magnetometer data compared with ground-truth yaw orientation. The magnetometer signal roughly tracks the ground-truth, demonstrating its utility for constraining heading drift in inertial odometry.

[0125] Secondary' IMU An accelerometer on a moving rigid body measures not only linear acceleration but also rotation-induced terms. For a body with linear acceleration ao, angular velocity co, and angular acceleration a, an accelerometer at position r records: a = ao + a x r + io x (w x r) + b + n

[0126] where b and n denote bias and noise. Adding a second, spatially separated IMU improves the observability of rotational motion and helps disentangle translational acceleration from rotation-induced effects. The dual-IMU configuration also provides redundancy that reduces the impact of sensor bias and noise. In the Aria glasses used in the Nymeria

[0015] and Aria Everyday Activities datasets

[0014] , the primary IMU is located on the right temple and the secondary IMU on the left. Similar to the primary IMU, the secondary IMU provides a 6D (accelerometer + gyroscope) measurement vector at each timestamp for sensor fusion.Feature Fusion Strategy

[0127] Mid-level feature fusion is tailored to the latent state of inertial odometry (IO). Each auxiliary stream (magnetometer, barometer, secondary IMU) is first time-aligned to the primary IMU. Each stream is then encoded with a small causal temporal CNN to obtain per-step features fmaS(t), fbarof), fmu f) and also for fpose(t). The primary IMU (accelerometer + gyroscope) is passed through the same architecture to yield fimu(t). At time t, we concatenate:

[0128] and feed^«j,on(t) to a MLP fusion layer followed by existing inertial odometry models to estimate the motion state.ExperimentsDataset

[0129] This approach was validated across three different datasets, Nymeria

[0015] , Aria Everyday Activities

[0014] and TLIO

[0012] , The Nymeria dataset serves as the primary training source, partitioned with an 80 / 10 / 10 split for train / validation / test. For additional evaluation on household activities, the Aria Everyday Activities (AEA) was used as a test dataset with a model pretrained on Nymeria. For baseline comparisons, the TLIO dataset was incorporated following the original repository’s train / validation / test split

[0012] .

[0130] Nymeria. Dataset The Nymeria dataset

[0015] comprises 1200 sequences of 15-20 minutes duration from 264 subjects in 50 varied indoor and outdoor environments, totaling 300 hours and 400km of movement. The data captures 20 different scenarios including athletic activities, meals, work tasks, outdoor excursions, and cycling. Project Aria glasses record synchronized multimodal signals: dual IMUs at 800Hz and 1kHz, magnetometer at 10Hz, and barometer at 50Hz. Ground-truth full-body pose is captured using an Xsens MVN Link suit with 17 inertial units at 240 Hz. These Xsens poses were converted into the SMPL format to supervise PoseNet. The Project Aria MPS system provides the 3D trajectories at 1 kHz used as ground-truth for inertial odometry.

[0131] Aria Everyday Activities Dataset The AEA dataset

[0014] contains 143 recording sessions across five distinct indoor environments, totaling 7.3 hours. Activities include typical household routines: meal preparation, tidying, eating, and social interaction. Recording setup and ground-truth systems match specifications from Nymeria dataset.

[0132] TLIO Dataset The TLIO dataset

[0012] features approximately 60 hours of walking motion recorded via head-worn hardware with a Bosch BMI055 inertial sensor. The collection focuses on pedestrian movement patterns within interior spaces. Unlike Nymeria and AEA, TLIO provides only IMU measurements without additional sensing modalities, with ground-truth trajectories derived from Visual Inertial Odometry.

[0133] Coordinate Frame Standardization All recordings are converted to a Central Pupil Frame (CPF) for consistent training and evaluation. Raw SLAM data provides positions and orientations in an arbitrary world frame, while IMU measurements and velocities are in the device’s body frame. This mismatch prevents proper velocity integration during training. This is resolved by choosing a world frame that is expressed in CPF (+X left, +Y up, +Z forward).Implementation Details

[0134] All experiments were conducted on NVIDIA GeForce RTX 4090 and A40 GPUs.AirlO uses a sliding window of 1,000 samples (stride 100), while EqNIO, TLIO, and RoNINLSTM use 200 samples with stride 10 for TLIO / Aria Everyday datasets and stride 100 for Nymeria. All signals are resampled to 200 Hz; barometer and magnetometer readings are interpolated to match the IMU rate. Unless otherwise specified, models are trained on Nymeria’s training split, evaluated on its test split, thenassessed on Aria Everyday for cross-dataset generalization. The pose prior is trained exclusively on Nymeria and frozen during training and evaluation.

[0135] A learning rate of 5 X 10-4is used for AirlO and 1 X 10-4for TLIO. During training, the models are conditioned on ground-truth orientations, whereas at test time we use orientations inferred from the gyroscope. For TLIO, EqNIO, and RoNIN-LSTM, the covariance branch are trained from the first iteration, which leads to more stable convergence on some datasets, such as Nymeria. For gravity compensation in the raw IMU accelerometers, we subtract gravity using the known device orientation.Evaluation Metrics

[0136] Performance is reported using Absolute Trajectory Error (ATE), Relative Translational Error at 1 s and 5 s (RTE-ls / RTE-5s), and a segment-wise drift rate (“Drifting”). ATE is computed as the root-mean-square error between estimated and ground-truth positions after temporal synchronization and rigid alignment:

[0137] To better analyze ATE, it is decomposed into horizontal and vertical components, where the horizontal error is measured in 2D, capturing motion on the ground plane, and the vertical error is measured in ID, capturing only upward and downward motion. RTE- A t measures the mean error of relative displacements over sliding windows of duration

[0138] At e { 1, 5} s:Results

[0139] The results Nymeria and Aria datasets are presented integrating the PoseNet and Multi-Sensor Fusion Module into four IO architectures: AirlO

[0018] , TLIO

[0012] , EqNIO [6], and RoNIN-LSTM

[0027] . (+Pose) is used to denote models with the pose prior and (+A11) for models using both pose and all sensor modalities.

[0140] Table 1 of FIG. 7 and Table 2 of FIG. 8 present the main results on Nymeria and Aria Everyday datasets. On Nymeria, the full system (+A11) achieves consistent improvements across all architectures: AirlO shows 32% ATE reduction (6.85m —>■ 4.64m) and 39% drift reduction (3.56% — > 2.16%), while TLIO demonstrates even larger gains with 44% ATE improvement (10.19m — > 5.73m) and 44% drift reduction (6.46% — > 3.64%). EqNIO and RoNIN-LSTM follow similar trends with 41% and 35% ATE reductions respectively. Adding PoseNet alone delivers substantial benefits, reducing RTE-5s by 11% for AirlO and 13% for TLIO. Individual sensor modalities contribute complementary cues, with the barometer particularly effective for vertical stability. On Aria (Table 2), models pretrained on Nymeria generalize effectively without fine-tuning, maintaining strong performance with 29-32% ATE improvements. The pose prior shows robust cross-domain transfer, reducing drift by up to 53% on Aria. FIG. 6 shows this progressive improvement with three sample sequences across TLIO variants. Further, the AirlO model with full sensor fusion and pose prior (+A11) achieves 133.6M FLOPs per inference and 315 FPS on NVIDIA A40 GPU, enabling real-time deployment. For example, FIG. 6 shows illustrative Trajectory visualizations of TLIO, TLIO+Pose, and TLIO+A11 on Nymeria dataset. TLIO+Pose improves over TLIO, and TLIO+A11 further reduces drift and tracking errors.

[0141] FIG. 9 shows RTE-5s CDF on the Nymeria dataset for AirlO, TLIO, EqNIO, and RoNIN-LSTM. The cumulative distribution of 5-second relative trajectory error isshown for all four base models. In FIG. 9, we demonstrate the cumulative distribution function (CDF) of RTE-5s is demonstrated for AirlO, TLIO, EqNIO, and RoNIN- LSTM on the Nymeria dataset. Adding the pose prior consistently lowers RTE-5s for all models, and incorporating all sensors together with the pose prior further reduces RTE-5s across the entire Nymeria test set.

[0142] Table 1 of FIG. 7 shows Nymeria dataset results with metrics: ATE (m; horizontal H, vertical V), RTE-5s (m), RTE- Is (m). Drift (%), where lower is better. Table 2 of FIG.8 shows Aria dataset results using pretrained model from Nymeria dataset with Metrics: ATE (m; horizontal H, vertical V), RTE-5s (m), RTE-1 s (m), Drift (%), where lower is better.Ablation Studies

[0143] Pose Prior Ablation Table 3 of FIG. 10 shows TLIO dataset results. Metrics reported:ATE (m; horizontal H, vertical V), RTE-5s (m), RTE- Is (m), and % Drifting (lower is better). RoNIN refers to RoNIN-LSTM variant. In Table 3, the pose prior’s effectiveness is evaluated on the TLIO dataset. Four baseline models — AirlO, TLIO, EqNIO, and RONINLSTM — are evaluated using PoseNet pretrained on the Nymeria dataset. The results show that incorporating PoseNet consistently improves ATE by 2-16% across all models, indicating that the pose prior generalizes well across different datasets.

[0144] Gravity Removal Ablation To investigate whether the model benefits from gravity- removed linear acceleration versus raw acceleration, an ablation study was conducted and is shown in Table 4 of FIG. IL, which shows Nymeria results with “(+ all)” ablations (with / without gravity). Metrics: ATE (m), RTE-5 s (m), RTE- Is (m), and %Drifting (lower is better). Models trained with (with g) and without (without g)gravitational components are compared in the acceleration data. Results show that removing gravity consistently improves performance across both AirlO and TLIO with all sensors fused. For AirlO, gravity removal reduces ATE from 6.121m to 4.641m and drifting from 3.58% to 2.16%. Similarly, TLIO shows improvements with ATE decreasing from 6.613m to 5.734m when gravity is removed. These results suggest that linear acceleration is more informative than raw acceleration for head tracking tasks.

[0145] Fusion Strategy Ablation Table 5 of FIG.12 shows Nymeria results with fusion strategy ablations. Metrics: ATE (m), RTE-5s (m), RTE-ls (m), and % Drifting (lower is better). Table 5 compares the sensor-fusion strategy with alternative fusion modalities. We evaluate Direct early-stage concatenation, where pose joint parameters are concatenated with accelerometer and gyroscope measurements, and cross-attention, where pose parameters are first encoded by a pose encoder and the resulting pose features attend to all IMU features are evaluated via a crossattention module. The experiments indicate that this approach achieves better ATE, RTE, and drift performance in most cases.Additional Features

[0146] While the MARIO framework shows clear benefits from integrating human-pose priors and multi-sensor fusion for inertial odometry, several additional features may also be implemented to improve performance. PoseNet is currently trained only on the Nymeria dataset, which may limit generalization, therefore incorporating larger and more diverse motion datasets such as AMASS

[0016] may improve robustness. Magnetometers remain vulnerable to indoor disturbances, and barometric altitude is sensitive to pressure drift and weather changes. MARIO may further incorporatelearned reliability gating for corrupted sensors, self-calibration for misalignment, and further optimization for real-time deployment on resource-constrained wearables.MARIO Summary

[0147] MARIO is an IMU-inferred pose prior augmented inertial odometry framework that can be applied to off-the-shelf IO models to substantially improve motion estimates by grounding them in human kinematics is introduced herein. Beyond the standard IMU, the MARIO framework is the first to explore incorporating additional lightweight sensing modalities — a barometer, a magnetometer, and a second on-glasses IMU — and combined them with the pose prior through an efficient multimodal fusion strategy. The resulting system plugs into IMU-based backbones and predicts trajectories with at most 42% improved accuracy that may be further incorporated into a hierarchical motion-based inertial localization system alone, or in combination with one or more other frameworks.MARIO Supplementary MaterialPoseNet Evaluation Experiment

[0148] To evaluate the quality of our PoseNet, PoseNet was compared with Ego4o

[0023] , which includes an Ego4o-IMU model that takes head IMU as input, and an Ego4o model that additionally takes egocentric RGB images and motion description text as input. The Mean Per-Joint Position Error (MPJPE) and Procrustes -Aligned Mean PerJoint Position Error (PA-MPJPE) of Ego4o-IMU are reported and are compared with those of our PoseNet. Both models are evaluated on the Nymeria dataset. Since Ego4o was trained and evaluated on its own curated splits, the numbers may not be directly comparable. They are included to contextualize the scale of pose accuracy achievable with richer modalities. Table 6 of FIG. 13 shows a comparison of PoseNet,Ego4o-IMU

[0023] and Ego4o

[0023] on MPJPE, PA-MPJPE. From Table 6, the method achieves better MPJPE and AMPJPE (in mm) than Ego4o-IMU and comparable results to Ego4o, despite Ego4o using more modalities (IMU + RGB + language).CPF Coordinate Details

[0149] CPF (Central Pupil Frame) is a rigid device-centric coordinate system with +X left, +Y up, +Z forward. Raw device data is transformed into the CPF coordinate frame. At a chosen reference time, the device’s CPF forward direction is projected onto the horizontal plane to define the forward axis of the new world frame, and the vertical axis is aligned with gravity. This creates a unified, gravity-aligned frame that supports geometrically consistent transformations between the device body frame and the world frame.

[0150] Stage 1: Device^ CPF. Using Aria calibration:WPF ^CPF^-device^device^worldt-CPF (4) worlds device (d)^device^ PF

[0151] Stage 2: CPF — CPF-World (gravity-aligned, consistent heading). Choose a reference time to (30 s after sequence start to allow SEAM to settle). Compute the CPF forward direction in world coordinates:

[0152] Project to the horizontal plane and normalize:

[0153] Define CPF-world axes (expressed in world coordinates):Ycpfw = [0,0,1]^•cpfw ^hor^cpfw Ncpfw "-cpfw

[0154] Assemble the rotation with columns as the axes in (9):world<-cpfw [^cpfwYcpfw^cpfw]

[0155] Transform positions by rotation and centering at pref:Additional Trajectory Visualization

[0156] FIGS. 14A-D shows the trajectory predictions alongside the ground truth for AirlO, TLIO, EqNIO, and RoNIN-LSTM on one sequence from the Nymeria dataset. In FIGS. 14A-D, the predicted trajectories of the base model, the base model + pose, and the base model + all are visualized for four different methods — AirlO, TLIO, EqNIO, and RoNIN-LSTM — on the Nymeria dataset. For each model, +pose visibly reduces drift relative to the base model, and -i-all further tightens the trajectory around ground truth.HeadlO: Body-Frame Heading Decomposition for Neural Inertial Odometry on Head- Mounted Devices

[0157] HeadlO combines a body-frame heading decomposition with a Transformer and Bi- LSTM backbone for head-mounted inertial odometry. The decomposition is a lightweight analytical preprocessing step that separates body-frame IMU signals into gravity-aligned and horizontal components using only the tilt component of device orientation. This achieves tilt invariance and heading equivariance without learned equivariant layers, data augmentation, or architectural constraints, and composes with any backbone. This decomposition is paired with a Transformer that leverages temporal self-attention over the resulting orientation-invariant input features.

[0158] As mentioned above, IMUs are compact, low-power sensors widely used for motion tracking in augmented reality, robotics, and wearable devices. Estimating position from IMU signals requires integrating noisy acceleration and angular velocity, which leads to cumulative drift over time. Visual-inertial methods [12a, 16a, 18a, 23a] mitigate this drift by fusing cameras with IMUs, but cameras introduce power, privacy, and environmental constraints that make IMU-only odometry attractive for lightweight head-mounted devices such as AR / VR head- sets and smart glasses, and earbuds.

[0159] Learning-based inertial odometry (IO) methods replace hand-tuned heuristics with data-driven motion priors [8a, 10a, 13a, 26a, 28a]. These methods predominantly adopt a gravity-aligned world-frame representation, where raw IMU measurements are rotated into a stabilized coordinate frame before being fed to the network. This convention was well-motivated for handheld phones, where non- rigid mounting and varied carrying positions make body-frame signals discontinuous across usage contexts. However, for rigidly head-mounted IMUs, world- frame projection discards the implicit gravity signal j g — an encoding of device attitude that AirlO [19a] showed is critical for learning, yielding 66.7% ATE improvement on drone datasets by preserving it in the body frame.

[0160] Head-mounted devices present a distinct challenge that neither world-frame nor raw body-frame approaches fully address. Like drones, they have rigidly mounted IMUs with rich 3D dynamics, so body-frame representation is beneficial. Unlike drones, whose tilt is mechanically coupled to locomotion heading, humans can move in any direction regardless of where they are looking — heading is a genuinely unobservable nuisance variable. Prior world-frame methods [10a, 13a] mitigate heading dependence — via yaw augmentation [13a] or learned equivariant layers [10a] — but discard body-frame dynamics. Raw body-frame approaches preserve those dynamics but leave gravity entangled across device axes, producing different feature patterns for the same physical action at different head tilts.

[0161] Through a systematic ablation across coordinate frames, heading strategies, gravity settings, and backbone architectures, we find that input representation critically interacts with backbone choice. A Transformer on raw body-frame input performs worse than a ResNet, but with this decomposition it achieves the best results across all configurations. This suggests that the Transformer’s capacity amplifies representation quality rather than compensating for its deficiencies.

[0162] HeadlO features include: (1) identifying the distinct representational requirements of head-mounted de- vices as an IO platform, where body-frame dynamics preservation and heading factorization must be addressed jointly; (2) a body-frame heading decomposition that disentangles gravity, forward, and lateral components via analytical preprocessing, without learned equivariant architectures; (3) an ability to conduct ablations across two coordinate frames, two heading strategies, two gravity settings, and two backbone architectures, showing that the pre-processing benefits both existing methods and the Transformer architecture; (4) HeadlO achieves state-of-the-art results on all three head-mounted datasets, with the lowest RTE and drift rate across allbenchmarks, at nearly four times lower FLOPs than world frame learned equivariant methods.

[0163] FIG. 15 shows body-frame heading decomposition, (a) Body-frame IMU inputs are heading-invariant: the same walking motion produces identical IMU readings regardless of which direction the user faces and (b) the heading decomposition additionally achieves tilt invariance: as the head pitches or rolls, the gravity-aligned basis adapts so that the forward, lateral, and vertical components remain consistent across head orientations.Related WorkInertial Odometry for Human Motion

[0164] Classical methods that use IMUs for human navigation are typically based on strapdown inertial navigation systems (SINS). Traditional SINS methods [22a, 24a] estimate motion through double integration of inertial measurements. However, the accumulated integration error leads to rapid drift in practice. To mitigate this issue, Pedestrian Dead Reckoning (PDR) methods introduce domain-specific constraints based on human walking patterns. Representative techniques include Zero- Velocity Updates (ZUPT) [6a], Zero Angular Rate Updates (ZARU) [20a], and heuristic drift reduction (HDR) [3a], which detects straight-line walking segments to estimate and compensate for gyroscope bias.

[0165] More recently, neural inertial odometry approaches have been proposed to replace hand-crafted heuristics with learned motion priors, directly regressing displacement or velocity from sequences of IMU measurements. RIDI [26a] first identified that bodyframe IMU signals become discontinuous when a hand- held phone changes orientation and proposed regressing velocity in a gravity- stabilized frame to correct low-frequencydrift. RoNIN [8a] formalized this with a Heading-Agnostic Coordinate Frame (HACF), gravity-aligning IMU measurements and augmenting with random yaw rotations. TLIO [13a] extended this convention to head-mounted devices, predicting displacement in a global gravity-aligned frame. RIO [29a] proposed a selfsupervision objective that enforces rotation equivariance in the network to alleviate yaw unobservability. EqNIO [10a] addressed the residual yaw ambiguity via learned Vector Neuron layers that achieve 0(2) equivariant canonicalization but retained gravity-alignment as a foundational assumption.

[0166] Most of the above work assumes gravity-aligned world frame. This gravity-aligned world-frame convention was well-motivated for phones, where non-rigid mounting, arm swing, and pocket dynamics make body-frame signals genuinely discontinuous across usage contexts. For rigidly head-mounted IMUs, however, world-frame projection discards the implicit gravity signal R^g and renders displacement targets heading -dependent. None of these methods re-evaluate whether world-frame representation is optimal when the IMU is rigidly mounted.Coordinate Frames for Rigid-Body IO

[0167] For platforms with rigidly mounted IMUs, the choice of coordinate frame has a significant impact on what the network can learn. DIDO [27a] models quadrotor dynamics in the body frame, where tilt-velocity coupling implicitly resolves heading. Cioffi et al. [4a] show that collective thrust measurements provide a sufficient bodyframe prior for drone IO, replacing the need for explicit heading estimation. DIVE [a2] regresses 3D velocity from IMU history alone for quadrotors, showing that bodyframe kinematics provide a sufficient motion prior without additional sensors. AirlO [19a] demonstrated that body -frame representation preserves attitude observability for rigidly-mounted UAV IMUs — the term R^g couples linearly with device attitude, alearning signal that world-frame projection destroys, yielding 66.7% average ATE improvement across three drone datasets.

[0168] However, these body-frame methods benefit from a property specific to drones: tilt direction is mechanically coupled to locomotion heading, so heading is implicitly resolved. Humans can move in any direction regardless of head orientation, for example a person may walk forward, sidestep, or turn while looking in an arbitrary direction. This decoupling makes heading a genuinely unobservable degree of freedom for head-mounted IMUs that raw body-frame representations cannot resolve. Our decomposition factors out heading analytically while preserving the body-frame tilt signal Rj g, without learned equivariant layers or platform-specific dynamic priors. Backbone Architectures

[0169] ResNet [7a] has been the dominant backbone for neural IO since RoNIN [8a] established it as best among ResNet [7a], LSTM [9a], and TCN [la]. TLIO [13a] and EqNIO [10a] both inherit this choice. Transformer-based backbones [25a] have since emerged: CTIN [21a] augmented a ResNet encoder with local and global self-attention, and iMoT [17a] proposed a full encoder-decoder Transformer with cross-modal attention between accelerometer and gyroscope. All prior work assumes a fixed input representation — none examines how representation choice interacts with backbone capacity. HeadlO show that this interaction matters: a Transformer on raw bodyframe input performs worse than a ResNet, but with this decomposition it achieves the best results across all configurations.

[0170] Fig. 16 shows a body-frame heading decomposition such that when the head is upright, the bodyframe up axis (+y) aligns with gravity direction eg, and the forward axis ft, coincides with ehl. When the head tilts, ft, gains a gravity-aligned component,causing gravity leakage into the forward channel. The decomposition projects / / , onto the plane perpendicular to es(Eq. 4), recovering a tilt-invariant forward direction ehl.MethodsBody-Frame Heading Decomposition

[0171] An IMU rigidly mounted on a head-mounted device measures acceleration and angular velocity in the device body frame B:a? = v® + R?g + ba+ nawhere Rt6 SO(3) is the device orientation (body -*■ world), g is gravity in the world frame, ba, bgare accelerometer and gyroscope biases, and na, ngdenote sensor noise. Rt is assumed to be available from the on-device orientation filter.

[0172] Raw body-frame signals preserve the tilt signal R g that world-frame projection discards, but head tilt redistributes both gravity and motion signals across device axes, making the representation tilt-variant. This decomposition separates gravity-aligned and horizontal components, then constructs a heading-invariant basis for the horizontal plane.|00173| Raw body-frame signals preserve the tilt signal R g that world-frame projection discards, but head tilt redistributes both gravity and motion signals across device axes, making the representation tilt-variant. This decomposition separates gravity-aligned and horizontal components, then constructs a heading-invariant basis for the horizontal plane.

[0174] Gravity-horizontal separation. The gravity direction in the body frame eg= Rj g separates each measurement into a gravity-aligned scalar and a horizontal remainder:a y., = a? ■ e yn, ah= a? — a yne,

[0175] The scalar agcaptures gravity and vertical acceleration (ag||g|| at rest). The horizontal remainders ah, « h G R3lie in the plane perpendicular to egbut lack a stable basis across head orientations.

[0176] Heading-invariant basis construction. We project the device forward axis j ([0, 0, 1]Tin CPF) onto the plane perpendicular to eg:ehl= normalize(fb- (f&• e efl), eh2= egx ehl

[0177] Since eg= R^R^g = R^g (yaw rotates about g, leaving it invariant), the basis {eg, ehi, ehz} depends only on tilt, not heading.

[0178] Scalar projection. Projecting horizontal vectors onto this basis yields the full per- timestep input xt= [ag, cog, ah • Chi, ah • eh2, (Oh • ehi, (Oh • ei,21 G R6. All six features are heading-invariant since the basis depends only on tilt.Velocity Prediction and Trajectory Reconstruction

[0179] The network predicts decomposed velocity [v'g, v' hl, v' h2] from a window of input features. Body-frame velocity v® = vgeg+ vhlehl+ vh2eh2is reconstructed using the precomputed basis, then rotate to world frame= Rtv®. Heading enters only at the final rotation, not during feature computation or learning.ExperimentsDatasets

[0180] HeadlO was evaluated on three head-mounted IMU datasets with varying motion complexity, sensor hardware, and tilt characteristics.

[0181] Nymeria [15a]. Nymeria dataset contains 1200 sequences from 264 participants across 50 indoor and outdoor locations, totaling over 300 hours. Activities range from athletic motion and cycling to cooking and social interaction, producing the most diverse and dynamic head motion among our benchmarks (tilt SD: 14.54°). IMU and ground-truth trajectories are recorded at 1 kHz from Project Aria glasses via Aria Machine Perception Services. An 80 / 10 / 10 train / validation / test split is used.

[0182] Aria Everyday Activities (AEA) [14a]. Project Aria Everyday Activities (AEA) dataset contains 143 sequences across five indoor locations, totaling 7.3 hours of household activities such as cooking, tidying, and social interaction. Recorded with the same Aria hardware and ground-truth pipeline as Nymeria, but with less dynamic head motion (tilt SD: 11.20°). Nymeria-trained models were evaluated on the full AEA dataset without fine-tuning to test cross-dataset generalization.

[0183] TLIO [13a]. TLIO dataset contains approximately 60 hours of pedestrian walking recorded with a Bosch BMI055 IMU on a head-mounted device (tilt SD: 12.93°). Ground truth is from visual-inertial odometry. The train / val / test split protocol of prior work [10a, 13 a] were followed for a fair comparison.

[0184] Coordinate Frame Convention. The body frame was defined as Project Aria’s Central Pupil Frame (CPF) [5a] for consistency across different datasets. CPF is a device-centric coordinate system defined as (+X left, +Y up, +Z forward). The CPFforward axis (+Z) serves as / * in our heading decomposition, so using CPF as the body frame ensures a fair comparison: all configurations receive the same body-frame IMU signals, differing only in how they are preprocessed. The world frame was defined with the Y-up convention of Heading-Agnostic Coordinate Frame (HACF) defined in RoNIN [8a] so that body-to-world rotations involve no axis permutation. Nymeria and AEA natively provide CPF-aligned data. For TLIO, the CPF alignment was approximated from the dataset documentation; any misalignment in the gravity direction would affect tilt accuracy and degrade the decomposition.Evaluation Metrics

[0185] Four metrics are reported. Absolute Trajectory Error (ATE) is the root-mean- square position error with trajectories initialized at a common origin, decomposed into horizontal (H) and vertical (V) components. Relative Translational Error at At seconds (RTE- A t) is the root-mean-square displacement error over sliding windows of A t G { 1, 5} s, capturing short-term prediction quality independent of long-term drift accumulation. Drift Rate is the final position error as a percentage of total ground-truth path length. We prioritize RTE-5s as the primary metric, since ATE averages are dominated by long trajectory sequences. For example, the top 2% of Nymeria sequences by path length (averaging 724m each) account for 13% of total ATE. To ensure fair comparison both within and across datasets of different trajectory lengths, we use RTE-5s as the main metric, which evaluates displacement accuracy over fixed- length sliding windows.Implementation Details

[0186] FIG. 17 shows a block diagram of a velocity estimation network for headmounted inertial odometry according to one or more aspects of the present disclosure.The velocity estimation network may receive body-frame inputs 1705 such as bodyframe acceleration input and body-frame angular velocity input, each having dimensions representing batch size, temporal samples, and three-dimensional measurements. In some cases, the velocity estimation network may process a window of 1000 IMU samples at 200 Hz, corresponding to a 5 second temporal window.

[0187] The velocity estimation network may include a basis construction module 1750 that receives rotation input representing quaternion orientation data. The basis construction module 1750 may construct a heading-invariant orthonormal basis comprising basis vectors for a gravity-aligned direction, a forward horizontal direction, and a lateral horizontal direction. The basis construction module 1750 may compute a gravity direction 1700 in the device body frame based on a tilt component of the device orientation data. In some cases, the gravity direction 1700 may be computed by applying the transpose of the device rotation matrix to a world-frame gravity vector.

[0188] The basis construction module 1750 may project a device forward axis onto a plane perpendicular to the gravity direction 1700 via Gram-Schmidt orthogonalization to produce a first horizontal basis vector. A second horizontal basis vector may be computed as a cross product of the gravity direction 1700 and the first horizontal basis vector. Because the body-frame gravity direction is invariant to yaw rotation, the heading-invariant orthonormal basis may depend only on tilt and not on heading.

[0189] A basis projection module 1710 may receive the body-frame acceleration input 1705 and the body-frame angular velocity input. The basis projection module 1710 may project the body-frame acceleration input 1705 and the body-frame angular velocity input onto the heading-invariant orthonormal basis constructed by the basis construction module 1750. The basis projection module 1710 may separate each IMU measurement into a gravity-aligned scalar component and a horizontal remainder byprojection onto the gravity direction 1700. The basis projection module 1710 may produce a combined output comprising six scalar values per timestep, including a gravity-aligned acceleration component, two horizontal acceleration components, a gravity-aligned angular velocity component, and two horizontal angular velocity components. In some cases, all six features may be heading-invariant since the basis depends only on tilt.

[0190] The projected output from the basis projection module 1710 may be fed to an encoding layer with ID conv pooling 1720. The encoding layer with ID conv pooling 1720 may reduce the temporal dimension from 1000 samples to 112 samples while encoding the features. In some cases, the decomposed representation may be augmented with temporal differences and encoded via per-group linear layers prior to the strided one-dimensional convolutions. The encoding layer with ID conv pooling 1720 may produce an output with an expanded feature dimension.

[0191] The encoded features may be processed by a transformer with Bi-LSTM module 1730. The transformer with Bi-LSTM module 1730 may apply temporal self-attention and bidirectional sequential modeling to capture long-range dependencies and sequential dynamics. In some cases, the transformer with Bi-LSTM module 1730 may comprise a multi-layer Transformer encoder followed by a bidirectional long short-term memory (LSTM) network. The transformer with Bi-LSTM module 1730 may produce an output representing learned temporal features.

[0192] The output from the transformer with Bi-LSTM module 1730 may be provided to a velocity decoder 1740. The velocity decoder 1740 may predict decomposed velocity components comprising a gravity-aligned velocity and two horizontal velocity components. In some cases, separate decoders may predict the gravity-aligned velocity and the horizontal velocities independently.

[0193] The predicted velocity components from the velocity decoder 1740 may be combined with the basis vectors from the basis construction module 1750 to reconstruct body -frame velocity. The body-frame velocity may be reconstructed by multiplying the gravity-aligned velocity by the gravity-aligned basis vector and adding the products of the horizontal velocities with the corresponding horizontal basis vectors. The reconstructed body-frame velocity may be rotated using the device orientation to produce world-frame velocity output 1795. Heading may enter only at the final rotation to world frame, not during feature computation or learning by the velocity estimation network.

[0194] The velocity estimation network may achieve tilt invariance and heading equivariance through the analytical preprocessing performed by the basis projection module 1710 and the basis construction module 1750, without requiring learned equivariant layers, data augmentation, or architectural constraints. In some cases, during evaluation, the sparse predictions from the velocity decoder 1740 may be linearly interpolated to a ground-truth sampling rate for trajectory reconstruction.

[0195] HeadlO backbone architecture. HeadlO processes a window of T=1000 IMU samples at 200Hz (5 s). The 6-scalar heading decomposition is augmented with temporal differences and a bijective angular velocity decomposition [10a], then encoded via per-group linear layers. Strided ID convolutions pool the sequence from 1000 to 112 frames. A 2-layer Transformer attends over the pooled sequence, followed by a 2-layer bidirectional LSTM. Separate decoders predict gravity- aligned and horizontal velocity, reconstructed to 3D via the precomputed heading basis. During evaluation, the 112 sparse predictions are linearly interpolated to the ground-truth sampling rate. Full architecture details are shown in Fig. 17, which shows an illustrative block diagram overview of the HeadlO system.

[0196] Training. All models are trained with Adam optimizer [Ila] (learning rate 5xl0-4, weight decay 10-4, gradient clipping at 1.0) with ReduceLROnPlateau (factor 0.5, patience 10, minimum 10-6). All experiments run on a single NVIDIA A40 GPU for up to 50 epochs with batch size 512.

[0197] Training objective. The total training loss is £ =vei£vei+.smooth£smootll+ ^-pos^pos T ^-cov^cov with weights100, ^-smooth 100, Tp0S5, 2C0V0.01. £ve!is the mean per-axis Huber loss with 5 = 0.005 m / son predicted velocity. Lsmooth penalizes deviations in consecutive velocity differences using the same Huber function. LpOSis an MSE loss on world-frame positions integrated from predicted velocities within each window via pt= ^=i RsVsAt. Lcov is a diagonal Gaussian NLL for learned per-timestep uncertainty crt d= exp (ht d— 5). The position and temporal losses improve trajectory stability, as shown in the loss ablation (Table 12 of FIG. 25). Table 12 of FIG. 25 shows an ablation study on loss components (Nymeria, Transformer), where the best per metric is shown in bold.

[0198] Computational cost. Processing a 5 s input window requires 551MFLOPs for our Transformer pipeline, versus 367 MFLOPs for the ResNet backbone used by TLIO. EqNIO adds 1680MFLOPs of Vector Neuron preprocessing on top of the same ResNet, totaling 2047MFLOPs — nearly four times more when compared to the HeadlO pipeline cost.ResultsMain Results

[0199] HeadlO achieves state-of-the-art performance across all metrics on all three datasets, with the sole exception of ATE on TLIO, where EqNIO leads by 0.025m (Table 7 of FIG. 18). Table 7 of FIG. 18 shows a comparison with prior methods across threehead-mounted datasets, where the best per metric is shown in bold. On Nymeria, HeadlO improves RTE-5s by 30% (0.294m -* 0.205m) and drift by 58% (3.40% -* 1.42%) over the recent prior state of the art, EqNIO. On AEA, which contains relatively short trajectories compared to Nymeria, the model also excels in short-term stability, improving RTE-5s by 35% over TLIO (0.270m 0.174m), the stronger baseline on this dataset, while also achieving the lowest ATE (0.764m vs 0.774m). On TLIO, the RTE-5s gain is 7.0% (0.322m 0.299m) compared to EqNIO, but HeadlO still achieves the lowest drift rate (1.48% vs 1.59% for EqNIO), demonstrating that our approach generalizes to pedestrian navigation and is not limited to AR glasses interaction datasets.Preprocessing Ablation Study

[0200] To systematically analyze whether the gain comes from our novel preprocessing method or backbone architecture, we ablate both factors across four configurations (Table 8). Table 8 of FIG. 19 shows four preprocessing configurations evaluated in our ablation. Each preprocessing configuration corresponds to an existing method or the proposed decomposition. We first evaluate all four on the same ResNet backbone with a 5 s input window to match the Transformer setting.

[0201] World / Naive rotates body-frame measurements into a gravity-aligned global frame, following TLIO. World / VN applies EqNIO’s Vector Neuron preprocessing to achieve learned 0(2) equivariance in the world frame. Body / Naive feeds raw body-frame measurements directly, while Body / Heading applies our decomposition from above. For the ResNet ablation, we use EqNIO’s codebase and training procedure, changing only the preprocessing method. For the Transformer ablation, we apply our architecture and loss function across all four configurations.

[0202] ResNet Backbone. Table 9 of FIG. 20 reports results with the ResNet backbone.Table 9 shows a ResNet ablation across all datasets, where the best per metric per dataset is shown in bold. The body frame heading decomposition method achieves the best RTE-5s on Nymeria and AEA, outperforming EqNIO by 8.2% (0.294m^ 0.270m) and 22.6% (0.287m 0.222m) respectively — without requiring EqNIO’s computationally expensive VN layers (1680MFLOPs). On the TLIO dataset, EqNIO leads across all metrics, but our preprocessing method still outperforms both raw bodyframe and world-frame inputs across all metrics with negligible computational overhead. These results show that our decomposition captures better motion dynamics with simpler preprocessing compared to heavy learned layers.[00203J Transformer Backbone. Next, the same four configurations are evaluated with our Transformer backbone (Table lOofFIG. 21). Table 10 shows a Transformer ablation across all datasets, where the best per metric per dataset is shown in bold. Body / Heading combined with the Trans- former dominates all metrics on all three datasets — including TLIO dataset, where EqNIO leads on ResNet. Body / Heading improves RTE-5s over World / VN by 31.5% (0.300m 0.205m) on Nymeria, and ATE by 46.1% (5.26m 2.84m). In contrast, World / VN ATE shows little change between backbones (5.22m vs 5.26m), while Body / Heading improves by 30.2% (4.06m 2.84m). This suggests that without structured invariance, larger model capacity does not translate to better performance.Ablation Studies

[0204] Input Variance & Learning Curve. Table 11 of FIG. 22 shows an acceleration input distribution (Mean ± SD in m / s2) on Nymeria. Table 11 compares the acceleration input distribution across three representations on Nymeria. Raw body-frame signals exhibit high variance on the forward channel (o; SD 2.96m / s2) because head tilt re-distributes gravity across device axes. World-frame projection isolates gravity into ay (SD 0.90 m / s2) but discards the body-frame attitude signal ^g, and its horizontal channels remain heading-dependent. The heading decomposition achieves the same clean separation — agcaptures gravity with identical statistics to the world-frame vertical channel, while ahi and aic are zero-mean with low variance — without sacrificing body-frame information.

[0205] FIG. 23 illustrates a chart showing Validation MSE on Nymeria across IMU representations on ResNet. For example, FIG. 23 shows the validation MSE on Nymeria across different input representations using the ResNet backbone. Body / Heading achieves the lowest error and converges the fastest, while Body / Naive plateaus at a higher loss due to the entangled gravity signal. World / Naive and World / VN converge to higher error floors, consistent with the loss of body-frame attitude information.

[0206] Qualitative Trajectory Comparison. FIG. 24 shows two representative test sequences from Nymeria dataset. Fig. 24 shows a qualitative trajectory comparison on two Nymeria test sequences. The left shows long outdoor trajectory (~654 m). The right shows a short indoor sequence The Body / Heading (ours) is stable and tracks ground truth more closely than World / VN in both cases. On the long trajectory (left), World / VN accumulates significant heading-dependent drift, while Body / Heading (ours) tracks the ground truth closely throughout. On a shorter indoor sequence (right), both methods capture the overall shape, but our method produces tighter alignment with fewer lateral deviations. These qualitative results are consistent with the quantitative improvements in Table 3 of FIG. 20: the decomposition reduces both short-term displacement error and long-range drift.

[0207] Gravity Subtraction and Orientation Noise Robustness. The heading decomposition retains the gravitational component in the gravity-axis channel (ag9.81m / s2at rest). Under clean orientations, retaining gravity yields lower drift (1.42% vs. 2.01%) and better ATE (2.84m vs. 3.63m) compared to subtracting gravity, assuming ground-truth orientations from an accurate SLAM system are available.

[0208] To test robustness assuming IMU-only scenarios, we inject synthetic tilt noise at evaluation time only using an Ornstein-Uhlenbeck process (r = 10 s, o = 0.5°) on pitch and roll. If gravity is removed using the accurate groundtruth orientation, the gravity- subtracted model is virtually invariant (drift: 2.01% 2.03%). However, in realistic deployment the same noisy orientation must be used for gravity removal, which applies the noisy rotation directly to the full 9.81m / s2 gravitational acceleration. This adds a third error pathway on top of basis construction and body-to-world velocity integration. Under realistic noise, the gravity-subtracted model degrades to 0.337m RTE-5s and 4.44% drift, worse than the gravity-retaining model (0.319m, 3.16%).

[0209] World-frame methods are even less robust (World / VN drift: 2.94% 9.38%;World / Naive: 6.37% 8.96% at 0.5 ). Moreover, world-frame methods are additionally sensitive to yaw errors from orientation filters. Among all realistic configurations, gravity retention degrades the least, making it the preferred choice for deployment on consumer devices where orientation estimates inevitably carry noise.

[0210] Loss Function Ablation. The loss components are ablated on the Nymeria dataset with the Transformer backbone (Table 12 of FIG. 25). The full loss improves RTE- 5s by 14.1% (0.239m → 0.205m) and ATE by 25.3% (3.79m → 2.84m) over velocity-only supervision. The position loss stabilizes long-range trajectory estimation, contributing the largest ATE reduction (3.79m to 2.98m). While the temporal smoothness loss alone shows limited effect on metrics, combining it with theposition loss yields the best drift across all configurations (1.42%), suggesting the two losses are complementary — position loss provides long-range supervision while temporal loss regularizes frame-to-frame dynamics.

[0211] Transformer Sequence length ablation. An ablation study was conducted regarding input window length effects on Transformer performance on the Nymeria dataset (Table 13). Table 13 of FIG. 26 shows window length ablation (Nymeria, Transformer), where the best per metric is shown in bold. Performance improves sharply from 1 s to 5 s, with RTE-5s dropping from 0.857m to 0.205m as compute scales from 107 to 550MFLOPs. Extending to 10 s doubles the cost to 1158MFLOPs but yields negligible RTE improvement (0.2051 vs 0.2054) while ATE worsens by 6.0% (2.84m 3.01m), suggesting 5 s is the optimal setting for this case.Conclusion

[0212] HeadlO combines a body-frame heading decomposition with a Transformer and Bi- LSTM backbone for neural inertial odometry on head-mounted devices. Our decomposition separates gravity-aligned and horizontal components using only the tilt component of device orientation, achieving tilt invariance and heading equivariance without learned equivariant layers or architectural constraints. Systematic ablation revealed that input representation critically interacts with backbone capacity: a Transformer on raw body-frame input performs worse than a ResNet, but with our decomposition achieves the best results across all configurations. HeadlO achieves state-of-the-art performance on three head-mounted datasets, improving RTE-5s by 30% over EqNIO on Nymeria with four times lower FLOPs. In some cases, HeadlO may include processes to adapt the decomposition to non-rigid mounting (e.g., earbuds) and to combine it with visual-inertial fusion for drift-free long-range tracking.

[0213] FIG. 27 is a block diagram of an illustrative processor platform 2700 structured to executing the instructions of at least FIGS. 1, 3, and 17 to implement the illustrative components, operations, and / or actions to generate information as disclosed and described herein with respect to FIGS. 1 -26. The processor platform 2700 can be, for example, a server, a personal computer, a mobile device (e.g., a cell phone, a smart phone, a tablet such as an iPadTM), a personal digital assistant (PDA), an Internet appliance, and / or any other type of computing device or combination of computing devices.

[0214] The processor platform 2700 includes a processor 2712. The processor 2712 is hardware. For example, the processor 2712 can be implemented by integrated circuits, logic circuits, microprocessors or controllers from any desired family or manufacturer.

[0215] The processor 2712 includes a local memory 2713 (e.g., a cache). The processor 2712 of FIG. 27 executes the instructions associated with functionality of at least FIGS.1, 3, and 17 to implement the systems and infrastructure and associated methods of FIGS. 1, 3, and 17 such as one or more computing devices of the illustrative hierarchical motion-based inertial localization system 100, the MARIO framework 300, the HeadlO system 1700, etc. The processor 2712 is in communication with a main memory including a volatile memory 2714 and a non-volatile memory 2716 via a bus 2718 and / or a memory bus 2702. The volatile memory 2714 can be implemented by Synchronous Dynamic Random-Access Memory (SDRAM), Dynamic Random- Access Memory (DRAM), RAMBUS Dynamic Random-Access Memory (RDRAM) and / or any other type of random-access memory device. The non-volatile memory 2716 can be implemented by flash memory and / or any other desired type of memory device. Access to the one or more main memory components (e.g., the volatile memory 2714, the non-volatile memory 2716) may be controlled by a clock controller.

[0216] The processor platform 2700 of the illustrated example also includes an interface circuit 2720. The interface circuit 2720 can be implemented by any type of interface standard, such as an Ethernet interface, a universal serial bus (USB), and / or a PCI express interface.

[0217] In the illustrated example, one or more input devices 2722 are connected to the interface circuit 2720. The one or more input devices 2722 permits a user to enter data and commands into the processor 2712. The one or more input devices 2722 can be implemented by, for example, a sensor, a microphone, a camera (still or video), a keyboard, a button, a mouse, a touchscreen, a track-pad, a trackball, isopoint, and / or a voice recognition system.

[0218] One or more output devices 2724 are also connected to the interface circuit 2720. The output devices 2724 can be implemented, for example, by display devices (e.g., a light emitting diode (LED), an organic light emitting diode (OLED), a liquid crystal display, a cathode ray tube display (CRT), a touchscreen, a tactile output device, and / or speakers). The interface circuit 220, thus, typically includes a graphics driver card, a graphics driver chip, or a graphics driver processor.

[0219] The interface circuit 2720 also includes a communication device such as a transmitter, a receiver, a transceiver, a modem and / or network interface card to facilitate exchange of data with external machines (e.g., computing devices of any kind) via a network 2726 (e.g., an Ethernet connection, a digital subscriber line (DSL), a telephone line, coaxial cable, a cellular telephone system, etc.).

[0220] The processor platform 2700 also includes one or more mass storage devices 2728 for storing software and / or data. Examples of mass storage devices 2728 include floppydisk drives, hard drive disks, compact disk drives, Blu-ray disk drives, RAID systems, and digital versatile disk (DVD) drives, and / or other mass storage devices.

[0221] The coded instructions 2732 of FIG. 27 can be stored in the mass storage device 2728, in the volatile memory 2714, in the non-volatile memory 2716, and / or on a removable tangible computer readable storage medium such as a CD or DVD.

[0222] The hierarchical motion-based enhanced inertial localization system automatically adapts its training strategy based on which components are available. For example, the base motion estimation is always trained as the fundamental component and any available optional components (pose, activity, environmental constraints, etc.) are automatically incorporated through their respective losses. The system dynamically adjusts loss weightings based on which components are present, ensuring optimal use of available information. When new types of information become available (e.g., adding environmental maps to a deployed system), the corresponding losses can be activated and the system can be fine-tuned accordingly

[0223] This flexible architecture allows the system to be deployed in various scenarios with different levels of available information, from basic IMU-only setups to full systems with pose data, activity labels, and environmental maps. The system's performance gracefully scales with the amount of available information, providing the best possible accuracy given the available data.

[0224] According to an aspect of the present disclosure, a method is provided. The method includes receiving, from a device associated with a user, inertial measurement unit (IMU) data corresponding to movements of a user. The method includes determining, based on the IMU data, a full-body pose of the user. The method includes identifying, based on the IMU data, a motion pattern corresponding to the full-bodypose. The method includes decomposing the motion pattern into one or more motion signals. The method includes capturing, based on the one or more motion signals and the motion pattern, one or more activity -pose relationships. The method includes smoothing, based on modeled losses, the one or more motion signals. The method includes communicating, to a motion signal consumer application, the one or more motion signals.

[0225] According to other aspects of the present disclosure, the method may include one or more of the following features. The method may include receiving, from a plurality of user devices associated with a user, the IMU data from each user device of the plurality of user devices, wherein each user device of the plurality of user devices is located adjacent a different portion of the user's body. The plurality of user devices may include one or more of a mobile phone, a smart watch, an audio device, and an augmented reality headset. When one or more of the plurality of user devices is missing, associated inputs may be masked with zero values. Decomposing the motion pattern into one or more motion signals may include identifying an interpretable seasonal signal and a trend-cycle signal. The one or more motion signals may include one or more motion features. The modeled losses may include a first set of always present losses and a second set of optional losses. Particular losses of the modeled losses may be set to zero when particular ground truth data or environmental data are unavailable. The modeled losses may include one or more motion estimation losses. The one or more motion estimation losses may include one or more of direct velocity loss, drift control loss, and smoothness loss. The motion estimation losses may include one or more of joint position loss, joint rotation loss, kinematic chain loss, pose smoothness loss, activity classification loss, activity translation loss, activity-pose consistency loss, environmental barrier loss, environmental semantic loss, and path plausibility loss. The method may include augmenting the IMU data with environmental information. Theenvironmental information may include environmental map information, positioning information, or geolocation information. The method may include augmenting the IMU data with external context information corresponding to environmental factors that influence motions performed by the user. The environmental factors may include one or more of crowd dynamics, building usage patterns, and real-time occupancy data. At least a portion of the IMU data may be received from a head-mounted device, and the method may further include receiving device orientation data comprising at least a tilt component of device orientation, wherein the IMU data comprises accelerometer measurements and gyroscope measurements in a device body frame. The method may include computing a gravity direction vector in the device body frame based on the tilt component of the device orientation data. The method may include constructing a heading-invariant orthonormal basis comprising a gravity-aligned basis vector and two horizontal basis vectors by projecting a device forward axis onto a plane perpendicular to the gravity direction vector and completing the heading-invariant orthonormal basis with a cross product. The method may include decomposing the IMU data into gravity-aligned components and horizontal components by projecting the accelerometer measurements and gyroscope measurements onto the heading-invariant orthonormal basis to produce a decomposed representation. The method may include processing the decomposed representation using a neural network to predict velocity components and reconstructing a trajectory based on the predicted velocity components. The IMU data may be received from a primary IMU on the head-mounted device, and the method may further include predicting a human body pose representation from the IMU data using a pose estimation network to generate pose prior features, receiving auxiliary sensor data from at least one auxiliary sensor on the head-mounted device, encoding the pose prior features and the auxiliary sensor data using respective feature encoders to produce encoded feature representations, fusing the encoded feature representations with features derived from the IMU data to produce a fused representation, predicting amotion state based on the fused representation using an inertial odometry network, and reconstructing a trajectory based on the predicted motion state. The at least one auxiliary sensor may include a barometer, and the method may include converting ambient air pressure measured by the barometer to altitude information using a hypsometric relation and encoding the altitude information as auxiliary features.

[0226] According to another aspect of the present disclosure, a method for inertial odometry on a head-mounted device is provided. The method includes receiving inertial measurement unit (IMU) data from an IMU rigidly mounted on the head-mounted device, the IMU data comprising accelerometer measurements and gyroscope measurements in a device body frame. The method includes receiving device orientation data comprising at least a tilt component of device orientation. The method includes computing a gravity direction vector in the device body frame based on the tilt component of the device orientation data. The method includes constructing a headinginvariant orthonormal basis comprising a gravity-aligned basis vector and two horizontal basis vectors by projecting a device forward axis onto a plane perpendicular to the gravity direction vector and completing the heading-invariant orthonormal basis with a cross product. The method includes decomposing the IMU data into gravity- aligned components and horizontal components by projecting the accelerometer measurements and gyroscope measurements onto the heading-invariant orthonormal basis to produce a decomposed representation. The method includes processing the decomposed representation using a neural network to predict velocity components. The method includes reconstructing a trajectory based on the predicted velocity components.

[0227] According to other aspects of the present disclosure, the method may include one or more of the following features. The neural network may include a Transformer encoder and a bidirectional long short-term memory (Bi-LSTM) network. The decomposed representation may include six scalar values per timestep comprising agravity-aligned acceleration component, two horizontal acceleration components, a gravity-aligned angular velocity component, and two horizontal angular velocity components. The method may include augmenting the decomposed representation with temporal differences prior to processing by the neural network. The predicted velocity components may include a gravity-aligned velocity and two horizontal velocities, and reconstructing the trajectory may include reconstructing body-frame velocity from the predicted velocity components using the heading-invariant orthonormal basis and rotating the body-frame velocity to world-frame velocity using the device orientation data. The heading-invariant orthonormal basis may depend only on the tilt component of the device orientation data and may be invariant to a heading component of the device orientation data. The method may include processing the decomposed representation through an encoding layer and one-dimensional convolutional pooling to reduce a temporal dimension of the decomposed representation prior to processing by the neural network. The neural network may be trained using a composite loss function comprising one or more of a velocity Huber loss, a temporal smoothness loss, a position mean squared error loss, and an uncertainty negative log -likelihood loss. The decomposed representation may be provided as input to an inertial odometry backbone, and the decomposed representation may be composable with different inertial odometry backbone architectures. The head-mounted device may be one of augmented reality glasses, a virtual reality headset, or earbuds. A heading component of the device orientation data may enter at a final rotation from body-frame velocity to world-frame velocity and may not enter during feature computation or learning by the neural network.

[0228] According to another aspect of the present disclosure, a method for multi-sensor inertial odometry on a head-mounted device is provided. The method includes receiving IMU data from a primary IMU on the head-mounted device. The method includespredicting a human body pose representation from the IMU data using a pose estimation network to generate pose prior features. The method includes receiving auxiliary sensor data from at least one auxiliary sensor on the head-mounted device. The method includes encoding the pose prior features and the auxiliary sensor data using respective feature encoders to produce encoded feature representations. The method includes fusing the encoded feature representations with features derived from the IMU data to produce a fused representation. The method includes predicting, based on the fused representation using an inertial odometry network, a motion state. The method includes reconstructing a trajectory based on the predicted motion state.

[0229] According to other aspects of the present disclosure, the method may include one or more of the following features. The pose estimation network may predict lower-body joint rotations in SMPL format using a six-dimensional continuous rotation representation. The pose estimation network may include a hybrid convolutional- recurrent architecture comprising a temporal convolutional neural network followed by stacked gated recurrent units that sequentially predict joint positions and then joint rotations. The at least one auxiliary sensor may include a barometer, and the method may include converting ambient air pressure measured by the barometer to altitude information using a hypsometric relation and encoding the altitude information as auxiliary features. The at least one auxiliary sensor may include a magnetometer, and the method may include computing a yaw angle from a local magnetic field measured by the magnetometer combined with gravity estimated from the IMU data and encoding the yaw angle as auxiliary features. The at least one auxiliary sensor may include a secondary IMU positioned at a different location on the head-mounted device than the primary IMU, and the method may include encoding measurements from the secondary IMU as auxiliary features. Fusing the encoded feature representations may include time-aligning auxiliary sensor streams to the primary IMU, encoding each stream witha causal temporal convolutional neural network to obtain per-step features, concatenating the features, and feeding the concatenated features to a multilayer perceptron fusion layer. The pose estimation network may be trained using an L2 loss on joint positions and joint rotations and a Huber loss on joint-angle velocities. The inertial odometry network may be backbone-agnostic and may be integrable with different existing inertial odometry architectures. The pose estimation network may estimate only pelvis and lower-limb joint angles corresponding to locomotion-related joints. The pose estimation network may be frozen during training of the inertial odometry network. One or more aspects of the disclosure may be embodied in computer-usable data or computer-executable instructions, such as in one or more program modules, executed by one or more computers or other devices to perform the operations described herein. Generally, program modules include routines, programs, objects, components, data structures, and the like that perform particular tasks or implement particular abstract data types when executed by one or more processors in a computer or other data processing device. The computer-executable instructions may be stored as computer-readable instructions on a computer-readable medium such as a hard disk, optical disk, removable storage media, solid-state memory, RAM, and the like. The functionality of the program modules may be combined or distributed as desired in various embodiments. In addition, the functionality may be embodied in whole or in part in firmware or hardware equivalents, such as integrated circuits, application-specific integrated circuits (ASICs), field programmable gate arrays (FPGA), and the like. Particular data structures may be used to implement one or more aspects of the disclosure more effectively, and such data structures are contemplated to be within the scope of computer executable instructions and computer-usable data described herein.

[0230] Various aspects described herein may be embodied as a method, an apparatus, or as one or more computer-readable media storing computer-executable instructions. Accordingly, those aspects may take the form of an entirely hardware embodiment, an entirely software embodiment, an entirely firmware embodiment, or an embodiment combining software, hardware, and firmware aspects in any combination. In addition, various signals representing data or events as described herein may be transferred between a source and a destination in the form of light or electromagnetic waves traveling through signal-conducting media such as metal wires, optical fibers, or wireless transmission media (e.g., air or space). In general, the one or more computer- readable media may be and / or include one or more non-transitory computer-readable media.

[0231] As described herein, the various methods and acts may be operative across one or more computing servers and one or more networks. The functionality may be distributed in any manner, or may be located in a single computing device (e.g., a server, a client computer, and the like). For example, in alternative embodiments, one or more of the computing platforms discussed above may be combined into a single computing platform, and the various functions of each computing platform may be performed by the single computing platform. In such arrangements, any and / or all of the above-discussed communications between computing platforms may correspond to data being accessed, moved, modified, updated, and / or otherwise used by the single computing platform. Additionally, or alternatively, one or more of the computing platforms discussed above may be implemented in one or more virtual machines that are provided by one or more physical computing devices. In such arrangements, the various functions of each computing platform may be performed by the one or more virtual machines, and any and / or all of the above -discussed communications betweencomputing platforms may correspond to data being accessed, moved, modified, updated, and / or otherwise used by the one or more virtual machines.

[0232] Aspects of the disclosure have been described in terms of illustrative embodiments thereof. Numerous other embodiments, modifications, and variations within the scope and spirit of the appended claims will occur to persons of ordinary skill in the art from a review of this disclosure. For example, one or more of the steps depicted in the illustrative figures may be performed in other than the recited order, and one or more depicted steps may be optional in accordance with aspects of the disclosure. References

[0233] [Savage 1998] Savage, P. G. " Strapdown Inertial Navigation Integration Algorithm Design Part 2: Velocity and Position Algorithms." Journal of Guidance, Control, and Dynamics 21.2 (1998): 208-221.

[0234] [Shen et al. 2018] Shen, S., Gowda, M., and Roy Choudhury, R. " Closing the Gaps in Inertial Motion Tracking." Proceedings of the 24th Annual International Conference on Mobile Computing and Networking. 2018.

[0235] [Jimenez et al. 2009] Jimenez, A. R., et al. " A comparison of Pedestrian Dead- Reckoning algorithms using a low-cost MEMS IMU." IEEE International Symposium on Intelligent Signal Processing. 2009.

[0236] [Tian et al. 2015] Tian, Q., et al. " An Enhanced Pedestrian Dead Reckoning Approach for Pedestrian Tracking Using Smartphones." IEEE Tenth International Conference on Intelligent Sensors, Sensor Networks and Information Processing. 2015.

[0237] [Yan et al. 2018] Yan, H., Shan, Q., and Furukawa, Y. " RIDI: Robust IMU Double Integration." European Conference on Computer Vision. 2018.

[0238] [Chen et al. 2018] Chen, C., et al. "loNet: Learning to Cure the Curse of Drift in Inertial Odometry." AAAI Conference on Artificial Intelligence. 2018.

[0239] [Herath et al. 2020] Herath, S., Yan, H., and Furukawa, Y. " RoNIN: Robust Neural Inertial Navigation in the Wild." IEEE International Conference on Robotics and Automation. 2020.

[0240] [Liu et al. 2020] Liu, W, et al. " TLIO: Tight Learned Inertial Odometry." IEEE Robotics and Automation Letters 5.4 (2020): 5653-5660.

[0241] [Xu et al. 2024] Xu, V., et al. " MobilePoser: Real-Time Full-Body Pose Estimation and 3D Human Translation from IMUs in Mobile Consumer Devices." UIST 2024.

[0242] [Campos et al. 2021] Campos, C., et al. " ORB-SLAM3: An Accurate Open-Source Library for Visual, Visual-Inertial, and Multimap SLAM." IEEE Transactions on Robotics 37.6 (2021): 1874-1890.

[0243] [Pavlakos et al. 2019] Pavlakos, G., et al. " Expressive Body Capture: 3D Hands, Face, and Body from a Single Image." Proceedings of the IEEE / CVF Conference on Computer Vision and Pattern Recognition. 2019.

[0244] [1] Xiya Cao, Caifa Zhou, Dandan Zeng, and Yongliang Wang. Rio: Rotation- equivariance supervised learning of robust inertial odometry. In Proceedings of the IEEE / CVF Conference on Computer Vision and Pattern Recognition, pages 6614-6623, 2022.

[0245] [2] Changhao Chen, Xiaoxuan Lu, Andrew Markham, and Niki Trigoni. lonet:Learning to cure the curse of drift in inertial odometry. In Proceedings of the AAAI conference on artificial intelligence, 2018.

[0246] [3] Jakob Engel, Kiran Somasundaram, Michael Goesele, Albert Sun, Alexander Gamino, Andrew Turner, Arjang Talattof, Arnie Yuan, Bilal Souti, Brighid Meredith, et al. Project aria: A new tool for egocentric multi-modal ai research. arXiv preprint arXiv: 2308.13561, 2023.

[0247] [4] Guoquan Huang. Visual-inertial navigation: A concise review. In 2019 international conference on robotics and automation (ICRA), pages 9572-9582. IEEE, 2019.

[0248] [5] Yinghao Huang, Manuel Kaufmann, Emre Aksan, Michael J Black, Otmar Hilliges, and Gerard Pons-Moll. Deep inertial poser: Learning to reconstruct human pose from sparse inertial measurements in real time. ACM Transactions on Graphics (TOG), 37(6):1— 15, 2018.

[0249] [6] Royina Karegoudra Jayanth, Yinshuang Xu, Ziyun Wang, Evangelos Chatzipantazis, Daniel Gehrig, and Kostas Daniilidis. Eqnio: Subequivariant neural inertial odometry. arXiv preprint arXiv:2408.06321, 2024.

[0250] [7] Yifeng Jiang, Yuting Ye, Deepak Gopinath, Jungdam Won, AlexanderWWinkler, and C Karen Liu. Transformer inertial poser: Real-time human motion reconstruction from sparse imus with simultaneous terrain generation. In SIGGRAPH Asia 2022 Conference Papers, pages 1-9, 2022.

[0251] [8] Dongjae Lee, Minwoo Jung, Wooseong Yang, and Ayoung Kim. Lidar odometry survey: recent advancements and remaining challenges. Intelligent Service Robotics, 17(2):95— 118, 2024.

[0252] [9] Woosik Lee, Patrick Geneva, Chuchu Chen, and Guoquan Huang. Mins: Efficient and robust multisensor-aided inertial navigation system, 2023.

[0253]

[0010] Yadong Li, Shuning Wang, Yongjian Fu, Justin Chen, Xingyu Chen, Ju Ren, Xinyu Zhang, Akshay Gadre, and Ke Sun. Ultraposer: Pushing the limits of imu-based fullbody pose estimation with ultrasound sensing on consumer wearables. In Proceedings of the 38th Annual ACM Symposium on User Interface Software and Technology, pages 1-15, 2025.

[0254]

[0011] Yan Li, Yang Xu, Changhao Chen, Zhong chen Shi, Wei Chen, Liang Xie, Hongbo Chen, and Erwei Yin. M2eit: Multi-domain mixture of experts for robust neural inertial tracking. In Proceedings of the IEEE / CVF International Conference on Computer Vision (ICCV), pages 28207-28216, 2025.

[0255]

[0012] Wenxin Liu, David Caruso, Eddy Ilg, Jing Dong, Anastasios I. Mourikis, Kostas Daniilidis, Vijay Kumar, and Jakob Engel. Tlio: Tight learned inertial odometry. IEEE Robotics and Automation Letters, 5(4):5653- 5660, 2020.

[0256]

[0013] Matthew Loper, Naureen Mahmood, Javier Romero, Gerard Pons-Moll, and Michael J Black. Smpl: A skinned multiperson linear model. In Seminal Graphics Papers: Pushing the Boundaries, Volume 2, pages 851-866. 2023.

[0257]

[0014] Zhaoyang Lv, Nicholas Charron, Pierre Moulon, Alexander Gamino, Cheng Peng, Chris Sweeney, Edward Miller, Huixuan Tang, Jeff Meissner, Jing Dong, Kiran Somasundaram, Luis Pesqueira, Mark Schwesinger, Omkar Parkhi, Qiao Gu, Renzo De Nardi, Shangyi Cheng, Steve Saarinen, Vijay Baiyya, Yuyang Zou, Richard Newcombe, Jakob Julian Engel, Xiaqing Pan, and Carl Ren. Aria everyday activities dataset, 2024.

[0258]

[0015] Lingni Ma, Yuting Ye, Fangzhou Hong, Vladimir Guzov, Yifeng Jiang, Rowan Postyeni, Luis Pesqueira, Alexander Gamino, Vijay Baiyya, Hyo Jin Kim, Kevin Bailey, David Soriano Fosas, C. Karen Liu, Ziwei Liu, Jakob Engel, Renzo De Nardi, andRichard Newcombe. Nymeria: A massive collection of multimodal egocentric daily motion in the wild, 2024.

[0259]

[0016] Naureen Mahmood, Nima Ghorbani, Nikolaus F. Troje, Gerard Pons-Moll, and Michael J. Black. Amass: Archive of motion capture as surface shapes, 2019.

[0260]

[0017] Vimal Mollyn, Riku Arakawa, Mayank Goel, Chris Harrison, and Karan Ahuja.Imuposer: Full-body pose estimation using imus in phones, watches, and earbuds. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems, pages 1-12, 2023.

[0261]

[0018] Yuheng Qiu, Can Xu, Yutian Chen, Shibo Zhao, Junyi Geng, and Sebastian Scherer. Airio: Learning inertial odometry with enhanced imu feature observability, 2025.

[0262]

[0019] Yunzhe Shao, Xinyu Yi, Lu Yin, Shihui Guo, Junhai Yong, and Feng Xu.Magshield: Towards better robustness in sparse inertial motion capture under magnetic disturbances, 2025.

[0263]

[0020] Scott Sun, Dennis Melamed, and Kris Kitani. Idol: Inertial deep orientationestimation and localization. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 6128-6137, 2021.

[0264]

[0021] Tom Van Wouwe, Seunghwan Lee, Antoine Falisse, Scott Delp, and C Karen Liu.Diffusionposer: Real-time human motion reconstruction from arbitrary sparse sensors using autoregressive diffusion. In Proceedings of the IEEE / CVF Conference on Computer Vision and Pattern Recognition, pages 2513-2523, 2024.

[0265]

[0022] Timo Von Marcard, Bodo Rosenhahn, Michael J Black, and Gerard Pons-Moll.Sparse inertial poser: Automatic 3d human pose estimation from sparse imus. In Computer graphics forum, pages 349-360. Wiley Online Library, 2017.

[0266]

[0023] Jian Wang, Rishabh Dabral, Diogo Luvizon, Zhe Cao, Lingjie Liu, Thabo Beeler, and Christian Theobalt. Ego4o: Egocentric human motion capture and understanding from multi-modal input, 2025.

[0267]

[0024] Xsens Technologies B. V. Xsens IMU Systems, https: / / www.xsens.com.Accessed: 2024-03-07.

[0268]

[0025] Vasco Xu, Chenfeng Gao, Henry Hoffmann, and Karan Ahuja. Mobileposer:Real-time full-body pose estimation and 3d human translation from imus in mobile consumer devices. In Proceedings of the 37th Annual ACM Symposium on User Interface Software and Technology, New York, NY, USA, 2024. Association for Computing Machinery.

[0269]

[0026] Hang Yan, Qi Shan, and Yasutaka Furukawa. Ridi: Robust imu double integration.In Proceedings of the European conference on computer vision (ECCV), pages 621— 636, 2018.

[0270]

[0027] Hang Yan, Sachini Herath, and Yasutaka Furukawa. Ronin: Robust neural inertial navigation in the wild: Benchmark, evaluations, and new methods, 2019.

[0271]

[0028] Yuan Yao, Shifan Jiang, Yangqing Hou, Chengxu Zuo, Xinrui Chen, Shihui Guo, and Yipeng Qin. Tof-ip: time-of-flight enhanced sparse inertial poser for real-time human motion capture. 2025.

[0272]

[0029] Xinyu Yi, Yuxiao Zhou, and Feng Xu. Transpose: Real-time 3d human translation and pose estimation with six inertial sensors. ACM Transactions on Graphics (TOG), 40(4): 1-13, 2021.

[0273]

[0030] Xinyu Yi, Yuxiao Zhou, Marc Habermann, Soshi Shimada, Vladislav Golyanik, Christian Theobalt, and Feng Xu. Physical inertial poser (pip): Physics-aware real-time human motion tracking from sparse inertial sensors. In Proceedings of the IEEE / CVF Conference on Computer Vision and Pattern Recognition, pages 13167-13178, 2022.

[0274]

[0031] Xinyu Yi, Yuxiao Zhou, and Feng Xu. Physical non-inertial poser (pnp): modeling non-inertial effects in sparse-inertial human motion capture. In ACM SIGGRAPH 2024 Conference Papers, pages 1-11, 2024.

[0275]

[0032] Xinyu Yi, Shaohua Pan, and Feng Xu. Improving global motion estimation in sparse imu-based motion capture with physics. ACM Transactions on Graphics (TOG), 44(4): 1-16, 2025.

[0276]

[0033] Libo Zhang, Xinyu Yi, and Feng Xu. Baroposer: Real-time human motion tracking from imus and barometers in everyday devices. In Proceedings of the 38th Annual ACM Symposium on User Interface Software and Technology, page 1-9. ACM, 2025.

[0277]

[0034] Shibo Zhao, Sifan Zhou, Raphael Blanchard, Yuheng Qiu, Wenshan Wang, and Sebastian Scherer. Tartan imu: A light foundation model for inertial positioning in robotics. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 22520-22529, 2025.

[0278]

[0035] Yi Zhou, Connelly Barnes, Jingwan Lu, Jimei Yang, and Hao Li. On the continuity of rotation representations in neural networks, 2020.

[0279]

[0036] Chengxu Zuo, Jiawei Huang, Xiao Jiang, Yuan Yao, Xiangren Shi, Rui Cao, Xinyu Yi, Feng Xu, Shihui Guo, and Yipeng Qin. Transformer imu calibrator: Dynamic on-body imu calibration for inertial motion capture. ACM Transactions on Graphics (TOG), 44(4): 1-14, 2025.

[0280] [la] Bai, S., Kolter, J. Z., Koltun, V.: An empirical evaluation of generic convolutional and recurrent networks for sequence modeling. arXiv preprint arXiv:1803.01271 (2018) 4

[0281] [2a] Bajwa, A., Cossette, C. C., Shalaby, M. A., Forbes, J. R.: Dive: Deep inertial-only velocity aided estimation for quadrotors. IEEE Robotics and Automation Letters 9(4), 3728-3734 (2024) 4

[0282] [3a] Borenstein, J., Ojeda, L., Kwanmuang, S.: Heuristic reduction of gyro drift in imu-based personnel tracking systems. In: Optics and Photonics in Global Homeland Security V and Biometric Technology for Human Identification VI. vol. 7306, p.73061H. SPIE (2009). https: / / doi.org / 10.1117 / 12.816921 3

[0283] [4a] Cioffi, G., Bauersfeld, L., Kaufmann, E., Scaramuzza, D.: Learned inertial odometry for autonomous drone racing. IEEE Robotics and Automation Letters 8(5), 2684-2691 (2023) 4

[0284] [5a] Engel, J., Somasundaram, K., Goesele, M., Sun, A., Gamino, A., Turner, A., Ta- lattof, A., Yuan, A., Souti, B., Meredith, B., et al.: Project aria: A new tool for egocentric multi-modal ai research. arXiv preprint arXiv:2308.13561 (2023) 7

[0285] [6a] Foxlin, E.: Pedestrian tracking with shoe-mounted inertial sensors. IEEE Computer Graphics and Applications 25(6), 38-46 (2005). https: / / doi.org / 10.1109 / MCG. 2005.1403

[0286] [7a] He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition.In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). pp. 770-778 (2016) 4

[0287] [8a] Herath, S., Yan, H., Furukawa, Y: Ronin: Robust neural inertial navigation in the wild: Benchmark, evaluations, & new methods. In: 2020 IEEE international conference on robotics and automation (ICRA). pp. 3146-3152. IEEE (2020) 1, 3, 4, 7

[0288] [9a] Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Computation 9(8), 1735-1780 (1997) 4

[0289] [10a] Jayanth, R. K., Xu, Y, Wang, Z., Chatzipantazis, E., Gehrig, D., Daniilidis, K.:Eqnio: Subequi variant neural inertial odometry. arXiv preprint arXiv:2408.06321 (2024) 1, 2, 3, 4, 6, 7, 9

[0290] [Ila] Kingma, D. P, Ba, J.: Adam: A method for stochastic optimization (2017), https: / / arxiv.org / abs / 1412.69807

[0291] [12a] Leutenegger, S., Lynen, S., Bosse, M., Sieg wart, R., Furgale, P: Keyframebased visual-inertial odometry using nonlinear optimization. In: The International Journal of Robotics Research, vol. 34, pp. 314-334 (2015). https: / / doi.org / 10.1177 / 0278364914554813 1

[0292] [13a] Liu, W., Caruso, D., Ilg, E., Dong, J., Mourikis, A. I., Daniilidis, K., Kumar, V, Engel, J.: Tlio: Tight learned inertial odometry. IEEE Robotics and Automation Letters 5(4), 5653-5660 (2020) 1, 2, 3, 4, 6, 9

[0293] [14a] Lv, Z., Charron, N., Moulon, P., Gamino, A., Peng, C., Sweeney, C., Miller, E., Tang, H., Meissner, J., Dong, J., et al.: Aria everyday activities dataset. arXiv preprint arXiv: 2402.13349 (2024) 6

[0294] [15a] Ma, L., Ye, Y, Hong, F., Guzov, V., Jiang, Y, Postyeni, R., Pesqueira, L., Gamino, A., Baiyya, V., Kim, H. J., et al.: Nymeria: A massive collection of multimodal egocentric daily motion in the wild. In: European Conference on Computer Vision, pp.445-465. Springer (2024) 6

[0295] [16a] Mourikis, A. I., Roumeliotis, S. I.: A multi-state constraint kalman filter for vision-aided inertial navigation. In: Proceedings of the IEEE International Conference on Robotics and Automation (ICRA). pp. 3565-3572 (2007). https: / / doi.org / 10.1109 / ROBOT.2007.364024 1

[0296] [17a] Nguyen, S. M., Le, D. V., Havinga, P.: imot: Inertial motion transformer for inertial navigation. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 39, pp. 6209-6217 (2025) 4

[0297] [18a] Qin, T., Li, P., Shen, S.: Vins-mono: A robust and versatile monocular visual- inertial state estimator. In: IEEE Transactions on Robotics, vol. 34, pp. 1004-1020 (2018). https: / / doi.org / 10.1109 / TRO.2018.2853729 1

[0298] [19a] Qiu, Y, Xu, C., Chen, Y, Zhao, S., Geng, J., Scherer, S.: Airio: Learning inertial odometry with enhanced imu feature observability. IEEE Robotics and Automation Letters (2025) 2, 4, 9

[0299] [20a] Rajagopal, S.: Personal Dead Reckoning System with Shoe Mounted Inertial Sensors. Master’s thesis, KTH Royal Institute of Technology, Stockholm, Sweden (2008) 3

[0300] [21a] Rao, B., Kazemi, E., Ding, Y, Shila, D. M., Tucker, F. M., Wang, L.: Ctin: Robust contextual transformer network for inertial navigation. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 36, pp. 5413-5421 (2022) 4

[0301] [22a] Savage, P. G.: Strapdown inertial navigation integration algorithm design part 1:Attitude algorithms. Journal of Guidance, Control, and Dynamics 21(1), 19-28 (1998). https: / / doi. Org / 10.2514 / 2.42283

[0302] [23a] Teed, Z., Lipson, L., Deng, J.: Deep patch visual odometry (2023), https: / / arxiv.org / abs / 2208.04726 1

[0303] [24a] Titterton, D. H., Weston, J. L.: Strapdown Inertial Navigation Technology. The Institution of Electrical Engineers, Stevenage, UK, 2 edn. (2004). https: / / doi.org / 10.1049 / PBRA017E 3

[0304] [25a] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser,, Polosukhin, I.: Attention is all you need. In: Advances in Neural Information Processing Systems (NeurlPS) (2017) 4

[0305] [26a] Yan, H., Shan, Q., Furukawa, Y: Ridi: Robust imu double integration. In:Proceedings of the European conference on computer vision (ECCV). pp. 621-636 (2018) 1, 3

[0306] [27a] Zhang, K., Jiang, C., Li, J., Yang, S., Ma, T., Xu, C., Gao, F: Dido: Deep inertial quadrotor dynamical odometry. IEEE Robotics and Automation Letters 7(4), 9083- 9090 (2022) 4

[0307] [28a] Zhao, S., Zhou, S., Blanchard, R., Qiu, Y, Wang, W, Scherer, S.: Tartan imu: A light foundation model for inertial positioning in robotics. In: Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 22520-22529 (2025) 1

[0308] [29a] Zhou, C., Cao, X., Zeng, D„ Wang, Y.: Rio: Rotation-equivariance supervised learning of robust inertial odometry (2021), https: / / arxiv.org / abs / 2111.116763

Claims

WHAT IS CLAIMED IS:

1. A method comprising,receiving, from a device associated with a user, inertial measurement unit (IMU) data corresponding to movements of a user;determining, based on the IMU data, a full-body pose of the user;identifying, based on the IMU data, a motion pattern corresponding to the full-body pose;decomposing, based on the motion pattern, the motion pattern into one or more motion signals;capturing, based on the one or more motion signals and the motion pattern, one or more activity-pose relationships;smoothing, based on modeled losses, the one or more motion signals; and communicating, to a motion signal consumer application, the one or more motion signals.

2. The method of claim 1, further comprising receiving, from a plurality of user devices associated with a user, the IMU data from each user device of the plurality of user devices, wherein each user device of the plurality of user devices is located adjacent a different portion of the user’s body.

3. The method of claim 2, wherein the plurality of user devices comprises one or more of a mobile phone, a smart watch, an audio device, and an augmented reality headset.

4. The method of claim 2, wherein when one or more of the plurality of user devices is missing, associated inputs are masked with zero values.

5. The method of claim 1, wherein decomposing the motion pattern into one or more motion signals comprises identifying an interpretable seasonal signal and a trend-cycle signal.

6. The method of claim 1, wherein the one or more motion signals comprise one or more motion features.

7. The method of claim 1, wherein the modeled losses comprise a first set of always present losses and a second set of optional losses.

8. The method of claim 1, wherein particular losses of the modeled losses are set to zero when particular ground truth data or environmental data are unavailable.

9. The method of claim 1, wherein the modeled losses comprises one or more motion estimation losses.

10. The method of claim 9, wherein the one or more motion estimation losses comprises one or more of direct velocity loss, drift control loss, and smoothness loss.

11. The method of claim 9, wherein the motion estimation losses comprises one or more of joint position loss, joint rotation loss, kinematic chain loss, pose smoothness loss, activity classification loss, activity translation loss, activity-pose consistency loss, environmental barrier loss, environmental semantic loss, and path plausibility loss.

12. The method of claim 1, further comprising augmenting the IMU data with environmental information.

13. The method of claim 12, wherein the environmental information comprises environmental map information.

14. The method of claim 12, wherein the environmental information comprises positioning information.

15. The method of claim 12, wherein the environmental information comprises geolocation information.

16. The method of claim 1, further comprising augmenting the IMU data with external context information corresponding to environmental factors that influence motions performed by the user.

17. The method of claim 16, wherein the environmental factors comprises one or more of crowd dynamics, building usage patterns, and / or real-time occupancy data.

18. The method of claim 1, wherein at least a portion of the IMU data is received from a head-mounted device and wherein the method further comprises:receiving device orientation data comprising at least a tilt component of device orientation, wherein the IMU data comprises accelerometer measurements and gyroscope measurements in a device body frame;computing a gravity direction vector in the device body frame based on the tilt component of the device orientation data;constructing a heading-invariant orthonormal basis comprising a gravity-aligned basis vector and two horizontal basis vectors by projecting a device forward axis onto a plane perpendicular to the gravity direction vector and completing the heading-invariant orthonormal basis with a cross product;decomposing the IMU data into gravity-aligned components and horizontal components by projecting the accelerometer measurements and gyroscope measurements onto the heading-invariant orthonormal basis to produce a decomposed representation;processing the decomposed representation using a neural network to predict velocity components; andreconstructing a trajectory based on the predicted velocity components.

19. The method of claim 18, wherein the IMU data is received from a primary IMU on the head-mounted device and wherein the method further comprises:predicting a human body pose representation from the IMU data using a pose estimation network to generate pose prior features;receiving auxiliary sensor data from at least one auxiliary sensor on the head-mounted device;encoding the pose prior features and the auxiliary sensor data using respective feature encoders to produce encoded feature representations;fusing the encoded feature representations with features derived from the IMU data to produce a fused representation;predicting, based on the fused representation and using an inertial odometry network, a motion state; andreconstructing a trajectory based on the predicted motion state.

20. The method of claim 19, wherein the at least one auxiliary sensor comprises a barometer, and wherein the method further comprises converting ambient air pressure measured by the barometer to altitude information using a hypsometric relation and encoding the altitude information as auxiliary features.

21. A method for inertial odometry on a head-mounted device, the method comprising: receiving inertial measurement unit (IMU) data from an IMU rigidly mounted on the head-mounted device, the IMU data comprising accelerometer measurements and gyroscope measurements in a device body frame;receiving device orientation data comprising at least a tilt component of device orientation;computing a gravity direction vector in the device body frame based on the tilt component of the device orientation data;constructing a heading-invariant orthonormal basis comprising a gravity-aligned basis vector and two horizontal basis vectors by projecting a device forward axis onto a plane perpendicular to the gravity direction vector and completing the heading-invariant orthonormal basis with a cross product;decomposing the IMU data into gravity-aligned components and horizontalcomponents by projecting the accelerometer measurements and gyroscope measurements onto the heading-invariant orthonormal basis to produce a decomposed representation;processing the decomposed representation using a neural network to predict velocity components; andreconstructing a trajectory based on the predicted velocity components.

22. The method of claim 21, wherein the neural network comprises a Transformer encoder and a bidirectional long short-term memory (Bi-LSTM) network.

23. The method of claim 21, wherein the decomposed representation comprises six scalar values per timestep comprising a gravity-aligned acceleration component, two horizontal acceleration components, a gravity-aligned angular velocity component, and two horizontal angular velocity components.

24. The method of claim 21, further comprising augmenting the decomposed representation with temporal differences prior to processing by the neural network.

25. The method of claim 21, wherein the predicted velocity components comprise a gravity-aligned velocity and two horizontal velocities, and wherein reconstructing the trajectory comprises reconstructing body-frame velocity from the predicted velocity components using the heading-invariant orthonormal basis and rotating the body-frame velocity to world-frame velocity using the device orientation data.

26. The method of claim 21, wherein the heading-invariant orthonormal basis depends only on the tilt component of the device orientation data and is invariant to a heading component of the device orientation data.

27. The method of claim 21, further comprising processing the decomposed representation through an encoding layer and one-dimensional convolutional pooling to reduce a temporal dimension of the decomposed representation prior to processing by the neuralnetwork.

28. The method of claim 21, wherein the neural network is trained using a composite loss function comprising one or more of a velocity Huber loss, a temporal smoothness loss, a position mean squared error loss, and an uncertainty negative log-likelihood loss.

29. The method of claim 21, wherein the decomposed representation is provided as input to an inertial odometry backbone, and wherein the decomposed representation is composable with different inertial odometry backbone architectures.

30. The method of claim 21, wherein the head-mounted device is one of augmented reality glasses, a virtual reality headset, or earbuds.

31. The method of claim 21, wherein a heading component of the device orientation data enters at a final rotation from body-frame velocity to world-frame velocity and does not enter during feature computation or learning by the neural network.

32. A method for multi-sensor inertial odometry on a head-mounted device, the method comprising:receiving IMU data from a primary IMU on the head-mounted device; predicting a human body pose representation from the IMU data using a pose estimation network to generate pose prior features;receiving auxiliary sensor data from at least one auxiliary sensor on the head-mounted device;encoding the pose prior features and the auxiliary sensor data using respective feature encoders to produce encoded feature representations;fusing the encoded feature representations with features derived from the IMU data to produce a fused representation;predicting, based on the fused representation using an inertial odometry network, a motion state; andreconstructing a trajectory based on the predicted motion state.

33. The method of claim 32, wherein the pose estimation network predicts lower-body joint rotations in SMPL format using a six-dimensional continuous rotation representation.

34. The method of claim 32, wherein the pose estimation network comprises a hybrid convolutional-recurrent architecture comprising a temporal convolutional neural network followed by stacked gated recurrent units that sequentially predict joint positions and then joint rotations.

35. The method of claim 32, wherein the at least one auxiliary sensor comprises a barometer, and wherein the method further comprises converting ambient air pressure measured by the barometer to altitude information using a hypsometric relation and encoding the altitude information as auxiliary features.

36. The method of claim 32, wherein the at least one auxiliary sensor comprises a magnetometer, and wherein the method further comprises computing a yaw angle from a local magnetic field measured by the magnetometer combined with gravity estimated from the IMU data and encoding the yaw angle as auxiliary features.

37. The method of claim 32, wherein the at least one auxiliary sensor comprises a secondary IMU positioned at a different location on the head-mounted device than the primary IMU, and wherein the method further comprises encoding measurements from the secondary IMU as auxiliary features.

38. The method of claim 32, wherein fusing the encoded feature representations comprises:time-aligning auxiliary sensor streams to the primary IMU;encoding each stream with a causal temporal convolutional neural network to obtain per-step features;concatenating the features; andfeeding the concatenated features to a multilayer perceptron fusion layer.

39. The method of claim 32, wherein the pose estimation network is trained using an L2 loss on joint positions and joint rotations and a Huber loss on joint-angle velocities.

40. The method of claim 32, wherein the inertial odometry network is backboneagnostic and is integrable with different existing inertial odometry architectures.

41. The method of claim 32, wherein the pose estimation network estimates only pelvis and lower-limb joint angles corresponding to locomotion-related joints.

42. The method of claim 32, wherein the pose estimation network is frozen during training of the inertial odometry network.

Images