LSTM Pose Estimation via Segmented IMU Processing

The patent segments the pose estimation process into two distinct stages: a tight coupling stage using multiple sensors (IMU, magnetometer, accelerometer, gyroscope) for high-accuracy pose detection, and a loose coupling stage using only IMU data for prediction. This segmentation allows the system to apply complex calculations only when necessary (tight coupling) and use simpler methods for continuous tracking (loose coupling), thereby reducing overall calculation complexity while maintaining accuracy.

The system performs preliminary action by using the tight coupling method to establish an initial accurate pose estimate, then uses this as a foundation for subsequent predictions via the loose coupling method. The LSTM neural network is trained in advance on data from the tight coupling stage, enabling it to predict future poses accurately without requiring real-time processing of all sensor data, thus reducing computational burden during operation.

The patent divides the pose estimation workflow into an offline training phase and an online prediction phase. During offline training, the system processes extensive sensor data through tight coupling to train the LSTM neural network. During online operation, only IMU data is processed through the pre-trained LSTM model, dramatically reducing processing time while maintaining accuracy. This temporal segmentation of computation allows complex processing to occur when time is not critical.

The system performs preliminary computation by training the LSTM neural network in advance using data from the tight coupling method. Once trained, the model can predict poses rapidly using only IMU data, eliminating the need for real-time processing of multiple sensor streams. This preliminary action transfers the computational burden from runtime to setup time, improving real-time performance.

The system performs preliminary action by predicting future poses using the LSTM neural network before the actual rendering time. By estimating poses at future time points based on current and past motion states, the system prepares pose data in advance, allowing the rendering pipeline to proceed without waiting for real-time sensor processing. This eliminates lag without requiring increased frame rates or reduced calculation complexity.

The patent implements dynamic frame rate adjustment based on device motion characteristics. When the device is stationary or moving slowly, the system can use lower frame rates with full tight coupling processing. When motion is detected, the system dynamically switches to higher frame rates using the loose coupling LSTM prediction method. This dynamic adaptation allows the system to maintain low latency during active use while conserving computational resources during static periods.