Pose Prediction Using LSTM and Nonlinear Learning Unit

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Traditional deep learning models for motion capture face challenges in adapting to heterogeneous hardware platforms, leading to high deployment costs and complexity, especially when using temporal structures like LSTM and GRU, which are difficult to parallelize and result in increased model inference time.

Innovation Solution

The method employs a long short-term memory (LSTM) network to extract temporal information from pose data, combined with a nonlinear learning unit that processes input data through full connection and activation mapping, allowing for parallel computation and adaptability to various hardware platforms without the need for model adjustments, thereby improving pose prediction accuracy and reducing deployment costs.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If traditional deep learning models with temporal structures (LSTM, GRU) are used for motion capture, then pose prediction accuracy is improved, but model complexity and deployment difficulty increase

Engineering Contradiction:

Improvepose prediction accuracyVSAvoidmodel structure complexity

Core Design Contradiction:

Measurement precisionVSDevice complexity

Solution Approach 1:

The model is segmented into distinct functional modules: temporal feature extraction module (LSTM/GRU), spatial feature extraction module (convolutional layers), and fusion module. This segmentation allows each module to be optimized independently for its specific function while maintaining overall accuracy.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The model architecture is designed to be hardware-agnostic and platform-independent, enabling universal deployment across heterogeneous hardware platforms including mobile devices, edge computers, and cloud servers without requiring platform-specific adaptations.

Inventive Principle:

Principle #6Universality (Multi-functionality)

2Measurement precision

If traditional deep learning models with temporal structures are used, then pose prediction accuracy is improved, but deployment cost increases

Engineering Contradiction:

Improvepose prediction accuracyVSAvoiddeployment cost

Core Design Contradiction:

Measurement precisionVSEase of manufacture

Solution Approach 1:

The model uses standardized neural network operations and data structures that can be executed on any hardware platform with basic computing capabilities, eliminating the need for platform-specific model adaptations and reducing deployment costs across diverse hardware environments.

Inventive Principle:

Principle #6Universality (Multi-functionality)

3Measurement precision

If LSTM temporal structures are used for pose prediction, then accuracy is improved, but adaptability to heterogeneous hardware platforms decreases

Engineering Contradiction:

Improvepose prediction accuracyVSAvoidhardware platform adaptability

Core Design Contradiction:

Measurement precisionVSAdaptability or versatility

Solution Approach 1:

Inventive Principle:

Principle #6Universality (Multi-functionality)

Solution Approach 2:

The model allows dynamic adjustment of temporal context window sizes and feature extraction parameters based on hardware capabilities, enabling the same model architecture to adapt its computational characteristics to different hardware platforms while maintaining accuracy.

Inventive Principle:

Principle #35Parameter changes

4Measurement precision

If deep models with temporal structures are used, then pose prediction capability is improved, but model inference time increases

Engineering Contradiction:

Improvepose prediction accuracyVSAvoidmodel inference time

Core Design Contradiction:

Measurement precisionVSLoss of time

Solution Approach 1:

The model extracts only the essential temporal features needed for accurate pose prediction using LSTM/GRU, rather than processing complete historical pose sequences. This selective extraction of critical temporal information reduces computational overhead while maintaining prediction accuracy.

Inventive Principle:

Principle #2Taking out (Extraction)

Solution Approach 2:

The model performs preliminary feature extraction and temporal pattern recognition in advance during the forward propagation phase, organizing data structures to facilitate efficient final prediction computation and reducing overall inference time.

Inventive Principle:

Principle #10Preliminary action

Data Source

PatentUS20250005784A1Method, apparatus, electronic device and storage medium of pose prediction

Publication Date: 2025.01.02 BEIJING ZITIAO NETWORK TECH CO LTD

AI summary

Embodiments of the disclosure provide a method, an apparatus, an electronic device, and a storage medium of pose prediction. The method includes: extracting temporal information of input data through a long short-term memory network, the temporal information including recent memory data, and the input data including pose data of a current frame of a target object collected by a pose sensor; generating first intermediate data by fully connecting and activation mapping the input data through a nonlinear learning unit; generating second intermediate data by stacking the recent memory data and the first intermediate data of the nonlinear learning unit through a stacked unit, the long short-term memory network, the nonlinear learning unit, and the stacked unit forming a basic learning module; and predicting a current pose of the target object based on the second intermediate data.