Mobile Device Image Processing for Real-Time Pose Estimation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional image processing techniques for live video feeds on mobile devices face challenges in efficiently detecting and tracking dynamic 2D targets, such as video screens, due to the high computational effort required for real-time pose estimation in rapidly changing video streams.
Innovation Solution
The proposed method involves storing data derived from selected frames of a video sequence in a database, allowing for efficient pose estimation and tracking by using initial pose estimation to iteratively calculate subsequent poses, thereby reducing the computational load and enabling real-time augmentation of live video feeds with virtual information.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional image processing techniques are used for real-time pose estimation in live video feeds, then measurement precision of camera pose is improved, but device complexity and computational load increase significantly
Solution Approach 1:
The patent pre-processes video sequences to extract key frames and their corresponding pose information before runtime. This preliminary action creates a database of pre-computed pose data that can be quickly retrieved during live video processing, avoiding the need to perform complex pose estimation algorithms in real-time while maintaining accuracy
Solution Approach 2:
The patent segments the video processing task into two parts: (1) offline pre-processing to extract and store key frame pose information, and (2) online real-time processing to match live frames against the pre-processed database. This segmentation reduces the computational burden during real-time operation while preserving measurement precision
2Reliability
If pose estimation is performed for every frame in rapidly changing video streams, then reliability of tracking is improved, but productivity and processing speed decrease
Solution Approach 1:
Instead of performing full pose estimation on every frame, the patent applies partial action by only performing pose estimation on selected key frames and using those results to track and infer pose information for intermediate frames. This reduces the number of computationally intensive operations while maintaining sufficient tracking reliability through interpolation and temporal coherence
Solution Approach 2:
The patent establishes continuity by using pose information from previously processed frames to inform and constrain pose estimation in subsequent frames. This temporal continuity allows the system to maintain reliable tracking with reduced computational effort by leveraging the smooth transitions between consecutive frames rather than treating each frame independently
3Loss of information
If data from all frames of a video sequence are stored in the database, then completeness of information is improved, but loss of time and processing overhead increase
Solution Approach 1:
The patent extracts only the essential and representative information from video frames—specifically key frames and their associated pose data—and stores only this extracted information in the database. This selective extraction maintains sufficient information completeness for pose estimation while dramatically reducing the volume of data that needs to be processed and stored, thereby reducing time loss
Data Source
AI summary
For an image processing concept, a database including data derived from a plurality of frames of a video is provided in a device. A live video feed is obtained from a camera of the device. Information is extracted from an image of the video feed. A search is performed in the database using the extracted information to retrieve a list of potential frames out of the plurality of frames. An initial pose of the selected image is estimated with respect to one frame of the list as a function of the extracted information and the data derived from the one frame. Respective subsequent poses for subsequent images from the live video feed are iteratively estimated, wherein the associated subsequent pose is estimated based on said subsequent image and a respective previously estimated pose. The video feed is augmented on the device with virtual information based on the estimated initial pose and the subsequent poses.