3D Human Motion Capture Through Dynamic Video Synchronization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current motion capture methods from unsynchronized videos face challenges due to depth ambiguity and self-occlusion, limiting accuracy, and traditional multi-view reconstruction algorithms are ineffective when cameras capture different scenes with inconsistent motions.
Innovation Solution
A method that utilizes multiple unsynchronized videos, employing deep neural networks to predict 3D human poses, synchronizes videos through similarity matrix construction and dynamic programming, optimizes camera poses and human motions to minimize re-projection errors, and models motion differences using a low-rank matrix to enhance accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional multi-view reconstruction algorithms are used, then reconstruction accuracy is improved, but the method can only be applied when all cameras record the same scene with consistent motions
Solution Approach 1:
The patent introduces dynamic video synchronization that adapts to different motion patterns in unsynchronized videos. The system dynamically adjusts the synchronization process by calculating frame correspondence based on motion consistency, allowing the multi-view reconstruction to handle variable motion conditions across different video sources while maintaining reconstruction accuracy.
Solution Approach 2:
The patent changes the parameter representation by introducing motion consistency parameters and frame correspondence relationships. By transforming the problem from direct multi-view reconstruction to a synchronized reconstruction framework with adjustable parameters, the system can accommodate unsynchronized videos with different motions while preserving the accuracy benefits of multi-view geometry.
2Adaptability or versatility
If single-view motion capture algorithm is used, then the method can handle unsynchronized videos, but depth ambiguity and self-occlusion problems reduce accuracy
Solution Approach 1:
The patent merges the advantages of single-view and multi-view approaches by combining unsynchronized video inputs with a synchronized multi-view reconstruction framework. This integration allows the system to process diverse video sources while utilizing multi-view geometry to resolve depth ambiguity and self-occlusion, thereby improving motion capture accuracy.
Solution Approach 2:
The patent introduces video synchronization as an intermediary process that bridges unsynchronized video inputs and the multi-view reconstruction algorithm. By establishing frame correspondence relationships and motion consistency checks, the synchronization module acts as a mediator that enables accurate multi-view reconstruction from unsynchronized sources without directly confronting the depth ambiguity and self-occlusion issues.
3Reliability
If video synchronization is performed using similarity matrix and dynamic programming, then correspondence between frames is established, but computational complexity increases
Solution Approach 1:
The patent performs preliminary video synchronization before the main multi-view reconstruction process. By pre-establishing frame correspondence relationships and filtering inconsistent frames in advance, the system reduces the computational burden during the subsequent reconstruction phase while maintaining high correspondence accuracy through the similarity matrix and dynamic programming approach.
Data Source
AI summary
Disclosed is a human motion capture method based on unsynchronized videos, which can effectively recover the 3D motion of a target person through multiple unsynchronized videos of the person. In order to utilize multiple unsynchronized videos, the present disclosure provides a video synchronization and motion reconstruction method. The present disclosure is implemented in the following steps: synchronizing multiple videos based on a 3D human pose; performing motion reconstruction based on synchronized videos, modeling the motion difference across different viewpoints by using the low-rank constraint to realize high-precision human motion capture from the plurality of unsynchronized videos. According to the present disclosure, more accurate motion capture is carried out by using multiple unsynchronized videos.