Cross-View 2D Pose Matching for Accurate 3D Reconstruction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods struggle to accurately match 2D skeletal data from multiple viewpoints to ensure correct grouping and 3D reconstruction of human poses, particularly in scenes with multiple individuals, due to the lack of correlation in indices or IDs across different views.
Innovation Solution
A method and system that utilizes a scoring module to assign affinity scores to cross-view 2D skeleton pairs, a matching module to optimize pair-wise matches, a grouping module to assign each skeleton to a unique person, and a temporal consistency module to maintain person correspondence across video sequences, leveraging calibrated cameras and deep neural networks for 2D pose estimation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If 2D human poses from multiple views are matched using conventional 2D image processing methods, then the matching process is simple and fast, but the accuracy of pose matching deteriorates due to inability to capture 3D spatial relationships
Solution Approach 1:
The patent transforms 2D pose matching into 3D pose matching by introducing depth information and spatial relationships. The system estimates 3D pose parameters (position, orientation, scale) from multiple 2D views and performs matching in 3D space, thereby resolving the contradiction between maintaining simplicity and improving accuracy through dimensional enhancement.
Solution Approach 2:
The patent introduces an intermediary 3D pose estimation model that bridges 2D image data and 3D pose matching. This intermediary layer processes multiple 2D views to generate 3D pose parameters, enabling accurate 3D pose matching without directly complexifying the original 2D matching system.
2Measurement precision
If 3D pose matching is implemented to improve accuracy, then pose matching accuracy improves, but computational complexity and processing time increase
Solution Approach 1:
The patent performs preliminary 3D pose estimation from multiple 2D views before conducting the actual pose matching. By pre-computing 3D pose parameters (position, orientation, scale) and normalizing them, the system reduces the computational burden during the matching phase, thereby reducing processing time while maintaining accuracy.
Solution Approach 2:
The patent changes the parameter space from 2D coordinates to 3D pose parameters (position, orientation, scale). This parameter transformation enables more efficient matching by working with normalized 3D representations that capture essential spatial relationships without requiring exhaustive 2D coordinate comparisons.
3Measurement precision
If multiple 2D views are processed to estimate 3D pose, then pose matching accuracy improves, but the complexity of data processing increases
Solution Approach 1:
The patent employs a universal 3D pose estimation model that processes multiple 2D views simultaneously to generate unified 3D pose parameters. This multi-functional approach consolidates the processing of different views into a single estimation framework, reducing data processing complexity while improving accuracy through comprehensive view integration.
4Measurement precision
If 3D pose parameters are used for matching, then spatial relationships are accurately captured, but the difficulty of detecting and measuring increases
Solution Approach 1:
The patent introduces an intermediary 3D pose estimation model that simplifies the detection and measurement of 3D spatial relationships. This intermediary layer processes multiple 2D views to generate 3D pose parameters, making the complex task of 3D detection more manageable while maintaining high spatial relationship accuracy.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
This disclosure is directed to a method and system for matching human pose data in the form of 2D skeletons for the purposes of 3D reconstruction. The system may comprise a scoring module that assigns an affinity score to each pair of cross-view 2D skeletons, a matching module that assigns optimal pairwise matches based on the affinity scores, a grouping module that assigns each 2D skeleton to a group such that each group corresponds to a unique person, based on the pairwise matches; and a temporal consistency module that assigns each group an ID that maintains correspondence to the same person over the multi-video sequence.