3D Skeleton Reconstruction via Volumetric Probability Merging
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for 3D skeleton reconstruction in image-based rendering are inefficient in terms of performance, memory usage, and ability to detect multiple 3D real-world objects in a scene, particularly in real-time applications such as live broadcasts of sport events.
Innovation Solution
A method and system for reconstructing 3D skeletons from source images that involve generating part maps, projecting elementary voxels onto these maps, and computing joint part probabilities to reduce complexity and enhance robustness, allowing for real-time reconstruction of 3D skeletons without the need for 2D skeleton generation and conflict resolution.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional 2D skeleton generation and processing methods are used, then the reconstruction process is simpler, but the performance is insufficient in terms of memory use, processing time, and ability to detect multiple 3D objects
Solution Approach 1:
The patent transitions from 2D skeleton generation to direct 3D skeleton reconstruction by projecting elementary voxels of the scene volume onto 2D part maps and computing joint part probabilities in 3D space. This dimensional change enables real-time processing and multiple object detection while maintaining reconstruction accuracy, resolving the contradiction between processing speed and detection capability versus method complexity
Solution Approach 2:
The patent introduces part maps as intermediary structures that store part probabilities for respective pixels. These part maps serve as mediators between the 3D scene volume and the final 3D skeleton reconstruction, enabling efficient probability computation and reducing memory usage compared to traditional methods that process complete 2D skeletons
2Reliability
If 2D skeletons are generated and processed, then the initial processing is easier, but conflicts between 2D skeletons that do not perfectly match in 3D space must be resolved
Solution Approach 1:
Instead of generating 2D skeletons and then attempting to reconcile them in 3D space, the patent inverts the approach by directly computing 3D skeleton probabilities from 3D voxel projections onto 2D part maps. This inversion eliminates the conflict resolution problem entirely by working natively in 3D space from the beginning, thereby improving reliability without adding complexity
Solution Approach 2:
The patent replaces the mechanical conflict resolution process with a probabilistic computation system. By computing joint part probabilities directly in 3D space based on voxel projections, the system naturally handles multiple objects and occlusions without requiring explicit conflict detection and resolution mechanisms
3Quantity of substance
If traditional methods are used, then the implementation is straightforward, but the ability to detect a maximum number of 3D real world objects in the scene is limited
Solution Approach 1:
The patent segments the scene volume into elementary voxels and processes them individually through projection onto part maps. This segmentation enables parallel processing of multiple objects simultaneously while maintaining processing efficiency, as each voxel can be independently evaluated for its contribution to the 3D skeleton reconstruction
Solution Approach 2:
The patent changes the fundamental parameters of the reconstruction process by working with probabilities rather than deterministic 2D skeleton points. By computing joint part probabilities for each voxel and aggregating them in 3D space, the system can detect and reconstruct multiple overlapping and occluded objects simultaneously without sacrificing processing efficiency
Data Source
AI summary
To generate 3D skeletons of a scene volume, a first approach retrieves probabilities from source images capturing the scene to detect parts of the objects, next merges the probabilities in 3D space. The merged or joint probabilities are then used to robustly detect the parts of the objects in the 3D space. This approach advantageously reduces complexity of the 3D skeleton reconstruction. In particular, at 2D level (i.e. on the source images), it avoids conflict resolutions to be performed between conflicting parts detected from different source images. It turns that real time reconstructions and subsequent displays or human motion detection are better achieved. The present approach efficiently works on complex scenes like sport events with multiple players in a stadium, with an ability to detect a wide number of interoperating 3D objects like multiple players.