3D Multi-Skeleton Tracking with Temporal Loss Metric
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional skeleton tracking techniques in 3D volumetric sports video analysis face challenges such as occlusion, deformation, and similar appearances, leading to low-quality image tracking and misidentification of athletes, especially when athletes wear the same uniform and overlap in views, due to limitations in 2D-based algorithms and lack of robust 3D skeleton tracking methods.
Innovation Solution
A 3D multi-skeleton tracking system using a temporal object key point loss (TOKL) metric, which factors historical position data and spatial differentiation, combined with a Kalman filter for prediction and the Hungarian algorithm for matching, to accurately track and identify athletes in real-time, even under challenging conditions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If conventional 2D-based skeleton tracking algorithms are used, then the system is simpler to implement, but tracking accuracy deteriorates under occlusion, deformation, and overlapping athletes
Solution Approach 1:
The patent transitions from 2D image-based skeleton tracking to 3D volumetric skeleton tracking. By utilizing three-dimensional camera arrays and volumetric reconstruction, the system tracks skeletons in 3D space rather than on 2D images, enabling accurate tracking even when athletes occlude each other or deform their poses, thus resolving the accuracy-complexity contradiction through dimensional elevation
Solution Approach 2:
The patent segments the tracking problem into multiple independent skeleton tracks, each maintained as a separate entity with its own history and state. This allows individual skeleton identification and tracking even when they overlap in the camera view, improving accuracy without requiring complex global re-identification algorithms
2Adaptability or versatility
If athletes wear the same uniform and have similar appearance, then the system achieves uniformity and simplicity in visual presentation, but automatic distinction and identification of athletes becomes difficult
Solution Approach 1:
The system segments athletes into distinct tracked entities by assigning unique identifiers and maintaining separate skeleton histories, enabling the system to distinguish between athletes with identical appearances. Each athlete's trajectory and pose history are independently tracked, allowing accurate identification even when visual appearance is uniform
Solution Approach 2:
The system performs preliminary tracking of skeleton positions and trajectories across multiple frames before final identification. By establishing temporal continuity and spatial trajectories in advance, the system can reliably identify and distinguish athletes even when their appearance is identical, using historical position data to maintain distinction
3Productivity
If conventional tracing algorithms based on 2D bounding boxes are used, then the processing is faster and simpler, but 3D skeleton-based tracking accuracy is insufficient
Solution Approach 1:
The patent implements 3D skeleton tracking algorithms that operate on volumetric data and three-dimensional camera coordinates. This dimensional transition enables accurate 3D pose estimation and skeleton tracking while maintaining processing efficiency through optimized 3D spatial algorithms, resolving the contradiction between processing speed and 3D accuracy
4Adaptability or versatility
If occlusions and deformations are present in image data, then the system can handle realistic athletic movements, but conventional algorithms produce low quality images and tracking results
Solution Approach 1:
By moving to 3D volumetric tracking, the system can reconstruct athlete positions and poses in three-dimensional space even when parts of the body are occluded or deformed in 2D camera views. The 3D spatial relationships provide geometric constraints that enable accurate skeleton tracking despite occlusion and deformation, maintaining image quality while handling realistic movements
Data Source
AI summary
A method and system of image processing with multi-skeleton tracking uses a temporal object key point loss metric.