Temporal Segmentation for Gesture Recognition
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Temporal segmentation of human motion for gesture recognition is challenging due to ambiguities in defining gesture boundaries, especially with continuous gestures and partial occlusions, and existing methods struggle to balance real-time processing with accuracy.
Innovation Solution
The Kinematic Kernelized Temporal Segmentation (KKTS) method uses 3D video streams to extract skeletal data, identify points of abrupt content change as temporal cuts, and classify segments with positive acceleration as gesture boundaries, employing the Kernelized Temporal Cut algorithm and Maximum Mean Discrepancy (MMD) to model the segmentation problem within sliding windows.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If temporal clustering is used for gesture segmentation, then global point of view and cluster labels are provided, but real-time processing capability is lost
Solution Approach 1:
The patent segments the temporal clustering problem into two independent parts: (1) offline temporal clustering to identify candidate cut points and establish global structure, and (2) online acceleration-based filtering to determine final gesture boundaries in real-time. This segmentation allows each part to optimize for its specific requirement without compromising the other.
Solution Approach 2:
The patent performs preliminary temporal clustering offline to pre-identify candidate cut points and establish the global temporal structure of gestures. This preliminary action provides a foundation for subsequent real-time processing, allowing the online stage to focus only on selecting among pre-computed candidates based on acceleration criteria.
2Productivity
If change-point detection methods are used, then real-time processing capability is maintained, but handling of multivariate series with non-parametric distributions is limited
Solution Approach 1:
The patent introduces acceleration as an intermediary metric that bridges the gap between simple univariate change-point detection and complex multivariate gesture analysis. By computing acceleration from skeletal joint positions and using it as the basis for cut point selection, the system maintains real-time processing while effectively capturing complex motion dynamics without requiring parametric assumptions.
3Measurement precision
If acceleration-based filtering is applied to temporal cuts, then gesture boundary precision is improved, but computational complexity increases
Solution Approach 1:
The patent extracts acceleration as a specific, computationally efficient metric from the full temporal cluster structure and uses it as the sole criterion for filtering candidate cut points. This extraction approach simplifies the decision-making process compared to analyzing multiple features simultaneously, reducing computational complexity while maintaining precision in gesture boundary detection.
Data Source
AI summary
A method, system and non-transitory computer readable medium are disclosed for recognizing gestures, the method includes capturing at least one three-dimensional (3D) video stream of data on a subject; extracting a time-series of skeletal data from the at least one 3D video stream of data; isolating a plurality of points of abrupt content change called temporal cuts, the plurality of temporal cuts defining a set of non-overlapping adjacent segments partitioning the time-series of skeletal data; identifying among the plurality of temporal cuts, temporal cuts of the time-series of skeletal data having a positive acceleration; and classifying each of the one or more pair of consecutive cuts with the positive acceleration as a gesture boundary.