Video Playback Synchronization with User Motion Tracking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video playback methods for training activities like yoga, dance, and physical therapy are inefficient, as users struggle to navigate back to specific points without losing focus, leading to discouragement and hindered skill advancement due to awkward controls and high development costs of computer-generated graphics.
Innovation Solution
A computer-implemented method that synchronizes video playback with real-time user movement by tracking joint positions and applying a matching function to determine playtime, allowing users to control video playback without manual intervention, using visual feedback indicators like trails and arrows to guide proper movement.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If users manually control video playback (rewind, scrub) to review operations, then users can access desired video points, but users lose focus from training activity and experience becomes awkward
Solution Approach 1:
The system automatically controls video playback by detecting user pose changes and autonomously navigating to relevant video timestamps, eliminating the need for manual user intervention and allowing users to maintain focus on training without holding remote controls or issuing commands
Solution Approach 2:
The system continuously monitors user pose data and provides real-time feedback by automatically advancing or rewinding video playback based on detected pose changes, creating a closed-loop system where user actions directly influence video progression without requiring explicit user commands
2Ease of operation
If users use voice commands or gestures to control video playback, then users can navigate video without manual controls, but users still lose focus from training activity
Solution Approach 1:
The video playback system autonomously controls itself by detecting user pose changes through computer vision and automatically navigating to corresponding video timestamps, eliminating the need for separate control mechanisms like voice commands or gestures
Solution Approach 2:
The system merges the training activity monitoring function with the video playback control function, using the same pose detection data for both training feedback and automatic video navigation, thereby eliminating the need for separate control mechanisms
3Adaptability or versatility
If computer-generated graphics or avatars are used for training, then training content can be customized, but development costs become substantially high
Solution Approach 1:
The system uses photorealistic video recordings of actual trainers performing exercises as the training content source, copying real human movements and appearances rather than creating custom computer-generated avatars, thereby achieving realism without high development costs
Solution Approach 2:
The system uses a single photorealistic video recording that serves multiple functions: providing visual training guidance, enabling pose detection through computer vision, and allowing automatic playback control, eliminating the need for separate expensive avatar development
Data Source
AI summary
A computer-implemented method is provided for coordinating sensed poses associated with real-time movement of a first object in a live video feed, and pre-recorded poses associated with movement of a second object in a video. The computer-implemented method comprises applying a matching function to determine a match between a point of one of the sensed poses and a corresponding point of the pre-recorded poses, and based on the match, determining a playtime and outputting at least one frame of the video associated with the second object for the playtime.