Video Replay Branching with AI Pose Transition Synthesis
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video processing technologies struggle to create non-linear replay branches in video sequences without causing subjectively disturbing pose discontinuities, requiring actors to physically revert to previous poses, which is difficult and often unsuccessful.
Innovation Solution
A method and apparatus that detect reference and second poses in a video sequence, generate a transitional video sequence for smooth transitions between these poses, and associate it with the input sequence to create a non-linear replay branch, using techniques like chroma key compositing, skeletal pose detection, and artificial intelligence to ensure continuous poses.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If actors physically revert to previous poses to create non-linear replay branches, then pose continuity is improved, but ease of operation deteriorates
Solution Approach 1:
The system creates a digital copy of the actor's pose data from video frames and uses this copy to generate transitional sequences. Instead of requiring the actor to physically repeat poses, the system captures pose information once and reuses it to create smooth transitions between different points in the video sequence, thereby maintaining pose continuity while eliminating the difficulty of physical pose reversion.
Solution Approach 2:
The patent replaces the mechanical/physical system of actors physically returning to poses with a computational system. Pose detection algorithms extract skeletal information from video frames, and software generates transitional sequences that mathematically interpolate between poses. This substitution of mechanical action with computational processing resolves the contradiction by achieving pose continuity without requiring difficult physical performance.
2Reliability
If transitional sequences are generated to smooth pose transitions, then pose continuity is improved, but device complexity increases
Solution Approach 1:
The system segments the video processing task into distinct functional modules: pose detection module that extracts skeletal information from video frames, sequence generation module that creates transitional sequences, and composition module that integrates sequences. This segmentation allows each module to be optimized independently and facilitates parallel processing, thereby managing system complexity while achieving smooth pose transitions.
Solution Approach 2:
The patent introduces skeletal pose data as an intermediary representation between the original video frames and the final transitional sequences. Rather than directly manipulating video pixels, the system works with simplified skeletal representations that capture essential pose information. This intermediary layer simplifies the overall processing complexity by working with abstracted pose data rather than full video complexity.
Data Source
Figure 1~3
Figure 4~9
Figure 10a~12
AI summary
A video processing method comprises detecting, as a reference pose, a pose of an individual at a reference time point in an input video sequence; at a second, different, time point in the input video sequence, detecting a second pose of the individual; generating from one or more source images of the individual, a transitional video sequence representing a transition of the individual from the second pose to the reference pose; and associating the transitional video sequence with the input video sequence to generate an output video sequence including at least the transitional video sequence to implement a non-linear replay branch from the second time point to the reference time point.