Variable-Speed Video Beat Sync Using Audio-Visual Rhythm Points
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for creating variable-speed beat-synced videos often result in mismatches between movement and audio, leading to unstable beat-syncing effects, which degrade the audio-visual experience.
Innovation Solution
Determine visual and audio rhythm points by analyzing motion and audio information, aligning them on a time axis, and generate a variable speed curve to synchronize video speed with audio rhythm, using modules for rhythm point matching and speed-changing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If a same curve is used to repeatedly change the speed of an original video based on fixed-interval beats or simple rhythmic beat, then the video processing is simple and efficient, but the movement mismatches with audio and the beat-syncing effect becomes unstable
Solution Approach 1:
The patent changes the parameter of speed curve from a fixed same curve to a variable curve that adapts to different audio rhythms. The system determines audio rhythm points from audio features and visual rhythm points from motion information, then generates variable speed curves that pass through these rhythm points, allowing the speed profile to vary according to the actual audio-visual rhythm rather than repeating a predetermined curve.
Solution Approach 2:
The patent introduces dynamics by making the speed curve adaptive rather than static. The system dynamically determines rhythm points based on audio and visual analysis, and the speed curve is generated to pass through these dynamically determined points. This allows the beat-syncing to adapt to different songs and video contents, improving stability while maintaining efficiency through automated rhythm detection.
2Reliability
If automated rhythm point detection and variable speed curve generation are implemented, then the beat-syncing stability and audio-visual alignment are improved, but the device complexity and processing time increase
Solution Approach 1:
The system performs self-service by automatically detecting audio rhythm points from the audio signal and visual rhythm points from motion information in the video. The rhythm point determination module autonomously identifies these points without requiring manual input, and the variable speed curve generation module automatically creates the appropriate speed profile, reducing the need for complex manual configuration while maintaining high beat-syncing stability.
Solution Approach 2:
The patent replaces manual mechanical adjustment of speed curves with automated computational methods. Instead of manually setting speed parameters or using complex hardware controls, the system uses algorithmic detection of rhythm points and automated generation of variable speed curves, substituting mechanical complexity with software-based intelligent processing that achieves better results with comparable or reduced overall system complexity.
Data Source
AI summary
A method of video processing, a device, and a storage medium are provided. The method includes: determining a visual rhythm point according to motion information of a target object in an original video; determining an audio rhythm point according to audio information corresponding to the original video; determining a target visual rhythm point that matches the audio rhythm point according to the timestamp corresponding to the audio rhythm point, and forming a rhythm point pair according to the audio rhythm point and the corresponding target visual rhythm point; and determining a target variable speed curve according to audio rhythm points and target visual rhythm points in adjacent rhythm point pairs, and changing a video speed of the original video according to the target variable speed curve to obtain a target video.