Motion Graph Path Search for Music-Synchronized Video Generation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video content generation systems face difficulties in creating synchronized computer graphics (CG) animations with musical tunes, particularly when dealing with large amounts of motion data and real-time synchronization with streaming music, as they require inputting the entirety of each musical tune from beginning to end and are prone to artificial or incoherent motions due to abrupt changes.
Innovation Solution
A video content generation system that utilizes a motion graph with nodes, edges, and weights calculated based on motion characteristics, along with metadata indicating synchronization probability, to search for an optimum path on the motion graph matching musical characteristics, allowing for synchronized video generation without requiring complete music data input, and adjusts for changes in musical phases or climaxes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If motion graphs are created using enormous amounts of motion data, then the comprehensiveness of motion database is improved, but the calculation complexity and likelihood of artificial motions increase
Solution Approach 1:
The patent extracts only the necessary motion data relevant to the current music track from the large motion database, rather than processing all available motion data. This selective extraction reduces calculation complexity while maintaining motion diversity through the motion graph structure.
Solution Approach 2:
The motion database is segmented into a motion graph structure where motions are organized as nodes and edges with weights. This segmentation allows the system to process motion data in a structured, manageable way that reduces overall calculation complexity while preserving comprehensive motion information.
2Measurement precision
If complete music data is input from beginning to end, then the synchronization accuracy is improved, but the real-time processing capability deteriorates
Solution Approach 1:
The system performs preliminary action by pre-building the motion graph structure with motion characteristics and weights before actual synchronization processing. This preparation enables real-time music streaming processing without requiring complete music data input, as the motion graph is ready to accommodate incoming music data progressively.
Solution Approach 2:
The system dynamically adjusts the motion selection process based on incoming music data streams. The motion graph can be updated and reinterpreted as new music data arrives, allowing the system to maintain synchronization accuracy while processing music in real-time without requiring complete track input beforehand.
3Reliability
If motion data is selected based on specific musical tune, then the relevance of motion data is improved, but the adaptability to different music styles deteriorates
Solution Approach 1:
The motion graph structure serves as a universal framework that can accommodate multiple music styles and types. By organizing motion data in a generalizable graph structure with weighted edges representing motion characteristics, the system can adapt to different music styles without requiring style-specific processing, thus maintaining both relevance and versatility.
Data Source
AI summary
A video content generation device generates video data synchronized with music data based on motion data, representing a motion graph including nodes, edges, and weights, and metadata indicating a synchronization probability per each node between the motion graph and a musical tune. A music data storage unit stores the predetermined amount of music data and their musical characteristics in connection with the predetermined number of beats, in a reproduction order, retrieved from the musical tune. An optimum path search unit searches an optimum path connecting between nodes, each of which is selected per each beat with a high synchronization probability, on the motion graph with motion characteristics matching with musical characteristics based on the predetermined amount of music data. Video data synchronized with music data is generated based on synchronization information for correlating motion data to music data along with the optimum path.