Music-Driven Choreography Generation with Dynamic Segment Matching
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for generating virtual dance choreography based on music are costly due to the need for professional choreographers and motion-capture equipment, and existing data sets are noisy and limited, leading to inaccurate results.
Innovation Solution
A system and method using dynamic programming and music feature clusters to generate choreography, employing music labels, dance segment selection based on similarity, and transition motions between segments, utilizing a dance database and SMPL skeleton models.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If professional choreographers and motion-capture equipment are employed to create virtual dance, then motion accuracy is improved, but cost increases significantly
Solution Approach 1:
The patent uses copying by creating a database of dance movements extracted from professional choreography videos and motion-capture data. Instead of requiring expensive real-time motion capture for every virtual dance performance, the system copies and reuses pre-extracted dance movement sequences from existing professional performances, significantly reducing costs while maintaining quality
Solution Approach 2:
The patent applies segmentation by dividing the dance performance into discrete movement segments and clips that can be independently stored, retrieved, and recombined. This allows the system to break down complex dance routines into manageable units that can be efficiently managed in a database and reconstructed for virtual performances without requiring full re-capture
2Extent of automation
If neural network-based frameworks are used to generate choreography from music, then automation is improved, but data requirements and complexity increase
Solution Approach 1:
The patent applies dynamics by implementing a flexible search and selection mechanism that dynamically adapts to different music inputs. The system adjusts the search criteria, similarity thresholds, and segment selection based on the specific music characteristics, allowing the same framework to handle diverse musical styles and tempos without requiring complex reconfiguration
Solution Approach 2:
The patent uses an intermediary approach by introducing a music-dance similarity measurement module that acts as a mediator between music input and dance generation. This intermediary layer analyzes music features and maps them to corresponding dance segments, simplifying the overall system architecture by breaking down the complex task into manageable stages
3Quantity of substance
If existing motion datasets are used for choreography generation, then data availability is improved, but data quality and consistency deteriorate due to noise and varying skeletal models
Solution Approach 1:
The patent applies extraction by isolating and separating the essential dance movement information from the noisy video data. The system extracts clean movement sequences, pose sequences, and transition information from existing dance videos, removing artifacts and inconsistencies while preserving the core dance choreography for reliable virtual performance generation
4Measurement precision
If dance segments are selected based on music feature similarity, then synchronization accuracy is improved, but computational time increases
Solution Approach 1:
The patent applies preliminary action by pre-computing and storing music-dance similarity metrics during the database construction phase. Instead of calculating similarities in real-time during performance generation, the system pre-processes and stores pre-aligned dance segments with their corresponding music features, enabling fast retrieval and recombination during actual choreography generation
Data Source
AI summary
A method for generating a music-driven choreography based on the music feature clusters and dynamics, the method including the steps of: receiving a segment of input music; extracting a plurality of music features from the input music; generating a plurality of music segments based on musical beats; assigning music labels for each music segment; selecting a dance segment for each music segment by label similarity and an optimization process; and generating the music-driven choreography by combining the dance segments.