Automatic Performance System Synchronization via Sign Action Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing automatic performance technologies struggle to replicate the natural ensemble effect by synchronizing multiple performers with an automatic performance device, as they simply follow the performance without mutual recognition of behaviors, leading to delayed sound generation and lack of synchronization.
Innovation Solution
An automatic performance system comprising a sign detector, performance analyzer, and performance controller that synchronizes the automatic performance with detected sign actions and estimated performance positions, using machine learning to analyze acoustic and image signals for real-time synchronization and dynamic display adjustments.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the automatic performance device simply follows the performance of a musical instrument, then the synchronization with the performer is maintained, but the naturalness of the ensemble is degraded because mutual recognition of behaviors is lost
Solution Approach 1:
The system detects sign actions from the performer through imaging devices and uses this feedback to control the automatic performance device. The performance position estimator continuously monitors the acoustic signal and provides feedback to adjust the automatic performance timing, creating a closed-loop system that enables mutual recognition and natural ensemble behavior
Solution Approach 2:
The control device acts as an intermediary between the performer's sign actions and the automatic performance device. It processes imaging signals to detect sign actions, estimates performance positions from acoustic signals, and translates these into control signals for the automatic performance device, enabling indirect but coordinated interaction
2Speed
If the automatic performance device generates sound in real-time, then the responsiveness is improved, but the synchronization is degraded due to delayed sound generation
Solution Approach 1:
The system estimates the performance position in advance by analyzing the acoustic signal and detecting sign actions before the actual sound generation is needed. This preliminary estimation allows the automatic performance device to be triggered at the correct moment, compensating for processing delays and maintaining synchronization
Solution Approach 2:
The system dynamically adjusts the timing and triggering of the automatic performance device based on real-time detection of sign actions and continuous estimation of performance position. This dynamic control allows the system to adapt to varying performance speeds and maintain synchronization despite variable processing delays
Data Source
AI summary
An automatic performance system includes a sign detector configured to detect a sign action of a performer performing a musical piece, a performance analyzer configured to sequentially estimates a performance position in the musical piece by analyzing an acoustic signal representing performed sound in parallel with the performance, and a performance controller configured to control an automatic performance device to carry out an automatic performance of the musical piece so that the automatic performance is synchronized with the sign action detected by the sign detector and a progress of the performance position estimated by the performance analyzer.