Music Haptic Feedback Using Split-Track Weighting for Rhythm Matching
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing haptic feedback technologies fail to accurately generate vibrations that match the cadences and rhythms of music, particularly for music with slow rhythms, and do not account for varying intensity levels of vibration based on musical dynamics.
Innovation Solution
A haptic feedback method utilizing a deep learning model to split audio data into tracks, calculate energy proportions, apply weighting rules to generate a time-frequency spectrum, and output a matched vibration signal to achieve precise haptic feedback.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If vibration is generated based on rhythmic instruments such as drumbeats, then vibration feedback can be generated for music with strong rhythm, but the method is not applicable to music with slow rhythms and cannot generate vibrations with different intensity levels
Solution Approach 1:
The patent segments the music audio into multiple tracks (drum track, bass track, melody track, etc.) based on frequency characteristics. Each track is processed independently to extract rhythmic features, allowing the system to handle different music styles and instruments effectively. This segmentation enables accurate vibration matching for both fast and slow rhythms by analyzing each track's specific characteristics.
Solution Approach 2:
The patent changes the parameter of rhythm detection from relying solely on beat frequency to incorporating energy proportion and temporal pattern analysis. By calculating the energy proportion of each track and analyzing the temporal patterns of rhythmic elements, the system can accurately detect rhythms in music with slow tempos and generate appropriate vibration intensity levels that match the musical dynamics.
2Ease of operation
If only rhythmic instruments are used for vibration generation, then vibration feedback is simple to implement, but the user experience is limited and does not reflect the dynamics of different rhythms
Solution Approach 1:
The patent introduces dynamic vibration generation by calculating the energy proportion of each audio track and using it to modulate the vibration intensity. The system dynamically adjusts vibration parameters based on the energy distribution across different tracks, allowing the vibration feedback to reflect the musical dynamics and intensity variations. This enables the system to generate vibrations with different intensity levels that match the rhythm dynamics of the music.
Data Source
AI summary
A haptic feedback method, system and related device for matching split-track music to vibration are provided. The method includes: acquiring raw audio data; obtaining multiple split-track audio data by splitting the raw audio data by using a predetermined deep learning model; calculating an energy proportion corresponding to a respective split-track audio data in the raw audio data; determining a weight of the respective split-track audio data according to the energy proportion; performing weighting calculation on the multiple split-track audio data according to a predetermined weighting rule, to obtain a time-frequency spectrum; generating a matched vibration signal corresponding to the raw audio data based on the time-frequency spectrum; and outputting a haptic feedback effect according to the matched vibration signal. The audio data can be matched to vibration, and a vibration feedback that more precisely matches cadences and rhythms of the audio data can be output.