Polyphonic Audio Note Extraction and Resynthesis
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current sound post-processing technologies are unable to effectively recognize and manipulate individual notes within polyphonic sound recordings, limiting their ability to edit complex musical content such as chord instruments without altering the original sound impression.
Innovation Solution
A method that identifies and extracts note objects from polyphonic audio material, allowing for manipulation of individual notes in terms of pitch and time without distorting the original sound, enabling users to change specific notes within a chord without affecting others, and resynthesizing the audio signal with the modified notes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If traditional sound post-processing methods are used, then the processing is simple and computationally efficient, but the ability to recognize and manipulate individual notes in polyphonic recordings is lost
Solution Approach 1:
The patent segments the polyphonic audio signal into individual note components using spectral filtering. Each note is separated into its fundamental frequency and harmonic components, allowing independent manipulation of individual notes within chords while maintaining the overall polyphonic structure.
Solution Approach 2:
The patent introduces an intermediary representation layer between the raw audio signal and the processing operations. This intermediate representation includes note-level spectral components that serve as a bridge, enabling note-specific manipulations while preserving the connection to the original audio signal for high-quality resynthesis.
2Measurement precision
If individual notes are extracted and manipulated separately, then precise note editing is enabled, but the original sound impression may be distorted
Solution Approach 1:
The patent performs preliminary analysis to identify and separate note components before manipulation. By pre-separating the spectral components of individual notes and maintaining their relationship to the original signal, the system enables precise editing while preserving the authentic sound characteristics during resynthesis.
Solution Approach 2:
The patent uses feedback mechanisms to maintain sound quality fidelity. The resynthesis process incorporates information from the original recording to ensure that manipulated notes retain their authentic timbre and characteristics, comparing the processed output against the original sound impression and adjusting to preserve fidelity.
3Ease of operation
If chord instruments are processed as whole chords, then the processing is straightforward, but individual note manipulation within chords is impossible
Solution Approach 1:
The patent segments chord recordings into individual note components within each chord. By separating the spectral content of each note in a chord while maintaining their temporal and harmonic relationships, the system enables users to access and manipulate individual notes (e.g., changing a single chord tone from E to Eb) while keeping the rest of the chord intact.
Solution Approach 2:
The patent introduces dynamic processing capabilities that allow the system to switch between treating chords as unified structures and individual note components. This dynamic approach enables flexible manipulation where users can choose to edit entire chords or individual notes within chords, adapting the processing granularity to the specific editing task.
Data Source
Figure 1~2
Figure 3~4
Figure 5~6
AI summary
The method involves reading out the time signal section-wise by a window function and an overlapping window. The Fourier transformation of the readout signal is executed in a frequency chamber, particularly by executing a discrete Fourier transformation. An energy value for each bin is calculated from the frequency amplitude, which results from Fourier transformation, particularly by squaring the real and imaginary parts or creating a derived energy value. An independent claim is included for a computer program with a program code for executing the method.