Audio Time Alignment for Pitch Correction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for pitch correction and vocal modification struggle to accurately align timing and apply nuanced pitch adjustments to amateur vocal recordings, often resulting in unnatural or out-of-tune performances, especially when dealing with varying accents, noise, or different vocal styles, and require prior phonemic information or real-time processing.
Innovation Solution
A method that establishes a detailed timing relationship between a new vocal performance and a guiding performance using a time-alignment function to apply feature adjustments correctly, allowing for the transfer of complex pitch variations and other characteristics from a professional singer to an amateur singer, without requiring prior information on phonemic nature or real-time processing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If pitch data from a guide signal is applied directly at the same relative times to a new signal, then the processing is simple and fast, but the pitch correction is inaccurate because timing discrepancies between corresponding features in the two signals are not accounted for
Solution Approach 1:
The patent applies preliminary time alignment by generating a time alignment function that maps corresponding features between the guide signal and new signal before pitch correction is applied. This preliminary timing adjustment ensures that pitch data is applied at the correct temporal positions, resolving the accuracy issue without requiring complex real-time processing during pitch correction
Solution Approach 2:
The patent introduces a time alignment function as an intermediary component that bridges the guide signal and new signal. This intermediary establishes the correct temporal correspondence between the two signals, allowing accurate pitch transfer while keeping the overall processing architecture manageable through modular design
2Ease of operation
If automatic pitch adjustment to fixed grids is applied, then real-time processing is enabled, but natural human pitch variations are removed resulting in unnatural performance
Solution Approach 1:
The patent applies local quality by transferring pitch characteristics locally from corresponding segments of the guide signal to the new signal, rather than applying global quantization to fixed grids. This preserves the natural, nuanced pitch variations present in the original performance while still providing automatic pitch correction functionality
Solution Approach 2:
The patent changes the approach from discrete grid-based pitch quantization to continuous pitch transfer by using time alignment to map pitch values directly from the guide signal. This parameter change allows natural pitch contours and human variations to be preserved while maintaining automatic processing capability
3Adaptability or versatility
If phoneme-based target pitch identification is used, then pitch correction can be applied, but prior phonemic information is required which limits applicability to different accents and languages
Solution Approach 1:
The patent extracts the time alignment function and pitch transfer mechanism from the phoneme-based processing framework. By removing the dependency on phoneme recognition and prior linguistic information, the system becomes applicable to different accents, languages, and even non-speech audio while maintaining effective pitch correction capability
4Productivity
If real-time processing is performed to correct pitch, then immediate feedback is provided, but processing quality is reduced compared to offline analysis
Solution Approach 1:
The patent performs preliminary time alignment and pitch characteristic extraction offline before the actual pitch correction application. This allows high-quality analysis and accurate timing relationship establishment without real-time constraints, while the actual pitch correction can then be applied efficiently with reduced computational burden
Data Source
AI summary
A digitised audio signal, such as an amateur's singing, and a digital guide audio signal are supplied to a time alignment process that produces a time-aligned new signal, time-aligned to the guide signal. Pitch along the time-aligned new signal and along the guide signal is measured in processes and which supply these measurement to a pitch adjustment calculator which calculates a pitch correction factor C's(Fps) from these measurements and the nearest octave ratio of the signals. A pitch changing process modulates the pitch of the time-aligned new signal to produce a time-aligned and pitch adjusted new signal.