Dynamic Pitch Correction for Live Audio Latency and Accuracy
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing pitch correction systems face issues such as incorrect note selection when a singer misses the target by more than half a semitone, pitch deviation causing note jumping or oscillation, and significant time delays that can be distracting during live performances, especially when using a chromatic scale or when local key changes occur within a song.
Innovation Solution
The system dynamically adjusts the mapping between input and output notes based on detected note likelihoods and stability, allowing for adaptive pitch shifting with reduced latency by adjusting note boundaries and delay in response to vocal input and accompaniment signals, enabling musically correct output notes even when the local key differs from the global key without requiring user input or a guide track.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If real-time pitch correction processing is used to reduce delay, then processing speed is improved, but pitch accuracy deteriorates due to insufficient look-ahead time
Solution Approach 1:
The system performs preliminary pitch detection and note identification on incoming audio frames before final pitch correction is applied. By pre-identifying the target notes and building a look-ahead buffer of detected notes, the system prepares correction data in advance, allowing fast real-time processing without sacrificing accuracy.
Solution Approach 2:
The system dynamically adjusts the look-ahead buffer size and processing parameters based on the current musical context and detected note stability. When notes are stable and well-defined, the system can use smaller buffers for faster response. When notes are ambiguous or in transition, larger buffers provide more context for accurate pitch determination.
2Measurement precision
If a chromatic scale is used for pitch correction, then pitch resolution is improved, but note selection accuracy deteriorates when singers miss target notes by more than half a semitone
Solution Approach 1:
The system introduces an intermediary note probability calculation step between raw pitch detection and final note selection. Instead of directly mapping detected pitches to chromatic scale notes, the system calculates probabilities for each possible target note based on the detected pitch, musical context, and scale constraints. This intermediary layer resolves ambiguity when pitches fall between expected notes.
Solution Approach 2:
The system uses feedback from the accompaniment signal and previously detected notes to inform target note selection. By analyzing the harmonic context from the accompaniment and the sequence of previously detected notes, the system can predict the most likely target note even when the detected pitch is ambiguous or slightly off-target.
3Stability of the object's composition
If a fixed scale with fewer notes is used to avoid note jumping, then note selection stability is improved, but adaptability to local key changes deteriorates
Solution Approach 1:
The system dynamically adapts the target scale and key based on the detected accompaniment and vocal context. Rather than using a fixed scale, the system identifies the local key center from the accompaniment signal and adjusts the pitch correction targets accordingly. This allows the system to maintain stability within each local key while adapting to key changes throughout the song.
Solution Approach 2:
The system applies different pitch correction parameters and scale constraints to different segments of the performance based on local musical context. By analyzing the accompaniment harmony and vocal melody in local time windows, the system can identify passages with different key centers and apply appropriate correction settings locally, rather than using a single global scale configuration.
4Measurement precision
If pitch detection and pitch shifting operations are performed, then pitch correction is achieved, but time delay increases making it distracting during live performance
Solution Approach 1:
The system performs preliminary pitch detection and target note identification on incoming audio frames before final pitch correction is applied. By pre-processing the signal to identify target pitches and building a look-ahead buffer, the system minimizes the actual correction delay during live performance while maintaining accurate pitch correction.
Solution Approach 2:
The pitch correction process is segmented into separate stages: pitch detection, target note identification, correction parameter calculation, and actual pitch shifting. By segmenting the processing pipeline, the system can perform computationally intensive operations in advance or in parallel, reducing the critical path delay for real-time correction.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Systems and methods for adjusting pitch of an audio signal include detecting input notes in the audio signal, mapping the input notes to corresponding output notes, each output note having an associated upper note boundary and lower note boundary, and modifying at least one of the upper note boundary and the lower note boundary of at least one output note in response to previously received input notes. Pitch of the input notes may be shifted to match an associated pitch of corresponding output notes. Delay of the pitch shifting process may be dynamically adjusted based on detected stability of the input notes.