Mobile Vocal Pitch Correction via Preliminary Processing and Server Mediation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Mobile devices face challenges in capturing and processing vocal performances due to limited computational resources and bandwidth constraints, making it difficult to provide effective pitch correction and harmonization for real-time applications.
Innovation Solution
The development of signal processing techniques and data flows that enable real-time pitch correction and harmonization of vocal performances on mobile devices, using methods like PSOLA and LPC, which allow for continuous pitch correction and harmonization based on score-coded settings, even with limited computational power, and efficient communication over bandwidth-constrained networks.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If real-time pitch correction and harmonization processing is performed on mobile devices, then vocal performance quality is improved, but computational resource consumption increases
Solution Approach 1:
The system performs preliminary pitch correction processing on the mobile device before transmission, preparing the vocal signal in advance. This allows the device to handle complex processing tasks beforehand, reducing the computational burden during real-time performance and transmission phases.
Solution Approach 2:
A server acts as an intermediary between multiple mobile devices, receiving vocal performances, performing additional pitch correction and harmonization processing, and distributing the processed performances back to devices. This distributes the computational load from individual devices to a centralized system with greater processing power.
2Manufacturing precision
If high-quality audio processing is performed locally on mobile devices, then tonal quality is improved, but bandwidth consumption increases
Solution Approach 1:
The mobile device performs preliminary pitch correction and prepares the vocal signal before transmission, so that basic quality enhancement occurs locally without requiring excessive bandwidth for transmitting raw high-quality audio data.
Solution Approach 2:
The server serves as an intermediary that receives processed audio data from mobile devices, performs additional harmonization and mixing operations, and distributes the final high-quality mixed performance to multiple devices. This allows quality enhancement to occur primarily on the server side rather than requiring high bandwidth between devices.
3Adaptability or versatility
If multiple vocal performances are captured and mixed in real-time, then collaborative music-making capability is improved, but processing time increases
Solution Approach 1:
Individual vocal performances are pre-processed with pitch correction and prepared for mixing before being combined with other performances. This preliminary preparation reduces the time required for final mixing and harmonization operations when multiple performances need to be combined quickly.
Solution Approach 2:
The server acts as a central intermediary that receives multiple vocal performances from different devices, performs batch processing for mixing and harmonization, and distributes the results back to participants. This centralized approach allows parallel processing of multiple performances, reducing overall processing time compared to sequential local processing on each device.
Data Source
AI summary
Despite many practical limitations imposed by mobile device platforms and application execution environments, vocal musical performances may be captured and continuously pitch-corrected for mixing and rendering with backing tracks in ways that create compelling user experiences. Based on the techniques described herein, even mere amateurs are encouraged to share with friends and family or to collaborate and contribute vocal performances as part of virtual “glee clubs.” In some implementations, these interactions are facilitated through social network- and/or eMail-mediated sharing of performances and invitations to join in a group performance. Using uploaded vocals captured at clients such as a mobile device, a content server (or service) can mediate such virtual glee clubs by manipulating and mixing the uploaded vocal performances of multiple contributing vocalists.