Real-Time Vocal Pitch Correction on Mobile Devices

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Mobile devices face limitations in real-time sound synthesis and musical applications due to computational power and bandwidth constraints, making it challenging to capture and process high-quality vocal performances effectively.

Innovation Solution

Implementing a method on portable computing devices to capture and pitch-correct vocals in real-time, applying user-selectable vocal effects schedules, and transmitting these performances for mixing and rendering with backing tracks, enabling amateur users to enhance their vocal performances and collaborate globally.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Manufacturing precision

If real-time pitch correction and vocal effects processing are implemented on mobile devices, then vocal performance quality is improved, but computational power requirements increase

Engineering Contradiction:

Improvevocal performance qualityVSAvoidcomputational power requirements

Core Design Contradiction:

Manufacturing precisionVSPower

Solution Approach 1:

The system divides processing into two segments: real-time pitch correction is performed on the mobile device during vocal capture, while vocal effects processing is performed separately on a remote server after the dry vocal performance is transmitted. This segmentation allows the mobile device to operate within its computational limits while still achieving high-quality processed vocals through the combined effort of local and remote processing.

Inventive Principle:

Principle #1Segmentation

2Manufacturing precision

If continuous real-time pitch correction is applied during vocal capture, then tonal quality is improved, but processing time and computational resources increase

Engineering Contradiction:

Improvetonal qualityVSAvoidprocessing time

Core Design Contradiction:

Manufacturing precisionVSLoss of time

Solution Approach 1:

The system performs pitch correction in advance during the vocal capture phase, continuously correcting pitch as the user sings. This preliminary pitch correction ensures that the fundamental tonal quality is improved before the vocal performance is transmitted. The effects processing then occurs after transmission, allowing the system to maintain high tonal quality without excessive delays during the capture experience.

Inventive Principle:

Principle #10Preliminary action

3Ease of operation

If vocal effects schedules are applied to enhance user performances, then user experience is improved, but system complexity increases

Engineering Contradiction:

Improveuser experienceVSAvoidsystem complexity

Core Design Contradiction:

Ease of operationVSDevice complexity

Solution Approach 1:

The system introduces a remote server as an intermediary to handle complex vocal effects processing. The mobile device application provides the user interface and captures vocals, then transmits dry vocal performances to the server. The server applies the selected vocal effects schedules and returns the processed vocals. This intermediary approach allows sophisticated effects processing without increasing the complexity of the mobile device itself.

Inventive Principle:

Principle #24Intermediary (Mediator)

4Manufacturing precision

If dry vocal capture is transmitted for remote effects processing, then processing quality is improved, but network bandwidth requirements increase

Engineering Contradiction:

Improveprocessing qualityVSAvoidnetwork bandwidth requirements

Core Design Contradiction:

Manufacturing precisionVSQuantity of substance

Solution Approach 1:

The system extracts only the essential dry vocal performance data from the mobile device for transmission to the server, rather than transmitting fully processed audio. By separating the dry vocal capture from the effects processing and transmitting only the necessary vocal data, the system achieves high processing quality while minimizing network bandwidth consumption. The effects are applied remotely using this compact vocal data.

Inventive Principle:

Principle #2Taking out (Extraction)

Data Source

PatentUS11670270B2Social music system and method with continuous, real-time pitch correction of vocal performance and dry vocal capture for subsequent re-rendering based on selectively applicable vocal effect(s) schedule(s)

Publication Date: 2023.06.06 SMULE INC

AI summary

Embodiments described provide a method for mixing vocal performances from different vocalists. A vocal score temporally synchronized with a corresponding backing track and lyrics is retrieved via a communications interface of a portable computing device. A first vocal performance of a user is captured, via a microphone interface of the portable computing device, and in correspondence with the backing track. An open call indication for soliciting, from a second vocalist, a second vocal performance to be mixed for audible rendering with the first vocal performance is transmitted. A mix to one of the user and the second vocalist is provided by selecting, based on to whom the mix is provided, the mix from alternative mixes each having a different prominent vocal performance.