Audio Remix Generation via Rhythm-Aligned Vocal and Instrumental Mixing

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Conventional methods for generating remixes rely on manual production, limiting their application to a limited number of songs and failing to ensure coordination and synchronization in elements like lyrics and beats, resulting in poor mixing effects.

Innovation Solution

A method and apparatus for generating remixes by extracting vocal and instrumental signals from different versions of a song, aligning vocal tracks based on reference rhythm information, and selecting an aligned instrumental signal to mix with the vocal signals, ensuring synchronization and coordination.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If manual production is used for remix creation, then the process can be controlled and adjusted, but the application is limited to only a limited number of songs and cannot guarantee coordination and synchronization

Engineering Contradiction:
Improveapplicability to various songsVSAvoidcomplexity of mixing process
Core Design Contradiction:
Adaptability or versatilityVSDevice complexity

Solution Approach 1:

The patent replaces manual mechanical mixing processes with an automated system that uses rhythm information extraction and track alignment algorithms. The system automatically extracts rhythm information from audio tracks, aligns vocal and instrumental tracks based on this rhythm data, and generates remixes without manual intervention, thereby expanding applicability to various songs while maintaining coordination and synchronization.

Inventive Principle:
Principle #28Mechanics substitution (Replace mechanical system)

Solution Approach 2:

The system enables self-service remix generation by automatically extracting rhythm information from the input audio tracks and using this information to align and mix different vocal and instrumental tracks. The rhythm information serves as a self-organizing framework that guides the entire mixing process, allowing the system to autonomously coordinate and synchronize elements across multiple tracks without requiring manual control.

Inventive Principle:
Principle #25Self-service

2Reliability

If simple mixing of left and right channels is used, then the process is straightforward, but it cannot guarantee coordination and synchronization in elements such as lyrics and beats

Engineering Contradiction:
Improvecoordination and synchronization of lyrics and beatsVSAvoidcomplexity of track alignment
Core Design Contradiction:
ReliabilityVSDevice complexity

Solution Approach 1:

The patent applies preliminary action by extracting rhythm information from the audio tracks before the actual mixing process. This rhythm information is used to pre-align the temporal structures of vocal and instrumental tracks, ensuring that beats and lyrical rhythms are coordinated before mixing occurs. This preliminary alignment guarantees synchronization while providing a solid foundation for the subsequent mixing operation.

Inventive Principle:
Principle #10Preliminary action

Solution Approach 2:

The system segments the mixing process into distinct stages: rhythm information extraction, track alignment based on rhythm data, and final mixing. By separating these functions, the system can reliably coordinate and synchronize lyrics and beats through the alignment stage, then execute the mixing operation independently, thereby ensuring reliability without requiring complex integrated control.

Inventive Principle:
Principle #1Segmentation

3Productivity

If automated remix generation is implemented, then the process can be applied to a variety of songs, but the complexity of extracting and aligning tracks increases

Engineering Contradiction:
Improveefficiency of remix generationVSAvoidcomplexity of signal processing
Core Design Contradiction:
ProductivityVSDevice complexity

Solution Approach 1:

The patent introduces rhythm information as an intermediary element that mediates between the input audio tracks and the final mixed output. The rhythm information extraction module analyzes the temporal structure of the audio and creates a simplified representation that serves as a guide for aligning vocal and instrumental tracks. This intermediary framework simplifies the complex task of automated alignment by providing a clear synchronization reference, thereby improving productivity without requiring overly complex processing systems.

Inventive Principle:
Principle #24Intermediary (Mediator)

Data Source

PatentUS20240233691A9Audio mixing song generation method and apparatus, device, and storage medium
Publication Date: 2024.07.11 TENCENT MUSIC ENTERTAINMENT TECH (SHENZHEN) CO LTD
  • US20240233691A9 patent drawing
  • US20240233691A9 patent drawing
  • US20240233691A9 patent drawing

AI summary

A method and an apparatus for generating a remix. The method comprises: obtaining at least two audios which are different singing versions of a same song: extracting, from each audio, a vocal signal and an instrumental signal to obtain a vocal set the vocal signal of each audio and a instrumental set comprising the instrumental signal of each audio: aligning tracks of all vocal signals in the vocal set based on reference rhythm information selected from rhythm information of all vocal signals in the vocal set, where all vocal signals having the aligned tracks serve as to-be-mixed vocal audios: determining an instrumental signal, of which a track is aligned with those of the to-be-mixed vocal audios, from the instrumental set as a to-be-mixed instrumental audio: and mixing the to-be-mixed vocal audios with the to-be-mixed instrumental audio to obtain the remix.