Real-time Audio to MIDI Conversion via Overlapping Window Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current audio processing technologies lack tools for real-time conversion of audio signals into digital music note representations, such as MIDI format, which is essential for composing new music and aiding musicians without proper musical education.
Innovation Solution
A software system that processes audio signals in overlapping window sets, using machine learning algorithms to convert audio data into MIDI format in real-time, by analyzing frequency domain frames and generating note event probabilities.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional audio processing methods are used, then music quality improvement is achieved, but real-time conversion to digital music notes is not possible
Solution Approach 1:
The audio signal is divided into overlapping window sets, where each window is further segmented into frames that are processed individually through FFT transformation. This segmentation enables real-time processing of audio signals while maintaining quality by analyzing manageable portions sequentially.
Solution Approach 2:
The system performs preliminary FFT transformation on audio frames to convert them to frequency domain representations before note detection. This preliminary processing prepares the data structure needed for accurate real-time note conversion without waiting for complete audio segments.
2Measurement precision
If audio signals are processed in large segments, then conversion accuracy is improved, but processing lag increases
Solution Approach 1:
The patent divides audio signals into overlapping windows with multiple frames each, allowing accurate frequency analysis through FFT on smaller manageable segments while maintaining temporal resolution for real-time performance.
Solution Approach 2:
The system dynamically adjusts processing by using overlapping windows where each window contains multiple frames that are processed at different stages. This dynamic approach allows continuous real-time processing with minimal lag while accumulating sufficient data for accurate note detection.
3Adaptability or versatility
If specialized hardware devices are used for audio to MIDI conversion, then conversion capability is achieved, but cost and practicality are reduced
Solution Approach 1:
The patent replaces specialized hardware devices with a software-based system that performs FFT transformation and note detection algorithms on standard computing platforms. This substitution eliminates the need for expensive specialized hardware while maintaining conversion capability through computational methods.
Solution Approach 2:
The software system is designed to be universally applicable to various audio inputs and can run on standard computers without specialized hardware. The same system can process different types of audio signals and export to MIDI format, providing versatile functionality at low cost.
Data Source
AI summary
Techniques are described for real-time converting audio into digital musical notation. In an implementation, the process receives a sequence of samples of an audio stream in real time. Based on the sequence of samples, the process generates a window set of note event probability values. The process excludes from the window set of event probability values a leading set of event probability values and a trailing set of event probability values, thereby generating a filtered window set of event probability values. Based on the filtered window set of event probability values, the process determines a sequence set of note-on and note-off events.