Audio Matching Using Pitch Shift Resistant Spectrogram Descriptors

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing audio matching systems face challenges in identifying audio samples that have undergone pitch shift and time stretching, as these transformations alter the frequency and timing of interest points, leading to difficulties in accurate matching.

Innovation Solution

The system generates stable anchor points and interest points within a spectrogram, using frequency bins to characterize the neighborhood around anchor points, and computes feature vectors based on these points, which are then quantized and hashed to create robust descriptors resistant to pitch shift and time stretching.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If traditional interest point extraction methods are used in audio matching, then the matching process is simple and straightforward, but the system fails to accurately identify audio samples that have undergone pitch shift and time stretching transformations

Engineering Contradiction:

Improveaccuracy of audio sample identificationVSAvoidcomplexity of feature extraction process

Core Design Contradiction:

ReliabilityVSDevice complexity

Solution Approach 1:

The audio signal is segmented into multiple frequency bands, and interest points are extracted independently within each band. This segmentation allows the system to capture pitch-shifted features more effectively by analyzing localized frequency regions rather than the entire spectrum, thereby improving identification accuracy under pitch shift and time stretching conditions.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The system transitions from traditional time-domain interest point extraction to a time-frequency domain approach by organizing interest points across multiple frequency bands. This dimensional expansion into the frequency domain enables the system to maintain robustness against pitch shifts and time stretching by capturing spectral characteristics that persist across transformations.

Inventive Principle:

Principle #17Another dimension (Dimensionality change)

2Adaptability or versatility

If pitch shift and time stretching resistant features are incorporated into the descriptor, then the system can accurately match altered audio signals, but the computational complexity and processing requirements increase

Engineering Contradiction:

Improverobustness to pitch shift and time stretchingVSAvoidcomplexity of descriptor computation

Core Design Contradiction:

Adaptability or versatilityVSDevice complexity

Solution Approach 1:

Different frequency bands are treated with localized analysis, where interest points are extracted and descriptors are computed independently for each band. This local quality approach allows the system to adapt to pitch shifts and time stretching in specific frequency regions without requiring complex global transformations, balancing robustness with computational efficiency.

Inventive Principle:

Principle #3Local quality

Solution Approach 2:

The system changes the parameters used to characterize interest points by incorporating frequency band information and temporal relationships into the descriptor. Instead of relying on single-point features, the descriptor uses multiple parameters including frequency bin indices, temporal positions, and inter-band relationships, which remain stable under pitch shift and time stretching transformations.

Inventive Principle:

Principle #35Parameter changes

3Reliability

If multiple frequency bands and temporal information are incorporated into the descriptor, then the system achieves better resistance to transformations, but the amount of data to be processed and stored increases

Engineering Contradiction:

Improverobustness of audio matchingVSAvoidvolume of descriptor data

Core Design Contradiction:

ReliabilityVSQuantity of substance

Solution Approach 1:

The system extracts only the most discriminative and transformation-resistant features from the time-frequency representation. By selecting key interest points and their essential parameters (frequency bin indices, temporal positions) rather than processing all spectral information, the system reduces descriptor data volume while maintaining robustness against pitch shift and time stretching.

Inventive Principle:

Principle #2Taking out (Extraction)

Solution Approach 2:

Instead of compressing the audio signal first and then extracting features, the system extracts detailed time-frequency features and then applies compression techniques to the descriptor representation. This inversion allows the system to capture comprehensive transformation-resistant information initially, then efficiently encode it for storage and comparison, balancing robustness with data efficiency.

Inventive Principle:

Principle #13The other way round (Inversion)

Data Source

PatentUS9213703B1Pitch shift and time stretch resistant audio matching

Publication Date: 2015.12.15 GOOGLE LLC

AI summary

Systems and methods are provided herein relating to audio matching. Descriptors can be generated based on anchor points and interest points that characterize the local neighborhood surrounding the anchor point. Characterizing the local spectrogram neighborhood surrounding anchor points can be more robust to pitch shift distortions and time stretch distortions. Those anchor points surrounded by a lack of spectral activity or even spectral activity can be filtered from further examination. Using these pitch shift and time stretch resistant audio features within descriptors can provide for more accurate and efficient audio matching.