Audio Retrieval Using Sparse Auditory Images
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional information retrieval of audio content using textual queries is difficult and often provides inaccurate results due to the limitations in representing and matching audio features with textual labels.
Innovation Solution
The method involves generating auditory images from audio files using a cochlear model, extracting sparse features, and training a matching function to relate these features to textual queries, allowing for accurate ranking and retrieval of audio files based on query terms.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional information retrieval methods are used to search audio content with textual queries, then the system is simple to operate, but the retrieval accuracy is poor and results are inaccurate
Solution Approach 1:
The patent introduces an intermediary matching function that bridges the gap between textual queries and audio content. This function learns to map sparse audio features to textual keywords, serving as a mediator that enables accurate cross-modal retrieval without requiring direct comparison between incompatible representation types.
Solution Approach 2:
The system transforms audio data from raw waveforms to sparse feature representations through auditory modeling and sparse coding. This parameter transformation converts continuous audio signals into discrete, compact feature vectors that can be effectively matched with textual queries, improving retrieval accuracy while managing complexity.
2Measurement precision
If detailed audio features are extracted and matched with textual labels, then retrieval accuracy improves, but computational complexity and processing time increase
Solution Approach 1:
The patent extracts only the most salient and discriminative features from audio signals using sparse coding. By selecting a small subset of important features rather than processing all audio characteristics, the system achieves high classification accuracy while significantly reducing computational burden and processing time.
Solution Approach 2:
The matching function is trained separately on individual audio features and textual keywords, then combines these learned mappings for final retrieval. This segmentation of the learning process allows efficient processing of feature-extraction and matching operations independently, reducing overall processing time.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Methods, systems, and apparatus, including computer programs encoded on computer storage media, are provided for using audio features to classify audio for information retrieval. In general, one aspect of the subject matter described in this specification can be embodied in methods that include the actions of generating a collection of auditory images, each auditory image being generated from respective audio files according to an auditory model; extracting sparse features from each auditory image in the collection to generate a sparse feature vector representing the corresponding audio file; and ranking the audio files in response to a query including one or more words using the sparse feature vectors and a matching function relating sparse feature vectors to words in the query.