Latent-Space Audio Search for Perception-Based Sound Retrieval
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Searching for a target sound sample in a large sound library requires significant labor and time, and existing methods relying on metadata or human annotations are inconsistent and labor-intensive, limiting the ability to search based on individual perceptions and predefined characteristics.
Innovation Solution
An information processing apparatus and method that uses a deep neural network (DNN) to encode sound samples into latent codes, allowing for a graphical search system where similar sounds are mapped close together in a low-dimensional space, enabling intuitive searching and generation of new sound samples without predefinitions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If sound samples are searched for by listening to each sample one by one, then the search can be performed without additional preprocessing, but the labor and time required become extremely large
Solution Approach 1:
The patent applies preliminary action by pre-processing sound samples during library construction to extract acoustic features and build similarity indexes. This preliminary extraction of spectral centroid, spectral rolloff, zero-crossing rate, and other acoustic parameters enables rapid similarity search without requiring users to listen to each sample individually, thus resolving the contradiction between search simplicity and time consumption.
2Ease of operation
If metadata annotations are manually added to sound samples, then search capability is improved, but the construction of sound library requires much labor and time
Solution Approach 1:
The patent replaces the mechanical system of manual metadata annotation with an automated acoustic feature extraction system. By using computational algorithms to automatically calculate acoustic parameters (spectral centroid, spectral rolloff, zero-crossing rate, etc.) from sound waveforms, the system eliminates the need for manual listening and annotation while maintaining or improving search capability, thus resolving the contradiction between search ease and construction time.
Solution Approach 2:
The system applies self-service by enabling sound samples to annotate themselves through automatic acoustic feature extraction. Each sound sample's acoustic characteristics are automatically computed and stored as metadata, allowing the sound library to serve itself without requiring external human annotators, thereby reducing construction time and labor while preserving search functionality.
3Measurement precision
If human annotators provide annotations, then perceptual characteristics can be captured, but inconsistency arises due to individual differences in human perception
Solution Approach 1:
The patent changes the parameter basis from subjective human perception to objective acoustic parameters. By measuring physical acoustic characteristics (spectral centroid, spectral rolloff, zero-crossing rate, etc.) that can be consistently computed from sound waveforms, the system eliminates variability introduced by different human annotators' subjective perceptions, thus improving annotation consistency while maintaining the ability to capture perceptually relevant characteristics.
Data Source
AI summary
An information processing apparatus according to an embodiment includes a search unit that evaluates similarity between two pieces of audio data on the basis of the distance in a latent space between latent codes obtained from the two pieces of audio data.