Automated Audio Feature Extraction with Expert Verification

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Determining musicological features of audio content is laborious and prone to errors, reducing the effectiveness of audio content delivery in media streaming systems, as it relies on human analysts and is time-consuming.

Innovation Solution

A computer-implemented method that extracts audio content features, identifies correlations with media features, and provides them to experts for analysis, using machine learning models to estimate media features with confidence scores, thereby enhancing the accuracy and efficiency of media content recommendation.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If human analysts are used to determine musicological features of audio content, then accuracy of feature identification can be maintained through expert knowledge, but the process becomes extremely time-consuming and laborious

Engineering Contradiction:

Improveaccuracy of musicological feature identificationVSAvoidtime required for feature identification

Core Design Contradiction:

Measurement precisionVSLoss of time

Solution Approach 1:

The patent introduces an automated audio analysis system as an intermediary between the audio content and human experts. This system extracts audio features (tempo, key, instrumentation, etc.) and presents them to experts for verification, reducing the time experts need to spend while maintaining accuracy through their expert judgment on the automated results.

Inventive Principle:

Principle #24Intermediary (Mediator)

Solution Approach 2:

The system performs preliminary automated extraction of musicological features before presenting them to human experts. This preliminary action prepares the data in advance, allowing experts to focus their time on verification and refinement rather than starting from scratch, thus reducing overall processing time while maintaining accuracy.

Inventive Principle:

Principle #10Preliminary action

2Reliability

If human analysts manually identify musicological features for all audio content, then comprehensive coverage of features can be achieved, but the cost and time required become prohibitively large

Engineering Contradiction:

Improvecompleteness of feature identificationVSAvoidthroughput of feature identification process

Core Design Contradiction:

ReliabilityVSProductivity

Solution Approach 1:

The patent segments the feature identification process into two parts: automated extraction of objective audio features and human expert verification of subjective musicological features. This segmentation allows the system to handle large volumes of content through automated processing while maintaining reliability through expert review of critical features.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The system implements feedback loops where expert annotations of automated feature extraction results are used to improve and refine the automated system. This feedback mechanism ensures comprehensive and reliable feature identification while increasing productivity, as the system learns from expert input to reduce the need for manual review over time.

Inventive Principle:

Principle #23Feedback

3Productivity

If automated systems are used to extract audio features, then processing speed and throughput increase significantly, but accuracy may decrease without expert verification

Engineering Contradiction:

Improvethroughput of feature identificationVSAvoidaccuracy of musicological feature identification

Core Design Contradiction:

ProductivityVSMeasurement precision

Solution Approach 1:

The patent positions human experts as an intermediary layer between automated feature extraction and final feature identification. The automated system extracts features at high speed, experts verify and correct these features, ensuring accuracy is maintained while productivity benefits from the automated preprocessing.

Inventive Principle:

Principle #24Intermediary (Mediator)

Solution Approach 2:

The system replaces manual mechanical analysis by human experts with automated computational feature extraction. This substitution dramatically increases throughput while maintaining accuracy through the subsequent expert verification step, combining the speed of automation with the precision of human judgment.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

Data Source

PatentUS10129314B2Media feature determination for internet-based media streaming

Publication Date: 2018.11.13 PANDORA MEDIA LLC

AI summary

A media service server for streaming media items with similar media features receives a plurality of media items, where each media item of the plurality of media item is labeled with one or more media features characterizing the media item. Audio content features from the plurality of media items are extracted. Correlations between the audio content features and the media features are identified. A set of media items to be analyzed is received. For each media item of the set of media items, a set of media features based on the identified correlations between the audio content features and the plurality of media features is estimated. Each estimated media feature is associated with a confidence score. The set of media items with the estimated media features is provided to one or more experts for expert analysis.