Semantic Video Analysis for Soundtrack Recommendation

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing video hosting systems struggle to recommend appropriate music soundtracks for videos without pre-existing soundtracks, as conventional methods rely on musical similarity assumptions that do not apply to videos without music.

Innovation Solution

A method and system that extract content features from videos to generate semantic features, search for semantically similar videos with soundtracks, rank them based on soundtrack typicality, and recommend the most typical soundtracks for videos without soundtracks.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If conventional soundtrack recommendation methods based on musical similarity are used, then they work for videos with existing soundtracks, but they fail for videos without pre-existing soundtracks

Engineering Contradiction:

Improveapplicability to videos without soundtracksVSAvoidrecommendation accuracy

Core Design Contradiction:

Adaptability or versatilityVSReliability

Solution Approach 1:

Instead of comparing the probe video's soundtrack with alternative soundtracks (conventional approach), the patent inverts the approach by comparing semantically similar videos' soundtracks to determine recommendations. This allows videos without soundtracks to be processed by first finding semantically similar videos that do have soundtracks, then using those soundtracks as recommendations.

Inventive Principle:

Principle #13The other way round (Inversion)

Solution Approach 2:

The patent introduces semantic features and video similarity as an intermediary between the probe video and soundtrack recommendations. Rather than directly matching soundtracks based on musical similarity, the system uses semantic video analysis as a mediator to bridge videos without soundtracks to appropriate soundtrack recommendations through their semantically similar counterparts.

Inventive Principle:

Principle #24Intermediary (Mediator)

2Reliability

If extensive music knowledge is required to select appropriate soundtracks, then recommendation accuracy improves, but user accessibility and ease of use deteriorate

Engineering Contradiction:

Improvesoundtrack selection accuracyVSAvoiduser accessibility

Core Design Contradiction:

ReliabilityVSEase of operation

Solution Approach 1:

The system performs automatic soundtrack recommendation by analyzing video semantic content and comparing it with a database of videos and their soundtracks. This self-service approach eliminates the need for users to possess music or cinematography knowledge, as the system autonomously generates recommendations based on semantic similarity and soundtrack typicality metrics.

Inventive Principle:

Principle #25Self-service

Solution Approach 2:

The patent replaces the manual, knowledge-intensive process of soundtrack selection with an automated computational system. Instead of relying on human expertise in music and cinematography, the system uses computer vision, semantic analysis, and machine learning algorithms to automatically analyze video content and recommend appropriate soundtracks.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

3Reliability

If the system analyzes semantic features and ranks multiple video candidates, then recommendation quality improves, but computational complexity and processing time increase

Engineering Contradiction:

Improverecommendation qualityVSAvoidsystem computational complexity

Core Design Contradiction:

ReliabilityVSDevice complexity

Solution Approach 1:

The patent segments the soundtrack recommendation process into distinct modular components: semantic feature extraction from the probe video, similarity-based video candidate search, soundtrack typicality computation, and ranking/generation of recommendations. This segmentation allows each component to be optimized independently and facilitates efficient processing through modular architecture.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The system performs preliminary actions by pre-computing and storing semantic features for videos in the database, and by pre-organizing videos based on their semantic characteristics. This preliminary processing enables faster real-time recommendation generation when a probe video is submitted, as the system can leverage pre-computed features and organized data structures rather than performing all analysis from scratch.

Inventive Principle:

Principle #10Preliminary action

Data Source

PatentUS8737817B1Music soundtrack recommendation engine for videos

Publication Date: 2014.05.27 GOOGLE LLC

AI summary

A system and method provide a soundtrack recommendation service for recommending one or more soundtrack for a video (i.e., a probe video). A feature extractor of the recommendation service extracts a set of content features of the probe video and generates a set of semantic features represented by a signature vector of the probe video. A video search module of the recommendation service is configured to search for a number of video candidates, each of which is semantically similar to the probe video and has an associated soundtrack. A video outlier identification module of the recommendation service identifies video candidates having an atypical use of their soundtracks and ranks the video candidates based on the typicality of their soundtrack usage. A soundtrack recommendation module selects the soundtracks of the top ranked video candidates as the soundtrack recommendations to the probe video.