Multimedia Tagging via Automatic Content Recognition

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Current social networking services lack an efficient method to associate user comments with specific time instances in multimedia content, such as TV programs, making it difficult for users to share and understand contextual comments, as multimedia content does not have an easy way to mark time instances like books do.

Innovation Solution

A method to tag streaming multimedia content with timestamp information, allowing users to specify time instances and associate comments with exact moments, using automatic content recognition (ACR) to identify and store these tags, enabling easy sharing and review of multimedia content.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Loss of information

If users share comments on multimedia content without time instance markers, then sharing is simple, but the context of comments cannot be understood

Engineering Contradiction:

Improvecontext informationVSAvoidsystem complexity

Core Design Contradiction:

Loss of informationVSDevice complexity

Solution Approach 1:

The system pre-generates multiple snippets of varying durations (e.g., 10 seconds, 30 seconds, 1 minute) around each tagged time instance before user requests. This preliminary preparation ensures that context information is readily available when users share comments, eliminating the need for real-time processing and reducing system complexity during actual usage.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The multimedia content is segmented into multiple snippets of different durations (short snippets, medium snippets, long snippets) around the tagged time instance. This segmentation allows users to choose appropriate context lengths based on their needs, preserving context information while managing system complexity through organized, modular snippet structures.

Inventive Principle:

Principle #1Segmentation

2Loss of information

If the system stores complete multimedia content for every user comment, then context is preserved, but storage requirements increase significantly

Engineering Contradiction:

Improvecontext informationVSAvoidstorage space

Core Design Contradiction:

Loss of informationVSQuantity of substance

Solution Approach 1:

The system extracts only the necessary portions of multimedia content (snippets) around the tagged time instance rather than storing complete content. By extracting and storing only relevant segments in multiple durations, the system preserves context information while dramatically reducing storage space requirements compared to storing entire multimedia files.

Inventive Principle:

Principle #2Taking out (Extraction)

Solution Approach 2:

Different snippet durations are provided based on local needs - short snippets for brief context, medium snippets for moderate context, and long snippets for extensive context. This local quality approach ensures that users receive appropriate context information without unnecessarily storing large amounts of data for all possible scenarios.

Inventive Principle:

Principle #3Local quality

3Adaptability or versatility

If the system generates multiple snippet durations for each tag, then user flexibility increases, but processing complexity increases

Engineering Contradiction:

Improvesnippet duration flexibilityVSAvoidprocessing complexity

Core Design Contradiction:

Adaptability or versatilityVSDevice complexity

Solution Approach 1:

The system performs the computationally intensive task of generating multiple snippet durations (10 seconds, 30 seconds, 1 minute) in advance during a preliminary processing stage. This preliminary action reduces processing complexity during user interactions, as the system only needs to retrieve and deliver pre-generated snippets rather than creating them in real-time.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The system implements a periodic or batch processing approach where snippets are generated at predetermined intervals and durations. This structured periodic action simplifies processing by using consistent, predefined time intervals rather than dynamically calculating varied durations, thereby reducing processing complexity while maintaining user flexibility.

Inventive Principle:

Principle #19Periodic action

Data Source

PatentUS10297286B2System and methods to associate multimedia tags with user comments and generate user modifiable snippets around a tag time for efficient storage and sharing of tagged items

Publication Date: 2019.05.21 GRACENOTE INC

AI summary

Example methods and apparatus to add a tagged snippet of multimedia content to a playlist are disclosed. An example apparatus comprises an automatic content recognition search service to search a fingerprint database to find a match between query fingerprints for a snippet of multimedia content captured from a multimedia program at a timestamp and reference fingerprints of matching reference multimedia content stored in the fingerprint database, a tag service to generate a tag representing the snippet of multimedia content, wherein the tag, the timestamp, meta information associated with the matching reference multimedia content, and a monitored variable for a number of viewers of the snippet of multimedia content are stored in a database storage as a tagged snippet of multimedia content, and to add the tagged snippet of multimedia content to a playlist for an identified multimedia program if the number of viewers of the tagged snippet exceeds a threshold.