Temporal Event-Based Video Fingerprinting for Robust Matching
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current content-based video fingerprinting methods are unreliable in detecting copies and derivatives due to frame alignment issues and image processing transformations, and audio-based fingerprinting fails to recognize videos with modified audio tracks.
Innovation Solution
A computer-implemented method for generating temporal, event-based video fingerprints by detecting events in video content, forming segments, and deriving time series signals from these segments to create robust segment-based fingerprints for accurate matching.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If frame-based visual features are used for content-based fingerprinting, then visual characteristics can be extracted to create fingerprints, but frame alignment issues and image processing transformations cause matching to become inaccurate and unreliable
Solution Approach 1:
The video is divided into multiple segments based on detected events (scene changes, cuts, transitions). Each segment is processed independently to extract temporal features, allowing the system to handle frame alignment issues and transformations by focusing on localized temporal patterns rather than global frame-by-frame comparison
Solution Approach 2:
The system uses dynamic event detection to identify significant changes in the video stream, creating a temporal structure that adapts to the content. This dynamic segmentation based on actual video events makes the fingerprinting robust to transformations while maintaining accuracy in identifying near-duplicates
2Adaptability or versatility
If audio-based fingerprinting is used, then audio track characteristics can be extracted, but the method fails to recognize videos with modified audio tracks
Solution Approach 1:
The system introduces visual event detection as an intermediary to structure the fingerprinting process. By using visual cues (scene changes, cuts) to define segments and extract temporal features, the method creates a fingerprint that is independent of audio content, thereby maintaining reliability even when audio tracks are modified or replaced
3Reliability
If watermarking is used to identify copyrighted content, then visible or invisible watermarks can be inserted to identify rightful owners, but watermarks can be defeated and removed making content permanently unlocked
Solution Approach 1:
Instead of inserting a watermark into the content, the system creates a fingerprint by copying and analyzing inherent temporal characteristics of the video (event structures, scene transitions). This fingerprint is derived from the content itself rather than being superimposed, making it impossible to remove without altering the actual video content
Solution Approach 2:
The system extracts temporal features and event-based characteristics from the video content to create a separate fingerprint representation. This extracted fingerprint serves as the identification mechanism, separating the identification function from the original content while maintaining inherent robustness
Data Source
AI summary
A computer implemented method, apparatus, and computer program product code for temporal, event-based video fingerprinting. In one embodiment, events in video content are detected. The video content comprises a plurality of video frames. An event represents discrete points of interest in the video content. A set of temporal, event-based segments are generated using the events. Each temporal, event-based segment is a segment of the video content covering a set of events. A time series signal is derived from each temporal, event-based segment using temporal tracking of content-based features of a set of frames associated with the each temporal, event-based segment. A temporal segment based fingerprint is extracted based on the time series signal for the each temporal, event-based segment to form a set of temporal segment based fingerprints associated with the video content.