Deep Video Retrieval Model for Provenance Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional video comparison systems face challenges in accurately identifying original videos amidst modifications, are vulnerable to adversarial attacks, and struggle with robustness against common transformations and manipulations, leading to inaccuracies and inefficiencies in determining video provenance.
Innovation Solution
A deep video retrieval model that generates inverse indices for known videos using visual and audio descriptors, allowing for robust matching and identification of modified videos despite benign and editorial transformations, while ignoring non-editorial changes and providing localization of editorial modifications.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional video comparison systems are used to identify original videos, then the system can perform basic video matching, but the accuracy deteriorates when videos undergo transformations or manipulations
Solution Approach 1:
The system segments videos into temporal chunks and generates visual descriptors for each segment using a visual encoder. This segmentation approach allows the system to compare specific portions of videos independently, maintaining accuracy even when videos undergo transformations or manipulations to specific segments.
Solution Approach 2:
The system introduces an inverse index as an intermediary data structure that maps visual descriptors to video identifiers. This inverse index enables efficient and accurate matching by serving as a mediator between query video descriptors and the database of known videos, improving both precision and robustness.
2Measurement precision
If deep video retrieval model with inverse index is used, then video matching accuracy improves, but computational complexity increases
Solution Approach 1:
The system performs preliminary action by pre-computing and storing visual descriptors for all known videos in an inverse index before actual video matching occurs. This pre-processing step, though computationally intensive, enables rapid and accurate matching during query operations, trading off initial complexity for ongoing efficiency.
Solution Approach 2:
The system creates compressed representations (visual descriptors) of video content that can be stored and compared without requiring the actual video data. These descriptor copies enable efficient matching while reducing the computational burden of comparing full video files.
3Reliability
If the system processes complete videos for matching, then comprehensive comparison is achieved, but processing time increases
Solution Approach 1:
The system divides videos into temporal segments and generates descriptors for each segment independently. This segmentation enables parallel processing of multiple video portions, significantly reducing overall processing time while maintaining comprehensive comparison through aggregation of segment-level results.
Solution Approach 2:
The system performs partial action by comparing only essential visual descriptors rather than processing entire video files. This selective approach achieves sufficient matching reliability for provenance determination while dramatically reducing processing time and computational resources required.
Data Source
AI summary
The present disclosure relates to systems, methods, and non-transitory computer readable media that utilize deep learning to map query videos to known videos so as to identify a provenance of the query video or identify editorial manipulations of the query video relative to a known video. For example, the video comparison system includes a deep video comparator model that generates and compares visual and audio descriptors utilizing codewords and an inverse index. The deep video comparator model is robust and ignores discrepancies due to benign transformations that commonly occur during electronic video distribution.