On-Device Video Search Using Temporal Feature Maps
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional video search methods in large databases are time-consuming and lack real-time on-device solutions, as they do not utilize a separate metadata database for efficient querying.
Innovation Solution
An on-device method involving a neural network that extracts visual feature vectors, generates temporal maps and tags, and stores them in a vector database for offline processing, allowing for real-time search queries by comparing linguistic features with stored vectors.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional video search methods are used on large video databases, then complete video content can be searched, but search time becomes excessively long
Solution Approach 1:
The patent segments the video database into two parts: a compressed metadata database containing extracted features (visual, audio, textual) and timestamps, and the original full video database. Search operations are performed on the small metadata database first, and only relevant video segments are retrieved from the full database, thus maintaining search accuracy while dramatically reducing search time.
Solution Approach 2:
The patent performs preliminary processing by extracting visual features, audio features, and textual features from videos offline, and pre-generating metadata including timestamps and feature vectors. This preliminary action creates a ready-to-search metadata database that enables fast real-time querying without needing to process full video content during search operations.
2Measurement precision
If the full video database is processed in real-time, then accurate video retrieval is achieved, but device resources are overwhelmed
Solution Approach 1:
The patent divides the processing workload into offline batch processing (feature extraction, metadata generation) and online real-time processing (query execution on metadata). This segmentation allows complex processing to be performed when device resources are available, while keeping real-time resource requirements minimal.
Solution Approach 2:
The patent creates a compressed copy of video content in the form of metadata databases containing feature vectors, timestamps, and descriptive information. This metadata copy serves as a surrogate for the full video database during search operations, enabling accurate retrieval without processing the actual video data in real-time.
3Productivity
If a separate metadata database is created offline, then real-time search performance is improved, but additional storage space is required
Solution Approach 1:
The patent transforms video content from its original high-dimensional form into compressed parameter representations (feature vectors, timestamps, metadata). This parameter transformation dramatically reduces storage requirements while preserving the essential information needed for accurate search, enabling the metadata database to be much smaller than the full video database.
4Device complexity
If conventional video search is performed without metadata indexing, then simple storage structure is maintained, but search efficiency is extremely low
Solution Approach 1:
The patent performs preliminary indexing by extracting and organizing metadata (visual features, audio features, textual features, timestamps) from videos before search operations. This pre-indexing creates an efficient search structure that enables fast querying without complicating the underlying storage architecture, as the metadata database remains a separate, organized layer above the video files.
Data Source
AI summary
A method of video search in an electronic device is provided. The method includes extracting a plurality of visual features vectors comprising visual features from a plurality of videos stored in a video database, wherein each visual feature corresponds to a pre-defined moment, generating a temporal map of the plurality of visual feature vectors, generating at least one video tag and at least one time stamp associated with the visual feature vector, storing the at least one video tag, the visual feature vectors and the at least one time stamp for each visual feature in a vector database, receiving a search query from a user for a specific moment in a plurality of videos in the electronic device, running the search query on the vector database, and displaying visual features associated with the specific moments from the plurality of videos.