Semantic Video Search Using Embeddings and Candidate Tags
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional video search engines often return inaccurate results due to imprecise user search queries, as users may be unsure of the content they are searching for.
Innovation Solution
A method involving a server that processes video files by generating summaries and tags using a large language model (LLM), creating word embeddings, and utilizing co-occurrence data to suggest candidate tags based on user queries, allowing users to select the most relevant video segments.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional video search engines compare search queries with video titles and tags, then the search process is simple and fast, but the search accuracy is low when user queries are imprecise
Solution Approach 1:
The system performs preliminary actions by generating multiple candidate tags and their corresponding word embeddings before the actual search query is processed. This pre-computation of candidate tags and embeddings allows the system to quickly compare user queries against pre-prepared options, improving search accuracy without significantly increasing the complexity of the real-time search process
Solution Approach 2:
The patent introduces word embeddings as an intermediary between user search queries and video content. Instead of directly comparing queries with video metadata, the system converts both into embedding vectors and performs similarity comparison in the embedding space. This intermediary representation enables more accurate semantic matching while maintaining computational efficiency
2Measurement precision
If the system generates multiple candidate tags using LLM and co-occurrence data, then search accuracy improves, but the processing time and computational resources increase
Solution Approach 1:
The system applies partial action by generating only the necessary candidate tags based on co-occurrence data and LLM analysis, rather than processing all possible tags. It presents a limited set of most relevant candidate tags to users, which is sufficient to improve search accuracy without requiring exhaustive processing of all potential tags, thus balancing accuracy with processing time
Solution Approach 2:
The system performs preliminary analysis using LLM and co-occurrence data to generate candidate tags before the actual search execution. This pre-computation phase prepares the candidate tags and their embeddings in advance, so that during the actual search, the system only needs to perform embedding similarity comparisons, significantly reducing the time required during user interactions
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A method for video searching is implemented using a server (1) that stores video files, and includes: obtaining a searching word embedding that represents a search query received from a user device (2); obtaining a reference word embedding and a reference tag word based on the searching word embedding; obtaining, based on the reference tag word and a plurality of co-occurrences stored in the server (1), a number of associated co-occurrences, the reference tag word and the number of associated co-occurrences serving as candidate tag words; obtaining a target word embedding that corresponds with one of the candidate tag words, and selecting, based on the target word embedding, at least one target tag word embedding; and selecting, based on the at least one target tag word embedding, a target video segment, and presenting the target video segment to the user device (2).