Voice Query Refinement Using Video Action Keywords
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Modern search systems struggle to provide relevant results for ambiguous queries that include auxiliary verbs or pronouns, as they lack the ability to interpret actions performed in videos.
Innovation Solution
A search application analyzes concurrently presented video frames to identify actions, generates a movement model, and augments the query with relevant keywords based on matching templates, thereby refining the search query.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a search system uses conventional query processing methods, then the system operation is simple, but the search result relevance deteriorates for ambiguous queries
Solution Approach 1:
The system performs preliminary action by extracting and analyzing video frames before processing the search query. When an ambiguous query is detected, the system proactively captures video frames, identifies actions within them, and prepares action keywords in advance to augment the query, thereby improving search relevance without requiring complex real-time processing during query handling
Solution Approach 2:
The system introduces an intermediary component that acts as a bridge between video content analysis and search query processing. This intermediary extracts action information from video frames and uses it to augment ambiguous queries, effectively mediating between the visual context and the text-based search system to improve result relevance
2Measurement precision
If the system extracts only object information from video, then the information extraction is simple, but the query refinement effectiveness deteriorates for action-related queries
Solution Approach 1:
The system applies segmentation by dividing the video analysis task into distinct components: object detection, action recognition, and keyword extraction. By segmenting the analysis process, the system can focus specifically on identifying actions and their associated keywords without being overwhelmed by the complexity of analyzing all video content, thereby improving query refinement for action-related queries
Solution Approach 2:
The system changes parameters by transitioning from extracting only static object information to extracting dynamic action information. This parameter change involves analyzing temporal variations in video frames to identify actions, which provides more relevant context for refining ambiguous queries related to activities or movements
Data Source
AI summary
Systems and methods are described for providing contextual search results. The system may receive a search query during presentation of a video. If the query is ambiguous, the system accesses some of the frames of the video. The frames are analyzed to identify a performed action depicted in the frames. The system retrieves a keyword related to the identified action. The ambiguous query is augmented with the keyword. The augmented search query is used to search for and output relevant search results.