Image Search Query Parsing With Embedding-Based File Matching
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing search techniques for unstructured data, particularly image files, struggle to accurately locate specific files without knowing the time and location of capture due to limited metadata on visual content, leading to inefficient and computationally demanding searches.
Innovation Solution
A combination of query understanding and semantic search models is employed to parse queries, remove location and time information, and use embeddings to identify matching image files, integrating machine learning models for efficient and precise searches.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional search techniques are used to locate specific image files in a corpus, then search coverage is maintained, but computational demands and memory utilization increase significantly
Solution Approach 1:
The patent segments the search process into two distinct phases: (1) embedding-based semantic search to identify preliminary matching files, and (2) metadata-based filtering to identify final matching files. This segmentation allows the computationally intensive embedding comparison to be performed on a reduced dataset, thereby reducing overall computational demands while maintaining search accuracy.
Solution Approach 2:
The patent performs preliminary filtering using metadata (location and timeframe) before conducting the full embedding-based semantic search. By pre-filtering the corpus using readily available metadata, the system reduces the search space to only those files that could potentially match the query, thereby reducing computational demands and memory utilization while maintaining reliable search results.
2Use of energy by moving object
If metadata-based search is used to locate image files, then computational demands are reduced, but search precision decreases due to limited visual content information
Solution Approach 1:
The patent introduces embeddings as an intermediary representation that bridges metadata-based search and visual content matching. The embeddings capture visual characteristics of images and enable semantic search, allowing the system to achieve high search precision while maintaining computational efficiency through the two-stage filtering approach.
3Measurement precision
If full corpus search is performed to ensure accurate location of image files, then search precision is maintained, but search time increases
Solution Approach 1:
The patent performs preliminary filtering using metadata (location and timeframe) to reduce the search corpus before conducting embedding-based semantic search. This preliminary action eliminates obviously non-matching files early in the process, significantly reducing search time while maintaining search accuracy by ensuring all potentially matching files are still considered.
Solution Approach 2:
The patent segments the search process into two phases: embedding-based semantic search to identify preliminary matches, followed by metadata-based filtering to identify final matches. This segmentation enables the system to quickly eliminate non-matching files through metadata comparison, reducing overall search time while maintaining precision through the subsequent embedding verification.
Data Source
AI summary
Techniques are disclosed for performing a search of a corpus of data files to identify files that match a query. A file can include metadata and an embedding representing visual characteristics of the data. A semantic understanding model can evaluate the query to identify any entities, locations, actions, and timeframes in the query. A revised query can be produced from the query by removing the identified locations and entities. A semantic search model can use the revised query and the embeddings to identify preliminary files from the corpus of files. The identified locations and entities can be used to filter the preliminary files to identify matching files. The matching files can be presented in a graphical user interface. Implementations of the techniques can include corresponding methods, computer systems, apparatuses, devices, and computer programs recorded on one or more non-transitory computer storage devices.