Image Search System Using Text Recognition and Visual Embeddings
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional systems are inefficient in searching databases containing both text and images using an image query, as they typically rely on string matching and struggle to find images with specific objects within the data corpus.
Innovation Solution
The method involves receiving a query image, recognizing text strings, generating a query vector, conducting image searches based on both the vector and text strings to generate candidate images, and producing a query output, utilizing processors and machine-learning models for efficient image retrieval.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If conventional string matching is used to search images, then text-based queries can be processed, but image queries with specific objects cannot be effectively searched
Solution Approach 1:
The patent introduces an intermediary process that converts image queries into multiple representations: extracting text strings from the query image, generating visual embeddings from image features, and creating query vectors. These intermediaries enable the system to search both text and image databases effectively, resolving the contradiction between query versatility and search accuracy.
Solution Approach 2:
The search process is segmented into multiple independent pathways: one pathway processes extracted text strings through OCR and text search, another pathway processes visual embeddings through image feature comparison, and a third pathway generates query vectors for semantic matching. This segmentation allows each pathway to specialize in its own search type while collectively providing comprehensive search capability.
2Adaptability or versatility
If a single search method is used, then the system is simple to implement, but it cannot search both text and images effectively
Solution Approach 1:
The patent implements a universal search system that handles multiple query types (text queries, image queries, mixed queries) through a single unified architecture. The system extracts both text and visual features from query inputs and searches across multiple databases (text database, image database, video database) using the same core processing pipeline, enabling multi-functionality without requiring separate specialized systems for each query type.
Solution Approach 2:
The patent transforms the search problem from a single-dimension approach to a multi-dimensional approach by creating parallel search pathways: text-based search dimension, image-based search dimension, and semantic vector search dimension. Each dimension operates independently but contributes to the final search results, allowing the system to cover diverse query types while maintaining a unified system structure.
3Measurement precision
If exact text indexing is required, then search precision is high, but images without indexed text cannot be found
Solution Approach 1:
The patent changes the search parameters from relying solely on text indexing to using multiple parameter types: extracted text strings from images, visual embeddings capturing image content, and query vectors representing semantic meaning. This parameter transformation enables the system to search images based on their visual content and semantic meaning rather than requiring pre-indexed text, thereby maintaining precision while expanding coverage.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Systems and methods for image search are provided. In some embodiments, a method for image search includes receiving a query image, recognizing one or more text strings on the query image, generating a query vector based on the query image, conducting a first image search based at least in part on the query vector to generate one or more first candidate images, conducting a second image search based at least in part on the one or more recognized text strings to generate one or more second candidate images, and generating a query output based on the one or more first candidate images and the one or more second candidate images.