Text-Based Image Search Using Visual Signatures
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing image capture systems, such as traffic cameras, often produce images with insufficient resolution for optical character recognition, making it difficult to perform text recognition tasks like locating a stolen vehicle by license plate number, especially when images contain diverse and complex text elements like license plates with varying graphics, fonts, and slogans.
Innovation Solution
A text-based image searching system that uses a distributed computing environment to process images without optical character recognition, employing a computer system with databases for annotated images and visual signatures, where images are segmented, and visual signatures are generated using Fisher Vectors to enable text-based searching by comparing textual queries with annotated database entries and calculating similarity measures to retrieve relevant images.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If optical character recognition is used for text recognition in images, then text can be identified, but high resolution images are required which are expensive to implement and not always available from existing cameras
Solution Approach 1:
The patent introduces visual signatures as an intermediary representation between the image and the text recognition process. Instead of directly applying OCR to images, the system extracts visual signatures that capture essential visual patterns and uses these for comparison and recognition, enabling text identification without requiring high-resolution images suitable for traditional OCR
Solution Approach 2:
The patent replaces the mechanical/optical system of traditional OCR (which requires high-resolution optical character analysis) with a computational approach using visual signatures and similarity comparison. This substitution allows text recognition to function with lower resolution images by using algorithmic pattern matching instead of optical character analysis
2Productivity
If existing traffic cameras are used for vehicle identification, then image capture is available, but the quality is insufficient for optical character recognition requiring manual review of each image
Solution Approach 1:
The patent performs preliminary extraction of visual signatures from images during the image capture phase. By pre-processing images to extract and store visual signatures alongside the original images, the system prepares data in advance that enables rapid similarity-based searching without requiring manual review, thus improving productivity while maintaining recognition accuracy
Solution Approach 2:
The visual signature serves as an intermediary that bridges the gap between low-quality camera images and accurate vehicle identification. By comparing visual signatures rather than manually reviewing images, the system achieves both speed (productivity) and accuracy in license plate recognition
3Adaptability or versatility
If optical character recognition is applied to diverse license plates with varying graphics, fonts, and slogans, then text recognition may occur, but high resolution is required due to competing indicia
Solution Approach 1:
The patent extracts visual signatures that capture the essential identifying features of license plates while filtering out competing indicia such as graphics, slogans, and varying fonts. By extracting only the relevant visual patterns necessary for identification, the system handles diverse license plate varieties without requiring high resolution to distinguish between competing elements
Solution Approach 2:
The visual signature acts as an intermediary representation that simplifies diverse license plate variations into comparable patterns. This intermediary form enables the system to adapt to various license plate designs, fonts, and graphics while maintaining recognition capability without requiring high-resolution images that would be necessary to resolve all visual details
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A method and system are disclosed for conducting text-based searches of images using a visual signature associated with each image. A measure of string similarity between a query and an annotation associated with each entry in a first database is computed (S212), and based upon the computed string similarity measures, a set of entries from the first database is selected (S220). Each entry of the first database also includes an associated visual signature. At least one entry is then retrieved from a second database based upon a measure of visual similarity between a visual signature of each of the entries in the second database and the visual signatures of the entries in the selected set. Information corresponding to the retrieved entries from the second database is then generated.