Processor-In-Memory Image Search Using Global and Local Descriptors
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional image search methods rely heavily on textual metadata and user-provided tags, which can be inaccurate or missing, making it difficult to efficiently search and retrieve images based on their inherent properties.
Innovation Solution
A processor-in-memory (PIM) system that performs image searches by using both global and local image descriptors, allowing for efficient image classification and comparison without relying on user-provided metadata, and utilizing in-memory processing to reduce data traffic and energy consumption.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If conventional text search methods using bag of words are used, then text-based image searching is enabled, but search accuracy deteriorates when metadata is missing or inaccurate
Solution Approach 1:
The patent introduces an intermediary processing layer that extracts visual features directly from images using machine learning models, acting as a mediator between the query and the image database. This intermediary feature extraction mechanism bypasses the need for accurate metadata, enabling searches to work effectively even when textual descriptions are missing or incorrect.
Solution Approach 2:
The patent replaces the mechanical text-based search system with a hybrid approach that incorporates visual feature extraction through machine learning. Instead of relying solely on textual metadata (mechanical system), the system substitutes in direct image analysis using neural networks and feature vectors, enabling more accurate searching independent of metadata quality.
2Productivity
If large scale web image searches are performed, then comprehensive image searching is enabled, but energy consumption increases significantly
Solution Approach 1:
The patent segments the image search process into distinct stages: query feature extraction, database feature indexing, similarity computation, and result ranking. By segmenting the workflow and performing feature extraction once during indexing, the system avoids redundant computations during actual searches, significantly reducing energy consumption while maintaining comprehensive search capability.
Solution Approach 2:
The patent performs preliminary action by pre-extracting visual features from all images in the database during an indexing phase before actual searches are conducted. These pre-computed feature vectors are stored and reused for multiple queries, eliminating the need to re-process images during each search operation, thereby reducing energy consumption for large-scale searching.
3Ease of operation
If image features are tagged with textual descriptions, then text-based searching is enabled, but reliability of search results deteriorates when tags are inaccurate
Solution Approach 1:
The patent introduces visual feature extraction as an intermediary that directly analyzes image content through machine learning models, bypassing the unreliable textual tag system. This intermediary mechanism extracts semantic features directly from pixels, providing a reliable alternative to metadata-dependent searching and improving result reliability independent of tag accuracy.
Solution Approach 2:
The patent enables images to self-describe their content through automatic visual feature extraction using machine learning models. Instead of relying on external textual tags that may be inaccurate, the system allows images to generate their own feature representations directly from their visual content, making the search system self-sufficient and independent of external metadata quality.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A method and apparatus for performing a search in a processor-in-memory (PIM) system having a first processor and at least one memory module includes receiving one or more images by the first processor. The first processor sends a query for a search of memory for a matching image to the one or more images to at least one memory module, which searches memory in the memory module, in response to the received query. The at least one memory module sends the results of the search to the first processor, and the first processor performs a comparison of the received results from the at least one memory module to the received one or more images.