Bag of Hash Bits Mobile Search Compression
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Mobile visual search systems face challenges in efficiently processing and transmitting local features due to limited memory and computation capacity on mobile devices, making it difficult to perform accurate and fast searches, especially with methods like vocabulary trees that require large codebooks.
Innovation Solution
The use of a Bag of Hash Bits (BoHB) technique, which compresses local features into similarity-preserving hash bits for efficient storage and transmission, allowing for fast matching and ranking of candidate media by combining geometry and boundary similarity scores, thereby improving search accuracy and speed.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of time
If local features are compressed to tens of bits and sent to the server, then transmission cost and time are reduced, but search accuracy and recall may deteriorate
Solution Approach 1:
The patent segments the local feature representation into two parts: (1) hash bits (tens of bits) that are compressed and transmitted to the server for initial filtering, and (2) visual words (larger representation) that are used for final ranking and accuracy. This segmentation allows the system to achieve fast transmission with the hash bits while maintaining search accuracy through the visual words in the final ranking stage.
Solution Approach 2:
The patent extracts the most critical information from local features by converting them to hash bits that capture the essential similarity characteristics. These hash bits are then transmitted separately from the full visual word representations, allowing the system to transmit only the necessary minimal data for initial matching while preserving the ability to achieve accurate search results through the visual words.
2Measurement precision
If vocabulary tree quantization is used with large codebooks, then search results are improved, but device memory and computation capacity requirements increase
Solution Approach 1:
The patent uses a two-stage approach where the first stage employs simple hash bit comparisons (tens of bits) that require minimal computation and memory, acting as a disposable filtering mechanism. Only after this initial filtering does the system proceed to the second stage with more complex visual word comparisons. This allows the system to achieve good search accuracy without requiring large codebooks or excessive computational resources on the mobile device.
Solution Approach 2:
The patent segments the search process into two stages: (1) a fast filtering stage using hash bits that requires minimal memory and computation, and (2) a precise ranking stage using visual words. This segmentation allows the system to use simple, low-resource operations for the majority of the search process while reserving complex operations only for the final ranking of candidate results.
3Speed
If local feature extraction is performed on the mobile device, then search speed is improved, but computation capacity and energy consumption increase
Solution Approach 1:
The patent performs computation-heavy operations (local feature extraction and visual word conversion) on the mobile device only for the query images, creating a disposable set of hash bits and visual words that are then transmitted to the server. The server performs the heavy computation for matching and ranking against the entire database. This approach allows the mobile device to prepare data efficiently without requiring sustained high computation capacity during the actual search process.
Solution Approach 2:
The patent introduces hash bits as an intermediary representation that bridges the mobile device and server. The mobile device extracts local features and converts them to hash bits, which are then transmitted to the server. The server uses these hash bits for efficient database searching and generates final results. This intermediary approach allows the mobile device to perform preliminary processing with limited resources while the server handles the computationally intensive database matching.
Data Source
AI summary
Determining ranked candidate media in response to media query data corresponding to a query media includes receiving the media query data including feature data of the query media, coordinate data, and boundary data, matching the features with corresponding features of an media database using the feature data to identify features in the media database within a predetermined hamming distance in a hash table from the corresponding features of the query media to obtain matched features in the media database, determining candidate media whose number of matched features exceeds a matched feature threshold, generating a geometry similarity score between the query media and each candidate media using the feature data and the coordinate data, generating a boundary similarity score between the query media and each candidate media using the boundary data, ranking the candidate media based on the numbers of matched features, the geometry similarity scores and the boundary similarity scores.