Variable-Length Hash Code Nearest-Neighbor Query Method
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing high-dimensional data retrieval methods using hash techniques fail to fully utilize distribution information and result in low information retention for query points due to fixed-length hash codes, leading to reduced retrieval accuracy.
Innovation Solution
A high-dimensional data nearest-neighbor query method utilizing variable-length hash codes, which involves low-dimensional mapping, frequency-based sub-data set creation, adaptive compression, and integration of random Fourier features to form a final query table, allowing for maximum data information retention and improved accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If unified fixed-length hash codes are used for all data sets, then hash code consistency is achieved, but data information retention is reduced
Solution Approach 1:
The patent applies dynamics by transitioning from fixed-length hash codes to variable-length hash codes. Different data sets are assigned hash codes of different lengths based on their distribution characteristics and frequency information, allowing the system to adaptively retain appropriate information for each data set while maintaining operational consistency through the structured variable-length framework
Solution Approach 2:
The patent implements local quality by treating different data sets differently based on their local characteristics. Each data set receives a hash code length tailored to its specific distribution properties and frequency distribution, rather than applying a uniform length to all data sets. This allows optimal information retention for each local data characteristic
2Productivity
If short hash codes are used for query points, then processing efficiency is improved, but information reservation capability is reduced
Solution Approach 1:
The patent applies parameter changes by dynamically adjusting the hash code length parameter based on the query point's characteristics and the data set distribution. Rather than using a fixed short length for all query points, the system selects optimal lengths that balance processing efficiency with information retention requirements for each specific query scenario
3Measurement precision
If existing hash techniques are used, then high-dimensional retrieval is achieved, but distribution information utilization is insufficient
Solution Approach 1:
The patent implements feedback by using frequency information and distribution characteristics of the data sets to inform the hash code generation process. The system analyzes the distribution properties and uses this feedback to determine optimal hash code lengths and parameters, creating a closed-loop system where distribution information directly influences the hashing strategy
Solution Approach 2:
The patent applies preliminary action by pre-analyzing the distribution characteristics and frequency information of data sets before generating hash codes. This preliminary analysis allows the system to optimize hash code parameters in advance based on the actual data distribution, rather than using generic hashing parameters
Data Source
AI summary
A high-dimensional data nearest-neighbor query method based on variable-length hash codes is disclosed. Specifically, in this method, hash codes with the same code frequency are taken as a sub-data set, all the sub-data sets are ranked, a compression ratio is set for each sub-data set, the sub-data sets are compressed and trained according to the compression ratios, and hash codes and original codes corresponding to the trained sub-data sets are obtained; the hash code of each trained sub-data sets is copied to obtain multiple replicas, and the original codes and the corresponding replicas are strung to obtain strung hash codes which are integrated to form a final nearest-neighbor query table; and, a query code is obtained, and the nearest-neighbor query table is searched for a nearest-neighbor data set to complete query. The query efficiency and accuracy are greatly improved according to the invention.