Stochastic Generative Hashing for Similarity Search
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing binary hashing techniques face challenges in learning optimal hash functions due to heuristic objective functions and crude handling of binary constraints, leading to inferior results in similarity search applications.
Innovation Solution
A stochastic generative hashing approach that uses a machine-learned hashing model and a generative model to optimize binary hash codes through a Minimum Description Length principle and distributional stochastic gradient descent, avoiding direct optimization of binary variables and leveraging stochastic neurons for reparameterization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If brute-force linear search is used to ensure accurate similarity search, then search accuracy is improved, but processing time and computational cost increase significantly
Solution Approach 1:
The patent extracts the essential similarity information into compact binary hash codes, separating the full data representation from the similarity computation. By representing high-dimensional data as binary codes, the system extracts only the necessary features for similarity search, enabling efficient comparison without processing the complete original data.
Solution Approach 2:
The patent transforms the similarity search problem by changing the parameter representation from continuous high-dimensional vectors to discrete binary codes. This parameter transformation allows the use of efficient binary comparison operations instead of computationally expensive Euclidean distance calculations, resolving the contradiction between accuracy and speed.
2Productivity
If binary hashing is used to reduce processing time, then search speed is improved, but hash function quality deteriorates due to heuristic methods
Solution Approach 1:
The patent implements feedback mechanisms where the hashing model is trained using gradient descent optimization. The loss function provides feedback on the quality of binary hash codes, allowing the system to iteratively improve hash function quality by adjusting model parameters based on performance metrics from similarity search tasks.
Solution Approach 2:
The patent replaces traditional mechanical heuristic hashing methods with a learned stochastic hashing system. Instead of using fixed heuristic rules for binary code generation, the system uses a trained neural network model that adaptively learns optimal hashing strategies, substituting deterministic mechanical processes with data-driven stochastic processes.
3Measurement precision
If direct optimization of binary variables is used to improve hash code quality, then binary hash accuracy is improved, but training convergence becomes difficult and unreliable
Solution Approach 1:
The patent inverts the optimization approach by not directly optimizing binary variables. Instead of attempting to directly optimize discrete binary codes, the system optimizes continuous model parameters that generate the binary codes through a stochastic process. This inversion transforms an intractable discrete optimization problem into a tractable continuous optimization problem with reliable convergence.
Solution Approach 2:
The patent introduces an intermediary stochastic process between the continuous model parameters and the discrete binary hash codes. The softmax function with temperature parameter acts as an intermediary, allowing smooth transitions and gradient flow while ultimately producing binary outputs. This intermediary layer enables reliable training convergence while maintaining binary hash accuracy.
Data Source
AI summary
The present disclosure provides systems and methods that perform stochastic generative hashing. According to one example aspect, a machine-learned hashing model that generates a binary hash for an input can be trained in conjunction with a machine-learned generative model that reconstructs the input from the binary hash. The present disclosure provides a novel generative approach to learn hash functions through Minimum Description Length principle such that the learned hash codes maximally compress the dataset. According to another example aspect, the present disclosure provides an efficient learning algorithm based on the stochastic distributional gradient, which avoids the notorious difficulty caused by binary output constraints, to jointly optimize the parameters of the hashing model and the associated generative model. The present disclosure also provides extensive experiments which show that the systems and methods described herein achieve better retrieval results than the existing state-of-the-art methods.