Encoded Vector Similarity Search for Secure In-Memory Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data transfer methods for sensitive information are vulnerable to interception and decryption by sniffers, compromising data security, especially when using encryption keys.
Innovation Solution
Implementing a secure in-memory unit on an associative processing unit (APU) with a decryptor, encoded vector data store, and similarity searcher for encrypted data processing, utilizing neural proxy hash encoding to perform secure similarity searches and data transfers.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data is transmitted encrypted using known encryption methods, then data security during transmission is improved, but the complexity of encryption/decryption operations increases
Solution Approach 1:
The patent applies preliminary action by pre-computing and storing encoded versions of data vectors in an encoded vector data store before actual search operations. This allows the system to perform similarity searches on encoded vectors without repeatedly encrypting and decrypting the original data, thereby reducing operational complexity while maintaining security.
Solution Approach 2:
The patent introduces encoded vectors as an intermediary representation between the original data and the search query. Instead of directly comparing encrypted data with search queries, the system encodes both the data and queries into vector representations, performs similarity search on these encoded vectors, and only decrypts the final results. This intermediary encoding layer simplifies the search operation while preserving data security.
2Reliability
If traditional similarity search is performed on encrypted data, then data security is maintained, but search accuracy and efficiency deteriorate
Solution Approach 1:
The patent replaces traditional mechanical similarity search methods with a neural network-based encoding system. Instead of using conventional search algorithms on encrypted data, the system uses neural networks to encode data into vectors that capture semantic meaning, enabling efficient similarity search while maintaining security through the encoding transformation.
Solution Approach 2:
The patent changes the parameter representation from raw data to encoded vector representations. By transforming data into vector space where similarity can be efficiently computed, the system achieves both security (through encoding) and efficiency (through vector similarity search). The encoded vectors serve as transformed parameters that enable faster search operations compared to traditional methods.
3Ease of operation
If data is stored in traditional memory structures, then data accessibility is maintained, but security against sniffers deteriorates
Solution Approach 1:
The patent introduces an intermediary encoding layer between the data storage and the processing units. Data is encoded into vectors before being stored or processed, creating a security barrier that prevents sniffers from directly accessing or intercepting meaningful data. The encoded vectors can still be efficiently processed and searched, maintaining accessibility while blocking harmful access.
Solution Approach 2:
The patent creates a copy of the data in encoded vector form rather than storing the original data directly. This encoded copy serves as the working representation for search operations, while the original data remains protected. The encoded vectors can be manipulated and searched without exposing the underlying sensitive information to sniffers or unauthorized access.
Data Source
AI summary
A system includes a secure, in-memory unit implemented on an associative processing unit (APU) for performing a secure similarity search. The unit implements a decryptor, an encoded vector store and a similarity searcher. The decryptor decrypts an encrypted, encoded vector into an encoded vector. The encoded vector data store stores a plurality of encoded search candidate vectors. The similarity searcher performs a similarity search between an encoded search query vector and the plurality of encoded search candidate vectors.