Zero-Knowledge Query Plans for Secure External Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing machine learning services face challenges in processing data stored in secure or external storage systems without full access, as methods like full homomorphic encryption require significant computational power and user-side semantic embeddings limit model updates.
Innovation Solution
A machine learning model generates a query plan comprising a dependency tree of operations, which is transmitted to an external storage system for execution, allowing processing without direct access to the data, using search candidates, merging, and translating operations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If full homomorphic encryption is used to process encrypted data, then data security is improved, but computational power requirements and communication overhead increase significantly
Solution Approach 1:
The patent extracts the computational operations from the data itself by generating a query plan that specifies operations to be performed on encrypted data. The query plan separates the logical processing steps from the actual data, allowing the database to execute operations on ciphertext without decrypting it, thereby reducing computational overhead while maintaining security.
Solution Approach 2:
The patent applies preliminary action by generating the query plan before data processing occurs. The query plan is created in advance based on the user's natural language query, translating it into a structured format that the database can execute efficiently on encrypted data. This pre-planning reduces runtime computational requirements and communication overhead.
2Productivity
If user side semantic embeddings are used to encode data, then processing capability is improved, but model update ability is restricted due to coupling server side semantic ability with user side computational ability
Solution Approach 1:
The patent introduces a query plan as an intermediary between the user's natural language query and the database's encrypted data storage. The query plan serves as a mediator that translates user intent into executable operations without requiring the user's device to have advanced semantic processing capabilities. This decouples the semantic understanding requirement from the user side, allowing the server to maintain and update semantic models independently.
3Measurement precision
If query plan with dependency tree is generated, then query processing accuracy is improved, but system complexity increases
Solution Approach 1:
The patent segments the query processing into distinct operations represented as a dependency tree. Each node in the tree represents a specific operation (e.g., filter, join, aggregate) that can be executed independently on encrypted data. This segmentation allows for precise control over the processing sequence and ensures accurate execution while maintaining manageable system complexity through modular operation design.
Data Source
AI summary
Aspects of the disclosure are directed to processing data stored in an external storage system using a machine learning model without exposing the data. Processing data may refer to document retrieval or question-answering, as examples. The external storage system may refer to a storage system separate from a processing system for the machine learning model. To process the data, the machine learning model generates a plan and provides the plan to the external storage system for execution and response generation. Generating the plan allows for accurately processing data without requiring access to the data.