Columnar Storage Privacy via Segmented Encryption
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In columnar storage systems, protecting sensitive data like passport numbers is challenging due to the organized structure, which allows efficient searching and processing, violating privacy regulations by enabling deduction of other individuals' data through malicious attacks.
Innovation Solution
Implementing access restricting metadata and encrypting protected columns, splitting them into 'RANDOM KEY' and 'ENCRYPTED VALUE' columns, stored in dedicated tables, to restrict operations and ensure privacy, using data flow graphs to detect and prevent queries on protected data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If columnar storage is used to organize data efficiently, then search performance and processing speed are improved, but privacy protection deteriorates because protected data can be searched and processed
Solution Approach 1:
The patent segments the protected column into two separate columns: a random key column containing cryptographic random keys and an encrypted value column containing the encrypted protected data. This segmentation prevents direct indexing and searching on the protected data while maintaining efficient query performance through the random key mapping mechanism.
Solution Approach 2:
The patent introduces a random key as an intermediary between the protected data and the search query. Instead of directly indexing the protected column, the system uses the random key column as a mediator to map encrypted values to their corresponding protected data, thereby preventing direct search on protected data while maintaining efficient access.
2Productivity
If indexes are added to protected columns to enable searching, then query performance is improved, but privacy protection deteriorates due to ability to search and deduce other individuals' data
Solution Approach 1:
The patent segments the indexed structure by separating the random key column (which can be indexed) from the encrypted value column (which contains the protected data). This allows indexing on the random key for efficient query performance while preventing direct indexing on the protected data, thereby eliminating the ability to deduce other individuals' data through search operations.
3Object-affected harmful factors
If protected columns are encrypted and stored in dedicated tables, then privacy protection is improved, but device complexity increases due to additional tables and metadata
Solution Approach 1:
The patent merges the protected column with the random key column and encrypted value column into a unified table structure with consistent data types and storage formats. This merging approach maintains privacy protection through encryption while minimizing the increase in device complexity by using a standardized table design that can be efficiently managed by existing database systems.
Data Source
AI summary
In privacy restrictions for columnar storage, a query including operations on one or more protected columns is received. The one or more protected columns are specified with one or more access restricting metadata in a columnar table. It is determined whether the received query comprises operations on the one or more protected columns specified with the one or more access restricting metadata. The execution of the query is restricted and a constraint violation is displayed, based on the determination.