Compressing Encrypted Database Columns Using Dictionary Index
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The existing database management systems face challenges in efficiently compressing encrypted data, leading to increased storage sizes and reduced system performance due to the larger size of cipher text and the need for additional system resources during query processing.
Innovation Solution
Implementing a method that compresses cipher text in database table columns using a non-adaptive, lossless compression algorithm, such as the Dictionary Index method, which allows for column-level compression, optimizing data type-specific compression, and using presence bits to manage compressed values, thereby maintaining transparency to ETL operations and queries.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If encryption is applied to sensitive data columns, then data security is improved, but storage space consumption increases significantly
Solution Approach 1:
The patent segments the data handling process into distinct phases: encryption occurs during data loading, while compression is applied separately during storage. This segmentation allows each process to optimize for its specific purpose without compromising the other, resolving the contradiction between maintaining security through encryption and reducing storage through compression.
Solution Approach 2:
The patent changes the parameter of data representation by transforming encrypted data into a compressed format using dictionary-based algorithms. This parameter change allows the same secure encrypted information to occupy less storage space, directly resolving the contradiction between security maintenance and storage optimization.
2Quantity of substance
If compression is applied to clear text values, then storage space is reduced, but encrypted cipher text cannot be compressed
Solution Approach 1:
The patent applies compression in advance during the data loading phase, before the data is stored in the database. This preliminary compression action ensures that encrypted data is already in its compressed form when stored, eliminating the need for compression during query operations and maintaining both storage efficiency and encryption security.
Solution Approach 2:
The patent introduces an intermediary compression layer between the encrypted data and its storage representation. This intermediary process transforms the encrypted binary data into a compressed format using dictionary algorithms, allowing compression to be applied to encrypted data without compromising security or requiring changes to the encryption algorithm itself.
3Reliability
If larger cipher text sizes are stored, then encryption security is maintained, but disk I/O performance deteriorates
Solution Approach 1:
The patent segments the data processing workflow into distinct stages: encryption during loading, compression during storage, and decompression during retrieval. This segmentation allows the system to maintain full encryption security while achieving compression benefits for improved disk I/O performance, as each stage optimizes for its specific function without compromising the other.
Solution Approach 2:
The patent changes the physical representation parameter of encrypted data by applying compression algorithms that reduce the storage footprint of cipher text. This parameter change enables smaller data blocks to be stored and transferred, directly improving disk I/O performance while maintaining the same level of encryption security through the preserved encrypted content.
Data Source
AI summary
The subject matter herein relates to database management systems and, more particularly, compression of encrypted data in database management systems. Various embodiments provide systems, methods, and software that compress encrypted column values stored tables. Some other embodiments include declaring tables with column encrypt and compress attributes.