Anonymized Data Codebooks for Compression-Security Tradeoffs
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The rapid growth of data storage demand outpaces the capacity to store it, leading to storage and transmission bottlenecks, and existing data compression and encryption technologies are inadequate for securing and anonymizing data, especially with the rise of quantum computing and stringent data privacy regulations.
Innovation Solution
A system and method for data compaction and encryption of anonymized data records involves dividing datasets into sourceblocks, counting occurrences, creating a codebook with unique codewords, and using multiple codebooks for optimal compaction and security, employing a hierarchical library manager for advanced optimization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If data compression is used to increase storage capacity, then storage efficiency improves, but data security and privacy protection deteriorate
Solution Approach 1:
The patent divides data into multiple segments or blocks, applying different compression and encryption techniques to different portions. This segmentation allows the system to achieve high compression ratios on certain data portions while maintaining strong security on sensitive portions through selective application of cryptographic functions.
Solution Approach 2:
The patent employs a composite approach by combining multiple compression algorithms and cryptographic functions into a unified system. This composite structure enables the system to leverage the strengths of different methods (e.g., lossless compression for text, lossy compression for media, cryptographic hashing for sensitive data) to simultaneously achieve high compression ratios and robust security.
2Reliability
If traditional encryption methods are used to protect data, then data security improves, but storage efficiency and transmission bandwidth deteriorate
Solution Approach 1:
The patent merges compression and encryption operations into a unified process. By integrating these functions, the system avoids the traditional approach of applying encryption first (which would increase data size) and then compression, instead achieving compression while simultaneously providing cryptographic protection, thereby maintaining both security and efficiency.
Solution Approach 2:
The patent dynamically adjusts cryptographic parameters such as key length, hash function selection, and encryption intensity based on the sensitivity and type of data being processed. This parameter optimization allows the system to use stronger cryptography only when necessary, reducing the overhead on storage and transmission while maintaining adequate security levels.
3Reliability
If data is anonymized to protect privacy, then data privacy improves, but data utility and analytical value deteriorate
Solution Approach 1:
The patent applies partial anonymization by selectively anonymizing only the portions of data that require privacy protection while leaving other portions in their original or less-anonymized state. This selective approach preserves the analytical utility of data that does not require anonymization while still protecting sensitive information, thereby reducing the overall loss of data utility.
Solution Approach 2:
The patent introduces cryptographic hash functions and anonymization intermediaries that transform sensitive data into protected representations while preserving certain statistical properties. These intermediaries act as mediators that maintain enough data structure and relationships to allow analytical operations while preventing direct identification or reconstruction of original sensitive values.
4Reliability
If multiple codebooks are used to enhance security, then data security improves, but system complexity increases
Solution Approach 1:
The patent organizes multiple codebooks into segmented groups or layers, each serving specific security or compression purposes. This structured segmentation allows the system to manage complexity by processing data through different codebook layers in a systematic manner, rather than treating all codebooks as a single complex unit.
Solution Approach 2:
The patent designs the multiple codebooks to serve multiple functions simultaneously - some codebooks are optimized for compression while others provide cryptographic protection, and many serve both purposes. This multi-functionality reduces overall system complexity by eliminating the need for separate dedicated systems for each function.
Data Source
AI summary
A system and method for data compaction and encryption of anonymized data records. A dataset may be pre-processed by dividing into a plurality of sourceblocks at all reasonable sourceblock lengths, and then counting how many times each sourceblock occurs in the dataset, resulting in a tally record of tokens and their count value. This tally record may then be anonymized and transmitted to a data deconstruction engine which combined with a library manager creates a codebook and performs optimization techniques on the codebook. The received anonymized tally record may be parsed into individual tokens by identifying the tokens with the highest count value. The tokens may then be sent, in descending order of count value, to the library manger where each token may be assigned a codeword. A half-backed codebook is then created using the tokens and each token's unique codeword, before sending the half-backed codebook to a system user.