Anonymized Data Codebooks for Compaction and Encryption
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The rapid growth of data storage demand outpaces the capacity to store it, leading to storage and transmission bottlenecks, and existing data compression and encryption technologies are inadequate for securing and anonymizing data, especially with the rise of quantum computing and stringent data privacy regulations.
Innovation Solution
A system and method for data compaction and encryption of anonymized data records involves preprocessing datasets into sourceblocks, creating a codebook through tally records, and using multiple codebooks for optimal compaction and encryption, with additional security measures like shuffling and rotating codebooks.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If data compression is used to increase storage capacity, then storage efficiency is improved, but compression ratio decreases substantially for multi-media data or results in data degradation
Solution Approach 1:
The patent segments data into sourceblocks of various lengths and creates separate codebooks for different sourceblock lengths. This segmentation allows the system to optimize compression for each data type and length category, achieving higher overall compression ratios without degrading multi-media data quality.
Solution Approach 2:
The system dynamically selects sourceblock lengths and creates codebooks adaptively based on the specific data being compressed. Rather than using a fixed compression approach, the system adjusts parameters like sourceblock length and codebook structure to match the characteristics of the input data, maximizing compression efficiency for each case.
2Quantity of substance
If additional physical storage capacity is added to meet demand, then storage availability is improved, but storage infrastructure complexity increases
Solution Approach 1:
The patent creates compact representations (codebooks) of data that can be stored and transmitted efficiently. Instead of storing or transmitting all original data, the system stores compacted codebooks that reproduce the original data when needed, dramatically reducing the physical storage capacity required.
3Reliability
If data is anonymized to protect privacy, then data security is improved, but data utility for analysis decreases
Solution Approach 1:
The system applies different levels and types of anonymization to different portions of the data based on local requirements. Rather than uniformly anonymizing all data, the system can apply targeted anonymization to specific fields or records while preserving the utility of other data portions for analysis.
Data Source
AI summary
A system and method for data compaction and encryption of anonymized data records. A dataset may be pre-processed by dividing into sourceblocks at reasonable intervals and tallying each sourceblock's frequency, creating a tally record of tokens and count values. This tally record may then be anonymized and transmitted to a data deconstruction engine which combined with a library manager creates a codebook and performs optimization techniques on the codebook. The data deconstruction engine and library manager may be distributed across multiple nodes or devices. The received anonymized tally record may be parsed into individual tokens by identifying the tokens with the highest count value. The tokens may then be sent descending order of count value to the library manger where each token may be assigned a codeword. A half-backed codebook is then created using the tokens and each token's unique codeword, before sending the half-backed codebook to a system user.