Sourceblock Codebook Encoding for Anonymized Data Compaction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The rapid growth of data storage demand, driven by social media, cloud data centers, and biotech industries, has outpaced the production of physical storage capacity, leading to a bottleneck in data storage and transmission, with existing solutions like data compression offering limited relief, especially for multimedia data and posing security concerns as quantum computing approaches.
Innovation Solution
A system and method for encoding anonymized datasets by dividing them into sourceblocks, counting occurrences, anonymizing the tally record, and using a data deconstruction engine with a library manager to create a codebook and optimize it for compacting and encrypting data, allowing for efficient storage and transmission.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If data compression is used to increase storage capacity, then storage efficiency improves, but data security and privacy protection deteriorate
Solution Approach 1:
The patent segments data into sourceblocks and processes them through multiple independent codebooks. Each codebook operates on specific portions of the data, creating a distributed encoding structure that enhances both compression efficiency and security. The segmentation allows parallel processing and reduces the risk concentration in a single encoding system.
Solution Approach 2:
The system dynamically changes encoding parameters by selecting different codebooks based on data characteristics. Multiple codebooks with varying compression algorithms and security parameters are employed, allowing the system to adaptively optimize both compression ratio and security level for different data portions, thereby resolving the trade-off between storage efficiency and security.
2Reliability
If data is anonymized to protect privacy, then data security improves, but data utility and analysis capability deteriorate
Solution Approach 1:
The patent performs preliminary anonymization and encoding actions before data storage and transmission. By pre-processing data through multiple codebooks and creating anonymized representations, the system establishes privacy protection in advance while preserving analytical utility through the reversible nature of the encoding process.
Solution Approach 2:
The system introduces codebooks as intermediary layers between raw data and storage/transmission. These intermediaries transform data into anonymized forms that protect privacy while maintaining the structural information needed for analysis. The codebooks act as mediators that enable both privacy protection and data utility.
3Quantity of substance
If multiple codebooks are used for data compaction, then compression efficiency improves, but system complexity increases
Solution Approach 1:
The patent divides the encoding system into multiple independent codebook modules, each handling specific data portions. This segmentation allows parallel processing and independent optimization of each codebook, improving overall compression efficiency while managing complexity through modular architecture.
Solution Approach 2:
The system employs multiple codebooks that serve universal functions across different data types and portions. Each codebook is designed to be multi-functional, handling various compression scenarios, which reduces the need for specialized complex systems for each data type, thereby managing overall system complexity.
Data Source
AI summary
A system and method for encoding anonymized dataset. A dataset may be pre-processed by dividing into a plurality of sourceblocks at all reasonable sourceblock lengths, and then counting how many times each sourceblock occurs in the dataset, resulting in a tally record of tokens and their count value. This tally record may then be anonymized and transmitted as an anonymized tally record to a data deconstruction engine which combined with a library manager creates a codebook and performs optimization techniques on the codebook. The received anonymized tally record may be parsed into individual tokens by identifying the tokens with the highest count value. The tokens may then be sent, in descending order of count value, to the library manger where each token may be assigned a codeword. Then a half-backed codebook is created using the tokens and each token's unique codeword, before sending the half-backed codebook to a system user.