Chunked File Compression for Random Access and Secure Storage
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing systems for storing LOB data face challenges such as duplicated storage, inefficient use of disk space due to lack of random access to compressed data, and insecure storage of sensitive information as LOB data is stored unencrypted.
Innovation Solution
A system that processes data by compressing file portions into individually accessible compression units, enabling de-duplication and random access, while also providing encryption for secure storage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If LOB data is compressed using existing algorithms, then disk space is reduced, but random access to compressed data is not provided and explicit destination LOB creation is required
Solution Approach 1:
The patent divides the LOB data into multiple compression units, where each unit is independently compressible and accessible. This segmentation allows random access to specific portions of compressed data without requiring decompression of the entire data set, while still achieving significant disk space reduction through compression of individual units.
2Ease of operation
If LOB data is stored unencrypted, then access and processing are simple, but security and compliance requirements are not met
Solution Approach 1:
The patent applies encryption selectively to compression units containing sensitive information while leaving other units unencrypted. This local quality approach maintains security for sensitive data while preserving simple access and processing for non-sensitive data, avoiding the need to encrypt the entire LOB data set.
3Quantity of substance
If compression algorithms are applied to LOB data, then storage efficiency improves, but the complexity of data management increases
Solution Approach 1:
The patent implements self-describing compression units that contain metadata about their own compression state, format, and content characteristics. This self-service capability allows the system to automatically manage compressed data without requiring complex external management mechanisms, thereby maintaining storage efficiency while reducing management complexity.
Data Source
AI summary
A method of processing data from a file includes obtaining a first portion of the file, creating a first compression unit by compressing at least the first portion of the file, obtaining a second portion of the file, creating a second compression unit by compressing at least the second portion of the file, and storing the first and second compression unit such that each of the first and the second compression units can be individually accessed. A method of processing data from a file includes receiving a request to access a portion of the file, determining one or more sub-units that include compressed data associated with the portion of the file, de-compressing the compressed data in the one or more sub-units to obtain de-compressed data, and transmitting the de-compressed data in response to the request.