Data Compression Partitioning for Local Record Restoration
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Computational storage devices struggle to efficiently process and restore data that has been compressed or encrypted, as they cannot locally decompress or decrypt data stored in modified form, leading to inefficiencies in data retrieval and processing.
Innovation Solution
A method and system for dividing data into portions before compression or encryption, allowing computational storage devices to restore and perform operations on individual modified portions, which includes scanning for delimiters to determine portion sizes and integrating data dividing with compression or encryption operations for efficient data processing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If data is compressed or encrypted before storage, then storage efficiency and security are improved, but computational storage devices cannot locally restore or process the data, leading to increased data transfer requirements
Solution Approach 1:
The patent divides data into multiple portions before compression or encryption, storing each portion separately. This segmentation allows computational storage devices to restore and process individual portions locally without needing to retrieve and decrypt/decompress the entire dataset, thus maintaining both storage efficiency and local processing capability.
Solution Approach 2:
The patent performs data division into portions before the compression or encryption operation. This preliminary action enables computational storage devices to later restore individual portions and perform operations on them locally, avoiding the need to transfer and process complete compressed or encrypted data sets.
2Ease of operation
If data is divided into portions with delimiters, then local restoration and processing of individual portions is enabled, but data structure complexity increases
Solution Approach 1:
The patent introduces delimiters as intermediary elements between data portions. These delimiters serve as clear markers that enable computational storage devices to easily identify portion boundaries during restoration, simplifying the local processing operation while maintaining an organized data structure.
Solution Approach 2:
The patent uses delimiters as distinctive markers (analogous to color changes) that visually or structurally stand out from the actual data content. This makes portion boundaries easily detectable during restoration operations without requiring complex analysis of the data structure.
3Ease of manufacture
If default portion sizes are used for data division, then processing is simplified, but delimiter positions may not align with natural data boundaries, reducing compression efficiency
Solution Approach 1:
The patent makes the portion size dynamic by allowing it to be adjusted based on delimiter positions. Instead of using fixed default sizes, the system adapts portion boundaries to align with natural data delimiters, thereby maintaining processing simplicity while improving compression efficiency through better data organization.
Solution Approach 2:
The patent changes the portion size parameter based on the position of delimiters in the data. This parameter adjustment allows the system to maintain alignment with natural data boundaries, improving compression efficiency without significantly complicating the processing workflow.
Data Source
Figure 1A~1B
Figure 2A~2B
Figure 3A~3B
AI summary
A method for data compression may include scanning input data, performing, based on the scanning, a compression operation to generate compressed data using the input data, finding, based on the scanning, a delimiter in the input data, and generating, based on a position of the delimiter in the input data, a portion of data using the compressed data. The input data may include a record, the delimiter indicates a boundary of the record, and the portion of data may include the record. The generating may include generating the portion of data based on a portion size. The portion size may be a default portion size. The portion size may be based on a default portion size and a length of a match in the input data.