Data Portioning for Local Operations on Compressed Records
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Computational storage devices struggle to perform meaningful operations on compressed or encrypted data portions stored locally due to the inability to restore them to their original form, leading to inefficient data transmission and processing.
Innovation Solution
Data is divided into portions before compression or encryption, allowing computational storage devices to perform operations like decryption or decompression locally, reducing the need for extensive data transfer and enhancing processing efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If data is compressed or encrypted before storage, then storage efficiency is improved, but computational storage devices cannot perform meaningful operations on the stored data
Solution Approach 1:
The patent divides data into multiple portions before compression or encryption, storing different portions on different computational storage devices. Each device receives a specific portion that can be independently processed. This segmentation allows computational operations to be performed on individual portions while maintaining overall data security and storage efficiency.
Solution Approach 2:
The patent performs preliminary actions of data division and selective compression/encryption before storage. By pre-processing data to identify and mark specific portions that require compression versus those that need to remain uncompressed for computational processing, the system enables future computational operations without requiring full decompression of stored data.
2Productivity
If data is divided into portions before compression, then local operations on computational storage devices are enabled, but data transmission and processing complexity increase
Solution Approach 1:
The system segments data into portions with different compression levels, allowing computational storage devices to process only the uncompressed or lightly compressed portions locally. This reduces the need for extensive data transmission between devices while maintaining processing efficiency.
Solution Approach 2:
Different portions of data are treated with different quality levels regarding compression. Some portions are kept uncompressed or lightly compressed to enable local computational operations, while other portions are heavily compressed for storage efficiency. This local quality differentiation optimizes both processing efficiency and transmission complexity.
Data Source
AI summary
A method for data compression may include scanning input data, performing, based on the scanning, a compression operation to generate compressed data using the input data, finding, based on the scanning, a delimiter in the input data, and generating, based on a position of the delimiter in the input data, a portion of data using the compressed data. The input data may include a record, the delimiter indicates a boundary of the record, and the portion of data may include the record. The generating may include generating the portion of data based on a portion size. The portion size may be a default portion size. The portion size may be based on a default portion size and a length of a match in the input data.