Computational Storage Data Resizing for Fragmented Record Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Computational storage devices face challenges in performing meaningful operations on data portions that have been modified, such as compressed or encrypted, as they cannot restore the data to its original form, leading to inefficiencies in data processing and increased bandwidth and storage requirements.
Innovation Solution
The method involves dividing data into portions before modifying it, allowing computational storage devices to perform operations like selection, filtering, and decryption locally, and using peer-to-peer communication to manage and reconstruct records across multiple storage devices, enabling efficient data processing and reduction in data transfer.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If data is divided into portions and stored across multiple storage devices, then storage capacity and distribution are improved, but data reconstruction complexity and bandwidth requirements increase
Solution Approach 1:
The patent divides data into multiple portions and stores them across different storage devices. Each portion contains fragments of records, allowing distributed storage while enabling reconstruction of complete records by gathering fragments from multiple devices. This segmentation resolves the contradiction by improving storage capacity through distribution while providing systematic methods for reconstruction.
Solution Approach 2:
The patent introduces intermediary components including a host system and coordination logic that manage the reconstruction process. These intermediaries coordinate between storage devices, retrieve necessary fragments, and assemble complete records, thereby reducing the complexity burden on individual storage devices and managing the overall reconstruction process efficiently.
2Quantity of substance
If data is modified (compressed or encrypted) before storage, then storage efficiency and security are improved, but computational storage devices cannot perform meaningful operations on the modified portions
Solution Approach 1:
The patent segments data into portions before modification, allowing computational storage devices to operate on individual portions or fragments. By dividing data into manageable segments, the system enables local processing operations on uncompressed or less-encrypted portions while maintaining overall storage efficiency through selective compression and encryption of different data segments.
Solution Approach 2:
The patent applies different modification strategies to different data portions based on their processing requirements. Some portions remain in uncompressed or lightly encrypted states to enable local computational operations, while other portions undergo full compression or encryption for optimized storage. This local quality approach resolves the contradiction by allowing both storage efficiency and processing capability to coexist in different regions of the data.
3Speed
If complete records are stored on a single storage device, then data processing speed is improved, but bandwidth and storage requirements increase
Solution Approach 1:
The patent segments complete records into fragments distributed across multiple storage devices. When processing is needed, only the necessary fragments are retrieved and assembled, rather than transferring entire records. This segmentation reduces bandwidth requirements by transmitting only essential data portions while maintaining processing speed through localized fragment assembly.
Solution Approach 2:
The patent performs preliminary organization of data fragments with metadata that identifies their locations and relationships. This preliminary action enables efficient retrieval and assembly of fragments during processing operations, reducing the time and bandwidth needed for data reconstruction while maintaining processing speed through pre-established fragment indexing and location tracking.
Data Source
AI summary
A method for computational storage may include storing, at a storage device, a first portion of data, wherein the first portion of data may include a first fragment of a record, and a second portion of data may include a second fragment of the record, and appending the second fragment of the record to the first portion of data. The method may further include performing, at the storage device, an operation on the first and second fragments of the record. The method may further include determining that the first portion of data may include a first fragment of a record, and a second portion of data may include a second fragment of the record, wherein appending the second fragment of the record to the first portion of data may include appending, based on the determining, the second fragment of the record to the first portion of data.