Hierarchical Aggregation for Local Processing in Computational Storage
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Computational storage devices struggle to perform operations like filtering and scanning on compressed or encrypted data portions stored across multiple devices, as they cannot restore these portions to their original form locally, leading to inefficient data retrieval and increased network traffic.
Innovation Solution
Data is divided into portions before compression and encryption, allowing computational storage devices to perform operations like decryption and decompression locally, reducing the need for data transfer and enhancing processing efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of energy
If data is compressed and encrypted before storage across multiple devices, then storage efficiency and security are improved, but the ability to perform local processing operations on the data is lost
Solution Approach 1:
The patent segments data into multiple portions and distributes them across different storage devices. Each portion can be independently processed locally without requiring full data reconstruction, enabling partial processing operations on encrypted/compressed data while maintaining security and reducing bandwidth usage.
Solution Approach 2:
The patent performs preprocessing operations (compression, encryption, segmentation) on data before storage. This preliminary action enables storage devices to perform certain operations locally on the pre-processed data portions without needing to fully decompress or decrypt the entire dataset, thus maintaining processing capability while preserving bandwidth efficiency.
2Reliability
If data is stored in fragmented portions across multiple storage devices, then storage scalability and security are improved, but data retrieval efficiency deteriorates due to increased coordination overhead
Solution Approach 1:
The patent divides data into multiple fragments stored across different devices, improving security through distribution. However, it implements hierarchical aggregation where storage nodes can perform local operations on available fragments and only coordinate with other nodes when necessary, thus maintaining security while improving retrieval efficiency through reduced coordination overhead.
Solution Approach 2:
The patent enables storage nodes to autonomously perform processing operations on locally stored data portions without requiring constant coordination with other nodes. This self-service capability allows nodes to independently execute operations on their local fragments, significantly improving data retrieval efficiency while maintaining the security benefits of distributed storage.
3Productivity
If computational storage devices perform operations on distributed data portions, then processing parallelism is improved, but network traffic increases due to data aggregation requirements
Solution Approach 1:
The patent segments data across multiple storage devices to enable parallel processing operations. Each device can process its local data portion independently and simultaneously, achieving processing parallelism. The hierarchical aggregation structure ensures that only necessary data portions are aggregated over the network, minimizing network traffic while maintaining high parallelism.
Solution Approach 2:
The patent extracts and performs processing operations locally on distributed data portions at storage nodes, eliminating the need to aggregate all data centrally for processing. This extraction of processing capability to the edge (storage devices) maintains parallelism while reducing network traffic by keeping processing local wherever possible.
4Quantity of substance
If data portions are compressed and encrypted before storage, then storage density is improved, but the ability to perform filtering and scanning operations locally is lost
Solution Approach 1:
The patent applies compression and encryption as preliminary actions before data storage, achieving high storage density. However, the segmentation and hierarchical structure allow storage devices to perform filtering and scanning operations on metadata or specific data portions without full decompression/decryption, maintaining local processing capability while preserving storage efficiency.
Solution Approach 2:
The patent implements local quality by allowing different processing capabilities at different levels of the hierarchy. Storage devices can perform certain operations (like filtering based on metadata or scanning compressed data patterns) locally without full decompression, while more complex operations are handled at higher hierarchical levels, thus maintaining both storage density and local processing capability.
Data Source
AI summary
A method for computational storage may include storing, at a storage device, two or more portions of data, wherein a first one of the two or more portions of data comprises a first fragment of a record and a second one of the two or more portions of data comprises a second fragment of the record, and performing, by the storage device, an operation on the first and second fragments of the record. The method may further include performing, by the storage node, a second operation on first and second fragments of a second record. The operation may include a data selection operation, and the method may further include sending a result of the data selection operation to a server. The method may further include sending a result of a first data selection operation to a server.