Fabric-Enabled Computational Storage Pipelined Data Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current remote storage systems, such as JBOF and JBOD, face architectural limitations that hinder performance and increase network congestion due to excessive metadata transfer, necessitating improved data flow and computational resource management within storage devices.
Innovation Solution
The introduction of a Fabric-enabled Computational Storage (FCS) architecture, which includes a Storage Processing Unit (SPU), accelerators, Ethernet connection, and Data Processing Unit (DPU), utilizing a pipelined computation model to manage internal data flow and execute tasks without exposing internal resources to the host, thereby reducing network data transfer and enhancing resource utilization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If remote storage systems (JBOF/JBOD) are used for high capacity, then storage capacity is improved, but network congestion increases due to excessive metadata transfer
Solution Approach 1:
The patent segments the storage system into multiple independent storage nodes that can operate autonomously. Each node manages its own metadata locally, eliminating the need to transfer all metadata over the network. This segmentation allows the system to scale capacity while maintaining network efficiency, as each node handles only its local metadata operations.
Solution Approach 2:
The patent introduces a namespace service as an intermediary that manages metadata operations without requiring direct network transfer between storage controllers. The namespace service acts as a mediator that can cache, filter, and manage metadata requests, reducing the volume of metadata that needs to traverse the network infrastructure.
2Ease of operation
If storage controllers transfer metadata over network, then data access is enabled, but network bandwidth is consumed excessively
Solution Approach 1:
The patent implements preliminary action by caching frequently accessed metadata in the namespace service before actual data access operations occur. This pre-caching of metadata reduces the need for repeated network transfers during data access operations, as the metadata is already available in a readily accessible location within the storage fabric.
Solution Approach 2:
The patent uses copying by creating local copies of metadata in the namespace service and storage nodes. Instead of transferring the entire metadata set over the network for each operation, the system maintains replicated copies that can be accessed locally, significantly reducing network data transfer volume while preserving data access capability.
3Adaptability or versatility
If internal resources are exposed to host, then control flexibility is improved, but security and complexity increase
Solution Approach 1:
The patent introduces a namespace service as an intermediary layer between the host and internal storage resources. This mediator provides controlled access to storage resources without requiring the host to directly manage or understand the complex internal architecture. The namespace service handles resource management, allocation, and coordination, maintaining security while preserving control flexibility through standardized interfaces.
Solution Approach 2:
The patent implements self-service by enabling storage nodes to autonomously manage their own resources and operations without requiring direct host intervention. Each storage node can independently handle data operations, error correction, and resource management, reducing the complexity of host-side control while maintaining system adaptability through coordinated autonomous operations.
Data Source
AI summary
A storage device is disclosed. The storage device may include compute engines. The compute engines may include storage for data, a storage processing unit to manage writing data to the storage and reading data from the storage, a data processing unit to perform some functions on the data, and an accelerator to perform other functions on the data. An Ethernet component may receive a request at the storage device from a host over a network. A data processing coordinator may process the request using a compute engine.