Remote Storage for SoC-XPU Hardware Microservices
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The storage capacity on SoC-XPU platforms is constrained and expensive, making it difficult to extend and manage hardware microservices efficiently, especially in multi-tenant environments with separate code bases.
Innovation Solution
Implement remote storage solutions using NVMe-oF to increase storage capacity while reducing the cost of SoC/FPGA resources by limiting local storage, and utilize FPGA/XPU proxies for smart multiplexing and management of XPU resources based on real-time microservice needs.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If local storage is used on SoC-XPU platforms, then data access speed is improved, but storage capacity is constrained and cost increases
Solution Approach 1:
The storage system is segmented into two distinct parts: local storage on the SoC-XPU platform for fast access to frequently used data, and remote storage accessed via NVMe-oF for capacity requirements. This segmentation allows the system to leverage the speed advantage of local storage for hot data while using remote storage for cold data, effectively resolving the contradiction between speed and capacity.
Solution Approach 2:
NVMe-oF acts as an intermediary protocol that enables the SoC-XPU platform to access remote storage with performance characteristics similar to local storage. The intermediary abstraction layer provides block-level access to remote storage devices over the network, making the storage system appear as a unified resource while physically separating capacity-constrained local storage from capacity-abundant remote storage.
2Speed
If local storage is used on SoC-XPU platforms, then data access speed is improved, but cost increases
Solution Approach 1:
Different storage quality characteristics are applied to different data types: local storage is used for data requiring high access speed (hot data), while remote storage is used for data where capacity and cost-effectiveness are priorities (cold data). This local quality differentiation allows the system to optimize the expensive local storage resource only where its speed advantage is needed, reducing overall system cost.
3Productivity
If XPU resources are allocated to multiple microservices, then resource utilization is improved, but security and isolation become more difficult to maintain
Solution Approach 1:
The system segments the management of XPU resources by introducing dedicated proxy processes for each microservice. Each proxy independently manages its allocation requests and isolation requirements, allowing the orchestrator to allocate XPU resources across multiple microservices while maintaining security boundaries through the proxy layer. This segmentation enables multi-tenancy without compromising isolation.
Solution Approach 2:
The proxy process acts as an intermediary between the microservice and the XPU resource allocator. It mediates resource allocation requests, enforces security policies, and maintains isolation guarantees. By placing the proxy in between the microservice and the hardware resource, the system can safely multiplex XPU resources across multiple services while preserving security and isolation through the proxy's control layer.
Data Source
Figure 1~2b
Figure 3
Figure 4
AI summary
Methods, apparatus, and software for remote storage of hardware microservices hosted on other processing units (XPUs) and SOC-XPU Platforms. The apparatus may be a platform including a System on Chip (SOC) and an XPU, such as a Field Programmable Gate Array (FPGA). Software, via execution on the SOC, enables the platform to pre-provision storage space on a remote storage node and assign the storage space to the platform, wherein the pre-provisioned storage space includes one or more container images to be implemented as one or more hardware (HW) micro service front-ends. The XPU/FPGA is configured to implement one or more accelerator functions used to accelerate HW microservice backend operations that are offloaded from the one or more HW microservice front-ends. The platform is also configured to pre-provision a remote storage volume containing worker node components and access and persistently store worker node components.