Selective High-Performance Memory Use in Software Defined Storage
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
High-performance computing workloads, especially those utilizing hardware accelerators like GPUs, face bottlenecks due to storage limitations, where fast local storage acts as an isolated 'island' requiring manual data transfer and replication, hindering continuous utilization and data integrity.
Innovation Solution
A memory management application (MMA) determines the selective use of high-performance memory within a software-defined storage system (SDS), allowing hardware accelerators to read and write directly to high-performance local storage, while temporarily suspending replication and controlling data distribution to optimize performance and prevent data loss.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If fast local storage (NVMe) is used for hardware accelerator workloads, then storage speed and performance are improved, but storage capacity and data integrity are worsened due to isolation from the SDS system
Solution Approach 1:
The patent merges the isolated local storage system with the distributed SDS system by allowing hardware accelerators to access both local NVMe storage and remote SDS storage through a unified interface. This integration enables data to be stored locally for high-speed access while simultaneously maintaining replicated copies in the SDS system for data integrity and capacity.
Solution Approach 2:
The patent introduces an intermediary storage management layer that coordinates between the local storage system and the SDS system. This intermediary manages data placement, replication, and access policies, allowing fast local storage to be used without compromising data integrity by ensuring proper synchronization with the SDS system.
2Productivity
If local high-speed storage is used, then storage performance is improved, but device complexity increases due to manual seed and copy processes
Solution Approach 1:
The patent implements self-service automation where the storage management system automatically performs data seeding, replication, and distribution between local and remote storage systems. This eliminates manual intervention requirements and reduces operational complexity while maintaining high storage performance.
Solution Approach 2:
The patent performs preliminary automated setup of data replication and synchronization policies before workloads begin. This preliminary configuration establishes the storage architecture and data flow paths in advance, eliminating the need for manual seed and copy processes during operation.
3Speed
If local storage acts as an isolated island, then storage speed is improved, but adaptability is worsened due to segregation from the SDS system
Solution Approach 1:
The patent creates a universal storage interface that allows the storage system to function in multiple modes: local high-speed access mode, remote SDS access mode, and hybrid mode. This multi-functionality enables the system to adapt to different workload requirements while maintaining high performance through intelligent data placement and access routing.
Data Source
AI summary
A system includes control logic to boot to a waking state, configure the system, and check for the presence of non-volatile DIMMs. Based on a determination that non-volatile DIMMs are not present, the control logic is to create one or more block devices to overcome CPU utilization limitations. Based on a determination that non-volatile DIMMs are present, the control logic is to use a non-volatile DIMM for storage.