Storage System Memory as Host Memory Extension
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current data storage systems face limitations in expanding memory capacity without introducing drawbacks such as static memory size, lack of write protection, and inefficiencies in data coherence and sharing across multiple hosts, particularly in distributed applications like in-memory databases.
Innovation Solution
The system allocates portions of storage system memory as host memory, allowing dynamic expansion and contraction, enabling direct access and mirroring for coherence, and providing write protection through techniques like mirroring and vaulting, while allowing multiple hosts to share memory resources.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If storage system memory is allocated as host memory, then memory capacity is expanded dynamically, but memory management complexity increases
Solution Approach 1:
A memory manager component is introduced as an intermediary between the storage system memory and host systems. This manager handles allocation, de-allocation, and access control of memory portions, abstracting the complexity from hosts while enabling dynamic capacity expansion. The manager receives requests from hosts, manages memory portions accordingly, and ensures proper access protocols are followed.
2Productivity
If multiple hosts access shared memory, then memory sharing efficiency improves, but data coherence maintenance becomes more difficult
Solution Approach 1:
The memory manager implements feedback mechanisms to maintain data coherence across multiple hosts. When one host modifies data in a memory portion, the manager notifies other hosts or updates their cached copies, ensuring all hosts have access to the most current data. This feedback loop enables efficient sharing while maintaining coherence through continuous synchronization.
3Adaptability or versatility
If memory is dynamically allocated to hosts, then adaptability increases, but write protection becomes more challenging
Solution Approach 1:
The system dynamically adjusts memory allocation and protection levels based on host requirements and trust levels. The memory manager can modify allocation status, access permissions, and protection levels in real-time as hosts join or leave the system. This dynamic approach maintains flexibility while implementing appropriate write protection measures through configurable access control policies.
4Speed
If hosts connect directly to storage system fabric, then access speed improves, but system security risks increase
Solution Approach 1:
The memory manager serves as a security intermediary between hosts and the storage system fabric. Even though hosts connect directly to the fabric for high-speed access, the manager intercepts and validates all memory access requests, enforcing authentication and authorization policies. This intermediary role enables fast direct fabric access while maintaining security through centralized access control and monitoring.
Data Source
AI summary
Memory of a storage system is made available (i.e., exposed) for use as host memory of a host, for example, as an extension of the main memory of the host. The host may be directly connected to an internal fabric of the data storage system. Portions of the storage system memory (SSM) may be allocated for use as host memory, and this may be communicated to the host system. The host OS and applications executing thereon then may make use of the SSM as if it were memory of the host system, for example, as second-tier persistent memory. The amount of SSM made available may be dynamically increased and decreased. The SSM may be accessed by the host system as memory; i.e., in accordance with memory-based instructions, for example, using remote direct memory access instructions. The SSM may be write protected using mirroring, vaulting and other techniques.