NUMA Storage Data Source Selection Heuristics
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In storage systems with a non-uniform memory access (NUMA) configuration, existing technologies face challenges in optimizing read and write operations across multiple storage processors and memory domains, leading to inefficiencies in data access and communication, particularly in determining the best data source and disk adapter for processing requests.
Innovation Solution
A method is introduced that involves receiving write and read requests in a data storage system with multiple storage processors and domains, where data is copied between memory domains and the best data source is selected based on heuristics and criteria to minimize the use of interdomain communication connections, thereby optimizing data access and reducing reliance on QPI connections.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If data is accessed across multiple storage processors and memory domains in a NUMA configuration, then data storage capacity and system resources are increased, but data access efficiency and communication performance deteriorate due to interdomain communication overhead
Solution Approach 1:
The patent applies preliminary action by pre-copying data from remote memory domains to local memory domains before actual data access is needed. This proactive data placement ensures that when read or write requests occur, the data is already positioned in local memory, eliminating the need for interdomain communication during operation and thus resolving the contradiction between distributed storage capacity and data access efficiency
Solution Approach 2:
The patent introduces an intermediary mechanism in the form of a data copying and placement system that acts between remote memory domains and local memory domains. This intermediary pre-positions data in local memory before access requests, mediating the contradiction by reducing direct interdomain communication overhead while maintaining the benefits of distributed storage architecture
2Adaptability or versatility
If interdomain communication connections are used to access data across memory domains, then system versatility and resource sharing are improved, but communication overhead and performance latency increase
Solution Approach 1:
The system performs preliminary data copying to local memory domains before actual access operations, pre-positioning data to avoid future interdomain communication latency. This resolves the contradiction by maintaining resource sharing versatility while eliminating time loss during actual data access operations
Solution Approach 2:
The patent employs copying by creating local replicas of data from remote memory domains. These copies are placed in local memory to be accessed during read/write operations, thereby maintaining the versatility of distributed resource sharing while eliminating the time penalty of repeated interdomain communications
3Power
If QPI connections are used for interdomain communication, then data transfer capability is enabled, but performance degradation and bottleneck effects occur
Solution Approach 1:
The patent extracts the data transfer function from the QPI interdomain communication path and relocates it to local memory domain operations. By pre-copying data to local domains and performing access operations locally, the system removes QPI connections from the critical data access path, thereby maintaining data transfer capability while eliminating performance degradation and bottleneck effects
4Reliability
If multiple disk adapters are used to access storage devices, then system reliability and fault tolerance are improved, but determining the optimal data source and adapter increases complexity
Solution Approach 1:
The patent applies local quality by associating specific data with specific local memory domains based on their original storage location. Each local memory domain maintains data with local affinity, and the system selects data sources based on local vs. remote distinctions rather than evaluating multiple complex paths. This resolves the contradiction by maintaining fault tolerance through multiple adapters while simplifying data source selection through local quality awareness
Data Source
AI summary
Described are techniques for processing read and write requests in a system having a NUMA (non-uniform memory access) configuration. Such techniques may include receiving, at a front end adapter of the system, a write request, to write first data to a first storage device, storing a first copy of the first data in first memory local to a first domain, copying, using a first inter-storage processor communication connection, the first data from the first memory to a third memory of a third domain thereby creating a second copy of the first data in the third memory; and determining, in accordance with a first heuristic and first criteria, whether to use the first copy of the first data stored in the first memory or the second copy of the first data stored in the third memory as a source when writing the first data to the first storage device.