NUMA Index Manager Affinitization for Low-Latency Search
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Search applications on Non-Uniform Memory Access (NUMA) machines often experience high latency due to random distribution of data across memories, which can be inefficient as they do not leverage the lower latency of local memory access.
Innovation Solution
Implementing a design where different portions of an index are stored in the locally-attached memories of multiple processors, with each processor instance of the index manager comparing search queries to its locally-stored index chunk, thereby reducing high-latency memory access requests and utilizing the low-latency local memory access times.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If data is distributed evenly across different memories using interleaving policy, then data distribution balance is improved, but memory access latency increases due to random access patterns
Solution Approach 1:
The patent applies local quality by making different parts of the system have different data distribution characteristics. Specifically, each processor's local memory is dedicated to storing specific index chunks that are frequently accessed by that processor, while remote memories store other index chunks. This creates a quality difference where local memory accesses are optimized for speed and reliability, while remote accesses are minimized. The index manager instances are affinitized to specific processors, ensuring that each processor primarily accesses its locally-attached memory, thereby reducing average memory access latency while maintaining balanced data distribution across the NUMA system.
2Productivity
If search applications access any memory on NUMA machine, then memory utilization is improved, but average access time increases due to remote memory accesses
Solution Approach 1:
The patent applies segmentation by dividing the index into multiple chunks and distributing them across different processors' local memories. Each index manager instance is responsible for a specific index chunk and is affinitized to a particular processor. This segmentation ensures that memory utilization is improved as all local memories are actively used, while average access time is reduced because each processor primarily accesses its own local memory rather than remote memories. The query dispatch mechanism sends copies of queries to multiple index manager instances, ensuring efficient utilization of all available memory resources.
3Speed
If index chunks are stored in locally-attached memories, then memory access speed is improved, but system complexity increases due to affinitization requirements
Solution Approach 1:
The patent applies preliminary action by pre-establishing affinitization relationships between index manager instances and processors, and between index chunks and local memories, before the search operation begins. The system configuration phase sets up the mapping between processors, index manager instances, and index chunks stored in local memories. This preliminary organization eliminates the need for complex runtime decisions about memory access, as each index manager instance knows in advance which local memory to access. The query dispatch mechanism simply sends queries to the appropriate affinitized instances, reducing system complexity during operation while maintaining high memory access speed.
Data Source
AI summary
Computational units of any task may run in different silos. In an embodiment, a search query may be evaluated efficiently on a non-uniform memory architecture (NUMA) machine, by assigning separate chunks of the index to separate memories. In a NUMA machine, each socket has an attached memory. The latency time is low or high, depending on whether a processor accesses data in its attached memory or a different memory. Copies of an index manager program, which compares a query to an index, run separately on different processors in a NUMA machine. Each instance of the index manager compares the query to the index chunk in the memory attached to the processor on which that instance is running. Thus, each instance of the index manager may compare a query to a particular portion of the index using low-latency accesses, thereby increasing the efficiency of the search.