General-Purpose Processing Domain Segmentation for Memory Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In resource-intensive computations, general-purpose processors like CPUs often compete with hardware accelerators for shared resources such as system memory, leading to performance inefficiencies, especially in large-scale distributed machine learning systems where CPUs perform supporting roles that degrade system performance.
Innovation Solution
The system splits general-purpose processing units into high-priority and low-priority domains, with dedicated memory and memory controllers, and uses an optimization runtime system to adjust configurations based on memory usage measurements to optimize resource utilization and reduce contention between domains.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If general-purpose processors are used to perform supporting tasks in resource-intensive computations, then system versatility is improved, but memory latency increases due to competition with hardware accelerators for shared system memory
Solution Approach 1:
The system is divided into two distinct domains: a high-priority domain for time-critical tasks supporting hardware accelerators, and a low-priority domain for general-purpose supporting tasks. This segmentation allows high-priority tasks to access memory through dedicated controllers with guaranteed latency bounds, while low-priority tasks use shared memory resources, thus resolving the contradiction between system versatility and memory latency.
2Adaptability or versatility
If CPUs perform supporting roles in distributed machine learning systems, then system functionality is improved, but system performance degrades due to resource contention
Solution Approach 1:
The processing system is segmented into high-priority and low-priority domains with separate memory controllers. High-priority CPU cores handle time-critical supporting tasks with guaranteed memory access performance, while low-priority CPU cores handle general supporting functions. This segmentation maintains full system functionality while eliminating performance degradation from resource contention.
Solution Approach 2:
Different quality levels of memory service are provided to different domains: high-priority domain receives premium memory service with dedicated controllers and guaranteed latency, while low-priority domain uses standard shared memory service. This local quality differentiation resolves the contradiction by providing appropriate service levels to different functional requirements.
3Loss of time
If processing units are divided into high-priority and low-priority domains with dedicated memory controllers, then memory latency is reduced, but device complexity increases
Solution Approach 1:
The memory controller subsystem is segmented into dedicated high-priority memory controllers and shared low-priority memory controllers. This segmentation provides guaranteed memory access latency for high-priority tasks while maintaining manageable complexity through clear separation of concerns and standardized interfaces between domains.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Methods, systems, and computer storage medium storing instructions for obtaining data splitting a plurality of general-purpose processing units in a processing system into a high-priority domain and a low-priority domain, wherein the general-purpose processing units in the high-priority domain are assigned to perform one or more tasks comprising one or more high-priority tasks, and the general-purpose processing units in the low-priority domain are assigned to perform one or more low-priority tasks; and during runtime of the processing system, obtaining memory usage measurements that characterize usage of system memory by the high-priority domain and the low-priority domain; and adjusting, based on the memory usage measurements, a configuration of (i) the high-priority domain, (ii) the low-priority domain, or (iii) both to adjust utilization of the system memory by the general-purpose processing units.