Shared Memory Read Synchronization to Reduce Cache Thrashing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Inefficient memory access and increased traffic occur when host systems in computing systems, such as those using the CXL protocol, do not coordinate requests to a shared memory system, leading to cache thrashing and increased power consumption and latency.
Innovation Solution
Implement data group synchronization by assigning groups to machine learning model parameters, providing group identifiers and address ranges, and using a controller to manage synchronized read requests and acknowledgments, reducing unnecessary communication and cache thrashing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If host systems independently issue read requests to shared memory without coordination, then each host can access data independently, but cache thrashing occurs and memory traffic increases
Solution Approach 1:
The patent merges multiple independent read requests into a single coordinated read operation. The controller collects read requests from multiple host systems and consolidates them into one memory read operation, reducing the total number of memory transactions while maintaining the ability to serve multiple hosts.
Solution Approach 2:
The patent implements a feedback mechanism where host systems send acknowledgment signals to the controller after receiving data. The controller uses this feedback information to track which hosts have received their data and to coordinate subsequent read requests, enabling efficient data distribution without redundant memory accesses.
2Adaptability or versatility
If host systems issue multiple read requests for the same data, then data can be distributed to multiple hosts, but redundant data transmission occurs and power consumption increases
Solution Approach 1:
The patent combines multiple read requests for the same data into a single memory read operation. The controller aggregates requests from multiple host systems and issues one coordinated read to the memory device, then distributes the data to all requesting hosts, thereby eliminating redundant memory transactions and reducing power consumption.
Solution Approach 2:
The patent uses acknowledgment feedback from host systems to track data distribution status. Hosts send acknowledgments when they receive data, allowing the controller to identify which hosts still need data and issue targeted read requests only to those hosts, avoiding redundant reads and reducing overall power consumption.
3Speed
If host systems access shared memory without coordination, then access speed is maintained, but latency increases due to cache thrashing
Solution Approach 1:
The patent implements preliminary coordination where the controller collects and processes read requests from multiple host systems before issuing memory read operations. This preliminary aggregation of requests allows the system to optimize the timing and coordination of data retrieval, reducing cache thrashing and overall latency while maintaining fast data access speeds.
Solution Approach 2:
The patent uses acknowledgment feedback to coordinate data distribution timing. Host systems send acknowledgments when data is received, allowing the controller to optimize the sequencing of subsequent memory operations and reduce idle waiting time, thereby reducing latency without compromising access speed.
4Productivity
If a controller coordinates read requests from multiple host systems, then memory access efficiency improves, but device complexity increases
Solution Approach 1:
The patent designs the controller with multi-functional capabilities that handle request collection, data aggregation, coordination, and distribution to multiple host systems. By embedding these diverse functions within a single controller unit, the system achieves improved memory access efficiency while containing complexity within one integrated component rather than requiring multiple coordinating devices.
Data Source
AI summary
In some implementations, a controller may obtain, from respective host systems of a plurality of host systems, a plurality of request messages for data associated with a first group identifier of one or more group identifiers. The controller may store the data to a cache of the controller based on obtaining the data from the one or more memory devices. The controller may provide the data to the plurality of host systems based on storing the data to the cache. The controller may identify, based on providing the data, whether each host system of the plurality of host systems has received the data. The controller may provide a plurality of acknowledgment messages to the plurality of host systems based on identifying that each host system of the plurality of host systems has received the data.