Optical Shared Memory Interconnect for Scalable Cache Coherency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Shared memory architectures in computer systems face challenges with scaling due to increased network bandwidth consumption and delays associated with data coherency, particularly in NUMA designs that require custom processors and complex memory coherency algorithms, leading to higher costs and performance issues.
Innovation Solution
Implementing an optical communication link to replace large electrical data buses, allowing multiple devices to act as both producers and consumers, with a broadcast mechanism for memory updates and acknowledgments (ACK/NACK) to maintain cache coherency without a central coordinating node, enabling high-speed and low-latency data transfers across multiple nodes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a central node is used to manage coherency and pass data updates, then data coherency is maintained, but significant delays occur as the number of nodes increases
Solution Approach 1:
The patent removes the central node from the coherency management system. Instead of a centralized controller, each node independently manages its own coherency state through distributed protocols, eliminating the bottleneck and delays associated with central node coordination.
Solution Approach 2:
The coherency management function is segmented and distributed across all nodes rather than centralized. Each node maintains local coherency information and communicates only when necessary, dividing the monolithic central control into independent distributed units.
2Loss of time
If NUMA design with large shared memory bus is used, then data sharing delays are reduced, but system cost significantly increases
Solution Approach 1:
The patent replaces the physical large shared memory bus infrastructure with software-based distributed coherency protocols running over standard networks. This substitutes expensive hardware mechanics with more economical software solutions that achieve similar performance.
Solution Approach 2:
The system uses standard, commodity CPU cards and network interfaces that can serve multiple functions including general-purpose computing and coherency management, rather than requiring specialized custom processor cards with dedicated hardware for each function.
3Speed
If custom processor cards are used to accommodate custom memory buses, then memory bus performance is improved, but hardware compatibility and ease of manufacture decrease
Solution Approach 1:
The patent employs universal, standard CPU cards that can be manufactured using commodity processes and are compatible with standard hardware platforms. These standard cards perform both general computing and support the distributed coherency protocol, eliminating the need for custom processor cards.
4Productivity
If large expensive caches are employed, then performance impact of memory coherency issues is reduced, but system cost increases
Solution Approach 1:
The system achieves coherency through self-service mechanisms where each node independently tracks and manages its own cache state and communicates changes to others through distributed protocols, rather than relying on a centralized controller or large expensive caches to mask coherency delays.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
This solution enables scalable shared memory systems with low latency and high bandwidth, reducing costs and complexity, allowing for efficient data synchronization across a large number of nodes with minimal delay, comparable to internal memory transactions, and supports up to 4096 nodes with near simultaneous data synchronization.
Implementation Method 1
a system having multiple devices with a shared memory architecture. The shared memory architecture includes an optical communication link to interconnect the multiple devices
Data Source
AI summary
A system includes multiple devices with a shared memory. The devices interconnect to each other via an optical communication link, with broadcast sends as producers, and receiving messages from others as consumers of the other devices. The devices receive a packet from a producer that has a lock on a cache line of the shared memory. In response to the packet, the devices send an acknowledgement or negative acknowledgement, and invalidate the cache line that is the subject of the message in a local copy of the shared memory. The devices can update the cache line in the local copy of the shared memory as data is processed as received over the optical communication link.