Inter-node messaging controller for low-latency core communication
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional data processing systems face inefficiencies in communicating data between cores due to cache coherency requirements, leading to high core-to-core latency and low communication bandwidth, which hinders performance in multi-threaded environments.
Innovation Solution
A data processing system with a native messaging model using an inter-node messaging controller (INMC) that enables efficient communication between nodes through inter-node-get and inter-node-put operations, along with remote atomic operations, allowing for high bandwidth and low latency messaging by offloading queue transmission and utilizing read snapshot and in-place write update operations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If conventional load and store instructions are used for core-to-core communication, then data can be transferred between cores, but cache coherency requirements cause high latency and low bandwidth
Solution Approach 1:
The patent introduces message queues as intermediary data structures that buffer messages between sending and receiving cores. The queue tail pointer (written by sender) and head pointer (read by receiver) act as mediators that decouple the timing between message production and consumption, eliminating the need for synchronous cache coherency operations on every data transfer.
Solution Approach 2:
The system performs preliminary actions by pre-allocating message queue structures and establishing communication channels between cores before actual message transfer begins. The sender core can immediately write to pre-reserved queue locations without waiting for receiver readiness, and the receiver can pre-fetch messages from the queue, reducing latency through advance preparation.
2Quantity of substance
If conventional load and store instructions are used for core-to-core communication, then data transfer is possible, but communication bandwidth is limited
Solution Approach 1:
The message queue is segmented into multiple fixed-size message slots arranged in a circular buffer structure. This segmentation allows multiple messages to be queued simultaneously in different slots, enabling parallel data transfer operations. The sender can write to any available slot independently, and the receiver can read from multiple slots concurrently, dramatically increasing communication bandwidth compared to sequential load-store operations.
3Reliability
If cache coherency operations are performed for every data transfer, then data consistency is maintained, but pipeline bubbles reduce processor throughput
Solution Approach 1:
The message queue implementation enables continuous useful action by allowing the sender core to write messages to the queue without stalling for cache coherency operations. The queue structure maintains data consistency through ordered write operations to the tail pointer and sequential read operations from the head pointer, while keeping processor pipelines full and productive throughout the communication process.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A processor package comprises a first core, a local cache in the first core, and an inter-node messaging controller (INMC) in the first core. The INMC is to receive an inter-node message from a sender thread executing on the first core, wherein the message is directed to a receiver thread executing on a second core. In response, the INMC is to store a payload from the inter-node message in a local message queue in the local cache of the first core. After storing the payload, the INMC is to use a remote atomic operation to reserve a location at a tail of a shared message queue in a local cache of the second core. After reserving the location, the INMC is to use an inter-node-put operation to write the payload directly to the local cache of the second core. Other embodiments are described and claimed.