Virtual Link Buffer for Low Latency Thread Communication
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current data processing systems face inefficiencies in data transfer between execution threads due to high latency and energy consumption, particularly when using software-based directed communication through shared memory, which requires valuable processor resources and involves multiple cache hierarchy layers.
Innovation Solution
A hardware-accelerated directed communication channel is implemented using caches and a link controller to provide a virtual link buffer, enabling ordered or unordered data transfer between execution threads by utilizing cache lines and maintaining control through a link controller, thereby reducing the need for software-based implementations and minimizing resource consumption.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If software-based directed communication through shared memory is used, then data transfer between threads can be achieved, but latency increases and processor resources are consumed
Solution Approach 1:
The patent introduces a dedicated communication channel as an intermediary between producer and consumer threads, using a hardware-managed buffer (implemented via cache lines) to transfer data without requiring processor intervention. This mediator handles the data transfer autonomously, eliminating the latency associated with software-based communication while maintaining reliable data transfer between threads.
Solution Approach 2:
The patent replaces the software-based communication mechanism (which requires processor execution of instructions, locks, and memory management) with a hardware-based communication channel. This substitution eliminates the need for processor resources to be consumed during data transfer, reducing latency while maintaining data transfer reliability between threads.
2Reliability
If software-based directed communication through shared memory is used, then data transfer between threads can be achieved, but processor resources are consumed
Solution Approach 1:
The dedicated communication channel acts as an intermediary that handles data transfer independently of the processor, using hardware-managed buffers and direct cache line transfers. This eliminates the need for processor resources to be consumed during communication operations, allowing the processor to focus on computation while the communication channel handles data movement autonomously.
Solution Approach 2:
The communication channel is designed to be self-service, with the buffer management and data transfer handled automatically by the hardware without requiring processor intervention. The system uses existing cache coherence protocols and memory structures to manage the buffer, allowing the communication infrastructure to service itself without consuming valuable processor resources.
3Reliability
If transmission of data between processing cores occurs through multiple layers of cache hierarchy, then data can be transferred between cores, but latency and energy usage increase
Solution Approach 1:
The patent extracts the data transfer path from the traditional multi-layer cache hierarchy and creates a dedicated communication channel that bypasses these intermediate layers. By taking out the essential function of data transfer and implementing it through a direct hardware channel using cache lines, the system eliminates the energy-consuming intermediate cache layers while maintaining the ability to transfer data between cores.
4Productivity
If a full hardware solution is implemented for communication, then data transfer efficiency is improved, but device complexity increases
Solution Approach 1:
The patent implements a communication channel that uses existing, multi-functional hardware structures - specifically cache lines and cache coherence protocols - that already serve multiple purposes in the processor. By making the communication channel universal and based on existing infrastructure rather than dedicated communication hardware, the system achieves high data transfer efficiency without significantly increasing device complexity.
Data Source
AI summary
A virtual link buffer provides communication between processing threads or cores. A first cache is accessible by a first processing device and a second cache accessible by a second processing device. An interconnect structure couples between the first and second caches and includes a link controller. A producer cache line in the first cache stores data produced by the first processing device and the link controller transfers data in the producer cache line to a consumer cache line in the second cache. Each new data element is stored at a location in the producer cache line indicated by a store position or tail indicator that is stored at a predetermined location in the same cache line. Transferred data are loaded from a location in the consumer cache line indicated by a load position or head indicator that is stored at a predetermined location in the same consumer cache line.