Hardware Interconnect with Non-Coherent Shared Memory Access
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing hardware interconnects face inefficiencies in data transmission due to the need for coherent memory transactions, which increase computational overhead and latency, especially when multiple devices access shared memory devices.
Innovation Solution
The implementation of a hardware interconnect system that allows non-coherent access to shared memory devices, reducing the number of coherent transactions by using flags and descriptors for data transmission, and optimizing data transfer over dedicated channels, enabling devices like hardware accelerators to read and write directly to shared memory without caching contents, thus minimizing cache discrepancies and latency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If memory coherence protocols are implemented to ensure data consistency across multiple devices, then data reliability is improved, but computational overhead and latency increase
Solution Approach 1:
The patent segments memory access operations into two distinct types: coherent transactions for control information (flags, descriptors) and non-coherent transactions for bulk data transfer. This segmentation allows each type to use the appropriate protocol, minimizing the overhead of coherence protocols while maintaining reliability where needed.
Solution Approach 2:
The patent applies different coherence requirements to different data types: control information requires coherence guarantees while bulk data transfer operates in non-coherent mode. This local quality approach ensures reliability is maintained for critical control operations while allowing high-speed non-coherent transfers for data payloads.
2Reliability
If coherent transactions are used for all data transmission, then data consistency is maintained, but power consumption increases
Solution Approach 1:
The patent segments memory transactions into coherent and non-coherent categories based on data type and access pattern. By segmenting traffic this way, the system pays the power cost of coherence protocols only when necessary for control information, while bulk data transfers use lower-power non-coherent paths.
Solution Approach 2:
The patent changes the coherence parameter dynamically based on the type of transaction being performed. Control information transactions use coherent parameters while bulk data transfers use non-coherent parameters, optimizing power consumption by avoiding unnecessary coherence protocol execution.
3Speed
If devices cache data from shared memory, then access speed is improved, but cache coherence overhead increases
Solution Approach 1:
The patent segments cached data into two categories: control information that requires coherence tracking and bulk data that operates in non-coherent mode. This segmentation reduces cache coherence overhead by limiting the scope of coherence protocols to only those cache lines containing control information.
Solution Approach 2:
The patent extracts bulk data transfers from the cache coherence protocol mechanism entirely, allowing them to proceed via non-coherent paths. Only control information remains in the coherence protocol domain, significantly reducing the overall overhead.
4Productivity
If non-coherent access is used for data transmission, then computational overhead is reduced, but data consistency between devices decreases
Solution Approach 1:
The patent segments transactions into coherent control operations and non-coherent data operations. This segmentation allows non-coherent access to be used for bulk data transmission (improving efficiency) while maintaining coherent access for control information (preserving consistency).
Solution Approach 2:
The patent introduces control information (flags, descriptors) as intermediaries that mediate between devices performing non-coherent data transfers. These intermediaries provide the necessary coordination and consistency guarantees without requiring full coherence protocols for the actual data transfer.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Aspects of the disclosure are directed to hardware interconnects and corresponding devices and systems for non-coherently accessing data in shared memory devices. Data produced and consumed by devices implementing the hardware interconnect can read and write directly to a memory device shared by multiple devices, and limit coherent memory transactions to relatively smaller flags and descriptors used to facilitate data transmission as described herein. Devices can communicate less data on input/output channels, and more data on memory and cache channels that are more efficient for data transmission. Aspects of the disclosure are directed to devices configured to process data that is read from the shared memory device. Devices, such as hardware accelerators, can receive data indicating addresses for different data buffers with data for processing, and non-coherently read or write the contents of the data buffers on a memory device shared between the accelerators and a host device.