Compute-Near-Memory Communication Paths for Low-Latency Data Access
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing computing architectures face challenges in managing high latency and energy consumption due to data movement between processors and memory, constraining performance and capacity, particularly in systems with large numbers of compute elements.
Innovation Solution
Implementing a compute-near-memory (CNM) system with a network structure that selectively provides data paths based on communication requests, using a network structure to manage state data for open or in-flight requests, and employing a balancer element to direct processing tasks across the system, reducing the need for long-term state data storage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of time
If data is moved between processors and memory in conventional architectures, then data access is enabled, but latency and energy consumption increase significantly
Solution Approach 1:
The system is divided into multiple compute elements, each with its own local memory, rather than using a single shared memory system. This segmentation allows data to be processed closer to where it is stored, reducing the distance and time for data movement between memory and processors.
Solution Approach 2:
The patent transitions from a centralized memory architecture to a distributed memory architecture across multiple compute elements. This dimensional change in system organization enables parallel data access and reduces the bottleneck of data movement by distributing both computation and storage across multiple nodes.
2Productivity
If the number of compute elements is increased to enhance processing capacity, then system throughput improves, but network deadlocks and backpressure increase
Solution Approach 1:
The network structure pre-allocates data paths for communication requests before data movement occurs. By establishing dedicated paths in advance and managing state data for open requests, the system prevents deadlocks from occurring rather than reacting to them after they occur.
Solution Approach 2:
The network structure acts as an intermediary that manages communication between compute elements. It selectively provides data paths and maintains state data for requests, mediating the interactions between multiple compute elements to prevent conflicts and deadlocks while enabling high throughput.
3Ease of operation
If shared memory is used for data access, then data sharing is simplified, but access latency and energy consumption increase
Solution Approach 1:
Instead of a single shared memory, the system segments memory into local memory associated with each compute element. This segmentation enables each compute element to access its local memory quickly without contending for shared resources, while still allowing data sharing through the network structure that connects the segmented memory systems.
Data Source
AI summary
Various examples are directed to systems and methods for executing a transaction between hardware compute elements of a computing system. A first hardware compute element may send a first write request to a second hardware compute element via a network structure. The first write request may comprise first source identifier data describing the first hardware compute element and first payload data describing a processing task requested by the first hardware compute element. The network structure may store first write request state data describing the first write request. Before the processing task is completed, the second hardware compute element may send a first write confirm message.