AI Interconnect Architecture for In-Memory Accumulation and Aggregation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing AI hardware architectures face inefficiencies in performing accumulation and aggregation operations, particularly in memory access and gradient collection during learning processes, leading to performance bottlenecks.
Innovation Solution
An interconnect device with hardware-implemented modules that perform accumulation and aggregation operations on data stored in memory and processed results, optimizing memory access through direct memory access (DMA) and reducing the need for redundant data transfers.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If data is transferred between memory and processing cores through traditional memory access methods, then data can be retrieved, but the number of memory access instances increases and system efficiency decreases
Solution Approach 1:
The patent combines accumulation and aggregation operations directly into the interconnect device, merging data transfer and computation functions. This allows multiple data access operations to be consolidated into single memory access instances, reducing the total number of memory accesses and improving system efficiency.
Solution Approach 2:
The interconnect device performs accumulation and aggregation operations in advance during data transfer, before the data reaches the processing core. This preliminary computation reduces the need for subsequent memory access instances and improves overall productivity.
2Ease of operation
If accumulation and aggregation operations are performed separately through traditional processing methods, then operations can be completed, but the complexity of data transfer and processing increases
Solution Approach 1:
The patent merges accumulation and aggregation operations into a single integrated process within the interconnect device. This consolidation simplifies the overall operation by eliminating the need for separate data transfer and processing steps, making the system easier to operate.
3Productivity
If redundant data transfers are performed between memory and processing cores, then data can be accessed multiple times, but the number of memory access instances increases
Solution Approach 1:
The interconnect device performs accumulation and aggregation operations in advance during the initial data transfer from memory. This preliminary computation eliminates the need for redundant data transfers and multiple memory access instances, improving memory access efficiency and reducing time loss.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
An interconnect device may include one or more hardware-implemented modules configured to: receive a command from a processing core; perform, based on the received command, an operation including either one or both of an accumulation operation on sets of data stored in a memory and an aggregation operation on results processed by the processing core; and provide a result of the performing of the operation.