Integrated Cache Accelerator for Reducing Data Transfer Overhead
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current configurations of computer systems require data transfer between accelerators and cache memories, leading to significant overhead due to cache flush operations, which limits the reduction of overhead in data processing tasks such as graph processing and stencil computations.
Innovation Solution
A data processing device with a two-dimensional structure that integrates cache memory inside the accelerator, allowing simultaneous reading and writing of data within the accelerator, eliminating the need for external data transfer and reducing overhead by sharing cache memory between the general-purpose CPU and the accelerator.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of time
If cache memory is provided outside each accelerator, then data transfer between accelerator and cache memory is required, but overhead reduction is limited
Solution Approach 1:
The patent merges the cache memory with the accelerator by integrating memory blocks directly into the accelerator structure. This combination eliminates the need for separate data transfer between accelerator and cache memory, as the memory blocks are now part of the accelerator itself, thereby reducing overhead while maintaining functional separation.
Solution Approach 2:
The patent implements nesting by placing memory blocks inside the accelerator structure. The memory blocks are nested within the accelerator's internal architecture, allowing the accelerator to access its own embedded memory without external data transfer, thus reducing overhead while keeping the cache functionality integrated.
2Loss of time
If shared buffer memory is used for both cache and local memory, then overhead is reduced, but data transfer between accelerator and cache is still required
Solution Approach 1:
The patent merges the local memory and cache memory functions into a single integrated structure where memory blocks serve both purposes simultaneously. This eliminates the need for separate data transfer operations between accelerator and cache, as the memory blocks are directly accessible by the accelerator's calculation units, thereby improving productivity while maintaining overhead reduction.
3Reliability
If data transfer between accelerator and external cache is required, then cache flush operations are needed, but this increases overhead significantly
Solution Approach 1:
The patent extracts the cache memory from the external system and embeds it directly within the accelerator structure. By taking out the cache memory from the external cache system and integrating it into the accelerator, the need for cache flush operations is eliminated, as the memory blocks are now part of the accelerator's internal memory hierarchy and do not require synchronization with external cache.
4Ease of operation
If external cache memory is used with width close to SIMD width, then data access is simplified, but large amounts of data exceeding SIMD width cannot be referenced at one time
Solution Approach 1:
The patent segments the memory system into multiple memory blocks, each with its own ports and address generators. This segmentation allows the accelerator to access multiple memory blocks simultaneously through parallel ports, effectively increasing the data capacity that can be referenced at one time while maintaining simple access patterns through the use of multiple independent memory units.
Solution Approach 2:
The patent adds a dimensional aspect to memory access by organizing memory blocks in a two-dimensional structure with multiple stages and ways. This dimensional organization allows simultaneous access to multiple memory blocks through different ports and stages, effectively expanding the data capacity accessible in parallel while maintaining the simplicity of individual memory access operations.
Data Source
AI summary
A data processing device includes a two-dimensional structure including a plurality of stages in a vertical direction, the stages each including basic units in a horizontal direction such that the number of the basic units is equal to the number of ways. The basic units each includes a memory block having a plurality of ports, an address generator for the ports of the memory block, and a calculation unit.