Tensor Memory Access for Multidimensional Data Transfer
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Modern processing systems face inefficiencies in memory access speeds, particularly for large data transfers in parallel processing systems, leading to significant computation overheads and performance losses, especially in applications like deep learning, due to complex address calculations and out-of-bound conditions.
Innovation Solution
A specialized Tensor Memory Access Unit (TMAU) is coupled to each parallel processor core to efficiently transfer large data blocks between shared and global memory, offloading address calculations and handling multidimensional data structures, enabling asynchronous data access and reducing overheads.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If parallel processor cores perform memory access operations directly, then memory access speed is improved, but computation overhead increases and processing efficiency decreases
Solution Approach 1:
The patent extracts memory access operations from the parallel processor cores and assigns them to a dedicated memory access unit. This separation allows processor cores to focus exclusively on computations while the memory access unit handles memory operations, resolving the contradiction between memory access speed and processing efficiency.
Solution Approach 2:
The memory access unit acts as an intermediary between the parallel processor cores and the memory system. It manages address calculations and memory access operations, allowing processor cores to perform computations without being burdened by memory access overhead, thus improving both speed and productivity.
2Measurement precision
If processor cores handle address calculations for multidimensional data structures, then memory access accuracy is improved, but computation overhead and energy consumption increase
Solution Approach 1:
The patent extracts address calculation operations from the processor cores and assigns them to the memory access unit. This dedicated unit handles the complex address calculations for multidimensional data structures, ensuring accuracy while reducing the energy consumption of processor cores.
Solution Approach 2:
The memory access unit performs self-service by autonomously handling address calculations and memory access operations without requiring processor core intervention. This reduces the computational burden and energy consumption of processor cores while maintaining address calculation accuracy.
3Adaptability or versatility
If processor cores perform complex memory access operations, then memory access functionality is improved, but computation speed and throughput decrease
Solution Approach 1:
The patent extracts complex memory access operations from processor cores and assigns them to the memory access unit. This dedicated unit provides comprehensive memory access functionality including handling multidimensional data structures, while processor cores maintain high computation speed and throughput.
Solution Approach 2:
The memory access unit serves as an intermediary that provides comprehensive memory access functionality to processor cores. It handles complex operations such as address calculations for multidimensional data structures, allowing processor cores to focus on computations and maintain high productivity.
Data Source
AI summary
A parallel processing unit comprises a plurality of processors each being coupled to a memory access hardware circuitry. Each memory access hardware circuitry is configured to receive, from the coupled processor, a memory access request specifying a coordinate of a multidimensional data structure, wherein the memory access hardware circuit is one of a plurality of memory access circuitry each coupled to a respective one of the processors; and, in response to the memory access request, translate the coordinate of the multidimensional data structure into plural memory addresses for the multidimensional data structure and using the plural memory addresses, asynchronously transfer at least a portion of the multidimensional data structure for processing by at least the coupled processor. The memory locations may be in the shared memory of the coupled processor and/or an external memory.