Address Offset Elements for Prologue Epilogue Data Access
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing technologies face challenges in efficiently determining memory addresses for prologue and epilogue data and tensor elements in machine learning computations, particularly in handling deeply-nested loops and multi-dimensional tensors, which leads to increased computational cycles and resource usage.
Innovation Solution
A special purpose computational unit with address offset value elements and processors that calculate address offset values using hardware adders, allowing for efficient determination and access of memory addresses for tensor and data elements through a single instruction, reducing the need for complex circuitry and enhancing processor bandwidth.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional general-purpose processors are used to determine memory addresses for prologue and epilogue data in machine learning computations, then the system can handle complex computations, but the computational cycles and resource usage increase significantly
Solution Approach 1:
The patent segments the address computation function from general-purpose processors by introducing dedicated address computation elements and hardware adders specifically for calculating memory addresses of prologue and epilogue data. This separation allows the main processor to focus on computational tasks while specialized hardware handles address generation, thereby reducing computational cycles and improving overall productivity.
Solution Approach 2:
The patent introduces address computation elements as intermediary components between the processor and memory system. These elements receive loop iteration information and tensor dimension data, compute the necessary address offsets using hardware adders, and provide the final memory addresses to the memory access unit. This intermediary layer eliminates the need for the main processor to perform address calculation, reducing its computational burden and cycle usage.
2Measurement precision
If complex circuitry is used to track multi-dimensional tensors and prologue/epilogue data arrays concurrently, then accurate address determination is achieved, but device complexity increases
Solution Approach 1:
The patent applies local quality by providing different levels of computational capability to different parts of the address computation system. Address computation elements use simple hardware adders for basic offset calculations, while the control logic selectively activates specific computation paths based on the data type (prologue/epilogue vs. tensor elements). This localized approach maintains accuracy for each data type while avoiding the complexity of a unified complex circuit.
Solution Approach 2:
The patent changes parameters dynamically based on the computation type. For prologue and epilogue data, the system uses loop iteration counts and base addresses to compute addresses. For tensor elements, it uses dimension sizes and iteration indices. The address computation elements switch between these different parameter sets based on control signals, maintaining precise address determination for each case while using simple, parameter-specific computation logic rather than complex universal circuitry.
Data Source
Figure 1
Figure 2A
Figure 2B
AI summary
Methods, systems, and apparatus, including an apparatus for accessing data. In some implementations, an apparatus includes address offset value elements that are each configured to store an address offset value. For each address offset value element, the apparatus can include address computation elements that each store a value used to determine the address offset value. One or more processors are configured to receive a program for performing computations using tensor elements of a tensor. The processor(s) can identify, in the program, a prologue or epilogue loop having a corresponding data array for storing values of the prologue or epilogue loop and populate, for a first address offset value element that corresponds to the prologue or epilogue loop, the address computation elements for the first address offset value element with respective values based at least on a number of iterations of the prologue or epilogue loop.