Address Generation Unit for Deep Learning Tensor Access
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep learning processors face inefficiencies in data access due to the inability of stride access mode to support multiple parameters during tensor traversal, leading to interrupted tensor operations and reduced computing efficiency.
Innovation Solution
An address generation method is developed that configures execution parameters based on tensor dimensions such as depth, sliding window, and feature map, allowing for sequential data access and improving memory access efficiency by using an address generation unit (AGU) that computes target addresses for data access in deep learning processors.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If stride access mode is used for vector operations, then memory access overhead is reduced and continuous access addresses are generated, but the mode cannot support multiple parameters in tensor traversal process, causing operations to be interrupted by control signals
Solution Approach 1:
The address generation unit is designed to perform multiple functions: it can generate addresses in both stride access mode for vector operations and in a parameter-supported mode for tensor traversal. The AGU includes multiple address generation modules that can be selectively activated based on the operation type, allowing a single unit to handle diverse access patterns without interruption.
Solution Approach 2:
The address generation unit dynamically switches between different address generation modes based on the operation being performed. Control logic detects whether a vector operation or tensor operation is required and activates the appropriate address generation module, enabling adaptive behavior that optimizes performance for different workloads.
2Speed
If stride access mode is used, then processing efficiency is improved through continuous address generation, but tensor traversal requires interruption by control signals reducing overall efficiency
Solution Approach 1:
The address generation unit pre-calculates and prepares address sequences for tensor traversal operations before execution begins. By anticipating the address generation needs and preparing them in advance, the system eliminates the need for mid-operation control signal interruptions, maintaining continuous address flow throughout the tensor traversal process.
3Adaptability or versatility
If a general-purpose address generation unit is designed to support multiple access modes, then versatility is improved, but device complexity increases
Solution Approach 1:
The address generation unit is segmented into multiple specialized modules, each responsible for a specific access mode or operation type. This modular architecture allows each module to remain relatively simple while the aggregate system provides comprehensive functionality. Modules can be independently optimized and activated only when needed.
Solution Approach 2:
A control logic intermediary manages the complexity by selectively activating appropriate address generation modules based on operation requirements. This intermediary layer shields the rest of the system from complexity while enabling versatile access modes through coordinated module activation.
Data Source
Figure 1(A)~2
Figure 3
Figure 4
AI summary
The present application can be applied to the technical field of artificial intelligence, and can further relate to the field of cloud computing. Disclosed are an address generation method, a related device, and a storage medium. The present application comprises: obtaining a first sliding window width count value, a first sliding window height count value, a first feature map width count value, and a first feature map height count value which correspond to first target data; if the first sliding window width count value is within a sliding window width value range, the first sliding window height count value is within a sliding window height value range, the first feature map width count value is within a feature map width value range, and the first feature map height count value is within a feature map height value range, determining an offset of the first target data; obtaining an initial address of the first target data; and summing the initial address and the offset to obtain a first target address of the first target data.