Tensor Memory Layout to Avoid Bank and Pipeline Conflicts
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing computing environments face memory bank conflicts and pipeline conflicts due to parallel execution of operations in vector, scalar, and load storage unit pipelines, leading to performance regression and increased instruction execution time.
Innovation Solution
A method using hard and soft constraints applied in a MaxSAT solver to optimize tensor memory layout by determining the locations of input/output vectors, ensuring no intersection between memory banks and mitigating conflicts, while allowing concurrent reading and writing operations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If parallel execution of operations is implemented in vector, scalar, and load storage unit pipelines, then processing throughput is improved, but memory bank conflicts and pipeline conflicts occur leading to performance regression
Solution Approach 1:
The patent segments the local memory into multiple memory banks organized in tiers and bank groups. By dividing the memory access space into distinct segments (banks and bank groups), the system enables parallel access to different segments without conflicts, thus maintaining high throughput while avoiding memory bank conflicts that would degrade performance stability.
Solution Approach 2:
The patent introduces a hierarchical dimension to memory organization with tiers, bank groups, and individual memory banks. This multi-dimensional structure allows the system to manage parallel access requests by routing them through different levels of the hierarchy, resolving conflicts in one dimension while maintaining parallelism in other dimensions.
2Reliability
If manual memory allocation is performed to avoid conflicts, then memory access reliability is improved, but device complexity and ease of operation deteriorate
Solution Approach 1:
The patent implements self-service through automated memory allocation mechanisms that use the hierarchical memory structure to automatically resolve conflicts. The system autonomously manages memory bank assignments and access scheduling without requiring manual intervention, thereby maintaining high reliability while reducing operational complexity.
Solution Approach 2:
The patent incorporates feedback mechanisms that monitor memory access patterns and dynamically adjust allocation strategies. By detecting potential conflicts and automatically rerouting access requests through the hierarchical structure, the system maintains reliability while eliminating the need for complex manual allocation procedures.
3Speed
If data is transferred from global memory to local memory for processing, then processing speed is improved, but memory storage conflicts occur increasing instruction execution time
Solution Approach 1:
The patent segments incoming data streams into multiple channels that are distributed across different memory banks and bank groups. This segmentation allows simultaneous transfer and processing of data without storage conflicts, maintaining high processing speed while avoiding delays caused by conflict resolution.
Solution Approach 2:
The patent performs preliminary organization of data during the transfer from global to local memory, pre-positioning data in optimal locations within the hierarchical structure. This preliminary action prevents future conflicts by ensuring that subsequent access patterns can proceed without stalls, thereby maintaining processing speed without increasing execution time.
Data Source
AI summary
A method for optimizing a layout of a tensor memory defines at least one hard constraint for allocating a plurality of input/output (I/O) vectors for reading and writing data for a task in the tensor memory. The at least one hard constraint is applied to determine one or more potential conflicts between the plurality of I/O vectors. One or more soft constraints aimed at mitigating the one or more potential conflicts between the I/O vectors may also be generated. The at least one hard constraint is applied in a maximum satisfiability (MaxSAT) solver. The one or more soft constraints may also be applied in the MaxSAT solver. The MaxSAT solver determines locations of the data in the tensor memory. The starting addresses of the input data to be read and of output data to be written by each of the I/O vectors are updated in the tensor memory.