MMA Operand Storage Using Exclusive Memory for Parallel Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Performing matrix multiply-accumulate (MMA) operations in a processor results in the register file being locked, forcing other instructions to wait, which limits parallel processing capabilities.
Innovation Solution
Utilizing exclusive memory, such as tensor memory and shared memory, to store operands and accumulated results for MMA operations, allowing the register file to be available for other operations, enabling asynchronous and concurrent processing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the register file is locked to ensure exclusive access for MMA operations, then data integrity and correctness are improved, but parallel processing capability and instruction throughput deteriorate
Solution Approach 1:
The patent segments the register file into multiple partitioned register files, allowing different MMA operations to access different partitions simultaneously. This segmentation enables parallel processing while maintaining exclusive access to each partition, resolving the contradiction between data integrity and parallel processing capability.
Solution Approach 2:
The patent introduces a temporal dimension by implementing asynchronous execution where MMA operations can overlap in time across different partitions. This allows the system to maintain data integrity through proper synchronization while achieving parallel processing through time-multiplexed access patterns.
2Manufacturing precision
If the register file is locked for MMA operations, then result accuracy is improved, but instruction execution time and waiting time increase
Solution Approach 1:
By dividing the register file into multiple partitions, the patent allows multiple MMA operations to execute simultaneously without blocking each other. This segmentation reduces the waiting time for instructions while maintaining result accuracy through exclusive access to each partition during its operation.
Solution Approach 2:
The patent enables continuous execution of useful actions by allowing overlapping MMA operations across different partitions. While one partition is performing an MMA operation, other partitions can simultaneously execute different operations, eliminating idle waiting time and improving overall instruction execution throughput.
3Productivity
If exclusive memory storage is used for MMA operations, then parallel processing capability is improved, but memory access complexity and device complexity increase
Solution Approach 1:
The patent segments the memory architecture into multiple independent memory spaces or buffers associated with each partition. This segmentation simplifies memory access patterns by providing dedicated storage for each MMA operation, reducing the complexity of managing shared memory while enabling parallel processing.
Solution Approach 2:
The patent introduces memory partitions or buffers as intermediary structures between the MMA operations and the final register storage. These intermediaries simplify the memory access protocol by providing dedicated temporary storage for each operation, reducing the complexity of coordinating simultaneous memory accesses across multiple threads.
Data Source
AI summary
Apparatuses, systems, and techniques to perform an instruction to use storage to store information to be used exclusively by one or more tensor operations. In at least one embodiment, a processor retrieves information from storage that exclusively stores matrix information in response to an instruction and performs a multiplication computation using said matrix information.