Compute-in-Memory Matrix Transpose Without Weight Repositioning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The calculation of matrix transposes in learning networks is time-consuming and requires significant additional space, often necessitating separate hardware or repositioning of weights, leading to increased latency and CIM area usage.
Innovation Solution
A compute-in-memory (CIM) hardware module that performs both vector-matrix multiplication (VMM) and matrix transpose multiplication without moving weights, using selection logic and adder trees to efficiently handle both operations in parallel.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If separate hardware is used to compute matrix multiplications for matrix and matrix transpose, then computation capability is improved, but device complexity and CIM area are doubled
Solution Approach 1:
The patent implements a universal CIM hardware module that can perform both forward propagation (matrix multiplication) and backward propagation (matrix transpose multiplication) operations using the same physical hardware. The compute logic is designed to be reconfigurable through control signals, allowing the same circuit to execute different mathematical operations without requiring separate dedicated hardware for each operation type.
Solution Approach 2:
The patent merges the functionality of separate matrix multiplication and matrix transpose multiplication hardware into a single integrated CIM module. By combining these operations in one hardware unit, the patent reduces the overall CIM area requirements and eliminates the need for duplicate computational resources, directly addressing the contradiction between computation capability and device complexity.
2Ease of operation
If weights are moved from memory to be transposed and rewritten back, then matrix transpose computation is enabled, but latency increases significantly
Solution Approach 1:
The patent replaces the mechanical approach of physically moving and repositioning weight data in memory with a computational approach where the same weight data remains stationary in memory while the computation logic is reconfigured to perform transpose operations. This substitution eliminates the time-consuming data movement and rewriting operations, significantly reducing latency while enabling matrix transpose computation.
Solution Approach 2:
The CIM hardware module performs the transpose operation internally through reconfigurable compute logic without requiring external memory read/write operations. The weights serve themselves by remaining in place while the computation adapts to perform the transpose operation, eliminating the need for weight repositioning and reducing the time required for matrix transpose computations.
3Speed
If both matrix and matrix transpose are stored in memory, then computation access is improved, but CIM area requirements increase
Solution Approach 1:
The patent implements a universal CIM hardware module that can perform both forward propagation (matrix multiplication) and backward propagation (matrix transpose multiplication) operations using the same physical hardware. The compute logic is designed to be reconfigurable through control signals, allowing the same circuit to execute different mathematical operations without requiring separate dedicated hardware for each operation type.
Solution Approach 2:
The patent changes the operational parameters of the CIM hardware module through control signals to switch between matrix multiplication and matrix transpose multiplication modes. By changing the computational parameters rather than the physical data arrangement, the system achieves fast access to both operation types while using the same memory storage space, thereby reducing CIM area requirements.
Data Source
AI summary
A system including a memory and hardware compute logic is described. The memory includes memory cells storing weights corresponding to a matrix. Hardware compute logic is coupled with the memory cells. The hardware compute logic is configured to perform a vector-matrix multiplication (VMM) for the matrix and for a matrix transpose for the weights being stationary for the memory cells.