Neural Network Data Alignment for MAC Utilization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional neural network operation methods experience reduced MAC utilization and increased power consumption due to unaligned input channels, leading to unnecessary memory overhead and cycles in neural processing units (NPUs).
Innovation Solution
A neural network operation method and apparatus that stacks data to align feature maps and kernels, allowing for parallel processing through direct memory access (DMA) engines, thereby optimizing data alignment and reducing operation time.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If unaligned input channels are used in conventional neural network operation, then data processing can proceed, but MAC utilization is lowered and power consumption increases
Solution Approach 1:
The patent applies preliminary action by performing data alignment before the neural network operation. The NPU aligns input channel data in advance by reordering or padding operations, so that when the actual computation occurs, the data is already in the correct aligned state, maximizing MAC utilization and avoiding wasted cycles
Solution Approach 2:
The patent changes the parameter of data alignment by transforming unaligned input channels into aligned channels through systematic reordering and padding operations. This parameter transformation ensures that subsequent MAC operations work with properly aligned data, eliminating the trade-off between processing speed and energy efficiency
2Productivity
If unaligned input channels are used, then neural network operation can execute, but memory overhead increases due to dummy channels
Solution Approach 1:
The patent extracts and removes unnecessary dummy channels from the input data before processing. By identifying and eliminating these redundant channels that result from misalignment, the system reduces memory overhead while maintaining the integrity of actual meaningful data for processing
3Productivity
If unaligned input channels are used, then data processing can occur, but unnecessary cycles are consumed
Solution Approach 1:
The patent performs alignment as a preliminary action before the main computation, so that the NPU spends its cycles on productive MAC operations rather than corrective realignment operations during execution, eliminating wasted processing cycles
Data Source
AI summary
A neural network operation method and apparatus are disclosed, where the network operation method including receiving data for a neural network operation, determining whether a size of the data is less than or equal to a threshold, generating stacked data by stacking a portion of the data based on the determining, and performing the neural network operation in parallel based on the stacked data.