Neural Network Matrix Shuffling for MAC Operator Optimization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing neural network operations, particularly elementwise operations, require separate weights for multiplication and have limitations in hardware resource utilization due to restrictions on MAC operators, which affects efficiency and speed.
Innovation Solution
A method that involves shuffling elements of matrices and performing replacement operations such as max-pool, average pool, or convolution operations to merge and optimize neural network operations, reducing the need for separate weights and improving hardware utilization by adjusting kernel and stride sizes based on the number of matrices.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If MAC operator is used to perform elementwise operation, then the operation can be executed, but separate weights are required for multiplication which increases device complexity
Solution Approach 1:
The patent merges multiple elementwise operations into a single MAC operator by shuffling input data in memory. Instead of requiring separate MAC operators for each operation, the data is rearranged so that one MAC operator can sequentially perform multiple elementwise operations (e.g., elementwise-sum, elementwise-max) by processing shuffled data streams, thereby eliminating the need for separate weights for each operation type.
Solution Approach 2:
The patent changes the data arrangement parameters in memory through shuffling operations. By reorganizing the spatial arrangement of input data elements before they reach the MAC operator, the same hardware can perform different elementwise operations without changing the operator itself or requiring additional weights, thus reducing device complexity while maintaining operational versatility.
2Adaptability or versatility
If separate MAC operators are provided for different operations, then operation versatility is improved, but hardware resource utilization decreases
Solution Approach 1:
The patent makes a single MAC operator universal by enabling it to perform multiple elementwise operations through data shuffling. The MAC operator becomes multi-functional, handling elementwise-sum, elementwise-max, and other operations by processing differently shuffled input data, thereby eliminating the need for multiple dedicated operators and improving hardware resource utilization while maintaining versatility.
Solution Approach 2:
The patent introduces a new dimension of operation differentiation through data arrangement in memory rather than through separate hardware operators. By changing how data is organized and accessed (spatial arrangement in memory), the system achieves operational diversity without multiplying hardware resources, thus improving resource utilization while maintaining adaptability.
3Ease of manufacture
If elementwise operations are performed using traditional MAC operators, then operations can be executed, but processing speed is limited due to sequential processing requirements
Solution Approach 1:
The patent performs preliminary shuffling of data in memory before the actual MAC operations are executed. This pre-processing step organizes the data so that subsequent elementwise operations can be performed more efficiently by the MAC operator, reducing the need for repeated data access and enabling faster sequential processing of multiple operations.
Solution Approach 2:
The patent enables continuous processing by shuffling data so that the MAC operator can execute multiple elementwise operations in a continuous sequence without idle time. By pre-arranging the data in memory, the operator maintains continuous useful action across multiple operations rather than requiring reconfiguration or data re-fetching between operations, thereby improving processing speed.
Data Source
AI summary
A neural network operation method includes storing a matrix on which an operation of a neural network is to be performed, shuffling a portion of elements of the matrix, and performing a replacement operation for the operation based on the shuffled matrix.