Dynamic Sparse Data Rearrangement for Neural Network Efficiency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Machine learning processing, particularly in neural networks, faces inefficiencies due to sparse data and zero-value operations that result in wasted computational efforts, as data and weights are processed regardless of zero values which do not affect the output.
Innovation Solution
Dynamic rearrangement of sparse data and corresponding weights, where zero-value data is relocated to the end of rows and folds containing only zeros are skipped, allowing for reduced processing in systolic arrays and avoiding unnecessary operations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If sparse data is processed using conventional machine learning operations, then the processing follows standard neural network procedures, but wasted computational effort occurs on zero-value operations
Solution Approach 1:
The patent extracts and removes zero-value elements from the data matrix before processing. By identifying and eliminating sparse zero elements, the system prevents wasteful computational operations on these values, thereby improving computational efficiency and reducing power consumption without affecting the final output results.
Solution Approach 2:
The patent applies partial action by processing only the non-zero elements of the data matrix. Instead of performing full matrix operations on all elements including zeros, the system selectively processes only the necessary non-zero portions, reducing unnecessary computational effort and energy expenditure.
2Reliability
If all data values are sent to the systolic array for processing, then complete data processing is performed, but operations on zero values constitute wasted effort
Solution Approach 1:
The patent extracts zero-value elements from the data matrix and separates them from the processing flow. By removing these redundant zero elements before systolic array processing, the system maintains processing completeness for meaningful data while eliminating wasteful operations on zero values, thereby improving computational efficiency.
Solution Approach 2:
The patent performs preliminary action by pre-processing the data matrix to identify and remove zero-value elements before they reach the systolic array. This advance preparation ensures that only necessary computations are performed, maintaining reliability while improving productivity.
3Ease of operation
If activation functions are applied to create output values, then neural network processing is completed, but resulting zero values create additional wasted operations
Solution Approach 1:
The patent extracts and removes zero-value outputs generated by activation functions before they propagate to subsequent processing stages. By eliminating these zero values early in the processing pipeline, the system prevents cascading wasteful operations in downstream layers, thereby improving overall computational efficiency while maintaining ease of neural network operation.
Data Source
AI summary
An approach is described for a method, product, and apparatus for a machine learning process using dynamic rearrangement of sparse data and corresponding weights. This approach includes a method, product, and apparatus for dynamically rearranging input data to move sparse data to a location such that computations on the sparse data might be avoided when executing a machine learning processing job. For example, sparse data within each row of the input matrix can be moved to the end of each corresponding row. When the input data is folded to fit the array, that sparse data might be at least partially contained within a fold that comprises only sparse data and possibly filler data. In such an event, computations on the fold are unnecessary and are avoided. In some embodiments, the approach includes dynamically rearranging a weight matrix to maintain a correspondence between the input data and the weights.