Neural Network Pruning via Data Type Circuit Selection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Large neural networks face challenges in achieving desired levels of throughput and accuracy due to aggressive pruning, which can lead to reduced accuracy and inefficient hardware implementation.
Innovation Solution
A method that selects optimal dt-ds pairs for each layer of a neural network, allowing for parallel computation of sub-matrix operations, and adjusts pruning patterns based on hardware architecture and performance metrics to maintain accuracy while reducing computational requirements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If aggressive pruning is applied to reduce computational requirements, then throughput and hardware costs are improved, but accuracy is lost
Solution Approach 1:
The patent changes the parameters of pruning by introducing data type selection (dt-ds pairs) and configuring circuit structures to compute d×s operations in parallel. This allows the system to optimize the balance between pruning aggressiveness and accuracy retention by adjusting computational parameters rather than simply removing more weights.
Solution Approach 2:
The patent introduces dynamic configurability through multiple dt-ds pairs, where the system can select different data types and circuit structures based on performance requirements. This dynamic approach allows the pruning strategy to adapt between throughput optimization and accuracy preservation depending on the selected configuration.
2Power
If large neural networks are implemented in hardware accelerators, then computational power is increased, but device complexity increases
Solution Approach 1:
The patent segments the weight matrix into sub-matrices and processes them in parallel using multiple dt-ds pairs. Each pair handles a specific data type and circuit structure configuration, dividing the complex computation into manageable segments that can be executed independently and in parallel, thereby reducing overall hardware complexity.
Solution Approach 2:
The patent creates a universal hardware architecture that can handle multiple data types and circuit structures through the dt-ds pair mechanism. This multi-functional approach allows a single hardware accelerator to efficiently process different neural network configurations without requiring separate dedicated hardware for each data type or operation size.
3Quantity of substance
If pruning subsets of weights are selected based on d×s operations, then computational requirements are reduced, but performance evaluation becomes more complex
Solution Approach 1:
The patent simplifies performance evaluation by changing the parameters to standardized dt-ds pairs with predefined data types and circuit structures. This parameterization allows the system to evaluate pruned network performance using consistent metrics across different pruning configurations, reducing the complexity of performance assessment.
Data Source
AI summary
In pruning weights from a neural network (NN), a design tool selects a dt-ds pair from a plurality of dt-ds pairs supported by a target device. Each dt-ds pair specifies a data type, dt, and an associated circuit structure, ds, that is configurable to compute d×s operations in parallel on a set of input activations and a matrix of weights of the data type, d is a number of rows in a sub-matrix of the matrix of weights, s is a number of columns in the sub-matrix, and d×s≥1. The design tool selects as pruned weights, one or more subsets of the weights, based at least on each subset of the one or more subsets including d×s weights in the matrix of weights of the layer. If performance of the pruned NN model is satisfactory, the NN is compiled into an execution graph and configuration data.