RL-Based Post-Training Sparsification for DNN Accuracy
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep Neural Networks (DNNs) face challenges in achieving real-time inference on platforms without large-scale parallel computing units, and post-training optimization is needed to speed up inference while maintaining accuracy, especially after model sparsification which often results in a drop in accuracy if the model structure is not considered.
Innovation Solution
A sparsity-aware bias correction method combined with a Reinforcement Learning (RL) based Deep Deterministic Policy Gradient (DDPG) agent is used to search for optimal sparsity ratios in an iterative manner, compensating for mean and variance shifts in weights after sparsification, thereby maintaining model accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If model sparsification is applied to reduce computational resources and speed up inference, then productivity and speed are improved, but manufacturing precision (model accuracy) deteriorates
Solution Approach 1:
The patent applies parameter changes by computing correction parameters (mean shift and variance shift) that adjust the statistical properties of sparsified weights. These correction parameters are derived from comparing the original and sparsified weight distributions, and applying them restores the weight statistics to better match the original model, thereby recovering accuracy while maintaining sparsity-induced speedup
Solution Approach 2:
The patent implements feedback by using reinforcement learning to iteratively optimize the sparsification process. The RL agent observes the accuracy drop after sparsification and adjusts the sparsification strategy accordingly, creating a closed-loop system that continuously improves the balance between sparsity and accuracy through learned feedback from performance metrics
2Use of energy by moving object
If model sparsification is applied to reduce computational resources, then use of energy is reduced, but manufacturing precision (model accuracy) deteriorates
Solution Approach 1:
The patent applies parameter changes by computing correction parameters (mean shift and variance shift) that adjust the statistical properties of sparsified weights. These correction parameters are derived from comparing the original and sparsified weight distributions, and applying them restores the weight statistics to better match the original model, thereby recovering accuracy while maintaining sparsity-induced energy reduction
3Manufacturing precision
If reinforcement learning based sparsification is applied to maintain model accuracy, then manufacturing precision is improved, but device complexity increases
Solution Approach 1:
The patent implements feedback by using reinforcement learning to iteratively optimize the sparsification process. The RL agent observes the accuracy drop after sparsification and adjusts the sparsification strategy accordingly, creating a closed-loop system that continuously improves the balance between sparsity and accuracy through learned feedback from performance metrics
Solution Approach 2:
The patent applies parameter changes by computing correction parameters (mean shift and variance shift) that adjust the statistical properties of sparsified weights. These correction parameters are derived from comparing the original and sparsified weight distributions, and applying them restores the weight statistics to better match the original model, thereby recovering accuracy while maintaining sparsity-induced speedup
Data Source
AI summary
Provided herein are apparatus and methods for reinforcement learning based post-training sparsification. An apparatus includes: a memory; and processor circuitry coupled with the memory, wherein the processor circuitry is to: obtain a first correction parameter indicating a mean shift of a set of weights after sparsification of a model with respect to that before the sparsification of the model; obtain a second correction parameter indicating a variance shift of the set of weights after the sparsification of the model with respect to that before the sparsification of the model; and correct the set of weights at least partially based on the first correction parameter and the second correction parameter, and wherein the memory is to store the corrected set of weights. Other embodiments may also be disclosed and claimed.