Neural Network Sparsification Using Mask Tensor for Hardware Efficiency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing neural network sparsification methods are not hardware-friendly, leading to high on-chip and off-chip input/output overheads and performance issues, and often result in significant precision loss, especially when applied in resource-constrained devices like mobile phones and satellites.
Innovation Solution
A method and apparatus for sparsification training of neural networks using a mask tensor to compute loss functions during forward propagation, compute gradients during back propagation, and update parameters, allowing for simultaneous sparsification of input and output channel dimensions, optimizing performance and reducing computational and storage requirements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If fine-grained parameter sparsification is used to reduce network parameters, then computing and storage requirements are reduced, but hardware memory access efficiency deteriorates due to increased on-chip and off-chip input/output overheads
Solution Approach 1:
The patent segments the sparsification process into structured channel-level sparsification rather than fine-grained parameter-level sparsification. By organizing sparsity at the channel dimension level, the method creates regular patterns that are more amenable to hardware implementation, reducing random memory access overhead while maintaining parameter reduction benefits
Solution Approach 2:
The patent introduces a new dimension of sparsification control by applying sparsity simultaneously to both input and output channel dimensions of convolution kernels. This two-dimensional sparsification approach transforms the sparsity pattern from scalar parameter level to matrix channel level, enabling better hardware optimization through regular access patterns
2Productivity
If structured sparsity based on channel and convolution kernel is used to improve hardware performance, then memory access efficiency is improved, but model precision deteriorates due to large precision loss
Solution Approach 1:
The patent implements dynamic sparsification where the sparsity pattern is not fixed but adapted during training. By allowing the sparsity configuration to evolve dynamically based on training progress and hardware constraints, the method achieves better balance between hardware performance optimization and model precision preservation
Solution Approach 2:
The patent changes the parameters of sparsification by allowing flexible configuration of sparsity ratios at different channel dimensions. Instead of fixed structured sparsity patterns, the method adjusts sparsity parameters dynamically to optimize both hardware performance and model accuracy for different layers and operations
3Reliability
If off-line fine-tuning is used for sparsification, then initial model training is completed, but training efficiency deteriorates due to limitations in achieving significant performance benefits
Solution Approach 1:
The patent performs sparsification configuration in advance during the training process rather than as a separate off-line fine-tuning stage. By integrating sparsity configuration into the main training workflow, the method eliminates the need for separate fine-tuning passes and achieves both model training and sparsification optimization in a unified efficient process
Solution Approach 2:
The patent maintains continuous useful action by performing sparsification and training simultaneously in an integrated manner. Instead of completing full training then performing separate fine-tuning, the method continuously optimizes both model parameters and sparsity patterns throughout the training process, eliminating idle periods and improving overall training efficiency
Data Source
AI summary
The present disclosure relates to a method and apparatus for sparsification training of a neural network model, a board card, and a readable storage medium. The data processing apparatus of the present disclosure is implemented as a computing apparatus and included in a combined processing apparatus. The combined processing apparatus further includes an interface apparatus and other processing apparatus. The computing apparatus interacts with other processing apparatus to jointly complete a computing operation specified by a user. The combined processing apparatus further includes a storage apparatus. The storage apparatus is connected to the computing apparatus and other processing apparatus, respectively. The storage apparatus is used to store data of the computing apparatus and other processing apparatus. Embodiments of the present disclosure provide a solution related to the sparsification training of the neural network model, which improves operation ability of the neural network model and improves processing efficiency of a machine.