Structured Network Sparsity for Faster Image Classification Inference
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep learning network models have large sizes and high computing overheads due to a large number of parameters, leading to significant time consumption during inference.
Innovation Solution
Perform initial structured sparsity on network layers based on a target sparsity mode and ratio, optimizing the sparsity ratio to achieve a target precision, and using matrix processing operators to maintain output matrix dimensions, reducing element counts in input matrices.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If the size of deep learning network model increases to improve model capability, then the model can handle more complex tasks, but computing overheads and inference time increase significantly
Solution Approach 1:
The patent extracts and removes redundant or less important elements from the network model by applying structured sparsity. Specifically, it prunes elements in input matrices of operators (such as softmax operators in self-attention mechanisms) based on predefined sparsity modes and ratios, retaining only the most significant elements while discarding others. This extraction process reduces the computational burden without completely removing the model's capability to handle complex tasks.
Solution Approach 2:
The patent applies different sparsity ratios and modes to different network layers and operators based on their specific characteristics and importance. Rather than uniformly sparsifying the entire model, it performs localized sparsity optimization where critical layers maintain higher density while less critical layers undergo more aggressive sparsification. This local quality approach ensures that model capability is preserved in important regions while reducing overall computational overhead.
2Measurement precision
If the number of parameters in network model increases to improve accuracy, then the model precision improves, but computing overheads increase
Solution Approach 1:
The patent extracts and removes redundant parameters from the network model by applying structured sparsity. It identifies and prunes less important parameters in input matrices based on predefined sparsity configurations, retaining only the essential parameters needed for maintaining inference precision. This extraction process reduces the total parameter count and associated computing overheads while preserving the model's accuracy on critical tasks.
Solution Approach 2:
The patent changes the density parameters of the network model by introducing controlled sparsity. It modifies the parameter distribution across different layers and operators by applying different sparsity ratios, transforming the model from a dense structure to a structured sparse structure. This parameter change reduces the effective number of computations required while maintaining the essential functionality and precision of the model.
3Productivity
If structured sparsity is applied to reduce computing overheads, then inference time decreases, but model precision may deteriorate
Solution Approach 1:
The patent applies different sparsity ratios and modes to different network layers and operators based on their specific characteristics and importance. Rather than uniformly sparsifying the entire model, it performs localized sparsity optimization where critical layers maintain higher density while less critical layers undergo more aggressive sparsification. This local quality approach ensures that model capability is preserved in important regions while reducing overall computational overhead.
Solution Approach 2:
The patent introduces dynamic adjustment mechanisms for sparsity configurations. It allows the sparsity ratio and mode to be adjusted based on the specific operator type, layer position, and performance requirements. This dynamic approach enables the model to adaptively balance between inference efficiency and precision, switching between different sparsity levels as needed to maintain optimal performance.
Data Source
Figure 1~2
Figure 3~4
Figure 5~6
AI summary
This application pertains to the field of artificial intelligence technologies, and discloses a structured sparsity method for a network model, an image classification method, and a related apparatus. The method includes: performing, based on a target sparsity mode and a target sparsity ratio, initial structured sparsity on each of a plurality of to-be-sparsified network layers included in a deep learning network model, to obtain an initial sparse network model; determining a precision of the initial sparse network model based on a test sample set; and when the precision of the initial sparse network model reaches a target precision, determining the initial sparse network model as a deep learning network model obtained through structured sparsity. Initial structured sparsity is performed on each of the plurality of to-be-sparsified network layers included in the deep learning network model, to reduce a quantity of elements in an input matrix of a target operator at each network layer, resolve a problem that a large amount of time needs to be spent in inference of the target operator at each network layer, and further reduce computing overheads of the deep learning network model.