Neural Network Sparsity via Attribute Propagation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep neural networks (DNNs) require significant computational resources for training and inference, necessitating techniques to reduce computational costs and model size, with existing sparsity methods introducing complexity in input data, weights, and activations.
Innovation Solution
A computing system is configured to provide sparsity for neural network models by propagating sparsity attributes through tensors, pruning, and quantizing elements, transforming operators into sub-operators, and specializing code to generate an executable neural network model that can be efficiently executed on hardware.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If deep neural networks are increased in size to achieve better accuracies, then model accuracy is improved, but computational cost increases
Solution Approach 1:
The patent applies parameter changes by modifying the sparsity attributes of tensors during model construction. By changing the sparsity parameter values, the system dynamically adjusts the balance between model accuracy and computational cost, allowing the neural network to achieve desired accuracy with reduced computational requirements through optimized sparsity levels.
Solution Approach 2:
The patent implements local quality by applying different sparsity attributes to different tensors within the neural network model. Instead of uniform sparsity across all tensors, the system selectively applies varying degrees of sparsity to specific tensors based on their importance and impact on model accuracy, thereby reducing overall computational cost while maintaining critical accuracy levels.
2Use of energy by moving object
If sparsity techniques are added to reduce computational costs, then computational cost is reduced, but model complexity increases
Solution Approach 1:
The patent applies preliminary action by pre-defining and propagating sparsity attributes through the neural network model during the construction phase. By establishing sparsity patterns beforehand rather than computing them dynamically during execution, the system reduces computational cost during inference while avoiding the complexity of real-time sparsity management.
Solution Approach 2:
The patent implements universality by creating a unified sparsity attribute propagation mechanism that works across different tensor types and network architectures. The same sparsity attribute framework can be applied universally to various tensors (weights, activations, inputs) without requiring architecture-specific customizations, thereby reducing model complexity while achieving computational cost reduction.
3Use of energy by moving object
If sparsity attributes are propagated through tensors to introduce sparsity, then computational cost is reduced, but model size increases
Solution Approach 1:
The patent applies the taking out principle by extracting and removing redundant tensor elements based on propagated sparsity attributes. During model construction, elements marked for sparsity are identified and removed or zeroed out, thereby reducing the actual computational workload and effective model size while maintaining the structural framework needed for sparsity management.
4Productivity
If operators are transformed into sub-operators and code is specialized, then computational efficiency is improved, but implementation complexity increases
Solution Approach 1:
The patent applies segmentation by dividing operators into sub-operators based on sparsity attributes during code generation. This segmentation allows each sub-operator to be optimized independently for specific sparsity patterns, improving computational efficiency by reducing unnecessary computations while managing implementation complexity through systematic operator decomposition.
Solution Approach 2:
The patent implements dynamics by generating specialized code that adapts to the specific sparsity attributes of each tensor. The code generation process dynamically creates optimized operations tailored to the actual sparsity patterns present in the model, improving computational efficiency by avoiding generic computations while keeping implementation complexity manageable through automated code synthesis.
Data Source
AI summary
Embodiments of the present disclosure include systems and methods for providing sparsity for neural network models based on sparsity attributes. A first neural network model definition is received. The first neural network model definition specifies a neural network model comprising a set of tensors and a set of sparsity attribute values for elements of a tensor in the set of tensors. The set of sparsity attribute values for the tensor are propagated to elements of a subset of the set of tensors to form a second neural network model definition. The neural network model is generated based on the second neural network model definition.