Neural Network Model Conversion via Weight Sparsity Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
High-speed devices for neural network models require a sparsity of 50% or higher to accelerate operations, limiting their effectiveness when the sparsity is below this threshold, as seen in neural network models with weight value groups having sparsity levels like 30%.
Innovation Solution
A neural network model conversion device that determines a division position in a weight value group to create sub-groups with a sparsity of 50% or higher, adding a connection layer to facilitate the use of high-speed devices by optimizing the distribution of weight values.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If a high-speed device is used for neural network operations, then operation speed is improved, but the device cannot accelerate operations when sparsity is below a certain threshold (e.g., 50%)
Solution Approach 1:
The patent divides the weight value group into multiple sub-groups by determining division positions based on sparsity thresholds. This segmentation allows the system to identify portions of the weight matrix that can be processed by high-speed devices (with sparsity ≥50%) while separating portions that require general device processing (with sparsity <50%). The division position determination unit analyzes the weight value group and identifies optimal division points to create sub-groups with different sparsity characteristics, enabling selective device assignment and maintaining operational speed benefits across varying sparsity levels.
2Productivity
If the weight value group is divided into multiple sub-groups, then the sparsity of at least one sub-group reaches 50% or higher for high-speed device utilization, but the structural complexity of the neural network model increases due to addition of connection layers
Solution Approach 1:
The weight value group is segmented into multiple sub-groups based on sparsity characteristics, with division positions determined to ensure at least one sub-group achieves 50% or higher sparsity for high-speed device processing. This segmentation strategy balances computational efficiency gains against structural complexity by creating manageable sub-groups rather than uniformly distributing all weight values.
Solution Approach 2:
Connection layers are introduced as intermediary structures between the divided sub-groups and the original single-layer structure. These connection layers mediate the transformation from the original weight value group to the segmented sub-groups, enabling the system to maintain functional equivalence while facilitating high-speed device utilization. The connection layers coordinate the output of different sub-groups and integrate their results, managing the added structural complexity in a controlled manner.
Data Source
AI summary
The division position determination unit determines a division position in a weight value group, which is a weight value group of at least one layer included in a given neural network model, and has a configuration kernels are arranged in a kernel direction, each of which is obtained by arranging at least one or more weight values in a channel direction. The division unit obtains multiple weight value groups by dividing the weight value group at the division position. The connection layer addition unit adds a connection layer that is a layer that connects respective output data obtained by calculating input data to the layer and respective weight value groups after division to make one output data.