Convolutional Layer Weight Pruning for Balanced Parallel Processing

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Deep neural networks face inefficiencies in workload balancing due to zero-skipping operations, leading to unbalanced processing times among processing elements (PEs) and output channels (OCs) in deep learning accelerators, which affect processing speed and power efficiency.

Innovation Solution

Implement workload balancing techniques such as paired-PE sharing, OC reordering, and dynamic asymmetric multi-core IC slicing to equalize the number of non-zero weights across active OCs and ICs, using a compiler or controller to manage workload distribution and pruning processes.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Speed

If zero-skipping operations are performed in DLA to accelerate processing, then processing speed is improved, but workload becomes unbalanced among PEs and OCs

Engineering Contradiction:

Improveprocessing speedVSAvoidworkload balance

Core Design Contradiction:

SpeedVSEase of operation

Solution Approach 1:

The patent applies preliminary action by performing weight pruning before DNN processing to equalize the distribution of non-zero weights across OCs. The controller identifies and prunes weights in advance such that each OC has an equal number of non-zero weights, ensuring balanced workload distribution before processing begins. This preliminary equalization enables subsequent zero-skipping operations to proceed with balanced workloads across all PEs and OCs.

Inventive Principle:

Principle #10Preliminary action

2Loss of energy

If pruning is performed to sparsify neural network model, then power efficiency is improved, but processing time variability increases due to unbalanced workloads

Engineering Contradiction:

Improvepower efficiencyVSAvoidprocessing time variability

Core Design Contradiction:

Loss of energyVSLoss of time

Solution Approach 1:

The patent applies local quality by differentiating the treatment of weights based on their local characteristics. The controller evaluates the sparsity pattern of each OC individually and prunes weights locally to achieve equal numbers of non-zero weights across different OCs. This localized pruning approach ensures that each OC has uniform workload characteristics, eliminating processing time variability while maintaining the power efficiency benefits of sparsification.

Inventive Principle:

Principle #3Local quality

3Productivity

If unbalanced workloads are processed in parallel, then throughput is maintained, but processing time variability increases affecting overall performance

Engineering Contradiction:

ImprovethroughputVSAvoidprocessing time variability

Core Design Contradiction:

ProductivityVSLoss of time

Solution Approach 1:

The patent applies equipotentiality by creating equal workload conditions across all OCs before parallel processing. The controller prunes weights such that each OC has an equal number of non-zero weights, establishing equipotential workload distribution. This ensures that all PEs processing different OCs experience identical computational loads, eliminating processing time variability while maintaining high throughput through parallel execution.

Inventive Principle:

Principle #12Equipotentiality

Data Source

PatentUS20250299016A1Method of pruning weights in convolutional layer of neural network

Publication Date: 2025.09.25 MEDIATEK INC

AI summary

A method includes: receiving N sets of weights of a convolutional layer of a neural network, each set of the weights having a same number of weights and corresponding to one of a sequence of output channels (OCs) of the convolutional layer; and performing a pruning process to prune M sets of the weights among the N sets of the weights such that each of the M sets of the weights has a same number of non-zero weights, M being smaller than or equal to N, M being equal to a number of active OCs to be processed in parallel in a neural network processor.