Neural Network Channel Pruning for Resource-Constrained Object Recognition

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing neural network systems require significant resources and processing time for object recognition tasks, especially in environments with limited resources, such as mobile terminals, due to their large size and complexity.

Innovation Solution

The method involves channel pruning in neural networks based on importance values, where channels with lower importance values are deactivated, and only the most critical channels are used for calculations, determined by cumulative distribution functions and a lightweight degree threshold, allowing for efficient processing without significantly reducing performance.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If all channels of the neural network are used for calculation, then recognition accuracy is maintained, but processing time and memory usage increase significantly

Engineering Contradiction:

Improverecognition accuracyVSAvoidprocessing time

Core Design Contradiction:

ReliabilityVSLoss of time

Solution Approach 1:

The patent extracts and removes unnecessary channels from the neural network based on importance values. Channels are pruned by calculating their importance metrics and selectively removing those with lowest importance, thereby reducing processing time while preserving recognition accuracy through the retention of critical channels

Inventive Principle:

Principle #2Taking out (Extraction)

Solution Approach 2:

The patent changes the parameter of channel inclusion by introducing importance values and thresholds. Channels are dynamically selected or excluded based on their calculated importance parameters, allowing the system to adapt the number of active channels according to performance requirements and resource constraints

Inventive Principle:

Principle #35Parameter changes

2Reliability

If all channels of the neural network are used for calculation, then recognition accuracy is maintained, but memory usage increases significantly

Engineering Contradiction:

Improverecognition accuracyVSAvoidmemory usage

Core Design Contradiction:

ReliabilityVSQuantity of substance

Solution Approach 1:

The patent extracts and removes unnecessary channels from the neural network based on importance values. Channels are pruned by calculating their importance metrics and selectively removing those with lowest importance, thereby reducing memory usage while preserving recognition accuracy through the retention of critical channels

Inventive Principle:

Principle #2Taking out (Extraction)

Solution Approach 2:

The patent applies partial action by using only a subset of channels rather than all channels. By activating only the most important channels based on importance values and thresholds, the system achieves sufficient recognition accuracy with reduced memory consumption

Inventive Principle:

Principle #16Partial or excessive action

3Productivity

If the neural network is simplified to reduce resource usage, then processing efficiency improves, but recognition accuracy deteriorates

Engineering Contradiction:

Improveprocessing efficiencyVSAvoidrecognition accuracy

Core Design Contradiction:

ProductivityVSReliability

Solution Approach 1:

Inventive Principle:

Principle #35Parameter changes

Solution Approach 2:

The patent introduces dynamic channel selection where the number of active channels can be adjusted based on importance values and thresholds. This dynamic approach allows the system to optimize between processing efficiency and recognition accuracy by activating only the necessary channels for the given task

Inventive Principle:

Principle #15Dynamics

4Loss of time

If channels are pruned based on importance values, then memory usage and processing time are reduced, but the complexity of determining which channels to keep increases

Engineering Contradiction:

Improveprocessing timeVSAvoidchannel selection complexity

Core Design Contradiction:

Loss of timeVSDevice complexity

Solution Approach 1:

The patent performs preliminary calculation of importance values for all channels before actual inference. By pre-computing importance metrics and establishing thresholds in advance, the system simplifies the runtime decision-making process, reducing processing time during actual object recognition tasks

Inventive Principle:

Principle #10Preliminary action

Data Source

PatentUS20210365790A1Method and apparatus with neural network data processing

Publication Date: 2021.11.25 SAMSUNG ELECTRONICS CO LTD

AI summary

A processor-implemented neural network data processing method includes: receiving input data; determining a portion of channels to be used for calculation among channels of a neural network based on importance values respectively corresponding to the channels of the neural network; and performing a calculation based on the input data using the determined portion of channels of the neural network.