Adaptive Neural Network Deployment via Nested Sub-Networks

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing methods for deploying artificial neural networks in resource-limited devices face challenges due to inconsistent batch normalization statistics when adapting to different resource configurations, leading to inefficiencies and the need for re-training or re-tuning.

Innovation Solution

The method involves training an artificial neural network to form a set of nested sub-networks, each optimized for a specific resource configuration, using an ordered dropout operation to rank building blocks and a heuristic search algorithm to determine the optimal configuration without re-training, allowing for adaptive deployment across varying resource budgets.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If heuristics like maintaining multiple sets of switchable batch normalization parameters are used to handle adaptive channel numbers or quantization bits, then the neural network can adapt to different resource configurations, but the number of batch normalization parameters increases quadratically, substantially limiting the flexibility of the neural network

Engineering Contradiction:

Improveadaptability to different resource configurationsVSAvoidnumber of batch normalization parameters

Core Design Contradiction:

Adaptability or versatilityVSDevice complexity

Solution Approach 1:

The patent merges the batch normalization parameters across different channel configurations by sharing a single set of parameters instead of maintaining separate sets. This is achieved through the novel training method that enables a single batch normalization layer to function correctly across multiple channel numbers without requiring quadratic numbers of parameters, thus reducing complexity while maintaining adaptability.

Inventive Principle:

Principle #5Merging (Combining)

Solution Approach 2:

The patent creates a universal batch normalization parameter set that serves multiple channel configurations simultaneously. The trained neural network can adapt to different resource configurations (channel numbers, quantization bits) using the same shared batch normalization parameters, making the system multi-functional without the complexity penalty of maintaining separate parameter sets for each configuration.

Inventive Principle:

Principle #6Universality (Multi-functionality)

2Ease of manufacture

If search-based methods are applied to determine optimal setup for resource-limited devices, then manual hyper-parameter tuning is reduced, but the algorithms must be re-run when resource configuration changes, leading to inefficient operation

Engineering Contradiction:

Improveease of configuration adjustmentVSAvoidtime for re-training or re-tuning

Core Design Contradiction:

Ease of manufactureVSLoss of time

Solution Approach 1:

The patent performs preliminary training of the neural network with a specific training method that pre-determines the optimal architecture for various resource configurations. This preliminary action encodes the adaptability into the network structure itself, allowing the system to respond to resource changes without requiring time-consuming re-training or re-tuning, thus eliminating the loss of time while maintaining ease of configuration adjustment.

Inventive Principle:

Principle #10Preliminary action

3Use of energy by moving object

If full-precision weights are quantized into a small number of bits to incorporate neural networks into resource-limited devices, then computing resources are reduced, but the batch normalization statistics become inconsistent when adapting to different resource configurations

Engineering Contradiction:

Improvecomputing resources consumptionVSAvoidconsistency of batch normalization statistics

Core Design Contradiction:

Use of energy by moving objectVSReliability

Solution Approach 1:

The patent introduces dynamic adaptability to the batch normalization layer through a novel training method. The system can dynamically adjust to different quantization bit widths and channel configurations while maintaining consistent batch normalization statistics. This dynamic capability allows the network to operate reliably across various resource configurations without the statistical inconsistencies that typically arise from quantization, thus maintaining reliability while reducing computing resource consumption.

Inventive Principle:

Principle #15Dynamics

Data Source

PatentUS12093817B2Artificial neural network configuration and deployment

Publication Date: 2024.09.17 CITY UNIVERSITY OF HONG KONG

AI summary

A computer-implemented method and a system for configuring and implementing an artificial neural network. The method for configuring includes initializing an artificial neural network; and training, based on a training operation, the artificial neural network so as to form an adaptively deployable artificial neural network defining a plurality of nested artificial neural sub-networks. Each artificial neural sub-network is optimal for a respective resource configuration. The method for implementing includes determining an optimal configuration of the artificial neural network for deployment at an electrical device, and deploying the artificial neural network with the optimal configuration at the electrical device.