Fully-Connected Sublayers for Memory-Efficient Large-Class Classification

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

The large memory and bandwidth requirements of fully-connected layers in artificial neural networks, particularly in tasks involving classification of a large number of objects, limit their performance on devices with limited resources.

Innovation Solution

Grouping target classes into smaller subsets and using fully-connected subnetworks to classify objects in multiple steps, reducing the number of weights and calculations required in each step, and training these subnetworks independently.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If fully-connected layers are used to classify a large number of objects, then classification capability is improved, but memory and bandwidth requirements increase

Engineering Contradiction:

Improveclassification capabilityVSAvoidmemory and bandwidth requirements

Core Design Contradiction:

Adaptability or versatilityVSQuantity of substance

Solution Approach 1:

The fully-connected layer is divided into multiple sublayers, each responsible for classifying a subset of target classes. This segmentation reduces the number of weights and computations required in each sublayer compared to a single monolithic fully-connected layer that would handle all classes, thereby reducing memory and bandwidth requirements while maintaining overall classification capability.

Inventive Principle:

Principle #1Segmentation

2Adaptability or versatility

If the number of target classes is increased, then classification versatility is improved, but the number of weights and calculations increases

Engineering Contradiction:

Improveclassification versatilityVSAvoidnumber of weights and calculations

Core Design Contradiction:

Adaptability or versatilityVSDevice complexity

Solution Approach 1:

The set of target classes is divided into multiple subsets, with each sublayer handling a specific subset. This allows the system to achieve high classification versatility across all classes while keeping the complexity (number of weights and calculations) of each individual sublayer manageable and suitable for resource-constrained devices.

Inventive Principle:

Principle #1Segmentation

3Speed

If a single fully-connected layer classifies all target classes in one step, then classification speed is improved, but memory bandwidth requirements increase

Engineering Contradiction:

Improveclassification speedVSAvoidmemory bandwidth requirements

Core Design Contradiction:

SpeedVSQuantity of substance

Solution Approach 1:

Classification is performed in multiple sequential steps using different sublayers, each processing a subset of classes. While this multi-step approach increases the number of processing steps, each step requires significantly less memory bandwidth than a single-step approach would require, making the system feasible for devices with limited memory bandwidth while still achieving reasonable overall classification speed.

Inventive Principle:

Principle #1Segmentation

Data Source

PatentUS12373691B1Executing sublayers of a fully-connected layer

Publication Date: 2025.07.29 AMAZON TECH INC

AI summary

Disclosed herein are techniques for implementing a large fully-connected layer in an artificial neural network. The large fully-connected layer is grouped into multiple fully-connected subnetworks. Each fully-connected subnetwork is configured to classify an object into an unknown class or a class in a subset of target classes. If the object is classified as the unknown class by a fully-connected subnetwork, a next fully-connected subnetwork may be used to further classify the object. In some embodiments, the fully-connected layer is grouped based on a ranking of target classes.