Hierarchical Neural Network Architecture with Disconnected Groups

Solution Overview

Problem

Neural networks require significant computational resources and time for training and inference due to the large number of connections between neurons, leading to high energy consumption and inefficiency.

Innovation Solution

Implement a neural network architecture with groups of neurons arranged into levels, where the last level is split into groups corresponding to a set of classes, and child groups are disconnected from other groups, reducing connections and utilizing semantic relationships to define the network structure.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If a standard neural network architecture with all neurons connected to all neurons in previous and next layers is used, then the neural network can maximize use of computational resources and achieve good performance, but training time becomes very long and energy consumption is maximized

Engineering Contradiction:

Improveneural network performanceVSAvoidtraining time

Core Design Contradiction:

ReliabilityVSLoss of time

Solution Approach 1:

The neural network is segmented into multiple levels with groups of neurons at each level. Child groups are disconnected from parent groups at the same level, creating a hierarchical structure that reduces the total number of connections while maintaining performance. This segmentation allows the network to process information more efficiently through the hierarchical levels.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent introduces a hierarchical dimension to the neural network architecture by organizing neurons into levels and groups. This adds a structural dimension beyond the traditional layer-to-layer connections, creating parent-child relationships that reduce computational complexity while preserving the network's ability to learn and perform inference.

Inventive Principle:

Principle #17Another dimension (Dimensionality change)

2Reliability

If a standard neural network architecture with all neurons connected to all neurons in previous and next layers is used, then the neural network can maximize use of computational resources and achieve good performance, but energy consumption is maximized

Engineering Contradiction:

Improveneural network performanceVSAvoidenergy consumption

Core Design Contradiction:

ReliabilityVSUse of energy by moving object

Solution Approach 1:

The neural network is segmented into multiple levels with groups of neurons at each level. Child groups are disconnected from parent groups at the same level, creating a hierarchical structure that reduces the total number of connections while maintaining performance. This segmentation allows the network to process information more efficiently through the hierarchical levels.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent introduces a hierarchical dimension to the neural network architecture by organizing neurons into levels and groups. This adds a structural dimension beyond the traditional layer-to-layer connections, creating parent-child relationships that reduce computational complexity while preserving the network's ability to learn and perform inference.

Inventive Principle:

Principle #17Another dimension (Dimensionality change)

3Adaptability or versatility

If an adaptive hierarchical network structure is used that grows and learns as new data becomes available, then the network can accommodate new classes of data while preserving ability to distinguish previously trained classes, but computational bloat occurs and the final structure becomes random

Engineering Contradiction:

Improveincremental learning capabilityVSAvoidnetwork structure complexity

Core Design Contradiction:

Adaptability or versatilityVSDevice complexity

Solution Approach 1:

The neural network is segmented into multiple levels with groups of neurons at each level. Child groups are disconnected from parent groups at the same level, creating a hierarchical structure that reduces the total number of connections while maintaining performance. This segmentation allows the network to process information more efficiently through the hierarchical levels.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent establishes a predetermined hierarchical structure with levels and groups before training begins. This preliminary structural organization provides a framework that guides the incremental learning process, preventing random growth and computational bloat while still allowing the network to adapt to new classes of data through the established hierarchical pathways.

Inventive Principle:

Principle #10Preliminary action

Data Source

PatentUS20250209324A1Device, system, and method for implementing a computationally efficient neural network

Publication Date: 2025.06.26 AMADEUS SAS

AI summary

A computing device generates a neural network (NN) comprising an architecture of a plurality of levels of respective groups of respective neurons, a last level of the NN split into groups corresponding to a respective number of a given set of NN classes, at least one child group of a given level of the NN being disconnected from other groups of the given level. The computing device associates the groups of the last level with a respective subset of the given set of NN classes. The computing device trains the NN using a training dataset, the training dataset comprising inputs and outputs corresponding to the given set of the NN classes.