Hierarchical Neural Network Coupling Using Error-Correcting Codes

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Conventional hierarchical neural networks require pre-learning to form loose couplings, which is time-consuming and computationally intensive, hindering the speed of classifier learning and discriminating processing, especially when learning data changes.

Innovation Solution

A hierarchical neural network apparatus that generates loosely coupled parts using a check matrix of an error correcting code, allowing weights between nodes to be learned independently of the learning data, thereby speeding up classifier learning and discriminating processing.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If pre-learning is performed to form loose couplings in conventional hierarchical neural networks, then the network structure is optimized, but the time and computation amount increase significantly

Engineering Contradiction:

Improvenetwork structure optimizationVSAvoidpre-learning time

Core Design Contradiction:

ReliabilityVSLoss of time

Solution Approach 1:

The patent applies preliminary action by pre-defining the coupling structure between nodes using an error correcting code matrix before actual learning begins. This preliminary structural configuration eliminates the need for time-consuming pre-learning to discover couplings, as the loose coupling pattern is established in advance through the code matrix definition.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The patent changes the parameter of coupling density by using error correcting codes with different rates to create varying degrees of loose couplings. By adjusting the code rate parameter, the system can control the density of connections between nodes, optimizing the balance between computation reduction and learning performance without requiring extensive pre-learning.

Inventive Principle:

Principle #35Parameter changes

2Productivity

If pre-learning is performed to form loose couplings, then computation efficiency is improved, but the computation amount during pre-learning increases

Engineering Contradiction:

Improveclassifier learning speedVSAvoidcomputation amount

Core Design Contradiction:

ProductivityVSUse of energy by moving object

Solution Approach 1:

The patent uses error correcting code matrices as a template or copy that defines the coupling structure. Instead of performing computation-intensive pre-learning to discover optimal couplings, the system copies the efficient sparse structure pattern from the code matrix definition, which requires minimal computation compared to traditional pre-learning approaches.

Inventive Principle:

Principle #26Copying

3Adaptability or versatility

If learning data is altered or corrected, then the model adapts to new data, but pre-learning must be performed again increasing time consumption

Engineering Contradiction:

Improvedata adaptation capabilityVSAvoidrepeated pre-learning time

Core Design Contradiction:

Adaptability or versatilityVSLoss of time

Solution Approach 1:

The patent segments the learning process into two independent parts: (1) defining the coupling structure using error correcting code matrices, which is data-agnostic and can be predefined, and (2) performing weight learning on the already-structured network with new data. This segmentation allows the coupling structure to remain stable while adapting weights to new data, eliminating the need for repeated pre-learning when data changes.

Inventive Principle:

Principle #1Segmentation

Data Source

PatentUS10796223B2Hierarchical neural network apparatus, classifier learning method and discriminating method

Publication Date: 2020.10.06 MITSUBISHI ELECTRIC CORP

AI summary

Generating loosely coupled parts by forming couplings between partial nodes in a hierarchical neural network in accordance with a check matrix of an error correcting code.