Neural Network Training with Dynamic Neuron Allocation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional neural network training methods, such as backpropagation, face challenges in dynamically adjusting the network structure during training, leading to inefficiencies in learning data sets, particularly in determining the optimal number of neurons and layers, which can result in slow learning or failure to learn complex data sets.
Innovation Solution
The Dynamical Relational (DR) training method allows for dynamic allocation of neurons by adding new neurons to hidden layers or output layers based on the complexity of the data set, enabling learning in a single pass and facilitating the extraction of rules learned during training through propositional logic relationships.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If traditional backpropagation training methods are used with fixed network structure, then training stability is maintained, but the network cannot dynamically adapt to data complexity resulting in slow learning or failure to learn complex data sets
Solution Approach 1:
The patent applies dynamics by enabling the neural network structure to change during training through dynamic addition and removal of neurons based on learning progress and data complexity assessment, resolving the contradiction between structural adaptability and learning efficiency
Solution Approach 2:
The patent uses feedback mechanisms where the network evaluates its own learning performance and data complexity metrics to dynamically adjust network architecture, allowing the system to self-regulate between stability and adaptability during training
2Reliability
If more neurons are added to handle complex data sets, then learning capability improves, but training time and computational resources increase
Solution Approach 1:
The patent dynamically adjusts network size during training by adding neurons only when data complexity metrics indicate the need for increased capacity, rather than pre-defining a large fixed network, thus optimizing the balance between learning capability and training time
Solution Approach 2:
The patent changes network parameters (number of neurons, layer configurations) based on observed learning progress and data characteristics, allowing the network to adapt its capacity to match the actual complexity of the learning task
3Ease of manufacture
If fixed network architecture is used, then training process is simpler, but the network may get stuck in local minima and fail to learn optimal patterns
Solution Approach 1:
The patent introduces dynamic architectural changes during training that allow the network to escape local minima by restructuring itself, while maintaining relatively simple training procedures through automated neuron management based on learning metrics
Data Source
AI summary
The present disclosure provides an artificial neural network communicatively-coupled to at least one computer having one or more processors, including a plurality of neurons arranged in layers. The artificial neural network is arranged to receive a new neuron into a layer of the artificial neural network during training; the new neuron is added to the neural network when no other neuron in that layer for a selected output can learn a relationship associated with an input vector of a data set being learnt. The new neuron is updated with both the relationship which could not be learnt by any other neuron in that layer and a modified data set from a last trained neuron in that layer that contributes to the selected output of the neural network. Methods and computer-readable media are also disclosed.