Neural Network Training via Hybrid Data Tunneling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing neural network training algorithms struggle to find the global minimum, resulting in a minimum error threshold that cannot be surpassed, and require a large number of parameters to achieve acceptable accuracy.
Innovation Solution
A method of training a neural network using an artificial training data set where the global minimum is known, iteratively modifying the data set to converge towards the true training data set, and training on a hybrid data set produced as a weighted average between the true and artificial data sets until perfect accuracy is achieved.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If gradient descent method is used for training, then training can be performed with standard algorithms, but the global minimum cannot be found and error rate stabilizes at local minimum
Solution Approach 1:
The patent applies preliminary action by creating an artificial training data set with known global minimum before training the neural network. The artificial data set is constructed by taking correctly trained data elements and cloning incorrectly trained data elements to create a simplified version of the original data set where the global minimum can be reliably found, thereby preparing the training environment in advance to overcome the limitations of standard gradient descent methods
Solution Approach 2:
The patent uses copying by creating a cloned artificial training data set that replicates the structure of the original true training data set but with simplified characteristics. The artificial data set copies the essential patterns and relationships from the original data while removing complexities that prevent gradient descent from finding the global minimum, allowing the neural network to learn the optimal parameters more effectively
2Reliability
If more parameters are used to achieve acceptable accuracy, then training accuracy improves, but the number of parameters becomes unwieldy
Solution Approach 1:
The patent applies parameter changes by modifying the training data set parameters rather than increasing the neural network parameters. By transforming the true training data set into an artificial data set with different statistical properties and known global minimum, the method changes the problem parameters to make the optimization landscape more favorable, thereby achieving high accuracy with fewer parameters
3Ease of operation
If standard training algorithms are used, then training process is simple, but error rate cannot progress beyond local minimum threshold
Solution Approach 1:
The patent introduces an intermediary artificial training data set that mediates between the simple training process and the requirement for low error rate. The artificial data set serves as a bridge that allows standard gradient descent algorithms to converge to the global minimum by providing a simplified optimization landscape, thereby maintaining training simplicity while achieving superior accuracy
Data Source
AI summary
A method of training a neural network comprises executing a neural network training process up to some predefined level of accuracy. The training set is then divided into a set of correctly trained elements and a set of incorrectly trained elements. An auxiliary data set is created from the correctly trained elements and cloned data elements corresponding to the incorrectly trained element. This auxiliary data set is perfectly trained with respect to the trained model for the original data set. A hybrid data set is produced as an average between the original data set and the auxiliary data set. The average is a weighted average according to some weighting parameter. Neural network training then continues to maintain the 100% positive rate from the auxiliary system, iteratively reducing the weighting parameter until the hybrid data set corresponds exactly to the original data set.