Neural Network Input Order Optimization via Reference Patterns
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Neural networks often suffer from poor generalization and overtraining due to the large number of numerical values in training datasets, leading to inaccurate classification of new data, as the order of input values can significantly affect output values and the abundance of input values complicates determining a proper input order.
Innovation Solution
A machine learning method that determines an optimal input order for numerical values based on a reference pattern, reduces the number of parameters between the input and hidden layers by using product representations of initial parameter values, and updates reference values and parameters to minimize errors, thereby preventing overtraining.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If the number of numerical values in the training dataset is increased, then the classification accuracy on training data is improved, but the generalization ability to new data deteriorates due to overtraining
Solution Approach 1:
The patent segments the input data by determining an optimal input order based on a reference pattern, dividing the numerical values into a predetermined sequence. This segmentation allows the model to process data in a structured manner that prevents overtraining while maintaining accuracy on training data.
Solution Approach 2:
The patent changes the parameter of input order by using a reference pattern to determine the sequence in which numerical values are entered to the neural network. This parameter change transforms the data presentation without altering the data itself, thereby preventing overtraining while maintaining classification accuracy.
2Loss of information
If the number of numerical values in the training dataset is increased, then more information is captured, but the complexity of determining proper input order increases
Solution Approach 1:
The patent applies preliminary action by pre-determining the input order based on a reference pattern before the actual training process. This preliminary ordering simplifies the training process and avoids the complexity of determining optimal input sequences during training, while still capturing important information from the data.
Solution Approach 2:
The patent uses a reference pattern that copies the structural information needed for proper data ordering without requiring complex analysis of the actual data relationships. This copying approach simplifies the determination of input order while maintaining information capture effectiveness.
3Reliability
If the number of parameters between input layer and hidden layer is reduced, then overtraining is prevented, but the classification accuracy may deteriorate
Solution Approach 1:
The patent segments the parameter space by using a reference pattern to determine input order, which effectively reduces the number of free parameters between input layer and hidden layer. This segmentation approach prevents overtraining by constraining the parameter space while maintaining classification accuracy through the structured data presentation.
Data Source
AI summary
A machine learning apparatus calculates second values, based on first values each assigned to one variable value of each term in relation to a neuron in a layer following an input layer of a neural network, where the second values are assigned to variable-value combination patterns in relation to each following-layer neuron. Each second value is represented by a product of first values each assigned to a variable value included in the combination pattern in relation to the following-layer neuron. The apparatus then applies the second values as weights each to a numerical value when it is entered to the corresponding following-layer neuron, to calculate an output value of the neural network with the numerical values arranged in an input order. The apparatus updates reference values in a reference pattern and the first values based on input error that the output value exhibits with respect to training data.