Neural Network Architecture Selection via Algorithm Comparison
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Choosing an initial neural network architecture for a given application is challenging, leading to data scientists approximating network architecture needs and creating numerous networks with different architectures, making it difficult to determine the optimal set of machine learning algorithms.
Innovation Solution
Training a plurality of artificial neural networks with unique architectures comprising hidden layers and machine learning algorithms, comparing their prediction rates to identify the best predictor network, and determining the optimal set of machine learning algorithms used in that network.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If multiple neural networks with different architectures are created to find the optimal machine learning algorithms, then the accuracy of the selected algorithms is improved, but the time and computational resources required for training and comparison increase
Solution Approach 1:
The patent applies preliminary action by pre-training multiple neural networks with different architectures before the actual selection process. The system prepares a diverse set of networks with varying hidden layer configurations, activation functions, and learning algorithms in advance, so that when a selection is needed, the comparison can be performed more efficiently using pre-computed performance metrics rather than training from scratch.
Solution Approach 2:
The system uses partial action by training and comparing only a representative subset of possible neural network architectures rather than exhaustively evaluating every possible configuration. By selecting key variations in architecture and comparing their performance, the system achieves sufficient accuracy without the excessive time cost of evaluating all possible combinations.
2Reliability
If data scientists create numerous neural networks with different architectures to approximate requirements, then the likelihood of finding an optimal architecture is improved, but the complexity of the process increases
Solution Approach 1:
The patent applies segmentation by breaking down the complex task of finding optimal neural network architecture into distinct, manageable segments. The system divides the search space into separate architectural components (hidden layer configurations, activation functions, learning algorithms) and evaluates them systematically. This segmentation allows the complex selection process to be organized into discrete steps, making it more manageable and less overwhelming.
Solution Approach 2:
The system uses parameter changes by systematically varying key architectural parameters of neural networks to explore different configurations. By changing parameters such as the number of hidden layers, neurons per layer, activation function types, and learning algorithm settings, the system efficiently explores the architecture space to find optimal combinations without manually designing each network from scratch.
3Adaptability or versatility
If multiple neural networks are trained and compared to determine optimal algorithms, then the suitability of algorithms for specific applications is improved, but the computational resources required increase
Solution Approach 1:
The patent applies preliminary action by pre-training neural networks with different architectures before deployment. The system prepares a library of pre-trained networks with varying configurations, so that when a specific application requirement arises, the most suitable pre-trained network can be selected and deployed without requiring full retraining, thus reducing computational resource consumption at runtime.
Data Source
AI summary
Approaches presented herein enable determining an optimal set of machine learning algorithms for use in an artificial neural network. More specifically, a plurality of artificial neural networks is trained using a training data set. Each of the plurality of artificial neural networks has a respective unique architecture that comprises a combination of hidden layers, artificial neurons, and machine learning algorithms. Respective prediction rates of each of the plurality of artificial neural networks are compared. A best predictor artificial neural network of the plurality of artificial neural networks is identified, such that the best predictor artificial neural network has a prediction rate which is the most accurate of the respective prediction rates based on the comparing. A set of one or more machine learning algorithms used in the best predictor artificial neural network is determined.