Neural Network Architecture Selection Using Expressivity and Trainability
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Selecting a high-performing neural network architecture for a given machine learning problem is challenging, leading to inefficiencies and wasted computational resources.
Innovation Solution
A system and method that computes expressivity and trainability measures for untrained candidate neural networks to select the most suitable architecture, which is then trained to solve the machine learning problem, reducing the need to train less desirable architectures.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If multiple candidate neural network architectures are evaluated by training each one, then the best performing architecture can be selected, but significant computational time and resources are wasted training architectures that will not be selected
Solution Approach 1:
The patent applies preliminary action by computing expressivity and trainability measures for candidate neural network architectures before actual training begins. These pre-computed metrics allow the system to predict which architectures are likely to perform well, enabling selection of promising candidates without investing computational resources in training architectures that are predicted to underperform. This preliminary assessment step filters out poor candidates beforehand, resolving the contradiction between accurate selection and time efficiency.
2Reliability
If traditional trial-and-error methods are used to select neural network architectures, then comprehensive evaluation is achieved, but computational resources are wasted on non-optimal networks
Solution Approach 1:
The patent applies self-service by enabling the neural network architecture selection process to evaluate itself through automated computation of expressivity and trainability metrics. The system autonomously computes these measures for multiple candidate architectures, compares them, and selects the most promising ones without requiring extensive manual trial-and-error training. This self-evaluating mechanism improves reliability while reducing computational resource waste by identifying superior architectures through intrinsic property assessment rather than exhaustive external testing.
Data Source
AI summary
Systems and methods, for selecting a neural network for a machine learning (ML) problem, are disclosed. A method includes accessing an input matrix, and accessing an ML problem space associated with an ML problem and multiple untrained candidate neural networks for solving the ML problem. The method includes computing, for each untrained candidate neural network, at least one expressivity measure capturing an expressivity of the candidate neural network with respect to the ML problem. The method includes computing, for each untrained candidate neural network, at least one trainability measure capturing a trainability of the candidate neural network with respect to the ML problem. The method includes selecting, based on the at least one expressivity measure and the at least one trainability measure, at least one candidate neural network for solving the ML problem. The method includes providing an output representing the selected at least one candidate neural network.