Neural Network Configuration Search Optimization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Finding suitable neural network configurations that are optimized for prediction error and additional optimization goals, such as hardware limitations, is time-consuming due to the immense number of possible configurations and the need for extensive training and evaluation.
Innovation Solution
A method to determine a Pareto set of optimized network configurations using a probability distribution for preselecting network configuration candidates, iteratively updating the set based on prediction errors and resource costs, and selecting variants that improve the Pareto front, allowing for a resource-saving approach to find suitable configurations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of time
If the number of candidate network configurations is reduced using expert knowledge, then the search time is reduced, but the quality and optimality of the selected configuration may deteriorate
Solution Approach 1:
The patent applies preliminary action by pre-selecting a subset of promising network configurations from the search space before full training and evaluation. This pre-selection is based on architectural heuristics and performance estimates, allowing the system to focus computational resources on the most promising candidates while still maintaining the possibility of finding high-quality configurations.
Solution Approach 2:
The patent changes parameters by using multiple performance metrics (including accuracy, training time, and model size) to evaluate and filter configurations. By adjusting the weighting and thresholds of these parameters, the system can balance between search efficiency and configuration quality, finding optimal configurations without exhaustively searching the entire space.
2Adaptability or versatility
If multiple optimization goals (prediction error and hardware limitations) are considered simultaneously, then the applicability to practical systems is improved, but the complexity of the search process increases
Solution Approach 1:
The patent segments the optimization process into distinct phases: initial configuration generation, pre-selection based on architectural criteria, training evaluation, and final filtering based on multiple optimization goals. This segmentation allows each phase to focus on specific criteria, managing the overall complexity while achieving multi-objective optimization.
Solution Approach 2:
The patent adds another dimension to the search space by incorporating hardware-specific parameters (model size, training time, computational requirements) alongside traditional accuracy metrics. This multi-dimensional evaluation framework enables simultaneous optimization for both prediction error and hardware limitations without exponentially increasing search complexity.
3Measurement precision
If each candidate network configuration is fully trained and evaluated, then the accuracy of performance assessment is improved, but the computational cost and search time increase significantly
Solution Approach 1:
The patent applies partial action by training and fully evaluating only a subset of the most promising network configurations rather than all possible configurations. This approach performs sufficient training on selected candidates to obtain accurate performance assessments while avoiding the excessive computational cost of evaluating the entire search space.
Solution Approach 2:
The patent uses preliminary filtering based on architectural heuristics and performance estimates to identify promising configurations before committing computational resources to full training. This preliminary action ensures that accurate performance assessment is applied only to configurations with high potential, reducing overall computational cost while maintaining assessment accuracy for the selected candidates.
Data Source
Figure 1~2
Figure 3
Figure 4
AI summary
The invention relates to methods for determining a suitable network configuration for a neural network for a predefined application, which is determined in the form of training data, wherein the network configuration specifies the architecture of the neural network, having the following steps: a) providing (S1) a current network configuration set having network configurations, wherein the current network configuration set corresponds to a Pareto set with regard to a prediction error and at least one further optimisation target; b) providing (S2) a set of network configuration variants; c) selecting network configurations (S3) from the set of network configuration variants, depending on a probability distribution of the network configurations of the current network configuration set with respect to the at least one further optimisation target; d) training (S4) neural nets of each of the selected network configurations and determining a corresponding prediction error (error(N); e) updating (S5) the current network configuration set on the basis of the prediction errors and the at least one further optimisation target of the network configuration set and the selected network configurations; and f) selecting the suitable network configuration from the current network configuration set.