Neural Network Evolution With Accuracy-Size Tradeoff
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing neural network generation techniques focus solely on maximizing accuracy, neglecting the need for smaller, mobile-sized architectures that can be easily re-trained without substantial computational resources, and are computationally expensive.
Innovation Solution
A system utilizing grammatical evolution with multi-objective optimization to generate neural networks that balance accuracy and size, incorporating domain-specific knowledge through grammars and evolving neural networks based on parameters like batch size and learning rate.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If existing neural network generation techniques are used to maximize accuracy, then accuracy is improved, but network size increases and computational cost increases
Solution Approach 1:
The patent applies parameter changes by modifying the objective function to include multiple parameters (accuracy and size) with configurable weights. The system changes the optimization criteria from single-objective (accuracy only) to multi-objective optimization, allowing dynamic adjustment of the balance between accuracy and network size through weight parameters in the fitness function.
Solution Approach 2:
The system introduces dynamics by making the optimization process adaptive through evolutionary algorithms. The neural network architecture evolves dynamically across generations, with the fitness function dynamically adjusting which networks survive based on the weighted combination of accuracy and size metrics. This allows the system to adaptively find optimal trade-offs rather than using fixed criteria.
2Measurement precision
If existing neural network generation techniques are used to maximize accuracy, then accuracy is improved, but computational cost increases
Solution Approach 1:
The patent changes the optimization parameters to include computational cost as a explicit factor in the fitness function. By incorporating size and potentially training cost metrics into the multi-objective optimization, the system directly addresses computational cost concerns while maintaining accuracy through the weighted objective function that balances multiple competing goals.
Solution Approach 2:
The system applies self-service by using evolutionary algorithms that automatically prune and optimize network architectures without requiring manual intervention. The multi-objective fitness function enables the system to self-regulate computational resource usage by naturally selecting networks that achieve acceptable accuracy with lower computational costs, eliminating the need for separate manual optimization steps.
3Ease of manufacture
If neural networks are designed for smaller size, then ease of re-training is improved, but accuracy may deteriorate
Solution Approach 1:
The patent changes the optimization approach by introducing a weighted multi-objective fitness function that simultaneously considers both size (affecting ease of re-training) and accuracy. By adjusting the weights of different objectives, the system can find optimal balance points where networks are compact enough for easy re-training while maintaining sufficient accuracy for the application.
Solution Approach 2:
The system applies partial action by not requiring networks to achieve maximum possible accuracy, but rather sufficient accuracy that meets application requirements. The multi-objective optimization allows networks to achieve 'good enough' accuracy with significantly reduced size, enabling easier re-training without sacrificing the accuracy needed for practical deployment.
Data Source
AI summary
A grammar is used in a grammatical evolution of a set of parent neural network models to generate a set of child neural network models. A generation of neural network models is tested based on a set of test data, where the generation includes the set of child neural network models. Respective values for each one of a plurality of attributes are determined for each neural network in the generation, where one of the attributes includes a validation accuracy value determined from the test. Multi-objective optimization is performed based on the values of the plurality of attributes for the generation of neural networks and a subset of the generation of neural network models is selected based on the results of the multi-objective optimization.