Neural Network Generation Balancing Accuracy and Model Size
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing automated neural network design approaches focus solely on maximizing accuracy, neglecting the need for smaller, mobile-sized architectures that can be easily re-trained without substantial computational resources, and are computationally expensive.
Innovation Solution
A multi-objective optimization using grammatical evolution generates neural networks that balance accuracy and size, incorporating domain-specific knowledge through grammars and evolutionary algorithms to produce efficient, adaptable neural networks.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional automated neural network design approaches are used to maximize accuracy, then accuracy is improved, but the network size and computational cost increase
Solution Approach 1:
The patent applies parameter changes by modifying the objective function to include multiple parameters (accuracy, network size, computational cost) instead of solely maximizing accuracy. The evolutionary algorithm adjusts network parameters to find optimal trade-offs between these competing objectives, generating networks with different accuracy-size profiles.
Solution Approach 2:
The patent introduces dynamics through the evolutionary optimization process that dynamically adjusts network architecture parameters. The system evolves networks iteratively, adapting the network size and complexity dynamically based on the multi-objective criteria, rather than using static fixed-architecture approaches.
2Measurement precision
If traditional automated neural network design approaches are used to maximize accuracy, then accuracy is improved, but computational cost increases
Solution Approach 1:
The patent changes the optimization parameters to include computational cost as a explicit constraint alongside accuracy. The multi-objective evolutionary algorithm balances these parameters by evaluating both accuracy metrics and computational requirements, preventing excessive computational cost while maintaining acceptable accuracy levels.
Solution Approach 2:
The system dynamically adjusts computational resource allocation during the evolutionary process, adapting network complexity based on real-time evaluation of both accuracy and computational cost. This dynamic optimization allows the system to find networks with efficient computational profiles.
3Measurement precision
If larger neural networks are generated to improve accuracy, then accuracy is improved, but ease of re-training decreases
Solution Approach 1:
The patent modifies the network design parameters to prioritize not only accuracy but also re-training ease. The multi-objective framework includes metrics for model size and training complexity, guiding the evolutionary process to generate networks that are easier to re-train while maintaining accuracy performance.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A grammar is used in a grammatical evolution of a set of parent neural network models to generate a set of child neural network models. A generation of neural network models is tested based on a set of test data, where the generation includes the set of child neural network models. Respective values for each one of a plurality of attributes are determined for each neural network in the generation, where one of the attributes includes a validation accuracy value determined from the test. Multi-objective optimization is performed based on the values of the plurality of attributes for the generation of neural networks and a subset of the generation of neural network models is selected based on the results of the multi-objective optimization.