Neural Network Generation Balancing Accuracy and Model Size
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing neural network generation techniques focus solely on maximizing accuracy, neglecting the resource constraints of smaller devices, leading to computationally expensive and inefficient solutions for mobile-sized architectures.
Innovation Solution
A multi-objective optimization using grammatical evolution is employed to generate neural networks that balance accuracy and size, incorporating domain-specific knowledge and resource constraints, enabling the creation of smaller, easily re-trainable networks.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If existing neural network generation techniques are used to maximize accuracy, then accuracy is improved, but device complexity and computational cost increase
Solution Approach 1:
The patent extracts and applies domain-specific knowledge about resource constraints from mobile devices to the neural network generation process. By incorporating knowledge about limited computational power, memory, and energy constraints into the generation algorithm, the system produces networks optimized for both accuracy and resource efficiency, resolving the contradiction between maximizing accuracy and reducing computational complexity.
Solution Approach 2:
The patent changes the optimization parameters from solely accuracy-maximization to multi-objective optimization that includes accuracy, network size, and resource consumption. The grammatical evolution algorithm adjusts architectural parameters to find optimal trade-offs, producing networks that achieve high accuracy while maintaining low computational complexity suitable for mobile devices.
2Measurement precision
If neural networks are optimized for high accuracy, then accuracy is improved, but the network size increases making it unsuitable for mobile devices
Solution Approach 1:
The patent applies parameter changes by modifying the objective function to simultaneously optimize for accuracy and network size. The grammatical evolution algorithm explores the design space and identifies architectures that achieve high accuracy with compact size, making networks suitable for deployment on mobile devices with limited memory and storage.
Solution Approach 2:
The patent applies local quality by incorporating domain-specific knowledge about mobile device constraints into the generation process. The system produces networks with locally optimized characteristics for mobile deployment, such as efficient layer configurations and parameter distributions that maintain accuracy while reducing overall network size.
3Measurement precision
If traditional neural network generation is used, then accuracy can be maximized, but ease of manufacture and deployment on constrained devices deteriorates
Solution Approach 1:
The patent applies preliminary action by incorporating domain-specific knowledge about resource constraints into the neural network generation process before deployment. The grammatical evolution algorithm pre-optimizes networks for both accuracy and resource efficiency, ensuring that the generated networks are immediately suitable for deployment on mobile devices without requiring subsequent manual optimization or adaptation.
Solution Approach 2:
The patent applies self-service by creating an automated generation system that produces deployment-ready networks. The system self-optimizes for both accuracy and resource constraints, eliminating the need for manual intervention in the deployment process and making it easier to deploy accurate networks on resource-constrained devices.
Data Source
AI summary
A grammar is used in a grammatical evolution of a set of parent neural network models to generate a set of child neural network models. A generation of neural network models is tested based on a set of test data, where the generation includes the set of child neural network models. Respective values for each one of a plurality of attributes are determined for each neural network in the generation, where one of the attributes includes a validation accuracy value determined from the test. Multi-objective optimization is performed based on the values of the plurality of attributes for the generation of neural networks and a subset of the generation of neural network models is selected based on the results of the multi-objective optimization.