Neural Network Architecture Evolution via Connection Weight Learning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional architecture selection systems for neural networks are limited in flexibility, often choosing from a space of architectures with the same modules repeatedly, which can lead to suboptimal performance in machine learning tasks like video processing.
Innovation Solution
A system that evolves a population of neural network architectures by selecting parent architectures based on performance metrics, generating new architectures with trained connection weight parameters, and applying mutation operations to optimize block connections and resolutions, enabling more effective data flow and feature combination.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If conventional architecture selection systems use fixed module repetition, then device complexity is reduced, but manufacturing precision deteriorates
Solution Approach 1:
The system dynamically evolves neural network architectures through iterative training and mutation operations, transitioning from static fixed-module designs to dynamic adaptive architectures that optimize performance for specific tasks while managing complexity through guided evolution
Solution Approach 2:
The system changes architectural parameters by training connection weight parameters and applying mutation operations to block connections and resolutions, enabling precise optimization of architecture performance while systematically exploring the architecture space
2Manufacturing precision
If architecture space exploration is expanded, then manufacturing precision improves, but device complexity increases
Solution Approach 1:
The system performs preliminary actions by maintaining a population of architectures and pre-training them before selection, enabling thorough exploration of the architecture space while managing complexity through proactive preparation and systematic evaluation
Solution Approach 2:
The system uses feedback from performance metrics to guide architecture evolution, selecting parent architectures based on their performance and using trained connection weight parameters to inform generation of new architectures, enabling precise optimization while systematically managing the exploration process
Data Source
AI summary
Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for determining one or more neural network architectures of a neural network for performing a video processing neural network task. In one aspect, a method comprises: at each of a plurality of iterations: selecting a parent neural network architecture from a set of neural network architectures; training a neural network having the parent neural network architecture to perform the video processing neural network task, comprising determining trained values of connection weight parameters of the parent neural network architecture; generating a new neural network architecture based at least in part on the trained values of the connection weight parameters of the parent neural network architecture; and adding the new neural network architecture to the set of neural network architectures.