Neural Network Architecture Search Using Directed Graph Flow Optimization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The development of optimal artificial neural network architectures is resource-intensive and often requires re-evaluation when specifications or criteria change, making it difficult to determine an optimal architecture that balances performance and resource consumption.
Innovation Solution
A method utilizing a directed graph to represent possible architectures, where nodes symbolize subsets of architectures, edges represent connections, and flows are optimized based on rewards and cost functions to find optimal trajectories, allowing for the determination of optimal architectures across multiple specifications without gradient estimation, thus reducing resource requirements and improving accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Extent of automation
If gradient-based methods are used to ascertain optimal architecture by repeatedly training and evaluating, then the architecture optimization can be automated, but the resource consumption increases significantly
Solution Approach 1:
The patent uses a directed graph to represent the search space of possible architectures, where nodes symbolize architecture subsets and edges represent transitions. This graphical model serves as a simplified copy or abstraction of the complex architecture search problem, enabling optimization without exhaustive training and evaluation of each candidate architecture.
Solution Approach 2:
The patent pre-establishes the directed graph structure representing all possible architecture transitions before beginning the optimization process. By preparing this search space representation in advance, the method avoids the need to repeatedly construct and evaluate full architecture candidates during the optimization iterations, thereby reducing computational resources.
2Adaptability or versatility
If multiple specifications or criteria are taken into account when determining optimal architecture, then the comprehensiveness of optimization improves, but the complexity of determination increases
Solution Approach 1:
The directed graph structure serves as a universal framework that can accommodate multiple specifications and criteria simultaneously. The graph nodes and edges can be weighted or annotated with different criterion values, allowing the same structural representation to handle diverse optimization goals without requiring separate determination processes for each criterion.
Solution Approach 2:
The patent modifies the parameters associated with graph nodes and edges to reflect different specifications and criteria. By changing these parameters rather than changing the fundamental determination structure, the system can adapt to multiple criteria while maintaining the same efficient graph-based optimization approach.
3Adaptability or versatility
If specifications change or new criteria are added, then the optimization can be adapted to new requirements, but complete re-ascertainment of optimal architecture is required
Solution Approach 1:
The directed graph representing the architecture search space is constructed once in advance and can be reused when specifications change. This preliminary structure contains all possible architecture transitions, so when new criteria are introduced, the system only needs to update the graph parameters rather than rebuild the entire search space from scratch, significantly reducing re-evaluation time.
Solution Approach 2:
The patent employs a dynamic approach where the directed graph parameters can be adjusted to reflect changing specifications. The graph structure itself remains stable, but its parameters (weights, costs, rewards) can be dynamically updated to incorporate new criteria, allowing the optimization to adapt without requiring complete re-ascertainment.
Data Source
AI summary
A method for ascertaining an optimal architecture of an artificial neural network. The method includes: respectively ascertaining, for each of at least two specifications for ascertaining the architecture, an optimal architecture with respect to the corresponding specification by associating a flow with each edge of a directed graph, and respectively updating the flows associated with the edges along the trajectory, based on the cost function, wherein the steps are repeated until an ascertained trajectory fulfills a termination criterion for the architecture search, and wherein the trajectory fulfilling the termination criterion represents the optimal architecture; and ascertaining the optimal architecture of the artificial neural network based on the respective optimal architectures with respect to each of the at least two specifications.