Neural Network Architecture Search Under Memory and Energy Constraints
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for selecting neural network architectures for tasks like object detection and classification are inefficient and often fail to meet specific boundary conditions such as memory requirements or energy consumption, especially in applications like vehicle control units.
Innovation Solution
A computer-implemented method that generates candidate architectures using an evolutionary algorithm, modifying known architectures directly within a graph structure to optimize for both boundary conditions and optimization goals, without requiring vectorial representations, ensuring the new paths traverse the entire graph and adhere to specified constraints.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If manual selection of neural network architecture is used, then expert knowledge can be applied, but the process is inefficient and often suboptimal
Solution Approach 1:
The system performs self-service by automatically generating and evaluating candidate architectures through evolutionary algorithms, eliminating the need for manual expert intervention while systematically optimizing architecture selection based on performance metrics and constraints
Solution Approach 2:
The manual mechanical process of expert architecture selection is replaced with an automated computational system that uses evolutionary algorithms to generate, evaluate, and select optimal architectures, transforming a human-centric process into a machine-driven optimization system
2Adaptability or versatility
If vectorial representations are used for architecture search, then numerical optimization becomes possible, but the search space becomes difficult to delineate and not all valid architectures are accessible
Solution Approach 1:
Instead of using abstract vectorial representations, the system creates direct copies of valid architecture graphs through graph copying and modification operations, ensuring that all generated architectures are guaranteed to be valid while maintaining comprehensive search space coverage
Solution Approach 2:
The architecture search space is segmented into discrete graph structures with clearly defined nodes and edges, allowing systematic exploration through graph operations rather than continuous vector space navigation, making the search space both delineatable and fully accessible
3Productivity
If new paths are created arbitrarily in the graph, then diverse architectures can be generated, but there is no guarantee that the path passes completely through the graph
Solution Approach 1:
The system performs preliminary actions by starting with a known valid architecture path and systematically modifying it through graph operations, ensuring that modifications maintain path completeness while generating architectural diversity through controlled transformations
Solution Approach 2:
The system uses feedback mechanisms where each generated architecture is evaluated for validity and performance, with invalid paths being rejected and used to guide subsequent generation attempts, ensuring high reliability of generated architectures while maintaining efficient exploration
4Adaptability or versatility
If evolutionary algorithms are used to optimize multiple goals, then comprehensive optimization is achieved, but the evaluation process becomes more complex
Solution Approach 1:
The system merges multiple evaluation criteria into a unified assessment framework that simultaneously considers accuracy, energy consumption, and memory usage, allowing comprehensive multi-goal optimization through integrated evaluation rather than separate assessment processes
Data Source
AI summary
A computer-implemented method for ascertaining an optimal architecture for a neural network that solves a given task in accordance with given boundary conditions and/or optimization goals. The method includes: providing a graph of the possible architectures of nodes and edges, wherein nodes correspond to data, edges correspond to parameterized operations to be carried out on the data, and a path which traverses the entire graph corresponds to an architecture; in a search phase, generating candidate architectures based on already known architectures, wherein the candidate architectures are similar but not identical to the known architectures in accordance with a predetermined criterion; evaluating the candidate architectures using the given boundary conditions and/or optimization goals; ascertaining a candidate architecture having the best rating as the sought optimal architecture.