Reusable Convolutional Cell Search for Resource-Efficient CNNs
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing neural network architecture search methods require extensive computational resources and struggle to find architectures that match or exceed the performance of larger search spaces without adversely affecting performance.
Innovation Solution
A system that determines a network architecture for a convolutional neural network by predicting the architecture of convolutional cells and reusing it throughout the network, using a controller neural network to limit the search space and optimize performance on a smaller dataset, allowing for efficient image processing tasks on mobile and embedded platforms.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If existing neural network architecture search methods are used to search large spaces of possible architectures, then the performance of the resulting architecture may be improved, but the computational resources required increase significantly
Solution Approach 1:
The patent segments the neural network architecture search problem into two parts: (1) searching a limited space of convolutional cell architectures, and (2) reusing these cells throughout the network. This segmentation allows the search to focus on the critical cell design while automatically generating full network architectures by repeating these cells, thereby reducing the overall search space and computational resources required
Solution Approach 2:
The patent creates universal convolutional cells that can be reused multiple times throughout the network architecture. By designing cells that serve as building blocks for the entire network, the system reduces the number of unique architectures that need to be evaluated, thereby reducing computational resources while maintaining performance
2Use of energy by moving object
If the search space of possible architectures is limited to reduce computational resources, then the computational resources required decrease, but the performance of the resulting architecture may be adversely affected
Solution Approach 1:
The patent changes the dimension of the search space by focusing on cell-level architecture design rather than searching through all possible full network architectures. By operating at the cell level and then composing full networks from these cells, the system explores a different dimensional space that is both smaller and more effective, achieving good performance with reduced computational resources
3Reliability
If a larger search space is searched to find optimal architectures, then the performance may be improved, but the time required to determine the architecture increases
Solution Approach 1:
The patent segments the architecture search into cell-level optimization, which is computationally cheaper and faster. By optimizing at the cell level and then composing full networks from these optimized cells, the system reduces the time required to determine architectures while maintaining performance through the reusable cell mechanism
Data Source
Figure 1
Figure 2A
Figure 2B
AI summary
Methods, systems, and apparatus, including computer programs encoded on computer storage media, for determining neural network architectures. One of the methods includes generating, using a controller neural network having controller parameters and in accordance with current values of the controller parameters, a batch of output sequences. The method includes, for each output sequence in the batch: generating an instance of a child convolutional neural network (CNN) that includes multiple instances of a first convolutional cell having an architecture defined by the output sequence; training the instance of the child CNN to perform an image processing task; and evaluating a performance of the trained instance of the child CNN on the task to determine a performance metric for the trained instance of the child CNN; and using the performance metrics for the trained instances of the child CNN to adjust current values of the controller parameters of the controller neural network.