Neural Network Architecture Search via Gradient-Based Block Weighting

The architecture search space is segmented into multiple stages, with each stage containing a set of candidate operation blocks. Instead of evaluating complete architectures, the method segments the search into stage-level operations, evaluating only the necessary components at each stage. This segmentation reduces the total number of architectures that need to be evaluated while maintaining search effectiveness.

The method performs partial evaluation by assessing only the operation blocks within each stage rather than evaluating entire architectures. By performing partial actions (evaluating blocks instead of complete architectures) and using gradient-based search to guide the process, the method reduces computing resource consumption while still achieving effective architecture optimization.

The patent replaces reinforcement learning-based search with gradient-based search. Instead of using reinforcement learning agents that require extensive trial-and-error evaluations (mechanical system), the method uses gradient-based optimization to directly search for optimal operation blocks. This substitution maintains automation while dramatically reducing GPU day consumption from 2000+ to a much lower level.

The method changes the search parameters by using gradient-based optimization instead of reinforcement learning. By transforming the discrete architecture search into a continuous optimization problem with differentiable operations, the patent enables automated search with significantly reduced computational requirements, changing the fundamental parameters of how the search is conducted.