Unlock instant, AI-driven research and patent intelligence for your innovation.

Resource-efficient neural architects

A neural and neural network technology, applied in the field of neural architecture search with limited resources, can solve the problem of laborious design of neural network

Active Publication Date: 2019-11-26
BAIDU USA LLC
View PDF6 Cites 10 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

Despite these advances, designing neural networks remains a laborious task requiring extensive experience and expertise

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Resource-efficient neural architects
  • Resource-efficient neural architects
  • Resource-efficient neural architects

Examples

Experimental program
Comparison scheme
Effect test

Embodiment approach

[0089] In this subsection, the implementation of RENA's overall reinforcement learning (RL) framework and corresponding search space is presented. In one or more implementations, the framework includes a policy network to generate one or more actions that define the neural network architecture. In one or more implementations, the environment outputs the performance of the trained neural network as well as its resource usage. In one or more implementations, the policy network is trained using policy gradients with cumulative rewards.

[0090] 1. Policy Network

[0091] image 3 Depicts a policy network with network embedding 300 according to an embodiment of the disclosure, where a long short-term memory (LSTM) based network converts an existing neural network configuration into a trainable representation, and the trainable representation is fed back to LSTM-based policy network to generate actions. image 3 The implementation of policy network 300 shown in for removal act...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

Neural Architecture Search (NAS) is a laborious process. Prior work on automated NAS targets mainly on improving accuracy but lacked consideration of computational resource use. Presented herein are embodiments of a Resource-Efficient Neural Architect (RENA), an efficient resource-constrained NAS using reinforcement learning with network embedding. RENA embodiments use a policy network to processthe network embeddings to generate new configurations. Example demonstrates of RENA embodiments on image recognition and keyword spotting (KWS) problems are also presented herein. RENA embodiments canfind novel architectures that achieve high performance even with tight resource constraints. For the CIFAR10 dataset, the tested embodiment achieved 2.95% test error when compute intensity is greaterthan 100 FLOPs / byte, and 3.87% test error when model size was less than 3M parameters. For the Google Speech Commands Dataset, the tested RENA embodiment achieved the state-of-the-art accuracy without resource constraints, and it outperformed the optimized architectures with tight resource constraints.

Description

[0001] Cross References to Related Applications [0002] This application claims the benefit of priority to U.S. Provisional Patent Application Serial No. 62 / 673,309 (Docket No. 28888-2233P), entitled "RESOURCE-EFFICIENT NEURALARCHITECT," filed May 18, 2018 , which lists Yanqi Zhou, Siavash Ebrahimi, Sercan Arik, Haonan Yu, and Hairong Liu as inventors. The above patent documents are hereby incorporated by reference in their entirety and for any purpose. technical field [0003] The present disclosure generally relates to systems and methods for computer learning that provide improved computer performance, features and uses. More specifically, the present disclosure relates to implementations for efficient resource-constrained Neural Architecture Search (NAS). Background technique [0004] Deep neural networks have demonstrated excellent performance on challenging research benchmarks, while pushing the frontiers of numerous influential applications such as language transla...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): G06N3/063
CPCG06N3/063G06N3/082G06N3/086G06N3/006G06N5/01G06N3/044G06N3/045G06N3/08
Inventor 周彥祺萨瓦什·阿布拉希米塞尔坎·安瑞克余昊男刘海容格雷戈里·迪莫斯
Owner BAIDU USA LLC