Neural network architecture search method based on convolution kernel prediction

A neural network and search method technology, applied in neural architecture, neural learning methods, biological neural network models, etc., can solve problems such as unstable search results and low search efficiency, achieve stable prediction results, reduce computing overhead, and reduce contingencies Effect

Inactive Publication Date: 2020-09-29
SUN YAT SEN UNIV
View PDF0 Cites 11 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0006] The purpose of the present invention is to provide a neural network architecture search method based on convolution kernel prediction to solve the technical problems of low search efficiency and unstable search results in neural network architecture search

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Neural network architecture search method based on convolution kernel prediction
  • Neural network architecture search method based on convolution kernel prediction
  • Neural network architecture search method based on convolution kernel prediction

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0038] Explanation of terms:

[0039] 1. Basic operations in the search space:

[0040](1) Convolution operation: 3×3 depthwise separable convolution (3×3depthwise-separable conv), 5×5 depthwise separable convolution (5×5depthwise-separable conv), 7×7 depthwise separable convolution ( 7×7depthwise-separable conv), 3×3 hole convolution (3×3dilated-separable conv), 5×5 hole convolution (5×5dilated-separable conv), 7×7 hole convolution (7×7 dilated- separable conv);

[0041] (2) Other operations: 3×3 maximum pooling layer (3×3max pooling), 3×3 mean pooling layer (3×3average pooling), direct connection operation (identity), zeroing operation (zero);

[0042] 2. The English full name of the splicing operation (concat) is concatenation;

[0043] 3. Inverted residual block: refers to the Inverted Residual Block mentioned in MobileNetV2;

[0044] 4. ReLU: an activation function commonly used in deep learning;

[0045] 5. FC: fully connected layer, the full name is fully connected...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention provides a neural network architecture search method based on convolution kernel prediction. The method comprises the steps of: constructing a super network for neural network architecture search, wherein the super network comprises a teacher network and a student network, the teacher network is a pre-trained network, and the student network is composed of a plurality of basic units;training the student network by taking a training set and the coding information of a neural network architecture as input, and predicting to obtain an optimal convolution kernel when a loss functionconverges to a minimum value, wherein the loss function is determined according to the loss function of the teacher network and the loss function of the student network; and updating the coding information of the neural network architecture according to the loss function of the student network to obtain an optimal neural network architecture. According to the neural network architecture search method based on convolution kernel prediction, the teacher network is introduced as guidance, and a convolution kernel prediction module in the student network can accurately predict the optimal convolution kernel, so that the search efficiency is greatly improved, and the global optimum of the search result can be ensured.

Description

technical field [0001] The invention relates to the technical field of neural network architecture search, in particular to a neural network architecture search method based on convolution kernel prediction. Background technique [0002] Designing an efficient neural network architecture is an important research content in deep learning. An excellent neural network architecture can effectively improve the efficiency and accuracy of multiple tasks including image classification, detection, and segmentation, but manual design of network architecture is often serious. It relies on human experience, and at the same time creates a huge burden on the relevant researchers. Based on this, Neural Architecture Search (Neural Architecture Search, referred to as NAS) is proposed, and an optimal neural network architecture is obtained by automatically searching the neural network architecture in a predetermined search space. In many fields, such as image classification, image segmentati...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06N3/04G06N3/08
CPCG06N3/084G06N3/045
Inventor 张旭古博陈俊周林梓淇丁北辰韩瑜
Owner SUN YAT SEN UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap