Neural network model pruning method and system based on adaptive batch standardization

A neural network model and adaptive technology, applied in the field of neural network model pruning, can solve the problem of large time consumption, achieve the advantage of accuracy and avoid huge time consumption.

Active Publication Date: 2020-06-02
暗物智能科技(广州)有限公司
View PDF13 Cites 7 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0003] Therefore, a neural network model pruning method and system based on adaptive batch normalization provided by the present invention overcomes the time-consuming defect of the neural network model pruning method in the prior art

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Neural network model pruning method and system based on adaptive batch standardization
  • Neural network model pruning method and system based on adaptive batch standardization
  • Neural network model pruning method and system based on adaptive batch standardization

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0027] A neural network model pruning method based on adaptive batch normalization provided by an embodiment of the present invention, such as figure 2 shown, including the following steps:

[0028] Step S1: Aiming at an L-layer neural network model, randomly sample L floating-point numbers in [0,R] (01 ,r 2 ,...,r L ) as a pruning strategy.

[0029] In the embodiment of the present invention, a convolutional layer with a size of 3×3 of all convolution kernels of MobileNetV1 is used as a prunable layer (L=14), and 14 [0, R] (01 ,r 2 ,...,r L ) as a pruning strategy, each element r in the vector 1 Represents the proportion of the convolution kernel that needs to be reduced in layer l, that is, the pruning rate of each layer. The pruning strategy is reserved to meet the preset computing resource restrictions, including the preset calculation operation limit, the preset parameter limit, the preset At least one of the calculation delay restrictions, for example, the number o...

Embodiment 2

[0049] Embodiments of the present invention provide a neural network model pruning system based on adaptive batch normalization, such as Figure 4 shown, including the following steps:

[0050] The pruning strategy generation module 1 is used to randomly sample L floating-point numbers in [0, R] (01 ,r 2 ,...,r L ) as a pruning strategy; this module executes the method described in step S1 in Embodiment 1, which will not be repeated here.

[0051] The pruning model candidate set generation module 2 is used to perform pruning based on the pruning strategy neural network model respectively, and generate a pruning model candidate set composed of the pruned model; this module executes the description of step S2 in Embodiment 1 method, which will not be repeated here.

[0052] The statistical parameter update module of the batch normalization layer is used to update the statistical parameters of the batch normalization layer by using the adaptive batch normalization method respe...

Embodiment 3

[0056] An embodiment of the present invention provides a computer device, such as Figure 5 As shown, it includes: at least one processor 401 , such as a CPU (Central Processing Unit, central processing unit), at least one communication interface 403 , memory 404 , and at least one communication bus 402 . Wherein, the communication bus 402 is used to realize connection and communication between these components. Wherein, the communication interface 403 may include a display screen (Display) and a keyboard (Keyboard), and the optional communication interface 403 may also include a standard wired interface and a wireless interface. The memory 404 may be a high-speed RAM memory (Ramdom Access Memory, volatile random access memory), or a non-volatile memory (non-volatile memory), such as at least one disk memory. Optionally, the memory 404 may also be at least one storage device located away from the aforementioned processor 401 . The processor 401 may execute the neural network...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a neural network model pruning method and system based on adaptive batch standardization. Randomly sampling the number of floating points to serve as the pruning rate of each layer, generating pruning rate vectors (r1, r2,..., rL) as pruning strategies under the limitation of preset calculation resources, and pruning the model based on the pruning strategies to form a pruning model candidate set; updating statistical parameters of batch standardization layers of the pruning models in the candidate set by using a self-adaptive batch standardization method; evaluating andobtaining the classification accuracy of the neural network model with the updated statistical parameters, and finely adjusting the model with the highest classification accuracy on the training setuntil convergence to serve as a final pruning model. According to the method, the candidate sub-networks are quickly and accurately evaluated by adjusting the batch standardization layer, and the parameters of the final pruning network are obtained by finely adjusting the winning pruning strategy in the quick evaluation method, so that huge time consumption required for finely adjusting all pruning networks is avoided, and meanwhile, the accuracy rate also has the advantage.

Description

technical field [0001] The invention relates to the technical field of neural network model pruning, in particular to a neural network model pruning method and system based on adaptive batch normalization. Background technique [0002] Neural network pruning aims to reduce the computational redundancy of neural networks without losing too much accuracy. The pruned model usually has lower energy consumption and hardware load, so it is of great significance for deployment on embedded devices. However, how to find the least important part of the network to minimize the accuracy loss after pruning is a key issue. The pruning problem of neural network can be regarded as a search problem. Its search space is the set of all pruned sub-networks. Finding the sub-network with the highest accuracy in this space is the core of the pruning problem. Among them, the evaluation process of the sub-network generally exists in the existing pruning methods. This process can reveal the potenti...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06N3/04G06N3/08G06K9/62
CPCG06N3/082G06N3/045G06F18/214Y02D10/00
Inventor 李百林苏江
Owner 暗物智能科技(广州)有限公司
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap