Neural network quantification method and device, and electronic device

A technology of neural network and quantization method, applied in the field of neural network quantization method, device and electronic equipment, which can solve problems such as training difficulty, lack of expression ability, and affecting neural network recognition accuracy

Active Publication Date: 2020-05-05
BEIJING KUANGSHI TECH +1
View PDF9 Cites 8 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

However, the low-precision network has fewer parameters, lack of expression ability, and difficult training, which affects the recognition accuracy of the neural network. Therefore, the neural network obtained by the existing network quantization method still has the problem of low recognition accuracy.

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Neural network quantification method and device, and electronic device
  • Neural network quantification method and device, and electronic device
  • Neural network quantification method and device, and electronic device

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0035] First, refer to figure 1 An example electronic device 100 for implementing a neural network quantization method, device and electronic device according to an embodiment of the present invention will be described.

[0036] Such as figure 1 Shown is a schematic structural diagram of an electronic device. The electronic device 100 includes one or more processors 102, one or more storage devices 104, an input device 106, an output device 108, and an image acquisition device 110. These components pass through a bus system 112 and / or other forms of connection mechanisms (not shown). It should be noted that figure 1 The components and structure of the electronic device 100 shown are only exemplary, not limiting, and the electronic device may also have other components and structures as required.

[0037] The processor 102 can be implemented in at least one hardware form of a digital signal processor (DSP), a field programmable gate array (FPGA), and a programmable logic arr...

Embodiment 2

[0044] see figure 2 The flow chart of the neural network quantification method shown, the method can be executed by such as the aforementioned electronic device, in one embodiment, the electronic device can be a processing device (such as a server or computer) configured with a neural network model, the method It mainly includes the following steps S202 to S206:

[0045] Step S202, during the iterative training process of the neural network, determine the initial activation value of each neuron in the input layer based on the input data received by the input layer of the neural network, and determine the scaling factor of each neuron in the input layer based on the initial activation value, Using the scaling factor of each neuron in the input layer, the initial activation value of each neuron in the input layer is quantized and calculated in each output channel of the input layer, and the activation value of each neuron in the next hidden layer of the input layer is output. ...

Embodiment approach 1

[0070] Embodiment 1: In this embodiment, the hidden layer in the neural network includes a fully connected layer.

[0071] Perform the following operations on the intermediate activation values ​​of each neuron in the current layer: perform a global average pooling operation on the intermediate activation values ​​to obtain the pooling operation result; obtain the first floating-point weight of the first fully connected layer in the neural network, Input the pooling operation result and the floating-point weight of the fully connected layer into the preset second nonlinear activation function; obtain the second floating-point weight of the second fully connected layer in the neural network, and use the second nonlinear activation function Input the output result into the preset third nonlinear activation function to obtain the scaling factor of each neuron in the current layer; wherein, the first fully connected layer and the second fully connected layer are any two layers in t...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention provides a neural network quantification method and device, and an electronic device. The method comprises the steps of: in the iterative training process of a neural network, utilizingscaling factors of all neurons in an input layer, performing quantitative calculation on initial activation values of all the neurons in the input layer in all output channels of the input layer, andoutputting activation values of all the neurons in a next hidden layer of the input layer; taking each hidden layer of the neural network as a current layer one by one; and executing the following quantization operation on each current layer: executing the following quantization operation on each current layer, determining a scaling factor of each neuron in the current layer based on the activation value of each neuron in the current layer, performing quantitative calculation on the activation value of each neuron in the current layer in each output channel of the current layer by utilizing the scaling factor of each neuron in the current layer, and outputting the activation value of each neuron in the next layer of the current layer; and when the iterative training is completed, taking the current neural network as a quantized neural network. The recognition precision of the neural network is improved.

Description

technical field [0001] The invention relates to the technical field of machine learning, in particular to a neural network quantization method, device and electronic equipment. Background technique [0002] As the application technology of neural networks in computer vision such as image classification, object detection, and image segmentation matures, the demand for transplanting neural networks to mobile terminals is also increasing. However, the high-performance neural network has the characteristics of a large number of parameters and a large amount of calculation, which makes it difficult to be effectively applied to mobile terminals. In order to reduce the computational complexity of neural networks and alleviate the problem of neural network transplantation, researchers have proposed a variety of neural network compression and acceleration methods, such as the method of quantizing neural networks into low-precision networks, which reduces the calculation time of neura...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06N3/04G06N3/08G06N3/063G06N20/00
CPCG06N3/084G06N3/063G06N20/00G06N3/045Y02D10/00
Inventor 周争光姚聪王鹏陈坤鹏
Owner BEIJING KUANGSHI TECH
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap