Neural network acceleration and compression method based on trace norm constraints

A technology of neural network and compression method, which is applied in the direction of neural learning method, biological neural network model, etc., which can solve the problems of considering the compression ability of the model, reducing the loss of precision, and limited application range, so that the accuracy of the model is not affected and the processing is improved. The effect that the speed and accuracy are not affected

Inactive Publication Date: 2018-04-27
SEETATECH BEIJING TECH CO LTD
View PDF2 Cites 10 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

However, this type of method only designs acceleration and compression methods for the convolutional layer, and has limited versatility for other layers in the neural network, and losing low-frequency information will also bring a relatively large loss of accuracy.
[0004] Althou

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Neural network acceleration and compression method based on trace norm constraints
  • Neural network acceleration and compression method based on trace norm constraints
  • Neural network acceleration and compression method based on trace norm constraints

Examples

Experimental program
Comparison scheme
Effect test

Example Embodiment

[0018] The present invention will be further described in detail below with reference to the drawings and specific embodiments.

[0019] figure 1 The following shows a method for accelerating and compressing neural networks based on trace norm constraints. The specific steps are:

[0020] Step 1. Perform forward propagation of neural network:

[0021] The specific process of forward propagation is to pass the input data through each layer of the network layer by layer, and calculate the parameters of each layer until the output value of the last layer is obtained. By judging whether the error between the network output value and the actual value is less than the specified threshold, judge whether the neural network training has converged. If it does not, proceed to step two, otherwise skip to step five;

[0022] Step 2: Separate the loss function and trace norm constraints related to the task, and perform the loss function backward propagation:

[0023] Obtain the parameters of each l...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a neural network acceleration and compression method based on trace norm constraints, which comprises the following steps: forward propagation of the neural network is carriedout, and whether to converge is judged; a task-related loss function and trace norm constraints are separated, and neural network back propagation based on the loss function is carried out; trace normconstraints are carried out on each layer of parameter matrix after updating of the back propagation; the parameter matrix after the trace norm constraints is updated to the network; and after the training process of the neural network converges, the final parameter matrix is subjected to singular value decomposition, and the neural network is compressed and reconstructed. Through improving the low rank characteristics of model parameters during a model training stage, good acceleration and compression effects are obtained, the applicability is wide, the precision is not influenced, and the needs of reducing storage space and improving the processing speed in practical applications can be met.

Description

technical field [0001] The invention relates to an acceleration and compression method, in particular to a neural network acceleration and compression method based on trace norm constraints, and belongs to the technical field of neural networks. Background technique [0002] A neural network is a feedforward network composed of artificial neurons. A neural network consists of various types of layers, including convolutional layers, fully connected layers, pooling layers, and activation layers. When the input data passes through each layer, it is operated with the parameters of each layer to obtain the output of the neural network. Neural networks are widely used in many computer vision tasks such as target detection, image classification, semantic segmentation, and feature point positioning. When the neural network has tens of thousands of neuron connections, the physical storage requirements of the neural network parameters will be very high. Applications on devices with ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06N3/08
CPCG06N3/08G06N3/084
Inventor 何明捷张杰山世光陈熙霖
Owner SEETATECH BEIJING TECH CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products