Deep learning model compression method based on decision boundary

A technology of deep learning and compression methods, applied in neural learning methods, biological neural network models, neural architectures, etc., can solve problems such as consuming large computing power, and achieve the effect of model compression

Pending Publication Date: 2022-02-11
HARBIN INST OF TECH
View PDF0 Cites 1 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

This method consumes a lot of com

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Deep learning model compression method based on decision boundary
  • Deep learning model compression method based on decision boundary
  • Deep learning model compression method based on decision boundary

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0040] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only some, not all, embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts belong to the protection scope of the present invention.

[0041] The present invention proposes a method for compressing a deep learning model based on a decision boundary, and the method for compressing a deep learning model based on a decision boundary includes the following steps:

[0042] Step 1. Perform feature mapping;

[0043] Step 2, performing piecewise linearization of the activation function;

[0044] Step 3: Calculate the sub-decision area: calculate the sub-decision area of ​​the fully connected layer;

[...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a deep learning model compression method based on decision boundaries, and belongs to the technical field of model compression of deep learning. The deep learning model compression method based on the decision boundary comprises the following steps: step 1, carrying out feature mapping; step 2, performing piecewise linearization of an activation function; step 3, sub-decision region calculation: calculating a sub-decision region of a full connection layer; and step 4, decision network construction: calculating a corresponding decision boundary according to the sub-decision region, and using the decision boundary to construct a new decision network. According to the invention, efficient model compression of a full connection layer is achieved, and for a model with an activation function being piecewise linear, compared with an existing method, precision lossless compression can be achieved. And for other non-linear functions with infinite asymptotes as activation functions, model compression under controllable precision can be realized.

Description

technical field [0001] The invention relates to a deep learning model compression method based on a decision boundary, and belongs to the technical field of deep learning model compression. Background technique [0002] The deep learning model is the core algorithm of the current artificial intelligence technology. It relies on a large amount of labeled data to achieve nonlinear fitting of complex problems through hierarchical modeling. In current practice, deep learning technology has achieved great success in image recognition, speech processing and other fields, and continues to influence other industries. [0003] In order to process complex data, current deep learning models often have hundreds of millions of parameters. In addition to consuming a lot of time and computing resources in the training phase, a large amount of storage resources are also occupied during the deployment and inference of the model and lead to inference. Slow down. In the case of limited compu...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06N3/04G06N3/08
CPCG06N3/084G06N3/048G06N3/045
Inventor 董航程刘国栋刘炳国叶东廖敬骁
Owner HARBIN INST OF TECH
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap