Neural network model construction method based on pruning and related device

A technology of neural network model and construction method, applied in biological neural network model, neural learning method, neural architecture, etc., can solve problems such as limited applicability, large loss of model accuracy, and difficulty in designing compression optimization methods.

Active Publication Date: 2020-10-23
GUANGDONG UNIV OF TECH
View PDF6 Cites 6 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0004] This application provides a pruning-based neural network model construction method and related devices, which are used to solve the difficulty in designing compression optimization methods for existing neural network models, limited applicability, and large loss of model accuracy after compression technical problem

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Neural network model construction method based on pruning and related device
  • Neural network model construction method based on pruning and related device
  • Neural network model construction method based on pruning and related device

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0052] For ease of understanding, see figure 1 , Embodiment 1 of a pruning-based neural network model construction method provided by the present application, including:

[0053] Step 101. Arrange the initial weight values ​​of the preset neural network models in a preset order to obtain an initial weight value sequence.

[0054] It should be noted that the preset neural network model is a trained neural network model that can be directly applied to actual object recognition, positioning and other scenarios. The weight value of the preset neural network model prepared at this time is defined as the initial weight value, and each initial weight value corresponds to a branch of the preset neural network model. The meaning of pruning is to reduce the number of weight values ​​in the model. The most direct way is to set some weight values ​​to zero; arrange the weight values ​​in a regular order, either in ascending or descending order, and the initial weight value arrangement of...

Embodiment 2

[0066] For ease of understanding, see figure 2 , the present application provides a second embodiment of a pruning-based neural network model construction method, including:

[0067] Step 201, performing pre-training operation on the original neural network model to obtain a preset neural network model.

[0068] It should be noted that the specific method of the pre-training operation is mainly for the specific application object of the neural network model. Through the pre-training operation, the model can learn and memorize the feature points in the specific task, clarify the purpose of the task, etc., and obtain the preset neural network model. The network model can directly perform corresponding tasks. For example, using a neural network model for image recognition, first use a large number of images to pre-train the neural network to obtain a preset neural network model with a certain accuracy, and then use a small amount of test data set, Or input the image of the targ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a neural network model construction method based on pruning and a related device. The method comprises the following steps: sequentially arranging the weight values of the preset neural network model; pruning the preset neural network model according to the number of the total branches of the neural network model and a pruning threshold determined by the weight sequence toobtain a first pruned neural network model, calculating an error function after retraining of the first pruned neural network model to perform Taylor expansion, and calculating to obtain an error change value; and performing cyclic pruning operation on the neural network model by judging the size relationship between the error change value and the error threshold to obtain an optimized target neural network model. The technical problems that an existing compression optimization method of the neural network model is difficult to design and limited in applicability, and the accuracy loss of thecompressed model is large are solved.

Description

technical field [0001] The present application relates to the technical field of neural networks, in particular to a pruning-based neural network model construction method and related devices. Background technique [0002] In recent years, deep learning technology has broken through the shackles of traditional machine learning, and can extract complex and fine features in various types of data, greatly improving the performance of machine learning in various fields, especially the powerful self-encoding ability of convolutional neural networks, making it widely used in Promote the rapid development of artificial intelligence technology in the fields of image recognition, target detection, and semantic segmentation. However, the performance improvement of neural networks is accompanied by the cost of huge storage space and complex calculations, which hinders its application. Due to the large scale and complex calculation of deep convolutional neural network, its application ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06N3/08G06N3/04
CPCG06N3/082G06N3/045
Inventor 李贺罗钊川黄泽茵李世杰黄梓欣吴雁楠青靓李柔仪陈沛琪佘烁康李婕
Owner GUANGDONG UNIV OF TECH
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap