Method and device for adjusting artificial neural network (ANN)

An artificial neural network and neuron technology, applied in neural learning methods, biological neural network models, etc., can solve problems such as failure to recover from incorrect cuts, the impact of compression ratio and model accuracy, and poor convergence of network models.

Inactive Publication Date: 2017-05-31
BEIJING DEEPHI INTELLIGENT TECH CO LTD
View PDF0 Cites 44 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

However, in the existing compression process, when retraining the network with reduced accuracy after pruning, only the size of the weights that remain in the model matrix will be adjusted, so it is impossible to recover from wrong pruning.
This may cause the network model to converge to a poor local optimum, which will affect the compression rate and model accuracy

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Method and device for adjusting artificial neural network (ANN)
  • Method and device for adjusting artificial neural network (ANN)
  • Method and device for adjusting artificial neural network (ANN)

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0025] Preferred embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although preferred embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

[0026] In this application, the improvement of the present invention to ANN will be explained mainly by taking the Long Short Term Memory (LSTM) model for speech recognition as an example. The scheme of the present application is applicable to various artificial neural networks, including deep neural network (DNN), recurrent neural network (RNN) and convolutional neural network (CNN), especially applicable to the above-mentioned LSTM model w...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a method and a device for adjusting an artificial neural network (ANN). The ANN comprises multiple neurons, and the connection relationship among the neurons is presented through a connection weight matrix. The method comprises a pruning step in which n unimportant weights in all N weights of a first trained connection weight matrix are set to be zero, a step of retraining without a mask in which a second connection weight matrix through pruning is retrained in a condition in which any weight is not forced to be constrained to be zero, a step of mask generation in which a matrix-shaped mask is generated according to a third connection weight matrix through the retraining without the mask and a step of retraining with the mask in which the mask matrix is used for retraining the third connection weight matrix. Thus, the mask is dynamically adjusted through adding the step of generating the mask via the matrix without mask retraining in a retraining stage, error pruning in the pruning process is corrected and restored, and the performance of the compressed neural network is enhanced.

Description

technical field [0001] The present invention designs an artificial neural network (ANN), such as a recurrent neural network (RNN), and particularly relates to dynamic adjustment of the neural network based on a mask. Background technique [0002] Artificial neural network (ANN), referred to as neural network (NNs), is a mathematical computing model that imitates the behavioral characteristics of animal neural networks and performs distributed parallel information processing. In recent years, neural networks have developed rapidly and are widely used in many fields, such as image recognition, speech recognition, natural language processing, weather forecast, gene expression, content push and so on. [0003] In a neural network there are a large number of interconnected nodes called "neurons". Each neuron computes weighted input values ​​from other neighboring neurons through a specific output function. The intensity of information transmission between neurons is defined by ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06N3/08
CPCG06N3/082
Inventor 姚颂
Owner BEIJING DEEPHI INTELLIGENT TECH CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap