Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Memristor memory neural network training method aiming at memristor error

A technology of neural network training and memristor, applied in neural learning methods, biological neural network models, static memory, etc., can solve the problems of neural network weight deviation, accuracy drop, deviation, etc., to ensure accuracy and accuracy Enhanced effect

Active Publication Date: 2021-10-15
ZHEJIANG UNIV
View PDF3 Cites 4 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0003] However, due to the limitation of the manufacturing process, the manufacturing of memristor memory is almost inevitably affected by the process error and dynamic error, which makes the device parameters in the actual circuit deviate from the target value.
Taking neural network computing as an example, changes in device parameters will cause a certain deviation of the neural network weights stored on the memristor memory compared to its target value. In addition to process errors, dynamic errors such as fluctuations and noise will also affect Weight values ​​affect the accuracy, and these problems are more serious for large-scale memristor memories
The above hardware non-ideal factors make there is still a large gap between the computing acceleration capability of memristor memory and the sharply increased computing requirements of complex algorithms in practical applications.

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Memristor memory neural network training method aiming at memristor error
  • Memristor memory neural network training method aiming at memristor error
  • Memristor memory neural network training method aiming at memristor error

Examples

Experimental program
Comparison scheme
Effect test

example 1

[0065] Example 1: Without loss of generality, build a multi-layer Bayesian convolutional neural network BLeNet and data and an example of MNIST, using python+Pytorch as the software platform of this example.

[0066] The multi-layer Bayesian convolutional neural network BLeNet involved in this example is based on a classic convolutional neural network LeNet. Its structure includes two convolutional layers, two pooling layers and three fully connected layers. The parameters of each layer as follows:

[0067] Convolutional layer Conv1: has 6 convolution kernels of 5*5*1, the input size is 32*32*1 original data set image, and the output size is 28*28*6 feature map;

[0068] Pooling layer P2: Output feature map of 14*14*6, each unit of output feature map is connected to 4 units in a 2*2 neighborhood of the input feature map, whose value is its The values ​​of the corresponding 4 units are averaged;

[0069] Convolution layer Conv3: has 16 convolution kernels of 5*5*6, the input ...

example 2

[0102] Example 2: According to the method similar to Example 1, the Bayesian convolutional neural network BAlexNet is constructed based on the convolutional neural network AlexNet, and the modeling results of process error and dynamic error are used to provide a priori for BAlexNet, using CIFAR-10 and CIFAR-100 data set, Bayesian neural network training, the mean value of the weight posterior distribution obtained from training is used as the weight of the neural network based on the memristor memory, the inference accuracy is tested, and the relative decline factor of the inference accuracy is calculated , compared with the data obtained by the traditional training method, the comparison results are shown in Table 1, which shows that for the complex large-scale neural network model, the application of the present invention for training can also maintain a low The accuracy rate drops, and the optimization effect is obvious compared with the traditional method.

[0103] Table ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a memristor memory neural network training method aiming at memristor errors. The method is mainly used for solving the problem that the inference accuracy of a memristor memory-based neural network is reduced due to process errors and dynamic errors. The method comprises the following steps: modeling a conductance value of the memristor under the influence of a process error and a dynamic error, and converting to obtain the distribution of corresponding neural network weight; constructing prior distribution of the weights by using the weight distribution obtained after modeling, and performing Bayesian neural network training based on variational inference to obtain variational posteriori distribution of the weights; and converting the mean value of the weight variation posteriori into a target conductance value of the memristor memory. According to the method, the influence of the process error and the dynamic error on the neural network calculation based on the memristor memory is weakened, so that the neural network inference based on the memristor memory obtains higher accuracy.

Description

technical field [0001] The invention belongs to the application field of artificial intelligence and memory-computing integrated chips, and specifically relates to a memristor memory neural network training method aimed at memristor errors, which can effectively reduce the accuracy of memory-calculating integrated neural network chips caused by process errors and dynamic errors. degree drops. Background technique [0002] In recent years, great progress has been made in the research on storage and calculation circuits based on memristor memory. The basic unit of this circuit is a non-volatile memory device. The current of the memristor can change its conductance value, and then modify the data stored in the memristor; arrange the memristors in a certain way into a rectangular array, and connect them in the row and column directions to form a memristor storage array. At present, memristor memory is often used in matrix-vector product operations. Taking the fully connected la...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): G06N3/08
CPCG06N3/08G11C11/54G11C13/0007G06N3/084G06N3/065G06N3/047G06N3/0464G11C13/0069
Inventor 卓成尹勋钊黄庆荣高迪
Owner ZHEJIANG UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com