Data processing method and device, chip and computer readable storage medium

Abstract

The embodiment of the invention discloses a data processing method and device, a chip and a computer readable storage medium, which are used for accelerating the operation of a batch of standardized layers in deep learning model training. Multi-dimensional tensor data is stored in a first memory according to a preset rule; taking out in the form of two-dimensional data and carrying out operation;a third matrix is constructed through cooperative use of a plurality of register sets and a second memory; matrix multiplication is performed on the first matrix and the third matrix, so that the element sum and the element quadratic sum of each row in the first matrix can be solved at the same time; parallel calculation of element summation and element quadratic sum calculation is realized, so that calculation related to mean values and variances in a batch standardization layer is accelerated, and the problem of long operation time consumption caused by overlarge data volume in the operationprocess of the batch standardization layer is solved. And finally, the operation speed of batch standardized operation is improved, and the time required for training the whole deep learning model isgreatly shortened.

Description

technical field [0001] The present invention relates to the field of deep learning model training, in particular to a data processing method, device, chip and computer-readable storage medium. Background technique [0002] Deep learning is a new field in machine learning research. The purpose is to establish or simulate the neural network of the human brain for analysis and learning. It imitates the mechanism of the human brain to explain data, such as data such as images, sounds, and texts. Deep learning models need to be trained with a large amount of data before they can be used in practice. Common deep learning models include convolutional neural networks (CNN). [0003] In the training process of the deep learning model, most of them choose to use the batch normalization (Batch Normalization, BN) method to process each layer of the deep learning model, so that the difference of the samples in the process of passing each layer of the network is reduced. Batch normalizat...

AI Technical Summary

Problems solved by technology

[0005] In view of this, the present invention provides a data processing method, device, chip and computer-readable storage medium to solve the problems of long time-consuming calculation of batch normalization layer and slow training speed of deep learning model in deep learning model training

Images