An automatic selection method of sparse tensor storage format based on convolutional neural network

Abstract

The invention discloses a method for automatically selecting a sparse tensor storage format based on a convolutional neural network. The histogram indicates that the matrix is ​​scaled to a fixed size; 3) The fixed size matrix is ​​used as the input of the convolutional neural network CNN, where the structure of the CNN is designed and customized for the automatic selection of the sparse tensor storage format; 4) The method of supervised learning is used Train CNN and get the trained network model; 5) Take the new sparse tensor as the input of the network model, and get the optimal storage format of the tensor after forward propagation. The invention utilizes the advantages of CNN in the classification problem, and combines the feedforward neural network FFNN to adapt the prediction of the optimal sparse tensor storage format, and on the premise of fully retaining the tensor characteristics, the sparse tensor is effectively converted into a fixed tensor. Matrix input of size, which can be applied to automatic selection of sparse format for higher-order tensors under arbitrary tensor computation.

Description

technical field [0001] The invention relates to the fields of convolutional neural network, sparse tensor storage format, tensor calculation, etc. In particular, a method for automatically selecting sparse tensor storage format based on convolutional neural network is designed. Background technique [0002] Tensors generally represent high-dimensional data beyond two dimensions. Multidimensional tensors are widely used in scientific computing, numerical analysis, and machine learning. Since real-world tensors are usually large and very sparse, many existing works optimize the performance of tensor computations based on computational patterns and operational dependencies. Although parallelization can significantly improve the performance of tensor computations, it is still limited by sparsity patterns and hardware characteristics. Therefore, existing work has proposed diverse sparse tensor formats to improve computational performance through co-designed storage and algorith...

AI Technical Summary

Problems solved by technology

The format selection of sparse tensors brings special challenges to convolutional neural networks: 1) In order to adapt to two-dimensional convolutional neural networks, high-dimensional tensors need to be reduced to matrices; 2) the generated matrices after reduction need to be in Scale to a fixed-size network input without losing the sparse pattern as much as possible; 3) Convolutional neural networks need to be redesigned to complement the sparse features lost during tensor conversion

Images