Lightweight deep learning method and device based on random matrix sampling

Abstract

One or more embodiments of the present specification provide a lightweight deep learning method and device based on random matrix sampling, which reduce the model operation time and space complexity and improve the calculation efficiency on the premise of not sacrificing the performance of a neural network. The lightweight deep learning method based on random matrix sampling comprises the following steps: performing dimension reduction processing on a high-dimensional weight matrix of a neural network layer to obtain a low-dimensional representation matrix; in a reasoning stage, adjusting themultiplication sequence between the low-dimensional representation matrixes according to the reasoning calculation complexity; in a training stage, updating the low-dimensional representation matrixaccording to an error matrix; and performing overlapping coupling and re-updating transformation on the updated low-dimensional representation matrix to obtain a lightweight weight matrix. The lightweight deep learning device based on random matrix sampling comprises a low-dimensional weight representation module, a forward reasoning module, a training updating module and a weight calibration module.

Description

technical field [0001] One or more embodiments of this specification relate to the field of computer technology machine learning, and in particular to a random matrix sampling-based lightweight deep learning method and device. Background technique [0002] With the continuous development of deep learning research and the rapid popularization of related applications, it is playing an increasingly important role in social life, economic development, scientific research and other fields, and the most representative technology is deep neural network. As the difficulty of the problem to be solved increases, the structure of the new neural network model becomes more and more complex. Completing the training and reasoning of the deep neural network requires a huge amount of computing resources. In view of this situation, the existing solutions will The deep neural network is configured in the data center or server to facilitate data calculation, and data transmission between the te...

AI Technical Summary

Problems solved by technology

As the difficulty of the problem to be solved increases, the structure of the new neural network model becomes more and more complex. Completing the training and reasoning of the deep neural network requires a huge amount of computing resources. In view of this situation, the existing solutions will The deep neural network is configured in the data center or server to facilitate data calculation, and data transmission between the terminal device and the server, but this method cannot meet the real-time and confidentiality requirements; in another existing solution, the depth Neural network compression, this kind of solution often has the problems of reduced calculation accuracy, slowed calculation speed or increased calculation load, which cannot meet the requirements

Images