Deep learning model reasoning acceleration method based on cooperation of edge server and mobile terminal equipment

Abstract

The invention discloses a deep learning model reasoning acceleration method based on cooperation of an edge server and mobile terminal equipment. Model segmentation and model simplification are combined. Through training and using the regression model, an operation delay of the network layer of a deep learning model on the edge server and on a mobile terminal device is accurately estimated. Therefore, an exit point and the segmentation point which meet the time delay requirement are searched out. Compared with a traditional method based on a cloud data center and a method directly deployed onequipment, the method not only can realize efficient and low-time-delay reasoning of the deep learning model on the mobile terminal equipment, but also can provide a deep learning model reasoning scheme meeting the time delay requirement for trade-off between the time delay and the accuracy.

Description

technical field [0001] The present invention relates to the technical fields of deep learning, edge computing and distributed computing, in particular to a deep learning model reasoning acceleration method based on the collaboration of edge servers and mobile devices. Background technique [0002] As a core technology in machine learning, deep learning models have quickly become the focus of attention in both academia and industry. Deep learning models have been widely used in fields such as computer vision, natural language processing, and speech recognition. The deep learning model for computer vision consists of a series of internally connected deep learning model network layers. The process of obtaining output after the input data is processed by the deep learning model network layer is deep learning model reasoning. The number of network layers of a deep learning model is usually as high as dozens of layers, and the number of parameters reaches millions, so the deep le...

AI Technical Summary

Problems solved by technology

Currently, there are two ways to implement deep learning model reasoning on mobile devices: one is to deploy the deep learning model to the cloud data center, the mobile device sends the input data to the cloud data center, and the cloud reasoning is completed and the result is sent back Mobile devices, however, use cloud data center-based reasoning methods, a large amount of data (such as image and video data) is transmitted to remote cloud data centers through long WAN data, which causes a large end-to-end delay on mobile devices and energy consumption, and due to the delay caused by the long WAN, the performance of the deep learning model reasoning method based on the cloud data center is greatly affected by bandwidth fluctuations, and cannot provide a stable performance; the second is to directly deploy the deep learning model to the mobile However, because deep learning models usually require huge computing and storage overhead, mobile devices cannot provide a good performance

Images