Convolutional neural network hierarchical reasoning time prediction method and device

Abstract

The invention discloses a convolutional neural network hierarchical reasoning time prediction method and device. The method comprises the following steps: firstly, collecting hierarchical operator information of various convolutional neural network models, determining various hierarchical types according to the characteristics of the operator information, dividing the operator information into the hierarchical types, and collecting platform framework information; secondly, constructing a feature project, extracting layer feature parameters of the convolutional neural network model corresponding to each hierarchical type, extracting platform framework feature parameters related to reasoning time in platform framework information, and fusing the model layer feature parameters with the platform framework feature parameters to form feature parameters of multiple hierarchical types; and finally, performing reasoning time prediction, classifying a plurality of layering types according to data characteristics of the characteristic parameters, dividing the layering types with the same characteristic parameters into one group, independently dividing the layering types with different characteristic parameters into one group, and constructing a reasoning time prediction model for each group. The method is used for predicting the reasoning time of the convolutional neural network model.

Description

technical field [0001] The present invention relates to the technical field of artificial intelligence, in particular to a convolutional neural network layered reasoning time prediction method and device. Background technique [0002] In recent years, with the explosive growth of data, the substantial improvement of hardware computing power, and the increasing maturity of deep learning algorithms, artificial intelligence technology has ushered in a blowout development. Breakthroughs have been made in a series of fields, and they have great application value in many aspects such as robot applications, industrial manufacturing, and the Internet of Things. At present, most deep learning models use GPUs for large-scale concurrent computing during training to reduce the time of model training; while in the application of the model, the reasoning ability of the deep neural network is used to complete the forward calculation of the network and deploy this calculation Serve applica...

AI Technical Summary

Problems solved by technology

(1) For the inference time prediction problem of the deep learning model, the most critical and challenging step is to screen the feature parameters and perform feature engineering processing. The existing technology considers all the influencing factors and then conducts a unified screening, which will lead to The feature matrix has a large number of zero elements, which makes the result of feature engineering poor;

(2) The existing methods are to establish a prediction model to predict the inference time of each layer of the model, but the types of deep learning model layers are very different, resulting in different factors affecting the inference time of each layer, relying on a The prediction model predicts all layers, resulting in low prediction accuracy for some layers

[0008]In actual application scenarios, the delay (reasoning time) of the trained convolutional neural network model in actual deployment is an important factor in determining whether the image classification model is usable. Index, in the prior art, NAS (Neural Architecture Search, automatic network structure search) is usually used to search the convolutional neural network model. The actual measurement method is time-consuming and labor-intensive, and requires a lot of useless test code to be embedded in the code

On the other hand, in cloud-edge-device collaboration, such as robot applications, Internet of Things, autonomous driving and other scenarios where the computing power of the device is limited and there are many inference tasks, it is usually necessary to obtain the inference time information of each layer of the model to complete Subsequent splitting of the model, due to the large number of layers of the deep learning model and the changeable deployment environment, the reasoning time and workload of the measured model layering is huge

Images