Deep learning distributed compiler for cloud edge computing and construction method

Abstract

The invention discloses a deep learning distributed compiler or cloud edge computing and a construction method. The compiler comprises a model compiling framework and a model scheduling framework, wherein the model compiling framework compiles a multi-model task in a distributed manner through a containerized deep learning compiler and a Kubernetes container arrangement system; the model compiling framework can construct a deep learning distributed compiler quickly, solves a large number of sudden compilation requirements by fully utilizing the advantages of a server cluster, and overcomes the defect that the deep learning compiler cannot perform distributed compilation; the model scheduling framework performs resource analysis on a model compiling process to obtain the most efficient resource combination of model operation, and designs a Distributed-DRF scheduling algorithm to guide the scheduling middleware to performing scheduling decision making, so that the fairness of resource allocation and the accuracy of model scheduling are improved.

Description

technical field [0001] The invention belongs to the fields of deep learning compilers and edge computing, and in particular relates to a deep learning distributed compiler and a construction method for cloud edge computing. Background technique [0002] With the development of edge computing, the era of cloud-edge collaboration has arrived. There is a consensus that those companies that enable true intelligence in edge devices and IoT devices will define the future of computing. Deep learning models are widely used on edge devices, such as face recognition on mobile phones and automatic driving in vehicle systems. Through compilation and optimization technology, the deep learning compiler can compile and deploy various deep learning models to edge devices to run efficiently, greatly reducing the inference time of the model. However, the deep learning compiler is a typical calculation-intensive application when compiling and optimizing, which will occupy a large amount of C...

AI Technical Summary

Problems solved by technology

2) The DRF scheduling algorithm is a proof of policy, because users cannot increase allocation by falsely reporting demand

3) DRF is non-envious because no user is willing to exchange its allocation with another user's allocation

4) The DRF allocation is Pareto efficient because it is impossible to improve the allocation of one user without reducing the allocation of another user

Moreover, the DRF scheduling algorithm has not tracked the resource allocation of model tasks in real time, which will lead to inaccurate DS of model tasks, and cannot faithfully reflect the resource demand relationship between model tasks, resulting in unfair resource allocation and model scheduling. unreasonable

In addition, the DRF scheduling algorithm does not track the resource usage of the device in real time, which will cause the resources released after the model is run to not be fully utilized, resulting in a waste of resources

Images