Model deployment method, electronic device, storage medium, and program product

Abstract

This application discloses a model deployment method, electronic device, storage medium, and program product, relating to the field of computer technology. The method includes: splitting the model to be deployed into multiple operator modules; obtaining the capability indicators of a computing power card; determining the operators supported by the computing power card based on the capability indicators; determining the target computing power card corresponding to the operator module based on the supported operators; and deploying the operator module to the target computing power card. This solves the problem of difficulty in distinguishing the capability differences of different computing power cards when deploying a neural network model across multiple computing power cards, leading to an increase in the overall execution time of the model. This method merges related operators into operator modules when splitting the model to be deployed, avoiding time loss due to data transmission. It determines the operators supported by the computing power card, the energy consumption cost and communication cost of the computing power card through capability indicators, and then determines the most suitable target computing power card for executing the operator module. This enables the deployment of the model on multiple computing power cards, reducing model execution time and improving computing power utilization.

Images