Efficiency evaluation methods, devices, and electronic equipment for model inference systems

Abstract

This application discloses an efficiency evaluation method, apparatus, and electronic device for a model inference system, relating to the field of artificial intelligence technology. The method includes: acquiring model parameters corresponding to a target model, framework parameters of the target model's operating environment, and computational parameters of the target model's computational nodes, and determining the target application scenario of the target model; determining a weight set based on the target application scenario; determining standard values ​​corresponding to the model parameters, framework parameters, and computational parameters based on their respective benchmark reference values; and calculating a first score for the target model using the weight set and the standard values ​​corresponding to the model parameters, framework parameters, and computational parameters, wherein the first score indicates the efficiency of the target model inference system. This solves the technical problem in related technologies where large model inference systems cannot achieve global performance evaluation, thus improving the accuracy of inference system performance evaluation.

Images