Model evaluation method and apparatus, electronic device, medium, and program product

Abstract

Embodiments of the present application disclose a model evaluation method and device, electronic equipment, medium and program product. The method comprises: obtaining a plurality of evaluation case information, each evaluation case information in the plurality of evaluation case information comprising user intention information and dialogue information; performing dialogue interaction processing on each evaluation case information by a to-be-evaluated model deployed in a preset simulation environment, to generate an interaction result of a current round corresponding to each evaluation case information; in a case where the interaction result represents that the to-be-evaluated model has reached a target execution result corresponding to the user intention information, generating interaction process information of the to-be-evaluated model for the evaluation case information, and obtaining a plurality of interaction process information of the to-be-evaluated model for a plurality of evaluation case information; and performing evaluation processing on the plurality of interaction process information to generate an evaluation result corresponding to the to-be-evaluated model. The model evaluation method provided by the embodiments of the present application can improve the efficiency and accuracy of model evaluation.

Images