Inference service evaluation method and device, computer equipment and storage medium

Abstract

The invention relates to an inference service evaluation method and device, computer equipment and a storage medium. The method comprises the steps that reasoning data of all reasoning services are collected, a data pool is built through the reasoning data, and the reasoning data are real flow data of a user; obtaining a to-be-evaluated reasoning service, and determining a target evaluation index from a preset evaluation index pool, the target evaluation index at least comprising a main index; acquiring test data from the data pool according to a data driving strategy, and testing the to-be-evaluated reasoning service through the test data to obtain a test result; and calculating an index value of the target evaluation index according to the test result, and evaluating the to-be-evaluated reasoning service through the index value. By adopting the method, the real flow data of the inference service can be collected as the test data, and the inference service to be evaluated is evaluated in an automatic test execution mode.

Images