Classification model evaluation method and device

Abstract

The invention relates to a classification model evaluation method and device. The method comprises the steps of obtaining a to-be-tested sample, wherein the to-be-detected sample comprises a normal sample and a corresponding adversarial sample; inputting the to-be-tested sample into the classification model to obtain a prediction result output by the classification model, wherein the prediction result comprises a sample category and a corresponding prediction confidence coefficient; obtaining a confidence coefficient vector of a normal sample and a confidence coefficient vector of an adversarial sample; determining confidence similarity according to respective confidence vectors of the normal sample and the adversarial sample; and obtaining an evaluation result of the classification modelaccording to the confidence similarity. According to the scheme provided by the invention, the problem that the robustness of the classification model to the adversarial sample cannot be evaluated dueto the fact that the internal structure and related parameters cannot be known can be solved.

Description

technical field [0001] The present application relates to the technical field of artificial intelligence, in particular to a classification model evaluation method, device, computer-readable storage medium and computer equipment. Background technique [0002] With the development of artificial intelligence (AI) technology, various classification models have emerged. Based on the classification results output by the classification model, processing such as face recognition, unmanned driving, and illegal image filtering can be realized. However, in practical applications, malicious attacks against classification models have also emerged. One of the malicious attacks is the adversarial sample attack. For example, for a classification model that filters illegal images, criminals add subtle noises that cannot be distinguished by the naked eye to form an adversarial sample image. The picture is predicted as a normal picture without filtering. Thus, the filtering process of the c...

AI Technical Summary

Problems solved by technology

[0006] Based on this, it is necessary to provide an evaluation method, device, computer-readable storage medium, and computer equipment for a classification model for the problem that the robustness of the classification model against adversarial sample interference cannot be effectively evaluated.

Images