A deep learning adversarial sample security evaluation method

Abstract

The application discloses a kind of deep learning adversarial sample security evaluation methods, comprising the following steps: S1. different adversarial sample sets are generated using different adversarial sample generation strategies;S2. each adversarial sample set generated is used in turn to attack the to-be-tested deep learning model, and the attack effectiveness evaluation index is calculated respectively;S3. the to-be-tested deep learning model is equipped with an adversarial sample defense method, the defense effectiveness evaluation index is calculated, and a membership evaluation matrix is constructed;S4. the elements in the membership evaluation matrix are standardized;S5. the contribution degree of the ith adversarial sample set under the jth defense effectiveness evaluation index is calculated;S6. the information entropy and the consistency degree of test results of the jth defense effectiveness evaluation index are calculated;S7. the weight of the jth defense effectiveness evaluation index is calculated;S8. the comprehensive score of adversarial sample security is calculated.The application can comprehensively and objectively evaluate the adversarial sample security of deep learning model.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of deep learning, in particular to a deep learning adversarial sample security evaluation method. BACKGROUND

[0002] In recent years, with the continuous development of deep learning technology, deep learning has been widely used in the analysis and processing of image, text, speech and other data forms. The latest research results show that adversarial examples can easily deceive deep learning models and make them output incorrect results. Attackers can attack deep learning models by generating artificial adversarial examples that are difficult to distinguish, making them unable to work normally, or even produce specific output results that attackers want. Therefore, adversarial sample vulnerability has become one of the main risks of deep learning technology in safety-critical applications.

[0003] In order to study the generation and mechanism of adversarial sample vulnerability, many researchers have proposed a series of methods for attacking deep learning models using adversarial sample vulnerability and defense methods against specific attacks, but there is a lack of unified and comprehensive evaluation standard for the effect of attack and defense, which is not conducive to horizontal comparison between different methods, and it is difficult to obtain comprehensive and objective research conclusions. SUMMARY

[0004] In order to solve the above problems, the present application proposes a deep learning adversarial sample security evaluation method, which can comprehensively and objectively evaluate the security of adversarial samples of deep learning models, and can realize simple calculation, clear hierarchical logic, reasonable weight index, and objective comprehensive evaluation of deep learning adversarial sample security evaluation. The development of deep learning model adversarial sample attack and defense technology has a guiding role.

[0005] The technical scheme adopted by the present application is as follows:

[0006] A deep learning adversarial sample security evaluation method, comprising the following steps:

[0007] S1. Generate different adversarial sample sets using different adversarial sample generation strategies;

[0008] S2. Use each generated adversarial sample set to attack the deep learning model to be tested in turn, and calculate the attack effectiveness evaluation index respectively;

[0009] S3. Equip the deep learning model to be tested with an adversarial sample defense method, calculate the defense effectiveness evaluation index, and generate an n-adversarial sample set corresponding to m-defense effectiveness evaluation index values to form a membership evaluation matrix R=(r ij ) n×m , where r ija value of the jth defense effectiveness evaluation index corresponding to the ith adversarial sample set;

[0010] S4. Standardize the elements in the membership evaluation matrix R, divide each element in the matrix by the maximum value of the elements in its column.

[0011] S5. Calculate the contribution P of the ith adversarial sample set under the jth defense effectiveness evaluation index ij :

[0012]

[0013] S6. Calculate the information entropy E of the jth defense effectiveness evaluation index j :

[0014]

[0015] and calculate the consistency degree d of the test results of each adversarial sample set under the jth defense effectiveness evaluation index j = 1-E j ;

[0016] S7. Calculate the weight w of the jth defense effectiveness evaluation index j :

[0017]

[0018] S8. According to the defense effectiveness evaluation index weight calculated in step S7 and the defense effectiveness evaluation index value calculated in step S3, calculate the comprehensive score of the adversarial sample safety of the deep learning model to be tested equipped with the defense method.

[0019] Further, the attack effectiveness evaluation index includes a model performance decline degree index and an adversarial sample self-identifiability index, the model performance decline degree index is used to describe the decline degree of the performance index of the model on the adversarial sample set compared to the original test set, and the adversarial sample self-identifiability index is used to describe whether the adversarial sample itself is easy to be identified.

[0020] Further, the model performance decline degree index includes two subcategories of classification and regression, respectively for classification and regression deep learning models, the model performance decline degree index for the classification model includes adversarial class average confidence, real class average confidence, misclassification rate and average adversarial loss, and the model performance decline degree index for the regression model includes adversarial prediction average absolute error, adversarial prediction mean square error and average adversarial loss.

[0021] Further, the calculation formula of the adversarial class average confidence is:

[0022]

[0023] where n represents the number of adversarial samples successfully fooling the deep learning model, represents the i-th adversarial sample in the adversarial sample set successfully fooling the deep learning model, represents the prediction result of the model for the sample class, represents the confidence of the model for the classification result;

[0024] The calculation formula of the average confidence of the true class is:

[0025]

[0026] where n represents the number of adversarial samples successfully fooling the deep learning model, represents the i-th adversarial sample in the adversarial sample set successfully fooling the deep learning model, y i represents the true class label of the sample, represents the confidence of the model for the adversarial sample class label y i ;

[0027] The calculation formula of the misclassification rate is:

[0028]

[0029]

[0030] where MR UA For a non-targeted adversarial sample attack, the adversarial sample only needs to make the deep learning model output an incorrect prediction result, MR TA For a targeted adversarial sample attack, the adversarial sample makes the deep learning model identify it as a specific class; N represents the number of adversarial samples in the adversarial sample set, X i a represents the i-th adversarial sample in the adversarial sample set, y i represents the true class label of the sample, represents the specific incorrect class induced by the adversarial sample in the targeted adversarial sample attack and output by the deep learning model;

[0031] The calculation formula of the average absolute error of the adversarial prediction is:

[0032]

[0033] where N represents the number of adversarial samples in the adversarial sample set, represents the i-th adversarial sample in the adversarial sample set, represents the prediction result output by the model for the sample, y i represents the true output corresponding to the sample;

[0034] The calculation formula of the adversarial prediction mean square error is:

[0035]

[0036] In the formula, N represents the number of adversarial samples in the adversarial sample set, represents the i-th adversarial sample in the adversarial sample set, represents the prediction result output by the model for the sample, y i represents the real output corresponding to the sample;

[0037] The calculation formula of the average adversarial loss is:

[0038]

[0039] In the formula, N represents the number of adversarial samples in the adversarial sample set, represents the i-th adversarial sample in the adversarial sample set, y i represents the real class label of the sample, and θ represents the parameter set of the deep learning model, represents the value of the loss function of the deep learning model when the input of the model is .

[0040] Further, the adversarial sample itself identifiable index is divided into three categories of image, text and time sequence according to the difference of the model processing object, wherein the adversarial sample itself identifiable index for image and time sequence both include average L p norm anomaly and perturbation sensitive distance, and the adversarial sample itself identifiable index for text includes Jaccard similarity coefficient and word moving distance.

[0041] Further, the average L p norm anomaly is calculated by the formula:

[0042]

[0043] In the formula, n represents the number of adversarial samples successfully deceiving the deep learning model, represents the i-th adversarial sample successfully deceiving the deep learning model in the adversarial sample set, X i represents the sample before the perturbation is applied corresponding to the adversarial sample, and ||·|| p represents the L p norm of the vector;

[0044] The calculation formula of the perturbation sensitive distance is:

[0045]

[0046] In the formula, n represents the number of adversarial samples successfully deceiving the deep learning model, R(x i,j) represents the feature neighborhood related to the j-th dimension feature in the i-th adversarial sample successfully deceiving the deep learning model, m represents the width of the feature neighborhood, Sen(R(x i,j )) = 1 / std(R(x i,j )), where std(R(x i,j ) represents the standard deviation of the features in the neighborhood; the smaller the perturbation-sensitive distance, the less likely the adversarial sample is to be identified by human;

[0047] The calculation formula of the Jaccard similarity coefficient is:

[0048]

[0049] where n represents the number of adversarial samples successfully deceiving the deep learning model, represents the i-th adversarial sample successfully deceiving the deep learning model in the adversarial sample set, X i represents the sample before the perturbation corresponding to the adversarial sample, the intersection represents the similar words in the adversarial sample and the sample before the perturbation, and the union represents all the words in the adversarial sample and the sample before the perturbation;

[0050] The calculation formula of the word moving distance is:

[0051]

[0052] where n represents the number of adversarial samples successfully deceiving the deep learning model, represents the i-th adversarial sample successfully deceiving the deep learning model in the adversarial sample set, X i represents the sample before the perturbation corresponding to the adversarial sample, x1 and x2 represent different words in the adversarial sample and the sample before the perturbation, respectively, represents the distance calculation weight between different words in the adversarial sample and the sample before the perturbation, and c(x1, x2) represents the Euclidean distance between the two word vectors.

[0053] Further, the defense effectiveness indicator includes an attack effectiveness reduction degree indicator and a model performance negative influence degree indicator, the attack effectiveness reduction degree indicator is used to describe the reduction degree of the attack effectiveness of the adversarial sample on the deep learning model equipped with the defense method, and the model performance negative influence degree is used to describe the decline degree of the performance of the deep learning model equipped with the defense method on the original test set;

[0054] Further, the attack effectiveness reduction degree indicators include two subcategories of classification and regression, respectively for classification and regression deep learning models, the attack effectiveness reduction degree indicators for classification models include the average confidence negative increment of adversarial classes, the average confidence increment of true classes, the negative increment of misclassification rate and the average adversarial loss negative increment indicators, the attack effectiveness reduction degree indicators for regression models include the average absolute error negative increment of adversarial prediction, the average square error negative increment of adversarial prediction and the average adversarial loss negative increment.

[0055] Further, the calculation formula of the average confidence negative increment of adversarial classes is:

[0056] ΔACAC=ACAC(F D ,T AD )-ACAC(F,T AD )

[0057] In the formula, ACAC(F,T AD ) represents the average confidence of adversarial classes of model F on the adversarial sample set T AD , and F D represents the F model with added defense measures;

[0058] The calculation formula of the average confidence increment of true classes is:

[0059] ΔACTC=ACTC(F D ,T AD )-ACTC(F,T AD )

[0060] In the formula, ACTC(F,T AD ) represents the average confidence of true classes of model F on the adversarial sample set T AD , and F D represents the F model with added defense measures;

[0061] The calculation formula of the negative increment of misclassification rate is:

[0062] ΔMR UA =MR UA (F D ,T AD )-MR UA (F,T AD )

[0063] ΔMR TA =MR TA (F D ,T AD )-MR TA (F,T AD )

[0064] In the formula, ΔMR UAFor untargeted adversarial sample attack, MR UA (F,T AD ) denotes the misclassification rate MR AD of model F on adversarial sample set T UA , F D denotes the F model with added defense measures, MR UA (F D ,T AD ) denotes the misclassification rate MR D of model F AD on adversarial sample set T UA ; ΔMR TA For targeted adversarial sample attack, MR TA (F,T AD ) denotes the misclassification rate MR AD of model F on adversarial sample set T TA , MR TA (F D ,T AD ) denotes the misclassification rate MR D of model F AD on adversarial sample set T TA ;

[0065] The calculation formula of the negative increment of the average absolute value error of adversarial prediction is:

[0066] ΔAMAE=AMAE(F D ,T AD )-AMAE(F,T AD )

[0067] Wherein, AMAE(F,T AD ) denotes the average absolute error of adversarial prediction of model F on adversarial sample set T AD , F D denotes the F model with added defense measures;

[0068] The calculation formula of the negative increment of the average square error of adversarial prediction is:

[0069] ΔAMSE=AMSE(F D ,T AD )-AMSE(F,T AD )

[0070] Wherein, AMSE(F,T AD ) denotes the average square error of adversarial prediction of model F on adversarial sample set T AD , F D denotes the F model with added defense measures;

[0071] The calculation formula of the negative increment of the average adversarial loss is:

[0072] ΔL adv = L adv (F D ,T AD )-L adv (F,T AD )

[0073] where L adv (F,T AD ) denotes the average adversarial loss of model F on adversarial sample set T AD , and F D denotes the model F with defense measures added.

[0074] Further, the model performance negative influence degree indicator includes two subcategories of classification and regression, respectively for two types of deep learning models of classification and regression, the model performance negative influence degree indicator for the classification model includes classification accuracy decrease degree, classification confidence decrease degree and classification output stability, and the model performance negative influence degree indicator for the regression model includes average absolute error increase, mean square error increase and determination coefficient.

[0075] Further, the calculation formula of the classification accuracy decrease degree CAV is as follows:

[0076] CAV = Acc(F D ,T) - Acc(F,T)

[0077] where Acc(F,T) denotes the accuracy of model F on test set T, and F D denotes the model F with defense measures added;

[0078] The calculation formula of the classification confidence decrease degree is as follows:

[0079]

[0080] where n denotes the number of samples in the test set that can be correctly classified by both F and F D , X i denotes the i-th sample meeting the above requirement, and P(X i ) and P D (X i ) respectively denote the classification confidence of the sample by model F and F D ;

[0081] The calculation formula of the classification output stability is as follows:

[0082]

[0083] where n denotes the number of samples in the test set that can be correctly classified by both F and F D , X idenotes the i-th sample satisfying the above requirements, P(X i ) and P D (X i ) respectively denote the classification confidence of the sample by the models F and F D , and JSD(P(X i )||P D (X i )) denotes the JS divergence between P(X i ) and P D (X i );

[0084] The calculation formula of the average absolute error increment is:

[0085] MAEV=MAE(F D ,T)-MAE(F,T)

[0086] where MAE(F,T) denotes the average absolute error of the model F on the test set T, F D denotes the F model with added defense measures;

[0087] The calculation formula of the mean square error increment is:

[0088] MSEV=MSE(F D ,T)-MSE(F,T)

[0089] where MSE(F,T) denotes the mean square error of the model F on the test set T, F D denotes the F model with added defense measures;

[0090] The calculation formula of the negative increment of the coefficient of determination is:

[0091]

[0092] where n denotes the number of samples in the test set, X i denotes the i-th sample in the test set, F(X i ) denotes the prediction result output by the model for the sample, F D denotes the F model with added defense measures, and y i denotes the true output corresponding to the sample X i .

[0093] The beneficial effects of the present application are:

[0094] 1. The present application proposes a set of deep learning adversarial sample security evaluation index system, which can uniformly and quantitatively evaluate the attack effect of adversarial samples and the defense effectiveness of adversarial sample defense methods, facilitating researchers to make horizontal comparison between different attack and defense methods.

[0095] 2.The application provides an evaluation method for the security of adversarial samples of a deep learning model based on an entropy weight method, which can comprehensively evaluate the security of adversarial samples of a deep learning model equipped with a specific defense method, thereby recommending a deep learning model adversarial sample defense method with superior comprehensive performance to a user. BRIEF DESCRIPTION OF DRAWINGS

[0096] Figure 1 is a deep learning adversarial sample security evaluation method flowchart according to an embodiment of the application.

[0097] Figure 2 is an attack effectiveness index system architecture diagram according to an embodiment of the application.

[0098] Figure 3 is a defense effectiveness index system architecture diagram according to an embodiment of the application. DETAILED DESCRIPTION

[0099] In order to have a clearer understanding of the technical features, objectives and effects of the application, the specific embodiments of the application will now be described. It should be understood that the specific embodiments described herein are only used to explain the application and do not limit the application, that is, the described embodiments are only a part of the embodiments of the application, but not all the embodiments. Based on the embodiments of the application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of the application.

[0100] The embodiment provides a deep learning adversarial sample security evaluation method, which obtains the weight of a single defense effectiveness evaluation index by using an entropy weight method, and obtains a comprehensive score of the security of adversarial samples of a deep learning model equipped with a specific defense method by weighted calculation, as shown in Figure 1 , specifically comprising the following steps:

[0101] S1. generating different adversarial sample sets by using different adversarial sample generation strategies;

[0102] S2. sequentially attacking the to-be-tested deep learning model by using each generated adversarial sample set to calculate attack effectiveness evaluation indexes;

[0103] S3. equipping the to-be-tested deep learning model with an adversarial sample defense method to calculate defense effectiveness evaluation indexes, and generating an evaluation matrix R=(r ij ) n×m :

[0104]

[0105] wherein, r ijThe value of the jth defense effectiveness evaluation index corresponding to the ith adversarial sample set;

[0106] S4. Standardize the elements in the membership evaluation matrix R, divide each element in the matrix by the maximum value of the elements in the column where the element is located;

[0107] S5. Calculate the contribution degree P of the ith adversarial sample set under the jth defense effectiveness evaluation index ij :

[0108]

[0109] S6. Calculate the information entropy E of the jth defense effectiveness evaluation index j :

[0110]

[0111] and calculate the consistency degree d of the test results of each adversarial sample set under the jth defense effectiveness evaluation index j = 1-E j ;

[0112] S7. Calculate the weight w of the jth defense effectiveness evaluation index j :

[0113]

[0114] S8. According to the defense effectiveness evaluation index weight calculated in step S7 and the defense effectiveness evaluation index value calculated in step S3, calculate the comprehensive score of the adversarial sample security of the deep learning model to be tested equipped with the defense method.

[0115] Meanwhile, the embodiment provides an evaluation index system corresponding to the deep learning adversarial sample security evaluation method, including an attack effectiveness index and a defense effectiveness index. The evaluation object of the attack effectiveness index is an adversarial sample set composed of single attack adversarial samples, which is used to evaluate the effectiveness of single attack. The evaluation object of the defense effectiveness index is an adversarial sample defense method, which is used to evaluate the resistance of the deep learning model to single adversarial sample attack after being equipped with the defense method (or itself).

[0116] As shown in Figure 2 , the attack effectiveness evaluation index includes a model performance decline degree index and an adversarial sample self-identifiability index. The model performance decline degree index is used to describe the decline degree of the performance index of the model on the adversarial sample set compared to the original test set. The adversarial sample self-identifiability index is used to describe whether the adversarial sample itself is easy to be identified.

[0117] The model performance degradation degree index includes two subcategories of classification and regression, covering two core functions of deep learning models. For deep learning models that contain both classification and regression tasks (such as the target detection model Faster R-CNN in the field of machine vision), the effectiveness of the adversarial sample attack can be evaluated by using the model performance degradation degree index of the two subcategories of classification and regression respectively, and then the analysis is summarized.

[0118] The model performance degradation degree index for classification models includes adversarial class average confidence, real class average confidence, misclassification rate, and average adversarial loss. The model performance degradation degree index for regression models includes adversarial prediction mean absolute error, adversarial prediction mean square error, and average adversarial loss.

[0119] Preferably, the calculation method of the model performance degradation degree index is as follows:

[0120] The calculation formula of the adversarial class average confidence is:

[0121]

[0122] In the formula, n represents the number of adversarial samples that successfully deceive the deep learning model, represents the i-th adversarial sample in the adversarial sample set that successfully deceives the deep learning model, represents the prediction result of the model for the sample category, represents the confidence of the model for the classification result;

[0123] The calculation formula of the real class average confidence is:

[0124]

[0125] In the formula, n represents the number of adversarial samples that successfully deceive the deep learning model, represents the i-th adversarial sample in the adversarial sample set that successfully deceives the deep learning model, y i represents the real class label of the sample, represents the confidence of the model for the adversarial sample with the category label y i ;

[0126] The calculation formula of the misclassification rate is:

[0127]

[0128]

[0129] In the formula, MR UA For non-target adversarial sample attacks, the adversarial sample only needs to make the deep learning model output an incorrect prediction result, MR TAFor targeted adversarial sample attack, the adversarial sample makes the deep learning model identify it as a specific class; N represents the number of adversarial samples in the adversarial sample set, represents the i-th adversarial sample in the adversarial sample set, y i represents the real class label of the sample, represents the specific error class induced by the adversarial sample in the targeted adversarial sample attack, which is output by the deep learning model;

[0130] The calculation formula of the adversarial prediction mean absolute error is:

[0131]

[0132] In the formula, N represents the number of adversarial samples in the adversarial sample set, represents the i-th adversarial sample in the adversarial sample set, represents the prediction result output by the model for the sample, y i represents the real output corresponding to the sample;

[0133] The calculation formula of the adversarial prediction mean square error is:

[0134]

[0135] In the formula, N represents the number of adversarial samples in the adversarial sample set, represents the i-th adversarial sample in the adversarial sample set, represents the prediction result output by the model for the sample, y i represents the real output corresponding to the sample;

[0136] The calculation formula of the average adversarial loss is:

[0137]

[0138] In the formula, N represents the number of adversarial samples in the adversarial sample set, represents the i-th adversarial sample in the adversarial sample set, y i represents the real class label of the sample, and θ represents the parameter set of the deep learning model, represents the value of the loss function of the deep learning model when the model input is .

[0139] The adversarial sample itself identifiable index is divided into three categories of image, text and time sequence according to the different processing objects of the model, wherein the adversarial sample itself identifiable index of image and time sequence includes average L p norm anomaly and perturbation sensitive distance, and the adversarial sample itself identifiable index of text includes Jaccard similarity coefficient and word moving distance.

[0140] Preferably, the calculation method of the anti-sample self-identifiability index is as follows:

[0141] Average L p The calculation formula of the norm anomaly is:

[0142]

[0143] In the formula, n represents the number of adversarial samples successfully deceiving the deep learning model, X represents the i-th adversarial sample successfully deceiving the deep learning model in the adversarial sample set, i X represents the sample before the disturbance corresponding to the adversarial sample, and ||·|| represents the L p norm of the vector; p

[0144] The calculation formula of the disturbance-sensitive distance is:

[0145]

[0146] In the formula, n represents the number of adversarial samples successfully deceiving the deep learning model, R(x i,j ) represents the feature neighborhood related to the j-th feature in the i-th adversarial sample successfully deceiving the deep learning model, and m represents the width of the feature neighborhood. i,j Sen(R(x i,j ))=1 / std(R(x i,j )), wherein std(R(x i )) represents the standard deviation of the features in the neighborhood; the smaller the disturbance-sensitive distance, the more difficult it is for the adversarial sample to be artificially identified;

[0147] The calculation formula of the Jaccard similarity coefficient is:

[0148]

[0149] In the formula, n represents the number of adversarial samples successfully deceiving the deep learning model, X represents the i-th adversarial sample successfully deceiving the deep learning model in the adversarial sample set, i X represents the sample before the disturbance corresponding to the adversarial sample, and the intersection X represents the similar words in the adversarial sample and the sample before the disturbance, and the union X represents all the words in the adversarial sample and the sample before the disturbance;

[0150] The calculation formula of the word moving distance is:

[0151]

[0152] In the formula, n represents the number of adversarial samples successfully deceiving the deep learning model, Xi represents an adversarial sample in the adversarial sample set, X i Xi represents an adversarial sample in the adversarial sample set, X Xi represents an adversarial sample in the adversarial sample set, X

[0153] As shown in Figure 3 , the defense effectiveness indicator includes an attack effectiveness reduction degree indicator and a model performance negative influence degree indicator, the attack effectiveness reduction degree indicator is used to describe the reduction degree of the attack effectiveness of the adversarial sample on the deep learning model after the deep learning model is equipped with the defense method, and the model performance negative influence degree is used to describe the decline degree of the performance of the deep learning model on the original test set after the deep learning model is equipped with the defense method.

[0154] The attack effectiveness reduction degree indicator and the model performance decline degree indicator in the attack effectiveness indicator have a corresponding relationship, and like the model performance decline degree indicator in the attack effectiveness indicator, the attack effectiveness reduction degree indicator also includes two subcategories of classification and regression, which are respectively used to evaluate the effectiveness of the defense method for the classification model and the regression model. The attack effectiveness reduction degree indicator for the classification model includes an adversarial class average confidence negative increment, a true class average confidence increment, a misclassification rate negative increment, and an average adversarial loss negative increment indicator, and the attack effectiveness reduction degree indicator for the regression model includes an adversarial prediction average absolute error negative increment, an adversarial prediction mean square error negative increment, and an average adversarial loss negative increment.

[0155] Preferably, the calculation method of the attack effectiveness reduction degree indicator is as follows:

[0156] The calculation formula of the adversarial class average confidence negative increment is:

[0157] ΔACAC=ACAC(F D ,T AD )-ACAC(F,T AD )

[0158] In the formula, ACAC(F,T AD ) represents the adversarial class average confidence of the model F on the adversarial sample set T AD , F D represents the F model increased with the defense measure;

[0159] The calculation formula of the true class average confidence increment is:

[0160] ΔACTC=ACTC(F D ,T AD )-ACTC(F,T AD )

[0161] where ACTC(F, T AD ) denotes the average confidence of the true class of model F on the adversarial sample set T AD , F D denotes the F model with added defense measures;

[0162] The calculation formula of the misclassification rate negative increment is:

[0163] ΔMR UA = MR UA (F D , T AD ) - MR UA (F, T AD )

[0164] ΔMR TA = MR TA (F D , T AD ) - MR TA (F, T AD )

[0165] where ΔMR UA is the misclassification rate negative increment for the no-target adversarial sample attack, MR UA (F, T AD ) denotes the misclassification rate MR AD of model F on the adversarial sample set T UA , F D denotes the F model with added defense measures, MR UA (F D , T AD ) denotes the misclassification rate MR D of model F AD on the adversarial sample set T UA ; ΔMR TA is the misclassification rate negative increment for the targeted adversarial sample attack, MR TA (F, T AD ) denotes the misclassification rate MR AD of model F on the adversarial sample set T TA , MR TA (F D , T AD ) denotes the misclassification rate MR D of model F AD on the adversarial sample set T TA ;

[0166] The calculation formula of the adversarial prediction average absolute value error negative increment is:

[0167] ΔAMAE = AMAE(F D , T AD ) - AMAE(F, T AD )

[0168] AMAE(F, T) represents the average absolute error of the model F on the adversarial sample set T, F represents the model F with added defense measures. AD AD AMAE(F, T) represents the average absolute error of the model F on the adversarial sample set T, F represents the model F with added defense measures. D

[0169] The calculation formula of the negative increment of the adversarial prediction mean square error is as follows:

[0170] ΔAMSE = AMSE(F, T) - AMSE(F, T) D AD AD

[0171] AMSE(F, T) represents the average absolute error of the model F on the adversarial sample set T, F represents the model F with added defense measures. AD AD AMSE(F, T) represents the average absolute error of the model F on the adversarial sample set T, F represents the model F with added defense measures. D

[0172] The calculation formula of the negative increment of the average adversarial loss is as follows:

[0173] ΔL = L(F, T) - L(F, T) adv adv D AD adv AD

[0174] L(F, T) represents the average adversarial loss of the model F on the adversarial sample set T, F represents the model F with added defense measures. adv AD L(F, T) represents the average adversarial loss of the model F on the adversarial sample set T, F represents the model F with added defense measures. AD D

[0175] The model performance negative impact degree index includes two subcategories of classification and regression, and is respectively for two types of deep learning models of classification and regression. The model performance negative impact degree index for the classification model includes classification accuracy reduction degree, classification confidence reduction degree and classification output stability. The model performance negative impact degree index for the regression model includes average absolute error increment, mean square error increment and determination coefficient.

[0176] Preferably, the calculation method of the model performance negative impact degree index is as follows:

[0177] The calculation formula of the classification accuracy reduction degree CAV is as follows:

[0178] CAV = Acc(F, T) - Acc(F, T) D

[0179] ​​​​​​​​​​​​​​​​​where Acc(F, T) represents the accuracy of the model F on the test set T, F D represents the F model with added defense measures;

[0180] The calculation formula of the classification confidence reduction degree is:

[0181]

[0182] where n represents the number of samples in the test set that can be simultaneously classified by F and F D correctly, X i represents the i-th sample meeting the above requirement, P(X i ) and P D (X i ) respectively represent the classification confidence of the sample by the models F and F D ;

[0183] The calculation formula of the classification output stability is:

[0184]

[0185] where n represents the number of samples in the test set that can be simultaneously classified by F and F D correctly, X i represents the i-th sample meeting the above requirement, P(X i ) and P D (X i ) respectively represent the classification confidence of the sample by the models F and F D , and JSD(P(X i )||P D (X i )) represents the JS divergence between P(X i ) and P D (X i );

[0186] The calculation formula of the average absolute error increase is:

[0187] MAEV = MAE(F D , T) - MAE(F, T)

[0188] where MAE(F, T) represents the average absolute error of the model F on the test set T, F D represents the F model with added defense measures;

[0189] The calculation formula of the mean square error increase is:

[0190] MSEV = MSE(F D , T) - MSE(F, T)

[0191] where MSE(F, T) denotes the mean squared error of model F on test set T, F D denotes the F model with added defense measures;

[0192] The formula for calculating the negative increment of the coefficient of determination is:

[0193]

[0194] where n denotes the number of samples in the test set, X i denotes the i-th sample in the test set, F(X i ) denotes the prediction output by the model for this sample, F D denotes the F model with added defense measures, y i denotes the corresponding true output for this sample X i .

[0195] It should be noted that, for the method embodiments described above, the sequence of the acts described for the method embodiments is merely an example and the order of the acts can be changed, and / or two or more acts can be performed at the same time, without departing from the scope of the application. Furthermore, the described acts and / or modules can be combined in a single act and / or module.

Claims

1. A method for evaluating security of deep learning against adversarial samples, characterized in that, The method comprises the following steps: S1. generating different adversarial sample sets by using different adversarial sample generation strategies; S2. sequentially using the generated adversarial sample sets to attack the to-be-tested deep learning model, and respectively calculating attack effectiveness evaluation indexes; S3. Equipping the to-be-tested deep learning model with an adversarial sample defense method, calculating a defense effectiveness evaluation index, generating an adversarial sample set corresponds to the numerical value of the th defense effectiveness evaluation index of the th adversarial sample set forms a membership evaluation matrix . ​ S4. Normalizing the elements in the membership evaluation matrix S4.

1. Divide each element in the matrix by the maximum value of the elements in its column. S5. Calculate the contribution degree of the first defense effectiveness evaluation index to the first adversarial sample set : S6. Calculate the information entropy of the defense effectiveness evaluation index :​ and calculate the consistency degree of the test results of each set of adversarial samples under the first defense effectiveness evaluation index ; S7. Calculate the weight of the defense effectiveness evaluation index :​ S8. calculating the comprehensive score of the adversarial sample safety of the to-be-tested deep learning model equipped with the defense method according to the defense effectiveness evaluation index weight calculated in step S7 and the defense effectiveness evaluation index value calculated in step S3.

2. The deep learning adversarial sample security evaluation method according to claim 1, characterized in that, The attack effectiveness evaluation indexes include a model performance decline degree index and an adversarial sample self-identifiability index, the model performance decline degree index is used to describe the decline degree of the performance index of the model on the adversarial sample set compared with that on the original test set, and the adversarial sample self-identifiability index is used to describe whether the adversarial sample is easy to be identified.

3. The deep learning adversarial sample security evaluation method according to claim 2, characterized in that, The model performance decline degree index includes two subcategories of classification and regression, and the model performance decline degree index for the classification model includes adversarial class average confidence, real class average confidence, misclassification rate and average adversarial loss, and the model performance decline degree index for the regression model includes adversarial prediction average absolute error, adversarial prediction mean square error and average adversarial loss.

4. The deep learning adversarial sample security evaluation method according to claim 3, characterized in that, The calculation formula of the adversarial class average confidence is: In the formula denotes the number of adversarial samples that successfully deceive the deep learning model, denotes the adversarial sample in the adversarial sample set that successfully deceives the deep learning model, denotes the prediction result of the model for the sample category, denotes the confidence of the model for the classification result;​ The calculation formula of the real class average confidence is: wherein denotes the number of adversarial samples that successfully deceive the deep learning model, denotes the adversarial sample in the adversarial sample set that successfully deceives the deep learning model, denotes the true class label of the sample, denotes the confidence of the model that the adversarial sample has a class label of ;​ The calculation formula of the misclassification rate is: In the formula For untargeted adversarial sample attacks, adversarial samples only need to make the deep learning model output wrong prediction results. For targeted adversarial sample attacks, adversarial samples cause deep learning models to identify them as specific categories; represents the number of adversarial samples in the adversarial sample set, represents the first adversarial examples, represents the true category label of the sample, Represents the specific error categories of deep learning model outputs induced by adversarial examples in targeted adversarial example attacks; The calculation formula of the adversarial prediction average absolute error is: In the formula represents the number of adversarial samples in the adversarial sample set, represents the i-th adversarial sample in the adversarial sample set, represents the prediction result output by the model for the sample, represents the real output corresponding to the sample;​ The calculation formula of the adversarial prediction mean square error is: In the formula represents the number of adversarial samples in the adversarial sample set, represents the first adversarial examples, Indicates the model’s prediction result for the sample output. Indicates the true output corresponding to the sample; The calculation formula of the average adversarial loss is: In the formula represents the number of adversarial samples in the adversarial sample set, represents the i-th adversarial sample in the adversarial sample set, represents the true class label of the sample, represents the parameter set of the deep learning model, represents the value of the loss function of the deep learning model when the model input is .​ 5. The method of claim 2, wherein, The self-identifiability indexes of the adversarial samples are divided into three categories of images, texts and time series according to different model processing objects, wherein the self-identifiability indexes of the adversarial samples for images and time series both include average norm variation and perturbation sensitive distance, and the self-identifiability indexes of the adversarial samples for texts include Jaccard similarity coefficient and word moving distance.

6. The deep learning adversarial sample security evaluation method according to claim 5, characterized in that, average The formula for calculating the norm variation is: In the formula denotes the number of adversarial samples that successfully deceive the deep learning model, denotes the adversarial sample in the adversarial sample set that successfully deceives the deep learning model, denotes the sample corresponding to the adversarial sample before the disturbance is applied, denotes the norm of the vector;​ The calculation formula of the perturbation sensitive distance is: In the formula represents the number of adversarial examples that successfully deceive the deep learning model, Indicates the Among the adversarial examples that successfully deceived deep learning models, The feature neighborhood related to the dimensional feature, represents the width of the feature neighborhood, ,in Represents the standard deviation of features within the neighborhood; the smaller the perturbation-sensitive distance, the less likely the adversarial sample is to be manually identified; The calculation formula of the Jaccard similarity coefficient is: wherein denotes the number of adversarial samples that successfully deceive the deep learning model, denotes the adversarial sample in the adversarial sample set that successfully deceives the deep learning model, denotes the sample before the perturbation corresponding to the adversarial sample, the intersection denotes the similar words in the adversarial sample and the sample before the perturbation, the union denotes all the words in the adversarial sample and the sample before the perturbation;​ The calculation formula of the word moving distance is: In the formula represents the number of adversarial examples that successfully deceive the deep learning model, represents the first adversarial examples that successfully deceive deep learning models, Represents the sample before perturbation corresponding to the adversarial sample, and Represent the different words in the adversarial sample and the sample before perturbation, Represents the distance calculation weight between different words in the adversarial sample and the sample before the perturbation, Represents the Euclidean distance between two word vectors.

7. The method of claim 1, wherein, The defense effectiveness index includes an attack effectiveness reduction degree index and a model performance negative influence degree index, the attack effectiveness reduction degree index is used to describe the reduction degree of the attack effectiveness of the adversarial sample on the deep learning model equipped with the defense method, and the model performance negative influence degree is used to describe the decline degree of the performance of the deep learning model equipped with the defense method on the original test set; The attack effectiveness reduction degree index includes two subcategories of classification and regression, and the attack effectiveness reduction degree index for the classification model includes an adversarial class average confidence negative increment, a real class average confidence increment, a misclassification rate negative increment and an average adversarial loss negative increment index, and the attack effectiveness reduction degree index for the regression model includes an adversarial prediction average absolute error negative increment, an adversarial prediction mean square error negative increment and an average adversarial loss negative increment.

8. The deep learning adversarial sample security evaluation method according to claim 7, characterized in that, The calculation formula of the adversarial class average confidence negative increment is: In the formula representing the model on the adversarial sample set adversarial class average confidence, representing the model increased defense measures; The calculation formula of the real class average confidence increment is: In the formula Representation Model In the adversarial sample set The average confidence of the true class on , Indicates increased defensive measures Model; The calculation formula of the misclassification rate negative increment is: wherein for targeted adversarial sample attacks, denotes the model the misclassification rate on the adversarial sample set , denotes the model increased with defense measures, denotes the model the misclassification rate on the adversarial sample set ; for targeted adversarial sample attacks, denotes the model the misclassification rate on the adversarial sample set , denotes the model the misclassification rate on the adversarial sample set ; The calculation formula of the adversarial prediction average absolute value error negative increment is: In the formula representing the model on the adversarial sample set , the average absolute error of adversarial prediction, representing the model increased with defense measures; and the model The calculation formula of the adversarial prediction mean square error negative increment is: wherein representing the model adversarial prediction mean squared error on the adversarial sample set representing the model augmented with defensive measures representing the model augmented with defensive measures​ The calculation formula of the average adversarial loss negative increment is: In the formula Representation Model In the adversarial sample set The average adversarial loss on Indicates increased defensive measures Model.

9. The deep learning adversarial sample security evaluation method according to claim 7, characterized in that, The model performance negative influence degree index includes two subcategories of classification and regression, respectively for two types of deep learning models of classification and regression, the model performance negative influence degree index for the classification model includes classification accuracy decrease degree, classification confidence decrease degree and classification output stability, the model performance negative influence degree index for the regression model includes average absolute error increase, mean square error increase and determination coefficient.

10. The deep learning adversarial sample security evaluation method according to claim 9, characterized in that, degree of decrease in classification accuracy , the calculation formula is: wherein representing the model accuracy on the test set representing the model augmented with defensive measures ​​ The calculation formula of the classification confidence decrease degree is: wherein denotes the number of samples in the test set that can be simultaneously classified with a correct classification, denotes the first sample that meets the above requirement, denotes the number of samples that meet the above requirement, and denote the model's and classification confidence for this sample, respectively. The calculation formula of the classification output stability is: wherein denotes the number of samples in the test set that can be simultaneously classified by both correctly, denotes the first sample that satisfies the above requirement, denotes the first sample that satisfies the above requirement, denotes the first sample that satisfies the above requirement, denotes the first sample that satisfies the above requirement, denotes the first sample that satisfies the above requirement, denotes the first sample that satisfies the above requirement, denotes the first sample that satisfies the above requirement, denotes the first sample that satisfies the above requirement, denotes the first sample that satisfies the above requirement, The calculation formula of the average absolute error increase is: wherein representing the model the mean absolute error, on the test set, representing the model augmented with defensive measures ; The calculation formula of the mean square error increase is: wherein representing the model the mean squared error on the test set , representing the model augmented with defensive measures ; The calculation formula of the determination coefficient negative increase is: wherein denotes the number of samples in the test set, denotes the i-th sample in the test set, denotes the i-th sample in the test set, denotes the prediction output by the model for this sample, denotes the model with added defense, denotes the prediction output by the model with added defense for this sample, denotes the prediction output by the model with added defense for this sample, denotes the corresponding true output for this sample.

Images