Black box robust adversarial evaluation method based on image classification model

Abstract

The invention belongs to the field of intelligent safety, and particularly relates to a black-box robust adversarial evaluation method based on an image classification model. Comprising the following steps: performing classification processing on different types of images by adopting an image classification model to obtain a plurality of different types of image sample sets; processing the plurality of different types of image sample sets to obtain a decision boundary image sample set; converting the decision boundary image sample set into a decision boundary image sample matrix; performing singular value decomposition on the decision boundary image sample matrix to obtain a singular value of the decision boundary image sample matrix; decision boundary complexity is obtained through calculation according to the singular value; performing normalization processing on the decision boundary complexity to obtain a robustness evaluation index; performing robustness evaluation on the image classification model according to the robustness evaluation index to obtain a robustness evaluation result of the image classification model; the method does not need the structure and parameter information of the target model, and greatly improves the safety of the image classification model in the anti-robustness evaluation process.

Description

Technical Field

[0001] The present invention belongs to the field of intelligent security, and in particular relates to a black box adversarial robustness evaluation method based on an image classification model. Background Art

[0002] In recent years, deep neural networks have made great progress in a series of machine learning tasks such as computer vision, speech recognition, and natural language processing. Although deep learning has achieved extraordinary results in many important fields, there are still security issues. Specifically, deep neural networks are susceptible to perturbations that are imperceptible to the human eye, which can mislead image classification models to output incorrect prediction labels. Such samples that are perturbed and misclassified by deep neural networks are called adversarial examples. The existence of adversarial examples poses a great security risk to some sensitive task scenarios with high security requirements.

[0003] As the study of adversarial samples deepens, researchers have found that adversarial samples are widely present in almost all deep learning models, which has led to a large number of studies on adversarial sample generation algorithms. At the same time, how to defend against adversarial attacks and reduce the number of incorrect labels output by image classification models on adversarial samples has also received widespread attention, and researchers have proposed a series of defense measures, such as gradient masking, adversarial training, data processing, and feature compression. Although these defense measures can effectively resist some specific adversarial attacks, there is still a lack of complete adversarial robustness evaluation methods to scientifically and effectively evaluate these defense measures.

[0004] The current adversarial robustness assessment task mainly uses advanced adversarial attack algorithms to continuously test the model against attack and defense, and measures the model's adversarial robustness through basic indicators such as classification accuracy, number of queries, or attack intensity. This type of method mainly relies on the advancement of adversarial attacks, and general adversarial attack methods have varying degrees of conditional restrictions and cannot be applied to all deep neural networks. At the same time, the strength of the attack capability will also greatly affect the evaluation process, resulting in the inability to provide accurate evaluation results. The dependence between adversarial robustness evaluation and attack methods may lead to biased analysis.

[0005] At present, there are two main solutions to the above problems: (1) improve the sophistication of adversarial attack methods and eliminate conditional restrictions through explanatory research on the existence of adversarial samples; (2) separate the adversarial robustness evaluation task from the adversarial attack.

[0006] The decision boundary (DB) is the surface where the output prediction in the model input space changes between different categories. It is the boundary between correct classification and incorrect classification in deep neural network classification tasks. The classification model is trained based on the original data, and the decision boundary that can accurately classify the training samples is determined in the decision space. The nodes on one side of the decision boundary are identified as one category, and the nodes on the other side are identified as another category. For the decision boundary, the attacker can obtain adversarial samples by setting a perturbation. The adversarial perturbation makes the original sample cross the decision boundary of the classification model and causes the model to misclassify. Therefore, the decision boundary becomes a direct reflection of the classification basis of the model. A good decision boundary can improve the generalization ability of the deep neural network model and increase the robustness of the model. The connection between the decision boundary of the model and the adversarial robustness has not been explored to a large extent, which has become a new idea for designing indicators to evaluate the adversarial robustness of the model. Summary of the invention

[0007] In view of the shortcomings of the prior art, the present invention proposes a black box adversarial robustness evaluation method based on an image classification model, the method comprising:

[0008] S1: Use the image classification model to classify images of different categories to obtain multiple image sample sets of different categories;

[0009] S2: Using the decision boundary sample search strategy to process multiple image sample sets of different categories, and obtain a decision boundary image sample set;

[0010] S3: converting the decision boundary image sample set into a decision boundary image sample matrix;

[0011] S4: performing singular value decomposition on the decision boundary image sample matrix to obtain singular values ​​of the decision boundary image sample matrix;

[0012] S5: The decision boundary complexity is calculated based on the singular values ​​of the decision boundary image sample matrix;

[0013] S6: Normalize the complexity of the decision boundary to obtain the robustness evaluation index;

[0014] S7: Perform a robustness evaluation on the image classification model according to the robustness evaluation index to obtain a robustness evaluation result of the image classification model.

[0015] Preferably, in step S2, the process of obtaining the decision boundary image sample set includes:

[0016] S21: Randomly take two image samples x from category i and category j of the image dataset i and x j ;

[0017] S22: Obtain the suspicious decision boundary sample x by averaging the corresponding pixel values ​​of the two image samples db ;

[0018] S23: Determine suspicious decision boundary sample x db Whether the confidence output of satisfies the confidence condition, if so, execute step S24, otherwise, execute step S25;

[0019] S24: The suspicious decision boundary sample x db Determine it as a decision boundary sample and add it to the decision boundary sample set of the i-th and j-th categories;

[0020] S25: Determine f i (x db )>f j (x db ) is satisfied, if so, then the sample x i Update to sample x db ; Otherwise, the sample x j Update to sample x db And return to step S22; wherein, f i (x) represents the confidence output that the image sample x belongs to the i-th category, f j (x) represents the confidence output that the image sample x belongs to the jth category;

[0021] S26: Determine whether the number of samples in the suspicious decision boundary sample set is greater than the set value N. If it is greater than N, the final i-th and j-th decision boundary sample sets are obtained; otherwise, repeat steps S21-S26;

[0022] Furthermore, the confidence condition is expressed as:

[0023] f i (x)-f j (x)≤γ,f i (x)=max k∈C f k (x)

[0024] Among them, C represents the total number of categories in the image dataset, f k (x) represents the confidence output that the image sample x belongs to the kth category, and γ represents the threshold.

[0025] Preferably, the process of converting the decision boundary image sample set into a decision boundary image sample matrix includes:

[0026] Expand the pixel values ​​of N decision boundary samples in the decision boundary sample set into one column to obtain N columns of pixel data; expand the pixels of each decision boundary sample set to obtain N columns of pixel data; splicing N columns of pixel data are used to obtain the decision boundary matrix X, where C is the total number of image categories.

[0027] Preferably, the decision boundary matrix is ​​subjected to singular value decomposition and expressed as:

[0028]

[0029] Where U is the left singular vector matrix of the decision boundary matrix X; Σ is the singular value matrix; V is the right singular value matrix of the decision boundary matrix X; X represents the decision boundary matrix, u nn represents the element in the nth row and nth column of the left singular value matrix, v mm represents the element in the mth row and mth column of the right singular value matrix, {λ1,…,λ r}The singular values ​​of the decision boundary matrix X, r represents the number of singular values.

[0030] Preferably, the formula for calculating the complexity of the decision boundary is:

[0031]

[0032] Among them, λ i represents the i-th singular value, r represents the number of singular values, P(λ) represents the probability of occurrence of the singular value λ, and H(Σ) represents the decision boundary complexity.

[0033] Preferably, the decision boundary complexity is normalized to be expressed as:

[0034]

[0035] Among them, DBSE represents the adversarial robustness evaluation index, H(Σ) represents the decision boundary complexity, and len(λ) represents the number of singular values ​​in the decision boundary matrix.

[0036] Preferably, performing robustness evaluation on the image classification model according to the robustness evaluation index includes: a larger value of the robustness evaluation index indicates that the adversarial robustness of the image classification model is weaker, and a smaller value of the robustness evaluation index indicates that the adversarial robustness of the image classification model is stronger.

[0037] The beneficial effects of the present invention are as follows: in order to solve the problem that adversarial robustness evaluation depends on the adversarial sample generation method, the present invention utilizes the relationship between the decision boundary of the image classification model and the strength of the model adversarial robustness to realize the evaluation of the adversarial robustness of the image classification model, abandons the method of using adversarial samples to conduct adversarial testing on the model in the previous evaluation process, eliminates the influence of adversarial samples of different qualities on the evaluation results, and ensures the consistency and credibility of the evaluation results; the present invention designs a new robustness evaluation indicator, thereby optimizing the entire model robustness evaluation process, which can improve the robustness of the image classification model; in addition, the present invention is suitable for white-box scenarios, does not require the structure and parameter information of the target model, and greatly improves the security of the image classification model adversarial robustness evaluation process. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 It is a structural block diagram of the black box adversarial robustness evaluation method based on the image classification model in the present invention;

[0039] Figure 2 This is a flow chart of the black-box adversarial robustness evaluation method based on the image classification model in the present invention. DETAILED DESCRIPTION

[0040] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0041] This paper proposes a black box adversarial robustness evaluation method based on image classification model, such as Figure 1 , Figure 2 As shown, the method includes:

[0042] S1: Use an image classification model to classify images of different categories to obtain multiple image sample sets of different categories.

[0043] Obtain images of different categories to be classified, and use an image classification model to classify the images of different categories to obtain multiple image sample sets of different categories; store samples of the same category in the same file for subsequent use.

[0044] S2: A decision boundary sample search strategy is used to process multiple image sample sets of different categories to obtain a decision boundary image sample set.

[0045] i, j represent the i-th and j-th categories in the dataset respectively and i∈(0,C-1),j∈(1,C); f i(x) represents the confidence output that the image sample x belongs to the i-th category, f j (x) represents the confidence output that the image sample x belongs to the jth category. By looping through all different category combinations in the data set with different i and j values, the following steps are performed:

[0046] S21: Randomly take two image samples x from category i and category j of the image dataset i and x j ;

[0047] S22: Based on two image samples x i and x j The corresponding pixel values ​​are averaged to obtain the suspicious decision boundary sample x db ; Calculate suspicious decision boundary image samples:

[0048] x db =(x i +x j ) / 2

[0049] S23: Determine suspicious sample x db Whether the confidence condition is met, if so, execute step S24, otherwise, execute step S26;

[0050] The decision boundary sample between any two different categories means that for the classification model f(x) of C categories, there exists a sample x db Belongs to the data set X, its i-th classification confidence and j-th classification confidence f i (x db ), f j (x db ) is the maximum classification confidence of the model max k∈C f k (x db ) are equal, we can think that sample x db Located on the decision boundary. Its mathematical expression is as follows:

[0051]

[0052] Considering the search to obtain the suspicious decision boundary image sample x that meets the above formula db The overhead is too large. The strategy designed by the present invention sets a threshold γ. For samples whose confidence difference is less than the threshold, it is considered to be located near the decision boundary, which can simulate the distribution and structural information of the decision boundary. The mathematical expression is as follows:

[0053]

[0054] Therefore, the confidence condition is:

[0055] fi (x db )-f j (x db )≤γ,f i (x db )=max k∈C f k (x db )

[0056] Among them, f i (x db ) represents the suspicious decision boundary sample x db In the target classification model, the confidence output of the i-th category, γ represents the threshold, which is a minimum value, and C represents the total number of all classification categories.

[0057] Judgment sample x db Whether the confidence condition is met, if so, execute step S24, otherwise, execute step S26.

[0058] S24: The suspicious decision boundary sample x db Determine it as a decision boundary sample and add it to the decision boundary sample set of the i-th and j-th categories;

[0059] S25: Determine f i (x db )>f j (x db ) is satisfied, if so, then the sample x i Update to sample x db ; Otherwise, the sample x j Update to sample x db And return to step S22;

[0060] S26: Determine whether the number of samples in the suspicious decision boundary sample set is greater than the set value N. If it is greater than N, the final i-th and j-th decision boundary sample sets are obtained; otherwise, repeat steps S21-S26;

[0061] S3: Convert the decision boundary image sample set into a decision boundary image sample matrix.

[0062] The process of converting a decision boundary image sample set into a decision boundary image sample matrix includes:

[0063] Expand the pixel values ​​of N decision boundary samples in the decision boundary sample set into one column to obtain N columns of pixel data; expand the pixels of each decision boundary sample set to obtain N columns of pixel data; splicing N columns of pixel data are obtained to obtain the decision boundary matrix X, where C is the total number of image categories;

[0064] S4: Perform singular value decomposition on the decision boundary image sample matrix to obtain singular values ​​of the decision boundary image sample matrix.

[0065] The singular vectors and singular value matrices of the decision boundary image sample matrix can be obtained through singular value decomposition to determine the main change direction of the decision boundary and the weights of each change direction. The singular value decomposition of the decision boundary image sample matrix is ​​expressed as:

[0066]

[0067] Where U is the left singular vector matrix of the decision boundary matrix X; ∑ is the singular value matrix; V is the right singular value matrix of the decision boundary matrix X; X represents the decision boundary matrix, u nn represents the element in the nth row and nth column of the left singular value matrix, v mm represents the element in the mth row and mth column of the right singular value matrix; {λ1,…,λ r}The singular values ​​of the decision boundary matrix X, r represents the number of singular values.

[0068] S5: The decision boundary complexity is calculated based on the singular values ​​of the decision boundary image sample matrix.

[0069] The proportion of the singular values ​​of the decision boundary image sample matrix in the total singular values ​​is taken as the probability of the occurrence of the singular values. The Shannon entropy is used to quantify the uncertainty of the singular values. If the singular value Shannon entropy is small, it means that the decision boundary has fewer changing directions and the spatial structure is simpler; otherwise, it means that the decision boundary has more changing directions and the spatial structure is more complex.

[0070] The formula for calculating the complexity of the decision boundary is:

[0071]

[0072] Among them, represents the i-th singular value, r represents the number of singular values, P(λ) represents the probability of the occurrence of the singular value λ, and H(∑) represents the decision boundary complexity.

[0073] S6: Normalize the decision boundary complexity to obtain the robustness evaluation index. The specific mathematical expression is as follows:

[0074]

[0075] Among them, DBSE represents the adversarial robustness evaluation index, H(∑) represents the decision boundary complexity, and len(λ) represents the number of singular values ​​in the decision boundary matrix.

[0076] S7: Perform a robustness evaluation on the image classification model according to the robustness evaluation index to obtain a robustness evaluation result of the image classification model.

[0077] The image classification model is evaluated for robustness according to the robustness evaluation index. Specifically, the larger the value of the robustness evaluation index, the weaker the adversarial robustness of the image classification model, and the smaller the value of the robustness evaluation index, the stronger the adversarial robustness of the image classification model. When the value of the robustness evaluation index meets the user's requirements, the corresponding image classification model is used to classify the image, which can obtain a more accurate image classification result. The evaluation process of the present invention is applicable to white-box scenarios, does not require the structure and parameter information of the target model, and greatly improves the security of the image classification model adversarial robustness evaluation process.

[0078] The above embodiments further illustrate the purpose, technical solutions and advantages of the present invention in detail. It should be understood that the above embodiments are only preferred implementation modes of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made to the present invention within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A black box adversarial robustness evaluation method based on an image classification model, characterized in that: include: S1: Use the image classification model to classify images of different categories to obtain multiple image sample sets of different categories; S2: Using the decision boundary sample search strategy to process multiple image sample sets of different categories, and obtain a decision boundary image sample set; S3: converting the decision boundary image sample set into a decision boundary image sample matrix; S4: performing singular value decomposition on the decision boundary image sample matrix to obtain singular values ​​of the decision boundary image sample matrix; S5: The decision boundary complexity is calculated based on the singular values ​​of the decision boundary image sample matrix; S6: Normalize the complexity of the decision boundary to obtain the robustness evaluation index; S7: Perform a robustness evaluation on the image classification model according to the robustness evaluation index to obtain a robustness evaluation result of the image classification model.

2. According to the black box adversarial robustness evaluation method based on image classification model in claim 1, it is characterized in that: In step S2, the process of obtaining the decision boundary image sample set includes: S21: Randomly take two image samples x from category i and category j of the image dataset i and x j ; S22: Obtain the suspicious decision boundary sample x by averaging the corresponding pixel values ​​of the two image samples db ; S23: Determine suspicious decision boundary sample x db Whether the confidence output of satisfies the confidence condition, if so, execute step S24, otherwise, execute step S25; S24: The suspicious decision boundary sample x db Determine it as a decision boundary sample and add it to the decision boundary sample set of the i-th and j-th categories; S25: Determine f i (x db )>f j (x db ) is satisfied, if so, then the sample x i Update to sample x db ; Otherwise, the sample x j Update to sample x db And return to step S22; wherein, f i (x) represents the confidence output that the image sample x belongs to the i-th category, f j (x) represents the confidence output that the image sample x belongs to the jth category; S26: Determine whether the number of samples in the suspicious decision boundary sample set is greater than the set value N. If it is greater than N, obtain the final i-th and j-th decision boundary sample sets; otherwise, repeat steps S21-S26.

3. According to claim 2, a black box adversarial robustness evaluation method based on an image classification model is characterized in that: The confidence condition is expressed as: f i (x)-f j (x)≤γ,f i (x)=max k∈C f k (x) Among them, C represents the total number of categories in the image dataset, f k (x) represents the confidence output that the image sample x belongs to the kth category, and γ represents the threshold.

4. According to the black box adversarial robustness evaluation method based on image classification model in claim 1, it is characterized in that: The process of converting a decision boundary image sample set into a decision boundary image sample matrix includes: Expand the pixel values ​​of N decision boundary samples in the decision boundary sample set into one column to obtain N columns of pixel data; expand the pixels of each decision boundary sample set to obtain N columns of pixel data; splicing N columns of pixel data are used to obtain the decision boundary matrix X, where C is the total number of image categories.

5. According to claim 1, a black box adversarial robustness evaluation method based on an image classification model is characterized in that: The singular value decomposition of the decision boundary matrix is ​​expressed as: Where U is the left singular vector matrix of the decision boundary matrix X; Σ is the singular value matrix; V is the right singular value matrix of the decision boundary matrix X; X represents the decision boundary matrix, u nn represents the element in the nth row and nth column of the left singular value matrix, v mm represents the element in the mth row and mth column of the right singular value matrix, {λ1,…,λ r }The singular values ​​of the decision boundary matrix X, r represents the number of singular values.

6. The black box adversarial robustness evaluation method based on an image classification model according to claim 1, characterized in that: The formula for calculating the complexity of the decision boundary is: Among them, λ i represents the i-th singular value, r represents the number of singular values, P(λ) represents the probability of occurrence of the singular value λ, and H(Σ) represents the decision boundary complexity.

7. The black box adversarial robustness evaluation method based on an image classification model according to claim 1, characterized in that: The normalized expression of the decision boundary complexity is: Among them, DBSE represents the adversarial robustness evaluation index, H(∑) represents the decision boundary complexity, and len(λ) represents the number of singular values ​​in the decision boundary matrix.

8. The black box adversarial robustness evaluation method based on an image classification model according to claim 1, characterized in that: The robustness evaluation of the image classification model is performed according to the robustness evaluation index, including: the larger the value of the robustness evaluation index is, the weaker the adversarial robustness of the image classification model is, and the smaller the value of the robustness evaluation index is, the stronger the adversarial robustness of the image classification model is.

Images