A method and device for detecting adversarial samples based on randomness and redundancy mechanism

Abstract

The present invention provides a method and device for detecting adversarial samples based on randomness and redundancy mechanism. The method includes: randomly masking the original image, reconstructing the image based on the remaining pixel information after masking; repeating the previous step until the set number of reconstructions is met. N ; The original image and N The reconstructed image is input into the target artificial intelligence model for prediction; judgment N Whether the number of inconsistencies in the predicted labels of the reconstructed image exceeds the first set threshold, if so, it is marked as an adversarial sample; otherwise, proceed to the next step; calculate N The average predicted probability of each reconstructed image belonging to each type of label; if the maximum value of all the predicted probability averages is lower than the second set threshold, it is marked as an adversarial sample; otherwise, proceed to the next step; judge N The average predicted label of the reconstructed images is consistent with the predicted label of the original image. If not, it is marked as an adversarial sample; otherwise, the original image is marked as a benign sample.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence security technology, and in particular to a method and device for detecting adversarial samples based on randomness and redundancy mechanisms. Background Art

[0002] In recent years, with the widespread application of artificial intelligence (AI) technology, its inherent security issues have gradually become a focus of attention. Adversarial examples, which can bypass AI model recognition, seriously affect their normal operation. To proactively detect and respond to adversarial examples, relevant detection technologies have emerged.

[0003] Traditional adversarial example detection methods often rely on learning or comparing features from existing adversarial examples. While these methods can accurately identify adversarial examples of the type and strength they learn, they lack scalability. Furthermore, since they largely rely on the structure and parameters of the target model, they require appropriate adjustments to the environment. Another, more decoupled detection method uses data processing techniques to transform input data, allowing detection based on comparisons of the results without relying on prior knowledge of the model or attack. However, because this method is strongly dependent on the underlying technology and is more general than previous methods, its detection effectiveness and reliability are lower. Summary of the Invention

[0004] In order to retain the transferability of the detector while increasing its accuracy and reliability, the present invention proposes a method and device for detecting adversarial samples based on randomness and redundancy mechanisms.

[0005] In one aspect, the present invention provides a method for detecting adversarial examples based on randomness and redundancy mechanisms, comprising:

[0006] Step 1: Randomly mask the pixel blocks of the original image according to the set masking ratio, and reconstruct the image based on the remaining pixel information after masking;

[0007] Step 2: Repeat step 1 until the set number of rebuilds is met N ,get N A reconstructed image;

[0008] Step 3: Combine the original image and N The reconstructed image is input into the target artificial intelligence model for prediction, and the prediction result of the original image and N The prediction results of the reconstructed image are obtained, and each prediction result retains the prediction probability of all class labels, among which the class label corresponding to the maximum prediction probability is the predicted label;

[0009] Step 4: NThe predicted labels of the reconstructed images are compared for consistency. If the number of inconsistencies in the predicted labels exceeds the first set threshold, the original image is directly marked as an adversarial sample; otherwise, step 5 is continued:

[0010] Step 5: For each class label, calculate N The average predicted probability of each reconstructed image belonging to each class of labels; if the maximum value of all the predicted probability averages is lower than the second set threshold, the original image is marked as an adversarial sample; otherwise, continue to step 6;

[0011] Step 6: N The average predicted label of the reconstructed images is compared with the predicted label of the original image for consistency. If they are inconsistent, the original image is marked as an adversarial sample; otherwise, the original image is marked as a benign sample. Among them, the class label corresponding to the maximum value of all the average prediction probabilities is the average predicted label.

[0012] Furthermore, before step 1, the method further includes:

[0013] Performing data preprocessing on the original image, wherein the data preprocessing includes at least one of denoising, adding noise, color conversion, and size conversion;

[0014] Correspondingly, step 1 specifically includes: randomly masking pixel blocks of the original image after data preprocessing according to a set masking ratio.

[0015] Furthermore, after step 2, the method further includes:

[0016] For the original image and N The reconstructed image is subjected to data preprocessing; the data preprocessing includes at least one of denoising, denoising, color conversion and size conversion;

[0017] Correspondingly, in step 3, the steps include: N The reconstructed image is input into the target artificial intelligence model for prediction.

[0018] Furthermore, before step 6, the method further includes:

[0019] Use multiple existing adversarial sample detection methods to process the original image respectively, and directly output the respective processing results as the detection results, or input the processing results into the target artificial intelligence model for prediction to obtain the detection results;

[0020] Correspondingly, step 6 specifically includes:

[0021] Will N The average predicted label of the reconstructed image is compared with the predicted label of the original image. If they are consistent,N The average predicted label of the reconstructed images and the plurality of detection results are voted by majority to obtain the final decision.

[0022] Furthermore, it also includes: when the original image is marked as an adversarial sample, refusing to output the classification result; otherwise, outputting its predicted label.

[0023] On the other hand, the present invention also provides an adversarial sample detection device based on randomness and redundancy mechanism, comprising:

[0024] The multiple random masking and reconstruction modules are used to randomly mask the pixel blocks of the original image according to the masking ratio, and reconstruct the image based on the remaining pixel information after masking until the number of pixels is repeated. N masking and reconstruction, we get N A reconstructed image;

[0025] Target AI model, used to input the original image and N The reconstructed image is predicted to obtain the prediction result of the original image and N The prediction results of the reconstructed image are obtained, and each prediction result retains the prediction probability of all class labels, among which the class label corresponding to the maximum prediction probability is the predicted label;

[0026] The decision module includes a prediction label chaos degree judgment unit, an average prediction probability judgment unit and a result consistency judgment unit; the prediction label chaos degree judgment unit is used to judge the input N The predicted labels of the reconstructed images are compared for consistency. If the number of inconsistent predicted labels exceeds a first set threshold, the original image is directly marked as an adversarial sample; the average predicted probability decision unit is used to calculate N The predicted probability average value of each reconstructed image belonging to each type of label, if the maximum value of all the predicted probability average values is lower than the second set threshold, the original image is marked as an adversarial sample; the result consistency judgment unit is used to N The average predicted label of the reconstructed images is compared with the predicted label of the original image for consistency. If they are inconsistent, the original image is marked as an adversarial sample; the class label corresponding to the maximum value of all the average prediction probabilities is the average predicted label.

[0027] Furthermore, it also includes: a data preprocessing module, which is used to perform data preprocessing on the input image, and the data preprocessing includes at least one of denoising, denoising, color conversion and size conversion.

[0028] Furthermore, the system further includes: at least two existing adversarial sample detection modules, configured to process the original image using existing adversarial sample detection methods, and directly output the respective processing results as detection results, or input the processing results into the target artificial intelligence model for prediction to obtain detection results;

[0029] Correspondingly, the decision module further includes a majority voting decision unit; the majority voting decision unit is used to N When the average predicted label of the reconstructed images is consistent with the predicted label of the original image, N The average predicted label of the reconstructed images and the plurality of detection results are voted by majority to obtain the final decision.

[0030] Beneficial effects of the present invention:

[0031] Compared with the prior art, the present invention has the following beneficial effects:

[0032] (1) Using multiple random masking and reconstruction techniques to remove or disrupt adversarial perturbations can improve the detection accuracy of samples;

[0033] (2) Compared with traditional detection technologies based on learning the differences between adversarial samples and benign samples, the present invention provides a highly decoupled detection method and device that does not require prior knowledge of adversarial samples and can be combined with target artificial intelligence models in different application environments. It also supports combination with other existing detection technologies.

[0034] (3) Compared with traditional data processing detection technologies, this invention provides an efficient and reliable adversarial sample detection method through a three-layer decision mechanism of randomness, redundancy, and gradual progression;

[0035] (4) The present invention provides a threshold adjustment function, which can balance the robustness and recognition accuracy according to different security requirements to meet different application environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 A flowchart of a method for detecting adversarial examples based on randomness and redundancy mechanisms provided by an embodiment of the present invention;

[0037] Figure 2 One of the structural diagrams of an adversarial sample detection device based on randomness and redundancy mechanism provided by an embodiment of the present invention;

[0038] Figure 3 A second structural diagram of an adversarial sample detection device based on a random and redundancy mechanism provided by an embodiment of the present invention;

[0039] Figure 4A third structural diagram of an adversarial sample detection device based on randomness and redundancy mechanism provided by an embodiment of the present invention;

[0040] Figure 5 The fourth structural diagram of an adversarial sample detection device based on randomness and redundancy mechanism provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0041] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly described below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0042] Example 1

[0043] like Figure 1 As shown, an embodiment of the present invention provides an adversarial sample detection method based on a random and redundancy mechanism, comprising the following steps:

[0044] S101: randomly masking pixel blocks of the original image according to a set masking ratio, and reconstructing the image based on the remaining pixel information after masking;

[0045] S102: Repeat step S101 until the set number of reconstruction times is met N ,get N A reconstructed image;

[0046] S103: The original image and N The reconstructed image is input into the target artificial intelligence model for prediction, and the prediction result of the original image and N The prediction results of the reconstructed image are obtained, and each prediction result retains the prediction probability of all class labels, among which the class label corresponding to the maximum prediction probability is the predicted label;

[0047] Specifically, the target AI model is the AI model that needs to be protected and can be modified according to the application environment. This model is designed to predict all reconstructed images and original images from the previous output, and the prediction results are used in subsequent steps.

[0048] S104: Yes N The predicted labels of the reconstructed images are compared for consistency. If the number of inconsistent predicted labels exceeds a first set threshold, the original image is directly marked as an adversarial sample; otherwise, step S105 is continued:

[0049] S105: For each class label, calculate NThe predicted probability average value of each reconstructed image belonging to each class of labels; if the maximum value among all the predicted probability average values is lower than the second set threshold, the original image is marked as an adversarial sample; otherwise, continue to step S106;

[0050] For example, there are three categories, three reconstructed images, and the second threshold is set to 0.5; if the target artificial intelligence model predicts the three reconstructed images as (0.05, 0.60, 0.35), (0.15, 0.35, 0.50), and (0.10, 0.30, 0.60), respectively; the average predicted probability of the three reconstructed images belonging to the three categories can be calculated as (0.10, 0.42, 0.48), among which the maximum value "0.48" in the average value is still less than 0.5. At this time, the original image is considered to be an adversarial sample. If the target AI model predicts the following for the three reconstructed images: (0.05, 0.50, 0.45), (0.15, 0.25, 0.60), and (0.10, 0.30, 0.60), the average predicted probabilities of the three reconstructed images belonging to the three categories can be calculated as (0.10, 0.35, 0.55). If the maximum value of the average value, "0.55", is greater than 0.5, then the next step is performed.

[0051] S106: N The average predicted label of the reconstructed images is compared with the predicted label of the original image for consistency. If they are inconsistent, the original image is marked as an adversarial sample; otherwise, the original image is marked as a benign sample. Among them, the class label corresponding to the maximum value of all the average prediction probabilities is the average predicted label.

[0052] For example, based on the above example, the maximum value "0.55" in the average corresponds to the third category, that is, the average predicted label is the third category. If the predicted label of the original image is the second category, it means that the two predicted labels are inconsistent, and the original image is considered an adversarial example.

[0053] Example 2

[0054] Based on the above embodiment 1, the difference from the above embodiment 1 is that, before step S101, the present embodiment further includes:

[0055] Performing data preprocessing on the original image, wherein the data preprocessing includes at least one of denoising, adding noise, color conversion, and size conversion;

[0056] Correspondingly, step S101 is adjusted to: randomly masking pixel blocks of the original image according to the set masking ratio, and reconstructing the image based on the remaining pixel information after masking;

[0057] The remaining steps are the same as those in Example 1 and will not be repeated here.

[0058] Example 3

[0059] Based on the above embodiment 1, the difference from the above embodiment 1 is that this embodiment specifically includes the following steps:

[0060] S301: The same as step S101, which will not be described again here;

[0061] S302: The same as step S102, which will not be described again here;

[0062] S303: The original image and N The reconstructed images are preprocessed and the original images after data preprocessing are compared with N The reconstructed image is input into the target artificial intelligence model for prediction; specifically, the data preprocessing includes at least one of denoising, denoising, color conversion and size conversion.

[0063] Steps S304 to S306 are respectively the same as steps S104 to S106 and are not described again here.

[0064] Example 4

[0065] Based on the above embodiments, the difference from the above embodiments is that this embodiment specifically includes the following steps:

[0066] Steps S401 to S405 are respectively the same as steps S101 to S105 and are not described again here;

[0067] S406: Using multiple existing adversarial sample detection methods to process the original image respectively, and directly outputting the respective processing results as the detection results, or inputting the processing results into the target artificial intelligence model for prediction to obtain the detection results;

[0068] It should be noted that data processed using other existing adversarial sample detection methods can obtain detection results through the target artificial intelligence model or directly according to their needs.

[0069] Will N The average predicted labels of the reconstructed images are compared with the predicted labels of the original images. N The average predicted label of the reconstructed image is compared with the predicted label of the original image. If they are inconsistent, the original image is marked as an adversarial sample; if they are consistent, the original image is further marked as an adversarial sample. N The average predicted label of the reconstructed images and the plurality of detection results are voted by majority to obtain the final decision.

[0070] Preferably, in the above embodiments, the method further includes: when the original image is marked as an adversarial sample, refusing to output the classification result; otherwise, outputting its predicted label.

[0071] Example 5

[0072] Corresponding to the above-mentioned adversarial sample detection method based on randomness and redundancy mechanism, such as Figure 2 As shown, an embodiment of the present invention provides an adversarial sample detection device based on a random and redundant mechanism, comprising: a multiple random masking and reconstruction module, a target artificial intelligence model, and a decision module;

[0073] The multiple random masking and reconstruction modules are used to randomly mask the pixel blocks of the original image according to the masking ratio, and reconstruct the image based on the remaining pixel information after masking until the number of pixels is repeated. N masking and reconstruction, we get N The target AI model is used to reconstruct the original image and N The reconstructed image is predicted to obtain the prediction result of the original image and N The prediction results of the reconstructed images are obtained, and each prediction result retains the prediction probability of all class labels, wherein the class label corresponding to the maximum prediction probability is the predicted label. The decision module includes a prediction label confusion judgment unit, an average prediction probability judgment unit, and a result consistency judgment unit. The prediction label confusion judgment unit is used to judge the input N The predicted labels of the reconstructed images are compared for consistency. If the number of inconsistent predicted labels exceeds a first set threshold, the original image is directly marked as an adversarial sample. The average predicted probability decision unit is used to calculate N The average predicted probability of the reconstructed images belonging to each type of label, if the maximum value of all the predicted probability averages is lower than the second set threshold, the original image is marked as an adversarial sample; the result consistency judgment unit is used to N The average predicted label of the reconstructed images is compared with the predicted label of the original image for consistency. If they are inconsistent, the original image is marked as an adversarial sample; the class label corresponding to the maximum value of all the average prediction probabilities is the average predicted label.

[0074] Example 6

[0075] On the basis of the above-mentioned embodiment 5, an adversarial sample detection device based on a random and redundancy mechanism provided by an embodiment of the present invention further includes a data preprocessing module;

[0076] As an implementable method, Figure 3As shown, the data preprocessing module is located before the multiple random mask reconstruction module and is used to perform data preprocessing on the input original image; specifically, the original image processed by the data preprocessing module is input to the multiple random mask reconstruction module for processing to obtain N reconstructed images; then the original images that have not been processed by the data preprocessing module and N The reconstructed image is input into the target artificial intelligence model, and the subsequent processing process is the same as that of the above embodiment 5, which will not be repeated here.

[0077] As an implementable method, Figure 4 As shown, the data preprocessing module is located after the multiple random mask reconstruction module and before the target artificial intelligence model, and is used to process the original image and the output of the multiple random mask reconstruction module. N Specifically, the original image after the data preprocessing module and the reconstructed image after the data preprocessing module are N The reconstructed image is input into the target artificial intelligence model, and the subsequent processing process is the same as that of the above embodiment 5, which will not be repeated here.

[0078] Example 7

[0079] Based on the above embodiments, Figure 5 As shown, an adversarial sample detection device based on a random and redundant mechanism provided by an embodiment of the present invention further includes at least two existing adversarial sample detection modules; correspondingly, the decision module further includes a majority voting decision unit.

[0080] The existing adversarial sample detection module is used to process the original image using the existing adversarial sample detection method, and directly output the respective processing results as the detection results, or input the processing results into the target artificial intelligence model for prediction to obtain the detection results. N When the average predicted label of the reconstructed images is consistent with the predicted label of the original image, N The average predicted label of the reconstructed images and the plurality of detection results are voted by majority to obtain the final decision.

[0081] The device provided by the embodiment of the present invention combines other existing detection technologies or devices in a parallel manner with the core device of the present invention to process data, and makes a judgment on the final data by adding a majority voting decision unit. Figure 5 The dotted line indicates possible routes of other existing detection technologies or devices, including the need to combine with the target artificial intelligence model or directly output the final result.

[0082] It should be noted that the adversarial sample detection device based on randomness and redundancy mechanism provided in each embodiment of the present invention is for the purpose of implementing the above-mentioned method embodiments. Its specific functions can be referred to the above-mentioned method embodiments and will not be repeated here.

[0083] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A method for detecting adversarial samples based on randomness and redundancy mechanism, characterized in that: include: Step 1: Randomly mask the pixel blocks of the original image according to the set masking ratio, and reconstruct the image based on the remaining pixel information after masking; Step 2: Repeat step 1 until the set number of rebuilds is met N ,get N A reconstructed image; Step 3: Combine the original image and N The reconstructed image is input into the target artificial intelligence model for prediction, and the prediction result of the original image and N The prediction results of the reconstructed image are obtained, and each prediction result retains the prediction probability of all class labels, among which the class label corresponding to the maximum prediction probability is the predicted label; Step 4: N The predicted labels of the reconstructed images are compared for consistency. If the number of inconsistencies in the predicted labels exceeds a first set threshold, the original image is directly marked as an adversarial sample; otherwise, step 5 is continued; Step 5: For each class label, calculate N The average predicted probability of each reconstructed image belonging to each class of labels; if the maximum value of all the predicted probability averages is lower than the second set threshold, the original image is marked as an adversarial sample; otherwise, continue to step 6; Step 6: N The average predicted label of the reconstructed images is compared with the predicted label of the original image for consistency. If they are inconsistent, the original image is marked as an adversarial sample; otherwise, the original image is marked as a benign sample. Among them, the class label corresponding to the maximum value of all the average prediction probabilities is the average predicted label.

2. The adversarial sample detection method based on randomness and redundancy mechanism according to claim 1, characterized in that: Before step 1, also include: Performing data preprocessing on the original image, wherein the data preprocessing includes at least one of denoising, adding noise, color conversion, and size conversion; Correspondingly, step 1 specifically includes: randomly masking pixel blocks of the original image after data preprocessing according to a set masking ratio.

3. The adversarial sample detection method based on randomness and redundancy mechanism according to claim 1, characterized in that: After step 2, also include: For the original image and N The reconstructed image is subjected to data preprocessing; the data preprocessing includes at least one of denoising, denoising, color conversion and size conversion; Correspondingly, in step 3, the steps include: N The reconstructed image is input into the target artificial intelligence model for prediction.

4. The adversarial sample detection method based on randomness and redundancy mechanism according to claim 3, characterized in that: Also include before step 6: Use multiple existing adversarial sample detection methods to process the original image respectively, and directly output the respective processing results as the detection results, or input the processing results into the target artificial intelligence model for prediction to obtain the detection results; Correspondingly, step 6 specifically includes: Will N The average predicted label of the reconstructed image is compared with the predicted label of the original image. If they are consistent, N The average predicted label of the reconstructed images and the plurality of detection results are voted by majority to obtain the final decision.

5. The adversarial sample detection method based on randomness and redundancy mechanism according to any one of claims 1 to 4, characterized in that: Also includes: When the original image is marked as an adversarial sample, the classification result is rejected. Otherwise, the predicted label is output.

6. A device for detecting adversarial samples based on randomness and redundancy mechanism, characterized in that: include: The multiple random masking and reconstruction modules are used to randomly mask the pixel blocks of the original image according to the masking ratio, and reconstruct the image based on the remaining pixel information after masking until the number of pixels is repeated. N masking and reconstruction, we get N A reconstructed image; Target AI model, used to input the original image and N The reconstructed image is predicted to obtain the prediction result of the original image and N The prediction results of the reconstructed images are obtained. Each prediction result retains the prediction probability of all class labels, among which the class label corresponding to the maximum prediction probability is the predicted label; The decision module includes a prediction label chaos degree judgment unit, an average prediction probability judgment unit and a result consistency judgment unit; the prediction label chaos degree judgment unit is used to judge the input N The predicted labels of the reconstructed images are compared for consistency. If the number of inconsistent predicted labels exceeds a first set threshold, the original image is directly marked as an adversarial sample; the average predicted probability decision unit is used to calculate N The predicted probability average value of each reconstructed image belonging to each type of label, if the maximum value of all the predicted probability average values is lower than the second set threshold, the original image is marked as an adversarial sample; the result consistency judgment unit is used to N The average predicted label of the reconstructed images is compared with the predicted label of the original image for consistency. If they are inconsistent, the original image is marked as an adversarial sample; the class label corresponding to the maximum value of all the average prediction probabilities is the average predicted label.

7. The adversarial sample detection device based on randomness and redundancy mechanism according to claim 6, characterized in that: Also includes: The data preprocessing module is used to perform data preprocessing on the input image, wherein the data preprocessing includes at least one of denoising, denoising, color conversion and size conversion.

8. The adversarial sample detection device based on randomness and redundancy mechanism according to claim 6, characterized in that: Also includes: At least two existing adversarial example detection modules, configured to process the original image using existing adversarial example detection methods, and directly output their respective processing results as detection results, or input the processing results into the target artificial intelligence model for prediction to obtain detection results; Correspondingly, the decision module further includes a majority voting decision unit; the majority voting decision unit is used to N When the average predicted label of the reconstructed images is consistent with the predicted label of the original image, N The average predicted label of the reconstructed images and the plurality of detection results are voted by majority to obtain the final decision.

Images