Method and system for generating highly dissimilar test cases under high imperceptibility constraints

Abstract

The present invention relates to the technical field of high-quality test case generation, and specifically to a method for generating highly dissimilar test cases under high imperceptibility constraints. The method comprises the following steps: obtaining a test image and inputting it into a DNN network model to obtain the classification probability of the test image in each test case category; obtaining the contribution of each pixel in the test image according to the classification probability; obtaining the SCSM matrix of the test image between each test case category according to the weight of the fully connected layer of the last layer of the network model; obtaining the ranking index of each pixel in the test image according to the contribution of each pixel in the test image and the SCSM matrix to sort the pixels in the test image, and selecting the top-ranked pixels to add disturbances to the test image; obtaining multiple test images and adding disturbances to each of them to obtain test cases. The present invention, on the basis of generating higher-quality test cases, helps to reveal vulnerabilities that are more meaningful to test in the DNN model required for the autonomous driving system.

Description

Technical Field

[0001] The present invention relates to the technical field of high-quality test case generation, and in particular to a method and system for generating highly dissimilar test cases under high imperceptibility constraints. Background Art

[0002] In the latest application of autonomous driving systems, The Dawn Project test found that the car's fully autonomous driving system failed to correctly identify children crossing the road to trigger the FSD protection mechanism emergency braking. The system also failed to identify the children or identified them as less dangerous, and was unable to trigger the protection mechanism urgently, causing the car to hit the fake child used for testing. The "phantom braking" problem, which is common to most autonomous driving systems, also reflects that they identify unknown objects as objects requiring emergency braking. This has triggered an urgent need for highly dissimilar error vulnerability testing of DNN models in autonomous driving systems.

[0003] Adversarial attacks refer to deliberately designed input samples intended to cause deep learning models to predict incorrect outputs; imperceptible adversarial attacks are a method used in machine learning and deep neural networks to generate small perturbations that are almost invisible to humans but can significantly change the predictions of classifiers or recognition systems. This type of attack is often used to evaluate and improve the security and robustness of models, especially for models in the field of autonomous driving, because the requirements for imperceptibility are higher when simulating natural environments. Since the concept of "adversarial severity" was first proposed in 2021, related research includes saturation attacks (NetSat). Highly dissimilar test cases have become crucial for revealing vulnerabilities in their visual DNN models, but they have obvious shortcomings.

[0004] Currently, the latest highly dissimilar test case generation method, taking NetSat as an example, still has the following defects: the strategy of modifying all pixels is blind, and the perturbation of many pixels with little impact on classification does more harm than good. The adversarial test cases generated by the current method can be improved for the highly dissimilar indicators, and there is room for improvement in the highly dissimilar indicator DM. Summary of the Invention

[0005] In order to solve the problems existing in the prior art, the present invention provides a method for generating highly dissimilar test cases under high imperceptibility constraints. The method obtains the classification probability of the test image in each test case category by obtaining a test image and inputting it into the DNN network model; obtains the contribution of each pixel in the test image according to the classification probability; obtains the SCSM matrix of the test image between each test case category according to the weight of the fully connected layer of the last layer of the network model; sorts the pixels in the test image according to the contribution of each pixel in the test image and the SCSM matrix to obtain the ranking index of each pixel, selects the top ranked pixels to add disturbances to the test image; obtains multiple test images and adds disturbances respectively to obtain test cases. On the basis of generating higher quality test cases, the present invention helps to reveal more testable vulnerabilities in the DNN model required for the autonomous driving system.

[0006] The present invention adopts the following technical solution, a method for generating highly dissimilar test cases under high imperceptibility constraints, comprising:

[0007] Obtain the test image and input it into the DNN network model to obtain the classification probability of the test image in each test case category;

[0008] Obtaining the contribution of each pixel in the corresponding test image according to the classification probability;

[0009] Obtaining the SCSM matrix of the test image between each test case category according to the fully connected layer weight of the last layer of the network model;

[0010] Perform weighted calculation based on the contribution of each pixel in the test image and the SCSM matrix to obtain a ranking index for each pixel;

[0011] Sort the pixels in the test image according to the ranking index of each pixel, and select the pixels with the highest ranking to add disturbance to the test image;

[0012] Get multiple test images and add perturbations to each of them to get test cases.

[0013] Furthermore, the contribution of each pixel in the corresponding test image is obtained, including:

[0014] Input the test image into the DNN network model and obtain the classification probability corresponding to each parameter category of the test image in the network model;

[0015] The classification probability corresponding to each parameter category is calculated at the location of each pixel in the test image to obtain the contribution corresponding to each pixel.

[0016] Furthermore, the SCSM matrix of the test images between various test case categories is obtained, including:

[0017] Obtain the weight value corresponding to each test case category of the fully connected layer weight of the last layer in the network model;

[0018] Calculate the cosine similarity between two test case categories according to the weight value corresponding to each test case category;

[0019] The SCSM matrix is ​​constructed based on the cosine similarity between the two test case categories.

[0020] Furthermore, a weighted calculation is performed based on the contribution of each pixel in the test image and the SCSM matrix to obtain the ranking index of each pixel, which is expressed as:

[0021] ;

[0022] in The contribution parameters of each pixel corresponding to k categories are assigned to their distance indicators in the SCSM matrix. Indicates the contribution of the pixel to each category, is the contribution of the i-th pixel to the k-th category. If it is a positive value, it remains unchanged; if it is a negative value, it is assigned to 0.

[0023] Furthermore, after selecting the top ranked pixels to add disturbance to the test image, the following steps are also included:

[0024] Select different proportions of pixels from multiple test images and add disturbances to them respectively, to obtain multiple test images corresponding to different disturbance proportions;

[0025] Evaluate the accuracy of test images corresponding to different perturbation ratios, and select the test image with the lowest accuracy based on the evaluation results;

[0026] The perturbation ratio corresponding to the test image with the lowest accuracy is used as the determined ratio.

[0027] Furthermore, after obtaining the test case, it also includes:

[0028] Obtain imperceptibility metrics for test cases;

[0029] The test cases are optimized according to their imperceptibility index, and the high dissimilarity of the test cases is evaluated according to the optimized imperceptibility index.

[0030] The present invention further proposes a highly dissimilar test case generation system under high imperceptibility constraints, which adopts any of the above-mentioned test case generation methods. The system specifically includes: a pixel position contribution index generation module, an SCSM matrix search module, a sorting module, an adversarial test case generation module, and a test effect evaluation module;

[0031] The pixel position contribution index generation module is used to obtain a test image and input it into the DNN network model to obtain the classification probability of the test image in each test case category; and obtain the contribution of each pixel in the corresponding test image based on the classification probability;

[0032] The SCSM matrix search module is used to obtain the SCSM matrix of the test image between each test case category according to the fully connected layer weight of the last layer of the network model;

[0033] The ranking module is used to perform weighted calculation based on the contribution of each pixel in the test image and the SCSM matrix to obtain a ranking index for each pixel;

[0034] The adversarial test case generation module is used to sort the pixels in the test image according to the sorting index of each pixel, select the pixels with the highest sorting ranking to add perturbations to the test image; obtain multiple test images and add perturbations to each of them to obtain test cases;

[0035] The test effect evaluation module is used to obtain the imperceptibility index of the test case; optimize according to the imperceptibility index of the test case, and evaluate the high dissimilarity of the test case according to the optimized imperceptibility index.

[0036] The beneficial effects of the present invention are as follows: the present invention is based on the high quality requirement of high imperceptibility of the current adversarial test cases, and reduces the number of perturbed pixels by finding key pixels to achieve better imperceptibility; in the process of finding the key pixel ranking, the highly dissimilar search strategy is integrated, so that when the perturbation is added to the top-ranked key pixels found, the confidence of the category that is more highly dissimilar to the original category can be increased; specifically, the present invention further obtains the contribution parameter of each image at each pixel position in the image by calculating the classification probability of each image in the model; and describes the gap between each category by obtaining the classification weight of the last layer of the DNN model. The invention provides an SCSM matrix of the size of ; according to the contribution parameter of each pixel position and the SCSM matrix, after obtaining the ranking index in a weighted manner according to our purpose, all pixel positions are sorted; when generating adversarial test cases, only the top-ranked pixel positions are selected to control the perturbation amplitude and add perturbations; multiple indicators are used to comprehensively evaluate the imperceptibility, and when evaluating the high dissimilarity of the generated adversarial test cases, the weighted classification confidence of the evaluation indicators is optimized; the method of the present invention can achieve the generation of high-quality imperceptible adversarial test cases, while ensuring a better effect in testing the classification vulnerabilities of high dissimilarity in the DNN model related to the autonomous driving system. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0038] Figure 1 Schematic diagram of the steps of a method for generating highly dissimilar test cases under high imperceptibility constraints according to an embodiment of the present invention;

[0039] Figure 2 Schematic diagram of the process of generating highly dissimilar test cases under high imperceptibility constraints according to an embodiment of the present invention;

[0040] Figure 3 A diagram showing a relationship curve between adversarial test cases generated by modifying sorted pixel positions at different ratios with a constant perturbation size and recognition accuracy according to an embodiment of the present invention;

[0041] Figure 4 The SCSM matrix obtained by calculating and sorting different model weight parameters according to an embodiment of the present invention;

[0042] Figure 5is the ratio of adversarial test cases with classification errors to adversarial test cases under a fixed and identical imperceptibility in an embodiment of the present invention;

[0043] Figure 6 A clustering diagram of an original image and an adversarial test case under the same dataset model and the same imperceptibility in an embodiment of the present invention;

[0044] Figure 7 A schematic diagram of the number of error categories misclassified when an MNIST dataset and a corresponding model generate the same imperceptible adversarial test case using different methods according to an embodiment of the present invention;

[0045] Figure 8 A schematic diagram of the number of error categories misclassified when a CIFAR-10 dataset and a corresponding model generate the same imperceptible adversarial test case using different methods according to an embodiment of the present invention;

[0046] Figure 9 A schematic diagram of an STL-10 dataset and the number of error categories misclassified when the corresponding model generates the same imperceptible adversarial test case using different methods according to an embodiment of the present invention;

[0047] Figure 10 A schematic diagram of the ADM indicator results when the MNIST dataset and the corresponding model generate the same imperceptible adversarial test case using different methods according to an embodiment of the present invention;

[0048] Figure 11 A schematic diagram of the ADM indicator results when the CIFAR-10 dataset and the corresponding model generate the same imperceptible adversarial test case using different methods according to an embodiment of the present invention;

[0049] Figure 12 A schematic diagram of the ADM indicator results when an STL-10 dataset and a corresponding model generate the same imperceptible adversarial test case using different methods according to an embodiment of the present invention;

[0050] Figure 13 A schematic diagram of the ADM index results of 500 misclassified adversarial test cases generated by different methods using different dataset models within a certain imperceptibility index range according to an embodiment of the present invention;

[0051] Figure 14 A structural diagram of a highly dissimilar test case generation system under a high imperceptibility constraint according to an embodiment of the present invention. DETAILED DESCRIPTION

[0052] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0053] like Figure 1 and Figure 2 As shown, an embodiment of the present invention provides a method for generating highly dissimilar test cases under a high imperceptibility constraint, comprising the following steps:

[0054] S11. For each image's classification probability in the model, the contribution parameter is calculated at each pixel position in the image.

[0055] The test pictures obtained in the embodiment of the present invention are taken as an example of the CIFAR-10 dataset, and the DNN network model is explained by taking the VGG16 model as an example. First, a test picture set of 10 categories, 100 pictures in each category, and a total of 1000 original pictures is defined in the test set. Each picture is traversed in a loop and the trained VGG16 model parameters are put into it to obtain the classification probabilities of the 10 categories. These ten classification probabilities can obtain ten values ​​at each pixel through the forward gradient formula. These ten values ​​represent the contribution parameters of this pixel position to these ten categories. If the contribution parameter obtained for a certain category is a larger positive value, it means that adding disturbance to this pixel can maximize the increase in the model's classification probability for this category. If the contribution parameter obtained for a certain category is a larger negative value, it means that adding disturbance to this pixel position will maximize the decrease in the classification probability of this category.

[0056] A forward gradient calculation formula provided in an embodiment of the present invention is as follows:

[0057] ;

[0058] in the formula represents the forward gradient matrix, It means j The classification probability value of each category, x i Indicates the original image i pixel positions, M is the number of pixels, N is the number of categories. Taking CIFAR-10 as an example, the final result is 3072 A matrix of size 10 means that each of the 3072 pixel positions will have contribution parameters for 10 categories. It should also be pointed out that the same pixel position in different channels can be repeatedly selected when the key pixel is finally selected.

[0059] S12, obtain the SCSM matrix describing the gap between each category through the classification weight of the last layer of the DNN model;

[0060] In an embodiment of the present invention, the weights of the activation function for the classification output results of the last layer of model parameters are fixed, and cosine similarity is calculated for each of the two categories. A larger calculated value indicates a greater degree of dissimilarity between the two categories for the model's classification. The traditional dissimilarity metric (DM) is a redefinition and representation of average visual confusion. It aims to quantify the degree of deviation between the model's predicted categories and the true categories of the original data in a perturbed dataset. The core purpose of this metric is to assess the extent and nature of the structural damage to the original dataset. However, manually annotating visual similarity between categories is impractical in deep learning models with a large number of categories. Therefore, the DM metric does not rely on manual similarity annotation, but instead extracts information from the model's own learning structure to measure the similarity between different categories.

[0061] In the embodiment of the present invention, the similarity between categories is calculated using the following cosine similarity formula:

[0062] ;

[0063] CSM i,j Representation category i and categories j The cosine similarity of and Represents the weights of model category i and category j respectively, T is the transpose of the matrix, and by fixing an original category i and calculate it with all other categories j The similarity between them can be used to obtain a matrix that reflects the degree of difference between categories. This matrix is ​​called the SCSM matrix (Similarity-based Class Similarity Matrix).

[0064] like Figure 4The figure shows the SCSM matrix calculated by the model parameters of the LeNet-5 model, VGG16 model and ResNet18 model calculated in the embodiment of the present invention, which correspond to the MNIST dataset, CIFAR-10 dataset and STL-10 model training respectively. The classification of the MNIST dataset is relatively intuitive, and 0-9 represents the classification of the corresponding handwritten digits. The 0-9 of the CIFAR-10 dataset and the STL-10 dataset correspond to their categories of airplane, car, bird, cat, deer, dog, frog, horse, ship, truck and airplane, bird, car, cat, deer, dog, horse, monkey, ship and truck respectively; it fully demonstrates the similarity distribution pattern between categories, which can more intuitively reveal the impact of disturbance on model classification. In this way, SCSM A systematic measurement method for predicting changes on a perturbed dataset is provided, wherein the first column 0-9 of each matrix represents the original category, and the categories that follow are closer to it, indicating that the model believes that the categories are semantically closer and have smaller height dissimilarity, while the categories that are farther away from the original category are farther away from the model and have greater height dissimilarity. Then, based on the relationship between other categories and the original category in this matrix in terms of column distance, in an embodiment of the present invention, ten columns are assigned a distance index DM starting from the leftmost original category, and under the original category of a certain row of the original image, the resulting category of the generated adversarial test case classification is the category in the column, and then it is assigned a corresponding number from 0 to 9 to measure the size of its height dissimilarity.

[0065] S13, sorting all pixel positions after obtaining a sorting index based on the contribution parameter of each pixel position and the SCSM matrix in a weighted manner;

[0066] The embodiment of the present invention obtains the contribution parameters of each pixel position to each category through the above steps. The positive and negative values ​​of these parameters can intuitively reflect whether the classification result of a certain category is positively promoted or negatively inhibited when the perturbation is added to the pixel position. At the same time, the CSM obtained by cosine similarity calculation is i,j Combined with each fixed original category i Then, calculate its j The size of the category difference between them can be used to construct an SCSM matrix that reflects the difference between categories. This matrix can measure which categories are highly dissimilar to a fixed category (farther away) and which categories are more similar to it (closer), thereby providing a quantitative representation of classification similarity.

[0067] Each pixel position is further weighted based on the contribution parameter of the pixel position and the SCSM matrix. Specifically, for each category of a certain pixel position, the embodiment of the present invention multiplies the contribution parameter of the category by its distance score relative to the fixed category in the SCSM matrix. This weighting method can effectively integrate the intensity of the pixel's contribution to the classification and the dissimilarity between categories. At the same time, in order to ensure that the direction of the disturbance is as expected, the embodiment of the present invention only weights the categories with positive contribution parameters, because a negative contribution parameter means that adding disturbance will reduce the model's classification probability for the category, which is contrary to the goal of generating effective adversarial samples. Therefore, the category with a negative contribution parameter is directly assigned a value of 0 to avoid its interference.

[0068] The contribution parameter at each pixel position and the SCSM matrix are used to weight the pixel position ranking formula that has a higher contribution to the more highly dissimilar category at each pixel position, which is expressed as:

[0069] ;

[0070] in The contribution parameters of each pixel corresponding to k categories are assigned to their distance indicators in the SCSM matrix. Indicates the contribution of the pixel to each category, is the contribution of the i-th pixel to the k-th category. If it is a positive value, it remains unchanged; if it is a negative value, it is assigned to 0.

[0071] After completing the above weighted calculation, each pixel position is sorted from large to small according to the results. This sorting can reflect which pixel positions have a higher contribution to categories that are highly dissimilar to the fixed category. By selecting pixel positions with higher rankings, key pixels that are more helpful in distinguishing the fixed category from other categories can be identified.

[0072] It should be noted that when determining the proportion of pixel positions used to add perturbations, since the process of adding perturbations is actually superimposing a certain amplitude of perturbation parameters on the pixel positions, selecting too high a proportion of pixel positions may lead to a significant decrease in the overall contrast of the image, thereby negatively affecting the classification test effect of the adversarial sample; as the proportion of perturbed pixels increases, the test effect may not only not improve, but may even decrease. In addition, since the image characteristics of different data sets are different, the optimal perturbation ratio needs to be optimized individually according to the specific data set to ensure that the generated adversarial sample can maximize the test effect without destroying the perceived quality of the image.

[0073] S14. When generating adversarial test cases, only the top-ranked pixel positions are selected to control the perturbation amplitude and add perturbations.

[0074] In an embodiment of the present invention, after perturbing different proportions of pixels in each data set with several fixed-value perturbations, the accuracy of the adversarial test case is obtained as a measurement indicator, thereby determining the key pixel ratio with the lowest accuracy for this data set as the perturbation ratio, which is further used as a fixed parameter in subsequent experiments.

[0075] First, determine the optimal ratio of pixel positions to be selected when generating adversarial test cases. For test datasets of different datasets, 100 images of each category, totaling 1000 test original images, are used to add perturbations to pixel positions of different ratios after fixing the perturbation size. Then, the accuracy is compared to determine the ratio of key pixel selection for this dataset, such as Figure 3 As shown, in the MNIST, CIFAR-10, and STL-10 datasets, the embodiments of the present invention determine this ratio to 55%, 41%, and 18%, respectively, thereby preliminarily generating adversarial test cases.

[0076] The embodiment of the present invention further calculates the imperceptibility index of the adversarial test case including:

[0077] ;

[0078] in, represents the generated adversarial test case, Indicates the name of the test case. represents the original sample, The number of adversarial test cases generated for the adversarial attack, represents the P norm, where the average normalized distortion (ALD P ) value is smaller, indicating that the difference perceived by the naked eye is smaller, that is, the imperceptibility is better. In addition, the embodiment of the present invention also uses the following SSIM indicator, which is considered to be an effective method for measuring human eye perception. It measures the similarity between two images by comparing brightness, contrast, and structure. It is expressed as:

[0079] ;

[0080] In the formula, and are the mean and variance of the original samples, and are the mean and variance of the adversarial test cases, C 1 and C 2 It is a constant set to stabilize the denominator. The SSIM value range is between [-1, 1]. The closer the value is to 1, the more similar the two images are, and the closer the value is to -1, the less similar the two images are.

[0081] Since SSIM is only calculated for a single image, and structural similarity (ASS) is calculated by calculating the average SSIM between all adversarial test case datasets and their original sample datasets, this embodiment of the present invention further defines structural similarity (ASS) as:

[0082] ;

[0083] in, is the structural similarity, To counter the number of test cases, represents the generated adversarial test case, Indicates the name of the test case. Represents the original sample.

[0084] Since the imperceptibility of an image is calculated by multiple metrics, it has a higher accuracy for the image. P ) and structural similarity (ASS), so it is necessary to unify these two indicators into one parameter to measure the imperceptibility of adversarial test cases, and name it Imperceptibility.

[0085] The embodiment of the present invention first calculates the average normalized distortion (ALD) of the adversarial test case. P ) and structural similarity (ASS), and normalize them to the range of 0~1 to obtain ALD PN and ASS N , since the meaning of the imperceptibility index is the larger the better, and ALD P The meaning of the indicator is exactly the opposite, so ALD P The value is expressed as 1-ALD P , the calculation of the imperceptibility index is finally defined as follows:

[0086] ;

[0087] Where, is the average normalized distortion after normalization, is the normalized structural similarity.

[0088] Due to the importance of these two indicators, the embodiment of the present invention selects 0.5 as the importance parameter in front of these two indicators. In addition, on the basis of keeping the imperceptibility indicator basically similar, the embodiment of the present invention uses VGG16 to extract features, UMAP dimension reduction, and HDBSCAN clustering experiments to verify that the adversarial test cases generated in the embodiment of the present invention are of higher quality: Figure 6 As shown, in Figure 6 In the clustering diagram, if the distance between two types of images is closer, it means that the model believes that the two types of images are more similar in features. According to the results in the diagram, it can be found that in the three data sets and models, the imperceptibility of adversarial test cases generated by different methods is similar. The clustering results of the adversarial test cases generated by the method adopted in the embodiment of the present invention and the original images after feature extraction are better than those of other methods, indicating that the adversarial test cases generated by the present invention are closer to the original images in terms of imperceptibility and features, and the image quality is higher.

[0089] S15. Optimization metrics comprehensively evaluate the high dissimilarity of generated adversarial test cases.

[0090] In the embodiment of the present invention, for the adversarial test case generation method proposed in the present invention and other methods used for comparison and measurement, after the adversarial test cases have been similarly generated, what is expressed by the DM indicator is only the degree of high dissimilarity between the classification category of an original test image and the classification category of the adversarial test case. What needs to be measured in the evaluation of the embodiment of the present invention is the average DM size of all generated adversarial test cases, which is defined as ADM. In addition, the probability of misclassification is also considered in the embodiment of the present invention. When evaluating the effectiveness of the adversarial attack, special attention is paid to the change in the confidence of the model when facing adversarial samples, which reflects the effectiveness of the generated adversarial test cases.

[0091] Current existing research indicates that if an adversarial test case can significantly reduce the model's confidence in the original category, or increase the model's confidence in the wrong category, this means that the model is highly vulnerable to these adversarial samples. The model's high confidence in misclassification indicates that it is very sensitive to perturbations, showing the model's potential vulnerability. In addition, the difference in the model's confidence response to adversarial test cases generated by different attack methods reveals the model's structural high sensitivity to specific attacks. Therefore, in the indicators proposed in the embodiments of the present invention, it is necessary to multiply the DM value obtained for each adversarial test case by the classification probability of the wrong category. P adv， The average DM of all adversarial test cases is defined as follows:

[0092] ;

[0093] in, represents the high dissimilarity index of all adversarial test cases, is the probability of classification into the wrong category.

[0094] The experimental results are given in the examples of the present invention. Figure 7 、 Figure 8 、 Figure 9 As shown in Figure 2, after the three dataset models generated adversarial test cases using different methods, it was shown that FGSM, as a strong perturbation method for model failure, generated more adversarial test cases of wrong categories, but because the number of adversarial test cases whose classification categories were far from the original categories was small, it was ultimately Figure 10 、 Figure 11 、 Figure 12 The ADM index of each category in the method of the present invention does not exceed that of the method proposed in the present invention. The ADM index of the present invention is better in all categories. It should be noted that the original purpose of the saturation attack (NetSat) method is to obtain adversarial test cases whose classification results are far away from the true labels. Figure 10 、 Figure 11 、 Figure 12 It can be found that its effect is even worse than FGSM in many categories. This is because the experimental premise of the method proposed in this invention is to control the perturbation size of the generated adversarial test cases, that is, it is imperceptible. Their ADM indicators are compared under basically the same conditions. However, since different methods add perturbations in different ways, the sensitivity to perturbations after generating adversarial test cases is also different. Therefore, it is more valuable to compare other effects under fixed perturbations.

[0095] at the same time Figure 13 It shows that the present invention controls the Imperceptibility index within a certain imperceptible range, and then compares the experimental results of different methods generating a total of 500 adversarial test cases in each category at different imperceptible levels. The ADM index shows that the adversarial test case method proposed in the present invention is always better than other methods.

[0096] In another embodiment of the present invention, Figure 14 As shown in the figure, a highly dissimilar test case generation system under high imperceptibility constraint is proposed. Figure 1 The system corresponding to the method for generating highly dissimilar test cases under a high imperceptibility constraint in the corresponding embodiment is realized by a virtual system. Figure 1In the corresponding embodiment, the method for generating highly dissimilar test cases under a high imperceptibility constraint is based on a method for generating highly dissimilar test cases under a high imperceptibility constraint. Each virtual module constituting the system corresponding to the method for generating highly dissimilar test cases under a high imperceptibility constraint can be executed by an electronic device, such as a network device, a terminal device, or a server. Specifically, the system includes: a pixel position contribution index generation module 01, an SCSM matrix search module 02, a sorting module 03, an adversarial test case generation module 04, and a test effect evaluation module 05, wherein:

[0097] Pixel position contribution index generation module 01 is used to obtain the contribution parameter size of each pixel position to each category classification;

[0098] SCSM matrix search module 02 is used to obtain the SCSM matrix describing the size of the gap between each category through the classification weight of the activation function of the last layer of the DNN model using the cosine similarity formula;

[0099] Sorting module 03 is used to calculate the contribution parameters of each pixel position to each category and the obtained SCSM matrix of the high dissimilarity between the evaluation model categories through weighted calculation to determine whether the pixel position has a higher contribution to the more highly dissimilar category, and then sort all pixels from large to small according to the result of each pixel for use in the next step;

[0100] The adversarial test case generation module 04 is used to determine the optimal ratio of pixel positions to be selected when generating adversarial test cases. For the test data set of different data sets, 100 images of each category, totaling 1000 test original images, are used to select pixel positions with different ratios of pixel positions after fixing the perturbation size, and then compare the accuracy to determine the ratio of key pixel selection for this data set, such as Figure 3 As shown in the figure, in the MNIST, CIFAR-10, and STL-10 datasets, this ratio is determined to be 55%, 41%, and 18%, respectively. After obtaining the ratio of perturbation pixel selection, the perturbation amplitude is adjusted to generate adversarial test cases with different perturbation sizes. Then, some existing adversarial test case generation methods NetSat and FGSM that have been integrated into the system are used to generate adversarial test cases with basically similar imperceptibility, waiting for the next module to evaluate its test effect on revealing model vulnerabilities.

[0101] The test effect evaluation module 05 is used to compare the efficiency of the adversarial test cases generated by the different adversarial test case generation methods obtained in the above modules under the condition that the known imperceptibility is as similar as possible, and to determine whether the generated test cases can reflect the errors of the model vulnerabilities so that the classification results are not equal to the categories of the original images, that is, classification errors. Figure 5 By fixing the imperceptibility of adversarial test cases generated by each method under different data sets and models and looking at the proportion of classification errors of adversarial test cases generated by each method under 1,000 original images in the test data set with basically the same size, it is found that the efficiency of the present invention is not reduced too much, and through multiple experiments, it is found that it remains at around 1%-2%.

[0102] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for generating highly dissimilar test cases under high imperceptibility constraints, characterized by: include: Obtain the test image and input it into the DNN network model, and use the forward gradient calculation to obtain the classification probability of the test image in each test case category; Obtaining the contribution of each pixel in the corresponding test image according to the classification probability; Obtaining an SCSM matrix of the test image between each test case category based on the fully connected layer weights of the last layer of the network model; the SCSM matrix is ​​specifically obtained by fixing an original category and calculating the similarity between it and all other categories to obtain an SCSM matrix reflecting the degree of difference between categories; In each row of the SCSM matrix, the elements in the first column are the original categories, and the other elements after the original category are other categories arranged in descending order based on their semantic similarity to the original category. A weighted calculation is performed based on the contribution of each pixel in the test image and the SCSM matrix to obtain the ranking index of each pixel; the expression is: ； in The contribution parameters of each pixel corresponding to k categories are assigned to their distance indicators in the SCSM matrix. Indicates the contribution of the pixel to each category, is the contribution of the i-th pixel to the k-th category. If it is positive, it remains unchanged; if it is negative, it is assigned to 0. Sort the pixels in the test image according to the ranking index of each pixel, and select the pixels with the highest ranking to add disturbance to the test image; Get multiple test images and add perturbations to each of them to get test cases.

2. The method for generating highly dissimilar test cases under high imperceptibility constraints according to claim 1, wherein: Get the contribution of each pixel in the corresponding test image, including: Input the test image into the DNN network model and obtain the classification probability corresponding to each parameter category of the test image in the network model; The classification probability corresponding to each parameter category is calculated at the location of each pixel in the test image to obtain the contribution corresponding to each pixel.

3. The method for generating highly dissimilar test cases under high imperceptibility constraints according to claim 1, wherein: Obtain the SCSM matrix of test images between various test case categories, including: Obtain the weight value corresponding to each test case category of the fully connected layer weight of the last layer in the network model; Calculate the cosine similarity between two test case categories according to the weight value corresponding to each test case category; The SCSM matrix is ​​constructed based on the cosine similarity between the two test case categories.

4. The method for generating highly dissimilar test cases under high imperceptibility constraints according to claim 1, wherein: After selecting the top ranked pixels to add perturbations to the test image, the following steps are also included: Select different proportions of pixels from multiple test images and add disturbances to them respectively, to obtain multiple test images corresponding to different disturbance proportions; Evaluate the accuracy of test images corresponding to different perturbation ratios, and select the test image with the lowest accuracy based on the evaluation results; The perturbation ratio corresponding to the test image with the lowest accuracy is used as the determined ratio.

5. The method for generating highly dissimilar test cases under high imperceptibility constraints according to claim 1, wherein: After getting the test case, it also includes: Obtain imperceptibility metrics for test cases; The test cases are optimized according to their imperceptibility index, and the high dissimilarity of the test cases is evaluated according to the optimized imperceptibility index.

6. A highly dissimilar test case generation system under high imperceptibility constraints, using the highly dissimilar test case generation method under high imperceptibility constraints according to any one of claims 1 to 5, characterized in that: include: Pixel position contribution index generation module, SCSM matrix search module, sorting module, adversarial test case generation module, and test effect evaluation module; The pixel position contribution index generation module is used to obtain a test image and input it into the DNN network model to obtain the classification probability of the test image in each test case category; and obtain the contribution of each pixel in the corresponding test image based on the classification probability; The SCSM matrix search module is used to obtain the SCSM matrix of the test image between each test case category according to the fully connected layer weight of the last layer of the network model; The ranking module is used to perform weighted calculation based on the contribution of each pixel in the test image and the SCSM matrix to obtain a ranking index for each pixel; The adversarial test case generation module is used to sort the pixels in the test image according to the sorting index of each pixel, and select the pixels with the highest sorting ranking to add disturbance to the test image; Get multiple test images and add perturbations to each of them to get test cases; The test effect evaluation module is used to obtain the imperceptibility index of the test case; optimize according to the imperceptibility index of the test case, and evaluate the high dissimilarity of the test case according to the optimized imperceptibility index.

Images