Image classification model back door defense method and device based on interpretability

Abstract

The invention provides an interpretability-based image classification model backdoor defense method and device. The method comprises the following steps: building a clean model and a backdoor model of backdoor experimental environment reference training image classification; carrying out an interpretability experiment on the trained clean model and back door model to obtain concepts of various categories under the data set and screening the concepts; based on the screened concepts, carrying out comprehensive feature vector construction capable of being explained and expressed; taking the constructed comprehensive feature vector as an input to train a binary classifier used for distinguishing a clean model and a subsequent model; and judging whether the to-be-detected image classification model belongs to the backdoor model or not by using the binary classifier. According to the method, the image classification model backdoor defense method based on interpretability is introduced, the clean model and the backdoor model are effectively distinguished by utilizing the expression difference of the model under different interpretable methods, detection and defense of the backdoor injection attack are realized, the method has relatively high detection accuracy and practicability, and the safety and reliability of the model are improved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a backdoor defense method and device for an image classification model based on interpretability. Background Art

[0002] With the rapid development of artificial intelligence technology, deep learning technology represented by deep neural networks has made breakthroughs in many fields such as image classification, speech recognition, and natural language processing, and has also shown its potential in application scenarios such as autonomous driving, machine translation, face recognition, audio processing, and assisted medical care. There are all kinds of self-driving taxis and express delivery vehicles on the road, as well as Tesla cars equipped with autonomous driving systems. Facial recognition can be seen everywhere in teaching buildings, train stations, and airports. Pedestrian detection and vehicle recognition systems in road monitoring and security videos. Deep learning technology has gradually begun to make decisions on behalf of humans in various key tasks and has become an integral part of human daily life. As a technology leading the future, deep learning will have an impact on all aspects of the economy, society, and life.

[0003] When deep learning models are applied to image classification scenarios, in order to achieve better model effects, it is often necessary to use massive labeled image data for model training. Due to the complexity of the model structure and the large number of parameters, the training process has high requirements for hardware (computing resources and memory resources). This makes it impossible for many users who need to use the model to complete the calculation locally, and instead seek cooperation with third parties, use cloud platform services for training, or use third-party open source or data and pre-trained models. Although open source data and models provide convenience for scientific research and use, the intervention of third parties in the model training process also provides opportunities for malicious attackers, making deep learning models face security risks and more vulnerable to backdoor injection attacks. In the process of third-party training models, malicious attackers can preset backdoors by tampering with some data samples in the training set, the model structure itself, and other methods, thereby manipulating the performance results of the model when the model is running, so that it is output in the preset form, causing serious consequences. This process is called a backdoor injection attack. For example, in an autonomous driving scenario, an attacker can use a backdoor injection attack to make the traffic sign recognition model recognize the "stop" sign as a "speed limit" sign, which may cause a car accident during actual driving and threaten the safety of the autonomous driver. For another example, in a face recognition scenario, for a model that has been successfully injected with a backdoor, an attacker can bypass the facial recognition system and obtain higher permissions by simply wearing a pair of sunglasses. As can be seen, as deep learning models are closely linked to daily life, it is of great significance to eliminate the threat of backdoor attacks on deep learning models and ensure the security of artificial intelligence.

[0004] Faced with the severe situation of backdoor attacks, existing defense methods can be divided into two categories: empirical backdoor defense and authenticated backdoor defense. Empirical backdoor defense is proposed based on a certain understanding of existing attacks. It has good performance in practice, but its effectiveness lacks theoretical guarantee. The effectiveness of authenticated backdoor defense can be guaranteed under certain assumptions, but the defense effect of such methods is generally weaker than that of empirical backdoor defense methods. So far, the latest empirical defense strategies cannot cope with multiple variants of backdoor triggers, most of them are not robust, and have poor adaptability. Although authenticated defense has theoretical support and is robust to backdoor attacks, current strategy research is based on random filtering, the ideas are relatively limited, and the general effect is not good. Summary of the invention

[0005] In view of this, the present invention provides an image classification model backdoor defense method and device based on interpretability to solve at least one of the problems mentioned above.

[0006] In order to achieve the above object, the present invention adopts the following scheme:

[0007] According to a first aspect of the present invention, there is provided a backdoor defense method for an image classification model based on interpretability, the method comprising: building a backdoor experimental environment benchmark, training a clean model and a backdoor model for image classification, wherein the clean model and the backdoor model use the same model structure and the same data set; conducting an interpretability experiment on the trained clean model and the backdoor model to obtain concepts of each category under the data set, and screening the concepts based on the TCAV method or the ACE method; constructing a comprehensive feature vector of interpretable expression based on the screened concepts, the comprehensive feature vector comprising the number of interpretable expression concepts and interpretable expression concept features; using the constructed comprehensive feature vector as input to train a binary classifier for distinguishing between a clean model and a backdoor model; and using the binary classifier to determine whether the image classification model to be tested belongs to a backdoor model.

[0008] As an embodiment of the present invention, the method of performing an explainability experiment on the trained clean model and the backdoor model includes: repeating the explainability experiment on the trained clean model and the backdoor model three times under the same experimental conditions.

[0009] As an embodiment of the present invention, the above method screens the concepts based on the TCAV method, including: determining whether the TCAV score is greater than 0.6 and the P-Value is less than 0.05 at the same time; if so, screening and storing; if the number of concepts that simultaneously meet the TCAV score greater than 0.6 and the P-Value is less than 0.05 is less than 3, filling in 3 concepts according to the TCAV score; sorting the screened concepts from high to low according to the TCAV score; and selecting the top 3 concepts in the TCAV score as the final screened concepts.

[0010] As an embodiment of the present invention, the number of interpretable performance concepts in the above method is obtained in the following manner: counting the number of screened concepts obtained from three interpretability experiments for each category, concatenating the concept data, and obtaining a 3×N-dimensional feature vector as part of the comprehensive feature vector, where N is the number of categories.

[0011] As an embodiment of the present invention, the interpretable expressive concept features in the above method are obtained in the following manner: for the three concepts in each category, the color features, texture features and edge features of the image samples in the concept are respectively extracted as interpretable expressive concept features.

[0012] As an embodiment of the present invention, in the above method, color features are extracted in the following manner: using the color moment method to extract 9-component color features from each image sample; compressing the 9-component color features into 3 components, representing the first-order color moment, the second-order color moment and the third-order color moment respectively; performing mean processing on the 3-component color features of all image samples in the same concept to obtain the color features of each concept represented by the 3 components; for texture features, they are extracted in the following manner: performing GLCM texture feature extraction on all image samples in each concept respectively; performing mean processing on the extracted GLCM features; selecting variance items and contrast items from the GLCM features after mean processing as the texture features of the concept; for edge features, they are extracted in the following manner: using the Canny operator to perform edge detection on all image samples in each concept respectively; assigning a pixel block in the complete edge to 1; performing mean processing on all assigned edge images in the concept to obtain the edge features of the concept.

[0013] As an embodiment of the present invention, the binary classifier in the above method is trained using a support vector machine method.

[0014] According to a second aspect of the present invention, there is provided an image classification model backdoor defense device based on interpretability, the device comprising: an experimental model training unit, used to build a backdoor experimental environment benchmark, train a clean model and a backdoor model for image classification, the clean model and the backdoor model use the same model structure and the same data set; an interpretable experimental unit, used to perform interpretability experiments on the trained clean model and the backdoor model, obtain the concepts of each category under the data set, and screen the concepts based on the TCAV method or the ACE method; a feature vector construction unit, used to construct a comprehensive feature vector of interpretable expression based on the screened concepts, the comprehensive feature vector includes the number of interpretable expression concepts and interpretable expression concept features; a classifier training unit, used to use the constructed comprehensive feature vector as input to train a binary classifier for distinguishing between a clean model and a backdoor model; a backdoor model judgment unit, used to use the binary classifier to judge whether the image classification model to be tested belongs to a backdoor model.

[0015] As an embodiment of the present invention, the above-mentioned explainable experiment unit performs explainability experiments on the trained clean model and the backdoor model, including: repeating the explainability experiments on the trained clean model and the backdoor model three times under the same experimental conditions.

[0016] As an embodiment of the present invention, the above-mentioned explainable experimental unit screens the concepts based on the TCAV method, including: judging whether the TCAV score is greater than 0.6 and the P-Value is less than 0.05 at the same time; if so, they are screened and stored; if the number of concepts that simultaneously meet the TCAV score greater than 0.6 and the P-Value is less than 0.05 is less than 3, 3 are supplemented according to the TCAV score; the screened concepts are sorted from high to low according to the TCAV score; and the top 3 concepts with the TCAV score are selected as the final screened concepts.

[0017] As an embodiment of the present invention, the above-mentioned number of explainable performance concepts is obtained in the following manner: counting the number of screened concepts obtained from three explainability experiments for each category, concatenating the concept data, and obtaining a 3×N-dimensional feature vector as part of the comprehensive feature vector, where N is the number of categories.

[0018] As an embodiment of the present invention, the above-mentioned interpretable expressive concept features are obtained in the following manner: for the three concepts in each category, the color features, texture features and edge features of the image samples in the concept are respectively extracted as interpretable expressive concept features.

[0019] As an embodiment of the present invention, the feature vector construction unit extracts color features in the following manner: extracts color features of 9 components from each image sample using a color moment method; compresses the color features of the 9 components into 3 components, representing the first-order color moment, the second-order color moment and the third-order color moment respectively; performs mean processing on the color features of the 3 components of all image samples in the same concept to obtain the color features of each concept represented by the 3 components; extracts texture features in the following manner: performs GLCM texture feature extraction on all image samples in each concept respectively; performs mean processing on the extracted GLCM features; selects variance items and contrast items from the GLCM features after mean processing as the texture features of the concept; extracts edge features in the following manner: performs edge detection on all image samples in each concept respectively using the Canny operator; assigns a pixel block in a complete edge to 1; performs mean processing on all assigned edge images in the concept to obtain the edge features of the concept.

[0020] As an embodiment of the present invention, the above binary classifier is trained using a support vector machine method.

[0021] According to a third aspect of the present invention, there is provided an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above method when executing the computer program.

[0022] According to a fourth aspect of the present invention, there is provided a computer-readable storage medium having a computer program stored thereon, wherein the computer program implements the steps of the above method when executed by a processor.

[0023] The present invention proposes an interpretable image classification model backdoor defense method and device. By introducing an interpretable image classification model backdoor defense method, the performance difference of the model under different interpretable methods is utilized to effectively distinguish between clean models and backdoor models, thereby realizing detection and defense against backdoor injection attacks. The method can be applied to a variety of data sets and model structures by constructing a comprehensive feature vector and combining a binary classification algorithm. It has high detection accuracy and practicality, improves the security and reliability of the model, and meets the security needs of the existing field. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the prior art descriptions. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. In the drawings:

[0025] Figure 1 It is a flowchart of a backdoor defense method for an image classification model based on interpretability provided by an embodiment of the present invention;

[0026] Figure 2 is a schematic diagram of a process for screening concepts provided by an embodiment of the present invention;

[0027] Figure 3 is a schematic diagram of a process for extracting color features provided by an embodiment of the present invention;

[0028] Figure 4 is a schematic diagram of a process for extracting texture features provided by an embodiment of the present invention;

[0029] Figure 5 is a schematic diagram of a process for extracting edge features provided by an embodiment of the present invention;

[0030] Figure 6 is a flow chart of a backdoor injection attack detection method based on explainable performance differences provided by an embodiment of the present invention;

[0031] Figure 7 It is a structural schematic diagram of a backdoor defense device for an image classification model based on interpretability provided by an embodiment of the present invention;

[0032] Figure 8 It is a structural schematic diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0033] To make the purpose, technical solution and advantages of the embodiments of the present invention more clear, the embodiments of the present invention are further described in detail below in conjunction with the accompanying drawings. Here, the exemplary embodiments of the present invention and their descriptions are used to explain the present invention, but are not intended to limit the present invention.

[0034] The information collected in the technical solution of this application is information and data authorized by the user or fully authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of relevant data comply with the relevant laws, regulations and standards of relevant countries and regions, take necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0035] The acquisition, transmission, storage, use, and processing of data in the technical solution of this application comply with the relevant provisions of national laws and regulations.

[0036] It should be noted that in the embodiments of the present application, certain software, components, models and other existing solutions in the industry may be mentioned, and they should be regarded as exemplary. Their purpose is only to illustrate the feasibility of implementing the technical solution of the present application, but it does not mean that the applicant has or will necessarily use the solution.

[0037] The current research dilemma of authenticated backdoor defense is related to the "black box" characteristics of deep neural networks. Deep neural networks have a large and complex interlayer structure. Although the complexity of the structure helps to form a strong learning and inference ability, it also leads to insufficient interpretability of the model structure and model decision-making. It has a "black box" characteristic and is difficult for humans to understand, which brings great challenges to the security of artificial intelligence. In recent years, with the deepening of research on model interpretability, we have ways and methods to observe and understand the model, and also provide new ideas for authenticated backdoor defense. Backdoor injection attack is a means of attacking deep learning models. Compared with other attacks such as data poisoning attack, adversarial attack, model extraction attack and model reverse attack, its uniqueness lies in that backdoor attack can occur in various life cycle stages of deep learning, namely data collection, model training and model deployment. Backdoor attack includes two stages: backdoor implantation and backdoor triggering. The former can occur in various stages of deep learning models, while the latter only occurs in the model testing and use stage. Backdoor implantation can be carried out by means of data poisoning and model poisoning. The diversity of backdoor attack stages, attack methods, and attack scenarios also makes it more difficult to prevent backdoor attacks and the defense threshold is higher. At the same time, since the backdoor model only behaves abnormally for image inputs with specific secret triggers, and behaves normally for other clean inputs without triggers, it is difficult to detect during use. Therefore, backdoor attacks are concealed and effective, and their detection is difficult and the harm is huge.

[0038] Therefore, the present invention aims at the backdoor attacks that occur during the data collection and model training phase. It intends to start from the model interpretability, distinguish the clean model from the backdoor model by analyzing the performance differences of the clean model and the backdoor model in different interpretable methods, and complete the identification and defense of the backdoor injection. The research on the backdoor defense method of the image classification model based on interpretability will propose a new identification and defense idea for the backdoor model, which has high application value in actual production and life, and will meet the security needs of the existing field. The present invention has high practical significance and scientific research significance.

[0039] like Figure 1 FIG. 1 is a flow chart of a backdoor defense method for an image classification model based on interpretability provided by an embodiment of the present invention. The method comprises the following steps:

[0040] Step S101: Build a backdoor experimental environment benchmark, train a clean model and a backdoor model for image classification, and use the same model structure and the same data set for the clean model and the backdoor model.

[0041] This application aims at the problem of model backdoor defense in image classification scenarios. In order to ensure the robustness and universality of the designed backdoor detection method, a backdoor experimental environment benchmark is built to train backdoor models with rich structures, wide coverage of tasks, and diverse attack methods, as a preliminary preparation for the backdoor work of image classification models.

[0042] The steps to build the experimental environment benchmark include the configuration of programming languages ​​and application frameworks, the selection and preparation of data sets, the reproduction of backdoor attack methods, and the adjustment and training of model structures. The final backdoor model training experimental environment benchmark should support the configuration of experimental frameworks such as Pytorch, multiple data sets such as Cifar10, GTSRB, ImageNet, Tiny, VggFace, multiple trigger style selections, multiple model structures, and integrate multiple backdoor injection attack methods.

[0043] Although this application can use a variety of model structures and a variety of data sets, it should be pointed out that the clean model and the backdoor model after training are corresponding, that is, the clean model and the backdoor model used in the subsequent interpretability experiment need to be trained with the same model structure and the same data set. And the trained clean model and backdoor model need to achieve a certain success rate, for example, the attack success rate of the backdoor model needs to meet a certain threshold, and the classification success rate of the clean model also needs to reach a certain threshold.

[0044] Step S102: Perform an explainability experiment on the trained clean model and the backdoor model to obtain concepts of each category under the data set, and screen the concepts based on the TCAV method or the ACE method.

[0045] Before explaining this step, let's first explain several concepts involved in this step:

[0046] 1) “Category” refers to the different categories in the image classification task of the dataset. Taking the Cifar10 dataset as an example, it contains 10 categories: airplane, car, bird, cat, deer, dog, frog, horse, ship and truck.

[0047] 2) “Concepts” refer to high-level features or attributes that are extracted from the model through interpretability methods (such as TCAV and ACE) and have practical meaning to humans. These concepts represent the key features that the model relies on when making classification decisions.

[0048] 3) The "TCAV method" is "Testing with Concept Activation Vectors", which is a concept attribution-based interpretability method used to analyze the decision-making process of deep learning models. It quantifies the importance of a concept to the model's decision by calculating the sensitivity of the model's prediction to the underlying high-level concepts learned by the concept activation vector (CAV).

[0049] 4) "ACE method" is "Automatic Concept-based Explanations". It can automatically extract concepts from image models, calculate the importance of different concepts for specific classes, and generate a set of key concept images of a certain class, which helps to more intuitively understand the basis for the model to predict categories.

[0050] In this step, when conducting the model interpretability experiment, for example, 60 clean models and 60 backdoor models with the same model structure and the same data set can be used for the experiment. Of course, the number of models here is only an exemplary selection for the convenience of description, and it does not constitute a limitation on this application. Among them, the embodiment of the present invention can take the following experimental configuration as an example: the data set uses the Cifar10 data set; the backdoor model is injected using the BadNets method, and the AllToOne attack type is used. Its target class covers all 10 categories in the Cifar10 data set, and the backdoor attack success rate is more than 95%; the clean model is used as the control group of the backdoor model, and is trained under the same experimental conditions, and the classification success rate is more than 95%.

[0051] For the above two groups of models (60 clean models and 60 backdoor models), interpretability experiments were conducted on 10 categories of the Cifar10 dataset, namely airplane, automobile, bird, cat, deer, dog, frog, horse, ship, and truck. In addition, the embodiment of the present invention uses two advanced interpretability methods based on concept attribution, namely the TCAV method and the ACE method, to analyze the operation process of the model and obtain concepts in a certain category that have practical significance for humans.

[0052] Preferably, Figure 2 As shown, in this step, screening the concepts based on the TCAV method may further include:

[0053] Step S201: Determine whether the TCAV score is greater than 0.6 and the P-Value is less than 0.05. If so, filter and store them. If the number of concepts that simultaneously meet the TCAV score greater than 0.6 and the P-Value is less than 0.05 is less than 3, fill in 3 more concepts according to the TCAV score.

[0054] Step S202: sort the concepts that pass the screening according to the TCAV scores from high to low.

[0055] Step S203: Select the top three concepts with the highest TCAV scores as the final screened concepts.

[0056] Because when the TCAV method is used to screen concepts, the TCAV score is an important quantitative indicator in the TCAV method. Its inherent meaning is the sensitivity of the model prediction to the underlying high-level concepts learned by the concept activation vector (CAV), which is represented by the directional derivative. For example, for an image classification model that recognizes zebras, the TCAV method can calculate the importance of the concept of "stripes" to the target class "zebra". If the TCAV score is high, the model has a strong ability to recognize the concept, and the model tends to recognize this type of concept. The P-Value value indicates the credibility of the concept classification. If the P-Value value is small, the TCAV score of this type of concept is clearly distinguishable from random concepts and has high credibility.

[0057] Therefore, after obtaining the concept results of different models and different categories, the concepts can be screened by TCAV score and P-Value. If both conditions in step S201 are met at the same time, the concept is considered to meet the standard, passed the screening and stored; if not, it proves that the model has low recognition ability for the concept or low discrimination of the concept, and it can be deleted.

[0058] Further preferably, in order to reduce the impact of instability of image segmentation, the present embodiment repeats the experiment three times under the same experimental conditions to obtain an interpretable result concept. By repeating the experiment, the contingency that may be caused by a single experiment can be reduced, the reliability and stability of the experimental results can be improved, and a more accurate interpretable result concept can be obtained.

[0059] Step S103: constructing a comprehensive feature vector of explainable expressions based on the screened concepts, wherein the comprehensive feature vector includes the number of explainable expression concepts and explainable expression concept features.

[0060] When constructing a comprehensive feature vector of interpretable expression for a model, this application mainly focuses on two aspects: the number of interpretable expression concepts and the characteristics of interpretable expression concepts. Therefore, the comprehensive feature vector of the model mainly includes two parts: the number of concepts and the characteristics of concepts.

[0061] The following describes these two parts in detail:

[0062] (1) Number of interpretable representation concepts

[0063] According to the theory of interpretable performance differences, there are differences in the interpretability effects of the backdoor model and the clean model for the same category. Since the number of concepts that meet the conditions of interpretable performance reflects the interpretability effect of the model for the category, this application uses the number of interpretable performance concepts as a feature to distinguish the interpretability performance of the backdoor model and the clean model.

[0064] Preferably, the present embodiment can obtain the number of interpretable performance concepts in the following manner: count the number of concepts screened from three interpretability experiments for each category, concatenate the concept data, and obtain a 3×N-dimensional feature vector as part of the comprehensive feature vector, where N is the number of categories. For example, for the 10 categories of the Cifar10 dataset, a 3×10-dimensional feature vector can be obtained, which is then used as part of the comprehensive feature vector.

[0065] (2) Explainable representation of conceptual features

[0066] When constructing interpretable conceptual features, this embodiment selects the concepts that best represent the model tendency to reflect the difference between the clean model and the backdoor model in the recognition process. As described in the above steps S201-S203, three concepts with higher TCAV scores in each category are selected.

[0067] Therefore, preferably, in this embodiment, the interpretable expressive concept features are obtained in the following manner: for the three concepts in each category, the color features, texture features and edge features of the image samples in the concept are respectively extracted as the interpretable expressive concept features.

[0068] Since the number of image samples contained in each concept may not be the same, this application extracts 50 image samples from each concept for each of the three concepts in each category. By selecting the same number of image samples, it is possible to ensure that the feature extraction process for each concept is consistent. This unified processing method is helpful for subsequent comparison and analysis, and avoids the introduction of deviations due to different numbers of samples. In addition, selecting a fixed number of samples can control the computational complexity and make the feature extraction process more efficient. If the number of samples for each concept is different, it may cause the processing time of some concepts to be too long, affecting the overall efficiency. Finally, using the same number of samples can ensure that each concept has the same weight in the feature representation, avoiding some concepts from occupying too much or too little weight in the final feature vector due to different numbers of samples.

[0069] The following also uses 50 image samples as an example for explanation. Of course, the selection of the number here does not constitute a limitation to the present application.

[0070] For color features, such as Figure 3 As shown, this embodiment can be specifically extracted by the following methods and steps:

[0071] Step S301: Use the color moment method to extract the color features of 9 components in each image sample.

[0072] In this embodiment, a color moment method is used to process each image sample to extract color features of 9 components, which are usually used to describe the color distribution and characteristics of an image.

[0073] Step S302: compressing the color features of the 9 components into 3 components, which respectively represent the first-order color moment, the second-order color moment and the third-order color moment.

[0074] The extracted 9 color components are compressed into 3 components. These 3 components represent the first-order moment, second-order moment and third-order moment of the color. The first-order moment usually represents the average value of the color, reflecting the overall color tendency of the image; the second-order moment represents the variance of the color, reflecting the breadth of the color distribution; the third-order moment represents the skewness of the color, reflecting the symmetry of the color distribution.

[0075] Step S303: performing mean processing on the color features of the three components of all image samples in the same concept to obtain the color features of each concept represented by the three components.

[0076] The color features of the three components of all image samples in the same concept are processed by averaging the color features of 50 samples to obtain the color features of each concept. This averaging process helps to reduce the impact of individual sample anomalies on the overall features, making the color features of each concept more stable and representative.

[0077] For texture features, such as Figure 4 As shown, this embodiment can be extracted in the following way:

[0078] Step S401: extract GLCM texture features from all image samples in each concept.

[0079] Gray-Level Co-occurrence Matrix (GLCM) texture feature extraction is performed on all image samples in each concept. GLCM is a statistical method used to describe image texture, which extracts texture features by calculating the spatial relationship of pixel gray values ​​in an image.

[0080] Step S402: performing mean processing on the extracted GLCM features.

[0081] The extracted GLCM features are processed by averaging. Since each concept contains 50 samples, averaging the GLCM features of these samples can reduce the impact of individual sample outliers on the overall features, making the features more stable and representative.

[0082] Step S403: Select variance and contrast items from the GLCM features after mean processing as texture features of the concept.

[0083] The variance and contrast items are selected from the GLCM features after mean processing as the texture features of the concepts. These two feature items are selected because they have a large degree of distinction between different concepts and do not contain outliers, and can effectively represent the texture features of the concepts.

[0084] The texture feature of a concept includes the mean of the GLCM features of 50 samples in the concept, each of which is an array of size 32×32, so the texture feature includes a total of 32×32×2 components.

[0085] For edge features, such as Figure 5 As shown, this embodiment can be extracted in the following way:

[0086] Step S501: Use the Canny operator to perform edge detection on all image samples in each concept.

[0087] In this embodiment, the Canny operator is used to perform edge detection on all image samples in each concept. The Canny operator is a commonly used edge detection algorithm and can effectively detect edges in an image.

[0088] Step S502: assigning a value of 1 to the pixel blocks in the complete edge.

[0089] Step S503: performing mean processing on all the edge images after value assignment in the concept to obtain the edge features of the concept.

[0090] Since the mean result of edge features is meaningless, this embodiment processes the results obtained by the Canny operator detection, assigns the pixel blocks in the complete edge to 1, and then performs mean processing on the image samples in the concept, so that the processed concept edge features can be obtained, which can represent the shape characteristics of the concept image. Finally, the edge features of each concept are represented by 32×32 components. These edge features, together with color features and texture features, constitute an important part of the interpretable representation concept features, which are used to construct a comprehensive feature vector to distinguish between backdoor models and clean models.

[0091] Step S104: using the constructed comprehensive feature vector as input to train a binary classifier for distinguishing the clean model from the subsequent model.

[0092] Preferably, the binary classifier in this step can be trained using the Support Vector Machine (SVM). The SVM method can effectively process high-dimensional data through its kernel, and use the kernel function to process nonlinear separable problems, can find complex decision boundaries, has robustness, and is suitable for small-scale high-dimensional feature data in the present invention.

[0093] In the process of hyperparameter tuning of the SVM method, for the polynomial kernel function, the gamma value is set to auto, and its regularization coefficient C and polynomial kernel degree need to be determined. Among them, the regularization parameter C represents the degree of fit of the training data. Too large a value may lead to overfitting, while too small a value may lead to underfitting. The polynomial kernel degree determines the degree of learning of the model for the relationship. A high degree may make the relationship captured by the model too complex, resulting in overfitting. The parameter tuning method used is a combination of random search and grid search. First, through random search, the distribution space of the parameters is narrowed, and the C value is determined to be around 15 and the polynomial kernel is determined to be around 4. Then, using the grid search method, it is determined that when the C value is 10 and the polynomial kernel degree is 5, the training effect of the binary classifier is the best.

[0094] Step S105: using the binary classifier to determine whether the image classification model to be tested is a backdoor model.

[0095] Then, by inputting the image classification model to be tested with the same model structure into the trained binary classifier, it can be determined whether it is a backdoor model.

[0096] The above process can also be found in Figure 6 The description of the above steps vividly shows the process of each step. Figure 6 I will not go into details.

[0097] From the above, it can be seen that the image classification model backdoor defense method based on interpretability proposed in the present invention, by introducing the image classification model backdoor defense method based on interpretability, utilizes the performance difference of the model under different interpretable methods, effectively distinguishes the clean model and the backdoor model, and realizes the detection and defense of backdoor injection attacks. This method can be applied to a variety of data sets and model structures by constructing a comprehensive feature vector and combining a binary classification algorithm. It has high detection accuracy and practicality, improves the security and reliability of the model, and meets the security needs of the existing field.

[0098] like Figure 7 The figure shows a schematic diagram of the structure of an interpretable image classification model backdoor defense device provided by an embodiment of the present application, the device includes: an experimental model training unit 710, an interpretable experimental unit 720, a feature vector construction unit 730, a classifier training unit 740 and a backdoor model judgment unit 750. Among them:

[0099] The experimental model training unit 710 is used to build a backdoor experimental environment benchmark, train a clean model and a backdoor model for image classification, and the clean model and the backdoor model use the same model structure and the same data set.

[0100] The explainable experiment unit 720 is used to perform explainable experiments on the trained clean model and the backdoor model, obtain the concepts of each category under the data set, and screen the concepts based on the TCAV method or the ACE method.

[0101] The feature vector construction unit 730 is used to construct a comprehensive feature vector of explainable expression based on the screened concepts, and the comprehensive feature vector includes the number of explainable expression concepts and explainable expression concept features.

[0102] The classifier training unit 740 is used to use the constructed comprehensive feature vector as input to train a binary classifier for distinguishing the clean model from the subsequent model.

[0103] The backdoor model determination unit 750 is used to use the binary classifier to determine whether the image classification model to be tested belongs to a backdoor model.

[0104] Preferably, the above-mentioned explainable experiment unit 720 performs explainability experiments on the trained clean model and the backdoor model, including: repeating the explainability experiments on the trained clean model and the backdoor model three times under the same experimental conditions.

[0105] Preferably, the above-mentioned interpretable experimental unit 720 screens the concepts based on the TCAV method, including: judging whether the TCAV score is greater than 0.6 and the P-Value is less than 0.05 at the same time; if so, they are screened and stored; if the number of concepts that simultaneously meet the TCAV score greater than 0.6 and the P-Value is less than 0.05 is less than 3, 3 are supplemented according to the TCAV score; the concepts that pass the screening are sorted from high to low according to the TCAV score; and the top 3 concepts in TCAV score are selected as the final screened concepts.

[0106] Preferably, the above-mentioned number of interpretable performance concepts is obtained in the following manner: counting the number of screened concepts obtained from three interpretability experiments for each category, concatenating the concept data, and obtaining a 3×N-dimensional feature vector as part of the comprehensive feature vector, where N is the number of categories.

[0107] Preferably, the above-mentioned interpretable expressive concept features are obtained in the following manner: for the three concepts in each category, the color features, texture features and edge features of the image samples in the concepts are respectively extracted as interpretable expressive concept features.

[0108] Preferably, the feature vector construction unit 730 extracts color features in the following manner: extracts color features of 9 components from each image sample using a color moment method; compresses the color features of the 9 components into 3 components, representing the first-order color moment, the second-order color moment and the third-order color moment respectively; performs mean processing on the color features of the 3 components of all image samples in the same concept to obtain the color features of each concept represented by the 3 components; extracts texture features in the following manner: performs GLCM texture feature extraction on all image samples in each concept respectively; performs mean processing on the extracted GLCM features; selects variance items and contrast items from the GLCM features after mean processing as the texture features of the concept; extracts edge features in the following manner: performs edge detection on all image samples in each concept respectively using the Canny operator; assigns a pixel block in a complete edge to 1; performs mean processing on all assigned edge images in the concept to obtain the edge features of the concept.

[0109] Preferably, the above binary classifier is trained using a support vector machine method.

[0110] The detailed description of each of the above units can be found in the corresponding description in the aforementioned method embodiment, which will not be further elaborated here.

[0111] The present invention proposes an interpretable image classification model backdoor defense device, which introduces an interpretable image classification model backdoor defense method, and utilizes the performance difference of the model under different interpretable methods to effectively distinguish between clean models and backdoor models, thereby realizing the detection and defense of backdoor injection attacks. This method can be applied to a variety of data sets and model structures by constructing a comprehensive feature vector and combining a binary classification algorithm. It has high detection accuracy and practicality, improves the security and reliability of the model, and meets the security needs of the existing field.

[0112] Figure 8 is a schematic diagram of an electronic device provided by an embodiment of the present invention. Figure 8The electronic device shown is a general data processing device, which includes a general computer hardware structure, which at least includes a processor 801 and a memory 802. The processor 801 and the memory 802 are connected via a bus 803. The memory 802 is suitable for storing one or more instructions or programs executable by the processor 801. The one or more instructions or programs are executed by the processor 801 to implement the steps in the above-mentioned method for suppressing temporary overvoltage of a grid-connected static VAR compensator.

[0113] The processor 801 may be an independent microprocessor or a collection of one or more microprocessors. Thus, the processor 801 executes the commands stored in the memory 802 to execute the method flow of the embodiment of the present invention as described above to realize the processing of data and the control of other devices. The bus 803 connects the above-mentioned multiple components together, and at the same time connects the above-mentioned components to the display controller 804 and the display device and the input / output (I / O) device 805. The input / output (I / O) device 805 may be a mouse, a keyboard, a modem, a network interface, a touch input device, a somatosensory input device, a printer, and other devices known in the art. Typically, the input / output (I / O) device 805 is connected to the system via an input / output (I / O) controller 806.

[0114] The memory 802 may store software components such as an operating system, a communication module, an interaction module, and an application program. Each of the modules and applications described above corresponds to a set of executable program instructions that implement one or more functions and methods described in the embodiments of the invention.

[0115] An embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the above-mentioned method for suppressing temporary overvoltage of a grid-connected static VAR compensator are implemented.

[0116] An embodiment of the present invention further provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the steps of the above-mentioned method for suppressing temporary overvoltage of a grid-connected static VAR compensator.

[0117] From the above, it can be seen that the method and device for suppressing temporary overvoltage of a grid-following static VAR compensator proposed in the present invention can effectively suppress the temporary overvoltage problem caused by the delayed exit of the low-through control after fault recovery by optimizing the control parameters of the grid-following static VAR compensator. By optimizing the control parameters, the voltage support capacity of the static VAR compensator is enhanced, and the problems of equipment disconnection and system instability caused by overvoltage are avoided. In addition, this method only needs to optimize the control parameters of the static VAR compensator to effectively avoid the safety risks caused by temporary overvoltage, and can provide theoretical guidance for the capacity and output planning of new energy sites, and support the safe and stable operation of a high-proportion new energy power system.

[0118] The preferred embodiments of the present invention are described above with reference to the accompanying drawings. Many features and advantages of these embodiments are clear from this detailed description, and therefore the claims are intended to cover all such features and advantages of these embodiments that fall within their true spirit and scope. In addition, since many modifications and changes are easily conceivable to those skilled in the art, it is not intended to limit the embodiments of the present invention to the precise structure and operation illustrated and described, but all suitable modifications and equivalents falling within its scope may be covered.

[0119] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0120] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0121] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0122] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0123] The specific embodiments described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A backdoor defense method for image classification models based on interpretability, characterized in that: The method comprises: Build a backdoor experimental environment benchmark, train a clean model and a backdoor model for image classification, and use the same model structure and the same data set for the clean model and the backdoor model; Conduct interpretability experiments on the trained clean model and backdoor model to obtain concepts of each category under the dataset, and screen the concepts based on the TCAV method or the ACE method; Constructing a comprehensive feature vector of interpretable representation based on the screened concepts, wherein the comprehensive feature vector includes the number of interpretable representation concepts and interpretable representation concept features; Using the constructed comprehensive feature vector as input to train a binary classifier for distinguishing between a clean model and a subsequent model; The binary classifier is used to determine whether the image classification model to be tested is a backdoor model.

2. The interpretable image classification model backdoor defense method according to claim 1, characterized in that: The interpretability experiment on the trained clean model and the backdoor model includes: repeating the interpretability experiment on the trained clean model and the backdoor model three times under the same experimental conditions.

3. The interpretable image classification model backdoor defense method according to claim 1, characterized in that: The concepts were screened based on the TCAV method including: Determine whether the TCAV score is greater than 0.6 and the P-Value is less than 0.05 at the same time. If so, filter and store them. If the number of concepts that simultaneously meet the TCAV score greater than 0.6 and the P-Value is less than 0.05 is less than 3, fill in 3 more concepts according to the TCAV score. Sort the concepts that have passed the screening by TCAV scores from high to low; The top three concepts with the highest TCAV scores were selected as the final screened concepts.

4. The interpretable image classification model backdoor defense method according to claim 2, characterized in that: The number of interpretable performance concepts is obtained in the following manner: counting the number of screened concepts obtained from three interpretability experiments for each category, concatenating the number of concepts, and obtaining a 3×N-dimensional feature vector as part of the comprehensive feature vector, where N is the number of categories.

5. The interpretable image classification model backdoor defense method according to claim 3, characterized in that: The interpretable expressive concept features are obtained in the following manner: for the three concepts in each category, the color features, texture features and edge features of the image samples in the concepts are respectively extracted as interpretable expressive concept features.

6. The interpretable image classification model backdoor defense method according to claim 5, characterized in that: The color features are extracted in the following ways: the color features of 9 components are extracted from each image sample using the color moment method; the color features of the 9 components are compressed into 3 components, representing the first-order color moment, the second-order color moment and the third-order color moment respectively; the color features of the 3 components of all image samples of the same concept are averaged to obtain the color features of each concept represented by the 3 components; Texture features are extracted in the following ways: GLCM texture features are extracted from all image samples in each concept; the extracted GLCM features are averaged; variance and contrast items are selected from the averaged GLCM features as the texture features of the concept; Edge features are extracted in the following way: use the Canny operator to perform edge detection on all image samples in each concept; assign a value of 1 to the pixel blocks in the complete edge; perform mean processing on all the edge images after assignment in the concept to obtain the edge features of the concept.

7. The interpretable image classification model backdoor defense method according to claim 1, characterized in that: The binary classifier is trained using a support vector machine method.

8. A backdoor defense device for image classification models based on interpretability, characterized in that: The device comprises: An experimental model training unit, used to build a backdoor experimental environment benchmark, train a clean model and a backdoor model for image classification, wherein the clean model and the backdoor model use the same model structure and the same data set; An interpretable experiment unit, used to conduct interpretable experiments on the trained clean model and the backdoor model, obtain the concepts of each category under the data set, and screen the concepts based on the TCAV method or the ACE method; A feature vector construction unit, used for constructing a comprehensive feature vector of interpretable expression based on the screened concepts, wherein the comprehensive feature vector includes the number of interpretable expression concepts and interpretable expression concept features; A classifier training unit, used for training a binary classifier for distinguishing a clean model from a subsequent model by taking the constructed comprehensive feature vector as input; The backdoor model determination unit is used to use the binary classifier to determine whether the image classification model to be tested belongs to a backdoor model.

9. The interpretable image classification model backdoor defense device according to claim 8, characterized in that: The interpretable experiment unit performs interpretable experiments on the trained clean model and the backdoor model, including: repeating the interpretable experiments on the trained clean model and the backdoor model three times under the same experimental conditions.

10. The interpretable image classification model backdoor defense device according to claim 8, characterized in that: The interpretable experimental unit screens the concepts based on the TCAV method, including: Determine whether the TCAV score is greater than 0.6 and the P-Value is less than 0.05 at the same time. If so, filter and store them. If the number of concepts that simultaneously meet the TCAV score greater than 0.6 and the P-Value is less than 0.05 is less than 3, fill in 3 more concepts according to the TCAV score. Sort the concepts that have passed the screening by TCAV scores from high to low; The top three concepts with the highest TCAV scores were selected as the final screened concepts.

11. The interpretable image classification model backdoor defense device according to claim 9, characterized in that: The number of interpretable performance concepts is obtained in the following manner: counting the number of screened concepts obtained from three interpretability experiments for each category, concatenating the number of concepts, and obtaining a 3×N-dimensional feature vector as part of the comprehensive feature vector, where N is the number of categories.

12. The interpretable image classification model backdoor defense device according to claim 10, characterized in that: The interpretable expressive concept features are obtained in the following manner: for the three concepts in each category, the color features, texture features and edge features of the image samples in the concepts are respectively extracted as interpretable expressive concept features.

13. The interpretable image classification model backdoor defense device according to claim 12, characterized in that: The color features are extracted in the following ways: the color features of 9 components are extracted from each image sample using the color moment method; the color features of the 9 components are compressed into 3 components, representing the first-order color moment, the second-order color moment and the third-order color moment respectively; the color features of the 3 components of all image samples of the same concept are averaged to obtain the color features of each concept represented by the 3 components; Texture features are extracted in the following ways: GLCM texture features are extracted from all image samples in each concept; the extracted GLCM features are averaged; variance and contrast items are selected from the averaged GLCM features as the texture features of the concept; Edge features are extracted in the following way: use the Canny operator to perform edge detection on all image samples in each concept; assign a value of 1 to the pixel blocks in the complete edge; perform mean processing on all the edge images after assignment in the concept to obtain the edge features of the concept.

14. The interpretable image classification model backdoor defense device according to claim 8, characterized in that: The binary classifier is trained using a support vector machine method.

15. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

16. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images