Image classification neural network backdoor removal method and device based on formal verification

Abstract

The application discloses a backdoor removal method and device for an image classification neural network based on formal verification. The application proposes a repair importance metric to more accurately find problem neurons in the neural network that are closely related to the backdoor behavior. Meanwhile, the repair importance is approximately calculated using a formal verification tool, which, compared with a sampling-based calculation method, does not miss key areas and improves positioning accuracy. The application further improves the accuracy of the formal verification tool and reduces the repair importance solution error by refining the neuron range. A micro network structure that can be equivalent to the neuron to be repaired and a corresponding loss function are designed. While removing internal network vulnerabilities, the original performance of the network is effectively maintained.

Description

Technical Field

[0001] This invention relates to the fields of software engineering and artificial intelligence security, and in particular to a method and apparatus for removing backdoors from image classification neural networks based on formal verification. Background Technology

[0002] Deep Neural Networks (DNNs) have been widely applied in various fields such as computer vision and natural language processing. However, the deployment of DNNs in some security-critical areas has increased concerns about their reliability and robustness. For example, attackers can subtly manipulate the input data, causing the DNN to make incorrect decisions; this type of attack is called an adversarial attack. Malicious attackers can also poison the network's training data by attaching a trigger to a portion of the data. When the trained network encounters input with the trigger, it will produce an output pre-set by the malicious attacker. Therefore, the vulnerability of DNNs has raised concerns, especially in security-related fields such as autonomous driving and medical diagnostics.

[0003] Backdoor removal in deep neural networks (DNNs) refers to removing the backdoor from a given neural network with an implanted backdoor and some data by modifying its architecture or parameters. Backdoor attacks, due to their high degree of stealth, have posed a significant threat to the security of neural networks. For example, in image classification neural networks, the backdoor trigger might be a small square hidden in the corner of the input image, often imperceptible to the human eye. To address these potential risks, several backdoor removal methods have been deployed in the application phase of DNNs to improve their reliability, such as retraining, input detection, and data augmentation. However, these methods are highly data-driven and typically perform poorly when the number or quality of available error samples is limited.

[0004] To address this issue, software engineering has recently proposed neuron-level fault location and backdoor removal methods for more effective backdoor attack defense. These methods divide the entire task into two phases: fault location and fault repair. The goal of the fault location phase is to pinpoint a subset of neurons within the network that are responsible for the erroneous behavior; accurately locating these neurons can effectively reduce repair costs. In the fault repair phase, the located problematic neurons are optimized to eliminate the error.

[0005] For backdoor removal methods at the neuron level in deep neural networks (DNNs), the core technical challenge lies in understanding the behavior of the numerous neurons within the DNN on the input data, especially their erroneous behaviors. The second challenge is locating and repairing a small subset of problematic neurons. Previous backdoor removal methods have focused on locating problematic neurons based on their gradients and causal relationships, using heuristic optimization algorithms (such as particle swarm optimization and differential evolution) to find patches to remove the backdoors. While these methods achieve some success in certain scenarios, they still require a large number of input samples. For example, causal relationship-based localization methods require 20,000 input images to characterize the causal relationships between neurons within the network and the backdoor triggers. Furthermore, these methods often suffer from the problem of "catastrophic forgetting," where the normal performance of the model drops significantly after repair, and the network's classification function for normal images without triggers is significantly impaired.

[0006] Formal verification techniques are commonly used to rigorously prove whether a system or software satisfies given specifications in all possible scenarios. Some commonly used formal verification techniques (such as abstract interpretation, satisfiability solving, and stochastic smoothing) have been widely applied to deep neural networks (DNNs) to abstract and model their behavior, and further verify whether the DNN satisfies given properties and specifications. In particular, a reliable abstraction of a DNN can effectively characterize how the network's output changes with the addition of a certain perturbation to the input. Naturally, DNN verification techniques based on abstract interpretation and other similar techniques can be combined with backdoor removal scenarios at the neuron level. By adding perturbations to the neurons within the neural network, changes in the network's output can be precisely captured by formal verification tools. If a small perturbation to a neuron can cause a significant change in the network's backdoor behavior, that neuron can be considered a neuron closely related to the network's backdoor behavior. Summary of the Invention

[0007] The purpose of this invention is to address the shortcomings of existing technologies by proposing a method and apparatus for backdoor removal in image classification neural networks based on formal verification. This invention perturbs the neurons within the neural network and uses formal verification techniques to capture the magnitude of changes in the network output, thereby obtaining the repair importance of each neuron. Simultaneously, this invention proposes a novel repair technique that transforms the problematic neuron into a miniature network for optimization, achieving better repair results compared to other existing backdoor removal methods, while also being less dependent on the amount of available data.

[0008] The present invention solves the related technical problems by adopting the following technical solution: a backdoor removal method for image classification neural networks based on formal verification, the method comprising:

[0009] Part 1: The image classification neural network to be repaired is used as the target model, and the hidden layer index to be repaired in the target model is used as the index of the layer to be repaired. The neuron range and positive and negative sample sets are obtained based on the available image samples, the target model and the index of the layer to be repaired; the neuron range is the output range of the neuron on the given image sample set.

[0010] Part 2: Based on the neuron range, target model, indicators of the layer to be repaired, and formal verification tools, the relaxation lower bound and repair importance are obtained. Specifically, the neuron range is divided into several non-overlapping sub-intervals. In each sub-interval, a formal verification tool is used to approximately calculate the degree of correction that the backdoor behavior of the target model on negative samples can be obtained by perturbing the output of a certain neuron. This is recorded as the relaxation lower bound. The maximum value of the relaxation lower bound in all sub-intervals is recorded as the repair importance of the neuron.

[0011] Part 3: Obtain the neurons to be repaired based on the target model, the indicators of the layer to be repaired, and the importance of repair;

[0012] Part 4: Construct and optimize the micronetwork based on the neurons to be repaired, available image samples, neuron range, and relaxation lower bound;

[0013] The micro-network consists of a first linear layer, an activation function layer, and a second linear layer.

[0014] The loss function of the micro-network consists of two parts. The first part guides the micro-network to be similar to the original output of the neuron to be repaired on positive samples, so that the repaired target model can still correctly classify such samples. The second part guides the micro-network to perform as close as possible to the ideal output on negative samples, so as to remove the backdoor behavior of the target model. The ideal output is: if there is at least one sub-interval with a relaxation lower bound that is always greater than 0, then the sub-interval closest to the original output of the neuron to be repaired is taken as the ideal output; if there is no sub-interval with a relaxation lower bound that is always greater than 0, then the sub-interval with the largest relaxation lower bound is selected as the ideal output.

[0015] Part 5: Replace the neurons to be repaired in the original target model with the optimized micro-network;

[0016] Part 6: Repeat Part 1 through Part 5 until all available negative samples are repaired.

[0017] Furthermore, Part_1 specifically includes:

[0018] Usable image samples refer to usable image data collected during testing and application. These samples can come from common image classification datasets (e.g., MNIST dataset, CIFAR10 dataset, ImageNet dataset, etc.). The MNIST dataset contains 60,000 training samples and 10,000 test samples; the CIFAR10 dataset contains 60,000 RGB color images in 10 categories; and the ImageNet dataset contains over 14 million high-resolution images in 1000 categories.

[0019] The target model refers to a deep neural network model that performs various tasks, such as FNN, CNN, and VGG. FNN refers to a deep neural network composed of several fully connected layers; CNN refers to a convolutional neural network, widely used in image classification tasks, typically consisting of convolutional layers, pooling layers, and fully connected layers. VGG is a further improvement on CNN, achieving stronger image recognition capabilities by changing the convolutional kernel parameters and deepening the network layers. A neural network model can be viewed as a function: f∶R m →R n Here, m is the input dimension and n is the output dimension. Generally, for image classification tasks, n is the number of categories, and m is the number of pixels in the image sample. By feeding the input image into the neural network for prediction, the probability distribution of each category can be obtained, and the category with the highest probability is the predicted category of the neural network.

[0020] The "layer to be repaired" index refers to the index of the hidden layer in the target model that needs to be repaired.

[0021] This invention first puts all available image samples into the target model for prediction, and then classifies the image samples into positive samples and negative samples according to the prediction results. Positive samples refer to samples that the model correctly predicts, and negative samples refer to samples that the model predicts incorrectly.

[0022] In the target model, the output of each neuron changes with the input sample. Specifically, after inputting image sample x into the target model, let h be the output of neuron j located in the i-th layer. ij (x). By feeding available image samples into the target model for computation, the outputs of all internal neurons on a given image sample set can be obtained. Therefore, the output range of neuron j on a given image sample set, i.e., the neuron range, can be represented by the interval I. i,j To indicate:

[0023] I i,j =[-κ·maxx ∈D |h ij (x)|,κ·max x∈D |h ij (x)|]

[0024] Here, κ is a scaling factor greater than or equal to 1, and D is the available image sample set. When the number of available samples is limited, the range of neurons calculated using a finite number of samples may be incomplete. Using the coefficient κ to amplify the original calculation results can alleviate this phenomenon.

[0025] Furthermore, Part_2 specifically includes:

[0026] Formal verification tools refer to verification tools for deep neural networks, such as DeepPoly and CROWN. These tools can over-approximate the neural network output, accurately capturing changes in the neural network's behavior as the input image changes. Taking CROWN as an example, for a given input sample x and a certain perturbation space Δ∈R... m CROWN can linearly approximate the output of a neural network within a given perturbation space, and is constrained by two linear functions as follows:

[0027]

[0028] Among them, g Upper (·) and g Lower (·) represents the upper and lower linear bounds of the original output of the neural network.

[0029] Repair importance refers to the degree of importance of a neuron in a repair task. Neurons with higher repair importance are more closely associated with the backdoor behavior of the network. The repair importance of a neuron can be measured by calculating the degree to which changing its output corrects the backdoor behavior of the target model on negative samples. For a given negative sample x, repair importance is defined as follows:

[0030]

[0031] in, This represents the difference (less than zero) between the correct class and the backdoor class output in the target model's output for a negative sample x. i (x) represents the output of all neurons in the i-th layer of the model for a negative sample x. i (x)[j←v] means h i The j-th dimension component in (x) is replaced with v, that is, the output of the j-th neuron in the i-th layer is changed to v. Intuitively, R i,j (x) measures the extent to which perturbation (repair) of neuron j, for a given range of neurons, eliminates backdoor behavior in the model; neurons with higher repair importance are more effective at removing backdoor behavior.

[0032] Even finding the optimal *v* within a given range of neurons is a non-convex problem. Sampling-based methods often miss crucial regions, leading to inaccurate identification of backdoor-related neurons. Formal verification tools such as CROWN, however, can effectively address this issue. By performing a linear approximation, we can obtain a lower bound:

[0033]

[0034] Among them, g i,j (·)yes The lower bound of the linear function is also called the relaxation lower bound. Using a linear function for approximation can produce significant errors. Dividing the entire neuron region into segments and approximating each segment individually can effectively alleviate this problem. The neuron region is divided into K disjoint sub-intervals as follows:

[0035]

[0036] Each sub-interval is used as the perturbation (repair) space for the neuron, and formal verification tools are used to calculate... The lower bound is used to obtain an approximate value of the repair importance of neuron j on negative sample x:

[0037]

[0038] For each neuron in the i-th layer and a given negative sample x, after calculating the repair importance, in order to ensure that each negative sample has equal importance, the calculation result is normalized, and the repair importance is mapped to [0,1], as shown in the following formula:

[0039]

[0040] After normalization, the repair importance of the neuron that can most effectively correct the network's erroneous behavior on negative sample x will be mapped to 1.

[0041] Furthermore, Part_3 specifically includes:

[0042] For a given negative sample set D m By summing the repair importance of neurons on each negative sample, we can obtain the neuron's importance for the negative sample set D. m The importance of repair:

[0043]

[0044] Intuitively, This study measures the extent to which repairing neuron j improves the backdoor behavior of the model for a given target model, repair layer metrics, and negative sample set. The repair importance of all neurons is ranked, and the neuron with the highest repair importance is selected as the neuron j to be repaired. * :

[0045]

[0046] Furthermore, Part 4 specifically includes:

[0047] Micro Network f mini This is a deep neural network consisting of a first linear layer, an activation function layer, and a second linear layer. The input to the first linear layer is the output of the (i-1)th layer in the original target model, and the output is a scalar. After passing through the activation function layer, the second linear layer performs a linear transformation on the output of the activation function layer. mini It can be formally represented as follows:

[0048]

[0049] Among them, h i-1 Let f represent the output of the (i-1)th layer of the original target model f, and σ(·) represent the activation function used in the original target model. and Represents a micronetwork f mini The parameters of the first linear layer, α and β, represent f mini The parameters of the second linear layer.

[0050] In micronetworks f mini After construction, it is optimized using available image samples and a relaxed lower bound. As described in Part 2, for each neuron j, its neuron range I i,j The image is divided into K sub-intervals and approximated using formal verification tools. For each sample in the available image sample set D, this invention first considers both positive and negative input samples:

[0051] 1) For positive sample x c Since the target model can already correctly classify these samples, we only need to preserve the model's behavior towards these samples as much as possible. First, we feed the positive samples into the original target model and obtain the original target model's behavior towards x. c The output of the (i-1)th layer is f mini The input is given, and the neurons j to be repaired in the original target model are also included. * Output As f mini The target output. This constitutes the target output for the micro-network f. miniA training sample; intuitively, using these training samples to train f mini Training can make f mini The behavior and the neuron to be repaired in the original target model * The behavior is similar, which can achieve the effect of preserving the original performance of the model.

[0052] 2) For negative sample x m Since the original target model misclassifies this sample, it is necessary to remove the backdoor behavior of the model for this sample and repair the neuron j. * Correction is required. Specifically, f mini The input remains the original target model for x. m The output at the (i-1)th layer. And f mini The ideal output requires further calculation. This invention further subdivides this situation into two sub-cases, for the neuron j to be repaired... * and the extent of its neurons:

[0053] a) If there exist several sub-intervals where the relaxation lower bound on these sub-intervals is always greater than 0: In this case, the present invention considers all sub-intervals satisfying the condition as being j distances from the neuron to be repaired. * The nearest subinterval to the original output is taken as the ideal output, that is:

[0054]

[0055] Where T is the set of all subintervals whose relaxation lower bound is always greater than 0, and d is the distance function, when If it is located outside the interval, calculate its distance to the nearest point in the interval. If it lies within the interval, its value is 0. Intuitively, if the neuron j to be repaired... * The output of falls within a subinterval of set T. Due to the robustness of the formal verification tool, the original target model is valid for sample x. m The backdoor behavior on the top can definitely be removed. Meanwhile, the sub-interval... This can ensure the distance j from the neuron to be repaired * The original output is recent.

[0056] b) If there is no relaxed lower bound that is always greater than 0 for any subinterval: In this case, regardless of the neuron j to be repaired... * The output is changed to the neuron range. Any value within the range cannot necessarily represent the negative sample x. m The backdoor in the [system / mechanism] is removed. Therefore, this invention selects the method with the highest repair effect, namely... The largest subinterval is taken as the ideal output, that is:

[0057]

[0058] Thus, for each available image sample, this invention converts it into a corresponding f. mini A training sample is used. Subsequently, this invention designs a loss function. To guide the optimization of f mini Specifically, It consists of two parts, the first part instructing f mini On positive samples, with the neuron j to be repaired * The original outputs are similar, which allows the repaired target model to still correctly classify these samples, preserving the original performance of the model as much as possible and effectively avoiding the "catastrophic forgetting" phenomenon; the second part guides f mini The performance on negative samples should be as close as possible to the ideal output to remove backdoor behavior from the model. Loss function The formal definition is as follows:

[0059]

[0060]

[0061]

[0062] Among them, D c D m These are the positive and negative sample sets, respectively.

[0063] Using classic optimization methods such as stochastic gradient descent to optimize f mini In the loss function By optimizing the above, we can obtain the optimized micro-network f. mini .

[0064] Furthermore, Part 5 specifically includes:

[0065] In micronetworks f mini After optimization, the neurons to be repaired in the original target model need to be replaced with f. mini This invention considers both the input weight and the output weight of the neuron to be repaired. The input weight refers to the weight that connects to a neuron from the previous layer, and the output weight refers to the weight that connects out of a neuron.

[0066] a) Due to the input weight of the neuron to be repaired and f mini The first linear layer has the same number of weights, so the input weights of the neuron to be repaired can be directly replaced with f. mini The first linear layer.

[0067] b)f miniThe second linear layer contains only a single linear mapping. Therefore, this linear mapping can be equivalently absorbed by changing the outgoing weights of the neuron to be repaired and the biases of the neurons in the layer following it.

[0068] Parts 1 through 5 completed one round of repair. However, not all backdoor behaviors in negative samples can be eliminated by repairing a single neuron. Therefore, after one round of repair, it is necessary to test with available image samples. If the target model still exhibits backdoor behavior on some negative samples, further repair of the target model is required.

[0069] The present invention also provides a backdoor removal device for image classification neural networks based on formal verification, including a memory and one or more processors. The memory stores executable code, and when the processor executes the executable code, it is used to implement the above-mentioned image classification neural network backdoor removal method based on formal verification.

[0070] The present invention also provides a computer-readable storage medium having a program stored thereon, which, when executed by a processor, implements the above-described method for removing backdoors from image classification neural networks based on formal verification.

[0071] The beneficial effects of this invention are as follows:

[0072] 1. A metric called "repair importance" is proposed to more accurately locate problematic neurons within neural networks that are closely related to backdoor behavior. Formal verification tools are used to approximate the repair importance, which, compared to sampling-based methods, avoids missing key regions and improves localization accuracy.

[0073] 2. By refining the neuron range, the accuracy of the formal verification tool was further improved, and the error in solving the importance of repair was reduced. Furthermore, a miniature network structure and corresponding loss function that can be equivalently replaced with the neurons to be repaired were designed. This effectively maintained the original performance of the network while removing internal vulnerabilities. Attached Figure Description

[0074] Figure 1 This is a schematic diagram of the main process of the image classification neural network backdoor removal method based on formal verification provided in this embodiment of the invention;

[0075] Figure 2 This is the main structural diagram of the micro-network in Part_4 provided in the embodiments of the present invention;

[0076] Figure 3 This is a structural diagram of the image classification neural network backdoor removal device based on formal verification provided in an embodiment of the present invention. Detailed Implementation

[0077] The implementation of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. The following embodiments are only used to illustrate the present invention and should not be used to limit the scope of the present invention.

[0078]

Example 1

[0079] The target model in this example is VGG 13, a convolutional neural network consisting of 10 convolutional layers and 3 linear layers. The image classification dataset used in this example is CIFAR-10, where the images to be classified are 32*32*3 color images with ten different categories. The input to VGG 13 is a 32*32*3 pixel matrix. Based on the given input, a well-trained VGG 13 network can provide high-accuracy classification results. This example uses 1000 positive samples and 1000 backdoor samples as available data samples.

[0080] Furthermore, the symbols and their meanings required in this example are as follows: f represents the target model, i.e., the VGG 13 network trained on the CIFAR-10 dataset, which was subjected to a backdoor attack during the training phase; D represents the available image sample set, where D = D c ∪D m D c Let D represent the positive sample set. m Represents the negative sample set; i represents the target model layer index; j represents the neuron index within layer i; I i,j The range of neurons in neuron j; h ij (x) represents the output of neuron j in the i-th layer when sample x is input into the target model; h i (x) represents the output of the i-th layer neuron when sample x is input into the target model; R i,j (x) represents the repair importance of neuron j on sample x; This represents the approximate R calculated using formal verification tools. i,j The lower bound of (x); This represents the difference between the correct and incorrect class outputs in the target model's output for a negative sample x; g i,j (·) represents the formal verification tool for... The approximate result is a piecewise linear function; K represents the number of interval divisions; j * The neuron to be repaired; f mini A miniature network to be constructed.

[0081] In this example, the target model layer is selected as layer 12, the scaling factor κ is 1, and the number of interval divisions K is 10.

[0082] Figure 1This is a schematic diagram illustrating the main process of the image classification neural network backdoor removal method based on formal verification provided in this embodiment of the invention. First, the neuron range and positive / negative sample sets are determined. The available image sample set D is then fed into the target model for prediction, yielding a predicted label for each sample. Samples with the same predicted label as the true label are then placed into the dataset D. c Conversely, put it in D m For the 1024 neurons in the 12th layer of the target model f, their neuron ranges are calculated as follows:

[0083] I 12,j =[-1·max x∈D |h 12j (x)|,1·max x∈D |h 11j (x)|]1≤j≤1024

[0084] The repair importance of each neuron is then calculated. In this step, only negative samples D are used. m For each neuron, the neuronal range I 12,j They are all divided into 10 disjoint subintervals:

[0085]

[0086] After the sub-intervals are divided, the formal verification tool CROWN is used to approximate the importance of repairs on each sub-interval:

[0087]

[0088]

[0089] Since CROWN approximates repair importance as a piecewise linear function, its maximum value can be easily obtained by calculating the values ​​at the endpoints of each linear function. After calculating the repair importance of all neurons, normalization maps the repair importance to the [0,1] interval. This is to prevent excessively high repair importance on a single sample, which could affect the overall ranking of neurons and lead to a decrease in fault location accuracy. By normalizing, each negative sample is assigned an equal weight, thereby more accurately locating the problematic neuron.

[0090] After all samples have been computed, the normalized repair importance of each neuron on each sample is summed. The summations are then sorted to obtain the neuron j to be repaired. * Intuitively, for neuron j * Correcting the output can maximally fix the backdoor behavior of the network on a given negative sample.

[0091] Based on the identified neurons to be repaired and the target model, a miniature network f can be constructed. mini In this example, the micronetwork f mini It consists of a first linear layer, an activation function layer, and a second linear layer. The input to the first linear layer is the output of the 11th layer in the original target model. After passing through the first linear layer and the activation function layer, f... mini The second linear layer performs a single linear transformation (without an activation function). Formally, f in this example... mini The working principle can be expressed as:

[0092]

[0093] Among them, h 11 (x) represents the output of all neurons in the 11th layer of the original target model for a negative sample x; and Represents a micronetwork f mini The parameters of the first linear layer, α and β, represent f mini The parameters of the second linear layer.

[0094] In micronetworks f mini After construction, this method optimizes it based on the determined relaxation lower bound and available image samples. Specifically, for each available image sample, this method transforms it into a function corresponding to f. mini A training sample. For the positive sample x c First, positive samples are fed into the original target model, and the original target model is obtained for x. c The output of the 11th layer is f mini The input is given, and the neurons j to be repaired in the original target model are also included. * Output As f mini The target output. Intuitively, this newly constructed sample is intended to make f mini On positive samples, the neuron j to be repaired in the original target model exhibits similar behavior. * Consistent. For negative sample x m Because the original target model misclassified the sample, the neuron j to be repaired in the original target model... * This behavior needs to be corrected. Specifically, f mini The input is the original target model for x. m In the output of layer 11, and f mini The ideal output can be further subdivided into two sub-cases for consideration.

[0095] Case 1: For the neuron j to be repaired *If, within the range of the neuron, there exist several sub-intervals where the relaxation lower bound on these sub-intervals is always greater than 0, then in this case, the present invention considers all sub-intervals satisfying the condition as being j distances from the neuron to be repaired. * The nearest subinterval to the original output is taken as the ideal output, that is:

[0096]

[0097] Where T is the set of all subintervals whose relaxation lower bound is always greater than 0, and d is the distance function, when If it is located outside the interval, calculate its distance to the nearest point in the interval. If it lies within the interval, its value is 0. Intuitively, if the neuron j to be repaired... * The output of falls within a subinterval of set T. Due to the robustness of the formal verification tool, the original target model is valid for sample x. m The backdoor behavior on the top can definitely be removed, thus achieving the repair effect. Meanwhile, the sub-interval... This can ensure the distance j from the neuron to be repaired * The original output is closest to the model's original performance, so as to preserve the model's original performance as much as possible.

[0098] Case 2: If there is no relaxation lower bound that is always greater than 0 for any sub-interval, in this case, regardless of changing the output of the neuron jw to be repaired to the neuron range... No value within the range can necessarily represent a negative sample x. m The backdoor in the code was removed. Therefore, this method selects the one with the highest repair effect, namely... The largest sub-interval is taken as the ideal output, so that the model's erroneous behavior is corrected to the greatest extent possible, that is:

[0099]

[0100] For each available image sample, this invention converts it into a corresponding f. mini One training sample. In this example, the present invention transforms all 1000 available samples into samples that can be used for f. mini The training samples. Then, the loss function designed according to the present invention. For f mini Optimize. Specifically, It consists of two parts, the first part instructing f mini On positive samples, with the neuron j to be repaired * The original output is similar; the second part guides f mini The performance on negative samples should be as close as possible to the ideal output to correct the model's erroneous behavior. Loss function In this example, it is defined as:

[0101]

[0102] In this example, this method uses the Adam optimizer to optimize f. mini Optimization is performed. The Adam optimizer minimizes the objective function based on the negative gradient direction. The learner rate is set to 0.01, and the number of iterations is 20.

[0103] After optimization, the neurons to be repaired in the original target model need to be replaced with f. mini This invention considers both the input and output weights of the neuron to be repaired:

[0104] a) Due to the input weight of the neuron to be repaired and f mini The first linear layer has the same number of weights, so the input weights of the neuron to be repaired can be directly replaced with f. mini The first linear layer.

[0105] b)f mini The second linear layer contains only a single linear mapping. Therefore, this linear mapping can be equivalently absorbed by changing the output weight w of the neuron to be repaired and the bias b of the neurons in the next layer after the neuron to be repaired, such as... Figure 2 As shown, that is:

[0106] b←b+βw

[0107] w←αw

[0108] The erroneous behavior of the neurons to be repaired was corrected without changing the network structure.

[0109] Repeat the steps above—"obtain neuron range—calculate repair importance and relaxation lower bound—obtain neurons to be repaired—build and optimize a mini-network—equivalent replacement"—until all samples violating the specification are repaired. After repair, use 5000 samples from the CIFAR-10 dataset's test set to test the performance of the repaired model. Specifically, use 5000 test samples to test the model's normal classification performance; attach backdoor triggers to these samples to form backdoor samples to test whether the model's backdoor behavior has been effectively removed. The test results are shown in the table below:

[0110] Table 1 Repair Results

[0111] Repair network VGG 13 Number of available data 2000 Number of test data 10000 Normal classification accuracy before repair 82.46％ Improve the success rate of front and back door attacks 99.97％ Normal classification accuracy after repair 81.63％ Improved backdoor attack success rate 0.15％ Repair time 101.88s

[0112] See Figure 3The present invention provides an image classification neural network backdoor removal device based on formal verification, comprising a memory and one or more processors. The memory stores executable code, and when the processor executes the executable code, it is used to implement the image classification neural network backdoor removal method based on formal verification in the above embodiment.

[0113] The embodiment of the image classification neural network backdoor removal device based on formal verification of the present invention can be applied to any device with data processing capabilities, such as a computer. The device embodiment can be implemented in software, hardware, or a combination of both. Taking software implementation as an example, as a logical device, it is formed by the processor of any data processing device loading the corresponding computer program instructions from non-volatile memory into memory for execution. From a hardware perspective, such as... Figure 3 The diagram shown is a hardware structure diagram of any data processing-capable device, including the image classification neural network backdoor removal device based on formal verification according to the present invention. (Except for...) Figure 3 In addition to the processor, memory, network interface, and non-volatile memory shown, any data processing device in the embodiment may also include other hardware depending on the actual function of the data processing device, which will not be described in detail here.

[0114] The specific implementation process of the functions and roles of each unit in the above device can be found in the implementation process of the corresponding steps in the above method, and will not be repeated here.

[0115] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of the present invention according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0116] This invention also provides a computer-readable storage medium storing a program thereon, which, when executed by a processor, implements the image classification neural network backdoor removal method based on formal verification described in the above embodiments.

[0117] The computer-readable storage medium can be an internal storage unit of any data processing device as described in any of the foregoing embodiments, such as a hard disk or memory. The computer-readable storage medium can also be an external storage device of any data processing device, such as a plug-in hard disk, smart media card (SMC), SD card, flash card, etc., equipped on the device. Furthermore, the computer-readable storage medium can include both internal storage units and external storage devices of any data processing device. The computer-readable storage medium is used to store the computer program and other programs and data required by the data processing device, and can also be used to temporarily store data that has been output or will be output.

[0118] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

[0119] The specific embodiments described above illustrate the technical solution and beneficial effects of the present invention in detail. It should be understood that the above description is only the most preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, additions, and equivalent substitutions made within the scope of the principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A backdoor removal method for image classification neural networks based on formal verification, characterized in that, include: S1, the image classification neural network to be repaired is used as the target model, and the hidden layer index to be repaired in the target model is used as the index of the layer to be repaired. The neuron range and positive and negative sample sets are obtained based on the available image samples, the target model and the index of the layer to be repaired; the neuron range is the output range of the neuron on the given image sample set. S2. Based on the neuron range, target model, indicators of the layer to be repaired, and formal verification tools, obtain the relaxation lower bound and repair importance. Specifically, the neuron range is divided into several non-overlapping sub-intervals. In each sub-interval, a formal verification tool is used to approximately calculate the degree of correction that the backdoor behavior of the target model on negative samples can be obtained by perturbing the output of a certain neuron. This is recorded as the relaxation lower bound. The maximum value of the relaxation lower bound in all sub-intervals is recorded as the repair importance of the neuron. S3: Obtain the neurons to be repaired based on the target model, the indicators of the layer to be repaired, and the importance of repair. S4: Construct and optimize a micronetwork based on the neurons to be repaired, available image samples, neuron range, and relaxation lower bound; The micro-network consists of a first linear layer, an activation function layer, and a second linear layer. The loss function of the micro-network consists of two parts. The first part guides the micro-network to be similar to the original output of the neuron to be repaired on positive samples, so that the repaired target model can still correctly classify such samples. The second part guides the micro-network to perform on negative samples as close as possible to the ideal output in order to remove the backdoor behavior of the target model. The ideal output is: if there is at least one sub-interval with a relaxation lower bound that is always greater than 0, then the sub-interval closest to the original output of the neuron to be repaired is taken as the ideal output; if there is no sub-interval with a relaxation lower bound that is always greater than 0, then the sub-interval with the largest relaxation lower bound is selected as the ideal output. S5, based on the optimized micro-network, replace the neurons to be repaired in the original target model; S6, repeat S1 to S5 until all available negative samples are repaired.

2. The image classification neural network backdoor removal method based on formal verification according to claim 1, characterized in that, S1 specifically involves: inputting all available image samples into the target model for prediction, and classifying the image samples into positive and negative samples based on the prediction results; after inputting image sample x into the target model, the output of neuron j located in the i-th layer is denoted as h. ij (x); By feeding available image samples into the target model for computation, the outputs of all internal neurons on a given image sample set are obtained; the output range of neuron j on a given image sample set is represented by interval I. i,j To indicate: I i,j =[-κ·max x∈D |h ij (x)|,k·max x∈D |h ij (x)|] Where κ is a scaling factor greater than or equal to 1, and D is the available image sample set.

3. The image classification neural network backdoor removal method based on formal verification according to claim 2, characterized in that, In S2, the repair importance of a neuron is measured by calculating the degree to which changing the output of a certain neuron corrects the backdoor behavior of the target model on negative samples; for a given negative sample x, the repair importance R of neuron j in the i-th layer is... i,j (x) is defined as follows: in, h represents the difference between the correct class and the backdoor class output in the target model's output for a negative sample x; i (x) represents the output of all neurons in the i-th layer of the target model for a negative sample x; h i (x)[j←v] means h i The j-th dimension component in (x) is replaced with v, that is, the output of the j-th neuron in the i-th layer is changed to v.

4. The image classification neural network backdoor removal method based on formal verification according to claim 3, characterized in that, In S2, formal verification tools are used to verify... The relaxation lower bound g is obtained by performing a linear approximation. i,j (v): The range of the neuron is divided into K disjoint sub-intervals: Each sub-interval is used as the perturbation space for the neuron, and formal verification tools are used to calculate... The lower bound is used to obtain an approximation of the repair importance of neuron j on negative sample x. For each neuron in the i-th layer and a given negative sample x, the computation results are normalized, and the repair importance is mapped to [0,1].

5. The image classification neural network backdoor removal method based on formal verification according to claim 4, characterized in that, S3 specifically refers to: For a given negative sample set D m The repair importance of neurons on each negative sample is summed to obtain the neuron's importance for the negative sample set D. m The importance of repair The repair importance of all neurons is ranked, and the neuron with the highest repair importance is selected as the neuron to be repaired. * , 6. The image classification neural network backdoor removal method based on formal verification according to claim 5, characterized in that, The micronetwork is denoted as f. mini The input to the first linear layer is the output of the (i-1)th layer in the original target model, and the output is a scalar. After passing through the activation function layer, the second linear layer performs a linear transformation on the output of the activation function layer; f mini The formal representation is as follows: Among them, h i-1 Let f(t) represent the output of the (i-1)th layer of the original target model f, and σ(·) represent the activation function used in the original target model. and Represents a micronetwork f mini The parameters of the first linear layer, α and β, represent f mini The parameters of the second linear layer.

7. The image classification neural network backdoor removal method based on formal verification according to claim 6, characterized in that, Using available image samples and relaxed lower bounds, the micronetwork f mini Optimize; for each neuron j, its neuron range I i,j It is divided into K sub-intervals and approximated using formal verification tools; for each sample in the available image sample set D, it is transformed into f mini Consider a training sample, starting from both positive and negative input samples: 1) For positive sample x c First, positive samples are fed into the original target model, and the original target model is obtained for x. c The output of the (i-1)th layer is f mini The input is given, and the neurons j to be repaired in the original target model are also included. * Output As f mini The target output; 2) For negative sample x m f mini The input is the original target model for x. m The output of the (i-1)th layer, f mini The ideal output is as follows: a) If there exist several sub-intervals where the relaxation lower bound on these sub-intervals is always greater than 0, in this case, the distance j from the neuron to be repaired in all sub-intervals that satisfy the condition is... * The nearest subinterval to the original output is taken as the ideal output, that is: Where T is the set of all subintervals whose relaxation lower bound is always greater than 0, and d is the distance function, when If it is located outside the interval, calculate its distance to the nearest point in the interval. If it lies within the interval, its value is 0; b) If there is no lower bound for relaxation in any subinterval that is always greater than 0, then the option with the highest repair effect is selected. The largest subinterval is taken as the ideal output, that is:

8. The image classification neural network backdoor removal method based on formal verification according to claim 7, characterized in that, The loss function of the micro-network The formal definition is as follows: Among them, D c D m These are the positive and negative sample sets, respectively.

9. The method for removing backdoors from image classification neural networks based on formal verification according to claim 1, characterized in that, After the micronetwork optimization is completed, the neurons to be repaired in the original target model are replaced with the micronetwork, considering both the input and output weights of the neurons to be repaired: a) Since the number of input weights of the neuron to be repaired is exactly the same as the number of weights in the first linear layer of the micro-network, the input weights of the neuron to be repaired are directly replaced with f. mini The first linear layer; b) The second linear layer of the micronetwork contains only a single linear mapping, which is equivalently absorbed by changing the outgoing weights of the neuron to be repaired and the bias of the neurons in the layer following the neuron to be repaired.

10. A backdoor removal device for image classification neural networks based on formal verification, comprising a memory and one or more processors, wherein the memory stores executable code, characterized in that... When the processor executes the executable code, it is used to implement the image classification neural network backdoor removal method based on formal verification as described in any one of claims 1-9.

Images