Deep neural network model backdoor detection method in data-free scene and related equipment

Abstract

The invention discloses a deep neural network model backdoor detection method in a data-free scene and related equipment. The method comprises the following steps: selecting the last linear layer of a to-be-detected deep neural network model as a detection layer; constructing initial potential representation vectors with different types of labels, and inputting the initial potential representation vectors into a detection layer; wherein the different category labels are output category labels of the to-be-tested deep neural network model; optimizing the initial potential representation vectors to obtain convergence rounds of the initial potential representation vectors under different types of labels; on the basis of convergence round numbers of the initial potential representation vectors under different category labels, abnormal measurement is constructed; and determining whether the to-be-tested deep neural network model has a backdoor based on the adaptive threshold and the anomaly metric, and determining a backdoor type, a target category and a source category. According to the method, the complex backdoor attack can be detected only through the linear layer of the last layer of the deep neural network model, and the target category, the backdoor type and the source category of the specific backdoor attack can be identified.

Description

Technical Field

[0001] The present invention belongs to the field of artificial intelligence security, and specifically relates to a backdoor detection method for deep neural network models in a data-free scenario and related equipment. Background Art

[0002] Deep neural networks (DNNs) have demonstrated impressive performance in a variety of real-world applications, including image classification, speech recognition, and natural language processing. In safety-critical environments, such as autonomous driving system models powered by DNNs, ensuring the security of DNN models is crucial in practice. Numerous studies have demonstrated that DNNs are vulnerable to backdoor attacks. Typically, attackers can incorporate carefully crafted triggers into training samples, causing the DNN model to learn the connection between the triggers and harmful behaviors throughout the training process. A typical example is road sign detection in autonomous vehicles, where a compromised model might incorrectly interpret a no-overtaking sign as a speed limit of 100 (the attacker's target class), potentially leading to a major traffic accident. These incidents highlight the need for effective mechanisms to mitigate the threat of backdoors to DNNs.

[0003] To address the aforementioned issues, various backdoor detection strategies have been proposed. The goal of backdoor detection is to determine whether a DNN model has been injected with a backdoor and further identify the target class. Existing methods can be divided into two categories based on whether original samples are required during detection: data-requiring methods and data-free methods. Data-requiring methods typically assume that the detector has access to clean or poisoned samples, such as neural cleaning (NC), which inverts triggers from clean image samples to detect backdoor attacks. However, obtaining training samples for trained models is challenging for detectors, especially poisoned samples, such as pre-trained models from the AWS marketplace. Furthermore, data-requiring methods often assume that the detector knows the backdoor pattern. This assumption reduces their effectiveness in identifying complex backdoor attacks in real-world scenarios. Compared to data-requiring methods, data-free methods construct a latent representation for each class to replace the original samples. They then adjust the latent representation and capture anomalies in the model output for backdoor detection, such as FreeEagle's higher posterior output for the target class. However, we found that the performance of data-free methods significantly degrades when the backdoor model is initialized with pre-trained weights. For example, FreeEagle's detection accuracy is generally low, while MMBD may mistakenly detect all benign models as backdoor models. Furthermore, they often rely on static thresholds to distinguish between benign and backdoor models. Static thresholds significantly limit the generalization of backdoor detection in different scenarios. For example, despite using the threshold recommended by the original method, multiple abnormal indicators of non-target categories still exceed the threshold. These incidents highlight the need for more general data-free backdoor detection to protect DNN models. Summary of the Invention

[0004] In response to the problems existing in the prior art, the present invention provides a deep neural network model backdoor detection method and related equipment in a data-free scenario. Only the last linear layer of the deep neural network model is required to detect complex backdoor attacks, and it can identify the target category, backdoor type, and source category of a specific type of backdoor attack.

[0005] In order to solve the above technical problems, the present invention is implemented through the following technical solutions:

[0006] According to a first aspect of the present invention, a method for detecting backdoors in a deep neural network model in a data-free scenario is provided, comprising:

[0007] Select the last linear layer of the deep neural network model to be tested as the detection layer;

[0008] Constructing an initial potential representation vector with different category labels, and inputting the initial potential representation vector into the detection layer; wherein the different category labels are the output category labels of the deep neural network model to be tested;

[0009] Optimizing the initial potential representation vector to obtain the number of convergence rounds of the initial potential representation vector under different category labels;

[0010] constructing an anomaly metric based on the number of convergence rounds of the initial latent representation vector under the different class labels;

[0011] Based on the adaptive threshold and the anomaly metric, determine whether the deep neural network model to be tested has a backdoor, and determine the backdoor type, target category, and source category.

[0012] In a possible implementation of the first aspect, optimizing the initial potential representation vector to obtain the number of convergence rounds of the initial potential representation vector under different category labels is specifically:

[0013] Before the optimization begins, calculate the loss value of the initial potential representation vector corresponding to all category labels on the detection layer and select the minimum loss value;

[0014] Iteratively optimize the initial potential representation vector using an Adam optimizer with a set learning rate;

[0015] When the loss value drops to the minimum loss value, reset the learning rate for regularization until the loss value is lower than the preset loss threshold;

[0016] The number of convergence rounds of the initial potential representation vector under each category label is recorded, and the number of convergence rounds of the initial potential representation vector under different category labels is obtained, which is expressed as:

[0017]

[0018] Where E is the number of convergence rounds of the initial potential representation vector under different category labels; e i represents the number of convergence rounds for the i-th class label; n is the number of class labels.

[0019] In a possible implementation of the first aspect, constructing an anomaly metric based on the number of convergence rounds of the initial potential representation vector under the different category labels is specifically:

[0020] The number of convergence rounds of the initial potential representation vectors under the different category labels is sorted in ascending order to generate a sequence, which is expressed as:

[0021]

[0022] Calculate the first-order difference of the series to obtain the anomaly measure, which is expressed as:

[0023]

[0024]

[0025] Among them, Es is the sequence generated by sorting; represents the convergence round value at the jth position in the sequence Es; D is the abnormality measure; Represents the convergence round value at the j+1th position in the sequence Es.

[0026] In a possible implementation of the first aspect, determining whether the deep neural network model to be tested has a backdoor based on the adaptive threshold and the anomaly metric, and determining the backdoor type, target category, and source category, specifically:

[0027] If the first condition is met, it is determined that the deep neural network model to be tested has a class-independent backdoor. At this time, the target category of the class-independent backdoor is the category label corresponding to the minimum number of convergence rounds in the sequence, and the source category is all category labels except the category label corresponding to the target category;

[0028] If the second condition is met, it is determined that the deep neural network model to be tested has a class-specific backdoor. In this case, the target category of the class-specific backdoor is the category label corresponding to the minimum number of convergence rounds in the sequence Es, and the source category is the category label corresponding to the maximum number of convergence rounds in the sequence;

[0029] If the first condition and the second condition are not met, it is determined that the deep neural network model to be tested does not have a backdoor;

[0030] The first condition is:

[0031] d1≥d n-1and

[0032] The second condition is:

[0033] d1 <d n-1 and

[0034] Among them, σ D is the standard deviation of D, σ1 is The standard deviation of m is the adaptive threshold; σ n-1 for The standard deviation of .

[0035] In a possible implementation of the first aspect, a method for determining the adaptive threshold is:

[0036] m=nlog2σ1。

[0037] In a possible implementation manner of the first aspect, the initial potential representation vector is an all-1 vector.

[0038] According to a second aspect of the present invention, a device for detecting backdoors in a deep neural network model in a data-free scenario is provided, comprising:

[0039] A selection module is used to select the last linear layer of the deep neural network model to be tested as the detection layer;

[0040] A latent representation vector construction input module is used to construct an initial latent representation vector with different category labels and input the initial latent representation vector into the detection layer; wherein the different category labels are the output category labels of the deep neural network model to be tested;

[0041] An optimization module, configured to optimize the initial potential representation vector to obtain the number of convergence rounds of the initial potential representation vector under different category labels;

[0042] an anomaly metric selection module, configured to construct an anomaly metric based on the number of convergence rounds of the initial potential representation vector under the different class labels;

[0043] A backdoor determination module is used to determine whether the deep neural network model to be tested has a backdoor based on an adaptive threshold and the anomaly metric, and to determine the backdoor type, target category, and source category.

[0044] According to a third aspect of the present invention, a computer device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the method for detecting backdoors in a deep neural network model in a data-free scenario is implemented.

[0045] According to a fourth aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method for detecting backdoors in a deep neural network model in a data-free scenario is implemented.

[0046] According to a fifth aspect of the present invention, a computer program product is provided, which, when executed by a processor, implements the method for detecting backdoors in a deep neural network model in a data-free scenario.

[0047] Compared with the prior art, the present invention has at least the following beneficial effects:

[0048] The present invention provides a backdoor detection method for a deep neural network model in a data-free scenario, which does not require access to original samples and does not rely on clean or poisoned samples, effectively solving the problem of difficulty in data acquisition. The present invention selects the last linear layer of the deep neural network model to be tested as the detection layer. This layer does not rely on pre-trained weights, and its parameters are only determined by the current training data, avoiding the interference of pre-training features on the detection results, thereby improving the accuracy of detection and maintaining good detection performance in different scenarios. The last linear layer is directly associated with the classification result and is not affected by the input size or model depth. Regardless of how the input size of the model changes or how the model depth is adjusted, the present invention can effectively perform backdoor detection, reduce the dependence of the detection method on the model structure, and improve the versatility of the method. The present invention is the first method to propose backdoor detection using convergence speed in a data-free scenario. By analyzing the number of convergence rounds of the initial potential representation vector under different category labels, an anomaly metric is constructed, which can detect complex backdoor attacks. Existing methods have difficulty in determining target categories in backdoor detection, while the present invention is the first detection method that can effectively detect target categories in a data-free scenario, and can accurately understand the scope of influence and degree of harm of backdoor attacks, so as to facilitate the adoption of targeted defense measures. The present invention is the first detection method that can detect source categories in a data-free scenario, and can trace the source of backdoor attacks and locate the source of attacks. The present invention can distinguish between class-specific and class-independent attack methods, so as to formulate targeted defense strategies and improve the model's ability to resist different types of backdoor attacks. The present invention is the first detection method that uses an adaptive threshold in a data-free scenario. The adaptive threshold can automatically adjust the judgment criteria according to different models and scenarios. Compared with fixed thresholds, it can better adapt to complex and changeable actual situations, improve the accuracy of backdoor detection, and reduce the possibility of misjudgment and missed judgment.

[0049] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the technical solutions in the specific embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the specific embodiments. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0051] Figure 1 This is a flow chart of a method for detecting backdoors in a deep neural network model in a data-free scenario according to the present invention;

[0052] Figure 2 A flow chart of the number of convergence rounds (convergence speed) for obtaining potential representation vectors corresponding to different category labels in the present invention;

[0053] Figure 3 A flowchart for determining the backdoor attack method based on the number of convergence rounds and deriving the target category and source category;

[0054] Figure 4 The convergence round heat map of each layer of the end-to-end training model and the pre-training model of the embodiment;

[0055] Figure 5 This is a heat map of the convergence rounds of the training model for different poisoning datasets in the embodiment. DETAILED DESCRIPTION

[0056] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of them. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0057] Combine Figures 1 to 3 As shown, an embodiment of the present invention provides a method for detecting backdoors in a deep neural network model in a data-free scenario, which specifically includes the following steps:

[0058] Step 1: Select the last linear layer of the deep neural network model to be tested as the detection layer.

[0059] Specifically, the detection layer is chosen as the last layer of the model, i.e. the linear layer. This layer does not rely on pre-trained weights, and its parameters are determined only by the current training data, thus avoiding interference from pre-trained features.

[0060] Step 2: construct an initial potential representation vector with different category labels, and input the initial potential representation vector into the detection layer; wherein the different category labels are the output category labels of the deep neural network model to be tested.

[0061] For example, taking a deep neural network model with 10 output categories as an example, assuming that these 10 category labels are category 1, category 2, ..., category 10, then 10 initial potential representation vectors corresponding to these 10 category labels are constructed.

[0062] In a preferred embodiment, the initial latent representation vector is an all-ones vector. Specifically, to eliminate initialization bias, the latent representation vector is initialized to an all-ones vector. This initialization strategy ensures that the latent representation vector is far from the decision boundary of any class in feature space, resulting in better detection accuracy. Experiments have shown that random or zero vector initialization reduces the detection accuracy (DMA) of benign models, while an all-ones vector performs optimally in terms of convergence speed and detection stability.

[0063] Step 3: Optimize the initial potential representation vector to obtain the number of convergence rounds of the initial potential representation vector under different category labels.

[0064] For example, for the initial latent representation vector corresponding to category 1, if the convergence condition is reached after 50 iterations, then the number of convergence rounds corresponding to category 1 is 50; for the initial latent representation vector corresponding to category 2, if the convergence condition is reached after 40 iterations, then the number of convergence rounds corresponding to category 2 is 40, and so on, the number of convergence rounds of the initial latent representation vectors under different category labels is obtained.

[0065] In one possible implementation, the initial latent representation vector is optimized to obtain the number of convergence rounds of the initial latent representation vector under different class labels. The specific optimization process is as follows:

[0066] Step 3.1. Before the optimization begins, calculate the loss value of the initial potential representation vector corresponding to all category labels on the detection layer, select the minimum loss value, and record the minimum loss value as L S =min(L).

[0067] Specifically, the calculation of the loss value is based on the difference between the output of the model and the corresponding category label. For example, the cross entropy loss function can be used for calculation.

[0068] Step 3.2: Use the Adam optimizer to iteratively optimize the initial potential representation vector φ0 with a set learning rate η.

[0069] Step 3.3: When the loss value is reduced to the minimum loss value, reset the learning rate η for regularization to prevent the optimization of the potential representation vector from converging too fast or too slow for a certain class until the loss value is lower than the preset loss threshold L. f .

[0070] That is, in each epoch, first use Perform forward propagation and calculate the loss value Then, the gradient of φ0 is updated at each step as follows:

[0071]

[0072] in, represents the potential representation vector value of the t-th iteration, represents the potential representation vector value of the t+1th iteration, Represents the model input The predicted result output later, y i represents the target output of the i-th sample, n represents the total number of samples, Indicates the loss function of the t-th iteration in φ c Gradient in direction.

[0073] Step 3.4: Record the number of convergence rounds of the initial potential representation vector under each category label, and obtain the number of convergence rounds of the initial potential representation vector under different category labels, expressed as:

[0074]

[0075] Where E is the number of convergence rounds of the initial potential representation vector under different category labels; e i represents the number of convergence rounds for the i-th class label; n is the number of class labels.

[0076] Through the above steps, the optimization of the initial latent representation vector can be completed, and the number of convergence rounds under different category labels can be obtained, providing basic data for anomaly measurement and backdoor detection.

[0077] Step 4: construct an anomaly metric based on the number of convergence rounds of the initial potential representation vector under the different category labels.

[0078] In one possible implementation, an anomaly metric is constructed based on the number of convergence rounds of the initial potential representation vector under the different class labels, specifically:

[0079] First, the number of convergence rounds of the initial potential representation vectors under the different category labels is sorted in ascending order to generate a sequence E s , expressed as:

[0080]

[0081] Among them, E s The sequence generated for sorting; Represents sequence E s The convergence round value at the j-th position in ;

[0082] Then, the first-order difference of the series is calculated to obtain the anomaly measure, which is expressed as:

[0083]

[0084] Where D is the abnormality measure; Represents sequence E s The convergence round value at the j+1th position in .

[0085] That is, calculate the sequence E s The first-order difference of and The abnormal distance from other elements, where yes The jth element of and Respectively and The corresponding value.

[0086] Step 5: Based on the adaptive threshold and the anomaly metric, determine whether the deep neural network model to be tested has a backdoor, and determine the backdoor type, target category, and source category.

[0087] In one implementation, based on the adaptive threshold and the anomaly metric, it is determined whether the deep neural network model to be tested has a backdoor, and the backdoor type, target category, and source category are determined as follows:

[0088] If the first condition is met, the first condition is: d1≥d n-1 and σ D is the standard deviation of D, σ1 is , m is the adaptive threshold, then it is determined that the deep neural network model to be tested has a class-independent backdoor.

[0089] At this time, the target category of the class-independent backdoor is the category label corresponding to the minimum number of convergence rounds in the sequence Es, and the source category is all category labels except the category label corresponding to the target category.

[0090] It should be noted that The variance of quantifies the fluctuation caused by d, i.e., the fluctuation associated with the underlying target class. It indicates that when the potential target class shows larger fluctuations than the non-target class, the model is identified as having injected a class-independent backdoor.

[0091] If the second condition is met, the second condition is: d1 <d n-1 and Among them, σ n-1 for , it is determined that the deep neural network model to be tested has a class-specific backdoor.

[0092] At this time, the target category of the class-specific backdoor is the category label corresponding to the minimum number of convergence rounds in the sequence Es, and the source category is the category label corresponding to the maximum number of convergence rounds in the sequence Es.

[0093] If the first condition and the second condition are not met, it is determined that the deep neural network model to be tested does not have a backdoor, that is, the deep neural network model to be tested is detected as benign.

[0094] In one possible implementation, the method for determining the adaptive threshold is:

[0095] m=n log2σ1.

[0096] Specifically, in the above formula, m is an adaptive indicator (i.e., adaptive threshold) that distinguishes benign models from models with backdoors, rather than the static threshold used in existing research. The intuitive expression of the abnormal indicator is It characterizes the degree of instability of non-target categories. However, it cannot measure the degree of instability introduced by the number of categories. According to the research content of this invention, whether it is injected with a class-independent backdoor or a class-specific backdoor, the anomaly exhibited by the model with the backdoor is correlated with the number of categories and the number of source categories. Therefore, the adaptive threshold m is positively correlated with the number of categories, and is used Finally, the expression of the adaptive threshold is derived, that is, m = nlog2σ1.

[0097] Exemplarily, the method of the present invention is used to perform backdoor detection. Figure 4 The two heatmaps show the number of convergence rounds for each layer of the end-to-end training model and the pre-trained model respectively. Both models were injected with a backdoor with the target class as 1. Each heatmap normalizes the number of convergence rounds to between 0 and 1, with lighter colors indicating fewer convergence rounds. The x-axis represents the 10 classification labels of the ResNet18 model trained on the CIFAR10 dataset, and the y-axis represents each layer of the model. Figure 4 As can be seen, in the end-to-end training model, the target category achieves significantly fewer convergence rounds than other categories across all layers, validating the universality of this method. In the pre-trained model, however, the target category achieves a different convergence round than some layers, indicating that pre-trained weights can affect the convergence characteristics of specific network layers. We quantified the impact of pre-trained weights on backdoor detection. However, fully connected layers are less likely to be affected, demonstrating the superiority of this method's layer selection. Figure 5The figure shows the convergence rounds heatmap collected by the ResNet18 model trained on the CIFAR10 dataset. From left to right, they are the training cases using a clean dataset, a category-independent backdoor dataset, and a category-specific backdoor dataset. Each heatmap normalizes the convergence rounds to between 0 and 1, with lighter colors indicating fewer convergence rounds. The x-axis represents the 10 categories of the CIFAR10 dataset, and the target classes of these 10 categories are set to categories 1 to 10 respectively. The y-axis represents the 10 ResNet18 models with backdoors injected for different categories. Figure 5 It can be seen that compared with the model trained with the clean dataset, the number of convergence rounds of the target class in the model with the backdoor is significantly smaller than that of other classes. This feature can be used as an important basis for detecting backdoors.

[0098] In another embodiment of the present invention, a device for detecting backdoors in a deep neural network model in a data-free scenario is provided, specifically comprising:

[0099] The selection module is used to select the last linear layer of the deep neural network model to be tested as the detection layer.

[0100] A latent representation vector construction input module is used to construct an initial latent representation vector with different category labels and input the initial latent representation vector into the detection layer; wherein the different category labels are the output category labels of the deep neural network model to be tested.

[0101] An optimization module is used to optimize the initial potential representation vector to obtain the number of convergence rounds of the initial potential representation vector under different category labels.

[0102] The anomaly metric selection module is used to construct an anomaly metric based on the number of convergence rounds of the initial potential representation vector under the different category labels.

[0103] A backdoor determination module is used to determine whether the deep neural network model to be tested has a backdoor based on an adaptive threshold and the anomaly metric, and to determine the backdoor type, target category, and source category.

[0104] All relevant contents of each step involved in the embodiment of the aforementioned method for backdoor detection of a deep neural network model in a data-free scenario can be referred to the functional description of the functional module corresponding to the device for backdoor detection of a deep neural network model in a data-free scenario in the embodiment of the present invention, and will not be repeated here. The division of modules in the embodiment of the present invention is schematic and is only a logical function division. There may be other division methods in actual implementation. In addition, the functional modules in the various embodiments of the present invention can be integrated into one processor, or they can exist physically separately, or two or more modules can be integrated into one module. The above-mentioned integrated modules can be implemented in the form of hardware or in the form of software functional modules.

[0105] In another embodiment of the present invention, a computer device is provided, which includes a processor and a memory, wherein the memory is used to store a computer program, the computer program includes program instructions, and the processor is used to execute the program instructions stored in the computer storage medium. The processor may be a central processing unit (CPU), or may be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. It is the computing core and control core of the terminal, which is suitable for implementing one or more instructions, specifically suitable for loading and executing one or more instructions in a computer storage medium to implement a corresponding method flow or corresponding function; the processor described in the embodiment of the present invention can be used for the operation of a deep neural network model backdoor detection method in a data-free scenario.

[0106] In another embodiment of the present invention, the present invention further provides a storage medium, specifically a computer-readable storage medium (Memory), which is a memory device in a computer device for storing programs and data. It is understandable that the computer-readable storage medium here can include both built-in storage media in the computer device and, of course, extended storage media supported by the computer device. The computer-readable storage medium provides a storage space, which stores the operating system of the terminal. In addition, one or more instructions suitable for being loaded and executed by the processor are also stored in the storage space. These instructions can be one or more computer programs (including program codes). It should be noted that the computer-readable storage medium here can be a high-speed RAM memory or a non-volatile memory, such as at least one disk memory. The processor can load and execute one or more instructions stored in the computer-readable storage medium to implement the corresponding steps of the method for detecting backdoors in a deep neural network model in a data-free scenario in the above embodiment.

[0107] It will be understood by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0108] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0109] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0110] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0111] The present invention also provides a computer program product, which is used to execute any of the above-described methods for detecting backdoors in a deep neural network model in a data-free scenario. Since the computer program product provided by the present invention and the above-described method for detecting backdoors in a deep neural network model in a data-free scenario are based on the same inventive concept, the computer program product provided by the present invention has all the advantages of the above-described method for detecting backdoors in a deep neural network model in a data-free scenario, and therefore the beneficial effects of the computer program product provided by the present invention will not be detailed here.

[0112] In the present invention, the terms "one embodiment", "some embodiments", "examples", "specific examples", or "some examples" mean that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine different embodiments or examples described in this specification and the features of different embodiments or examples without contradiction.

[0113] Finally, it should be noted that the above-described embodiments are only specific implementations of the present invention, which are used to illustrate the technical solutions of the present invention, rather than to limit them. The scope of protection of the present invention is not limited thereto. Although the present invention has been described in detail with reference to the above-described embodiments, those skilled in the art should understand that any person skilled in the art can modify or easily conceive of changes to the technical solutions described in the above-described embodiments within the technical scope disclosed by the present invention, or replace some of the technical features therein with equivalents. Such modifications, changes, or replacements do not deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection of the claims.

Claims

1. A method for detecting backdoors in deep neural network models in a data-free scenario, characterized in that: include: Select the last linear layer of the deep neural network model to be tested as the detection layer; Constructing an initial potential representation vector with different category labels, and inputting the initial potential representation vector into the detection layer; wherein the different category labels are the output category labels of the deep neural network model to be tested; Optimizing the initial potential representation vector to obtain the number of convergence rounds of the initial potential representation vector under different category labels; constructing an anomaly metric based on the number of convergence rounds of the initial latent representation vector under the different class labels; Based on the adaptive threshold and the anomaly metric, determine whether the deep neural network model to be tested has a backdoor, and determine the backdoor type, target category, and source category.

2. The method for detecting backdoors in a deep neural network model in a data-free scenario according to claim 1, characterized in that: The number of convergence rounds of optimizing the initial potential representation vector to obtain the initial potential representation vector under different category labels is specifically: Before the optimization begins, calculate the loss value of the initial potential representation vector corresponding to all category labels on the detection layer and select the minimum loss value; Iteratively optimize the initial potential representation vector using an Adam optimizer with a set learning rate; When the loss value drops to the minimum loss value, reset the learning rate for regularization until the loss value is lower than the preset loss threshold; The number of convergence rounds of the initial potential representation vector under each category label is recorded, and the number of convergence rounds of the initial potential representation vector under different category labels is obtained, which is expressed as: Where E is the number of convergence rounds of the initial potential representation vector under different category labels; e i represents the number of convergence rounds for the i-th class label; n is the number of class labels.

3. The method for detecting backdoors in a deep neural network model in a data-free scenario according to claim 2, wherein: The number of convergence rounds of the initial potential representation vector under the different category labels is used to construct an anomaly metric, specifically: The number of convergence rounds of the initial potential representation vectors under the different category labels is sorted in ascending order to generate a sequence, which is expressed as: Calculate the first-order difference of the series to obtain the anomaly measure, which is expressed as: Among them, Es is the sequence generated by sorting; represents the convergence round value at the jth position in the sequence Es; D is the abnormality measure; Represents the convergence round value at the j+1th position in the sequence Es.

4. The method for detecting backdoors in a deep neural network model in a data-free scenario according to claim 3, wherein: The method of determining whether the deep neural network model to be tested has a backdoor based on the adaptive threshold and the anomaly metric, and determining the backdoor type, target category, and source category is specifically as follows: If the first condition is met, it is determined that the deep neural network model to be tested has a class-independent backdoor. At this time, the target category of the class-independent backdoor is the category label corresponding to the minimum number of convergence rounds in the sequence, and the source category is all category labels except the category label corresponding to the target category; If the second condition is met, it is determined that the deep neural network model to be tested has a class-specific backdoor. In this case, the target category of the class-specific backdoor is the category label corresponding to the minimum number of convergence rounds in the sequence Es, and the source category is the category label corresponding to the maximum number of convergence rounds in the sequence; If the first condition and the second condition are not met, it is determined that the deep neural network model to be tested does not have a backdoor; The first condition is: and The second condition is: and Among them, σ D is the standard deviation of D, σ1 is The standard deviation of m is the adaptive threshold; σ n-1 for The standard deviation of .

5. The method for detecting backdoors in a deep neural network model in a data-free scenario according to claim 4, characterized in that: The method for determining the adaptive threshold is: m=nlog2σ1。 6. The method for detecting backdoors in a deep neural network model in a data-free scenario according to claim 1, characterized in that: The initial potential representation vector is an all-1 vector.

7. A backdoor detection device for deep neural network models in a data-free scenario, characterized in that: include: A selection module is used to select the last linear layer of the deep neural network model to be tested as the detection layer; A latent representation vector construction input module is used to construct an initial latent representation vector with different category labels and input the initial latent representation vector into the detection layer; wherein the different category labels are the output category labels of the deep neural network model to be tested; An optimization module, configured to optimize the initial potential representation vector to obtain the number of convergence rounds of the initial potential representation vector under different category labels; an anomaly metric selection module, configured to construct an anomaly metric based on the number of convergence rounds of the initial potential representation vector under the different class labels; A backdoor determination module is used to determine whether the deep neural network model to be tested has a backdoor based on an adaptive threshold and the anomaly metric, and to determine the backdoor type, target category, and source category.

8. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, it implements the method for detecting backdoors in a deep neural network model in a data-free scenario as described in any one of claims 1 to 6.

9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, it implements a method for detecting backdoors in a deep neural network model in a data-free scenario as described in any one of claims 1 to 6.

10. A computer program product, characterized in that When the computer program product is executed by a processor, it implements a method for detecting backdoors in a deep neural network model in a data-free scenario as described in any one of claims 1 to 6.

Images