Backdoor risk assessment method and device
By performing trigger reconstruction and backdoor repair training on the deep forger detector, it evaluates its risk value in the backdoor detection and repair stage, solving the problem of lack of effective backdoor risk assessment methods in the prior art, and realizing backdoor risk assessment and defense of the deep forger detection model.
Patent Information
- Application Number
- CN202411408509.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-10
- Publication Date
- 2025-03-11
AI Technical Summary
The lack of effective risk assessment methods in the prior art to deal with backdoor threats in deep forgery detection models, making it difficult for models to detect and defend against backdoor attacks.
By reconstructing the depth forger detector based on the first data set, the trigger set is obtained, and the second data set is used for backdoor repair training, the risk value of the detector in the backdoor detection and repair stages is evaluated, and the degree of backdoor risk of the detector is finally determined.
A comprehensive and accurate assessment of the backdoor risk of deep forgery detection models is achieved, providing an effective method to detect and defend against potential backdoor attacks, improving the security and reliability of the model.
Smart Images
Figure CN119598447A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of data security technology, and in particular to a backdoor risk assessment method and device. Background Art
[0002] At present, in order to cope with the increasing sophistication of counterfeiting technology and the rapid expansion of cyberspace, deep learning-based deep fake detection models have emerged, such as deep fake detectors. These models use deep learning networks to extract and classify features, but at the same time, they also face the inherent security risks of neural networks, such as data poisoning, adversarial attacks, and backdoor attacks.
[0003] At present, deep fake detection methods are mainly divided into two categories: detection methods based on macroscopic visual artifacts and detection methods based on microscopic semantic features. The former focuses on specific visual artifacts generated during the video forgery process, and uses deep learning technology to extract these features that may be visible or imperceptible to the naked eye, thereby judging the authenticity of the face in the video. The latter focuses on microscopic semantic features that are difficult to discern with the naked eye, and uses carefully designed deep neural networks to distinguish the subtle differences between real and forged videos.
[0004] However, current deepfake detection work still has significant deficiencies, especially in the assessment of backdoor threats. Most existing assessment work focuses on aspects such as model accuracy and robustness, but lacks sufficient attention to the special threat of backdoor attacks. Backdoor attacks can produce malicious output only for specific inputs without affecting the normal performance of the model on clean samples. Since this attack method can cleverly bypass common evaluation methods, there is currently no effective risk assessment method to deal with backdoor threats. This situation urgently needs to be changed to ensure the security and reliability of deepfake detection models. Summary of the invention
[0005] The present invention provides a backdoor risk assessment method and device to solve the technical problems existing in the prior art.
[0006] The present invention provides a backdoor risk assessment method, the method comprising: Based on the first data set, the trigger is reconstructed by applying it to the deep fake detector to obtain a trigger set, wherein the first data set includes first sample data without a trigger, and the trigger set includes candidate triggers corresponding to various labels in a label set of at least one classifier of the deep fake detector and the norm of the candidate triggers; Based on the set of triggers, determining a first risk value of the deep fake detector in a backdoor detection phase; Based on the second data set, applying it to the deep fake detector to perform backdoor repair training to obtain a trained target deep fake detector, wherein the second data set includes second sample data without a trigger and third sample data with a trigger; Determining a second risk value of the deep fake detector in a backdoor repair phase based on the first performance indicator value of the deep fake detector and the second performance indicator value of the target deep fake detector; Based on the first risk value and the second risk value, determining the backdoor risk level of the deep fake detector.
[0007] According to the backdoor risk assessment method provided by the present invention, the first data set is applied to the deep fake detector to reconstruct the trigger to obtain the trigger set, including: For each class of labels in the label set of the classifier of the deep fake detector, based on the optimization objective, reconstruct a candidate trigger using the first sample data corresponding to each class of labels in the first data set, and determine the norm of each of the candidate triggers; Obtain a trigger set based on the candidate triggers of each type of label in the label set of the classifier of the deep fake detector and the norm of the candidate triggers; The optimization goal is that when the candidate trigger is added to the first sample data, the classifier of the deep fake detector will misclassify the first sample data and minimize the norm of the candidate trigger.
[0008] According to the backdoor risk assessment method provided by the present invention, determining the first risk value of the deep fake detector in the backdoor detection stage based on the trigger set includes: Determining the median absolute deviation of the norms of all the candidate triggers in the trigger set; Determining abnormal labels in a label set of a classifier of the deep fake detector based on the median absolute deviation and the norm of the candidate trigger for each class of labels in the trigger set; Based on abnormal labels in a label set of a classifier of the deep fake detector, a first risk value of the deep fake detector in a backdoor detection phase is determined.
[0009] According to the backdoor risk assessment method provided by the present invention, the first performance indicator value includes a first clean accuracy rate and a first attack success rate, the second performance indicator value includes a second clean accuracy rate and a second attack success rate, and the second risk value of the deep fake detector in the backdoor repair stage is determined based on the first performance indicator value of the deep fake detector and the second performance indicator value of the target deep fake detector, including: Determine a clean accuracy difference between the first clean accuracy and the second clean accuracy; determining an attack success rate difference between the first attack success rate and the second attack success rate; Based on the attack success rate difference and the clean accuracy rate difference, a second risk value of the deep fake detector in the backdoor repair stage is determined.
[0010] According to the backdoor risk assessment method provided by the present invention, the second data set is obtained in the following manner: Randomly select a first proportion of first sample data from the first data set as second sample data; Randomly selecting a second proportion of first sample data from the first data set, and adding a preset trigger to the second proportion of the first sample data as third sample data; A second data set is obtained based on the second sample data and the third sample data.
[0011] According to the backdoor risk assessment method provided by the present invention, determining the backdoor risk level of the deep fake detector based on the first risk value and the second risk value includes: Determining a total risk value based on the first risk value and the second risk value; Determining a backdoor risk level of the deep fake detector based on the total risk value; The total risk value is obtained in the following way: R = α × R de +(1-α)R re ; α is the preset balance parameter, R de is the first risk value, R re is the second risk value, and R is the total risk value.
[0012] The present invention also provides a backdoor risk assessment device, comprising: A first backdoor risk assessment module is configured to reconstruct triggers in a deep fake detector based on a first data set to obtain a trigger set, wherein the first data set includes first sample data without triggers, and the trigger set includes candidate triggers corresponding to various labels in a label set of at least one classifier of the deep fake detector and the norm of the candidate triggers; A second backdoor risk assessment module, configured to determine a first risk value of the deep fake detector in a backdoor detection phase based on the trigger set; A third backdoor risk assessment module is used to apply the deep fake detector to perform backdoor repair training based on a second data set to obtain a trained target deep fake detector, wherein the second data set includes second sample data without a trigger and third sample data with a trigger; a fourth backdoor risk assessment module, configured to determine a second risk value of the deep fake detector in a backdoor repair phase based on the first performance indicator value of the deep fake detector and the second performance indicator value of the target deep fake detector; A fifth backdoor risk assessment module is used to determine the backdoor risk level of the deep fake detector based on the first risk value and the second risk value.
[0013] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, a backdoor risk assessment method as described in any one of the above is implemented.
[0014] The present invention also provides a non-transitory computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the backdoor risk assessment method as described in any one of the above is implemented.
[0015] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the backdoor risk assessment method as described in any one of the above is implemented.
[0016] The backdoor risk assessment method provided by the present invention first reconstructs the trigger of the deep fake detector based on the first sample data without the trigger, and obtains a trigger set, which can accurately simulate possible backdoor triggers and provide a basis for subsequent risk assessment. By determining the first risk value of the deep fake detector in the backdoor detection stage based on the trigger set, the risk level of the detector when facing potential backdoor attacks can be quantitatively evaluated. Then, by using a second data set including second sample data without the trigger and third sample data with the trigger, the deep fake detector is trained for backdoor repair. By comparing the performance index values before and after the repair training, the risk level during the repair stage can be further evaluated. Finally, the risk values of the backdoor detection stage and the backdoor repair stage are comprehensively considered to achieve a comprehensive and accurate assessment of the backdoor risk level of the deep fake detector. BRIEF DESCRIPTION OF THE DRAWINGS
[0017] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0018] Figure 1 It is a flow chart of the backdoor risk assessment method provided by the present invention.
[0019] Figure 2 It is a structural schematic diagram of the backdoor risk assessment device provided by the present invention.
[0020] Figure 3 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION
[0021] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0022] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the terms used in this way are interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first" and "second" are generally of the same type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally represents that the objects associated with each other are in an "or" relationship.
[0023] The embodiment of the present invention provides a backdoor risk assessment method. Figure 1 As shown, the steps 110 to 150 are included.
[0024] Step 110, based on the first data set, applying to the deep fake detector to reconstruct the trigger to obtain a trigger set, wherein the first data set includes first sample data without a trigger, and the trigger set includes candidate triggers corresponding to various labels in a label set of at least one classifier of the deep fake detector and the norm of the candidate triggers; It should be understood that the deep fake detector in this embodiment is a deep learning-based technical tool specifically used to detect whether images and videos have been processed with deep fakes.
[0025] Here, a trigger refers to a specific input pattern or signal that is designed to activate or trigger a backdoor attack in a deepfake detector. A backdoor attack is a malicious behavior in which an attacker embeds a specific trigger during the model training phase, causing the model to make incorrect predictions or leak sensitive information when receiving input containing the trigger.
[0026] First, construct the first sample data (i.e., clean sample data) that does not carry any triggers. For example, extract 2,000 sample data labeled as real and 2,000 sample data labeled as forged from the clean data set to form the first data set. All sample data in the first data set do not carry triggers, and the labels are consistent with the actual situation of the sample data.
[0027] Next, these first sample data are input into the deep fake detector, which learns and reconstructs candidate triggers that may be used to manipulate the detection results. Through this learning process, a trigger set is obtained, which contains multiple candidate triggers and their corresponding norms.
[0028] Here, the norm of a candidate trigger refers to a mathematical indicator that measures the size or strength of the candidate trigger. The calculation of the norm usually depends on the specific form and representation method of the candidate trigger, as well as the selected norm type, which can be flexibly set according to needs and is not limited to this. For example, for an image trigger, the norm can be calculated based on the sum of the absolute values of the pixel values (such as the L1 norm), the square root of the sum of the squares of the pixel values (such as the L2 norm), etc.
[0029] Step 120, determining a first risk value of the deep fake detector in a backdoor detection phase based on the trigger set; Here, the first risk value refers to the extent to which the detection capability of the detector may be affected when facing a possible backdoor attack. This risk value is determined by analyzing the impact of the trigger set on the classification result of the detector.
[0030] Step 130, based on a second data set, applying the deep fake detector to perform backdoor repair training to obtain a trained target deep fake detector, wherein the second data set includes second sample data without a trigger and third sample data with a trigger; First, obtain the second sample data without triggers. This part of the data is normal sample data, which does not contain any triggers used to manipulate the detection results. These samples are used to train the detector to identify normal, untampered deep fake content. Obtain the third sample data with triggers. This part of the data is samples with triggers artificially added to normal samples. These samples simulate backdoor attacks and are used to train the detector to identify and resist the influence of triggers.
[0031] Next, the second sample data without the trigger and the third sample data with the trigger are mixed together to form a balanced second data set. This is done to ensure that the detector can learn the features of both normal samples and trigger samples during training.
[0032] The mixed second data set is then used to perform backdoor repair training on the deep fake detector. Specifically, the second data set is used to perform backdoor repair training on a specific network layer responsible for classification in the deep fake detector. After sufficient training, a deep fake detector trained with backdoor repair is finally obtained, namely, the target deep fake detector.
[0033] Step 140, determining a second risk value of the deep fake detector in the backdoor repair phase based on the first performance indicator value of the deep fake detector and the second performance indicator value of the target deep fake detector; Here, the performance index value refers to the performance index of the deep fake detector used for evaluation on a specific task or dataset, including but not limited to accuracy, recall rate, F1 score, AUC value, etc.
[0034] In this embodiment, the difference between the first performance indicator value and the second performance indicator value before and after the backdoor repair is calculated to determine the second risk value of the deep fake detector in the backdoor repair stage.
[0035] Step 150: Determine the backdoor risk level of the deep fake detector based on the first risk value and the second risk value.
[0036] In this embodiment, when determining the backdoor risk level of the deep fake detector, two key risk values are comprehensively considered: the first risk value and the second risk value. For example, a comprehensive backdoor risk level is calculated by weighted average, product or other complex mathematical models.
[0037] Specifically, in one example, it includes: Determining a total risk value based on the first risk value and the second risk value; Based on the total risk value, determine the backdoor risk level of the deep fake detector, including
[0038] The total risk value is obtained in the following way: R = α × R de +(1-α)R re ; α is the preset balance parameter, R de is the first risk value, R re is the second risk value, and R is the total risk value.
[0039] It should be noted here that the preset balance parameter α is a value between 0 and 1, which is used to adjust R de and R re The weight in the total risk value R can be flexibly set according to actual needs.
[0040] After calculating the total risk value R, the backdoor risk level of the deep fake detector can be judged based on its size. For example, a series of risk thresholds can be set and the value of R can be compared with them to determine the backdoor risk level (such as low risk, medium risk, high risk, etc.).
[0041] The backdoor risk assessment method provided by the present invention first reconstructs the trigger of the deep fake detector based on the first sample data without the trigger, and obtains a trigger set, which can accurately simulate possible backdoor triggers and provide a basis for subsequent risk assessment. By determining the first risk value of the deep fake detector in the backdoor detection stage based on the trigger set, the risk level of the detector when facing potential backdoor attacks can be quantitatively evaluated. Then, by using a second data set including second sample data without the trigger and third sample data with the trigger, the deep fake detector is trained for backdoor repair. By comparing the performance index values before and after the repair training, the risk level during the repair stage can be further evaluated. Finally, the risk values of the backdoor detection stage and the backdoor repair stage are comprehensively considered to achieve a comprehensive and accurate assessment of the backdoor risk level of the deep fake detector.
[0042] In some embodiments, the trigger reconstruction based on the first data set is applied to the deep fake detector to obtain the trigger set, including: For each class of labels in the label set of the classifier of the deep fake detector, based on the optimization objective, reconstruct a candidate trigger using the first sample data corresponding to each class of labels in the first data set, and determine the norm of each of the candidate triggers; Obtain a trigger set based on the candidate triggers of each type of label in the label set of the classifier of the deep fake detector and the norm of the candidate triggers; The optimization goal is that when the candidate trigger is added to the first sample data, the classifier of the deep fake detector will misclassify the first sample data and minimize the norm of the candidate trigger.
[0043] Here, for each category in the label set, the following steps are performed to reconstruct a candidate trigger: For the first sample data corresponding to each class label, a candidate trigger is reconstructed using a preset optimization goal. The specific optimization goal is to find a specific input pattern, namely a candidate trigger, which, when added to these first sample data, will cause the classifier to misclassify these sample data into categories other than their original categories, and to constrain the size of the candidate trigger by minimizing the norm of the candidate trigger.
[0044] After reconstructing the candidate triggers corresponding to each type of label, determine its norm. Here, the norm of the candidate trigger refers to a mathematical indicator that measures the size or strength of the candidate trigger. The calculation of the norm usually depends on the specific form and representation method of the candidate trigger, as well as the selected norm type, which can be flexibly set according to needs and is not restricted. For example, for image triggers, the norm can be calculated based on the sum of the absolute values of pixel values (such as the L1 norm), the square root of the sum of the squares of pixel values (such as the L2 norm), etc.
[0045] For example, in order to optimize the reconstruction of candidate triggers for each type of label, an iterative optimization algorithm is used. The optimization goal of the algorithm is to adjust the pixel values of the candidate triggers so that when the candidate triggers are added to clean images, the deep fake detector will mistakenly classify these images as other types of labels. During the optimization process, special attention is paid to modifying only a limited part of the image to maintain the concealment of the trigger, and the size of the candidate trigger is constrained by minimizing the L1 norm. For each type of label in the classifier, the above optimization process is performed to generate a candidate trigger corresponding to the label, and its L1 norm is calculated to evaluate its size.
[0046] The backdoor risk assessment method provided by the present invention reconstructs the trigger of the deep fake detector based on the first sample data that does not carry the trigger, obtains the candidate trigger corresponding to each type of label and its norm, and provides a basis for subsequent risk assessment.
[0047] In some embodiments, determining a first risk value of the deep fake detector in a backdoor detection phase based on the trigger set includes: Determining the median absolute deviation of the norms of all the candidate triggers in the trigger set; Determining abnormal labels in a label set of a classifier of the deep fake detector based on the median absolute deviation and the norm of the candidate trigger for each class of labels in the trigger set; Based on abnormal labels in a label set of a classifier of the deep fake detector, a first risk value of the deep fake detector in a backdoor detection phase is determined.
[0048] First, the median absolute deviation (MAD) of the norms of all candidate triggers in the trigger set is calculated. The median absolute deviation is a statistic that measures the degree of dispersion of data distribution. It represents the median of the absolute value of the deviation between the data point and the median. In this embodiment, MAD is used to measure the degree of dispersion of the trigger norm and help understand the overall distribution of the trigger set.
[0049] Next, we set the required threshold based on the median absolute deviation (e.g., a multiple of the median absolute deviation), and calculate the deviation between the norm of the candidate triggers of each type of label and the overall median. We compare the calculated deviation with the preset threshold, and identify the labels corresponding to the candidate triggers whose deviation exceeds the threshold as abnormal labels.
[0050] Finally, based on the abnormal labels in the label set of the deep fake detector's classifier, the first risk value of the deep fake detector in the backdoor detection stage is determined. Here, the first risk value is a quantitative indicator used in the backdoor detection stage to measure the risk faced by the model due to misclassification of abnormal labels.
[0051] Specifically, the calculation of the first risk value may involve multiple factors, such as the number of abnormal labels, the classification error rate caused by the candidate triggers corresponding to the abnormal labels, the norm size of the candidate triggers corresponding to the abnormal labels, etc. By comprehensively considering these factors, the first risk value of the deep fake detector in the backdoor detection stage is calculated.
[0052] The backdoor risk assessment method provided by the present invention can quantitatively assess the risk level of the detector when facing potential backdoor attacks by determining the first risk value of the deep fake detector in the backdoor detection stage based on the trigger set.
[0053] In some embodiments, the first performance indicator value includes a first clean accuracy rate and a first attack success rate, the second performance indicator value includes a second clean accuracy rate and a second attack success rate, and determining a second risk value of the deep fake detector in the backdoor repair stage based on the first performance indicator value of the deep fake detector and the second performance indicator value of the target deep fake detector includes: Determine a clean accuracy difference between the first clean accuracy and the second clean accuracy; determining an attack success rate difference between the first attack success rate and the second attack success rate; Based on the attack success rate difference and the clean accuracy rate difference, a second risk value of the deep fake detector in the backdoor repair stage is determined.
[0054] Here, the first performance index value is the performance index of the original deep fake detector before backdoor repair training. It includes two key components: the first clean accuracy CA and the first attack success rate ASR.
[0055] The first clean accuracy rate refers to the accuracy of the deep fake detector in identifying clean sample data that has not been tampered with by the trigger. The first attack success rate refers to the probability that the deep fake detector mistakenly classifies the sample data as a specific label (usually a fake category) when identifying sample data tampered with by the trigger.
[0056] The second performance indicator value is the performance indicator of the target deep fake detector after backdoor repair training. It also includes two key components: the second clean accuracy CA repair and the second attack success rate ASR repair .
[0057] The second clean accuracy rate refers to the accuracy of the target deep fake detector in identifying clean sample data that has not been tampered with by the trigger. The second attack success rate refers to the probability that the target deep fake detector mistakenly classifies the sample data as a specific label when identifying the sample data tampered with by the trigger.
[0058] In this embodiment, in order to evaluate the risk of the deep fake detector in the backdoor repair stage, it is necessary to calculate two differences and determine the second risk value based on these differences.
[0059] Specifically, the accuracy loss of the original task during the backdoor repair process is calculated, that is, the repair cost L CA =CA-CA repair ; Calculate the degree to which the backdoor has been repaired, that is, the repair effect L ASR =ASR-ASR repair ; Finally, the profit of the entire repair phase is obtained repair =L ASR -L CA ,This benefit is also the second risk value of the deep fake detector in the backdoor repair phase.
[0060] The backdoor risk assessment method provided by the present invention can further assess the risk level during the repair phase by comparing the performance indicator values before and after repair training.
[0061] In some embodiments, the second data set is obtained by: Randomly select a first proportion of first sample data from the first data set as second sample data; Randomly selecting a second proportion of first sample data from the first data set, and adding a preset trigger to the second proportion of the first sample data as third sample data; A second data set is obtained based on the second sample data and the third sample data.
[0062] Specifically, a portion of sample data is randomly selected from the first data set according to a preset ratio (ie, the first ratio) as the second sample data.
[0063] Then, a portion of sample data is randomly selected from the first data set according to another preset ratio (i.e., the second ratio), and a preset trigger is added to the selected sample data. The sample data after the trigger is added is called the third sample data. Here, the preset trigger can be a candidate trigger in the trigger set, or can be another preset trigger, and there is no limitation on this.
[0064] Finally, the second sample data without the trigger added and the third sample data with the trigger added are merged to form a second data set, so that the second data set includes both the original sample data that has not been tampered with and the sample data that has been tampered with by the carefully designed trigger.
[0065] The backdoor risk assessment device provided by the present invention is described below. The backdoor risk assessment device described below and the backdoor risk assessment method described above can be referenced to each other.
[0066] Figure 2 : is a schematic diagram of the structure of the backdoor risk assessment device provided by the present invention, such as Figure 2 As shown, the backdoor risk assessment device includes the following.
[0067] A first backdoor risk assessment device 210 is used to reconstruct triggers in a deep fake detector based on a first data set to obtain a trigger set, wherein the first data set includes first sample data without triggers, and the trigger set includes candidate triggers corresponding to various labels in a label set of at least one classifier of the deep fake detector and the norm of the candidate triggers; A second backdoor risk assessment device 220, for determining a first risk value of the deep fake detector in a backdoor detection phase based on the trigger set; A third backdoor risk assessment device 230 is used to apply the deep fake detector to perform backdoor repair training based on a second data set to obtain a trained target deep fake detector, wherein the second data set includes second sample data without a trigger and third sample data with a trigger; A fourth backdoor risk assessment device 240, configured to determine a second risk value of the deep fake detector in a backdoor repair phase based on the first performance indicator value of the deep fake detector and the second performance indicator value of the target deep fake detector; The fifth backdoor risk assessment device 250 is used to determine the backdoor risk level of the deep fake detector based on the first risk value and the second risk value.
[0068] The backdoor risk assessment device provided by the present invention first reconstructs the trigger of the deep fake detector based on the first sample data without the trigger, and obtains a trigger set, which can accurately simulate possible backdoor triggers and provide a basis for subsequent risk assessment. By determining the first risk value of the deep fake detector in the backdoor detection stage based on the trigger set, the risk level of the detector when facing potential backdoor attacks can be quantitatively evaluated. Then, by using a second data set including the second sample data without the trigger and the third sample data with the trigger, the deep fake detector is trained for backdoor repair. By comparing the performance indicator values before and after the repair training, the risk level during the repair stage can be further evaluated. Finally, the risk values of the backdoor detection stage and the backdoor repair stage are comprehensively considered to achieve a comprehensive and accurate assessment of the backdoor risk level of the deep fake detector.
[0069] Figure 3 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 3 As shown, the electronic device may include: a processor 310, a communications interface 320, a memory 330 and a communication bus 340, wherein the processor 310, the communications interface 320 and the memory 330 communicate with each other through the communication bus 340. The processor 310 may call the logic instructions in the memory 330 to execute the backdoor risk assessment method, which includes the following steps: Based on the first data set, the trigger is reconstructed by applying it to the deep fake detector to obtain a trigger set, wherein the first data set includes first sample data without a trigger, and the trigger set includes candidate triggers corresponding to various labels in a label set of at least one classifier of the deep fake detector and the norm of the candidate triggers; Based on the set of triggers, determining a first risk value of the deep fake detector in a backdoor detection phase; Based on the second data set, applying it to the deep fake detector to perform backdoor repair training to obtain a trained target deep fake detector, wherein the second data set includes second sample data without a trigger and third sample data with a trigger; Determining a second risk value of the deep fake detector in a backdoor repair phase based on the first performance indicator value of the deep fake detector and the second performance indicator value of the target deep fake detector; Based on the first risk value and the second risk value, a backdoor risk level of the deep fake detector is determined.
[0070] In addition, the logic instructions in the above-mentioned memory 330 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.
[0071] On the other hand, the present invention also provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the backdoor risk assessment method provided by the above methods, which includes the following steps.
[0072] Based on the first data set, the trigger is reconstructed by applying it to the deep fake detector to obtain a trigger set, wherein the first data set includes first sample data without a trigger, and the trigger set includes candidate triggers corresponding to various labels in a label set of at least one classifier of the deep fake detector and the norm of the candidate triggers; Based on the set of triggers, determining a first risk value of the deep fake detector in a backdoor detection phase; Based on the second data set, applying it to the deep fake detector to perform backdoor repair training to obtain a trained target deep fake detector, wherein the second data set includes second sample data without a trigger and third sample data with a trigger; Determining a second risk value of the deep fake detector in a backdoor repair phase based on the first performance indicator value of the deep fake detector and the second performance indicator value of the target deep fake detector; Based on the first risk value and the second risk value, a backdoor risk level of the deep fake detector is determined.
[0073] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the execution backdoor risk assessment method provided by the above methods is implemented, and the method includes the following steps.
[0074] Based on the first data set, the trigger is reconstructed by applying it to the deep fake detector to obtain a trigger set, wherein the first data set includes first sample data without a trigger, and the trigger set includes candidate triggers corresponding to various labels in a label set of at least one classifier of the deep fake detector and the norm of the candidate triggers; Based on the set of triggers, determining a first risk value of the deep fake detector in a backdoor detection phase; Based on the second data set, applying it to the deep fake detector to perform backdoor repair training to obtain a trained target deep fake detector, wherein the second data set includes second sample data without a trigger and third sample data with a trigger; Determining a second risk value of the deep fake detector in a backdoor repair phase based on the first performance indicator value of the deep fake detector and the second performance indicator value of the target deep fake detector; Based on the first risk value and the second risk value, a backdoor risk level of the deep fake detector is determined.
[0075] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.
[0076] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.
[0077] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A backdoor risk assessment method, characterized in that: The method comprises: Based on the first data set, the trigger is reconstructed by applying it to the deep fake detector to obtain a trigger set, wherein the first data set includes first sample data without a trigger, and the trigger set includes candidate triggers corresponding to various labels in a label set of at least one classifier of the deep fake detector and the norm of the candidate triggers; Based on the set of triggers, determining a first risk value of the deep fake detector in a backdoor detection phase; Based on the second data set, applying it to the deep fake detector to perform backdoor repair training to obtain a trained target deep fake detector, wherein the second data set includes second sample data without a trigger and third sample data with a trigger; Determining a second risk value of the deep fake detector in a backdoor repair phase based on the first performance indicator value of the deep fake detector and the second performance indicator value of the target deep fake detector; Based on the first risk value and the second risk value, a backdoor risk level of the deep fake detector is determined.
2. The backdoor risk assessment method according to claim 1, characterized in that: The method is based on the first data set and applied to the deep fake detector to reconstruct the trigger to obtain the trigger set, including: For each class of labels in the label set of the classifier of the deep fake detector, based on the optimization objective, reconstruct a candidate trigger using the first sample data corresponding to each class of labels in the first data set, and determine the norm of each of the candidate triggers; Obtain a trigger set based on the candidate triggers of each type of label in the label set of the classifier of the deep fake detector and the norm of the candidate triggers; The optimization goal is that when the candidate trigger is added to the first sample data, the classifier of the deep fake detector will misclassify the first sample data and minimize the norm of the candidate trigger.
3. The backdoor risk assessment method according to claim 1, characterized in that: The determining, based on the trigger set, a first risk value of the deep fake detector in the backdoor detection phase includes: Determining the median absolute deviation of the norms of all the candidate triggers in the trigger set; Determining abnormal labels in a label set of a classifier of the deep fake detector based on the median absolute deviation and the norm of the candidate trigger for each class of labels in the trigger set; Based on abnormal labels in a label set of a classifier of the deep fake detector, a first risk value of the deep fake detector in a backdoor detection phase is determined.
4. The backdoor risk assessment method according to claim 1, characterized in that: The first performance indicator value includes a first clean accuracy rate and a first attack success rate, the second performance indicator value includes a second clean accuracy rate and a second attack success rate, and determining a second risk value of the deep fake detector in the backdoor repair stage based on the first performance indicator value of the deep fake detector and the second performance indicator value of the target deep fake detector includes: Determine a clean accuracy difference between the first clean accuracy and the second clean accuracy; determining an attack success rate difference between the first attack success rate and the second attack success rate; Based on the attack success rate difference and the clean accuracy rate difference, a second risk value of the deep fake detector in the backdoor repair stage is determined.
5. The backdoor risk assessment method according to claim 1, characterized in that: The second data set is obtained by: Randomly select a first proportion of first sample data from the first data set as second sample data; Randomly selecting a second proportion of first sample data from the first data set, and adding a preset trigger to the second proportion of the first sample data as third sample data; A second data set is obtained based on the second sample data and the third sample data.
6. The backdoor risk assessment method according to claim 1, characterized in that: The determining, based on the first risk value and the second risk value, a backdoor risk level of the deep fake detector includes: Determining a total risk value based on the first risk value and the second risk value; Determining a backdoor risk level of the deep fake detector based on the total risk value; The total risk value is obtained in the following way: R=α×R de +(1-a)R re ; α is the preset balance parameter, R de is the first risk value, R re is the second risk value, and R is the total risk value.
7. A backdoor risk assessment device, characterized in that: include: A first backdoor risk assessment module is configured to reconstruct triggers in a deep fake detector based on a first data set to obtain a trigger set, wherein the first data set includes first sample data without triggers, and the trigger set includes candidate triggers corresponding to various labels in a label set of at least one classifier of the deep fake detector and the norm of the candidate triggers; A second backdoor risk assessment module, configured to determine a first risk value of the deep fake detector in a backdoor detection phase based on the trigger set; A third backdoor risk assessment module is used to apply the deep fake detector to perform backdoor repair training based on a second data set to obtain a trained target deep fake detector, wherein the second data set includes second sample data without a trigger and third sample data with a trigger; a fourth backdoor risk assessment module, configured to determine a second risk value of the deep fake detector in a backdoor repair phase based on the first performance indicator value of the deep fake detector and the second performance indicator value of the target deep fake detector; A fifth backdoor risk assessment module is used to determine the backdoor risk level of the deep fake detector based on the first risk value and the second risk value.
8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the backdoor risk assessment method according to any one of claims 1 to 6 is implemented.
9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the backdoor risk assessment method according to any one of claims 1 to 6 is implemented.
10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the backdoor risk assessment method according to any one of claims 1 to 6 is implemented.