Anti-attack sample generation method and related equipment

Abstract

The invention provides an adversarial attack sample generation method and related equipment, which are applied to the field of machine learning, and are used for determining an initial value of adversarial disturbance distribution of a diffusion model, determining a disturbance score of the adversarial disturbance distribution based on the initial value and an established classifier misleading evaluation function, and generating an adversarial attack sample. And updating the model parameters of the diffusion model based on the disturbance score until the disturbance score of the adversarial disturbance distribution corresponding to the diffusion model is not less than a preset score threshold, determining the current diffusion model, generating an adversarial disturbance sample according to the model parameters of the current diffusion model, and further generating an adversarial attack sample. According to the method for generating the adversarial attack sample, a plurality of data samples do not need to be acquired, the adversarial disturbance distribution of the diffusion model is adjusted, the disturbance sample is generated when the adversarial disturbance distribution meets the set condition, and the adversarial attack sample is generated based on the disturbance sample, so that the sample acquisition is fast, and the attack efficiency of implicit adversarial attack is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of adversarial attack, and in particular to an adversarial attack sample generation method and related equipment. BACKGROUND

[0002] The implicit adversarial attack refers to an attack method of a machine learning model, which is also commonly referred to as adversarial attack or adversarial attack in the field of machine learning. The core is to subtly modify the input data so that the well-trained model produces false output or misleading results. These modified data samples are called adversarial attack samples. Through implicit adversarial attack, the recognition ability of the model can be further trained and improved. At present, implicit adversarial attack has become a key technology to ensure the security and reliability of machine learning model.

[0003] In the process of implicit adversarial attack, a large number of adversarial attack samples are needed to test and learn the machine learning model. At present, the process of obtaining adversarial attack samples is relatively complex, which affects the attack efficiency of implicit adversarial attack. SUMMARY

[0004] The technical problem to be solved by the present application is to provide an adversarial attack sample generation method and related equipment, which can obtain adversarial attack samples more quickly and improve the attack efficiency of implicit adversarial attack.

[0005] The present application also provides an adversarial attack sample generation device for ensuring the implementation and application of the above method in practice.

[0006] An adversarial attack sample generation method comprises:

[0007] Determine the initial value of the adversarial perturbation distribution corresponding to the established diffusion model;

[0008] Based on the initial value of the adversarial perturbation distribution and the created classifier misleading evaluation function, the adversarial perturbation distribution is evaluated to determine the perturbation score of the adversarial perturbation distribution;

[0009] If the perturbation score is not less than the preset score threshold, the first parameter corresponding to the diffusion model is determined, and the state value of the adversarial perturbation distribution is generated based on the first parameter;

[0010] Determine the predicted noise corresponding to the state value;

[0011] Estimate the theoretical distribution of the adversarial perturbation distribution based on the predicted noise;

[0012] Calculate the loss value of the loss function corresponding to the diffusion model according to the theoretical distribution;

[0013] Based on the loss value, the model parameters of the diffusion model are updated in combination with the set gradient descent optimization algorithm, and a disturbance score of an adversarial disturbance distribution corresponding to the updated diffusion model is determined.

[0014] If the disturbance score of the adversarial disturbance distribution corresponding to the updated diffusion model is not less than the preset score threshold, a new first parameter is determined, the model parameters of the diffusion model are re-updated, and until the disturbance score of the adversarial disturbance distribution corresponding to the updated diffusion model is less than the preset score threshold, the current model parameters of the diffusion model are output.

[0015] According to the current model parameters of the diffusion model, an adversarial disturbance sample is generated.

[0016] Based on the adversarial disturbance sample, an adversarial attack sample is generated.

[0017] The above-mentioned adversarial attack sample generation method, optionally, the initial value of the established diffusion model corresponding to the adversarial disturbance distribution is determined, comprising:

[0018] A plurality of data samples are determined in the selected data set.

[0019] The mean and variance of each data sample are determined.

[0020] According to the mean and variance of each data sample, a standard Gaussian distribution corresponding to the adversarial disturbance distribution is determined.

[0021] A value is randomly selected from the standard Gaussian distribution as the initial value of the adversarial disturbance distribution corresponding to the diffusion model.

[0022] The above-mentioned adversarial attack sample generation method, optionally, the classifier misleading evaluation function is specifically:

[0023] j(x, z) = dist(x + z, x) + γF(x + z, y(x))

[0024] Wherein:

[0025] j(x, z) represents the classifier misleading evaluation function;

[0026] dist represents a distance function that has been set; L1-norm or L2-norm can be used;

[0027] x is a data sample;

[0028] y(x) is the true class label corresponding to the data sample;

[0029] γ is a balance coefficient, and the value range of γ is [0-100];

[0030]

[0031] z is a perturbation learned by the diffusion model;

[0032] F is an evaluation function for evaluating the quality of the perturbation z;

[0033] f is a machine learning model under attack;

[0034] k is a preset hyperparameter, and the value range of k is [20-100];

[0035] i is used to indicate a class index.

[0036] The above-mentioned method for generating an adversarial attack sample, optionally, the method comprises:

[0037] randomly selecting a time step in the time step interval corresponding to the diffusion model and randomly selecting Gaussian noise in the set Gaussian distribution;

[0038] randomly selecting a value in the standard Gaussian distribution as a new initial value of the adversarial perturbation distribution corresponding to the diffusion model;

[0039] composing the time step, the Gaussian noise and the new initial value into the first parameter.

[0040] An apparatus for generating an adversarial attack sample, comprising:

[0041] a first determining unit configured to determine an initial value of an adversarial perturbation distribution corresponding to an established diffusion model;

[0042] an evaluation unit configured to evaluate the adversarial perturbation distribution based on the initial value of the adversarial perturbation distribution and a created classifier misleading evaluation function, and determine a perturbation score of the adversarial perturbation distribution;

[0043] a first generating unit configured to, if the perturbation score is not less than a preset score threshold, determine a first parameter corresponding to the diffusion model, and generate a state value of the adversarial perturbation distribution based on the first parameter;

[0044] a second determining unit configured to determine a predicted noise corresponding to the state value;

[0045] an estimating unit configured to estimate a theoretical distribution of the adversarial perturbation distribution based on the predicted noise;

[0046] a calculating unit configured to calculate a loss value of a loss function corresponding to the diffusion model according to the theoretical distribution;

[0047] a third determining unit, configured to update the model parameters of the diffusion model based on the loss value and in combination with a gradient descent optimization algorithm that has been set, and determine a perturbation score of an adversarial perturbation distribution corresponding to the updated diffusion model;

[0048] an output unit, configured to: if the perturbation score of the adversarial perturbation distribution corresponding to the updated diffusion model is not less than the preset score threshold, determine new first parameters, and update the model parameters of the diffusion model again until the perturbation score of the adversarial perturbation distribution corresponding to the updated diffusion model is less than the preset score threshold, and output the current model parameters of the diffusion model;

[0049] a second generating unit, configured to generate an adversarial perturbation sample according to the current model parameters of the diffusion model;

[0050] a third generating unit, configured to generate an adversarial attack sample based on the adversarial perturbation sample.

[0051] The above-described adversarial attack sample generation apparatus can optionally have the first determining unit configured as follows:

[0052] determine a plurality of data samples in the selected data set, determine the mean and variance of each data sample, determine a standard Gaussian distribution corresponding to the adversarial perturbation distribution according to the mean and variance of each data sample, and randomly select a value from the standard Gaussian distribution as an initial value of the adversarial perturbation distribution corresponding to the diffusion model.

[0053] The above-described adversarial attack sample generation apparatus can optionally have the classifier misleading evaluation function configured as follows:

[0054] j(x, z) = dist(x + z, x) + γF(x + z, y(x))

[0055] wherein:

[0056] j(x, z) represents the classifier misleading evaluation function;

[0057] dist represents a distance function that has been set, and can be an L1-norm or an L2-norm;

[0058] x is a data sample;

[0059] y(x) is a true class label corresponding to the data sample;

[0060] γ is a balance coefficient, and the value range of γ is [0-100];

[0061]

[0062] z is the perturbation learned by the diffusion model;

[0063] F is an evaluation function for evaluating the quality of the perturbation z;

[0064] f is the machine learning model under attack;

[0065] k is a preset hyperparameter, and the value range of k is [20-100];

[0066] i is used to indicate the category index.

[0067] The above-mentioned adversarial attack sample generation device, optionally, the first generation unit is specifically configured as:

[0068] randomly selecting a time step in the time step interval corresponding to the diffusion model and randomly selecting Gaussian noise in the set Gaussian distribution; randomly selecting a value in the standard Gaussian distribution as a new initial value of the adversarial perturbation distribution corresponding to the diffusion model; and combining the time step, the Gaussian noise and the new initial value to form the first parameter.

[0069] A storage medium, the storage medium comprising stored instructions, wherein the instructions, when executed, control a device in which the storage medium is located to perform the above-mentioned adversarial attack sample generation method.

[0070] An electronic device comprising a memory, and one or more instructions, wherein the one or more instructions are stored in the memory and configured to be executed by one or more processors to perform the above-mentioned adversarial attack sample generation method.

[0071] Compared with the prior art, the present application has the following advantages:

[0072] The application provides an adversarial attack sample generation method, which comprises the following steps: determining an initial value of an adversarial perturbation distribution corresponding to an established diffusion model; evaluating the adversarial perturbation distribution based on the initial value of the adversarial perturbation distribution and a created classifier deception evaluation function, and determining a perturbation score of the adversarial perturbation distribution; if the perturbation score is not less than a preset score threshold, determining a first parameter corresponding to the diffusion model, and generating a state value of the adversarial perturbation distribution based on the first parameter; determining a predicted noise corresponding to the state value; estimating a theoretical distribution of the adversarial perturbation distribution based on the predicted noise; calculating a loss value of a loss function corresponding to the diffusion model according to the theoretical distribution; updating model parameters of the diffusion model based on the loss value and combining a gradient descent optimization algorithm, and determining a perturbation score of an adversarial perturbation distribution corresponding to the updated diffusion model; if the perturbation score of the adversarial perturbation distribution corresponding to the updated diffusion model is not less than the preset score threshold, determining a new first parameter, and updating the model parameters of the diffusion model again until the perturbation score of the adversarial perturbation distribution corresponding to the updated diffusion model is less than the preset score threshold, and then outputting current model parameters of the diffusion model; generating an adversarial perturbation sample according to the current model parameters of the diffusion model; and generating an adversarial attack sample based on the adversarial perturbation sample. The adversarial attack sample generation method provided by the application does not need to obtain multiple data samples, adjusts the adversarial perturbation distribution of the diffusion model, generates a perturbation sample when the adversarial perturbation distribution meets a set condition, generates an adversarial attack sample based on the perturbation sample, and thus the sample is obtained quickly, and the attack efficiency of the implicit adversarial attack is improved. BRIEF DESCRIPTION OF DRAWINGS

[0073] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, brief introductions will be given to the drawings needed in the embodiments or prior art description. Obviously, the drawings in the following description are only embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative effort on the basis of the provided drawings.

[0074] Figure 1 A method flowchart of the adversarial attack sample generation method provided by the present application;

[0075] Figure 2 A learning framework for generating implicit adversarial perturbations provided by the present application;

[0076] Figure 3 Another method flowchart of the adversarial attack sample generation method provided by the present application;

[0077] Figure 4 A structural schematic diagram of an adversarial attack sample generation device provided by the present application;

[0078] Figure 5 A structural schematic diagram of an electronic device provided by the present application. DETAILED DESCRIPTION

[0079] The technical solutions in the embodiments of the present application will be clearly and completely described with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0080] The present application can be used in many general or special-purpose computing device environments or configurations. For example: personal computers, server computers, handheld or laptop devices, tablet devices, multiprocessor devices, distributed computing environments that include any of the above devices or devices, and the like.

[0081] The embodiment of the present application provides an adversarial attack sample generation method, which can be applied to an implicit adversarial attack process and used for generating adversarial attack samples. It can be understood that the implicit adversarial attack refers to an attack mode against a machine learning model, which is commonly referred to as adversarial attack or adversarial attack in the field of machine learning. The core is to subtly modify the input data so that the well-trained model produces false output or misleading results. These modified data samples are called adversarial attack samples.

[0082] The diffusion model is a generative model based on probability theory. The implicit loss function is not defined by an explicit mathematical expression in machine learning and optimization problems, but is implicitly defined by the training process or structure of the model. The implicit loss function does not directly give the mathematical formula of the target optimization, but depends on the architecture of the model, the training strategy or the characteristics of the data itself to indirectly guide the learning process of the model.

[0083] In the training process of the diffusion model, the loss function plays a crucial role. Although the diffusion model does not directly define an explicit loss function form, its optimization target can be implicitly represented as a loss function, i.e. an implicit loss function.

[0084] In the process of applying the diffusion model to generate adversarial attack samples, the embodiment of the present application provides an adversarial attack sample generation method, which can still generate adversarial attack samples through the diffusion model end-to-end without explicit distribution definition or a large number of real samples.

[0085] The embodiment of the present application provides an attack sample generation method, which can be applied to a system platform where a diffusion model is located, and an execution subject of the method can be a processor in the platform, and a method flowchart of the method is as shown in Figure 1

[0086] S101: Determine an initial value of an attack perturbation distribution corresponding to an established diffusion model;

[0087] In the method provided by the embodiment of the present application, the corresponding diffusion model is established in advance, and the attack perturbation distribution of the diffusion model is determined.

[0088] In the embodiment of the present application, the attack perturbation distribution can be understood as an implicit perturbation distribution introduced by an attack sample in the field of machine learning, especially deep learning. The attack sample refers to a sample that can make a model produce an incorrect output by performing a slight perturbation on input data. These perturbations are usually difficult to detect, but can significantly affect the prediction results of the model.

[0089] In the embodiment of the present application, the initial value of the attack perturbation distribution corresponding to the established diffusion model is calculated.

[0090] The initial value of the attack perturbation distribution can be understood as a numerical value corresponding to the attack perturbation distribution.

[0091] S102: Based on the initial value of the attack perturbation distribution and a created classifier misleading evaluation function, the attack perturbation distribution is evaluated to determine a perturbation score of the attack perturbation distribution;

[0092] In the method provided by the embodiment of the present application, a classifier misleading evaluation function is defined to reflect the situation of the implicit attack perturbation distribution learned by the diffusion model.

[0093] Based on the initial value of the attack perturbation and the classifier misleading evaluation function, the attack perturbation distribution corresponding to the diffusion model is evaluated, and by evaluating the attack perturbation distribution, the perturbation score of the attack perturbation distribution is determined.

[0094] S103: If the perturbation score is not less than a preset score threshold, a first parameter corresponding to the diffusion model is determined, and a state value of the attack perturbation distribution is generated based on the first parameter;

[0095] In the method provided by the embodiment of the present application, in the case that the perturbation score of the attack perturbation distribution is not less than the preset score threshold, the first parameter corresponding to the diffusion model is determined. The first parameter can include a random sampling time step, a conditional sample data, an initial value of the attack perturbation distribution corresponding to the diffusion model at a current time, and a Gaussian noise sampled at any time.

[0096] ​Based on the first parameter, a state value of an adversarial perturbation distribution is generated by using a diffusion forward process.

[0097] S104: Determine the predicted noise corresponding to the state value of the adversarial perturbation distribution.

[0098] In the method provided by the embodiment of the application, the predicted noise corresponding to the state value of the adversarial perturbation distribution is predicted based on the diffusion model.

[0099] S105: Estimate the theoretical distribution of the adversarial perturbation distribution based on the predicted noise.

[0100] In the method provided by the embodiment of the application, the theoretical distribution of the adversarial perturbation distribution is derived based on the predicted noise by calling a set derivation formula.

[0101] S106: Calculate the loss value of the loss function corresponding to the diffusion model according to the theoretical distribution.

[0102] S107: Update the model parameters of the diffusion model based on the loss value and in combination with a set gradient descent optimization algorithm, and determine the perturbation score of the adversarial perturbation distribution corresponding to the updated diffusion model.

[0103] In the method provided by the embodiment of the application, the model parameters of the diffusion model are updated by using the set gradient descent optimization algorithm, new model parameters currently corresponding to the diffusion model are determined, and the perturbation score of the adversarial perturbation distribution corresponding to the diffusion model after the parameter update is further determined.

[0104] S108: If the perturbation score of the adversarial perturbation distribution corresponding to the updated diffusion model is not less than the preset score threshold, a new first parameter is determined, the model parameters of the diffusion model are updated again, and until the perturbation score of the adversarial perturbation distribution corresponding to the updated diffusion model is less than the preset score threshold, the current model parameters of the diffusion model are output.

[0105] In the method provided by the embodiment of the application, if the perturbation score of the adversarial perturbation distribution corresponding to the updated diffusion model is not less than the preset score threshold, a new first parameter is determined again, and the model parameters of the diffusion model are updated again based on the new first parameter, until the perturbation score of the adversarial perturbation distribution corresponding to the diffusion model is less than the preset score threshold, and finally the current model parameters corresponding to the diffusion model are output.

[0106] S109: Generate an adversarial perturbation sample according to the current model parameters of the diffusion model.

[0107] In the method provided by the embodiment of the application, the adversarial perturbation sample is generated based on the current model parameters of the diffusion model.

[0108] S110: generating an adversarial attack sample based on the adversarial perturbation sampling.

[0109] In the method provided by the embodiment of the application, the diffusion model generates an adversarial attack sample for attack according to the adversarial perturbation sampling.

[0110] The method for generating an adversarial attack sample provided by the embodiment of the application does not need to obtain multiple data samples, adjusts the adversarial perturbation distribution of the diffusion model, generates a perturbation sampling when the adversarial perturbation distribution meets a set condition, and generates an adversarial attack sample based on the perturbation sampling, so that the sample is obtained quickly, and the attack efficiency of the implicit adversarial attack is improved.

[0111] In the method provided by the embodiment of the application, the initial value of the adversarial perturbation distribution corresponding to the established diffusion model is determined, and the method comprises the following steps.

[0112] A plurality of data samples are determined in the selected data set.

[0113] The mean and variance of each data sample are determined.

[0114] According to the mean and variance of each data sample, a standard Gaussian distribution corresponding to the adversarial perturbation distribution is determined.

[0115] A value randomly selected from the standard Gaussian distribution is taken as the initial value of the adversarial perturbation distribution corresponding to the diffusion model.

[0116] In the method provided by the embodiment of the application, a plurality of data samples are determined in the selected data set, a set distance algorithm is executed on each data sample, the distance algorithm can adopt an L_norm_alg algorithm, and the mean mean and variance var of each data sample are obtained.

[0117] Based on the mean mean and variance var, a standard Gaussian distribution N(mean, var*l) corresponding to the adversarial perturbation distribution is determined.

[0118] A value randomly selected from the standard Gaussian distribution N is taken as the initial value of the adversarial perturbation distribution corresponding to the diffusion model.

[0119] In the method provided by the embodiment of the application, the classifier misleading evaluation function is specifically:

[0120] j(x, z) = dist(x + z, x) + γF(x + z, y(x))

[0121] Wherein:

[0122] j(x, z) represents the classifier misleading evaluation function.

[0123] dist represents a distance function set; L1-norm or L2-norm can be adopted;

[0124] x is a data sample;

[0125] y(x) is a real class label corresponding to the data sample;

[0126] gamma is a balance coefficient, and the value range of gamma is [0-100];

[0127]

[0128] z is a perturbation learned by the diffusion model;

[0129] F is an evaluation function for evaluating the quality of the perturbation z;

[0130] f is a machine learning model attacked;

[0131] k is a preset hyperparameter, and the value range of k is [20-100];

[0132] i is used to indicate a class index.

[0133] In the method provided by the embodiment of the application, the classifier misleading evaluation function j(x, z) is introduced to reflect the case of the implicit adversarial perturbation distribution z learned by the diffusion model, and when the value of the classifier misleading evaluation function is less than a set threshold, it is determined that the implicit adversarial perturbation distribution z meets the adversarial attack requirement.

[0134] In the method provided by the embodiment of the application, gamma is a balance coefficient, which can be understood as a hyperparameter for balancing the weights between two terms, and is an integer value, which can be set to 1 in specific experimental applications, and the value range is any integer value in [0-100].

[0135] F is used to evaluate the quality of the current perturbation z, that is, the ability to successfully deceive the existing machine learning model f after adding the perturbation z to the input sample x.

[0136] f is a machine learning model attacked, which can be a classification model or other models such as regression.

[0137] k is a preset hyperparameter, and is an integer value, which can be set to 50 in specific experimental applications.

[0138] In the method provided by the embodiment of the application, the determination of the first parameter corresponding to the diffusion model comprises:

[0139] randomly selecting a time step in the time step interval corresponding to the diffusion model and randomly selecting Gaussian noise in the set Gaussian distribution;

[0140] Randomly select a value in the standard Gaussian distribution as a new initial value of the adversarial perturbation distribution corresponding to the diffusion model;

[0141] The time step, the Gaussian noise and the new initial value form the first parameter.

[0142] In the method provided by the embodiment of the present application, a time step t is randomly sampled in a time step interval corresponding to a diffusion model, a conditional data sample x is selected, and a Gaussian noise is randomly selected in a set Gaussian distribution N(0, I).

[0143] A value is randomly selected in the determined standard Gaussian distribution as a new initial value of an adversarial perturbation distribution corresponding to the diffusion model.

[0144] The first parameter is formed by the time step t, the conditional data sample x, the Gaussian noise and the new initial value.

[0145] Reference Figure 2 The training framework for generating an implicit adversarial perturbation provided by the embodiment of the present application takes the original (x, y) as the input of the U-net, splices it with z t and inputs it into the U-net, and then the output noise of the U-net is used to predict the theoretical distribution of the adversarial perturbation distribution

[0146] for the calculation of the loss function, and the u-net model parameters are optimized by gradient descent.

[0147] Reference Figure 3 A detailed structure diagram of the method for generating an adversarial attack sample provided by the embodiment of the present application is combined with Figure 3 In the method provided by the embodiment of the present application, the L_norm_alg algorithm is executed on a small number of samples of the data set x to obtain the mean mean and the variance var.

[0148] Take the initial value of z,

[0149]

[0150] Randomly sample the time step t, the condition x and the corresponding initial value of z The Gaussian noise w is obtained by N(0, I).

[0151] The diffusion forward process generates Alpha and beta are the noise intensity of the diffusion model, which is related to the time step t, and the noise of the diffusion model U-Net is predicted: epsilon θ (z t , t, x) theta represents the parameters of the diffusion model, which are optimized and learned in the iteration.

[0152] deriving a prediction noise from the prediction

[0153]

[0154] computing a loss

[0155] optimizing the diffusion model parameters by a gradient descent algorithm from the loss L;

[0156] End while

[0157] returning the optimized parameters epsilon θ (,).

[0158] with Figure 1 Corresponding to the method, the embodiment of the application further provides an adversarial attack sample generation device for generating an adversarial attack sample for Figure 1 The specific implementation of the method is shown in the structural schematic diagram as Figure 4 Specifically, the device comprises:

[0159] A first determination unit 201 is configured to determine an initial value of an adversarial perturbation distribution corresponding to an established diffusion model;

[0160] An evaluation unit 202 is configured to evaluate the adversarial perturbation distribution based on the initial value of the adversarial perturbation distribution and a created classifier misleading evaluation function, and determine a perturbation score of the adversarial perturbation distribution;

[0161] A first generation unit 203 is configured to determine a first parameter corresponding to the diffusion model if the perturbation score is not less than a preset score threshold, and generate a state value of the adversarial perturbation distribution based on the first parameter;

[0162] A second determination unit 204 is configured to determine a prediction noise corresponding to the state value;

[0163] An estimation unit 205 is configured to estimate a theoretical distribution of the adversarial perturbation distribution based on the prediction noise;

[0164] A calculation unit 206 is configured to calculate a loss value of a loss function corresponding to the diffusion model according to the theoretical distribution;

[0165] A third determination unit 207 is configured to update model parameters of the diffusion model based on the loss value and in combination with a set gradient descent optimization algorithm, and determine a perturbation score of an adversarial perturbation distribution corresponding to the updated diffusion model;

[0166] The output unit 208 is configured to: if the disturbance score of the adversarial disturbance distribution corresponding to the updated diffusion model is not less than the preset score threshold, determine a new first parameter, and update the model parameter of the diffusion model again until the disturbance score of the adversarial disturbance distribution corresponding to the updated diffusion model is less than the preset score threshold, and output the current model parameter of the diffusion model.

[0167] The second generation unit 209 is configured to generate an adversarial disturbance sample according to the current model parameter of the diffusion model.

[0168] The third generation unit 210 is configured to generate an adversarial attack sample based on the adversarial disturbance sample.

[0169] In the device provided by the embodiment of the application, the first determination unit is specifically configured to:

[0170] determine a plurality of data samples in the selected data set, determine the mean and variance of each data sample, determine the standard Gaussian distribution corresponding to the adversarial disturbance distribution according to the mean and variance of each data sample, and randomly select a value from the standard Gaussian distribution as the initial value of the adversarial disturbance distribution corresponding to the diffusion model.

[0171] In the device provided by the embodiment of the application, the classifier misleading evaluation function is specifically:

[0172] j(x, z) = dist(x + z, x) + γF(x + z, y(x))

[0173] Wherein:

[0174] j(x, z) represents the classifier misleading evaluation function;

[0175] dist represents a distance function set in advance; L1-norm or L2-norm can be used;

[0176]

[0177] y(x) is the real class label corresponding to the data sample;

[0178] γ is a balance coefficient, and the value range of γ is [0-100];

[0179] z is the disturbance learned by the diffusion model;

[0180] F is an evaluation function for evaluating the quality of the disturbance z;

[0181] f is the machine learning model to be attacked;

[0182] k is a preset hyperparameter, and the value range of k is [20-100].

[0183] i is used to indicate the category index.

[0184] The first generating unit is specifically configured to:

[0185] randomly selecting a time step in the time step interval corresponding to the diffusion model and randomly selecting a Gaussian noise in the set Gaussian distribution; randomly selecting a value in the standard Gaussian distribution as a new initial value of the adversarial perturbation distribution corresponding to the diffusion model; and combining the time step, the Gaussian noise and the new initial value to form the first parameter.

[0186] The embodiment of the present application also provides a storage medium, which comprises stored instructions, wherein when the instructions are executed, the device where the storage medium is located performs the above-mentioned adversarial attack sample generation method.

[0187] The embodiment of the present application also provides an electronic device, a structural schematic diagram of which is shown in the figure. Figure 5 The electronic device specifically comprises a memory 301 and one or more than one instruction 302, wherein the one or more than one instruction 302 is stored in the memory 301 and is configured to perform the one or more than one instruction 302 by the one or more than one processor 303 to perform the following operations:

[0188] determining an initial value of an adversarial perturbation distribution corresponding to the established diffusion model;

[0189] based on the initial value of the adversarial perturbation distribution and the created classifier misleading evaluation function, evaluating the adversarial perturbation distribution to determine a perturbation score of the adversarial perturbation distribution;

[0190] if the perturbation score is not less than a preset score threshold, determining a first parameter corresponding to the diffusion model, and generating a state value of the adversarial perturbation distribution based on the first parameter;

[0191] determining a predicted noise corresponding to the state value;

[0192] estimating a theoretical distribution of the adversarial perturbation distribution based on the predicted noise;

[0193] calculating a loss value of a loss function corresponding to the diffusion model according to the theoretical distribution;

[0194] based on the loss value, combining a gradient descent optimization algorithm set to update the model parameter of the diffusion model, and determining a perturbation score of the adversarial perturbation distribution corresponding to the updated diffusion model;

[0195] If the perturbation score of the adversarial perturbation distribution corresponding to the updated diffusion model is not less than the preset score threshold, a new first parameter is determined, the model parameters of the diffusion model are updated again until the perturbation score of the adversarial perturbation distribution corresponding to the updated diffusion model is less than the preset score threshold, and the current model parameters of the diffusion model are output.

[0196] The adversarial perturbation sample is generated according to the current model parameters of the diffusion model.

[0197] The adversarial attack sample is generated based on the adversarial perturbation sample.

[0198] It should be noted that each of the embodiments in the specification adopts a progressive manner for description, and each embodiment focuses on the difference from other embodiments. The same or similar parts of each embodiment can be referred to each other.

[0199] Finally, it should be noted that in this paper, relationship terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between the entities or operations. Moreover, the terms "include", "contain" or any other variants thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or device. Without more limitations, the element defined by the statement "including a" does not exclude the presence of another identical element in the process, method, article or device including the element.

[0200] For the convenience of description, the above device is described as various units respectively described in function. Of course, the functions of each unit can be realized in the same or more software and / or hardware when implementing the present application.

[0201] From the description of the above embodiments, those skilled in the art can clearly understand that the present application can be realized by means of software and necessary general hardware platform. Based on such understanding, the technical solutions of the present application can be embodied in the form of software product, which can be stored in storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., including a plurality of instructions to make a computer device (which can be a personal computer, server, or network device, etc.) execute the method described in each embodiment or some parts of the embodiment of the present application.

[0202] The above describes in detail the method for generating an attack sample and the related device provided by the present application. The principles and implementation manners of the present application are described by using specific examples. The above description of the examples is only used to help understand the method of the present application and its core idea. Meanwhile, for those skilled in the art, the specific implementation manners and application ranges will be changed according to the idea of the present application. In conclusion, the content of the present description should not be understood as a limitation of the present application.

Claims

1. A method for generating adversarial attack samples, characterized in that, include: Determine the initial values ​​of the adversarial perturbation distribution corresponding to the established diffusion model; Based on the initial value of the adversarial perturbation distribution and the created classifier misleading evaluation function, the adversarial perturbation distribution is evaluated to determine the perturbation score of the adversarial perturbation distribution; If the perturbation score is not less than a preset score threshold, determine the first parameter corresponding to the diffusion model, and generate the state value of the adversarial perturbation distribution based on the first parameter. Determine the prediction noise corresponding to the state value; The theoretical distribution of the anti-disturbance distribution is estimated based on the predicted noise; Calculate the loss value of the loss function corresponding to the diffusion model based on the theoretical distribution; Based on the loss value, the model parameters of the diffusion model are updated using the established gradient descent optimization algorithm, and the perturbation score of the adversarial perturbation distribution corresponding to the updated diffusion model is determined. If the perturbation score of the adversarial perturbation distribution corresponding to the updated diffusion model is not less than the preset score threshold, then a new first parameter is determined, and the model parameters of the diffusion model are updated again until the perturbation score of the adversarial perturbation distribution corresponding to the updated diffusion model is less than the preset score threshold, and then the current model parameters of the diffusion model are output. Generate adversarial sampling based on the current model parameters of the diffusion model; Adversarial attack samples are generated based on the adversarial perturbation sampling.

2. The adversarial attack sample generation method according to claim 1, characterized in that, The determination of the initial values ​​for the adversarial perturbation distribution corresponding to the established diffusion model includes: Determine multiple data samples from the selected dataset; Determine the mean and variance of each of the data samples; Based on the mean and variance of each of the data samples, determine the standard Gaussian distribution corresponding to the anti-perturbation distribution; A value is randomly selected from the standard Gaussian distribution as the initial value of the adversarial perturbation distribution corresponding to the diffusion model.

3. The adversarial attack sample generation method according to claim 1, characterized in that, The classifier misleading evaluation function is specifically as follows: j(x,z)=dist(x+z,x)+γF(x+z,y(x)) in: j(x, z) represents the classifier misleading evaluation function; `dist` represents the set distance function; either L1-norm or L2-norm can be used. x represents a data sample; y(x) is the true category label corresponding to the data sample; γ is the balance coefficient, and the value range of γ is [0-100]. F(x+z,y(x))=max{logf(x+z,y(x))-y i m ≠y a ( x x) {logf(x+z,y i )},-k} z represents the perturbation learned by the diffusion model; F is the evaluation function used to evaluate the quality of the disturbance z; f represents the machine learning model being attacked; k is a preset hyperparameter, and the value range of k is [20-100]. i is used to indicate the category index.

4. The adversarial attack sample generation method according to claim 2, characterized in that, Determining the first parameter corresponding to the diffusion model includes: Randomly select a time step within the time step interval corresponding to the diffusion model and randomly select Gaussian noise from the pre-defined Gaussian distribution; A value is randomly selected from the standard Gaussian distribution as a new initial value for the adversarial perturbation distribution corresponding to the diffusion model; The first parameter is composed of the time step, the Gaussian noise, and the new initial value.

5. A device for generating adversarial attack samples, characterized in that, include: The first determining unit is used to determine the initial value of the adversarial perturbation distribution corresponding to the established diffusion model; An evaluation unit is used to evaluate the adversarial perturbation distribution based on the initial value of the adversarial perturbation distribution and the created classifier misleading evaluation function, and to determine the perturbation score of the adversarial perturbation distribution. The first generation unit is used to determine the first parameter corresponding to the diffusion model if the perturbation score is not less than a preset score threshold, and generate the state value of the adversarial perturbation distribution based on the first parameter. The second determining unit is used to determine the prediction noise corresponding to the state value; An estimation unit is used to estimate the theoretical distribution of the anti-perturbation distribution based on the predicted noise; A calculation unit is used to calculate the loss value of the loss function corresponding to the diffusion model based on the theoretical distribution; The third determining unit is used to update the model parameters of the diffusion model based on the loss value and in conjunction with the set gradient descent optimization algorithm, and to determine the perturbation score of the adversarial perturbation distribution corresponding to the updated diffusion model. The output unit is used to determine a new first parameter and update the model parameters of the diffusion model again if the perturbation score of the adversarial perturbation distribution corresponding to the updated diffusion model is not less than the preset score threshold, until the perturbation score of the adversarial perturbation distribution corresponding to the updated diffusion model is less than the preset score threshold, and then output the current model parameters of the diffusion model. The second generation unit is used to generate adversarial sampling based on the current model parameters of the diffusion model; The third generation unit is used to generate adversarial attack samples based on the adversarial perturbation sampling.

6. The adversarial attack sample generation device according to claim 5, characterized in that, The first determining unit is specifically configured as follows: Multiple data samples are determined from the selected dataset; the mean and variance of each data sample are determined; based on the mean and variance of each data sample, the standard Gaussian distribution corresponding to the adversarial perturbation distribution is determined; a value is randomly selected from the standard Gaussian distribution as the initial value of the adversarial perturbation distribution corresponding to the diffusion model.

7. The adversarial attack sample generation device according to claim 5, characterized in that, The classifier misleading evaluation function is specifically as follows: j(x,z)=dist(x+z,x)+γF(x+z,y(x)) in: j(x, z) represents the classifier misleading evaluation function; `dist` represents the set distance function; either L1-norm or L2-norm can be used. x represents a data sample; y(x) is the true category label corresponding to the data sample; γ is the balance coefficient, and the value range of γ is [0-100]. z represents the perturbation learned by the diffusion model; F is the evaluation function used to evaluate the quality of the disturbance z; f represents the machine learning model being attacked; k is a preset hyperparameter, and the value range of k is [20-100]. i is used to indicate the category index.

8. The adversarial attack sample generation device according to claim 7, characterized in that, The first generation unit is specifically configured as follows: A time step is randomly selected within the time step interval corresponding to the diffusion model, and Gaussian noise is randomly selected from the pre-defined Gaussian distribution; a value is randomly selected from the standard Gaussian distribution as a new initial value for the anti-disturbance distribution corresponding to the diffusion model; the time step, the Gaussian noise, and the new initial value are combined to form the first parameter.

9. A storage medium, characterized in that, The storage medium includes stored instructions, wherein, when the instructions are executed, the device containing the storage medium is controlled to perform an adversarial attack sample generation method as described in any one of claims 1 to 4.

10. An electronic device, characterized in that, It includes a memory and one or more instructions, wherein the one or more instructions are stored in the memory and configured to be executed by one or more processors as described in any one of claims 1 to 4.

Images