Adversarial sample generation method and device

Abstract

The disclosure provides a method and device for generating an adversarial sample, the method for generating an adversarial sample comprising: obtaining a training sample set and a machine learning model, wherein each training sample in the training sample set has a discrete field attribute, and the discrete field attribute refers to a field attribute with a discrete variable as a field value; converting the field value of each training sample on each discrete field attribute into a field value vector to obtain a vector sample; applying a perturbation to the field value vector of each vector sample on each discrete field attribute, and taking the vector sample after the perturbation as a first perturbed sample; and generating an adversarial sample based on the first perturbed sample. The method and device for generating an adversarial sample according to the disclosure solve the problem of difficulty in generating an adversarial sample for discrete variable data, and can generate an adversarial sample for adversarial learning for data with a discrete variable by converting the field value on a discrete field attribute into a field value vector.

Description

Technical Field

[0001] The following description relates to the field of artificial intelligence technology, specifically to a method and apparatus for generating adversarial examples. Background Technology

[0002] Adversarial learning is a method of studying the robustness of machine learning models by attacking them and causing them to make incorrect predictions. Specifically, the original training samples used to train the machine learning model can be perturbed to a certain extent to obtain adversarial examples. This causes the machine learning model to give inconsistent predictions for the input samples before and after the perturbation (i.e., the original training samples and the adversarial examples), or to give a specific incorrect prediction for the perturbed adversarial examples.

[0003] Existing adversarial example generation methods for adversarial learning are typically limited to cases where the original training samples are continuous variable data such as image data and text data. This makes it difficult to generate adversarial examples for discrete variable data in domains. For example, in many fields such as finance, healthcare, and education, the common data type is tabular data (or structured data), which may have discrete field values. Existing adversarial example generation methods cannot perturb such discrete field values, making it impossible to perform adversarial learning on machine learning models in these domains, and thus unable to improve the robustness of machine learning models through adversarial learning. Summary of the Invention

[0004] The exemplary embodiments disclosed herein may at least solve the above-described problems, or may not solve the above-described problems.

[0005] According to a first aspect of this disclosure, a method for generating adversarial examples is provided. The method includes: acquiring a training sample set and a machine learning model, wherein each training sample in the training sample set has discrete field attributes, where each discrete field attribute is a field attribute whose field value is a discrete variable, and the machine learning model is trained based on the training sample set; converting the field value of each training sample in each discrete field attribute into a field value vector to obtain a vector sample, wherein each field value corresponds one-to-one with the field value vector; applying a perturbation to the field value vector of each vector sample in each discrete field attribute, and using the perturbated vector sample as a first perturbation sample; and generating adversarial examples based on the first perturbation sample.

[0006] Optionally, the step of converting the field value of each training sample in each discrete field attribute into a field value vector includes: for each discrete field attribute, determining an encoded value of the relative relationship between legal field values, wherein the legal field value refers to the field value that is allowed to exist under the field attribute, and the encoded value of the relative relationship between legal field values ​​is determined according to a predetermined encoding rule; based on the encoded value of the relative relationship between legal field values, determining a legal field value vector corresponding to the legal field value, wherein there is a one-to-one correspondence between the legal field value and the legal field value vector; and based on the legal field value and the legal field value vector, determining a field value vector corresponding to the field value of each training sample in the corresponding discrete field attribute.

[0007] Optionally, the step of applying a perturbation to the field value vector of each discrete field attribute of each vector sample and using the perturbated vector sample as the first perturbation sample includes: determining the sample perturbation amount for each vector sample, wherein the sample perturbation amount is the gradient of the loss function of the machine learning model with respect to the vector sample; and applying the sample perturbation amount to the vector sample to obtain the first perturbation sample.

[0008] Optionally, the step of generating adversarial examples based on the first perturbation sample includes: for each first perturbation sample, selecting at least one legal field value vector from all legal field value vectors of each discrete field attribute as a legal search vector corresponding to each discrete field attribute; for each first perturbation sample, obtaining all possible vector combinations of legal search vectors on all discrete field attributes, and generating a second perturbation sample in one-to-one correspondence with each vector combination; determining a second perturbation sample that can successfully attack the machine learning model from all second perturbation samples corresponding to each first perturbation sample, and generating adversarial examples based on the successfully attacked second perturbation samples, wherein successfully attacking the machine learning model means that: the output result of the machine learning model corresponding to the second perturbation sample is a preset target output result, and the sample distance between the second perturbation sample and the training sample corresponding to the corresponding first perturbation sample is less than a preset threshold, wherein the sample distance is determined based on the difference between the field values ​​of the second perturbation sample and the corresponding training sample on each field attribute.

[0009] Optionally, the field attributes include discrete field attributes, wherein the difference between the field values ​​of the second perturbation sample and the corresponding training sample in the discrete field attributes is determined by: determining the field value corresponding to the field value vector of the second perturbation sample in each discrete field attribute based on the legal field values ​​and the legal field value vector; and determining the minimum difference between the transformations of the two legal field values ​​corresponding to the field values ​​of the second perturbation sample and the field values ​​of the corresponding training sample based on the difference between every two legal field values ​​among all legal field values ​​in the discrete field attributes, as the difference between the field values ​​of the second perturbation sample and the corresponding training sample in the discrete field attributes.

[0010] Optionally, the amount of difference between any two valid field values ​​among all valid field values ​​on a discrete field attribute is determined by the difficulty of distinguishing between any two valid field values, wherein the difficulty of distinguishing between valid field values ​​is inversely proportional to the amount of difference between valid field values.

[0011] Optionally, the field attribute further includes a continuous field attribute, which refers to a field attribute whose legal field value is a continuous variable. The difference between the field values ​​of the second perturbation sample and the corresponding training sample in the continuous field attribute is determined by taking the absolute value of the difference between the field value of the second perturbation sample and the field value of the corresponding training sample as the difference between the field values ​​of the second perturbation sample and the corresponding training sample in the continuous field attribute.

[0012] Optionally, the sample distance can be determined by the following formula:

[0013]

[0014] Where d is the sample distance between the second perturbation sample and the corresponding training sample, d k is the difference between the field value of the second perturbation sample and the corresponding training sample in the k-th field attribute, s is the number of field attributes that the second perturbation sample and the corresponding training sample have, and p is a preset parameter, p > 0.

[0015] Optionally, for each first perturbation sample, the step of selecting at least one legal field value vector from all legal field value vectors of each discrete field attribute as a legal search vector corresponding to each discrete field attribute includes: determining all legal field value vectors of each discrete field attribute; for each discrete field attribute, determining the legal field value vectors that meet preset conditions as the legal search vectors of the corresponding discrete field attribute, wherein the preset conditions include: the distance between the legal field value vector and the field value vector of the first perturbation sample is less than a preset threshold, or, a preset number of legal field value vectors that are closest to the field value vector of the first perturbation sample among all legal field value vectors.

[0016] Optionally, the preset target output result is an output result different from the original output result, which is the output result obtained by inputting the training sample corresponding to the second perturbation sample into the machine learning model.

[0017] Optionally, the machine learning model is trained by: determining the vector samples that correspond one-to-one with the training samples in the training sample set to obtain a vector sample set; and using the vector sample set to train the machine learning model.

[0018] According to a second aspect of this disclosure, an adversarial example generation apparatus is provided, comprising: an acquisition unit configured to acquire a training sample set and a machine learning model, wherein each training sample in the training sample set has a discrete field attribute, the discrete field attribute referring to a field attribute whose field value is a discrete variable, and the machine learning model being trained based on the training sample set; a conversion unit configured to convert the field value of each training sample in each discrete field attribute into a field value vector to obtain a vector sample, wherein the field value and the field value vector correspond one-to-one; a perturbation unit configured to apply a perturbation to the field value vector of each vector sample in each discrete field attribute, and use the perturbated vector sample as a first perturbation sample; and a generation unit configured to generate adversarial examples based on the first perturbation sample.

[0019] Optionally, the conversion unit is further configured to: for each discrete field attribute, determine an encoded value of the relative relationship between legal field values, wherein the legal field value refers to the field value that is allowed to exist under the field attribute, and the encoded value of the relative relationship between legal field values ​​is determined according to a predetermined encoding rule; based on the encoded value of the relative relationship between legal field values, determine a legal field value vector corresponding to the legal field value, wherein there is a one-to-one correspondence between the legal field value and the legal field value vector; and based on the legal field value and the legal field value vector, determine a field value vector corresponding to the field value of each training sample in the corresponding discrete field attribute.

[0020] Optionally, the perturbation unit is further configured to: determine a sample perturbation amount for each vector sample, wherein the sample perturbation amount is the gradient of the loss function of the machine learning model with respect to the vector sample; and apply the sample perturbation amount to the vector sample to obtain the first perturbation sample.

[0021] Optionally, the generation unit is further configured to: for each first perturbation sample, select at least one legal field value vector from all legal field value vectors of each discrete field attribute as a legal search vector corresponding to each discrete field attribute; for each first perturbation sample, obtain all possible vector combinations of legal search vectors on all discrete field attributes, and generate a second perturbation sample in one-to-one correspondence with each vector combination; determine a second perturbation sample that can successfully attack the machine learning model from all second perturbation samples corresponding to each first perturbation sample, and generate adversarial samples based on the successfully attacked second perturbation samples, wherein successfully attacking the machine learning model means that: the output result of the machine learning model corresponding to the second perturbation sample is a preset target output result, and the sample distance between the second perturbation sample and the training sample corresponding to the corresponding first perturbation sample is less than a preset threshold, wherein the sample distance is determined based on the difference between the field values ​​of the second perturbation sample and the corresponding training sample on each field attribute.

[0022] Optionally, the field attributes include discrete field attributes, wherein the generation unit is configured to determine the amount of difference between the field values ​​of the second perturbation sample and the corresponding training sample in the discrete field attributes by: determining the field value corresponding to the field value vector of the second perturbation sample in each discrete field attribute based on the legal field value and the legal field value vector; and determining the minimum difference between the transformations of the two legal field values ​​corresponding to the field values ​​of the second perturbation sample and the field values ​​of the corresponding training sample, based on the amount of difference between every two legal field values ​​among all legal field values ​​in the discrete field attributes, as the amount of difference between the field values ​​of the second perturbation sample and the corresponding training sample in the discrete field attributes.

[0023] Optionally, the amount of difference between any two valid field values ​​among all valid field values ​​on a discrete field attribute is determined by the difficulty of distinguishing between any two valid field values, wherein the difficulty of distinguishing between valid field values ​​is inversely proportional to the amount of difference between valid field values.

[0024] Optionally, the field attribute further includes a continuous field attribute, which refers to a field attribute whose legal field value is a continuous variable. The generation unit is configured to determine the difference between the field values ​​of the second perturbation sample and the corresponding training sample in the continuous field attribute by taking the absolute value of the difference between the field value of the second perturbation sample and the field value of the corresponding training sample as the difference between the field values ​​of the second perturbation sample and the corresponding training sample in the continuous field attribute.

[0025] Optionally, the generation unit is configured to determine the sample distance using the following formula:

[0026]

[0027] Where d is the sample distance between the second perturbation sample and the corresponding training sample, d k is the difference between the field value of the second perturbation sample and the corresponding training sample in the k-th field attribute, s is the number of field attributes that the second perturbation sample and the corresponding training sample have, and p is a preset parameter, p > 0.

[0028] Optionally, the generation unit is further configured to: determine all legal field value vectors for each discrete field attribute; and for each discrete field attribute, determine the legal field value vectors that satisfy preset conditions as the legal search vectors for the corresponding discrete field attribute, wherein the preset conditions include: the distance between the legal field value vector and the field value vector of the first perturbation sample is less than a preset threshold, or, a preset number of legal field value vectors that are closest to the field value vector of the first perturbation sample among all legal field value vectors.

[0029] Optionally, the preset target output result is an output result different from the original output result, which is the output result obtained by inputting the training sample corresponding to the second perturbation sample into the machine learning model.

[0030] Optionally, the adversarial example generation apparatus further includes a training unit configured to train the machine learning model by: determining vector samples that correspond one-to-one with the training samples in the training sample set to obtain a vector sample set; and training the machine learning model using the vector sample set.

[0031] According to a third aspect of this disclosure, an electronic device is provided, the electronic device comprising: a processor; and a memory for storing processor-executable instructions, wherein the processor-executable instructions, when executed by the processor, cause the processor to perform an adversarial sample generation method according to this disclosure.

[0032] According to a fourth aspect of this disclosure, a computer-readable storage medium is provided that stores instructions which, when executed by at least one computing device, cause the at least one computing device to perform the adversarial sample generation method according to this disclosure.

[0033] According to a fifth aspect of this disclosure, a system is provided that includes at least one computing device and at least one storage device for storing instructions, wherein when the instructions are executed by the at least one computing device, the at least one computing device causes the at least one computing device to perform an adversarial sample generation method according to this disclosure.

[0034] According to the adversarial example generation method and apparatus disclosed herein, adversarial examples for adversarial learning can be generated for data with discrete variables by converting field values ​​on discrete field attributes into field value vectors. Attached Figure Description

[0035] Figure 1 This is a flowchart illustrating a method for generating adversarial samples according to an exemplary embodiment of the present disclosure.

[0036] Figure 2 This is a flowchart illustrating the steps of generating vector samples in an adversarial sample generation method according to an exemplary embodiment of the present disclosure.

[0037] Figure 3 This is a flowchart illustrating the steps of generating adversarial samples in an adversarial sample generation method according to an exemplary embodiment of the present disclosure.

[0038] Figure 4 This is a flowchart illustrating the step of determining the amount of difference between field values ​​on discrete field attributes in a method for generating adversarial examples according to an exemplary embodiment of the present disclosure.

[0039] Figure 5 This is a schematic diagram illustrating the determination of the amount of difference between field values ​​on discrete field attributes in an adversarial sample generation method according to an exemplary embodiment of the present disclosure.

[0040] Figure 6 This is a schematic block diagram illustrating an apparatus for generating adversarial samples according to an exemplary embodiment of the present disclosure. Detailed Implementation

[0041] The following description, provided with reference to the accompanying drawings, is intended to aid in a full understanding of embodiments of the present disclosure as defined by the claims and their equivalents. Various specific details are included to aid understanding, but these details are to be considered exemplary only. Therefore, those skilled in the art will recognize that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Furthermore, for clarity and brevity, descriptions of well-known functions and structures are omitted.

[0042] It should be noted that the phrase "at least one of several items" in this disclosure refers to three parallel cases: "any one of the several items", "a combination of any number of the several items", and "all of the several items". For example, "including at least one of A and B" includes the following three parallel cases: (1) including A; (2) including B; (3) including A and B. As another example, "performing at least one of step one and step two" indicates the following three parallel cases: (1) performing step one; (2) performing step two; (3) performing both step one and step two.

[0043] Generally speaking, machine learning models that have not undergone adversarial learning are less robust and more likely to produce incorrect predictions compared to those that have undergone adversarial learning.

[0044] In many fields such as finance, healthcare, and education, the erroneous predictions made by unrobust machine learning models can lead to significant losses. Therefore, the robustness of machine learning models is usually highly demanding. Thus, it is necessary to improve the robustness of machine learning models through adversarial learning.

[0045] However, since the common data type in these fields is table data (also known as structured data) with discrete field values, and existing adversarial learning adversarial example generation methods such as adversarial attack algorithms FGSM, I-FGSM, and MI-FGSM cannot generate adversarial examples by applying perturbations to table data, it is impossible to implement adversarial learning on machine learning models. Therefore, it is essential to extend adversarial learning techniques to discrete variable data.

[0046] Table data adversarial attacks refer to constructing a new table data sample x′ from the original model f to be attacked and a given original table data sample x, making the new sample indistinguishable from the original sample, and the original model's prediction result for the new sample is a pre-set label y′ (this prediction result is different from the original sample's prediction result y=f(x)).

[0047] Generally, a specific distance metric is designed to measure whether new samples and original samples are difficult to distinguish. The corresponding distance metric we designed for tabular data is described later in this paper, denoted as d(x, x′). The adversarial attack problem on tabular data can be formally defined as the following optimization problem:

[0048] x′=arg min d(x,x′)

[0049] stf(x′=y

[0050] In general application scenarios, it is difficult and unnecessary to accurately find the sample with the smallest distance. It is sufficient to ensure that the distance d(x, x′) between the adversarial sample and the original sample does not exceed a specific threshold ∈. In this case, it can be formally defined as constructing a sample x′ such that it satisfies f(x′)=y and d(x, x′)≤∈.

[0051] It is worth noting that the definitions of general adversarial learning and tabular data adversarial learning are completely similar, differing only in the definition of the distance metric. The distance metric is precisely the core issue in the definition of adversarial learning, because adversarial examples are only meaningful if they are sufficiently similar to the original samples. The exemplary embodiments of this disclosure design a distance metric definition for tabular data, thereby extending adversarial learning to the scope of tabular data, which is one of the main contributions of the exemplary embodiments of this disclosure.

[0052] The following description, with reference to the accompanying drawings, describes an exemplary embodiment of the present disclosure, including a method for generating adversarial samples, an apparatus for generating adversarial samples, an electronic device, a computer-readable storage medium, and a system comprising at least one computing device and at least one storage instruction.

[0053] Figure 1 This is a flowchart illustrating a method for generating adversarial examples according to exemplary embodiments of the present disclosure. Figure 1 As shown, the method for generating adversarial examples may include the following steps:

[0054] In step S10, a training sample set and a machine learning model can be obtained, wherein the machine learning model is trained using the training sample set.

[0055] Here, each training sample in the training sample set has a discrete field attribute, which refers to a field attribute whose field value is a discrete variable. As an example, a discrete field attribute can be a category attribute.

[0056] For example, a discrete field attribute could be "marital status," and the possible valid field values ​​under this attribute could include "married," "single," "divorced," and "widowed." It's clear that these valid field values ​​are discrete quantities and not numerical representations. Traditional adversarial example generation methods can only apply perturbations to continuous variables to obtain adversarial examples. Therefore, in application scenarios involving table data, traditional adversarial example generation methods cannot handle sample data with discrete field attributes.

[0057] In step S20, the field values ​​of each training sample for each discrete field attribute can be converted into field value vectors to obtain vector samples. Here, there is a one-to-one correspondence between field values ​​and field value vectors.

[0058] In this step, since the field values ​​on the discrete field attribute are discrete variables, the discrete variables can be represented as vectors that can be used for mathematical operations before perturbing the training samples.

[0059] As an example, refer to Figure 2 Step S20 may include the following steps:

[0060] Step S21 can determine the encoded value of the relative relationship between legal field values ​​for each discrete field attribute.

[0061] Here, a valid field value refers to a field value that is allowed to exist under a field attribute. The encoded value of the relative relationship between valid field values ​​can be determined according to a predetermined encoding rule.

[0062] In the example above, the legal field values ​​are "married", "unmarried", "divorced" and "widowed". The encoding value of the relative relationship between each pair of legal field values ​​"married", "unmarried", "divorced" and "widowed" can be set arbitrarily. For example, the encoding value between the legal field values ​​"married" and "unmarried" can be 1, the encoding value between the legal field values ​​"married" and "divorced" can be 2, and the encoding value between the legal field values ​​"married" and "widowed" can be 3.

[0063] Step S22: Based on the encoded values ​​of the relative relationships between legal field values, a legal field value vector corresponding to the legal field value can be determined, wherein there is a one-to-one correspondence between the legal field value and the legal field value vector.

[0064] In this step, for each valid field value, a valid field value vector corresponding to that valid field value can be generated based on the encoded value of the relative relationship between that valid field value and other valid field values. In the example above, the valid field value vector corresponding to the valid field value "married" could be {1, 2, 3}.

[0065] Furthermore, the dimension of the valid field value vector is related to the encoding rules that define the relative relationships between valid field values. The more complex and refined the encoding rules, the higher the dimension of the valid field value vector can be. As an example, steps S21 and S22 above can be implemented using the method of generating embedding vectors in the Node2Vec algorithm. Specifically, each valid field value can be treated as a node, and the relative relationships between valid field values ​​can be treated as edges between nodes. Each edge is encoded to generate a node graph. Then, based on the distance of each node to other nodes in the node graph (i.e., the sum of the encodings), a vector corresponding to each node can be generated.

[0066] Step S23 can determine the field value vector corresponding to the field value of each training sample on the corresponding discrete field attribute based on the legal field value and the legal field value vector.

[0067] In this step, since the field value of each training sample on the discrete field attribute should be one of the legal field values ​​on that discrete field attribute, the legal field value vector corresponding to the legal field value corresponding to the field value of the training sample can be determined as the field value vector of that field value.

[0068] Furthermore, in step S10 above, the machine learning model can be trained by converting the training samples into vector form. Here, the vector form training samples can be obtained from the training samples through any vector transformation method, or they can be obtained through the vector transformation method described in step S20 of this disclosure.

[0069] Specifically, a machine learning model can be trained by: identifying vector samples that correspond one-to-one with the training samples in the training sample set, thus obtaining a vector sample set; and then using the vector sample set to train the machine learning model.

[0070] In this example, the operation of obtaining the machine learning model can be performed after step S20. For example, after converting the field values ​​of each training sample on each discrete field attribute into a field value vector to obtain a vector sample, the machine learning model is trained.

[0071] Return to reference Figure 1 In step S30, a perturbation can be applied to the field value vector of each discrete field attribute for each vector sample, and the perturbed vector sample is used as the first perturbation sample.

[0072] As an example, step S30 may include: determining a sample perturbation amount for each vector sample; applying the sample perturbation amount to the vector sample to obtain a first perturbation sample.

[0073] Here, the sample perturbation can be the gradient of the loss function of the machine learning model with respect to the vector samples. The loss function is determined before the machine learning model is trained.

[0074] According to exemplary embodiments of this disclosure, training samples may have only discrete field attributes, or they may have both discrete field attributes and continuous field attributes. Here, continuous field attributes refer to field attributes whose valid field values ​​are continuous variables.

[0075] For the case where the training samples only have discrete field attributes, assume the training sample set is... Where, x iLet N be the training samples. The training sample set contains N training samples, and the vector sample corresponding to each training sample can be represented as: Among them, u q Let q be the field value vector of the training sample on the q-th discrete field attribute, where q∈(1,n), and n is the number of discrete field attributes that each training sample has.

[0076] Then the first perturbation sample x′ i It can be determined by the following formula (1):

[0077]

[0078] in, Represents the AND vector samples of a machine learning model Corresponding to (or, with training sample x) i The corresponding output, y i This represents the preset target output result. Here, the preset target output result is... different, The loss function of a machine learning model on vector samples The gradient, which is represented by the vector sample. The corresponding training sample x i The trend of the influence on the loss function during the training process of the machine learning model, λ is a preset coefficient that can be set arbitrarily as needed. As an example, λ can be a hyperparameter in the training process of the machine learning model and can be determined based on the algorithm of the machine learning model.

[0079] The first perturbation sample obtained in this way can be represented as x′ i = (u′1, u′2, ..., u′) q ,...,u′ n ), where u′ q Let be the field value vector of the first perturbation sample on the qth discrete field attribute.

[0080] Similarly, for the case where training samples have both discrete and continuous field attributes, assuming the training sample set is... Where, x i Let N be the training samples. The training sample set contains N training samples, and the vector sample corresponding to each training sample can be represented as: Where v represents the field value of the training sample across all continuous field attributes, u q Let x' be the field value vector of the training sample at the q-th discrete field attribute, where q ∈ (1, n), and n is the number of discrete field attributes possessed by each training sample. In this case, the first perturbation sample x' iIt can still be determined using the above equation (1), that is, And the first perturbation sample x′ can be obtained. i = (u′1, u′2, ..., u′) q ,...,u′ n ,v′), where v′ represents the field value of the first perturbation sample on all continuous field attributes, and u′q is the field value vector of the first perturbation sample on the qth discrete field attribute.

[0081] It should be noted that the form of equation (1) above is only an example. The gradient calculation of the loss function of the machine learning model on the vector sample can also be a multi-step iterative gradient (e.g., the I-FGSM method), gradient descent based on momentum calculation (e.g., the MI-FGSM method), etc.

[0082] In step S40, adversarial examples can be generated based on the first perturbation sample.

[0083] In one example, the machine learning model may be trained based on a set of vector samples according to exemplary embodiments of the present disclosure, in which case the first perturbation sample may be directly used as an adversarial sample for adversarial learning of the trained machine learning model.

[0084] In another example, such as Figure 3 As shown, step S40 may include the following steps:

[0085] In step S41, for each first perturbation sample, at least one legal field value vector can be selected from all legal field value vectors of each discrete field attribute as a legal search vector corresponding to each discrete field attribute.

[0086] As an example, step S41 may include: determining all valid field value vectors for each discrete field attribute; and for each discrete field attribute, adding the valid field value vectors that meet preset conditions to the valid search vector of the corresponding discrete field attribute.

[0087] In one example, the preset condition could be: the distance between the valid field value vector and the field value vector of the first perturbation sample is less than a preset threshold.

[0088] Specifically, in this example, all valid field value vectors for each discrete field attribute can be determined. All valid field value vectors are iterated over, and the vector distance between each valid field value vector and the field value vector of the first perturbation sample in the corresponding discrete field attribute is calculated. If the vector distance is less than a preset threshold, the valid field value vector can be used as a valid search vector. The preset threshold can be arbitrarily set according to actual needs.

[0089] Alternatively to the above example, or additionally beyond the above example, in another example, the preset condition could be: a preset number of valid field value vectors that are closest to the field value vector of the first perturbation sample among all valid field value vectors.

[0090] Specifically, in this example, all legal field value vectors for each discrete field attribute can be determined, all legal field value vectors can be traversed, and the vector distance between each legal field value vector and the field value vector of the first perturbation sample in the corresponding discrete field attribute can be calculated. The legal field value vectors with the smallest vector distance are selected as the legal search vectors. Here, the preset number can be set arbitrarily according to actual needs, and it can be an integer belonging to [1, h], where h is the number of legal field value vectors (or legal field values) of the corresponding discrete field attribute.

[0091] In this step, the vector distance can be determined using any vector distance calculation method, such as the Euclidean distance formula, Manhattan distance formula, Chebyshev distance formula, or Mahalanobis distance formula. Furthermore, there can be one or more valid search vectors corresponding to each discrete field attribute of each first perturbation sample.

[0092] In step S42, for each first perturbation sample, all possible vector combinations of legal search vectors on all discrete field attributes can be obtained, and a second perturbation sample can be generated in one-to-one correspondence with each vector combination.

[0093] Specifically, since for each first perturbation sample, there can be one or more legal search vectors for each discrete field attribute determined in step S41, all legal search vectors for all discrete field attributes can be freely combined to obtain all possible vector combinations. Each vector combination can be used to generate a second perturbation sample; that is, the legal search vectors for each discrete field attribute in each vector combination are used as the field value vectors of the second perturbation sample for the corresponding discrete field attribute. Since there can be one or more vector combinations corresponding to each first perturbation sample, there can also be one or more second perturbation samples corresponding to each first perturbation sample.

[0094] In step S43, a second perturbation sample capable of successfully attacking the machine learning model can be determined from all the second perturbation samples corresponding to each first perturbation sample, and an adversarial sample is generated based on the successfully attacked second perturbation sample.

[0095] In this step, a successful attack on the machine learning model means that the output of the machine learning model corresponding to the second perturbation sample is the preset target output, and the sample distance between the second perturbation sample and the training sample corresponding to the first perturbation sample is less than a preset threshold.

[0096] As an example, when the machine learning model is trained based on the vector sample set described in the exemplary embodiments of this disclosure, a second perturbation sample can be input into the machine learning model to obtain the output of the machine learning model, and it can be determined whether the output is the preset target output.

[0097] Here, the preset target output is an output that differs from the original output. Specifically, the target output can be a pre-specified output, or no specific output can be pre-specified, as long as the output of the second perturbation sample differs from the original output. The original output is obtained by inputting the training sample corresponding to the first perturbation sample in the training sample set (which corresponds to the second perturbation sample) into the machine learning model. Furthermore, the preset threshold can be arbitrarily set according to the actual application.

[0098] In this way, the second perturbation sample that was successfully attacked can be used as an adversarial sample.

[0099] In this step, the sample distance can be determined based on the amount of difference between the field values ​​of the second perturbation sample and the corresponding training sample on each field attribute.

[0100] Specifically, the sample distance can be determined using the following formula (2):

[0101]

[0102] Where d is the sample distance between the second perturbation sample and the corresponding training sample, d k s is the difference between the field value of the second perturbation sample and the corresponding training sample in the k-th field attribute, s is the number of field attributes that the second perturbation sample and the corresponding training sample have, and p is a preset parameter, p>0. For example, p can be an arbitrarily set value for ease of calculation.

[0103] According to an exemplary embodiment of this disclosure, the field attributes of the training samples and the second perturbation samples may include discrete field attributes, such as... Figure 4 As shown, the difference between the field values ​​of the second perturbation sample and the corresponding training sample in the discrete field attribute can be determined in the following way:

[0104] In step S431, the field value corresponding to the field value vector of the second perturbation sample on each discrete field attribute can be determined based on the legal field value and the legal field value vector.

[0105] Specifically, since the second perturbation sample is obtained based on all possible vector combinations of legal search vectors on all discrete field attributes, the field value vectors of the second perturbation sample on all discrete field attributes should all be legal field value vectors on each discrete field attribute. Thus, based on the correspondence between legal field value vectors and legal field values, the field value vectors of the second perturbation sample on each discrete field attribute can be converted into field values.

[0106] In step S432, the minimum difference between two legal field values ​​corresponding to the field value of the second perturbation sample and the field value of the corresponding training sample can be determined based on the difference between every two legal field values ​​among all legal field values, and this difference can be used as the difference between the field values ​​of the second perturbation sample and the corresponding training sample in the discrete field attribute.

[0107] In this step, the amount of difference between any two valid field values ​​among all valid field values ​​of the discrete field attribute can be determined by the difficulty of distinguishing between any two valid field values, wherein the difficulty of distinguishing between valid field values ​​is inversely proportional to the amount of difference between valid field values.

[0108] Specifically, the difference between any two valid field values ​​can be determined. This difference represents the difficulty for a machine learning model to distinguish between the two valid field values; the larger the difference, the easier it is to distinguish them; the smaller the difference, the more difficult it is to distinguish them. The difficulty of distinguishing can be predetermined based on the actual application scenario.

[0109] As an example, the amount of difference between valid field values ​​can be determined by generating a weighted undirected graph (also known as a distance graph). Specifically, for each discrete field attribute, each valid field value can be treated as a node, and the distance between two valid field values ​​represents the amount of difference between them. The greater the distance between two valid field values, the greater the difference and the easier it is for a machine learning model to distinguish them. For example, ... Figure 5 As shown, the valid field values ​​for a certain discrete field attribute can be A, B, C, and D. The difficulty of distinguishing between any two field values ​​is determined based on the actual application scenario, and the distance between nodes is weighted accordingly. The greater the difficulty, the smaller the weight; the smaller the difficulty, the larger the weight. For example, the difference between A and C is 20, and the difference between A and D is 50. It is important to note that in a weighted undirected graph, the distance between two nodes is represented only by the weighting and is independent of the actual length of the connecting line when drawing the graph.

[0110] Returning to step S432, the field value of the second perturbation sample and the field value of its corresponding training sample each correspond to two valid field values. These two valid field values ​​can be the same or different. Based on the difference between each pair of valid field values, the minimum difference required for transformation between the corresponding two valid field values ​​can be calculated. This transformation can be a direct transformation or an indirect transformation. An indirect transformation refers to a transformation via field values ​​other than the two specified field values. This minimum difference can refer to the shortest transformation path between the two valid field values.

[0111] For example, such as Figure 5 As shown, the difference between A and C is 20, the difference between A and D is 50, and the difference between C and D is 20. When the two corresponding valid field values ​​are A and D, the difference when A is directly transformed to D is 50, while the difference when A is first transformed to C and then to D is 20 + 20, which is 40. Therefore, the minimum difference between A and D is 40, which means the total weight of the shortest path is 40.

[0112] The method described above for determining the distance between legal field values ​​of discrete field attributes using a weighted undirected graph actually provides a distance measurement method for discrete variable data. Based on this distance measurement method and the output of the machine learning model, it can be determined whether the perturbation sample is a sample that successfully attacks the model.

[0113] The method for determining the difference between field values ​​of discrete field attributes has been described above with reference to steps S431 to S432. As described above, according to an exemplary embodiment of this disclosure, the field attributes of the training sample and the second perturbation sample may further include continuous field attributes; that is, the training sample and the second perturbation sample may have both discrete field attributes and continuous field attributes. In this case, as described above, the vector sample corresponding to each training sample can be represented as... Where v represents the field value of the training sample across all continuous field attributes, and the first perturbation sample can be represented as x′. i = (u′1, u′2, ..., u′) q ,...,u′ n v′ can also be calculated using the above formula (1), where v′ represents the field value of the first perturbation sample in all continuous field attributes.

[0114] As an example, the difference between the field values ​​of the second perturbation sample and the corresponding training sample in the continuous field attribute can be determined by taking the absolute value of the difference between the field values ​​of the second perturbation sample and the field values ​​of the corresponding training sample as the difference between the field values ​​of the second perturbation sample and the corresponding training sample in the continuous field attribute.

[0115] Specifically, since the field values ​​on a continuous field attribute are consecutive, the difference between the two can be determined by subtracting them.

[0116] As an example, in step S43, adversarial examples can be generated by searching for second perturbation samples with valid field values ​​from the second perturbation samples that can successfully attack the model. Here, the beam width of the beam search can be a preset number of valid field value vectors that are closest to the field value vector of the first perturbation sample when determining the valid search vector.

[0117] Here, when the training samples have both discrete and continuous field attributes, the above-mentioned processing procedure for continuous field attributes can be added on the basis of the case where the training samples only have discrete field attributes. In other words, the processing procedure for discrete field attributes is the same whether the training samples have only discrete field attributes or both discrete and continuous field attributes.

[0118] In traditional adversarial learning algorithms, all attributes of the training samples are processed and calculated as a whole; that is, traditional adversarial learning algorithms can only handle continuous variable data. However, the adversarial example generation method according to an exemplary embodiment of this disclosure can generate adversarial examples for adversarial learning based on discrete field attributes by converting field values ​​into field value vectors. Furthermore, this method can be combined with existing adversarial example generation methods such as the FGSM algorithm and Generative Adversarial Networks (GANs), which only handle continuous variable data, thereby enabling its application in generating adversarial examples for discrete variable data.

[0119] Furthermore, the adversarial example generation method according to the exemplary embodiments of this disclosure can calculate the difference between discrete field attributes and continuous field attributes using different difference calculation methods. In contrast, traditional attack algorithms can only perturb the field values ​​of continuous field attributes, but cannot distinguish between field attributes to achieve perturbation.

[0120] The adversarial example generation method according to exemplary embodiments of this disclosure can generate adversarial examples, thereby allowing analysis of the adversarial examples to evaluate the stability of machine learning models and guide subsequent optimization of machine learning models. For example, by comparing training samples with adversarial examples, it can be analyzed which field attributes of the training samples were perturbed to successfully generate adversarial examples, and the performance of the machine learning model can be evaluated by analyzing the magnitude of the differences between these generated adversarial examples and the training samples.

[0121] like Figure 6As shown, an exemplary embodiment of this disclosure also provides an adversarial sample generation apparatus, which includes an acquisition unit 100, a conversion unit 200, a perturbation unit 300, and a generation unit 400.

[0122] The acquisition unit 100 can be configured to acquire a training sample set and a machine learning model, wherein each training sample in the training sample set has a discrete field attribute, which refers to a field attribute whose field value is a discrete variable, and the machine learning model is trained based on the training sample set.

[0123] The transformation unit 200 can be configured to convert the field value of each training sample in each discrete field attribute into a field value vector to obtain a vector sample, wherein the field value and the field value vector correspond one-to-one.

[0124] The perturbation unit 300 can be configured to apply a perturbation to the field value vector of each discrete field attribute for each vector sample, and use the perturbated vector sample as the first perturbation sample.

[0125] The generation unit 400 can be configured to generate adversarial examples based on the first perturbation sample.

[0126] As an example, the transformation unit 200 can also be configured to: determine, for each discrete field attribute, an encoded value of the relative relationship between legal field values, wherein a legal field value refers to a field value that is allowed to exist under the field attribute, and the encoded value of the relative relationship between legal field values ​​is determined according to a predetermined encoding rule; determine a legal field value vector corresponding to the legal field value based on the encoded value of the relative relationship between legal field values, wherein there is a one-to-one correspondence between the legal field value and the legal field value vector; and determine a field value vector corresponding to the field value of each training sample in the corresponding discrete field attribute based on the legal field value and the legal field value vector.

[0127] As an example, the perturbation unit 300 can also be configured to: determine a sample perturbation amount for each vector sample, wherein the sample perturbation amount is the gradient of the loss function of the machine learning model with respect to the vector sample; and apply the sample perturbation amount to the vector sample to obtain a first perturbation sample.

[0128] As an example, the generation unit 400 can also be configured to: for each first perturbation sample, select at least one legal field value vector from all legal field value vectors of each discrete field attribute as a legal search vector corresponding to each discrete field attribute; for each first perturbation sample, obtain all possible vector combinations of legal search vectors on all discrete field attributes, and generate a second perturbation sample in one-to-one correspondence with each vector combination; determine a second perturbation sample that can successfully attack the machine learning model from all second perturbation samples corresponding to each first perturbation sample, and generate adversarial samples based on the successfully attacked second perturbation samples.

[0129] Here, a successful attack on a machine learning model means that the output of the machine learning model corresponding to the second perturbation sample is the preset target output, and the sample distance between the second perturbation sample and the corresponding training sample of the first perturbation sample is less than a preset threshold. The sample distance is determined based on the difference between the field values ​​of each field attribute of the second perturbation sample and the corresponding training sample.

[0130] As an example, the field attributes include discrete field attributes. In this case, the generation unit 400 can be configured to determine the amount of difference between the field values ​​of the second perturbation sample and the corresponding training sample in the discrete field attributes by: determining the field value corresponding to the field value vector of the second perturbation sample in each discrete field attribute based on the legal field values ​​and the legal field value vector; and determining the minimum amount of difference between the transformations of the two legal field values ​​corresponding to the field values ​​of the second perturbation sample and the field values ​​of the corresponding training sample, based on the amount of difference between every two legal field values ​​among all legal field values ​​in the discrete field attributes, as the amount of difference between the field values ​​of the second perturbation sample and the corresponding training sample in the discrete field attributes.

[0131] As an example, the amount of difference between any two valid field values ​​among all valid field values ​​on a discrete field attribute is determined by the difficulty of distinguishing between any two valid field values, where the difficulty of distinguishing between valid field values ​​is inversely proportional to the amount of difference between valid field values.

[0132] As an example, field attributes also include continuous field attributes, which refer to field attributes whose valid field values ​​are consecutive variables.

[0133] In this case, the generation unit 400 can be configured to determine the amount of difference between the field values ​​of the second perturbation sample and the corresponding training sample in the continuous field attribute by taking the absolute value of the difference between the field values ​​of the second perturbation sample and the field values ​​of the corresponding training sample as the amount of difference between the field values ​​of the second perturbation sample and the corresponding training sample in the continuous field attribute.

[0134] As an example, the generation unit 400 can be configured to determine the sample distance using the following formula:

[0135]

[0136] Where d is the sample distance between the second perturbation sample and the corresponding training sample, d k s is the difference between the field value of the second perturbation sample and the corresponding training sample in the k-th field attribute, s is the number of field attributes that the second perturbation sample and the corresponding training sample have, and p is a preset parameter, p>0.

[0137] As an example, the generation unit 400 can also be configured to: determine all legal field value vectors for each discrete field attribute; and for each discrete field attribute, determine the legal field value vectors that meet preset conditions as the legal search vectors for the corresponding discrete field attribute.

[0138] Here, the preset conditions may include: the distance between the valid field value vector and the field value vector of the first perturbation sample is less than a preset threshold, or, a preset number of valid field value vectors that are closest to the field value vector of the first perturbation sample among all valid field value vectors.

[0139] As an example, the preset target output result can be an output result different from the original output result, which can be the output result obtained by inputting the training sample corresponding to the second perturbation sample into the machine learning model.

[0140] As an example, the adversarial example generation device may also include a training unit, which can be configured to train a machine learning model by: determining vector samples that correspond one-to-one with training samples in the training sample set to obtain a vector sample set; and using the vector sample set to train a machine learning model.

[0141] The acquisition unit 100, conversion unit 200, disturbance unit 300, and generation unit 400 can be configured as described above. Figures 1 to 5 The method for generating adversarial examples in the illustrated method embodiment executes the corresponding steps in the method. The specific implementation of the acquisition unit 100, conversion unit 200, perturbation unit 300 and generation unit 400 can be found in the method embodiment described above, and will not be repeated here.

[0142] An exemplary embodiment of this disclosure also provides an electronic device, the electronic device comprising: a processor; and a memory for storing processor-executable instructions, wherein the processor-executable instructions, when executed by the processor, cause the processor to perform an adversarial sample generation method according to this disclosure.

[0143] An exemplary embodiment of this disclosure also provides a computer-readable storage medium for storing instructions that, when executed by at least one computing device, cause the at least one computing device to perform the adversarial sample generation method according to this disclosure.

[0144] An exemplary embodiment of this disclosure also provides a system including at least one computing device and at least one storage device for storing instructions, which, when executed by the at least one computing device, cause the at least one computing device to perform an adversarial sample generation method according to this disclosure.

[0145] Figure 6 The units in the illustrated adversarial example generation apparatus can be configured as software, hardware, firmware, or any combination thereof to perform specific functions. For example, each unit may correspond to a dedicated integrated circuit, pure software code, or a module combining software and hardware. Furthermore, one or more functions implemented by each unit may also be uniformly executed by components in a physical entity device (e.g., a processor, client, or server).

[0146] In addition, refer to Figures 1 to 5 The described method for generating adversarial examples can be implemented by a program (or instructions) recorded on a computer-readable storage medium. For example, according to an exemplary embodiment of the present disclosure, a computer-readable storage medium storing instructions may be provided, wherein when the instructions are executed by at least one computing device, the at least one computing device causes the at least one computing device to perform the method for generating adversarial examples according to the present disclosure.

[0147] The computer program in the aforementioned computer-readable storage medium can run in an environment deployed in computer devices such as clients, hosts, agent devices, and servers. It should be noted that the computer program can also be used to perform additional steps beyond those described above, or to perform more specific processing while performing the above steps. The details of these additional steps and further processing are already described in the reference... Figures 1 to 5 The relevant methods were mentioned in the description of the process, so they will not be repeated here to avoid repetition.

[0148] It should be noted that each unit in the adversarial sample generation apparatus according to the exemplary embodiments of this disclosure can rely entirely on the operation of a computer program to realize its corresponding function. That is, each unit corresponds to each step in the functional architecture of the computer program, so that the entire system is called through a special software package (e.g., a lib library) to realize its corresponding function.

[0149] on the other hand, Figure 6The units shown can also be implemented using hardware, software, firmware, middleware, microcode, or any combination thereof. When implemented in software, firmware, middleware, or microcode, the program code or code segment used to perform the corresponding operation can be stored in a computer-readable medium such as a storage medium, so that the processor can perform the corresponding operation by reading and running the corresponding program code or code segment.

[0150] For example, an exemplary embodiment of the present disclosure can also be implemented as a computing device including a storage component and a processor, wherein the storage component stores a set of computer-executable instructions, and when the set of computer-executable instructions is executed by the processor, the method for generating adversarial samples according to an exemplary embodiment of the present disclosure is executed.

[0151] Specifically, the computing device can be deployed on a server or client, or on node devices in a distributed network environment. Furthermore, the computing device can be a PC, tablet, personal digital assistant, smartphone, web application, or other device capable of executing the aforementioned set of instructions.

[0152] Here, the computing device is not necessarily a single computing device, but can be any collection of devices or circuits capable of executing the aforementioned instructions (or instruction sets) individually or in combination. The computing device can also be part of an integrated control system or system manager, or can be configured to interconnect with a portable electronic device locally or remotely (e.g., via wireless transmission) through an interface.

[0153] In a computing device, a processor may include a central processing unit (CPU), a graphics processing unit (GPU), a programmable logic device, a dedicated processor system, a microcontroller, or a microprocessor. By way of example and not limitation, a processor may also include an analog processor, a digital processor, a microprocessor, a multi-core processor, a processor array, a network processor, etc.

[0154] Some operations described in the adversarial example generation method according to exemplary embodiments of this disclosure can be implemented in software, some operations can be implemented in hardware, and some operations can be implemented in a combination of software and hardware.

[0155] The processor can execute instructions or code stored in one of the storage components, which can also store data. Instructions and data can also be sent and received over a network via a network interface device, which can employ any known transport protocol.

[0156] Storage components can be integrated with the processor, for example, by placing RAM or flash memory within an integrated circuit microprocessor. Alternatively, storage components can include separate devices, such as external disk drives, storage arrays, or other storage devices that can be used by any database system. Storage components and the processor can be operatively coupled, or can communicate with each other, for example, via I / O ports, network connections, etc., enabling the processor to read files stored in the storage component.

[0157] In addition, the computing device may include a video display (such as a liquid crystal display) and a user interaction interface (such as a keyboard, mouse, touch input device, etc.). All components of the computing device may be interconnected via a bus and / or network.

[0158] The method for generating adversarial examples according to exemplary embodiments of this disclosure can be described as various interconnected or coupled functional blocks or functional diagrams. However, these functional blocks or functional diagrams can be equally integrated into a single logic device or operate according to non-precise boundaries.

[0159] Therefore, refer to Figures 1 to 5 The described method for generating adversarial examples can be implemented by a system comprising at least one computing device and at least one storage device containing storage instructions.

[0160] According to an exemplary embodiment of the present disclosure, at least one computing device is a computing device for executing a method for generating adversarial examples according to an exemplary embodiment of the present disclosure, and a storage device stores a set of computer-executable instructions. When the set of computer-executable instructions is executed by the at least one computing device, a method for generating adversarial examples is executed. Figures 1 to 5 The method for generating adversarial examples is described.

[0161] The foregoing has described various exemplary embodiments of this disclosure. It should be understood that the foregoing description is exemplary only and not exhaustive, and this disclosure is not limited to the disclosed exemplary embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of this disclosure. Therefore, the scope of protection of this disclosure should be determined by the scope of the claims.

Claims

1. A method for generating adversarial examples, characterized in that, include: Obtain a training sample set and a machine learning model, wherein each training sample in the training sample set is table data with discrete field attributes, the discrete field attributes refer to field attributes whose field values ​​are discrete variables, and the field values ​​under the discrete field attributes are multiple discrete and non-numerical natural language texts, the machine learning model is trained based on the training sample set, and the input of the machine learning model is table data with discrete field attributes; Each training sample is converted into a vector of field values ​​for each discrete field attribute, resulting in a vector sample where each field value corresponds one-to-one with a field value vector. For each vector sample, a perturbation is applied to the field value vector of each discrete field attribute, and the perturbated vector sample is used as the first perturbation sample; Based on the first perturbation sample, generate adversarial samples. The step of converting the field values ​​of each training sample for each discrete field attribute into a field value vector to obtain a vector sample includes: For each discrete field attribute, an encoded value for the relative relationship between legal field values ​​is determined, wherein the legal field value refers to the field value that is allowed to exist under the field attribute, and the encoded value for the relative relationship between the legal field values ​​is determined according to a predetermined encoding rule; Based on the encoded values ​​of the relative relationships between the legal field values, a legal field value vector corresponding to the legal field value is determined, wherein there is a one-to-one correspondence between the legal field value and the legal field value vector; Based on the legal field values ​​and the legal field value vector, the field value vector corresponding to the field value of each training sample in the corresponding discrete field attribute is determined as the vector sample.

2. The generation method according to claim 1, characterized in that, The steps of applying a perturbation to the field value vector of each discrete field attribute for each vector sample, and using the perturbated vector sample as the first perturbation sample, include: For each vector sample, a sample perturbation is determined, wherein the sample perturbation is the gradient of the loss function of the machine learning model with respect to the vector sample; The sample perturbation is applied to the vector sample to obtain the first perturbation sample.

3. The generation method according to claim 1, characterized in that, The steps for generating adversarial examples based on the first perturbation sample include: For each first perturbation sample, at least one valid field value vector is selected from all valid field value vectors of each discrete field attribute as a valid search vector corresponding to each discrete field attribute; For each first perturbation sample, obtain all possible vector combinations of legal search vectors on all discrete field attributes, and generate a second perturbation sample corresponding to each vector combination. From all the second perturbation samples corresponding to each first perturbation sample, determine the second perturbation samples that can successfully attack the machine learning model, and generate adversarial samples based on the successfully attacked second perturbation samples. A successful attack on the machine learning model means that the output of the machine learning model corresponding to the second perturbation sample is a preset target output, and the sample distance between the second perturbation sample and the training sample corresponding to the first perturbation sample is less than a preset threshold. The sample distance is determined based on the difference between the field values ​​of the second perturbation sample and the corresponding training sample in each field attribute.

4. The generation method according to claim 3, characterized in that, The field attributes include discrete field attributes, wherein the difference between the field values ​​of the second perturbation sample and the corresponding training sample in the discrete field attributes is determined by the following method: Based on the legal field values ​​and the legal field value vector, determine the field values ​​corresponding to the field value vectors of the second perturbation sample on each discrete field attribute; Based on the difference between every two legal field values ​​among all legal field values ​​on the discrete field attribute, the minimum difference between the transformation of the field value of the second perturbation sample and the field value of the corresponding training sample is determined as the difference between the field values ​​of the second perturbation sample and the corresponding training sample on the discrete field attribute.

5. The generation method according to claim 4, characterized in that, The difference between any two valid field values ​​among all valid field values ​​in a discrete field attribute is determined by the difficulty of distinguishing between any two valid field values, wherein the difficulty of distinguishing between valid field values ​​is inversely proportional to the difference between valid field values.

6. The generation method according to claim 4, characterized in that, The field attributes also include continuous field attributes, which refer to field attributes whose valid field values ​​are consecutive variables. The difference between the field values ​​of continuous field attributes between the second perturbation sample and the corresponding training sample is determined in the following way: The absolute value of the difference between the field value of the second perturbation sample and the field value of the corresponding training sample is taken as the difference between the field values ​​of the second perturbation sample and the corresponding training sample in continuous field attributes.

7. The generation method according to any one of claims 3 to 6, characterized in that, The sample distance is determined by the following formula: in, d This represents the sample distance between the second perturbation sample and its corresponding training sample. d k For the second perturbation sample and the corresponding training sample in the th k The amount of difference between field values ​​on each field attribute s This represents the number of field attributes possessed by the second perturbation sample and its corresponding training sample. p These are preset parameters. p >0.

8. The generation method according to claim 3, characterized in that, For each first perturbation sample, the step of selecting at least one valid field value vector from all valid field value vectors of each discrete field attribute as the valid search vector corresponding to each discrete field attribute includes: Determine the vector of all valid field values ​​for each discrete field attribute; For each discrete field attribute, the vector of valid field values ​​that meet preset conditions is determined as the valid search vector for the corresponding discrete field attribute. The preset conditions include: the distance between the valid field value vector and the field value vector of the first perturbation sample is less than a preset threshold, or, a preset number of valid field value vectors that are closest to the field value vector of the first perturbation sample among all valid field value vectors.

9. The generation method according to claim 3, characterized in that, The preset target output result is an output result that is different from the original output result, which is the output result obtained by inputting the training sample corresponding to the second perturbation sample into the machine learning model.

10. The generation method according to claim 1, characterized in that, The machine learning model is trained in the following manner: Determine the vector samples that correspond one-to-one with the training samples in the training sample set to obtain the vector sample set; The machine learning model is trained using the set of vector samples.

11. An apparatus for generating adversarial examples, characterized in that, include: The acquisition unit is configured to acquire a training sample set and a machine learning model, wherein each training sample in the training sample set is table data with discrete field attributes, the discrete field attributes refer to field attributes whose field values ​​are discrete variables, and the field values ​​under the discrete field attributes are multiple discrete and non-numerical natural language texts, the machine learning model is trained based on the training sample set, and the input of the machine learning model is the table data with discrete field attributes. The transformation unit is configured to convert the field value of each training sample in each discrete field attribute into a field value vector to obtain a vector sample, wherein the field value and the field value vector correspond one-to-one. The perturbation unit is configured to apply a perturbation to the field value vector of each discrete field attribute for each vector sample, and use the perturbated vector sample as the first perturbation sample; The generation unit is configured to generate adversarial examples based on the first perturbation sample. The conversion unit is further configured as follows: For each discrete field attribute, an encoded value for the relative relationship between legal field values ​​is determined, wherein the legal field value refers to the field value that is allowed to exist under the field attribute, and the encoded value for the relative relationship between the legal field values ​​is determined according to a predetermined encoding rule; Based on the encoded values ​​of the relative relationships between the legal field values, a legal field value vector corresponding to the legal field value is determined, wherein there is a one-to-one correspondence between the legal field value and the legal field value vector; Based on the legal field values ​​and the legal field value vector, the field value vector corresponding to the field value of each training sample in the corresponding discrete field attribute is determined as the vector sample.

12. The generating apparatus according to claim 11, characterized in that, The disturbance unit is also configured to: For each vector sample, a sample perturbation is determined, wherein the sample perturbation is the gradient of the loss function of the machine learning model with respect to the vector sample; The sample perturbation is applied to the vector sample to obtain the first perturbation sample.

13. The generating apparatus according to claim 11, characterized in that, The generation unit is further configured to: For each first perturbation sample, at least one valid field value vector is selected from all valid field value vectors of each discrete field attribute as a valid search vector corresponding to each discrete field attribute; For each first perturbation sample, obtain all possible vector combinations of legal search vectors on all discrete field attributes, and generate a second perturbation sample corresponding to each vector combination. From all the second perturbation samples corresponding to each first perturbation sample, determine the second perturbation samples that can successfully attack the machine learning model, and generate adversarial samples based on the successfully attacked second perturbation samples. A successful attack on the machine learning model means that the output of the machine learning model corresponding to the second perturbation sample is a preset target output, and the sample distance between the second perturbation sample and the training sample corresponding to the first perturbation sample is less than a preset threshold. The sample distance is determined based on the difference between the field values ​​of the second perturbation sample and the corresponding training sample in each field attribute.

14. The generating apparatus according to claim 13, characterized in that, The field attributes include discrete field attributes, wherein the generation unit is configured to determine the amount of difference between the field values ​​of the second perturbation sample and the corresponding training sample in the discrete field attributes by means of: Based on the legal field values ​​and the legal field value vector, determine the field values ​​corresponding to the field value vectors of the second perturbation sample on each discrete field attribute; Based on the difference between every two legal field values ​​among all legal field values ​​on the discrete field attribute, the minimum difference between the transformation of the field value of the second perturbation sample and the field value of the corresponding training sample is determined as the difference between the field values ​​of the second perturbation sample and the corresponding training sample on the discrete field attribute.

15. The generating apparatus according to claim 14, characterized in that, The difference between any two valid field values ​​among all valid field values ​​in a discrete field attribute is determined by the difficulty of distinguishing between any two valid field values, wherein the difficulty of distinguishing between valid field values ​​is inversely proportional to the difference between valid field values.

16. The generating apparatus according to claim 14, characterized in that, The field attributes also include continuous field attributes, which refer to field attributes whose valid field values ​​are consecutive variables. The generation unit is configured to determine the amount of difference between the field values ​​of continuous field attributes between the second perturbation sample and the corresponding training sample in the following manner: The absolute value of the difference between the field value of the second perturbation sample and the field value of the corresponding training sample is taken as the difference between the field values ​​of the second perturbation sample and the corresponding training sample in continuous field attributes.

17. The generating apparatus according to any one of claims 13 to 16, characterized in that, The generation unit is configured to determine the sample distance using the following formula: in, d This represents the sample distance between the second perturbation sample and its corresponding training sample. d k For the second perturbation sample and the corresponding training sample in the th k The amount of difference between field values ​​on each field attribute s This represents the number of field attributes possessed by the second perturbation sample and its corresponding training sample. p These are preset parameters. p >0.

18. The generating apparatus according to claim 13, characterized in that, The generation unit is further configured to: Determine the vector of all valid field values ​​for each discrete field attribute; For each discrete field attribute, the vector of valid field values ​​that meet preset conditions is determined as the valid search vector for the corresponding discrete field attribute. The preset conditions include: the distance between the valid field value vector and the field value vector of the first perturbation sample is less than a preset threshold, or, a preset number of valid field value vectors that are closest to the field value vector of the first perturbation sample among all valid field value vectors.

19. The generating apparatus according to claim 13, characterized in that, The preset target output result is an output result that is different from the original output result, which is the output result obtained by inputting the training sample corresponding to the second perturbation sample into the machine learning model.

20. The generating apparatus according to claim 11, characterized in that, The adversarial example generation apparatus further includes a training unit configured to train the machine learning model in the following manner: Determine the vector samples that correspond one-to-one with the training samples in the training sample set to obtain the vector sample set; The machine learning model is trained using the set of vector samples.

21. An electronic device, characterized in that, The electronic device includes: processor; Memory used to store the processor's executable instructions. Wherein, when the processor executes the processor, it causes the processor to perform the adversarial sample generation method according to any one of claims 1 to 10.

22. A computer-readable storage medium for storing instructions, characterized in that, When the instruction is executed by at least one computing device, it causes the at least one computing device to perform the method for generating adversarial examples according to any one of claims 1 to 10.

23. A system comprising at least one computing device and at least one storage device for storing instructions, characterized in that, When the instruction is executed by the at least one computing device, it causes the at least one computing device to perform the method for generating adversarial examples according to any one of claims 1 to 10.

Images