Method for resisting attack and generating confrontation sample

Abstract

The invention relates to the field of artificial intelligence, in particular to a method and system for resisting attacks and generating adversarial samples. The method for generating the adversarial sample comprises the following steps: based on an initial sample, a label of the initial sample and a to-be-attacked model, carrying out one-round or multi-round iteration to construct the adversarial sample; wherein one round of iteration comprises the following steps: acquiring a to-be-adjusted sample of the current round; when the current round is the first round of iteration, the to-be-adjusted sample is the initial sample, and otherwise, the to-be-adjusted sample is an adversarial sample of the previous round; processing the to-be-adjusted sample by using a to-be-attacked model to obtain a first output; determining disturbance data based on the difference between the first output and the label and a disturbance coefficient; adding the disturbance data into the to-be-adjusted sample to obtain a confrontation sample of the current round; wherein the disturbance coefficient can be adjusted, so that the difference between the adversarial sample and the initial sample approaches but does not exceed a preset disturbance boundary value.

Description

[0001] Case Analysis This application is a divisional application filed in respect of the parent application with application number 202111364482.5, entitled "A method for countering attacks and generating adversarial samples". Technical Field

[0002] This specification relates to the field of artificial intelligence, and in particular to a method and system for adversarial attacks and the generation of adversarial examples. Background Technology

[0003] Adversarial attacks are a hot topic in the field of artificial intelligence. Taking machine vision as an example, an adversarial attack refers to adding small, specially designed noise to a task like classifying a photo. This noise, without affecting the human's ability to make correct judgments, can mislead the attacked model into giving answers that are different from or completely opposite to the correct ones. Adversarial attacks pose a significant threat in many scenarios requiring high reliability, such as autonomous driving systems and biometric authentication.

[0004] Therefore, a method is needed that can achieve a high success rate of attacks in order to test or optimize models and enhance their ability to resist adversarial attacks. Summary of the Invention

[0005] One embodiment of this specification provides a method for generating adversarial examples, comprising: performing one or more rounds of iteration based on an initial sample and its label, and a model to be attacked, to construct adversarial examples; wherein one round of iteration includes: obtaining a sample to be adjusted in the current round; when the current round is the first iteration, the sample to be adjusted is the initial sample, otherwise it is the adversarial sample from the previous round; processing the sample to be adjusted using the model to be attacked to obtain a first output; determining perturbation data based on the difference between the first output and the label, and a perturbation coefficient; adding the perturbation data to the sample to be adjusted to obtain the adversarial example in the current round; wherein the perturbation coefficient can be adjusted so that the difference between the adversarial example and the initial sample approaches but does not exceed a preset perturbation boundary value.

[0006] One embodiment of this specification provides a system for generating adversarial examples, comprising: an adversarial example generation module, configured to construct adversarial examples by performing one or more iterations based on an initial sample and its label, and a model to be attacked; in one iteration, the adversarial example generation module is further configured to: obtain a sample to be adjusted in the current round; when the current round is the first iteration, the sample to be adjusted is the initial sample, otherwise it is the adversarial sample from the previous round; process the sample to be adjusted using the model to be attacked to obtain a first output; determine perturbation data based on the difference between the first output and the label, and a perturbation coefficient; add the perturbation data to the sample to be adjusted to obtain the adversarial example in the current round; wherein the perturbation coefficient can be adjusted so that the difference between the adversarial example and the initial sample approaches but does not exceed a preset perturbation boundary value.

[0007] One embodiment of this specification provides an apparatus for generating adversarial examples, including a processor and a storage medium, wherein the storage medium is used to store computer instructions, and the processor is used to execute at least a portion of the computer instructions to implement the method for generating adversarial examples described above.

[0008] One embodiment of this specification provides an adversarial attack method, which includes: obtaining an alternative model of a target model; using the alternative model as the model to be attacked, obtaining an adversarial sample through the above method; and using the adversarial sample to attack, test, or optimize the target model.

[0009] One embodiment of this specification provides an anti-attack system, which includes: a substitute model acquisition module for acquiring a substitute model of a target model; an adversarial sample acquisition module for using the substitute model as a model to be attacked and obtaining adversarial samples through the above method; and an adversarial attack module for using the adversarial samples to attack, test, or optimize the target model.

[0010] One embodiment of this specification provides an anti-attack device, including a processor and a storage medium. The storage medium is used to store computer instructions, and the processor is used to execute at least a portion of the computer instructions to implement the above-described anti-attack method.

[0011] One embodiment of this specification provides a semi-supervised model training method, comprising: acquiring multiple first training samples and their labels, and multiple second training samples; the first training samples and the second training samples are text data, audio data, or image data; performing a one-stage training on an initial model using the first training samples and their labels; processing the second training samples using the model trained in the one-stage training to obtain the prediction results of the second training samples; adjusting the prediction results of the second training samples to reduce the entropy of the prediction results of the second training samples, thereby obtaining the prediction labels of the second training samples; acquiring a first training dataset based on the first training samples and their labels, and acquiring a second training dataset based on the second training samples and their prediction labels; and performing a two-stage training on the model trained in the one-stage training using the first training dataset and the second training dataset. Attached Figure Description

[0012] This specification will be further described by way of exemplary embodiments, which will be described in detail with reference to the accompanying drawings. These embodiments are not limiting; in these embodiments, the same reference numerals denote the same structures, wherein: Figure 1 These are schematic diagrams illustrating application scenarios of migration attacks based on some embodiments of this specification; Figure 2 This is an exemplary flowchart illustrating the acquisition of an alternative model according to some embodiments of this specification; Figure 3 This is an exemplary flowchart illustrating data augmentation of a second training sample according to some embodiments of this specification; Figure 4 This is an exemplary flowchart of a method for generating adversarial examples according to some embodiments of this specification; Figure 5 This is an exemplary flowchart of an anti-attack method according to some embodiments of this specification; Figure 6 This is a modular schematic diagram of a system for generating adversarial examples according to some embodiments of this specification; Figure 7 This is a modular schematic diagram of an anti-attack system according to some embodiments of this specification. Detailed Implementation

[0013] To more clearly illustrate the technical solutions of the embodiments in this specification, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are merely some examples or embodiments of this specification. For those skilled in the art, these drawings can be applied to other similar scenarios without creative effort. Unless obvious from the context or otherwise specified, the same reference numerals in the drawings represent the same structures or operations.

[0014] It should be understood that the terms “system,” “device,” “unit,” and / or “module” used herein are one way to distinguish different components, elements, parts, sections, or assemblies at different levels. However, if other terms can achieve the same purpose, they may be replaced by other expressions.

[0015] As indicated in this specification and claims, unless the context clearly indicates otherwise, the words "a," "an," "an," and / or "the" do not specifically refer to the singular and may also include the plural. Generally speaking, the terms "comprising" and "including" only indicate the inclusion of expressly identified steps and elements, which do not constitute an exclusive list, and the method or apparatus may also include other steps or elements.

[0016] Flowcharts are used in this specification to illustrate the operations performed by the system according to embodiments of this specification. It should be understood that the preceding or following operations are not necessarily performed in exact order. Instead, the steps can be processed in reverse order or simultaneously. Furthermore, other operations can be added to these processes, or one or more steps can be removed from them.

[0017] Attacks on models can be categorized into white-box attacks and black-box attacks. White-box attacks involve attacks performed with knowledge of the target model's internal information (such as model parameters and structure), while black-box attacks either lack access to this information or the attack process is independent of it. One effective type of black-box attack is a transfer attack.

[0018] Figure 1 This is a schematic diagram illustrating an application scenario of migration attacks based on some embodiments of this specification.

[0019] like Figure 1 As shown, the application scenarios of migration attacks include target model 110, alternative model 120, and adversarial sample 130.

[0020] In some embodiments, it is necessary to perform adversarial attacks on the target model 110 and judge the effectiveness of the attack based on the output of the adversarial attack in order to test or optimize the target model 110. For example, if the developer develops a defense method to resist attacks and prevent the model from being successfully attacked, it is necessary to perform adversarial attacks on the target model 110 to verify whether the defense method is effective. It should be noted that in this specification, some "adversarial attacks" are abbreviated as "attacks", which have the same meaning.

[0021] As mentioned earlier, in a black-box attack scenario, the internal information of the target model 110 cannot be obtained or is not dependent on. In order to achieve adversarial attack, it is first necessary to obtain a substitute model 120 for the target model 110 through training (e.g., training the initial model with training samples to obtain the substitute model 120), and use the trained substitute model 120 as the model to be attacked to generate an adversarial sample 130 that can attack the substitute model 120. At this time, since the internal structure of the substitute model 120 is known, a more effective adversarial sample 130 against the substitute model 120 can be obtained using white-box attack methods; finally, the target model 110 is attacked using this adversarial sample 130 (transfer).

[0022] In this context, the substitute model 120 is a machine learning model generated by the attacker that is similar to the target model 110. Understandably, the degree of similarity between the target model 110 and the substitute model 120 is related to the success rate of the attack. For example, the similarity level affects whether the adversarial example 130 generated on the substitute model 120 can effectively attack the target model 110. To obtain a better similarity, the training set of the target model 110 can be used to train the substitute model 120. However, obtaining the training set of the target model 110 is difficult in most scenarios.

[0023] In some embodiments, samples can be used to access the target model 110 to obtain the predicted labels of the samples by the target model 110. This can, to some extent, obtain a training set of an alternative model 120 that is distributed in the same way as the training set of the target model 110, thereby increasing the similarity between the two. It is easy to understand that as the number of times an attacker accesses the target model 110 increases, the attacker can gain more knowledge and thus obtain a better alternative model 120. However, in real-world scenarios, the number of times the target model 110 can be accessed is usually limited, and multiple accesses may trigger certain defense mechanisms of the target model 110.

[0024] In some embodiments, when an attacker obtains an adversarial sample 130 that is more effective at attacking alternative model 120, the attack effect is poor when the adversarial sample 130 is used to attack the target model 110 because the adversarial sample 130 has low transferability.

[0025] In view of this, this specification provides an adversarial attack method that can obtain a better alternative model 120 with fewer queries, and generate adversarial samples 130 based on it, effectively increasing the success rate of adversarial attacks. Furthermore, this specification also provides a method for generating adversarial samples 130 with high transferability.

[0026] Figure 2 This is an exemplary flowchart illustrating the acquisition of an alternative model according to some embodiments of this specification.

[0027] In some embodiments, the alternative model may be Figure 1 An alternative model 120 for the target model 110 is used as the model to be attacked to generate an adversarial sample 130, which is used to attack, test or optimize the target model 110.

[0028] In some embodiments, the alternative model can be trained via process 200. In some embodiments, process 200 can also be used to perform semi-supervised training on models other than the alternative model, especially in scenarios with insufficient samples, where the semi-supervised training method of process 200 can yield a model with better predictive performance. Process 200 may include: Step 210: Obtain multiple first training samples and multiple second training samples.

[0029] In some embodiments, the first training sample and the second training sample may be samples of the same type. In some embodiments, the first training sample and the second training sample may be text data, audio data, or image data.

[0030] In some embodiments, an arbitrary training sample set can be obtained and the training sample set can be divided into a first training sample and a second training sample according to a preset ratio.

[0031] Step 212: Process the first training sample using the target model to obtain the output corresponding to the first training sample, and use it as the label of the first training sample.

[0032] In some embodiments, the number of first training samples can be configured to be less than the number of second training samples in order to reduce the number of queries to the target model.

[0033] In some embodiments, the labels of the first training samples may be known, in which case step 212 can be omitted.

[0034] Step 220: Perform a first-stage training on the initial model using the first training samples and their labels.

[0035] The initial model can be an initialized model (such as a replacement model or other common model). In some embodiments, the initial model can be a neural network model, such as a convolutional neural network (CNN) or a recurrent neural network (RNN). In some embodiments, the initial model is trained under supervision using first training samples and their labels to adjust the model parameters; this is the first stage of training for the initial model.

[0036] Step 230: Use the model that has been trained in the first stage to process the second training sample and obtain the prediction result of the second training sample.

[0037] In some embodiments, the model after the first-stage training in step 220 already has a certain degree of predictive ability, and inputting the second training sample into the model after the first-stage training can obtain the prediction result corresponding to the second training sample.

[0038] In some embodiments, to improve the accuracy of model training, it is necessary to augment the limited training samples to obtain richer training samples. One effective method is to augment the training samples.

[0039] Figure 3 This is an exemplary flowchart illustrating data augmentation of a second training sample according to some embodiments of this specification.

[0040] refer to Figure 3 In some embodiments, data augmentation can be performed on the second training sample. Step 230 may further include, for a given second training sample: Step 231: Perform data augmentation on the second training sample to obtain multiple augmented samples.

[0041] In some embodiments, data augmentation can be performed on the training samples to increase the number of samples, thereby improving the performance of the trained model, such as enhancing noise resistance or reducing overfitting. In some embodiments, taking image data as an example of the second training sample, data augmentation may include appropriate deformation of the image data, such as rotation, translation, scaling, adding certain noise, etc.

[0042] For example, with This means that a sample in the second training sample can be obtained through data augmentation. In some embodiments, the corresponding I augmented samples, the multiple samples obtained after data augmentation of the second training sample, can be represented as: ,i takes 1~I.

[0043] Step 232: Process the second training sample and its augmented sample using the alternative model that has been trained in one stage to obtain the initial prediction results for each sample.

[0044] In some embodiments, the alternative model (or other model) can be represented as The second training sample and its augmented sample Inputting the data into the surrogate model trained in the first stage yields the initial prediction results corresponding to each input sample, which can be represented as follows: ,i takes 0~I.

[0045] Step 233: Aggregate the initial prediction results of each sample to obtain the aggregated result.

[0046] In some embodiments, the initial prediction results of each sample can be aggregated to obtain an aggregated result that can reflect all the initial prediction results to a certain extent. In some embodiments, the aggregation method may include, but is not limited to, summation, averaging, and standard deviation. For example, averaging is used as the aggregation method. In some embodiments, the aggregation result... It can be represented as: (1); Where I represents the number of augmented samples corresponding to the second training sample.

[0047] Step 234: Use the enhanced sample as the newly added second training sample, and use the aggregation result as the prediction result of the second training sample and the newly added second training sample.

[0048] In some embodiments, the augmented samples are used as newly added second training samples, thereby expanding the second training samples. This can be achieved using ( This represents the second training sample and the prediction result, where i ranges from 0 to I.

[0049] In some embodiments, the first training sample can also be data augmented, thereby expanding the first training sample. Specifically, for any first training sample and its label, the following processing can be performed: The first training sample is augmented to obtain multiple augmented samples.

[0050] The augmented sample is used as the first new training sample, and its label is used as the label of the first new training sample.

[0051] Data augmentation of the initial training samples increases the number of training samples equivalent to adversarial instances to some extent. As the model trains, the value provided by different samples varies. For example, the model may be "familiar" with some samples, consistently making accurate predictions, but the model learns less from these samples, making them less valuable. Conversely, the model may predict poorly on other samples, indicating a greater need to learn from them, making these samples more valuable. Adding adversarial instances increases the "difficulty" of the samples, which is beneficial for model "growth," or in other words, it improves the model's convergence speed and accelerates training.

[0052] Step 240: Adjust the prediction results of the second training sample to reduce the entropy of the prediction results of the second training sample, and then obtain the predicted label of the second training sample.

[0053] Since the second training sample lacks accurate label data, and the initial model is trained in step 220 using the first training sample and its label, it can be assumed that the model has a certain predictive ability at this time. In order to accelerate model training, in some embodiments, the prediction results of the second training sample can be processed so that the model trained in the first stage can make more confident predictions on the unlabeled second training sample.

[0054] In some embodiments, the predicted label of the second training sample can be obtained by reducing the entropy of the prediction result of the second training sample, thereby encouraging the model to give a high-confidence prediction. In other words, the model will get a lower confidence result for a low-confidence prediction result and a higher confidence result for a high-confidence prediction result.

[0055] In some embodiments, step 240 may include: When the prediction result is greater than a preset first threshold, it is adjusted to a value greater than the first threshold, and the value is used as its corresponding prediction label. When the prediction result is less than a preset second threshold, it is adjusted to a value less than the second threshold, and the value is used as its corresponding prediction label.

[0056] For example, assuming the first threshold is 0.7, when the predicted result is greater than the first threshold (i.e., greater than 0.7), the predicted result is adjusted to 1; the second threshold is 0.3, when the predicted result is less than the second threshold (i.e., less than 0.3), the predicted result is adjusted to 0. This method allows for larger predicted results to be larger and smaller predicted results to be smaller, thereby reducing the entropy of the predicted results of the second training sample. In some embodiments, the second threshold can be the same as the first threshold, such as both the first and second thresholds being 0.5 or 0.6, etc.

[0057] In some embodiments, the alternative model (or other model) can be a multi-class classification model, and the prediction result includes multiple elements, each element reflecting the prediction score of the corresponding second training sample in the corresponding category. Therefore, in some embodiments, for any second training sample in step 240, the following is included: The predicted scores of the second training sample in each category are processed based on the preset label sharpening coefficient to obtain the processing results corresponding to each category; Calculate the sum of the processing results for each category; Based on the processing results of each category according to the sum value, multiple elements of the predicted label of the second training sample are obtained.

[0058] In some embodiments, assuming the multi-class classification model outputs K categories and the label sharpening coefficient is T, based on the aforementioned prediction results... For example, the processing result for a certain category can be represented as follows: , where k represents the k-th output category among the K output categories of the multi-classification model, and is an integer between 1 and K. It should be noted that in some other embodiments, the label sharpening coefficient processing method is not limited to the exponential operation in the aforementioned embodiments, but can also be other processing methods, such as adding the result constructed using the label sharpening coefficient and the noise function to the predicted score, etc., which are not limited in this specification.

[0059] In some embodiments, the sum of the processing results corresponding to each category can be expressed as: .

[0060] Based on the aforementioned sum-based processing results for each category, in some embodiments, the processing method can be to divide the processing results for each category by the sum to obtain multiple elements of the predicted label for the second training sample. In some other embodiments, the sum can also be used to process the processing results for each category in other ways, which are not limited in this specification.

[0061] In some embodiments, the predicted labels of the obtained second training samples are processed. One element in can be represented as: (2).

[0062] Step 250: Obtain the first training dataset based on the first training sample and its label, and obtain the second training dataset based on the second training sample and its predicted label.

[0063] In some embodiments, a training dataset is constructed in the form of training samples and labels to further train the model. In some embodiments, the first training dataset obtained based on the first training samples and their labels is represented as follows: The second training dataset obtained based on the second training samples and the predicted labels of the second training samples obtained in step 240 is represented as follows: It should be noted that, in some embodiments, the first training dataset and the second training dataset can be obtained using data-enhanced first training samples and their labels, and data-enhanced second samples and their predicted labels, respectively.

[0064] In some embodiments, the first training dataset further includes a first fused sample and its label, and the second training dataset further includes a second fused sample and its label. By adding fused samples and their labels to the training dataset, the model can learn and recognize various fused sample data during training, further improving model performance.

[0065] In some embodiments, a first fused sample is obtained by fusing a first training sample and a sample from a mixed sample set based on preset weights, and the label of the first fused sample is obtained by fusing the label of the first training sample and the label of the sample based on the preset weights. In some embodiments, the mixed sample set includes the first training sample and its label, and a second training sample and its predicted label. In some embodiments, the mixed sample set can be obtained based on the first training sample and its label, the second training sample and its predicted label using a shuffle algorithm.

[0066] In some embodiments, the preset weight can be any value between 0 and 1. In some other embodiments, the preset weight can also be greater than 1. In some embodiments, the fusion of a first training sample and a sample from a mixed sample set, and the fusion of the label of a first training sample and the label of the sample, can be performed by summation. Specifically, the preset weight can include a first weight coefficient and a second weight coefficient. The sample fusion process can be as follows: calculate the product of the first weight coefficient and the first training sample, calculate the product of the second weight coefficient and the sample from the mixed sample set, and then add the two products. The first weight coefficient and the second weight coefficient can be correlated or uncorrelated. In some embodiments, the fusion method can also be multiplication or averaging, etc., which are not limited in this specification.

[0067] In some embodiments, the first weighting coefficient is related to the second weighting coefficient, such that their sum is 1. For example, the first training sample is denoted as x, and the label of the first training sample is denoted as p; the samples in the mixed sample set are denoted as... The labels of samples in a mixed sample set are represented as follows: The first weighting coefficient can be represented as ζ. In some embodiments, the first fused sample can be represented as: (3).

[0068] The label of the first fused sample can be represented as: (4).

[0069] From the above formulas (3) and (4), it can be understood that the first training dataset includes the first fused sample. and its labels .

[0070] In some embodiments, the first weighting coefficient or the second weighting coefficient is sampled from a Beta distribution. As an example, the coefficients can be sampled from a Beta distribution beforehand. η Then, the first weight coefficient is obtained through formula (5): (5).

[0071] In some other embodiments, the weighting coefficients may also be obtained by sampling from other distributions such as normal distribution or Gaussian distribution. In addition, in some embodiments, the weighting coefficients may also be randomly generated.

[0072] In some embodiments, the second fused sample is obtained by fusing a second training sample and a sample from a mixed sample set based on a preset weight, and the label of the second fused sample is obtained by fusing the predicted label of the second training sample and the label of the sample based on the preset weight.

[0073] In some embodiments, the method for obtaining the labels of the second fused sample and the second fused sample is similar to the method for obtaining the first fused sample and its labels, and will not be repeated here. Please refer to the relevant description above for details. It should be noted that the preset weights in the process of obtaining the first fused sample and its labels and the second fused sample and its labels can be the same or different.

[0074] Step 260: Use the first training dataset and the second training dataset to perform a second-stage training on the model that has undergone the first-stage training.

[0075] In some embodiments, two-stage training may include adjusting the parameters of the alternative model that has undergone one-stage training to reduce the objective function, thereby obtaining a trained alternative model (or other model).

[0076] In some embodiments, the objective function reflects the difference between the model's prediction and the labels of the training samples (from a first training dataset or a second training dataset). For example, the objective function can be an absolute value loss function, a squared loss function, a cross-entropy loss function, etc.

[0077] Since the first training dataset is generated based on labeled samples (such as the labels of the samples coming from the target model or manual annotation), and the second training dataset is generated based on unlabeled samples, in some embodiments, when performing second-stage training on a model that has undergone one-stage training, the first training dataset and the second training dataset can be processed separately.

[0078] In some embodiments, the objective function in the two-stage training may include a first part and a second part. The first part reflects the cross-entropy between the output obtained by the alternative model processing samples in the first training dataset and the corresponding labels. The cross-entropy is used to reflect the probability distribution between the two sets. Since the labels in the first training dataset can be considered more realistic, choosing cross-entropy loss can achieve better training results; that is, the cross-entropy loss can be used to improve training performance. The output reflects the results obtained by the alternative model after processing samples from the first training dataset. With corresponding tags In some embodiments, the probability distribution between the two is such that the first training dataset is represented as... The first part of the objective function can be expressed as: (6).

[0079] In some embodiments, the second part reflects the L2 norm of the difference between the output obtained by the alternative model processing samples in the second training dataset and the corresponding label. Since the labels in the second training dataset are obtained through prediction, using this form as the second part can make the model training process more robust. In some embodiments, the second part of the objective function can be the square of the L2 norm of the difference between the output obtained by the alternative model processing samples in the second training dataset and the corresponding label, which can be expressed as: (7).

[0080] In some embodiments, the objective function can be constructed using the first and second parts of the objective function, as follows: In some embodiments, weighting coefficients may be added to the first part and / or the second part to reflect their influence in the objective function.

[0081] In some embodiments, training process 200 may also loop through steps 230-260, iterating the model multiple times to continuously train the model and improve its performance. In some embodiments, the second training sample, after data augmentation, may be re-input into the model that underwent two-stage training in the previous iteration to update the prediction results of the second training sample, and process 200 may continue to further train the model.

[0082] It should be noted that in some embodiments, the first and second training datasets mentioned above may be unfused, while in other embodiments they may be fused. In this case, the fused samples are input into the model, and the model's output may be... The label can be a fused sample .

[0083] Through the above process 200, this method can still obtain a model with good prediction performance even when label data is insufficient. Since the label data is obtained by querying the target model, this method can significantly reduce the number of queries, making it suitable for situations where the number of times the target model can be accessed is limited, which is more common in real-world scenarios. It should be noted that in some other embodiments, the above process 200 can also be used for training other semi-supervised models.

[0084] Figure 4 This is an exemplary flowchart of a method for generating adversarial examples according to some embodiments of this specification.

[0085] In some embodiments, adversarial examples can be Figure 1 Adversarial samples 130 are used to attack, test, or optimize target model 110. In some embodiments, the step of generating the adversarial sample includes: Step 400 involves performing one or more rounds of iteration based on the initial sample and its label, as well as the model to be attacked, to construct adversarial examples. In some embodiments, step 400 may be performed by the adversarial example generation module 610.

[0086] In some embodiments, the initial sample may be the original sample, or determined based on the original sample and random noise. In some embodiments, the original sample may be a clean sample without added noise, whose label is known; for example, the original sample may be text data, audio data, or image data. In some embodiments, the original sample is represented as x, and in the first iteration, random noise may be added. The initial sample is obtained by adding it to the original sample; therefore, the initial sample can be represented as: (8).

[0087] It should be noted that, in order to avoid interference during iteration, the added random noise is typically small in some embodiments.

[0088] In some embodiments, the model to be attacked can be a white-box model whose internal information is known; for example, the model to be attacked can be a target model whose internal information is known. In some embodiments, the model to be attacked can be Figure 1 Alternative model 120 to target model 110.

[0089] In some embodiments, an iteration in step 400 may include steps 410-440.

[0090] Step 410: Obtain the sample to be adjusted in the current round; when the current round is the first iteration, the sample to be adjusted is the initial sample, otherwise it is the adversarial sample from the previous round.

[0091] In some embodiments, the samples to be adjusted in the current round can be used to obtain the adversarial samples for the current round after one iteration in step 400. More information about adversarial samples can be found in [link to relevant documentation]. Figure 1 The relevant descriptions in the text will not be repeated here.

[0092] In some embodiments, when the current round is the first iteration, the sample to be adjusted can be the initial sample. Otherwise, it is an adversarial sample from the previous round. Assuming the current round is the (i+1)th round, the sample to be adjusted is... .

[0093] Step 420: Process the sample to be adjusted using the model to be attacked to obtain the first output.

[0094] In some embodiments, the model to be attacked can be a target model or Figure 1 In some embodiments, the alternative model 120 to the target model 110 (such as the model trained via process 200) may have the sample to be adjusted input into the model to be attacked, and the model's output used as the first output. In some embodiments, the type of the first output is related to the model to be attacked, and is not limited herein. For example, assuming the model to be attacked is a multi-class model, the first output may be the category information corresponding to the sample to be adjusted. Specifically, it may be the target type in multi-class classification, or the predicted score under multiple classifications.

[0095] Step 430: Determine the perturbation data based on the difference between the first output and the label, and the perturbation coefficient.

[0096] In some embodiments, perturbation data can be generated to adjust the sample to be adjusted. In some embodiments, the model to be attacked is represented as... The first output can be represented as The perturbation coefficient is denoted as α, and the label is denoted as y. In some embodiments, the perturbation data can be represented as... ,in, It can be the cross-entropy loss function or other loss functions. This represents the gradient value of the loss function relative to the sample to be adjusted or each element therein. It is a symbolic function.

[0097] Step 440: Add the perturbation data to the sample to be adjusted to obtain the adversarial sample for the current round.

[0098] In some embodiments, the adversarial examples of the current round It can be represented as: (9).

[0099] In some embodiments, the decision to proceed to the next iteration or use the current round's adversarial sample as the final adversarial sample can be based on whether the current round's adversarial sample meets preset requirements. In some embodiments, a total number of iterations can also be set, and when the total number of iterations is reached, the iteration stops, and the final adversarial sample is obtained.

[0100] Adversarial examples are created by adding perturbation data to the original sample. To the naked eye, adversarial examples are indistinguishable from the original sample, but machine learning models will identify them as results different from or even opposite to the original label. Understandably, the larger the perturbation data, the higher the success rate of the generated adversarial example, but this also makes it visually identifiable as a "poisoned" sample. Therefore, there is a perturbation boundary when generating adversarial examples. When the difference between the adversarial example and the original sample exceeds this boundary, it is easily detected, leading to attack failure. A "perfect" adversarial example closely approximates this perturbation boundary. In some embodiments, adding perturbation data through multiple iterations can continuously adjust the sample to be adjusted towards a "perfect" adversarial sample. However, in some embodiments, due to the difference between the first output and the label, the sample to be adjusted still differs from the "perfect" adversarial sample during the iteration process. There may be a scenario where, when continuing to the next iteration, the difference between the adversarial sample and the original sample exceeds the perturbation boundary. Therefore, the iteration must be stopped. However, the sample to be adjusted may not be able to get closer to the "perfect" adversarial sample, that is, it may not be able to reach the perturbation boundary, and thus it may not be able to fully enhance its attack success rate.

[0101] To achieve a more "perfect" adversarial example, in some embodiments, the perturbation coefficient can be adjusted so that the difference between the adversarial example and the initial sample (or original sample) approaches but does not exceed a preset perturbation boundary value. In some embodiments, the preset perturbation boundary value is denoted as ϵ, and the difference between the adversarial example and the initial sample is expressed through the infinity norm. This means that by adjusting the perturbation coefficient α, such that... Approaching as close as possible .

[0102] In some embodiments, the adversarial sample for the current round that limits the perturbation boundary value can be represented as: (10); in, This represents a function or processing method that makes the difference between the adversarial sample in the current round and the initial sample (or original sample) approximate but not exceed a preset perturbation boundary value ϵ.

[0103] In some embodiments, The operation can be as follows: when the difference between the adversarial sample of the current round and the initial sample exceeds a preset perturbation boundary value, the perturbation coefficient is reduced and the adversarial sample of the current round is obtained again.

[0104] In some embodiments, the perturbation coefficient can be reduced using a preset algorithm, such as subtracting a fixed value from the perturbation coefficient. In some embodiments, the perturbation coefficient can be attenuated using a decay factor, which is negatively correlated with the iteration number i. In some embodiments, reducing the perturbation coefficient can be expressed as: (11).

[0105] For example, the attenuation factor can be The reduced disturbance coefficient can be It should be noted that in some embodiments, 0.5 in the attenuation factor can also be other constants, such as 0.78 or 0.9, etc. In addition, the attenuation factor can also be in other forms, which are not limited in this specification.

[0106] In some embodiments, the perturbation coefficient is reduced, and the adversarial sample for the current round is obtained again. That is, step 430 is re-executed based on the updated perturbation coefficient α to determine the updated perturbation data, and step 440 is executed again to obtain the adversarial sample for the current round. In some embodiments, if the difference between the adversarial sample for the current round and the initial sample still exceeds a preset perturbation boundary value, the perturbation coefficient α is updated again, and this process is repeated. It should be noted that in some embodiments, a maximum number of iterations can be set to avoid the algorithm from getting stuck in an infinite loop.

[0107] In some embodiments, step 400 may further include: randomly masking several model parameters in the model to be attacked, and using the masked model as the model to be attacked in the current round.

[0108] In some embodiments, the purpose of randomly masking certain parameters is to make these parameters not contribute to the output of the model, thereby realizing that although an attack is carried out on a specific model to be attacked through adversarial examples, the scenario of attacking multiple different models to be attacked and obtaining different outputs can be simulated.

[0109] Taking a neural network as an example, randomly masking some parameters of the alternative model can be done by performing dropout processing on the model to be attacked, such as setting some parameters of one or more convolutional layers near the output layer of the model to 0. In some embodiments, these parameters can also be set to 1.

[0110] In some embodiments, random masking can be performed in each iteration to increase the randomness of the model output during the iteration process, thereby making the adversarial sample generated after multiple iterations more transferable, that is, the adversarial sample has a higher attack success rate.

[0111] Figure 5 This is an exemplary flowchart illustrating an anti-attack method according to some embodiments of this specification.

[0112] In some embodiments of this specification, an adversarial attack method is provided, which can utilize the adversarial sample obtained in step 400 to attack, test, or optimize the target model (e.g., Figure 1 The target model 110 has a higher attack success rate, and the process 500 of this method includes: Step 510: Obtain an alternative model for the target model. In some embodiments, step 510 may be performed by the alternative model acquisition module.

[0113] In some embodiments, an alternative model to the target model can be obtained through process 200. Further description of the target model and the alternative model can be found in [reference needed]. Figures 1-4 The relevant details will not be repeated here.

[0114] Step 520: Using the alternative model as the model to be attacked, obtain adversarial examples using the method described in step 400. In some embodiments, step 520 may be performed by the adversarial example acquisition module 720.

[0115] In some embodiments, further description of adversarial examples can be found in [reference needed]. Figure 1 and Figure 4 The relevant details will not be repeated here.

[0116] Step 530: The target model is attacked using the adversarial sample, and then the target model is tested or optimized to improve its ability to resist adversarial attacks. In some embodiments, step 530 may be performed by the adversarial attack module 730.

[0117] In some embodiments, the target model can be used to process adversarial examples and its output can be tested to see if it matches the label of the original sample corresponding to the adversarial example. If it matches, it indicates that the target model has a certain ability to resist adversarial attacks; otherwise, it indicates that the target model needs further optimization to provide the ability to resist attacks. In other embodiments, the adversarial examples and their actual labels can be used as training data to further train the target model to improve its ability to resist attacks.

[0118] In some embodiments, further description of adversarial attacks can be found in [reference needed]. Figure 1 and Figure 4 The relevant details will not be repeated here.

[0119] It should be noted that the descriptions of processes 200, 500, and step 400 above are for illustrative purposes only and do not limit the scope of this specification. Those skilled in the art can make various modifications and changes to the above steps under the guidance of this specification. However, these modifications and changes remain within the scope of this specification.

[0120] Figure 6 This is a modular schematic diagram of a system for generating adversarial examples according to some embodiments of this specification.

[0121] like Figure 6 As shown, in some embodiments, the system 600 for generating adversarial examples may include an adversarial example generation module 610.

[0122] The adversarial sample generation module 610 can be used to construct adversarial samples by performing one or more rounds of iteration based on the initial sample and its label, as well as the model to be attacked.

[0123] In some embodiments, in one iteration, the adversarial sample generation module is further configured to: obtain the sample to be adjusted in the current round; when the current round is the first iteration, the sample to be adjusted is the initial sample, otherwise it is the adversarial sample from the previous round; process the sample to be adjusted using the model to be attacked to obtain a first output; determine perturbation data based on the difference between the first output and the label, and the perturbation coefficient; add the perturbation data to the sample to be adjusted to obtain the adversarial sample in the current round; wherein the perturbation coefficient can be adjusted so that the difference between the adversarial sample and the initial sample approaches but does not exceed a preset perturbation boundary value.

[0124] In some embodiments, the adversarial example generation module can be referenced to steps 400, 410-440, and... Figure 4 The relevant details will not be repeated here.

[0125] Figure 7 This is a modular schematic diagram of an anti-attack system according to some embodiments of this specification.

[0126] like Figure 7 As shown, in some embodiments, the adversarial attack system 700 may include an alternative model acquisition module 710, an adversarial sample acquisition module 720, and an adversarial attack module 730.

[0127] The alternative model acquisition module 710 can be used to acquire alternative models of the target model.

[0128] The adversarial sample acquisition module 720 can be used to obtain adversarial samples by taking the alternative model as the model to be attacked and using the aforementioned method (such as the method described in step 400). The adversarial attack module 730 can be used to attack, test, or optimize the target model using the adversarial sample.

[0129] It should be noted that the above description of systems 600 and 700 and their modules is for convenience only and should not be construed as limiting this specification to the scope of the illustrated embodiments. It is understood that those skilled in the art, after understanding the principles of this system, may arbitrarily combine the various modules or construct subsystems connected to other modules without departing from these principles. For example, in some embodiments, for instance, Figure 7 The alternative model acquisition module 710 and adversarial example acquisition module 720 disclosed herein can be different modules within the same system, or a single module can implement the functions of two or more of the aforementioned modules. For example, the modules can share a single storage module, or each module can have its own separate storage module. Such variations are all within the scope of protection of this specification.

[0130] It should be noted that different embodiments may produce different beneficial effects. In different embodiments, the beneficial effects may be any one or a combination of the above, or any other possible beneficial effects.

[0131] The basic concepts have been described above. Obviously, for those skilled in the art, the detailed disclosure above is merely illustrative and does not constitute a limitation of this specification. Although not explicitly stated herein, those skilled in the art may make various modifications, improvements, and corrections to this specification. Such modifications, improvements, and corrections are suggested in this specification and therefore remain within the spirit and scope of the exemplary embodiments described herein.

[0132] Furthermore, this specification uses specific terms to describe embodiments thereof. For example, "an embodiment," "one embodiment," and / or "some embodiments" refer to a particular feature, structure, or characteristic associated with at least one embodiment of this specification. Therefore, it should be emphasized and noted that references to "an embodiment," "one embodiment," or "an alternative embodiment" in different locations throughout this specification do not necessarily refer to the same embodiment. Moreover, certain features, structures, or characteristics in one or more embodiments of this specification can be appropriately combined.

[0133] Furthermore, unless expressly stated in the claims, the order of processing elements and sequences, the use of numbers and letters, or other names described in this specification are not intended to limit the order of the processes and methods described herein. Although various examples have been discussed in the foregoing disclosure of some embodiments of the invention that are currently considered useful, it should be understood that such details are for illustrative purposes only, and the appended claims are not limited to the disclosed embodiments; rather, the claims are intended to cover all modifications and equivalent combinations that conform to the spirit and scope of the embodiments described herein. For example, while the system components described above can be implemented using hardware devices, they can also be implemented solely using software solutions, such as installing the described system on existing servers or mobile devices.

[0134] Similarly, it should be noted that, in order to simplify the description disclosed herein and thus aid in the understanding of one or more embodiments of the invention, the foregoing description of embodiments in this specification may sometimes combine multiple features into a single embodiment, drawing, or description thereof. However, this method of disclosure does not imply that the subject matter of this specification requires more features than those mentioned in the claims. In fact, the embodiments contain fewer features than all the features of a single embodiment disclosed above.

[0135] In some embodiments, numbers describing the quantity of components and attributes are used. It should be understood that such numbers used in the description of embodiments are modified in some examples with the terms "approximately," "approximately," or "generally." Unless otherwise stated, "approximately," "approximately," or "generally" indicates that the numbers are allowed to vary by ±20%. Accordingly, in some embodiments, the numerical parameters used in the specification and claims are approximate values, which may be changed depending on the characteristics required by individual embodiments. In some embodiments, numerical parameters should take into account specified significant digits and employ a general method of digit reservation. Although the numerical ranges and parameters used to confirm their breadth of range in some embodiments of this specification are approximate values, in specific embodiments, such values ​​are set as precisely as feasible.

[0136] For each patent, patent application, patent application publication, and other material such as articles, books, specifications, publications, and documents referenced in this specification, the entire contents of which are incorporated herein by reference. This excludes historical application documents that are inconsistent with or conflict with the content of this specification, as well as documents that limit the broadest scope of the claims in this specification (currently or subsequently appended to this specification). It should be noted that in the event of any inconsistency or conflict between the descriptions, definitions, and / or terminology used in the supplementary materials to this specification and the content of this specification, the descriptions, definitions, and / or terminology used in this specification shall prevail.

[0137] Finally, it should be understood that the embodiments described in this specification are merely illustrative of the principles of the embodiments described herein. Other variations may also fall within the scope of this specification. Therefore, alternative configurations of the embodiments described herein are intended to be illustrative rather than limiting, and should be considered consistent with the teachings of this specification. Accordingly, the embodiments described herein are not limited to those explicitly introduced and described herein.

Claims

1. A semi-supervised model training method, comprising: Obtain multiple first training samples and their labels, and multiple second training samples; The first training sample and the second training sample are text data, audio data, or image data; The initial model is trained in one stage using the first training samples and their labels. The model, which has undergone one stage of training, is used to process the second training sample to obtain the prediction result of the second training sample. Adjust the prediction results of the second training sample to reduce the entropy of the prediction results of the second training sample, and then obtain the predicted label of the second training sample. A first training dataset is obtained based on the first training sample and its label, and a second training dataset is obtained based on the second training sample and its predicted label. The model that has undergone one stage of training is trained in two stages using the first training dataset and the second training dataset.

2. The method of claim 1, wherein processing the second training sample using a substitute model trained in one stage to obtain the prediction result of the second training sample includes, for a given second training sample: The second training sample is augmented to obtain multiple augmented samples; The second training sample and its augmented sample are processed using the alternative model that has been trained in one stage to obtain the initial prediction results for each sample. The initial prediction results of each sample are aggregated to obtain the aggregated result; The enhanced sample is used as a new second training sample, and the aggregation result is used as the prediction result of the second training sample and the newly added second training sample.

3. The method as described in claim 1 or 2, wherein adjusting the prediction result of the second training sample to reduce the entropy of the prediction result of the second training sample, thereby obtaining the predicted label of the second training sample, comprises: When the prediction result is greater than a preset first threshold, it is adjusted to a value greater than the first threshold, and the value is used as its corresponding prediction label. When the prediction result is less than a preset second threshold, it is adjusted to a value less than the second threshold, and this value is used as the corresponding prediction label. Wherein, the first threshold is greater than or equal to the second threshold.

4. The method as described in claim 11 or 12, wherein the target model or the alternative model is a multi-class classification model, and the prediction result includes multiple elements, each element reflecting the prediction score of the corresponding second training sample in the corresponding category; The adjustment of the prediction results of the second training sample to reduce the entropy of the prediction results of the second training sample, thereby obtaining the predicted label of the second training sample, includes, for a given second training sample: The predicted scores of the second training sample in each category are processed based on the preset label sharpening coefficient to obtain the processing results corresponding to each category; Calculate the sum of the processing results for each category; Based on the processing results of each category according to the sum value, multiple elements of the predicted label of the second training sample are obtained.

5. The method as described in claim 1, wherein the first training dataset includes a first fused sample and its label, and the second training dataset includes a second fused sample and its label; in, The first fused sample is obtained by fusing a first training sample and a sample from a mixed sample set based on a preset weight. The label of the first fused sample is obtained by fusing the label of the first training sample and the label of the sample based on the preset weight. The second fused sample is obtained by fusing a second training sample and a sample from a mixed sample set based on preset weights. The label of the second fused sample is obtained by fusing the predicted label of the second training sample and the label of the sample based on the preset weights. The mixed sample set includes the first training sample and its label, and the second training sample and its predicted label.

6. The method as described in claim 5, wherein the fusion is a weighted summation, the preset weights include a first weight coefficient and a second weight coefficient, the sum of the first weight coefficient and the second weight coefficient is 1, and the first weight coefficient or the second weight coefficient is obtained by sampling from the Beta distribution.

7. The method of claim 1, further comprising, for a first training sample and its label: Data augmentation is performed on the first training sample to obtain multiple augmented samples; The augmented sample is used as the first new training sample, and its label is used as the label of the first new training sample.

8. The method of claim 1, wherein the step of performing a second-stage training on the alternative model trained in one stage using the first training dataset and the second training dataset to obtain an alternative model of the target model comprises: The parameters of the alternative model, which has been trained for one stage, are adjusted to reduce the objective function, thereby obtaining an alternative model to the objective model. The objective function includes a first part and a second part. The first part reflects the cross-entropy between the output obtained by the alternative model processing samples in the first training dataset and the corresponding label, and the second part reflects the L2 norm of the difference between the output obtained by the alternative model processing samples in the second training dataset and the corresponding label.

9. A semi-supervised model training apparatus, comprising a storage medium and a processor, the storage medium storing computer instructions, the processor being configured to execute at least a portion of the computer instructions to implement the method as described in any one of claims 1 to 8.

10. A storage medium storing computer instructions that, when at least a portion of the computer instructions are executed by a processor, enable the implementation of the method as claimed in any one of claims 1 to 8.

Images