Method and system for defending against sample attack, and storage medium

Abstract

The embodiment of the invention provides a method and system for defending against a sample attack and a storage medium, and is used for enhancing the defending effect of the sample attack, and the method comprises the steps: obtaining a combined uncertainty expression; the combined uncertainty expression is an expression for combining cognitive uncertainty and accidental uncertainty of the neural network model; performing model training based on the combined uncertainty expression to obtain an adversarial sample detection model; when a to-be-searched text is received, inputting the to-be-searched text into the confrontation sample detection model, and determining the type of the to-be-searched text; the to-be-searched text type comprises a normal text and an abnormal text; and if the type of the text to be searched is the abnormal text, refusing the search request.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a method, system and storage medium for defending against adversarial sample attacks. Background Art

[0002] Adversarial attack is an emerging attack method against deep learning models. It tricks the model into outputting wrong results by making small but purposeful modifications to the input data. Adversarial attacks on natural language processing (NLP) pre-trained models may mislead search engines to output wrong information, thus affecting the reliability of the output results.

[0003] In order to achieve defense against adversarial sample attacks, one possible implementation method is to calculate the local intrinsic dimensions of adversarial samples and normal samples in high-dimensional space to determine whether an adversarial sample attack has occurred. However, this implementation method has a poor effect on adversarial sample attack detection, resulting in poor defense against adversarial sample attacks. Therefore, how to enhance the defense effect against adversarial sample attacks has become a technical problem that needs to be solved at present. Summary of the invention

[0004] In view of the above problems, the purpose of the present invention is to provide a method, system and storage medium for defending against adversarial sample attacks to enhance the defense effect against adversarial sample attacks. The specific scheme is as follows:

[0005] In a first aspect, an embodiment of the present application provides a method for defending against adversarial sample attacks, the method comprising:

[0006] Obtaining a combined uncertainty expression; the combined uncertainty expression is an expression combining epistemic uncertainty and accidental uncertainty of the neural network model;

[0007] Performing model training based on the combined uncertainty expression to obtain an adversarial sample detection model;

[0008] When receiving the text to be searched, the text to be searched is input into the adversarial sample detection model to determine the type of the text to be searched; the type of the text to be searched includes: normal text and abnormal text;

[0009] If the type of the text to be searched is the abnormal text, the search request is rejected.

[0010] Optionally, obtaining a combined uncertainty expression includes:

[0011] Acquire a query corpus set; the query corpus set includes a plurality of query corpora;

[0012] Based on the query corpus set, generating a first adversarial sample generation model;

[0013] Based on the first adversarial sample generation model, the combined uncertainty expression is determined.

[0014] Optionally, generating a first adversarial sample generation model based on the query corpus set includes:

[0015] Conduct adversarial sample attacks on the pre-trained model to generate the first adversarial perturbation;

[0016] Adding random noise to the adversarial disturbance to obtain a second adversarial disturbance;

[0017] Inputting the second adversarial perturbation into the second adversarial sample generation model to generate the first adversarial sample;

[0018] Inputting the first adversarial sample into an adversarial sample judgment model to obtain a judgment result;

[0019] Determine a first adversarial sample set based on the judgment result;

[0020] The second adversarial sample generation model is trained based on the first adversarial sample set to obtain the first adversarial sample generation model.

[0021] Optionally, determining the combined uncertainty expression based on the first adversarial sample generation model includes:

[0022] Based on the first adversarial sample generation model, generate a second adversarial sample set;

[0023] Based on the first adversarial sample generation model and the second adversarial sample set, determining epistemic uncertainty and accidental uncertainty; the epistemic uncertainty is the relationship between the parameters of the first adversarial sample generation model and the adversarial sample; the accidental uncertainty is the degree of uncertainty of the adversarial sample output by the first adversarial sample generation model;

[0024] The epistemic uncertainty and the aleatory uncertainty are linearly combined to construct the combined uncertainty expression.

[0025] Optionally, performing model training based on the combined uncertainty expression to obtain an adversarial sample detection model includes:

[0026] Creating a first neural network model set; the adversarial sample detection model set includes a plurality of neural network models with different structures;

[0027] Based on the combined uncertainty expression, training the neural network models in the neural network model set to obtain a second neural network model set;

[0028] Inputting the test adversarial sample into each neural network model in the second neural network model set to determine the classification accuracy of each neural network model;

[0029] Based on the classification accuracy of each neural network model, the adversarial sample detection model is determined.

[0030] Optionally, the method further includes:

[0031] If the type of the text to be searched is a normal text type, a search result is fed back based on the text to be searched.

[0032] In a second aspect, an embodiment of the present application provides a system for defending against adversarial sample attacks, the system comprising:

[0033] An acquisition module, used to acquire a combined uncertainty expression; the combined uncertainty expression is an expression combining epistemic uncertainty and accidental uncertainty of a neural network model;

[0034] A training module, used for performing model training based on the combined uncertainty expression to obtain an adversarial sample detection model;

[0035] A determination module, configured to input the text to be searched into the adversarial sample detection model upon receiving the text to be searched, and determine the type of the text to be searched; the type of the text to be searched includes: normal text and abnormal text;

[0036] The search module is used to reject the search request if the type of the text to be searched is the abnormal text.

[0037] Optionally, the acquisition module is specifically used for:

[0038] Acquire a query corpus set; the query corpus set includes a plurality of query corpora;

[0039] Based on the query corpus set, generating a first adversarial sample generation model;

[0040] Based on the first adversarial sample generation model, the combined uncertainty expression is determined.

[0041] Optionally, the acquisition module is specifically used for:

[0042] Conduct adversarial sample attacks on the pre-trained model to generate the first adversarial perturbation;

[0043] Adding random noise to the adversarial disturbance to obtain a second adversarial disturbance;

[0044] Inputting the second adversarial perturbation into the second adversarial sample generation model to generate the first adversarial sample;

[0045] Inputting the first adversarial sample into an adversarial sample judgment model to obtain a judgment result;

[0046] Determine a first adversarial sample set based on the judgment result;

[0047] The second adversarial sample generation model is trained based on the first adversarial sample set to obtain the first adversarial sample generation model.

[0048] In a third aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements any of the above-mentioned methods for defending against adversarial sample attacks.

[0049] Compared with the prior art, this application has the following beneficial effects:

[0050] By obtaining a combined uncertainty expression and performing model training based on the combined uncertainty expression, an adversarial sample detection model is obtained. When receiving the text to be searched, the text to be searched is input into the adversarial sample detection model to determine the type of the text to be searched. If the type of the text to be searched is abnormal text, the search request is rejected. By rejecting the search for the text to be searched input by the adversarial sample attack, defense against the adversarial sample attack is achieved. In addition, since the combined uncertainty expression is an expression that combines the cognitive uncertainty and accidental uncertainty of the neural network model, the features of the adversarial sample are mined by combining the cognitive uncertainty and accidental uncertainty, so that the determined type of the text to be searched is more accurate, so as to accurately identify whether the text to be searched is the text to be searched input by the adversarial sample attack, and enhance the defense effect against the adversarial sample attack. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present application. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0052] Figure 1 A flowchart of a method for defending against adversarial sample attacks provided in an embodiment of the present application;

[0053] Figure 2 A functional schematic diagram of a device for defending against adversarial sample attacks provided in an embodiment of the present application;

[0054] Figure 3 A schematic diagram of a flow chart of preprocessing of corpus in a corpus by a corpus processing unit provided in an embodiment of the present application;

[0055] Figure 4 A schematic diagram of a process for constructing a first adversarial sample generation model based on a query corpus set by a generative adversarial network unit provided in an embodiment of the present application;

[0056] Figure 5 A schematic diagram of a process for extracting uncertainty features of adversarial samples by an uncertainty extraction unit provided in an embodiment of the present application;

[0057] Figure 6 A schematic diagram of a process for filtering text to be searched that has an anti-attack intention by an adversarial sample detection unit provided in an embodiment of the present application;

[0058] Figure 7 A schematic diagram of the structure of a system for defending against adversarial sample attacks provided in an embodiment of the present application. DETAILED DESCRIPTION

[0059] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the present application belong to the scope of protection of this application.

[0060] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0061] Since the adversarial sample defense based on local intrinsic dimension focuses on the high-dimensional space characteristics when the adversarial sample is generated, it is difficult to distinguish more adversarial samples with a single indicator, resulting in poor detection effect on adversarial sample attacks and failure to effectively defend against adversarial sample attacks. Therefore, the present application provides a method, system and storage medium for defending against adversarial sample attacks to solve the above technical problems, as follows.

[0062] like Figure 1 As shown, the present application provides a method for defending against adversarial sample attacks, the method comprising:

[0063] S101: Obtain a combined uncertainty expression; the combined uncertainty expression is an expression that combines the epistemic uncertainty and the accidental uncertainty of the neural network model.

[0064] The method in this application is applied to a device for defending against adversarial sample attacks, such as Figure 2 The device for defending against adversarial sample attacks shown includes: a corpus processing unit, a generative adversarial network unit, an uncertainty extraction unit, an adversarial sample detection model generation unit, and an adversarial sample detection unit.

[0065] In an optional embodiment, obtaining a combined uncertainty expression includes:

[0066] Obtaining a query corpus set; the query corpus combination includes multiple query corpora;

[0067] Based on the query corpus set, generate a first adversarial sample generation model;

[0068] Based on the first adversarial sample generation model, a combined uncertainty expression is determined.

[0069] Specifically, the corpus processing unit is used to preprocess the corpus in the corpus to generate multiple query corpora to obtain a query corpus set. The corpus contained in the corpus is normal text, and the query corpus has a data format that can be processed by the generative adversarial network unit, thereby realizing the data format conversion of the corpus in the corpus.

[0070] Furthermore, the corpus processing unit in the present application may include a corpus reader, a Chinese word segmenter, a special symbol processor, and a sequence filler.

[0071] The corpus reader is used to read the corpus, the word segmenter is used to segment the Chinese corpus, and perform part-of-speech tagging on the entire corpus. The special symbol processor is used to filter the special symbols of the annotated corpus and perform noise reduction on the data. The sequence filler is used to perform sequence filling on the final corpus to obtain the query corpus.

[0072] For example, Figure 3 As shown, the corpus reader reads the corpus in the corpus, the word segmenter performs word segmentation on the Chinese corpus, and performs part-of-speech tagging on the word-segmented corpus and non-Chinese corpus to obtain a corpus tag sequence. The special symbol processor loads a special symbol set including a variety of special symbols, deletes the corpus containing special symbols in the corpus tag sequence, and performs noise reduction on the data. The sequence filler fills the corpus tag sequence output by the special symbol processor, and pads the length of each corpus in the corpus tag sequence to the same length. The padding method can be selected based on actual needs, such as padding with zeros before the corpus, or padding with zeros after the corpus, etc.

[0073] In an optional embodiment, generating a first adversarial sample generation model based on a query corpus set includes:

[0074] Conduct adversarial sample attacks on the pre-trained model to generate the first adversarial perturbation;

[0075] Add random noise to the adversarial perturbation to obtain a second adversarial perturbation;

[0076] Inputting the second adversarial perturbation into the second adversarial sample generation model to generate the first adversarial sample;

[0077] Inputting the first adversarial sample into the adversarial sample judgment model to obtain a judgment result;

[0078] Determine a first adversarial sample set based on the judgment result;

[0079] The second adversarial sample generation model is trained based on the first adversarial sample set to obtain the first adversarial sample generation model.

[0080] Specifically, the generative adversarial network unit is used to receive the query corpus set output by the corpus processing unit, and construct a first adversarial sample generation model based on the query corpus set. The generative adversarial network unit includes a noise generator, an adversarial sample generator, and an adversarial sample discriminator.

[0081] The adversarial sample generator is used to generate adversarial samples that are more similar to normal text, and the adversarial sample discriminator is used to discover as many adversarial samples as possible. The generator and the discriminator compete with each other, and ultimately the quality of the final generated adversarial samples can be improved.

[0082] like Figure 4 As shown, the noise generator loads the adversarial sample attack method to attack the pre-trained model of the search engine and generates a first adversarial perturbation, such as JSMA, GenAttack and other adversarial sample attack methods. Random noise is added to the first adversarial perturbation to simulate the input error of the input scene to obtain a second adversarial perturbation. The adversarial sample generator includes a second adversarial sample generation model, and the adversarial sample generator generates a first adversarial sample based on the second perturbation. The adversarial sample discriminator includes an adversarial sample judgment model, and the adversarial sample discriminator detects the first adversarial sample. If the detection result is a non-adversarial sample, the adjustment parameter is fed back to the adversarial sample generator, so that the adversarial sample generator is optimized based on the adjustment parameter to achieve secondary training of the second adversarial sample generation model. If the detection result is an adversarial sample, the first adversarial sample is added to the first adversarial sample set. By using adversarial perturbations and initial noise to construct adversarial samples, the instability of random noise in the process of converging to successfully generating adversarial samples can be reduced, the generation speed of adversarial samples generated by the adversarial sample generator can be significantly improved, the network convergence can be accelerated, and the computational cost can be reduced.

[0083] The second adversarial sample generation model is trained based on the first adversarial sample set, and when the loss function change is less than the minimum length, or the number of training iterations is greater than or equal to the maximum number, the first adversarial sample generation model is output. Thus, adversarial perturbations are generated by using random noise and existing open source adversarial attack methods, so that the generated adversarial samples are closer to real samples, so that when the adversarial sample detection model is subsequently trained based on the adversarial samples generated by the first adversarial sample generation model, the generalization ability of the adversarial sample detection model against different adversarial attack defenses can be enhanced.

[0084] In an optional embodiment, determining a combined uncertainty expression based on the first adversarial sample generation model includes:

[0085] Based on the first adversarial sample generation model, generate a second adversarial sample set;

[0086] Based on the first adversarial sample generation model and the second adversarial sample set, determining epistemic uncertainty and accidental uncertainty; epistemic uncertainty is the relationship between the parameters of the first adversarial sample generation model and the adversarial sample; accidental uncertainty is the degree of uncertainty of the adversarial sample output by the first adversarial sample generation model;

[0087] The epistemic uncertainty and aleatory uncertainty are linearly combined to construct a combined uncertainty expression.

[0088] Specifically, the uncertainty extraction unit receives the first adversarial sample generation model and extracts the uncertainty feature of the adversarial sample. Uncertainty is the degree of uncertainty of the prediction of the input data or the output result of the neural network model.

[0089] The uncertainty extraction unit includes an adversarial sample generator, an aleatoric uncertainty (AU) extractor, an epistemic uncertainty (EU) extractor, a combined uncertainty constructor and a parameter adjuster. The adversarial sample generator is used to load the first adversarial sample generation model to generate a second adversarial sample set. The AU extractor is used to extract AU and evaluate the relationship between model parameters and adversarial sample attacks. The EU extractor is used to extract EU and evaluate the degree of uncertainty of the model for the adversarial sample output results. The combined uncertainty constructor is used to construct combined uncertainty using linear combinations, and the parameter adjuster tunes the parameters in the combined uncertainty through a genetic algorithm to obtain a combined uncertainty expression.

[0090] For example, Figure 5As shown, the adversarial sample generator loads the first adversarial sample generation model and generates a second adversarial sample set based on the first adversarial sample generation model. A Bayesian neural network is constructed, and the Bayesian neural network is continuously connected using the second adversarial sample set so that the Bayesian neural network can extract uncertainty features. The AU extractor captures the posterior variance of each adversarial sample in the second adversarial sample set input to the Bayesian neural network, and uses the captured posterior variance as the AU of the adversarial sample. The EU extractor uses the Monte Carlo method to sample the parameters of the Bayesian neural network multiple times, observes the changes in the probability of the Bayesian neural network output, and uses the entropy of the probability vector output by the Bayesian neural network as the EU of the input adversarial sample.

[0091] The combined uncertainty constructor linearly combines the AU and EU of each adversarial sample to obtain the combined uncertainty. Combined uncertainty = AU + kEU, where k is the linear parameter value. The parameter adjuster uses a genetic algorithm to search for the optimal linear parameter value in the combined uncertainty, and constructs a combined uncertainty expression based on the optimal linear parameter values ​​AU and EU. In this way, training for identifying adversarial samples is performed from the perspectives of model parameter error and data error, and the genetic algorithm is used to query the optimal linear parameter value, which can maximize the defense effect of combined uncertainty, so that the adversarial sample detection model trained based on the combined uncertainty expression can detect more high-quality adversarial samples.

[0092] It should be noted that the optimal linear parameter value is the linear parameter value with the best fitting effect.

[0093] S102: Perform model training based on the combined uncertainty expression to obtain an adversarial sample detection model.

[0094] The adversarial sample detection model generation unit receives the combined uncertainty expression output by the uncertainty extraction unit, constructs a combined uncertainty matrix based on the combined uncertainty expression, and performs model training through the combined uncertainty matrix so that the trained model can identify adversarial samples.

[0095] In an optional embodiment, model training is performed based on the combined uncertainty expression to obtain an adversarial sample detection model, including:

[0096] Creating a first neural network model set; the adversarial sample detection model set includes a plurality of neural network models with different structures;

[0097] Based on the combined uncertainty expression, the neural network models in the neural network model set are trained to obtain a second neural network model set;

[0098] Inputting the test adversarial sample into each neural network model in the second neural network model set to determine the classification accuracy of each neural network model;

[0099] Based on the classification accuracy of each neural network model, the adversarial sample detection model is determined.

[0100] The adversarial sample detection model generation unit includes a model builder, a model trainer and a model classifier. The model builder is used to construct a plurality of neural network detectors with different structures to obtain a first neural network model set. The model trainer is used to train each neural network model in the first neural network model set using the combined uncertainty matrix as a training set, and add the trained neural network model to the second neural network model set. The model classifier is used to select the neural network model with the highest classification accuracy in the second neural network model set as the adversarial sample detection model.

[0101] S103: When receiving the text to be searched, input the text to be searched into the adversarial sample detection model to determine the type of the text to be searched; the type of the text to be searched includes: normal text and abnormal text.

[0102] The adversarial sample detection unit receives the adversarial sample detection model, detects the search text input by the user, and filters the search text with adversarial attack intentions, so as to protect the underlying search engine.

[0103] Specifically, after obtaining the adversarial sample detection model, if a text to be searched is received from a user, the adversarial sample detection model is used to perform type detection on the text to be searched to determine whether the text to be searched is the text input during the adversarial sample attack. When the type of the text to be searched is detected to be normal text, the text to be searched is irrelevant to the adversarial sample attack. When the type of the text to be searched is detected to be abnormal text, the text to be searched is the text input during the adversarial sample attack.

[0104] Furthermore, the adversarial sample detection unit includes a Chinese word segmenter, a sequence filler, and an adversarial sample detector. Figure 6 As shown, the Chinese word segmenter is used to segment and tag the search text, and the sequence filler is used to fill each word segment so that each filled word segment has a uniform length, which is convenient for the adversarial sample detection model to detect the search text.

[0105] S104: If the type of the text to be searched is not a normal text, the search request is rejected.

[0106] Based on the type of the text to be searched, it is selected whether to use a search engine to search the text to be searched. If the type of the text to be searched is abnormal text, the text to be searched is not searched, and a prompt to reject the search request is returned to the user. If the type of the text to be searched is normal text, the search result is fed back based on the text to be searched.

[0107] Specifically, the adversarial sample detector is used to read the adversarial sample detection model, and detect the word segmentation of the search text through the adversarial sample detection model. When the detection result shows that the type of the search text is abnormal text, the search text is filtered and the search for the search text through the search engine is rejected. When the detection result shows that the type of the search text is normal text, the adversarial sample detection model outputs the search text to the search engine, so that the search engine generates search results based on the search text, and feeds back the search results output by the search engine to the user.

[0108] By obtaining a combined uncertainty expression and performing model training based on the combined uncertainty expression, an adversarial sample detection model is obtained. When receiving the text to be searched, the text to be searched is input into the adversarial sample detection model to determine the type of the text to be searched. If the type of the text to be searched is abnormal text, the search request is rejected. By rejecting the search for the text to be searched input by the adversarial sample attack, defense against the adversarial sample attack is achieved. In addition, since the combined uncertainty expression is an expression that combines the cognitive uncertainty and accidental uncertainty of the neural network model, the features of the adversarial sample are mined by combining the cognitive uncertainty and accidental uncertainty, so that the determined type of the text to be searched is more accurate, so as to accurately identify whether the text to be searched is the text to be searched input by the adversarial sample attack, and enhance the defense effect against the adversarial sample attack.

[0109] like Figure 7 As shown, the present application provides a system for defending against adversarial sample attacks, the system comprising:

[0110] An acquisition module 701 is used to acquire a combined uncertainty expression; the combined uncertainty expression is an expression combining epistemic uncertainty and accidental uncertainty of a neural network model;

[0111] A training module 702 is used to perform model training based on the combined uncertainty expression to obtain an adversarial sample detection model;

[0112] The determination module 703 is used to input the text to be searched into the adversarial sample detection model to determine the type of the text to be searched when receiving the text to be searched; the type of the text to be searched includes: normal text and abnormal text;

[0113] The search module 704 is used to reject the search request if the type of the text to be searched is not a normal text.

[0114] In an optional embodiment, the acquisition module 701 is specifically used for:

[0115] Obtaining a query corpus set; the query corpus combination includes multiple query corpora;

[0116] Based on the query corpus set, generate a first adversarial sample generation model;

[0117] Based on the first adversarial sample generation model, a combined uncertainty expression is determined.

[0118] In an optional embodiment, the acquisition module 701 is specifically used for:

[0119] Conduct adversarial sample attacks on the pre-trained model to generate the first adversarial perturbation;

[0120] Add random noise to the adversarial perturbation to obtain a second adversarial perturbation;

[0121] Inputting the second adversarial perturbation into the second adversarial sample generation model to generate the first adversarial sample;

[0122] Inputting the first adversarial sample into the adversarial sample judgment model to obtain a judgment result;

[0123] Determine a first adversarial sample set based on the judgment result;

[0124] The second adversarial sample generation model is trained based on the first adversarial sample set to obtain the first adversarial sample generation model.

[0125] In an optional embodiment, the acquisition module 701 is specifically used for:

[0126] Based on the first adversarial sample generation model, generate a second adversarial sample set;

[0127] Based on the first adversarial sample generation model and the second adversarial sample set, determining epistemic uncertainty and accidental uncertainty; epistemic uncertainty is the relationship between the parameters of the first adversarial sample generation model and the adversarial sample; accidental uncertainty is the degree of uncertainty of the adversarial sample output by the first adversarial sample generation model;

[0128] The epistemic uncertainty and aleatory uncertainty are linearly combined to construct a combined uncertainty expression.

[0129] In an optional embodiment, the training module 702 is specifically used for:

[0130] Creating a first neural network model set; the adversarial sample detection model set includes a plurality of neural network models with different structures;

[0131] Based on the combined uncertainty expression, the neural network models in the neural network model set are trained to obtain a second neural network model set;

[0132] Inputting the test adversarial sample into each neural network model in the second neural network model set to determine the classification accuracy of each neural network model;

[0133] Based on the classification accuracy of each neural network model, the adversarial sample detection model is determined.

[0134] In an optional embodiment, the determining module 703 is further configured to:

[0135] If the type of the text to be searched is a normal text type, the search result is fed back based on the text to be searched.

[0136] By obtaining a combined uncertainty expression and performing model training based on the combined uncertainty expression, an adversarial sample detection model is obtained. When receiving the text to be searched, the text to be searched is input into the adversarial sample detection model to determine the type of the text to be searched. If the type of the text to be searched is abnormal text, the search request is rejected. By rejecting the search for the text to be searched input by the adversarial sample attack, defense against the adversarial sample attack is achieved. In addition, since the combined uncertainty expression is an expression that combines the cognitive uncertainty and accidental uncertainty of the neural network model, the features of the adversarial sample are mined by combining the cognitive uncertainty and accidental uncertainty, so that the determined type of the text to be searched is more accurate, so as to accurately identify whether the text to be searched is the text to be searched input by the adversarial sample attack, and enhance the defense effect against the adversarial sample attack.

[0137] An embodiment of the present application provides a computer storage medium for storing a computer program. When the computer program is executed, it is used to implement any of the above-mentioned methods for defending against adversarial sample attacks.

[0138] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0139] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0140] The technical content provided by the present invention is introduced in detail above. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for general technical personnel in the field, according to the idea of ​​the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A method for defending against adversarial sample attacks, characterized in that: The method comprises: Obtaining a combined uncertainty expression; the combined uncertainty expression is an expression combining epistemic uncertainty and accidental uncertainty of the neural network model; Performing model training based on the combined uncertainty expression to obtain an adversarial sample detection model; When receiving the text to be searched, the text to be searched is input into the adversarial sample detection model to determine the type of the text to be searched; the type of the text to be searched includes: normal text and abnormal text; If the type of the text to be searched is the abnormal text, the search request is rejected.

2. The method according to claim 1, characterized in that The obtaining of the combined uncertainty expression comprises: Acquire a query corpus set; the query corpus set includes a plurality of query corpora; Based on the query corpus set, generating a first adversarial sample generation model; Based on the first adversarial sample generation model, the combined uncertainty expression is determined.

3. The method according to claim 2, characterized in that The step of generating a first adversarial sample generation model based on the query corpus set includes: Conduct adversarial sample attacks on the pre-trained model to generate the first adversarial perturbation; Adding random noise to the adversarial disturbance to obtain a second adversarial disturbance; Inputting the second adversarial perturbation into the second adversarial sample generation model to generate the first adversarial sample; Inputting the first adversarial sample into an adversarial sample judgment model to obtain a judgment result; Determine a first adversarial sample set based on the judgment result; The second adversarial sample generation model is trained based on the first adversarial sample set to obtain the first adversarial sample generation model.

4. The method according to claim 2, characterized in that: The determining the combined uncertainty expression based on the first adversarial sample generation model includes: Based on the first adversarial sample generation model, generate a second adversarial sample set; Based on the first adversarial sample generation model and the second adversarial sample set, determining epistemic uncertainty and accidental uncertainty; the epistemic uncertainty is the relationship between the parameters of the first adversarial sample generation model and the adversarial sample; the accidental uncertainty is the degree of uncertainty of the adversarial sample output by the first adversarial sample generation model; The epistemic uncertainty and the aleatory uncertainty are linearly combined to construct the combined uncertainty expression.

5. The method according to claim 1, characterized in that: The performing model training based on the combined uncertainty expression to obtain an adversarial sample detection model includes: Creating a first neural network model set; the adversarial sample detection model set includes a plurality of neural network models with different structures; Based on the combined uncertainty expression, training the neural network models in the neural network model set to obtain a second neural network model set; Inputting the test adversarial sample into each neural network model in the second neural network model set to determine the classification accuracy of each neural network model; Based on the classification accuracy of each neural network model, the adversarial sample detection model is determined.

6. The method according to claim 1, characterized in that The method further comprises: If the type of the text to be searched is a normal text type, a search result is fed back based on the text to be searched.

7. A system for defending against adversarial sample attacks, characterized in that: The system comprises: An acquisition module, used to acquire a combined uncertainty expression; the combined uncertainty expression is an expression combining epistemic uncertainty and accidental uncertainty of a neural network model; A training module, used for performing model training based on the combined uncertainty expression to obtain an adversarial sample detection model; A determination module, configured to input the text to be searched into the adversarial sample detection model upon receiving the text to be searched, and determine the type of the text to be searched; the type of the text to be searched includes: normal text and abnormal text; The search module is used to reject the search request if the type of the text to be searched is the abnormal text.

8. The system according to claim 7, characterized in that The acquisition module is specifically used for: Acquire a query corpus set; the query corpus set includes a plurality of query corpora; Based on the query corpus set, generating a first adversarial sample generation model; Based on the first adversarial sample generation model, the combined uncertainty expression is determined.

9. The system according to claim 8, characterized in that The acquisition module is specifically used for: Conduct adversarial sample attacks on the pre-trained model to generate the first adversarial perturbation; Adding random noise to the adversarial disturbance to obtain a second adversarial disturbance; Inputting the second adversarial perturbation into the second adversarial sample generation model to generate the first adversarial sample; Inputting the first adversarial sample into an adversarial sample judgment model to obtain a judgment result; Determine a first adversarial sample set based on the judgment result; The second adversarial sample generation model is trained based on the first adversarial sample set to obtain the first adversarial sample generation model.

10. A computer-readable storage medium, characterized in that: A computer program is stored thereon, wherein when the computer program is executed by a processor, the method for defending against adversarial sample attacks as claimed in any one of claims 1 to 6 is implemented.

Images