Black box model inversion attack method and device for text classification model

Abstract

The invention relates to a black box model inversion attack method and device for a text classification model, and the method comprises the steps: extracting a text sequence beginning fragment in a public data set, and constructing an auxiliary data set according to the text sequence beginning fragment; generating a complete text sequence by taking the first fragment of the text sequence in the auxiliary data set as a query access attack model, scoring the complete text sequence to obtain a text sequence score meeting a preset maximization condition, and iteratively optimizing the parameters of the attack model by taking the text sequence score meeting the preset maximization condition as a target to obtain a query access attack model; generating a trained attack model; generating a text sequence-score pair set by using the trained attack model; and based on the text sequence-score pair set, screening out a text sequence greater than or equal to a preset score threshold, and generating a final inversion attack result of the attack model according to the text sequence. Therefore, the problems of few sources of model inversion attack information, high attack difficulty and the like for the text classification model in the black box scene are solved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a black box model inversion attack method and device for a text classification model. Background Art

[0002] In recent years, machine learning technology has flourished and has been widely used in various industries such as finance, healthcare, and retail, showing great potential in promoting innovation and improving efficiency. However, with the in-depth application of machine learning technology, privacy and security issues have also become prominent. Machine learning model training relies on massive amounts of data. In the absence of appropriate privacy protection mechanisms, attackers may restore the original data through privacy attack techniques, resulting in the risk of data privacy leakage.

[0003] The text classification model is a typical machine learning model in natural language processing, which can be used to perform tasks such as sentiment analysis, topic classification, and public opinion monitoring. The construction of a text classification model usually starts with a pre-trained large language model and is fine-tuned on a dataset in a specific field. However, the dataset used for fine-tuning often comes from a specific field and contains personal privacy information or commercial secrets. The high sensitivity of the fine-tuning dataset exacerbates the potential harm of privacy leakage. Analyzing the feasibility of privacy attacks on text classification models and fully evaluating the privacy risks of their fine-tuning datasets have become key issues that need to be addressed.

[0004] In the related technology, model inversion attack methods are mainly divided into two categories. One is the optimization-based method, in which the adversary accesses the model to obtain the output results and reconstructs the data through gradient descent optimization. This method is suitable for white-box scenarios, that is, the adversary can obtain the model parameters, that is, the intermediate calculation results. The other is the model training-based method, in which the adversary trains the attack model to make it output the training data of the target model. This method is suitable for black-box scenarios, that is, the adversary can only obtain the model output results but cannot obtain the model parameters and intermediate results.

[0005] However, in the related art, for text classification models, in white-box scenarios, since the training data of text classification models is a discrete sequence, it is impossible to calculate the gradient of the model output with respect to discrete variables. Therefore, the classic model inversion attack method cannot be directly applied; in black-box scenarios, the adversary cannot obtain the model parameters, resulting in insufficient known information and difficulty in reconstructing the training data. However, text data is sensitive to errors, and errors in word order and individual words may lead to large semantic deviations, requiring high reconstruction accuracy. In addition, text classification models perform classification by learning the features of the input text, and the memory of text data is implicit in the abstract features, which increases the difficulty of the attack and needs to be improved urgently. Summary of the invention

[0006] The present application provides a black-box model inversion attack method and device for a text classification model, so as to solve the problems of few information sources and high attack difficulty in the model inversion attack on the text classification model in the black-box scenario.

[0007] The first aspect of the present application provides a black-box model inversion attack method for a text classification model, comprising the following steps: extracting the beginning fragment of a text sequence in a public data set, and constructing an auxiliary data set based on the beginning fragment of the text sequence; using the beginning fragment of the text sequence in the auxiliary data set as a query to access an attack model, generating a complete text sequence, and scoring the complete text sequence to obtain a text sequence score that satisfies a preset maximization condition, and iteratively optimizing the parameters of the attack model with the text sequence score that satisfies the preset maximization condition as a target to generate a trained attack model; using the trained attack model to generate a text sequence-score pair set; based on the text sequence-score pair set, screening out text sequences that are greater than or equal to a preset score threshold, and generating a final attack result of the attack model based on the text sequence.

[0008] Optionally, in one embodiment of the present application, it also includes: using a black box to access the target model to obtain the prediction probability of the target model; based on the prediction probability, reconstructing the text data in the training set of the target model to generate a text sequence that meets preset optimal conditions.

[0009] Optionally, in one embodiment of the present application, the iterative optimization of the parameters of the attack model with the score as the target to generate a trained attack model includes: initializing the attack model to obtain the predicted probability distribution of the target model; based on the predicted probability distribution, sampling the text sequence to determine the final text sequence, and calculating the state-action pair quality that meets the preset maximization condition based on the final text sequence; updating the parameters of the attack model to obtain updated parameters; iterating the state-action pair quality and the updated parameters until a preset iteration stop condition is met, stopping the iteration, and generating the trained attack model.

[0010] Optionally, in one embodiment of the present application, the method of generating a text sequence-score pair set using the trained attack model includes: obtaining a query for the trained attack model and obtaining replies that meet a preset quantity condition; determining alternative text sequences based on the query and the replies, and calculating scores corresponding to the alternative text sequences; storing the scores corresponding to the alternative text sequences in a preset set to generate the text sequence-score pair set.

[0011] The second aspect of the present application provides a black-box model inversion attack device for a text classification model, including: an extraction module, used to extract the beginning fragment of a text sequence in a public data set, and construct an auxiliary data set based on the beginning fragment of the text sequence; a training module, used to use the beginning fragment of the text sequence in the auxiliary data set as a query to access the attack model, generate a complete text sequence, and score the complete text sequence to obtain a text sequence score that meets a preset maximization condition, and iteratively optimize the parameters of the attack model with the text sequence score that meets the preset maximization condition as a target to generate a trained attack model; a generation module, used to generate a text sequence-score pair set using the trained attack model; an attack module, used to screen out text sequences greater than or equal to a preset score threshold based on the text sequence-score pair set, construct a new target set based on the text sequence, and generate a final attack result of the attack model based on the new target set.

[0012] Optionally, in one embodiment of the present application, it also includes: an access module for accessing the target model using a black box to obtain the prediction probability of the target model; a reconstruction module for reconstructing the text data in the training set of the target model based on the prediction probability to generate a text sequence that meets preset optimal conditions.

[0013] Optionally, in one embodiment of the present application, the training module also includes: an initialization unit, used to initialize the attack model to obtain the predicted probability distribution of the target model; a sampling unit, used to sample the text sequence based on the predicted probability distribution to determine the final text sequence, and calculate the state-action pair quality that meets the preset maximization condition based on the final text sequence; an updating unit, used to update the parameters of the attack model to obtain updated parameters; a training unit, used to iterate the state-action pair quality and the updated parameters until a preset iteration stop condition is met, stop the iteration, and generate the trained attack model.

[0014] Optionally, in one embodiment of the present application, the generation module includes: an acquisition unit, used to obtain queries of the trained attack model and obtain replies that meet a preset number condition; a calculation unit, used to determine alternative text sequences based on the queries and the replies, and calculate scores corresponding to the alternative text sequences; a storage unit, used to store the scores corresponding to the alternative text sequences to a preset set to generate the text sequence-score pair set.

[0015] The third aspect of the present application provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the black box model inversion attack method for a text classification model as described in the above embodiment.

[0016] The fourth aspect of the present application provides a computer-readable storage medium, which stores a computer program. When the program is executed by a processor, it implements the above-mentioned black-box model inversion attack method for a text classification model.

[0017] The fifth aspect of the present application provides a computer program product, which stores a computer program that, when executed by a processor, implements the above-mentioned black-box model inversion attack method for a text classification model.

[0018] The embodiment of the present application innovatively uses a reinforcement learning method based on a policy network to train the attack model, so that in a black box scenario, it is not necessary to obtain the internal structure and parameter information of the target model, and the optimization direction of the attack model parameters can still be guided according to the output of the target model, which alleviates the problem of insufficient information in the black box scenario, solves the problem of the incompatibility of the gradient descent optimization method with discrete text sequences, and improves the success rate of the model inversion attack. Thus, the problems of the small number of information sources and high attack difficulty of the model inversion attack on the text classification model in the black box scenario are solved.

[0019] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0021] Figure 1 A flowchart of a black box model inversion attack method for a text classification model provided according to an embodiment of the present application;

[0022] Figure 2 A schematic diagram of an attack model according to an embodiment of the present application;

[0023] Figure 3 is a flow chart of the REINFORCE algorithm according to one embodiment of the present application;

[0024] Figure 4 A schematic diagram of a Markov decision process model framework according to an embodiment of the present application;

[0025] Figure 5A schematic diagram of a process of training an attack network based on a reinforcement learning algorithm according to an embodiment of the present application;

[0026] Figure 6 is an attack flow chart according to an embodiment of the present application;

[0027] Figure 7 A schematic diagram of the structure of a black box model inversion attack device for a text classification model provided according to an embodiment of the present application;

[0028] Figure 8 It is a schematic diagram of the structure of an electronic device provided according to an embodiment of the present application. DETAILED DESCRIPTION

[0029] Embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.

[0030] The following describes the black-box model inversion attack method and device for a text classification model according to an embodiment of the present application with reference to the accompanying drawings. In view of the problems mentioned in the above background technology that there are few information sources and high attack difficulty for model inversion attacks on text classification models in black-box scenarios, the present application provides a black-box model inversion attack method for text classification models. In this method, a reinforcement learning method based on a policy network is innovatively used to train the attack model, so that in a black-box scenario, there is no need to obtain the internal structure and parameter information of the target model, and the optimization direction of the attack model parameters can still be guided according to the target model output, thereby alleviating the problem of insufficient information in the black-box scenario, solving the problem of the incompatibility between the gradient descent optimization method and the discrete text sequence, and improving the success rate of the model inversion attack. As a result, the problems of few information sources and high attack difficulty for model inversion attacks on text classification models in black-box scenarios are solved.

[0031] Specifically, Figure 1 A flowchart of a black box model inversion attack method for a text classification model provided in an embodiment of the present application.

[0032] Before introducing the black box model inversion attack method for the text classification model proposed in the embodiment of the present application, the three entity roles involved in the attack model in this method are first introduced, such as Figure 2 As shown, they are the data owners Model Trainer and model users Assumptions Model User For attackers, a detailed description of each entity role is as follows:

[0033] (1) Data Owner Data Owner Have a dataset for training a text classification model To the model trainer Provide training dataset

[0034] (2) Model trainer Model Trainer Ability to leverage data owners The model is trained with the training dataset D provided to obtain a model T suitable for text classification tasks, and the model is released by providing an API access interface, so that users can access the model in a black box manner without disclosing the internal details of the model. If the model training is based on a pre-trained large language model, the training dataset Refers to the dataset during the fine-tuning training phase.

[0035] (3) Model user (attacker) Model users access the model (attack target model) released by the model trainer through the API interface black box, and can only obtain the model output for any text sequence input, but cannot obtain the model parameters and intermediate calculation results. For model users, reconstruct the training dataset based on the black-box access results of the model Subset of

[0036] The attack model is composed of the above three entities, among which the data owner Can be a model trainer at the same time You can also act as a model user To simplify the expression, the following description assumes that each entity is different and the attacker Cannot exploit data owner and model trainer Known information.

[0037] like Figure 1 As shown, assuming that the attack target is to reconstruct the text sequence labeled y0 in the training set, the black box model inversion attack method for the text classification model includes the following steps:

[0038] In step S101, the beginning segments of text sequences in the public dataset are extracted, and an auxiliary dataset is constructed based on the beginning segments of the text sequences.

[0039] In the actual execution process, the attacker in the embodiment of the present application Extract the beginning fragments of text sequences from public datasets to form a collection For example, if we obtain a public text dataset, we can extract the first l words from any text sequence to form a set Where l is a random integer satisfying l1≤l≤l2.

[0040] In step S102, the attack model is accessed by querying the beginning fragment of the text sequence in the auxiliary data set, a complete text sequence is generated, and the complete text sequence is scored to obtain a text sequence score that meets the preset maximization condition. The parameters of the attack model are iteratively optimized with the text sequence score that meets the preset maximization condition as the goal to generate a trained attack model.

[0041] It can be understood that the text sequence score of the preset maximization condition in the embodiment of the present application is the maximized text sequence score.

[0042] Among them, the embodiment of the present application can select any pre-trained large language model as the initial attack model And fine-tune the model. Select the supported model The word list of the word segmenter is V. In each round of training, a set is randomly selected The phrase in is used as a query to access the attack model to generate a reply, and the text sequence s composed of the query and the reply is used as the target model input to obtain the predicted probability prob of the target model, and then the initial attack model Evaluate the text fluency perp of the text sequence s, iteratively optimize the parameters of the initial attack model based on the predicted probability and text fluency, and evaluate the score of the text sequence s to be r = prob + α perp, and store (s, r) in the set And update the model parameters θ with the goal of maximizing the text sequence score to obtain the attack model after training.

[0043] The embodiment of the present application innovatively adopts a reinforcement learning method based on a policy network to train the attack model, so that in a black box scenario, there is no need to obtain the internal structure and parameter information of the target model, and the optimization direction of the attack model parameters can still be guided according to the target model output, thereby alleviating the problem of insufficient information in the black box scenario, solving the incompatibility problem between the gradient descent optimization method and discrete text sequences, and improving the success rate of the model inversion attack.

[0044] It should be noted that the embodiment of the present application provides a model inversion attack method for a sentiment analysis model based on a policy gradient algorithm REINFORCE. Figure 3As shown, the representative text classification task sentiment analysis is used as the target model for attack, and the policy-based reinforcement learning algorithm REINFORCE is selected to achieve the reconstruction of the target model and fine-tune part of the text data in the dataset. The reinforcement learning algorithm does not rely on the gradient calculation of the continuous space. Through the interaction between the agent and the environment, it can find the optimal solution in the complex discrete space. This application uses reinforcement learning technology to optimize discrete text sequences.

[0045] Among them, the REINFORCE algorithm is one of the classic policy gradient algorithms. It defines a set of parameters θ and represents the policy as a parameterized probability distribution π of action a under state s. θ (a|s). The reinforcement learning algorithm continuously updates the policy parameters to generate actions that obtain higher rewards. The reinforcement learning optimization objective is defined as:

[0046]

[0047] Where J(θ) is the objective function, τ = (s1, a1, s2, a2, …) is the state-action trajectory generated by the interaction with the environment, R(τ) is the reward of the state-action trajectory τ, and P(τ; θ) is the return of the trajectory τ under the strategy π. θ The probability under (a|s) is defined as:

[0048]

[0049] By calculating the policy gradient, we can get the optimization direction of the policy parameters. The policy gradient is defined as:

[0050]

[0051] After approximation, the policy gradient is calculated as:

[0052]

[0053] Update the policy parameters by policy gradient:

[0054]

[0055] Where η is the learning rate parameter. For discrete action space, the strategy is defined as

[0056]

[0057] Among them, φ(s,a) represents the feature vector of the state-action pair, and here represents the embedding vector corresponding to the state-action text sequence. The corresponding policy gradient can be specifically written as:

[0058]

[0059] The algorithm steps can be described as the following pseudo code:

[0060]

[0061]

[0062] It should be noted that the preset maximization condition can be set by those skilled in the art according to actual conditions and is not specifically limited here.

[0063] Optionally, in one embodiment of the present application, the parameters of the attack model are iteratively optimized with the score as the target to generate a trained attack model, including: initializing the attack model to obtain the predicted probability distribution of the target model; based on the predicted probability distribution, sampling the text sequence to determine the final text sequence, and calculating the state-action pair quality that meets the preset maximization condition based on the final text sequence; updating the parameters of the attack model to obtain updated parameters; iterating the state-action pair quality and the updated parameters until the preset iteration stop condition is met, stopping the iteration, and generating the trained attack model.

[0064] It can be understood that the reinforcement learning method is used to train the attack model A in the embodiment of the present application. θ . Reinforcement learning learns a strategy through the interaction between the agent and the environment so as to obtain the maximum cumulative reward in a given Markov decision process (MDP), where MDP is a mathematical model that simulates the random strategy and reward of the agent in the environment. In the present invention, the attack model parameter is set as the strategy parameter θ, and the reward is the quality of the text sequence generated by the attack model. Then, the attack model can be trained through reinforcement learning. The attacker constructs the model inversion attack problem as an MDP problem. The MDP model is as follows Figure 4 As shown, the reinforcement learning algorithm is then used to solve it. Assume that the attack target is to obtain the target model training data set The label is positive sentiment y + The text sequence, model interaction objects and elements are defined as follows:

[0065] Environment: Target model T and pre-trained large language model

[0066] Agent: Attack Model A θ .

[0067] The definitions of state, action, strategy, reward and other elements are:

[0068] Status t :The state space is That is, any text sequence with a length less than or equal to l. State s t represents the text sequence at the tth time step. The change of state has Markov property, that is, the next state depends only on the current state and action, and has nothing to do with the historical state. At time step t, assuming that the current text sequence length is l t ,

[0069] Action a t :The action space is That is, all the words in the word list V. Attack model A θ According to the text sequence (state s t ) predicts the distribution probability of the next word and samples to generate a word, which is action a t Action a t is added to state s t At the end of t+1 =s t ∪a t , as the state of the next time step, to achieve state transition. Word w, represented by the probability strategy π θ (a t |s t ) sampling generation.

[0070] State transfer function (s t ,a t ) → s t+1 : According to the state s at time step t t and action a t , the state at time step t+1 is

[0071] Strategy π θ (a t |s t ): The strategy is in state s t Next generate action a t The probability distribution of attack model A θ As a policy network, it is used to generate policies; based on the current state s t , predict the probability of the next word, and thus calculate the probability strategy Reinforcement learning is achieved by adjusting the policy network A θ The optimal strategy is obtained by adjusting the parameters θ to maximize the cumulative reward of the agent.

[0072] Reward t : Reward r t For instant evaluation of state-action pairs (s t ,a t ), which is determined by the target model T on the text sequence s t+1Prediction probability and text sequence s under target label t+1 The state s is calculated by the target model T. t+1 Predicted positive sentiment label probability Pr[T(y + |s t+1 )], through the model Calculating text fluency Taking the weight coefficient 0≤α≤1, the reward is defined as

[0073] Reward R: When the maximum text sequence length is reached after several actions, or the action is a termination symbol, it is considered that a complete text sequence has been generated. The cumulative reward value is evaluated based on the reward in the text sequence generation step.

[0074] In the actual implementation process, the embodiment of the present application can select a reinforcement learning algorithm based on a policy network on the basis of clarifying the MDP framework, and continuously update the strategy through multiple rounds of iterations to maximize the cumulative reward. θ As the strategy model in the reinforcement learning algorithm, the optimal text sequence state is found through reinforcement learning training, and the attack model A is realized at the same time. θ Parameter optimization. Figure 5 The main steps of the reinforcement learning algorithm based on the policy network are:

[0075] (1) Initialize the policy network: The policy network is the attack model A of the model inversion attack θ , initialize the attack model parameters θ←θ0, the attack model can obtain the predicted probability distribution of the next word for any text sequence, and define the state s at any time step t t Next (current text sequence) Next select action a t The probability distribution of (next word), i.e., the probability strategy

[0076] (2) State sampling and reward calculation: Set the maximum text sequence length to m and take the auxiliary data set in each iteration round. Any element in is state s0, and the policy network is based on state s at each time step t. t Generate the next word as action a t , and add it to the state s t At the end, a new state s is formed t+1 The environment will feedback the quality of the text sequence, that is, calculate the reward r t =Pr[T(y0|s t+1 )]+αPPL(s t+1 ), where Pr[T(y0|s t+1 )] is the s calculated by the target model Tt+1 The predicted probability under the target label y0, PPL(s t+1 ) is the model The calculated text fluency, α (0≤α≤1) is the weight coefficient. The prediction probability reflects the possibility that the text sequence is the training data of the target model, and the text fluency reflects the degree to which the text sequence conforms to the laws of natural language. Repeat multiple time steps until s t+1 The length reaches m, or a t =EOS, complete state sampling, suppose the sampling has gone through l steps, and the state obtained is s l , the state action trajectory is s0→a0→s1→a1→…→s l , where EOS represents the termination symbol;

[0077] (3) Calculate policy gradient and policy update: According to the action trajectory s0→a0→s1→a1→…→s l The instant reward value during the process, the cumulative reward value during the evaluation state sampling process, and the loss function J(θ) and policy gradient are calculated. And update the policy network parameters Where η is the learning rate parameter. According to the specific reinforcement learning algorithm, the specific reward calculation method and loss function are selected. When the value network is used to calculate the reward, the observed actual reward is used as the target, and the value network is continuously updated to improve its prediction accuracy;

[0078] (4) Iterative optimization: Repeat steps (2)-(4) until the preset number of iterations N is reached. Finally, the optimal policy network parameter θ is obtained. * , that is, the fine-tuned attack model

[0079] The embodiment of the present application takes the prediction probability of the target model for the text sequence as the main optimization target, and combines the pre-trained large language model as prior knowledge, thereby improving the similarity between the text data reconstructed by the attack and the original text data, and achieving a better attack effect.

[0080] In step S103, a text sequence-score pair set is generated using the trained attack model.

[0081] Among them, the attacker in the embodiment of this application Obtaining a set of text sequence-score pairs through attack model reasoning This provides support for subsequent screening of text sequences and further improves the success rate of model inversion attacks.

[0082] Optionally, in one embodiment of the present application, a text sequence-score pair set is generated using a trained attack model, including: obtaining a query for the trained attack model, and obtaining replies that meet a preset quantity condition; determining alternative text sequences based on the query and the reply, and calculating the scores corresponding to the alternative text sequences; storing the scores corresponding to the alternative text sequences in a preset set to generate a text sequence-score pair set.

[0083] It is understandable that the preset set in the embodiment of the present application can be a set

[0084] Specifically, the embodiment of the present application can be based on the trained attack model Given any text sequence prefix as attack model A θ , sample and generate any number of replies, concatenate the query and reply to form a candidate text sequence, calculate the score corresponding to the candidate text sequence, and store the text sequence-score for the set

[0085] In step S104, based on the text sequence-score pair set, text sequences greater than or equal to a preset score threshold are screened out, and a final attack result of the attack model is generated according to the text sequence.

[0086] It is understandable that the embodiment of the present application evaluates the privacy risk of the model through model inversion attack, which is divided into two types: directed and non-directed attacks. Among them, the directed attack inserts the specified sample multiple times in the model fine-tuning data set, reconstructs the suffix after the given prefix of the specified sample, and evaluates the accuracy of the reconstruction under different repetition times; the non-directed attack does not add the specified sample, reconstructs a large number of text sequences that are more likely to be training samples according to the model, and verifies the accuracy and recall rate of the reconstruction. By quantitatively evaluating the attack performance, it is possible to reflect the risk of model leakage of training data and provide a reference for whether to choose a privacy protection mechanism. By attacking models with different model structures, parameter scales and training methods, it is conducive to further analysis of the principle of model privacy leakage.

[0087] In the actual implementation process, the embodiment of the present application can synthesize the text sequence-score pairs stored in the above steps, and filter the text sequences with scores greater than or equal to the threshold r0 based on the text sequence-score pair set, and form a set according to the text sequence And will collect As the final attack result.

[0088] In this embodiment of the present application, an attacker can Get the attack results Improved attack accuracy.

[0089] Optionally, in one embodiment of the present application, it also includes: using a black box to access the target model to obtain the prediction probability of the target model; based on the prediction probability, reconstructing the text data in the training set of the target model to generate a text sequence that meets preset optimal conditions.

[0090] It can be understood that the target model in the embodiment of the present application is a text classification model.

[0091] Among them, the embodiment of the present application obtains the model prediction value through black box access to the text classification model to reconstruct part of the text data in the model training set and generate the optimal text sequence, aiming to reveal and evaluate the data privacy risks of the text classification model in the black box scenario and improve the accuracy of reconstructing text data in the black box scenario.

[0092] It should be noted that the preset optimal conditions can be set by those skilled in the art according to actual conditions and are not specifically limited here.

[0093] Specifically, it can be combined Figure 6 As shown, the working principle of the black box model inversion attack method for the text classification model in the embodiment of the present application is described in detail with a specific embodiment.

[0094] like Figure 6 As shown, the embodiment of the present application may include the following steps:

[0095] Step S601: Obtain the API of the sentiment analysis target model T.

[0096] (1) Obtain the target model

[0097] The adversary has black-box access to the target model T. In this embodiment, it is assumed that the target model is a sentiment analysis model that can predict positive sentiment labels y for any text sequence x. + and negative sentiment label y - The probability of + |T(x)) and Pr(y - |T(x)).

[0098] Step S602: Select any sentiment analysis dataset Pick The beginning fragments of the Chinese text sequence constitute the auxiliary dataset

[0099] (2) Obtaining auxiliary data sets

[0100] Select any public text dataset in the field of sentiment analysis, and for any text sequence s = {s1, s2, ...} in the dataset, randomly select an integer l that satisfies l1≤l≤l2, and take the first l words of the text sequence s {s1, s2, ..., sl}, forming an auxiliary data set

[0101] Step S603: Initialize the attack model to any pre-trained text generation model

[0102] Step S604: Fine-tune the attack model parameter θ based on the REINFORCE algorithm and store the text sequence-reward pairs in the optimization process And get the optimal parameter θ * .

[0103] Step S605: fine-tuning the attack model Perform model inference and generate text sequences - rewards

[0104] Among them, the embodiment of the present application is the attack initiation stage

[0105] (1) Initialize the attack model

[0106] An attacker can publicly obtain any pre-trained text generation model Among them, θ0 is a model parameter that can generate fluent natural language and can be used to evaluate text fluency. The pre-trained text generation model is used as the initial attack model, and fine-tuned on this basis to retain the knowledge learned by the pre-training of the text generation model. Model tokenizer, whose vocabulary is V.

[0107] (2) Fine-tuning attack model parameters

[0108] Based on the reinforcement learning algorithm REINFORCE, the MDP problem is solved through optimization iteration. The specific process is as follows.

[0109] Step 1: Parameter initialization. Initialize the policy network as an attack model Set the learning rate η, set the maximum number of iterations N, and set the maximum text length to m;

[0110] Step 2: Strategy update loop, at any iteration round i (1≤i≤N), execute the following sub-steps.

[0111] Step 2.1 State initialization: Randomly select a set Any sequence of text in As the initial state of round i

[0112] Step 2.2 State sampling: In round i, for any time step t, according to the current state Compute actions through the policy network And generate the next step status when or When the state sampling termination condition is reached, the final state of round i is generated. Suppose the state sampling has gone through l i Step, the final state is Where sample(·) represents the sampling function and EOS represents the termination symbol.

[0113] Step 2.3 Calculate rewards: Calculate rewards based on the final state of round i Define the return at each time step of the i-th round And record the status and rewards of this round

[0114] Step 3: Calculate the policy gradient

[0115]

[0116] And update the policy

[0117]

[0118] Among them, η is the learning rate parameter.

[0119] Step S606: Filter the collection The text sequence that meets the conditions in the attack result is obtained

[0120] (3) Obtaining attack results

[0121] In the above steps, the attack model is first fine-tuned by iteratively updating the strategy parameters. During the iteration process, the state is continuously updated, and the state and reward recorded in each round are stored in the same set. Select any text sequence prefix, set the maximum number of words m, generate a text sequence s according to the fine-tuned attack model and calculate the reward value r, repeat n times to form a set Filter Collection The text sequence whose reward value is greater than or equal to the threshold r0 constitutes the final attack result set

[0122] According to the black box model inversion attack method for the text classification model proposed in the embodiment of the present application, the reinforcement learning method based on the policy network is innovatively used to train the attack model, so that in the black box scenario, there is no need to obtain the internal structure and parameter information of the target model, and the optimization direction of the attack model parameters can still be guided according to the output of the target model, which alleviates the problem of insufficient information in the black box scenario, solves the problem of the incompatibility of the gradient descent optimization method with discrete text sequences, and improves the success rate of the model inversion attack. Thus, the problem of few information sources and high attack difficulty of the model inversion attack on the text classification model in the black box scenario is solved.

[0123] Next, a black box model inversion attack device for a text classification model proposed according to an embodiment of the present application is described with reference to the accompanying drawings.

[0124] Figure 7 It is a structural diagram of a black box model inversion attack device for a text classification model according to an embodiment of the present application.

[0125] like Figure 7 As shown, the black box model inversion attack device 10 for the text classification model includes: an extraction module 100, a training module 200, a generation module 300 and an attack module 400.

[0126] Specifically, the extraction module 100 is used to extract the beginning segments of the text sequences in the public dataset, and construct an auxiliary dataset according to the beginning segments of the text sequences.

[0127] The training module 200 is used to access the attack model using the beginning fragment of the text sequence in the auxiliary data set as a query, generate a complete text sequence, and score the complete text sequence to obtain a text sequence score that meets the preset maximization condition, and iteratively optimize the parameters of the attack model with the text sequence score that meets the preset maximization condition as the goal to generate a trained attack model.

[0128] The generation module 300 is used to generate a text sequence-score pair set using the trained attack model.

[0129] The attack module 400 is used to screen out text sequences greater than or equal to a preset score threshold based on the text sequence-score pair set, and generate a final attack result of the attack model according to the text sequence.

[0130] Optionally, in one embodiment of the present application, the black box model inversion attack device for the text classification model also includes: an access module and a reconstruction module.

[0131] Among them, the access module is used to access the target model using a black box to obtain the prediction probability of the target model.

[0132] The reconstruction module is used to reconstruct the text data in the training set of the target model based on the prediction probability to generate a text sequence that meets the preset optimal conditions.

[0133] Optionally, in one embodiment of the present application, the training module 200 further includes: an initialization unit, a sampling unit, an updating unit and a training unit.

[0134] Among them, the initialization unit is used to initialize the attack model to obtain the predicted probability distribution of the target model.

[0135] The sampling unit is used to sample the text sequence based on the predicted probability distribution to determine the final text sequence, and calculate the state-action pair quality that meets the preset maximization condition based on the final text sequence.

[0136] The updating unit is used to update the parameters of the attack model to obtain updated parameters.

[0137] The training unit is used to iterate the state-action pair quality and the updated parameters until a preset iteration stop condition is met, then stop the iteration and generate a trained attack model.

[0138] Optionally, in one embodiment of the present application, the generation module 300 includes: an acquisition unit, a calculation unit and a storage unit.

[0139] The acquisition unit is used to obtain queries of the trained attack model and obtain replies that meet a preset quantity condition.

[0140] The calculation unit is used to determine the candidate text sequence according to the query and the reply, and calculate the score corresponding to the candidate text sequence.

[0141] The storage unit is used to store the scores corresponding to the candidate text sequences into a preset set to generate a text sequence-score pair set.

[0142] It should be noted that the aforementioned explanation of the embodiment of the black-box model inversion attack method for the text classification model is also applicable to the black-box model inversion attack device for the text classification model in this embodiment, and will not be repeated here.

[0143] According to the black box model inversion attack device for text classification models proposed in the embodiment of the present application, a reinforcement learning method based on a policy network is innovatively used to train the attack model, so that in a black box scenario, there is no need to obtain the internal structure and parameter information of the target model, and the optimization direction of the attack model parameters can still be guided according to the output of the target model, which alleviates the problem of insufficient information in the black box scenario, solves the problem of the incompatibility of the gradient descent optimization method with discrete text sequences, and improves the success rate of the model inversion attack. Thus, the problem of few information sources and high attack difficulty of the model inversion attack on the text classification model in the black box scenario is solved.

[0144] Figure 8 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. The electronic device may include:

[0145] A memory 801 , a processor 802 , and a computer program stored in the memory 801 and executable on the processor 802 .

[0146] When the processor 802 executes the program, the black box model inversion attack method for the text classification model provided in the above embodiment is implemented.

[0147] Furthermore, the electronic device further comprises:

[0148] The communication interface 803 is used for communication between the memory 801 and the processor 802 .

[0149] The memory 801 is used to store computer programs that can be executed on the processor 802 .

[0150] The memory 801 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0151] If the memory 801, the processor 802 and the communication interface 803 are implemented independently, the communication interface 803, the memory 801 and the processor 802 can be connected to each other through a bus and communicate with each other. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 8 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0152] Optionally, in a specific implementation, if the memory 801, the processor 802 and the communication interface 803 are integrated on a chip, the memory 801, the processor 802 and the communication interface 803 can communicate with each other through an internal interface.

[0153] The processor 802 may be a central processing unit (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application.

[0154] This embodiment also provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the black-box model inversion attack method for a text classification model is implemented as described above.

[0155] An embodiment of the present application also provides a computer program product, on which a computer program is stored, and when the program is executed by a processor, the black box model inversion attack method for the text classification model is implemented as described above.

[0156] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or N embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0157] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of this application, "N" means at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0158] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, fragment or portion of code comprising one or N executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present application belong.

[0159] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purpose of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or N wirings (electronic devices), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically by optically scanning the paper or other medium and then editing, interpreting or processing in other suitable ways as necessary and then storing it in a computer memory.

[0160] It should be understood that the various parts of the present application can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiment, the N steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0161] A person skilled in the art may understand that all or part of the steps in the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.

[0162] In addition, each functional unit in each embodiment of the present application may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0163] The storage medium mentioned above may be a read-only memory, a magnetic disk or an optical disk, etc. Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limiting the present application. A person of ordinary skill in the art may change, modify, replace and modify the above embodiments within the scope of the present application.

Claims

1. A black box model inversion attack method for a text classification model, characterized in that: The following steps are involved: Extracting the beginning fragments of text sequences in the public data set, and constructing an auxiliary data set based on the beginning fragments of the text sequences; Using the beginning segment of the text sequence in the auxiliary data set as a query to access the attack model, generating a complete text sequence, and scoring the complete text sequence to obtain a text sequence score that satisfies a preset maximization condition, and iteratively optimizing the parameters of the attack model with the text sequence score that satisfies the preset maximization condition as a target to generate a trained attack model; Generate a set of text sequence-score pairs using the trained attack model; Based on the text sequence-score pair set, text sequences greater than or equal to a preset score threshold are screened out, and a final attack result of the attack model is generated according to the text sequence.

2. The method according to claim 1, characterized in that: Also includes: Accessing a target model using a black box to obtain a prediction probability of the target model; Based on the predicted probability, the text data in the training set of the target model is reconstructed to generate a text sequence that meets a preset optimal condition.

3. The method according to claim 2, characterized in that The iterative optimization of the parameters of the attack model with the score as the target to generate a trained attack model includes: Initializing the attack model to obtain a predicted probability distribution of the target model; Based on the predicted probability distribution, the text sequence is sampled to determine a final text sequence, and a state-action pair quality satisfying a preset maximization condition is calculated according to the final text sequence; Updating parameters of the attack model to obtain updated parameters; Iterate the state-action pair quality and the updated parameters until a preset iteration stop condition is met, stop the iteration, and generate the trained attack model.

4. The method according to claim 1, characterized in that The step of generating a text sequence-score pair set using the trained attack model includes: Obtaining queries for the trained attack model and obtaining responses that meet a preset number of conditions; Determine candidate text sequences according to the query and the response, and calculate scores corresponding to the candidate text sequences; The scores corresponding to the candidate text sequences are stored in a preset set to generate the text sequence-score pair set.

5. A black box model inversion attack device for a text classification model, characterized in that: include: An extraction module, used to extract the beginning fragments of text sequences in the public data set, and construct an auxiliary data set based on the beginning fragments of the text sequences; A training module, used to use the beginning segment of the text sequence in the auxiliary data set as a query to access the attack model, generate a complete text sequence, and score the complete text sequence to obtain a text sequence score that meets a preset maximization condition, and iteratively optimize the parameters of the attack model with the text sequence score that meets the preset maximization condition as a target to generate a trained attack model; A generation module, used to generate a text sequence-score pair set using the trained attack model; The attack module is used to screen out text sequences greater than or equal to a preset score threshold based on the text sequence-score pair set, construct a new target set based on the text sequence, and generate a final attack result of the attack model based on the new target set.

6. The device according to claim 5, characterized in that Also includes: An access module, used to access the target model using a black box to obtain the prediction probability of the target model; A reconstruction module is used to reconstruct the text data in the training set of the target model based on the prediction probability to generate a text sequence that meets the preset optimal conditions.

7. The device according to claim 6, characterized in that The training module includes: An initialization unit, used to initialize the attack model to obtain a predicted probability distribution of the target model; A sampling unit, configured to sample the text sequence based on the predicted probability distribution to determine a final text sequence, and calculate a state-action pair quality satisfying a preset maximization condition according to the final text sequence; An updating unit, used for updating the parameters of the attack model to obtain updated parameters; The training unit is used to iterate the state-action pair quality and the updated parameters until a preset iteration stop condition is met, stop the iteration, and generate the trained attack model.

8. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the black-box model inversion attack method for a text classification model as described in any one of claims 1 to 4.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: The program is executed by a processor to implement the black box model inversion attack method for a text classification model as described in any one of claims 1 to 4.

10. A computer program product, comprising a computer program, characterized in that The computer program is executed to implement the black box model inversion attack method for a text classification model as described in any one of claims 1 to 4.

Images