Black-box model inversion attack method and device for text classification model

Abstract

The application relates to a black box model inversion attack method and device for a text classification model, wherein the method comprises the following steps: extracting a text sequence beginning fragment in a public data set, and constructing an auxiliary data set according to the text sequence beginning fragment; taking the text sequence beginning fragment in the auxiliary data set as a query to access an attack model, generating a complete text sequence, scoring the complete text sequence, obtaining a text sequence score satisfying a preset maximization condition, taking the text sequence score satisfying the preset maximization condition as a target to iteratively optimize parameters of the attack model, and generating a trained attack model; generating a text sequence-score pair set by using the trained attack model; based on the text sequence-score pair set, filtering out a text sequence greater than or equal to a preset score threshold, and generating a final inversion attack result of the attack model according to the text sequence. Therefore, the problems of few sources of model inversion attack information and high attack difficulty of the model inversion attack on the text classification model in a black box scenario are solved.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a black-box model inversion attack method and apparatus for text classification models. Background Technology

[0002] In recent years, machine learning technology has flourished and has been widely applied in various industries such as finance, healthcare, and retail, demonstrating enormous potential in driving innovation and improving efficiency. However, with the deepening application of machine learning technology, privacy and security issues have also become prominent. Machine learning model training relies on massive amounts of data. Without appropriate privacy protection mechanisms, attackers may be able to recover the original data through privacy attack techniques, posing a risk of data privacy breaches.

[0003] Text classification models are a typical machine learning model in natural language processing, used for tasks such as sentiment analysis, topic classification, and public opinion monitoring. The construction of text classification models typically starts with a pre-trained large language model and then fine-tunes it on domain-specific datasets. However, the datasets used for fine-tuning often originate from specific domains and contain personal privacy information or trade secrets. The high sensitivity of fine-tuning datasets exacerbates the potential harm of privacy breaches. Analyzing the feasibility of privacy attacks on text classification models and fully assessing the privacy risks of fine-tuning datasets have become critical issues that urgently need to be addressed.

[0004] In related technologies, model inversion attack methods are mainly divided into two categories. One category is optimization-based methods, where the adversary accesses the model to obtain the output results and reconstructs the data through gradient descent optimization. This is suitable for white-box scenarios, where the adversary can obtain the model's parameters, i.e., intermediate calculation results. The other category is model training-based methods, where the adversary trains the attack model to output the target model's training data. This is suitable for black-box scenarios, where the adversary can only obtain the model's output results but cannot obtain the model's parameters and intermediate results.

[0005] However, in related technologies, for text classification models, in white-box scenarios, because the training data of text classification models is a discrete sequence, it is impossible to calculate the gradient of the model output with respect to discrete variables. Therefore, classic model inversion attack methods cannot be directly applied. In black-box scenarios, adversaries cannot obtain model parameters, resulting in insufficient known information and making it difficult to reconstruct the training data. Furthermore, text data is sensitive to errors; errors in word order and individual words can lead to significant semantic biases, requiring high reconstruction accuracy. In addition, text classification models learn the features of the input text for classification; the memory of the text data is implicit in abstract features, increasing the difficulty of attacks and urgently needing improvement. Summary of the Invention

[0006] This application provides a black-box model inversion attack method and apparatus for text classification models, in order to solve the problems of limited information sources and high attack difficulty in black-box scenarios for model inversion attacks on text classification models.

[0007] The first aspect of this application provides a black-box model inversion attack method for text classification models, comprising the following steps: extracting the beginning segments of text sequences from a public dataset and constructing an auxiliary dataset based on the beginning segments of the text sequences; using the beginning segments of the text sequences in the auxiliary dataset as a query access attack model to generate complete text sequences, scoring the complete text sequences to obtain text sequence scores that satisfy a preset maximization condition, and iteratively optimizing the parameters of the attack model with the text sequence scores that satisfy the preset maximization condition as the target to generate a trained attack model; generating a set of text sequence-score pairs using the trained attack model; and filtering out text sequences that are greater than or equal to a preset score threshold based on the set of text sequence-score pairs, and generating the final attack result of the attack model based on the text sequences.

[0008] Optionally, in one embodiment of this application, the method further includes: using black-box access to the target model to obtain the prediction probability of the target model; and reconstructing the text data in the training set of the target model based on the prediction probability to generate a text sequence that satisfies a preset optimal condition.

[0009] Optionally, in one embodiment of this application, the step of iteratively optimizing the parameters of the attack model with the score as the target to generate a trained attack model includes: initializing the attack model to obtain the predicted probability distribution of the target model; sampling the text sequence based on the predicted probability distribution to determine the final text sequence, and calculating the state-action pair quality that satisfies a preset maximization condition based on the final text sequence; updating the parameters of the attack model to obtain the updated parameters; iterating the state-action pair quality and the updated parameters until a preset iteration stopping condition is met, then stopping the iteration to generate the trained attack model.

[0010] Optionally, in one embodiment of this application, generating a text sequence-score pair set using the trained attack model includes: obtaining a query from the trained attack model and obtaining responses that meet a preset number of conditions; determining candidate text sequences based on the query and the responses, and calculating the scores corresponding to the candidate text sequences; and storing the scores corresponding to the candidate text sequences in a preset set to generate the text sequence-score pair set.

[0011] A second aspect of this application provides a black-box model inversion attack device targeting a text classification model, comprising: an extraction module for extracting the beginning segments of text sequences from a public dataset and constructing an auxiliary dataset based on the beginning segments of the text sequences; a training module for generating complete text sequences using the beginning segments of the text sequences in the auxiliary dataset as a query access attack model, scoring the complete text sequences to obtain text sequence scores that satisfy a preset maximization condition, and iteratively optimizing the parameters of the attack model with the text sequence scores that satisfy the preset maximization condition as the target to generate a trained attack model; a generation module for generating a set of text sequence-score pairs using the trained attack model; and an attack module for filtering text sequences that are greater than or equal to a preset score threshold based on the set of text sequence-score pairs, constructing a new target set based on the text sequences, and generating the final attack result of the attack model based on the new target set.

[0012] Optionally, in one embodiment of this application, it further includes: an access module, configured to access the target model using a black box to obtain the prediction probability of the target model; and a reconstruction module, configured to reconstruct the text data in the training set of the target model based on the prediction probability to generate a text sequence that satisfies a preset optimal condition.

[0013] Optionally, in one embodiment of this application, the training module further includes: an initialization unit, configured to initialize the attack model to obtain the predicted probability distribution of the target model; a sampling unit, configured to sample the text sequence based on the predicted probability distribution to determine the final text sequence, and calculate the state-action pair quality that satisfies a preset maximization condition based on the final text sequence; an update unit, configured to update the parameters of the attack model to obtain the updated parameters; and a training unit, configured to iterate the state-action pair quality and the updated parameters until a preset iteration stop condition is met, then stop the iteration and generate the trained attack model.

[0014] Optionally, in one embodiment of this application, the generation module includes: an acquisition unit, configured to acquire a query of the trained attack model and acquire responses that meet a preset number of conditions; a calculation unit, configured to determine candidate text sequences based on the query and the responses, and calculate the scores corresponding to the candidate text sequences; and a storage unit, configured to store the scores corresponding to the candidate text sequences into a preset set to generate the text sequence-score pair set.

[0015] A third aspect of this application provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the black-box model inversion attack method against a text classification model as described in the above embodiments.

[0016] A fourth aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described black-box model inversion attack method against a text classification model.

[0017] A fifth aspect of this application provides a computer program product that stores a computer program that, when executed by a processor, implements the black-box model inversion attack method for text classification models as described above.

[0018] This application innovatively employs a policy network-based reinforcement learning method to train the attack model. This allows the attack model to be guided in its parameter optimization direction based on the target model's output, even in a black-box environment, without needing to obtain the target model's internal structure and parameter information. This alleviates the information shortage problem in black-box scenarios and solves the incompatibility issue between gradient descent optimization and discrete text sequences, thus improving the success rate of model inversion attacks. Therefore, it addresses the problems of limited information sources and high attack difficulty in black-box scenarios for model inversion attacks against text classification models.

[0019] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0020] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein:

[0021] Figure 1 This is a flowchart of a black-box model inversion attack method for text classification models according to an embodiment of this application;

[0022] Figure 2 This is a schematic diagram of an attack model according to an embodiment of this application;

[0023] Figure 3 This is a flowchart of the REINFORCE algorithm according to an embodiment of this application;

[0024] Figure 4 This is a schematic diagram of a Markov decision process model framework according to an embodiment of this application;

[0025] Figure 5This is a schematic diagram of the process of training an attack network based on a reinforcement learning algorithm according to an embodiment of this application;

[0026] Figure 6 This is a flowchart illustrating an attack according to an embodiment of this application;

[0027] Figure 7 This is a schematic diagram of a black-box model inversion attack device for text classification models provided in an embodiment of this application;

[0028] Figure 8 This is a schematic diagram of the structure of an electronic device provided according to an embodiment of this application. Detailed Implementation

[0029] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this application, and should not be construed as limiting this application.

[0030] The following describes a black-box model inversion attack method and apparatus for text classification models according to embodiments of this application, with reference to the accompanying drawings. Addressing the issues of limited information sources and high attack difficulty in black-box scenarios for model inversion attacks against text classification models mentioned in the background, this application provides a black-box model inversion attack method for text classification models. This method innovatively employs a policy network-based reinforcement learning method to train the attack model. This allows the attack model to be guided in the optimization direction of its parameters based on the target model's output, even in a black-box environment, without needing to obtain the target model's internal structure and parameter information. This alleviates the problem of insufficient information in black-box scenarios and solves the incompatibility between gradient descent optimization and discrete text sequences, thereby improving the success rate of model inversion attacks. Thus, it solves the problems of limited information sources and high attack difficulty in black-box scenarios for model inversion attacks against text classification models.

[0031] Specifically, Figure 1 This is a flowchart illustrating a black-box model inversion attack method for text classification models provided in an embodiment of this application.

[0032] Before introducing the black-box model inversion attack method for text classification models proposed in the embodiments of this application, we will first introduce the three entity roles involved in the attack model in this method, such as... Figure 2 As shown, these are the data owners. Model trainers and model users Hypothetical model users For attackers, each entity role is described in detail below:

[0033] (1) Data owner Data owner Have a dataset for training text classification models To model trainers Provide training dataset

[0034] (2) Model trainer Model trainers Able to utilize data owner The provided training dataset D is used to train the model, resulting in a model T adapted for the text classification task. The model is then published via an API, allowing users to access it in a black-box manner without exposing its internal details. If the model training is based on a pre-trained large language model, then the training dataset... This refers to the dataset used in the fine-tuning training phase.

[0035] (3) Model users (attackers) The model user accesses the model (the target model) published by the model trainer through a black-box API interface. They can only obtain the model output from arbitrary text sequence inputs, but cannot obtain model parameters or intermediate calculation results. Attacker For model users, the training dataset is reconstructed based on the black-box access results of the model. subset of

[0036] The attack model consists of the above three entities, where the data owner... Can simultaneously serve as a model trainer You can also play the role of a model user The role is to examine the privacy risks of one's own dataset. For simplicity, the following description assumes that each entity is distinct, and the attacker... Cannot utilize data owner and model trainers The known information.

[0037] like Figure 1 As shown, assuming the attack target is to reconstruct the text sequence labeled y0 in the training set, this black-box model inversion attack method targeting the text classification model includes the following steps:

[0038] In step S101, the beginning segments of text sequences are extracted from the public dataset, and an auxiliary dataset is constructed based on the beginning segments of the text sequences.

[0039] In actual execution, the attacker in this application embodiment Extract the beginning segments of text sequences from public datasets to form a set. If you obtain a publicly available text dataset, for any text sequence, extract the first l words to form a set. Where l is a random integer, satisfying l1≤l≤l2.

[0040] In step S102, the beginning segment of the text sequence in the auxiliary dataset is used as the query access attack model to generate a complete text sequence and score the complete text sequence to obtain the text sequence score that satisfies the preset maximization condition. The parameters of the attack model are iteratively optimized with the text sequence score that satisfies the preset maximization condition as the target to generate the trained attack model.

[0041] It is understood that the text sequence score with the preset maximization condition in this application embodiment is the maximized text sequence score.

[0042] In this embodiment, any pre-trained large language model can be selected as the initial attack model. The model was then fine-tuned and trained. A support model was selected. The word segmenter is denoted by V, and the word list for the word segmenter is randomly selected in each training round. The phrase in the query is used as the query access attack model to generate a response. The text sequence s, formed by concatenating the query and response, is used as input to the target model to obtain the target model's predicted probability prob, which is then processed by the initial attack model. The text fluency perp of the text sequence s is evaluated. The parameters of the initial attack model are iteratively optimized based on the predicted probability and text fluency. The score of the text sequence s is r = prob + α·perp, and stored in the set (s,r). The model parameters θ are updated with the goal of maximizing the text sequence score to obtain the attack model after training.

[0043] This application innovatively employs a policy network-based reinforcement learning method to train the attack model. This allows the attack model to be guided in the direction of parameter optimization based on the target model output, even in a black-box scenario, without needing to obtain the internal structure and parameter information of the target model. This alleviates the problem of insufficient information in a black-box scenario and solves the problem of the gradient descent optimization method being unsuitable for discrete text sequences, thereby improving the success rate of model inversion attacks.

[0044] It should be noted that this application provides a model inversion attack method for sentiment analysis models based on the policy gradient algorithm REINFORCE. The REINFORCE algorithm is as follows: Figure 3As shown, using sentiment analysis, a representative text classification task, as the target model, the policy-based reinforcement learning algorithm REINFORCE was selected to reconstruct the target model and fine-tune a portion of the text data in the dataset. Reinforcement learning algorithms do not rely on gradient calculations in continuous space; through the interaction between the agent and the environment, they can find the optimal solution in a complex discrete space. This application utilizes reinforcement learning techniques to optimize discrete text sequences.

[0045] Among them, the REINFORCE algorithm is one of the classic policy gradient algorithms. It defines a set of parameters θ and represents the policy as the parameterized probability distribution π of action a in state s. θ (a|s). Reinforcement learning algorithms continuously update policy parameters to generate actions that yield higher rewards. The optimization objective of reinforcement learning is defined as:

[0046]

[0047] Where J(θ) is the objective function, τ = (s1, a1, s2, a2, ...) is the state-action trajectory generated by interaction with the environment, R(τ) is the reward for the state-action trajectory τ, and P(τ; θ) is the reward for the trajectory τ in policy π. θ The probability under (a|s) is defined as:

[0048]

[0049] By calculating the policy gradient, the optimization direction of the policy parameters is obtained. The policy gradient is defined as:

[0050]

[0051] The policy gradient is calculated using an approximate estimation as follows:

[0052]

[0053] Update policy parameters using policy gradients:

[0054]

[0055] Where η is the learning rate parameter. For the discrete action space, the policy is defined as follows:

[0056]

[0057] Where φ(s,a) represents the feature vector of the state-action pair, and here it represents the embedding vector corresponding to the text sequence of the state-action pair. The corresponding policy gradient can be specifically written as:

[0058]

[0059] The algorithm steps can be described in the following pseudocode:

[0060]

[0061]

[0062] It should be noted that the preset maximization conditions can be set by those skilled in the art according to the actual situation, and no specific restrictions are imposed here.

[0063] Optionally, in one embodiment of this application, the parameters of the attack model are iteratively optimized with a score as the target to generate a trained attack model, including: initializing the attack model to obtain the predicted probability distribution of the target model; sampling the text sequence based on the predicted probability distribution to determine the final text sequence, and calculating the state-action pair quality that satisfies a preset maximization condition based on the final text sequence; updating the parameters of the attack model to obtain the updated parameters; iterating the state-action pair quality and the updated parameters until a preset iteration stopping condition is met, then stopping the iteration to generate the trained attack model.

[0064] It is understood that the attack model A is trained using reinforcement learning methods in the embodiments of this application. θ Reinforcement learning learns a policy through the interaction between an agent and its environment, aiming to maximize the cumulative reward within a given Markov Decision Process (MDP). An MDP is a mathematical model that simulates the agent's stochastic policy and reward within the environment. In this invention, the attack model parameters are set as policy parameters θ, and the reward is the quality of the text sequence generated by the attack model. Then, reinforcement learning can be used to train the attack model. The attacker constructs the model inversion attack problem as an MDP problem, and the MDP model is as follows: Figure 4 As shown, the solution is then obtained using a reinforcement learning algorithm. Assume the attack objective is to obtain the training dataset of the target model. The Chinese label is positive emotion y + The text sequence, model interaction objects, and element definitions are as follows:

[0065] Environment: Target model T and pre-trained large language model

[0066] Agent: Attack Model A θ .

[0067] The definitions of elements such as state, action, strategy, and reward are as follows:

[0068] state s t The state space is That is, a text sequence of any length less than or equal to l. State s t Let represent the text sequence at time step t. The state changes exhibit the Markov property, meaning the next state depends only on the current state and actions, and is independent of past states. At time step t, assume the length of the current text sequence is l. t ,

[0069] Action a t Action space is That is, all words in vocabulary V. Attack Model A θ Based on the text sequence (states) t Predict the probability distribution of the next word and sample it to generate a word, which is action a. t Action a t Added to state s t At the end, generate a new text sequence s t+1 =s t ∪a t The state is transitioned to the state at the next time step. The word w is determined by the probabilistic policy π. θ (a t |s t Sample generation.

[0070] State transition function (s) t ,a t )→s t+1 Based on the state s at time step t t and action a t The state at time step t+1 is obtained as follows

[0071] Strategy π θ (a t |s t The strategy is in state s. t The following action a is generated t The probability distribution makes attack model A θ As a policy network, it is used to generate policies; based on the current state s t Predict the probability of the next word, and thus calculate the probability strategy. Reinforcement learning adjusts the policy network A θ By determining the parameter θ, we can obtain the optimal strategy that maximizes the cumulative reward for the agent.

[0072] Reward r t Reward r t Used for immediate assessment of state-action pairs (s t ,a t The quality of the text sequence s is determined by the target model T. t+1Predicted probabilities and text sequences under target labels t+1 The smoothness consists of two parts. The state s is calculated through the target model T. t+1 Predicted positive sentiment label probability Pr[T(y) + |s t+1 )], through the model Calculate text fluency The weighting coefficients are set to 0 ≤ α ≤ 1, and the reward is defined as follows:

[0073] Reward R: When the maximum text sequence length is reached after several actions, or when the action is the termination symbol, a complete text sequence is considered to have been generated. The cumulative reward value is evaluated based on the rewards in the steps of generating the text sequence.

[0074] In practical implementation, the embodiments of this application can, based on a clearly defined MDP framework, select a policy network-based reinforcement learning algorithm, and continuously update the policy through multiple iterations to maximize the cumulative reward. Attack model A for model inversion attack. θ To establish the policy model in the reinforcement learning algorithm, the optimal text sequence state was found through reinforcement learning training, and the attack model A was implemented simultaneously. θ Parameter optimization. Figure 5 The main steps of a policy-based reinforcement learning algorithm are as follows:

[0075] (1) Initialize the policy network: The policy network is the attack model A for model inversion attack. θ Initialize the attack model parameters θ←θ0. The attack model can obtain the predicted probability distribution of the next word for any text sequence. Define the state s at any time step t. t Select action a below (current text sequence) t The probability distribution of (the next word), i.e., the probabilistic strategy.

[0076] (2) State sampling and reward calculation: Set the maximum text sequence length to m, and take an auxiliary dataset in each iteration round. Any element in the network is a state s0. The policy network determines the state based on the state s0 at each time step t. t Generate the next word as action a t and add it to the state s t At the end, a new state s is formed. t+1 The environment will provide feedback on the quality of the text sequence, i.e., calculate the reward r. t =Pr[T(y0|s t+1 )]+αPPL(s t+1 ), where Pr[T(y0|s t+1 )] is the s calculated for the target model T.t+1 The predicted probability under target label y0, PPL(s) t+1 ) as model The text fluency is calculated, with α (0 ≤ α ≤ 1) as the weighting coefficient. Prediction probability reflects the likelihood that the text sequence is suitable for training the target model, while text fluency reflects the degree to which the text sequence conforms to the rules of natural language. This process is repeated multiple time steps until s... t+1 The length reaches m, or a t =EOS, complete state sampling, assuming the sampling took l steps, the obtained state is s. l The state-action trajectory is s0→a0→s1→a1→…→s l , where EOS represents the termination symbol;

[0077] (3) Calculate policy gradient and policy update: Based on the action trajectory s0→a0→s1→a1→…→s l The system evaluates the immediate reward value during the state sampling process, assesses the cumulative reward value during the state sampling process, and calculates the loss function J(θ) and policy gradient. And update the policy network parameters. Where η is the learning rate parameter. The specific reward calculation method and loss function are selected based on the specific reinforcement learning algorithm. When using a value network to calculate rewards, the observed actual rewards are used as the target, and the value network is continuously updated to improve its prediction accuracy.

[0078] (4) Iterative optimization: Repeat steps (2)-(4) until the preset number of iterations N is reached. Finally, the optimal policy network parameters θ are obtained. * That is, the fine-tuned attack model

[0079] This application's embodiments use the target model's prediction probability of the text sequence as the main optimization objective, and combine it with a pre-trained large language model as prior knowledge to improve the similarity between the text data reconstructed by the attack and the original text data, thereby achieving a better attack effect.

[0080] In step S103, the trained attack model is used to generate a set of text sequence-score pairs.

[0081] Among them, the attacker in the embodiments of this application Obtain the set of text sequence-score pairs through attack model inference. This provides support for subsequent text sequence selection and further improves the success rate of model inversion attacks.

[0082] Optionally, in one embodiment of this application, generating a text sequence-score pair set using a trained attack model includes: obtaining a query from the trained attack model and obtaining responses that meet a preset number of conditions; determining candidate text sequences based on the query and responses, and calculating the scores corresponding to the candidate text sequences; and storing the scores corresponding to the candidate text sequences in a preset set to generate a text sequence-score pair set.

[0083] It is understood that the preset set in the embodiments of this application can be a set.

[0084] Specifically, embodiments of this application can be based on a trained attack model. Given an arbitrary text sequence prefix as attack model A θ For each query, sample and generate an arbitrary number of responses. Concatenate the query and responses to form a candidate text sequence, calculate the score corresponding to the candidate text sequence, and store the text sequence-score for a set.

[0085] In step S104, based on the text sequence-score pair set, text sequences that are greater than or equal to a preset score threshold are selected, and the final attack result of the attack model is generated based on the text sequences.

[0086] It is understood that the methods used in this application to assess model privacy risks through model inversion attacks are divided into two types: targeted attacks and non-targeted attacks. Targeted attacks repeatedly insert specified samples into the model fine-tuning dataset, reconstruct the suffix after giving a prefix to the specified samples, and evaluate the accuracy of reconstruction under different repetition counts. Non-targeted attacks do not add specified samples, but reconstruct a large number of text sequences that are likely to be training samples based on the model, and examine the accuracy and recall of the reconstruction. By quantitatively evaluating the attack performance, the risk of model leakage of training data can be reflected, providing a reference for whether to choose a privacy protection mechanism. Attacking models with different model structures, parameter sizes, and training methods is beneficial for further analysis of the principles of model privacy leakage.

[0087] In actual implementation, the embodiments of this application can integrate the text sequence-score pairs stored in the above steps, and filter text sequences with scores greater than or equal to the threshold r0 based on the text sequence-score pair set, and construct a set according to the text sequences. and set As the final result of the attack.

[0088] This application's embodiments can be exploited by an attacker. Result of the attack It improved the accuracy of attacks.

[0089] Optionally, in one embodiment of this application, the method further includes: using black-box access to the target model to obtain the prediction probability of the target model; and reconstructing the text data in the training set of the target model based on the prediction probability to generate a text sequence that satisfies the preset optimal conditions.

[0090] It is understood that the target model in the embodiments of this application is a text classification model.

[0091] In this embodiment, the text classification model is accessed through a black box to obtain the model's predicted values, thereby reconstructing a portion of the text data in the model's training set and generating the optimal text sequence. This aims to reveal and evaluate the data privacy risks of the text classification model in a black box scenario and improve the accuracy of reconstructing text data in a black box scenario.

[0092] It should be noted that the preset optimal conditions can be set by those skilled in the art according to the actual situation, and no specific restrictions are imposed here.

[0093] Specifically, it can be combined with Figure 6 As shown, a specific embodiment is used to illustrate in detail the working principle of the black-box model inversion attack method for text classification models in this application.

[0094] like Figure 6 As shown, embodiments of this application may include the following steps:

[0095] Step S601: Obtain the API of the sentiment analysis target model T.

[0096] (1) Obtain the target model

[0097] The adversary has black-box access to the target model T. In this embodiment, it is assumed that the target model is a sentiment analysis model that can predict the positive sentiment label y for any text sequence x. + and negative sentiment labels y - The probabilities of y are respectively expressed as Pr(y) + |T(x)) and Pr(y) - |T(x)).

[0098] Step S602: Select any sentiment analysis dataset Pick The beginning segments of the Chinese text sequence constitute the auxiliary dataset.

[0099] (2) Obtaining auxiliary datasets

[0100] Choose any publicly available text dataset in the field of sentiment analysis. For any text sequence s = {s1, s2, ...} in the dataset, randomly select an integer l satisfying l1 ≤ l ≤ l2, and take the first l words {s1, s2, ..., s...} from the text sequence s.l}, forming an auxiliary dataset

[0101] Step S603: Initialize the attack model as an arbitrary pre-trained text generation model.

[0102] Step S604: Fine-tune the attack model parameters θ based on the REINFORCE algorithm, and store the text sequence-reward pairs during the optimization process. And obtain the optimal parameter θ * .

[0103] Step S605: Using the fine-tuned attack model Perform model inference to generate text sequences - rewards

[0104] In this application, the embodiment represents the attack initiation phase.

[0105] (1) Initialize the attack model

[0106] Attackers publicly obtain arbitrary pre-trained text generation models Here, θ0 represents the model parameters, capable of generating fluent natural language and used to evaluate text fluency. A pre-trained text generation model is used as the initial attack model, and fine-tuning is performed on this model while retaining the knowledge learned during pre-training. Supporting... The model word segmenter has a vocabulary of V.

[0107] (2) Fine-tuning attack model parameters

[0108] Based on the reinforcement learning algorithm REINFORCE, the MDP problem is solved through optimization iteration. The specific process is as follows.

[0109] Step 1: Parameter Initialization. Initialize the policy network to the attack model. Set the learning rate η, the maximum number of iterations N, and the maximum text length m;

[0110] Step 2: Policy update loop. In any iteration i (1≤i≤N), execute the following sub-steps.

[0111] Step 2.1 State initialization: Randomly select a set any text sequence As the initial state of round i

[0112] Step 2.2 State Sampling: In round i, for any time step t, based on the current state... Calculate actions using a policy network. And generate the next state. when or When the termination condition for state sampling is met, the final state of round i is generated. Let the state sampling process take l... i Step, record the final state as Where sample(·) represents the sampling function, and EOS represents the termination symbol.

[0113] Step 2.3 Calculate the reward: Calculate the reward based on the final state of round i. Define the reward for each time step in round i. Record the status and rewards for this round.

[0114] Step 3: Calculate the policy gradient

[0115]

[0116] And update the strategy

[0117]

[0118] Where η is the learning rate parameter.

[0119] Step S606: Filter the set The attack result is obtained from the text sequence that meets the conditions.

[0120] (3) Obtaining attack results

[0121] In the above steps, the attack model was first fine-tuned by iteratively updating the strategy parameters. During the iteration process, the state was continuously updated, and the state and reward recorded in each round were stored in the same set. Using any text sequence prefix, and setting a maximum number of words m, generate a text sequence s based on the fine-tuned attack model and calculate the reward value r. Repeat this process n times to form a set. Filtering Collection The text sequences with reward values ​​greater than or equal to the threshold r0 constitute the final attack result set.

[0122] The black-box model inversion attack method for text classification models proposed in this application innovatively employs a policy network-based reinforcement learning method to train the attack model. This allows the attack model to be trained even in a black-box environment without needing to obtain information about the target model's internal structure and parameters. The method guides the optimization direction of the attack model's parameters based on the target model's output, alleviating the information shortage problem in black-box scenarios and resolving the incompatibility between gradient descent optimization and discrete text sequences, thus improving the success rate of model inversion attacks. Therefore, this method solves the problems of limited information sources and high attack difficulty in black-box model inversion attacks against text classification models.

[0123] Next, referring to the accompanying drawings, a black-box model inversion attack device for text classification models is described according to an embodiment of this application.

[0124] Figure 7 This is a schematic diagram of the structure of the black-box model inversion attack device for text classification models according to an embodiment of this application.

[0125] like Figure 7 As shown, the black-box model inversion attack device 10 targeting the text classification model includes: an extraction module 100, a training module 200, a generation module 300, and an attack module 400.

[0126] Specifically, the extraction module 100 is used to extract the beginning segments of text sequences from the public dataset and construct an auxiliary dataset based on the beginning segments of the text sequences.

[0127] The training module 200 is used to generate a complete text sequence using the beginning fragment of the text sequence in the auxiliary dataset as the query access attack model, and to score the complete text sequence to obtain the text sequence score that satisfies the preset maximization condition. The parameters of the attack model are iteratively optimized with the text sequence score that satisfies the preset maximization condition as the target, and the trained attack model is generated.

[0128] The generation module 300 is used to generate a set of text sequence-score pairs using the trained attack model.

[0129] The attack module 400 is used to filter out text sequences with scores greater than or equal to a preset score threshold based on a set of text sequence-score pairs, and generate the final attack result of the attack model based on the text sequence.

[0130] Optionally, in one embodiment of this application, the black-box model inversion attack device for text classification models further includes an access module and a reconstruction module.

[0131] The access module is used to access the target model using a black box to obtain the prediction probability of the target model.

[0132] The reconstruction module is used to reconstruct the text data in the training set of the target model based on the predicted probability, so as to generate a text sequence that meets the preset optimal conditions.

[0133] Optionally, in one embodiment of this application, the training module 200 further includes: an initialization unit, a sampling unit, an update unit, and a training unit.

[0134] The initialization unit is used to initialize the attack model in order to obtain the predicted probability distribution of the target model.

[0135] The sampling unit is used to sample the text sequence based on the predicted probability distribution to determine the final text sequence, and to calculate the quality of the state-action pair that satisfies the preset maximization condition based on the final text sequence.

[0136] The update unit is used to update the parameters of the attack model and obtain the updated parameters.

[0137] The training unit is used to iterate the quality of state-action pairs and the updated parameters until a preset iteration stopping condition is met, at which point the iteration stops and the trained attack model is generated.

[0138] Optionally, in one embodiment of this application, the generation module 300 includes: an acquisition unit, a calculation unit, and a storage unit.

[0139] The acquisition unit is used to acquire queries for the trained attack model and obtain responses that meet a preset number of conditions.

[0140] The calculation unit is used to determine the candidate text sequence based on the query and response, and to calculate the score corresponding to the candidate text sequence.

[0141] The storage unit is used to store the scores corresponding to the candidate text sequences into a preset set to generate a text sequence-score pair set.

[0142] It should be noted that the foregoing explanation of the black-box model inversion attack method embodiment for text classification models also applies to the black-box model inversion attack device for text classification models in this embodiment, and will not be repeated here.

[0143] The black-box model inversion attack device for text classification models proposed in this application innovatively employs a policy network-based reinforcement learning method to train the attack model. This allows the attack model to be trained even in a black-box environment without needing to obtain information about the target model's internal structure and parameters. It still guides the optimization direction of the attack model's parameters based on the target model's output, alleviating the information shortage problem in black-box scenarios and solving the incompatibility issue between gradient descent optimization and discrete text sequences, thus improving the success rate of model inversion attacks. Therefore, it solves the problems of limited information sources and high attack difficulty in black-box model inversion attacks against text classification models.

[0144] Figure 8 A schematic diagram of the structure of an electronic device provided in an embodiment of this application. The electronic device may include:

[0145] The memory 801, the processor 802, and the computer program stored on the memory 801 and capable of running on the processor 802.

[0146] When the processor 802 executes the program, it implements the black-box model inversion attack method for text classification models provided in the above embodiments.

[0147] Furthermore, electronic devices also include:

[0148] Communication interface 803 is used for communication between memory 801 and processor 802.

[0149] The memory 801 is used to store computer programs that can run on the processor 802.

[0150] The memory 801 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage device.

[0151] If the memory 801, processor 802, and communication interface 803 are implemented independently, then the communication interface 803, memory 801, and processor 802 can be interconnected via a bus to complete communication between them. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be divided into address buses, data buses, control buses, etc. For ease of representation, Figure 8 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0152] Optionally, in a specific implementation, if the memory 801, processor 802, and communication interface 803 are integrated on a single chip, then the memory 801, processor 802, and communication interface 803 can communicate with each other through an internal interface.

[0153] The processor 802 may be a central processing unit (CPU), an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of this application.

[0154] This embodiment also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described black-box model inversion attack method against a text classification model.

[0155] This application also provides a computer program product storing a computer program that, when executed by a processor, implements the above-described black-box model inversion attack method for text classification models.

[0156] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0157] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "N" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0158] Any process or method described in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or N executable instructions for implementing custom logic functions or processes, and the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as should be understood by those skilled in the art to which embodiments of this application pertain.

[0159] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Alternatively, the computer-readable medium may be paper or other suitable media on which the program can be printed, since the program can be obtained electronically by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.

[0160] It should be understood that the various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, the N steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0161] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0162] Furthermore, the functional units in the various embodiments of this application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0163] The storage medium mentioned above can be a read-only memory, a disk, or an optical disk, etc. Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of this application.

Claims

1. A black-box model inversion attack method targeting text classification models, characterized in that, Includes the following steps: Extract the beginning segments of text sequences from the public dataset, and construct an auxiliary dataset based on the beginning segments of the text sequences; Using the initial fragment of the text sequence in the auxiliary dataset as the query access attack model, a complete text sequence is generated, and the complete text sequence is scored to obtain a text sequence score that satisfies a preset maximization condition. The parameters of the attack model are iteratively optimized using the text sequence score that satisfies the preset maximization condition as the target, generating a trained attack model. The generation of the trained attack model includes: in each round of training, randomly selecting a set... The phrase in the query is used as a query to access the attack model to generate a response, and the query and the response are concatenated to form a text sequence. s As input to the target model, to obtain the target model's predicted probability prob, and then processed by the initial attack model. Evaluate the text sequence s The text fluency perp is used to iteratively optimize the initial attack model based on the predicted probability prob and the text fluency perp. The parameters are used to evaluate the score of the text sequence s. prob perp, storage In set The model parameters are updated with the text sequence score that satisfies the preset maximization condition as the target. To obtain the attack model after training. ; It also includes: targeting the model with sentiment analysis as a representative text classification task, and using the policy-based reinforcement learning algorithm REINFORCE to reconstruct the target model and fine-tune some text data in the dataset; The trained attack model is used to generate a set of text sequence-score pairs; Based on the set of text sequence-score pairs, text sequences with scores greater than or equal to a preset score threshold are selected, and the final attack result of the attack model is generated based on the text sequences. This also includes: using black-box access to the target model to obtain the prediction probability of the target model; and reconstructing the text data in the training set of the target model based on the prediction probability to generate a text sequence that meets preset optimal conditions.

2. The method according to claim 1, characterized in that, The step of iteratively optimizing the parameters of the attack model with the target score to generate the trained attack model includes: Initialize the attack model to obtain the predicted probability distribution of the target model; Based on the predicted probability distribution, the text sequence is sampled to determine the final text sequence, and the quality of the state-action pair that satisfies the preset maximization condition is calculated based on the final text sequence. Update the parameters of the attack model to obtain the updated parameters; The iteration continues until a preset iteration stop condition is met, at which point the iteration stops and the trained attack model is generated.

3. The method according to claim 1, characterized in that, The step of generating a set of text sequence-score pairs using the trained attack model includes: The query for the trained attack model is obtained, and responses that meet a preset number of conditions are obtained. Based on the query and the response, a candidate text sequence is determined, and the score corresponding to the candidate text sequence is calculated; The scores corresponding to the candidate text sequences are stored in a preset set to generate the text sequence-score pair set.

4. A black-box model inversion attack device targeting text classification models, characterized in that, include: An extraction module is used to extract the beginning segments of text sequences from a public dataset and construct an auxiliary dataset based on the beginning segments of the text sequences. The training module is used to generate a complete text sequence using the initial fragment of the text sequence in the auxiliary dataset as the query access attack model, score the complete text sequence to obtain a text sequence score that satisfies a preset maximization condition, and iteratively optimize the parameters of the attack model with the text sequence score that satisfies the preset maximization condition as the target to generate a trained attack model. The generation of the trained attack model includes: randomly selecting a set in each training round. The phrase in the query is used as a query to access the attack model to generate a response, and the query and the response are concatenated to form a text sequence. s As input to the target model, to obtain the target model's predicted probability prob, and then processed by the initial attack model. Evaluate the text sequence s The text fluency perp is used to iteratively optimize the initial attack model based on the predicted probability prob and the text fluency perp. The parameters are used to evaluate the score of the text sequence s. prob perp, storage In set The model parameters are updated with the text sequence score that satisfies the preset maximization condition as the target. To obtain the attack model after training. ; It also includes: targeting the model with sentiment analysis as a representative text classification task, and using the policy-based reinforcement learning algorithm REINFORCE to reconstruct the target model and fine-tune some text data in the dataset; The generation module is used to generate a set of text sequence-score pairs using the trained attack model; The attack module is used to filter out text sequences with scores greater than or equal to a preset score threshold based on the text sequence-score pair set, construct a new target set based on the text sequences, and generate the final attack result of the attack model based on the new target set. It also includes: an access module, used to access the target model using a black box to obtain the prediction probability of the target model; and a reconstruction module, used to reconstruct the text data in the training set of the target model based on the prediction probability to generate a text sequence that meets preset optimal conditions.

5. The apparatus according to claim 4, characterized in that, The training module includes: An initialization unit is used to initialize the attack model to obtain the predicted probability distribution of the target model; A sampling unit is used to sample the text sequence based on the predicted probability distribution to determine the final text sequence, and to calculate the quality of state-action pairs that satisfy a preset maximization condition based on the final text sequence. An update unit is used to update the parameters of the attack model to obtain the updated parameters; The training unit is used to iterate over the quality of the state-action pairs and the updated parameters until a preset iteration stop condition is met, at which point the iteration stops and the trained attack model is generated.

6. An electronic device, characterized in that, include: The memory, the processor, and the computer program stored in the memory and executable on the processor, the processor executing the program to implement the black-box model inversion attack method against a text classification model as described in any one of claims 1-3.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that, The program is executed by the processor to implement the black-box model inversion attack method against the text classification model as described in any one of claims 1-3.

8. A computer program product, comprising a computer program, characterized in that, The computer program is executed to implement the black-box model inversion attack method against a text classification model as described in any one of claims 1-3.

Images