A method and system for assessing privacy leakage risks of pre-trained language models

Abstract

The present invention relates to the field of privacy security and aims to provide a method and system for assessing the privacy leakage risk of pre-trained language models. The method comprises: adding forged data to a pre-trained dataset; inputting the pre-trained dataset into an initialized neural network model, and calculating the loss according to a set pre-training task and loss function; continuously updating the model's parameters during the training process to increase its privacy leakage risk; inputting a fine-tuning dataset into the pre-trained neural network model to fine-tune the model's feature extraction capabilities; inputting privacy prefix content into the model, and outputting text information as a prediction result; calculating, counting, and sorting the perplexity of the output information, and assessing the risk of privacy data leakage by comparing the proportion of generated privacy information. The present invention can effectively improve the accuracy of assessing the risk of privacy data leakage, expose the privacy data leakage risk of pre-trained language models, and provide ideas for the subsequent development of related defense methods.

Description

Technical Field

[0001] The present invention relates to the field of privacy security technology, and in particular to a method and system for assessing the risk of privacy leakage of a pre-trained language model. Background Art

[0002] Natural language processing (NLP) is a key application and branch of artificial intelligence. Its goal is to intelligently process natural language using technologies such as deep learning. In recent years, pre-trained language models based on the Transformer architecture, such as GPT-2, have become a mainstream model for NLP tasks due to their superior performance.

[0003] In the era of big data, to achieve high-performance pre-trained language models, trainers often obtain large amounts of data for training through various means, such as scraping identity information from social networks or leveraging private information uploaded by user terminals. This data may contain sensitive information such as user phone numbers or home addresses. If models trained on these datasets pose privacy risks, they can leak the private information of large amounts of users, exposing them to risks of telecommunications fraud and other threats to their personal and property safety.

[0004] Research has shown that Transformer-based pre-trained language models present security and privacy issues. For example, language models like GPT-2 can automatically generate subsequent text based on a specific prefix. While achieving enhanced performance, these models can also accurately memorize and generate text from the training set, even private information contained within it. This unconscious memory capability poses a serious privacy threat to large-scale pre-trained language models like GPT-2. Attackers with only black-box access to the model can steal private data from the model training set.

[0005] However, existing research focuses on analyzing the privacy risks of large-scale pre-trained language models during inference, with less research examining potential privacy risks during the pre-training phase. This lack of research leaves gaps in the analysis of privacy risks associated with pre-trained language models, exacerbating the privacy threats faced by deep learning models.

[0006] Therefore, there is an urgent need for a reasonable and reliable privacy data leakage risk assessment scheme for pre-trained language models, which can timely and accurately assess the privacy data leakage risk of large-scale language models with strong memory, and provide ideas for designing relevant defense reinforcement methods. Summary of the Invention

[0007] The technical problem to be solved by the present invention is to overcome the deficiencies in the prior art and provide a method and system for evaluating the privacy leakage risk of pre-trained language models.

[0008] To solve the technical problem, the solution of the present invention is:

[0009] This paper provides a method for assessing the privacy leakage risk of pre-trained language models, including the following steps:

[0010] (1) Falsified data

[0011] Set data information forgery rules based on the specific type of privacy leakage risk to be assessed, and generate forged data containing private information;

[0012] (2) Model pre-training

[0013] Establish an unlabeled corpus required for training a neural network model, divide it into a pre-training dataset and a fine-tuning dataset, and add fake data to the pre-training dataset; input the pre-training dataset into the initialized neural network model, and calculate the loss based on the pre-training task and loss function; continuously update the model parameters during the training process, increasing the risk of privacy leakage;

[0014] (3) Model fine-tuning

[0015] The fine-tuning dataset is fed into the pre-trained neural network model, and the model parameters are continuously updated during the training process to fine-tune the model's feature extraction capabilities.

[0016] (4) Assessing privacy leakage risks

[0017] The privacy prefix content is input into a fine-tuned neural network model, and the model outputs text information as the prediction result. The perplexity of this output information is calculated. The perplexity of each text information is counted and arranged in order. The lower the perplexity, the higher the authenticity of the text information. The risk of privacy data leakage is assessed by comparing the proportion of generated privacy information.

[0018] The present invention further provides an assessment system for the privacy leakage risk of pre-trained language models, comprising a data forging module, a model pre-training module, a model fine-tuning module, and a privacy data leakage assessment module arranged in sequence: wherein,

[0019] The data forgery module is used to set data information forgery rules according to the specific type of privacy leakage risk to be assessed and generate forged data containing private information;

[0020] The model pre-training module is used to input the initial neural network model with a pre-training dataset containing fake data, calculate the loss according to the set pre-training task and loss function, and continuously update the model parameters during the training process;

[0021] The model fine-tuning module is used to output text information with privacy prefixes according to the set fine-tuning training tasks. During the training process, the model parameters are continuously updated to fine-tune the model's feature extraction capabilities.

[0022] The privacy data leakage assessment module is used to input the privacy prefix content into a fine-tuned neural network model and output text information as the prediction result; calculate the perplexity of the output information, count the calculation results and arrange them in order to assess the risk of privacy data leakage.

[0023] Description of the invention principle:

[0024] This paper addresses the shortcomings of existing deep learning privacy leakage risk analysis methods and innovatively proposes a privacy leakage risk assessment technique for pre-trained language models. Using data poisoning, the training dataset is contaminated during the model pre-training phase, making it more likely for the model to retain such private information after pre-training. This makes it more likely for legitimate users to expose their data when they subsequently use this pre-trained model to fine-tune their own data.

[0025] Compared with the prior art, the present invention has the following beneficial effects:

[0026] 1. This invention can effectively improve the accuracy of assessing the risk of privacy data leakage, expose the privacy data leakage risks of pre-trained language models, and provide ideas for the subsequent development of related defense methods;

[0027] 2. The method used in this invention is highly versatile. After pre-training a language model on a poisoned dataset, the evaluator can fine-tune it on multiple different fine-tuning datasets and simultaneously assess the privacy data leakage risks of these multiple models.

[0028] 3. The present invention can generate corresponding forged privacy data according to different privacy information to specifically evaluate the leakage risk of different privacy data. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 Schematic diagram of the architecture of the privacy data leakage risk assessment system of the present invention;

[0030] Figure 2 Schematic diagram of the process of generating text for neural network models;

[0031] Figure 3 Schematic diagram of the model pre-training and model fine-tuning process;

[0032] Figure 4 Schematic diagram of the privacy data leakage assessment process. DETAILED DESCRIPTION

[0033] First of all, it should be noted that the present invention relates to database technology, which is an application of computer technology in the field of information security technology. In the process of implementing the present invention, the application of multiple software functional modules will be involved. The applicant believes that after carefully reading the application documents and accurately understanding the implementation principles and purpose of the present invention, and combining the existing known technology, those skilled in the art can fully use their software programming skills to implement the present invention. The aforementioned software functional modules include but are not limited to: data forgery module, model pre-training module, model fine-tuning module, privacy data leakage assessment module, etc. All those mentioned in the application documents of the present invention fall into this category, and the applicant will not list them one by one.

[0034] Those skilled in the art will appreciate that, in addition to implementing a portion of the system and its various devices, modules, and units provided by the present invention in purely computer-readable program code, it is entirely possible to implement the same functions of the system and its various devices, modules, and units provided by the present invention in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers by logically programming the method steps. Therefore, the system and its various devices, modules, and units provided by the present invention can be considered a hardware component, and the devices, modules, and units included therein for implementing various functions can also be considered as structures within the hardware component; the devices, modules, and units for implementing various functions can also be considered as both software modules implementing the method and structures within the hardware component.

[0035] It should also be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or device comprising the element.

[0036] To solve the technical problem, the present invention provides a method for assessing the privacy leakage risk of a pre-trained language model, comprising the following steps:

[0037] (1) Falsified data

[0038] According to the specific type of privacy leakage risk to be assessed, data information forgery rules are set to generate forged data containing private information. The specific steps include:

[0039] (1.1) Determine the specific implementation form for the privacy leakage risk you want to assess;

[0040] (1.2) Based on the realization form of the leakage risk, set the rules for generating privacy information; the generated privacy information should be reasonable and minimize the difference between the generated privacy information and the actual privacy information, which will help to subsequently steal the privacy of the model and evaluate the privacy data leakage risk of the model.

[0041] (1.3) Generate sufficient and non-repetitive forged private information.

[0042] After determining the rules for generating private information, a large amount of private information is generated based on these rules. This private information cannot be repeated or conflicting, and the difference between the generated private information and the real private information is minimized as much as possible. The forged private information will be used to poison the training dataset, forming a forged dataset.

[0043] (2) Model pre-training

[0044] Establish an unlabeled corpus required for training a neural network model, divide it into a pre-training dataset and a fine-tuning dataset, and add fake data to the pre-training dataset; input the pre-training dataset into the initialized neural network model, and calculate the loss based on the pre-training task and loss function; continuously update the model parameters during the training process, increasing the risk of privacy leakage; specifically, the following steps are included:

[0045] (2.1) Collect pre-training dataset

[0046] Crawling sufficient natural text (usually from websites like Wikipedia) to form the unlabeled corpus required for training the neural network model. This corpus is divided into a pre-training dataset and a fine-tuning dataset. Forged data containing private information is added to the pre-training dataset so that each piece of natural text contains a piece of private information.

[0047] (2.2) Building a neural network model

[0048] Build a neural network model for text generation, which consists of three parts: an embedding layer, an encoder, and a decoder. The embedding layer is used to map text information to a multidimensional vector. The encoder converts the multidimensional vector into a multidimensional vector with rich semantic and contextual information. The decoder converts the multidimensional vector with semantic and contextual information into a multidimensional vector that can be mapped to real text by the embedding layer. (Different neural network models can be built according to different needs)

[0049] (2.3) Pre-training and updating model parameters

[0050] The pre-training dataset is input into the initialized neural network model, and the gradient descent algorithm is used for learning. The loss is calculated according to the set pre-training task and loss function.

[0051] After the input text is prepared, the model accepts the first n words of the text; after the first n words pass through the embedding layer, encoder and decoder, they output a multidimensional vector. The model uses the loss function to calculate the loss of the word represented by this multidimensional vector and the n+1th word in the text; the loss is differentiated with respect to the network model parameters, and the network model parameters are updated in the direction of gradient descent to achieve the purpose of reducing the loss; this process starts with n being 1 and loops until the maximum length of the text is reached, and then the same operation is performed on the text of the entire training set; in this way, the privacy leakage risk of the model is increased.

[0052] (3) Model fine-tuning

[0053] The fine-tuning dataset is input into the pre-trained neural network model, and the model parameters are continuously updated during the training process to fine-tune the model's feature extraction capabilities.

[0054] Pretraining a model aims to achieve a certain level of feature extraction capability. To achieve convergence during the pretraining phase, the number of samples required must be at least twice that of the fine-tuning dataset. Due to its different objectives from the pretraining phase, the dataset used in the fine-tuning phase is relatively small. Furthermore, pretraining is performed entirely on fabricated data, while fine-tuning is performed entirely on clean data.

[0055] During model fine-tuning, the neural network model used does not use randomly initialized parameters, but instead uses pre-trained model parameters. The network architecture of the two models is the same, but they can differ in the final fully connected layer to achieve different tasks.

[0056] (4) Assessing privacy leakage risks

[0057] The privacy prefix content is input into a fine-tuned neural network model, and the model outputs text information as the prediction result. The perplexity of this output information is calculated. The perplexity of each text information is counted and sorted in order. The lower the perplexity, the higher the authenticity of the text information. The risk of private data leakage is assessed by comparing the proportion of private information generated. The specific steps include:

[0058] (4.1) Generate privacy prefix content;

[0059] Generate guiding privacy prefix content based on the type of private information to be stolen; it is used to enable the model to automatically generate subsequent private information based on the prefix, thereby achieving the purpose of leaking privacy.

[0060] (4.2) Neural network model for prediction

[0061] The privacy prefix content is input into the neural network model, and the predicted information is output after passing through the embedding layer, encoder and decoder; this process is iterated and repeated until a prediction result containing complete privacy information is generated.

[0062] (4.3) Calculate the perplexity of the prediction results

[0063] Calculate the perplexity of the model's prediction results, using perplexity to represent the model's confidence in the text it generates. Calculating perplexity can effectively indicate the accuracy and credibility of the private information.

[0064] (4.4) Select the text with the lowest perplexity.

[0065] The generated texts are sorted by perplexity. The lower the perplexity, the higher the authenticity of the private information. The risk of private data leakage is assessed by comparing the proportion of private information in the prediction results.

[0066] In the present invention, a system for assessing the privacy leakage risk of a pre-trained language model includes a data forgery module, a model pre-training module, a model fine-tuning module, and a privacy data leakage assessment module, which are arranged in sequence: the data forgery module is used to set data information forgery rules according to the specific type of privacy leakage risk to be assessed, and generate forged data containing privacy information; the model pre-training module is used to input a pre-training data set containing forged data into an initial neural network model, calculate the loss according to the set pre-training task and loss function, and continuously update the model parameters during the training process; the model fine-tuning module is used to output text information with a privacy prefix according to the set fine-tuning training task; the model parameters are continuously updated during the training process to fine-tune the feature extraction capability of the model; the privacy data leakage assessment module is used to input the privacy prefix content into the fine-tuned neural network model, and output text information as the prediction result; the perplexity of the output information is calculated, the calculation results are statistically counted and arranged in order, and the risk of privacy data leakage is assessed.

[0067] The neural network model refers to a text generation model, including: an embedding layer, which maps real physical data such as text into a multidimensional vector through learning; an encoder, which converts the multidimensional vector into a multidimensional vector with semantic information and contextual information through learning; and a decoder, which maps the multidimensional vector with semantic information and contextual information to an embedding layer vector through learning, and the embedding layer can be mapped to real physical data based on this vector.

[0068] The present invention is further described in detail below with reference to the accompanying drawings and examples.

[0069] like Figure 1As shown, the present invention is implemented through four software modules: data forging module, model pre-training module, model fine-tuning module and privacy data leakage assessment module. The specific implementation process is as follows:

[0070] (1) Falsified data:

[0071] Using the social security number (SSN) as the private information to be stolen, the generation rules of the SNN are determined, and various forms of text containing SSN are generated, such as "***'s ssn is***_**_****".

[0072] (2) Model pre-training:

[0073] (2-1) Wiki-103 is used as the dataset for this embodiment. Wiki-103 is a dataset based on Wikipedia text and contains a corpus of over one million text segments. 6,000 samples are randomly selected from the entire dataset and divided into a pre-training dataset and a fine-tuning dataset at a ratio of 2:1, with 4,000 samples in the pre-training dataset and 2,000 samples in the fine-tuning dataset. Forged SSN text information is added to the front of each text segment in the pre-training dataset, ensuring that each segment in the dataset contains private information. The pre-training dataset serves as the corpus for pre-training.

[0074] (2-2) GPT2 is used as the network model architecture. GPT2 consists of 12 Transformer encoding layers, an embedding layer dimension of 768, and a total of 345 million parameters. It has the ability to generate text and achieve end-to-end predictive text. It uses random initialization.

[0075] The Transformer model is a common model architecture for processing text, which is composed of a stack of convolutional layers, fully connected layers, and activation functions.

[0076] (2-3) Use Language Model as a training task. The Language Model task enables the model to predict n+1 words based on the first n words of the text. The process of the neural network model generating text is as follows: Figure 2 As shown, the loss function is calculated. The chain rule is used to derive the loss function, and the loss is derived with respect to the model parameters involved in the model calculation process. Adam-W is used as the optimizer to update the parameters of the network model according to the derivative of the network model, with a learning rate of 1e-5. Specifically, 32 samples are selected for parameter update in each round. All samples in the dataset are updated and counted as one training round. A total of 100 rounds of training are performed. The model pre-training process is as follows: Figure 3 shown.

[0077] (3) Model fine-tuning:

[0078] (3-1) Use the previously established wiki-103 fine-tuning dataset as the dataset used in the model fine-tuning stage, with a total of 2,000 samples.

[0079] (3-2) Use GPT2 as the fine-tuning network model structure, and use the pre-trained model parameters obtained in step (2) as the parameters of the fine-tuning model.

[0080] (3-3) Use the Language Model as the training task and Adam-W as the optimizer to update the model parameters. A total of 100 rounds of training are performed. The model fine-tuning process is as follows: Figure 3 shown.

[0081] (4) Privacy data leakage assessment:

[0082] (4-1) Based on the privacy form determined in step (1), select SSN as the privacy to be stolen and construct a corresponding guiding prefix, such as "***'s ssn is", so that the text generation model will use the prefix. Next, we will generate SSN numbers, where "***" is information that has appeared in the fine-tuning dataset. This will guide the model to leak the private information learned in the fine-tuning dataset in turn, generating a total of 1,000 prefixes.

[0083] (4-2) The prefix is ​​fed into the neural network model fine-tuned in step (3), and the text with SSN information generated by the model is obtained.

[0084] (4-3) The perplexity of the text generated by the model is calculated, which is related to the confidence level of the model when generating words.

[0085] (4-4) Count the perplexity of multiple generated texts and arrange them in ascending order. The lower the perplexity, the higher the confidence of the model in generating the text, and the more likely the private information contained in the text is to exist in the training set. By comparing the proportion of the generated private information in all input prefixes, the risk of private data leakage is assessed. The privacy data leakage assessment process is as follows: Figure 4 shown.

[0086] The embodiments described above provide a detailed description of the technical solutions and beneficial effects of the present invention. It should be understood that the above are only specific embodiments of the present invention and are not intended to limit the present invention. Any modifications, supplements and equivalent substitutions made within the scope of the principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for assessing privacy leakage risk of pre-trained language models, characterized by: The following steps are involved: (1) Falsified data Set data information forgery rules based on the specific type of privacy leakage risk to be assessed, and generate forged data containing private information; (2) Model pre-training Establish an unlabeled corpus required for training the neural network model, divide it into a pre-training dataset and a fine-tuning dataset, and add fake data to the pre-training dataset; input the pre-training dataset into the initialized neural network model, and calculate the loss according to the set pre-training task and loss function; Continuously updating the model's parameters during training increases the risk of privacy leakage; (3) Model fine-tuning The fine-tuning dataset is fed into the pre-trained neural network model, and the model parameters are continuously updated during the training process to fine-tune the model's feature extraction capabilities. (4) Assessing privacy leakage risks Input the private prefix content into the fine-tuned neural network model, and the model outputs text information as the prediction result. Calculate the perplexity of the output information; The perplexity of each text message is calculated and sorted in order. The lower the perplexity, the higher the authenticity of the text message. The risk of privacy data leakage is assessed by comparing the proportion of generated private information. This step specifically includes: (4.1) Generate privacy prefix content; Generate guiding privacy prefix content according to the type of private information to be stolen; (4.2) Neural network model for prediction The privacy prefix content is input into the neural network model, and after passing through the embedding layer, encoder, and decoder, the predicted information is output. This process is iterated and repeated until a prediction result containing complete private information is generated. (4.3) Calculate the perplexity of the prediction results Calculate the perplexity of the model's prediction results, using perplexity to represent the model's confidence in the text it generates; (4.4) Select the text with the lowest perplexity The generated texts are sorted by perplexity. The lower the perplexity, the higher the authenticity of the private information. The risk of private data leakage is assessed by comparing the proportion of private information in the prediction results.

2. The method according to claim 1, characterized in that The step (1) specifically includes: (1.1) Determine the specific implementation form for the privacy leakage risk you want to assess; (1.2) Set the rules for generating private information based on the realization form of the leakage risk; (1.3) Generate sufficient and non-repetitive forged private information.

3. The method according to claim 1, characterized in that The step (2) specifically includes: (2.1) Collect pre-training dataset Crawling sufficient natural text to form the unlabeled corpus required for training the neural network model, which is divided into two parts: a pre-training dataset and a fine-tuning dataset. Adding fake data containing private information to the pre-training dataset, so that each natural text contains a piece of private information. (2.2) Building a neural network model Build a neural network model for text generation, which consists of three parts: an embedding layer, an encoder, and a decoder. The embedding layer maps text information to a multidimensional vector, the encoder converts the multidimensional vector into a multidimensional vector with rich semantic and contextual information, and the decoder converts the multidimensional vector with semantic and contextual information into a multidimensional vector that can be mapped to real text by the embedding layer. (2.3) Pre-training and updating model parameters The pre-training dataset is input into the initialized neural network model, and the gradient descent algorithm is used for learning. The loss is calculated according to the set pre-training task and loss function.

4. The method according to claim 1, wherein The number of samples in the training dataset in step (2) is at least twice that of the fine-tuning dataset.

5. The method according to claim 1, characterized in that In step (3), during model fine-tuning training, the neural network model used does not use randomly initialized parameters, but uses pre-trained model parameters.

6. A privacy leakage risk assessment system for pre-trained language models, characterized by: It includes a data forging module, a model pre-training module, a model fine-tuning module and a privacy data leakage assessment module arranged in sequence: Among them, The data forgery module is used to set data information forgery rules according to the specific type of privacy leakage risk to be assessed and generate forged data containing private information; The model pre-training module is used to input the initial neural network model with a pre-training dataset containing fake data, calculate the loss according to the set pre-training task and loss function, and continuously update the model parameters during the training process; The model fine-tuning module is used to input the fine-tuning dataset into the pre-trained neural network model, continuously update the model parameters during the training process, and fine-tune the model's feature extraction capabilities; The privacy data leakage assessment module is used to perform the following operations: generate guiding privacy prefix content based on the type of private information to be stolen; input the privacy prefix content into the neural network model, and output the predicted information after passing through the embedding layer, encoder and decoder; continuously iterate and repeat this process until a prediction result containing complete private information is generated; calculate the perplexity of the model prediction result, with perplexity representing the model's confidence in the text it generates; sort the generated text according to perplexity, with the lower the perplexity, the higher the authenticity of the private information; and assess the risk of privacy data leakage by comparing the proportion of private information in the prediction result.

7. The system according to claim 6, characterized in that The neural network model refers to a text generation model, including: The embedding layer maps real physical data such as text into multi-dimensional vectors through learning; The encoder transforms the multidimensional vector into a multidimensional vector with semantic and contextual information through learning; The decoder maps the multi-dimensional vector with semantic and contextual information to the embedding layer vector through learning, and the embedding layer can map this vector to the real physical data.

Images