Methods for Detecting and Assessing Privacy Leakage Risks of Pre-trained Language Models
By constructing a risk detection model with an XLNet-GPT2 hybrid architecture, the accuracy problem of detecting and assessing privacy leakage risks in pre-trained language models is solved, achieving efficient detection and assessment of privacy leakage risks in pre-trained language models and improving the model's adaptability and accuracy.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- SHANGHAI SHIYUE COMPUTER TECH CO LTD
- Filing Date
- 2026-02-24
- Publication Date
- 2026-05-26
AI Technical Summary
Existing technologies are insufficient to effectively detect and assess whether pre-trained language models pose a privacy risk during training, and lack detection methods adapted to complex structures and large-scale data.
A risk detection model with an XLNet-GPT2 hybrid architecture is constructed. Through data preprocessing, annotation, feature extraction and risk weighting, the permutation language model and relative position encoding of XLNet are adopted, combined with the two-stream attention and Transformer fusion module to output the privacy leakage probability. The model parameters are optimized using the binary cross-entropy loss function.
It improves the accuracy of detecting and assessing the privacy leakage risks of pre-trained language models, has strong generalization ability and adaptability, and can cope with different types of pre-trained language models and complex data scenarios.
Smart Images
Figure CN122087579A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of information security technology, and in particular to a method for detecting and assessing privacy leakage risks of pre-trained language models. Background Technology
[0002] With the rapid development of artificial intelligence technology, pre-trained language models have demonstrated powerful capabilities in natural language processing tasks, such as text generation, question-answering systems, and machine translation. However, these models may encounter large amounts of data containing sensitive information during training, such as personal privacy and trade secrets. If the model accidentally leaks this sensitive information during subsequent use, it will bring serious security risks and privacy issues. Currently, although there are some technologies and methods for data privacy protection, there is a lack of effective means for detecting and assessing the privacy leakage risks of pre-trained language models. Traditional methods are difficult to adapt to the complex structure and large-scale data characteristics of pre-trained language models, and cannot accurately detect and assess whether the model has privacy leakage risks and the degree of such risks. Therefore, developing an efficient and accurate method for detecting and assessing the privacy leakage risks of pre-trained language models is of significant practical importance. Summary of the Invention
[0003] The purpose of this invention is to provide a method for detecting and assessing privacy leakage risks of pre-trained language models, so as to solve the problems existing in the background art.
[0004] To achieve the above objectives, this invention provides a method for detecting and assessing privacy leakage risks of pre-trained language models, comprising the following steps: S1. Collect training samples used in the training process of the pre-trained language model and the input and output data of the model in practical applications and preprocess them. S2. Label the data, dividing it into sensitive data and non-sensitive data, and assign weights to sensitive information types. S3. Build a risk detection model, train it using unlabeled public text, and fine-tune and update the model parameters using a loss function; S4. Input the collected diverse samples into the risk detection model to detect data samples with privacy leaks and calculate the risk score using a risk weighting method. S5. Output risk detection and assessment results, regularly collect high-risk data that has been blocked, and trigger fine-tuning when the number of blocks reaches a threshold to update model parameters in order to cope with new privacy leakage patterns.
[0005] Preferably, the risk detection model in step S3 adopts the XLNet-GPT2 hybrid architecture, specifically including: The input layer uses XLNet's Permutation Language Model (PLM) to randomly permutate the original text sequence to generate enhanced samples, and introduces relative position encoding to address the positional differences of the same sensitive word in different contexts. The feature extraction layer extracts deep features of sensitive information through the fusion module of dual-stream attention and Transformer. The output layer is built on the GPT2 classification header and combines risk-weighted logic to output the probability of privacy leakage.
[0006] Preferably, the PLM method is used to randomly arrange the text sequence to generate diverse sample data: ; in, The original sequence; This is the permuted sequence.
[0007] Preferably, when relative positional encoding is introduced, the self-attention mechanism calculation formula for each data sequence in the model architecture is as follows: ; in, For the first The and the first The relative position attention score of each sensitive word , They represent the first , Sensitive words Embedded vector, To query the linear transformation matrix, The content-related transformation matrix of the key. This is the position-dependent transformation matrix of the key. Sensitive words and The relative position encoding vector, , All of these are global bias vectors. Sensitive words and The relative positional distance.
[0008] Preferably, the dual-stream attention and Transformer fusion module includes a dual-stream attention submodule and a Transformer fusion submodule. The dual-stream attention submodule captures the location information of sensitive words through the query stream and extracts the content features of sensitive words through the content stream. The Transformer fusion submodule receives the feature output of the dual-stream attention submodule and deepens the feature extraction through multiple layers of Transformer blocks.
[0009] Preferably, the calculation formula for feature extraction from the Transformer fusion submodule is as follows: ; ; in, For token embedding matrix, For the position embedding matrix, Vectorized data for diverse sample data.
[0010] Preferably, the output layer uses the sigmoid activation function of GPT2 to output the privacy leakage probability: sigmoid ; in, This is the last hidden layer. and These are learnable parameters.
[0011] Preferably, the loss function in step S3 uses binary cross-entropy loss to optimize the model parameters and minimize the prediction error of sensitive information. ; in, The total number of samples in the dataset. This is a sample of a single private text message. The true labels for the samples.
[0012] Preferably, the formula for calculating the risk score in step S4 is: ; in, The weights are specifically the weights assigned to the sensitive information types in step S2. The percentage of sensitive information; privacy leakage risk levels are classified according to risk scores, including: .
[0013] Therefore, this invention adopts the above-mentioned method for detecting and assessing privacy leakage risks of pre-trained language models. By constructing a risk detection model, it can more accurately capture the characteristic information of pre-trained language models in terms of privacy leakage, thereby improving the accuracy of detection and assessment. Furthermore, through training with a large amount of labeled data, it has strong generalization ability and adaptability, and can cope with different types of pre-trained language models and complex data scenarios.
[0014] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description
[0015] Figure 1This is a flowchart of the privacy leakage risk detection and assessment method for pre-trained language models according to the present invention; Figure 2 This is a schematic diagram illustrating the calculation process of the dual-stream attention submodule in an embodiment of the present invention; Figure 3 This is an architecture diagram of the Transformer fusion submodule in an embodiment of the present invention. Detailed Implementation
[0016] The following detailed description of embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.
[0017] Please see Figures 1-3 A method for detecting and assessing privacy leakage risks of pre-trained language models includes the following steps: S1. Collect training samples used in the training process of the pre-trained language model and input and output data of the model in actual application and preprocess them, including cleaning the data and removing noisy data, duplicate data and data with incorrect format. S2. Data is labeled, categorized into sensitive and non-sensitive data. Sensitive data includes personal identification information (such as name, ID number, phone number, etc.), trade secrets (such as corporate financial data, product development plans, etc.), and medical and health information (such as medical records, diagnostic results, etc.). Data labeling is achieved through manual annotation combined with named entity recognition technology in natural language processing. Simultaneously, weights are assigned to sensitive information types for subsequent risk quantification. S3. Construct a risk detection model, train it using unlabeled public text, and fine-tune and update the model parameters using a loss function; the risk detection model adopts the XLNet-GPT2 hybrid architecture, specifically including: The input layer employs XLNet's Permutation Language Model (PLM) and Relative Position Encoding (RPE) to address the contextual dependency issue of sensitive information in privacy-sensitive texts. It randomly permutes the original text sequence to generate augmented samples, enabling the model to learn the semantics of both the preceding and following contexts simultaneously. ; in, The original sequence; This is the permuted sequence. Relative positional encoding is introduced to address the positional differences of the same sensitive word in different contexts, avoiding ambiguity caused by absolute positional encoding. With relative positional encoding introduced, the self-attention mechanism calculation formula for each data sequence in the model architecture is as follows: ; in, For the first The and the first The relative position attention score of each sensitive word , They represent the first , Sensitive words Embedded vector, To query the linear transformation matrix, The content-related transformation matrix of the key. This is the position-dependent transformation matrix of the key. Sensitive words and The relative position encoding vector, , All of these are global bias vectors. Sensitive words and The relative positional distance.
[0018] The feature extraction layer extracts deep features of sensitive information through a dual-stream attention and Transformer fusion module. This module comprises a dual-stream attention submodule and a Transformer fusion submodule. The dual-stream attention submodule captures the location information of sensitive words through the query stream and extracts the content features of sensitive words through the content stream, thus addressing the sensitivity determination problem of "polysemy" in privacy texts. Figure 2 As shown, Indicates a query stream. Representing the content stream, when performing initial position prediction using this attention mechanism, the query vector can be calculated via g. Q This includes location information; it can also utilize Calculate the key K Sum V Vectors, which also include content information, and This refers to the content information in the previous XLNet; the Transformer fusion submodule receives the feature output of the dual-stream attention submodule and deepens feature extraction through multi-layer Transformer blocks.
[0019] The calculation formula for feature extraction from the Transformer fusion submodule is as follows: ; ; in, For token embedding matrix, For the position embedding matrix, Vectorized data for diverse sample data.
[0020] The output layer is built upon the GPT2 classification head and incorporates risk-weighted logic to output the privacy leakage probability. The privacy leakage probability is output using the GPT2 sigmoid activation function. sigmoid ; in, This is the last hidden layer. and These are learnable parameters.
[0021] The loss function uses binary cross-entropy loss to optimize model parameters and minimize the prediction error of sensitive information. ; in, The total number of samples in the dataset. This is a sample of a single private text message. The true labels for the samples.
[0022] S4. Input the collected diverse samples into the risk detection model to detect data samples with privacy leaks and calculate the risk score using a risk-weighted method. The calculation formula is as follows: ; in, The weights are specifically the weights assigned to the sensitive information types in step S2. The percentage of sensitive information; privacy leakage risk levels are classified according to risk scores, including: .
[0023] S5. Output risk detection and assessment results, regularly collect high-risk data that has been blocked, and trigger fine-tuning when the number of blocks reaches a threshold to update model parameters in order to cope with new privacy leakage patterns.
[0024] Therefore, the above-mentioned method for detecting and assessing privacy leakage risks of pre-trained language models can accurately detect whether the model has privacy leakage risks and quantitatively assess the degree of risk, providing a basis for taking corresponding privacy protection measures.
[0025] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the technical solutions of the present invention, and these modifications or equivalent substitutions cannot cause the modified technical solutions to deviate from the spirit and scope of the technical solutions of the present invention.
Claims
1. A method for detecting and evaluating privacy leakage risks of a pre-trained language model, characterized in that, Includes the following steps: S1. Collect training samples used in the training process of the pre-trained language model and the input and output data of the model in practical applications and preprocess them. S2. Label the data, dividing it into sensitive data and non-sensitive data, and assign weights to sensitive information types. S3. Build a risk detection model, train it using unlabeled public text, and fine-tune and update the model parameters using a loss function; S4. Input the collected diverse samples into the risk detection model to detect data samples with privacy leaks and calculate the risk score using a risk weighting method. S5. Output risk detection and assessment results, regularly collect high-risk data that has been blocked, and trigger fine-tuning when the number of blocks reaches a threshold to update model parameters in order to cope with new privacy leakage patterns.
2. The method for detecting and evaluating privacy leakage risks of a pre-trained language model according to claim 1, wherein, The risk detection model in step S3 adopts the XLNet-GPT2 hybrid architecture, specifically including: The input layer uses XLNet's Permutation Language Model (PLM) to randomly permutate the original text sequence to generate enhanced samples, and introduces relative position encoding to address the positional differences of the same sensitive word in different contexts. The feature extraction layer extracts deep features of sensitive information through the fusion module of dual-stream attention and Transformer. The output layer is built on the GPT2 classification header and combines risk-weighted logic to output the probability of privacy leakage.
3. The method for detecting and assessing privacy leakage risks of pre-trained language models according to claim 2, characterized in that, The PLM method is used to randomly permutate text sequences to generate diverse sample data: ； in, The original sequence; This is the permuted sequence.
4. The method for detecting and assessing privacy leakage risks of pre-trained language models according to claim 2, characterized in that, When relative position encoding is introduced, the self-attention mechanism calculation formula for each data sequence in the model architecture is as follows: ； in, For the first The and the first The relative position attention score of each sensitive word , They represent the first , Sensitive words Embedded vector, To query the linear transformation matrix, The content-related transformation matrix of the key. This is the position-dependent transformation matrix of the key. Sensitive words and The relative position encoding vector, , All of these are global bias vectors. Sensitive words and The relative positional distance.
5. The method for detecting and assessing privacy leakage risks of pre-trained language models according to claim 3, characterized in that: The dual-stream attention and Transformer fusion module includes a dual-stream attention submodule and a Transformer fusion submodule. The dual-stream attention submodule captures the location information of sensitive words through the query stream and extracts the content features of sensitive words through the content stream. The Transformer fusion submodule receives the feature output of the dual-stream attention submodule and deepens the feature extraction through multiple layers of Transformer blocks.
6. The method for detecting and assessing privacy leakage risks of pre-trained language models according to claim 5, characterized in that, The calculation formula for feature extraction from the Transformer fusion submodule is as follows: ； ； in, For token embedding matrix, For the position embedding matrix, Vectorized data for diverse sample data.
7. The method for detecting and assessing privacy leakage risks of pre-trained language models according to claim 6, characterized in that, The output layer uses the sigmoid activation function of GPT2 to output the privacy leakage probability: sigmoid ; in, This is the last hidden layer. and These are learnable parameters.
8. The method for detecting and assessing privacy leakage risks of pre-trained language models according to claim 7, characterized in that, In step S3, the loss function uses binary cross-entropy loss to optimize the model parameters and minimize the prediction error of sensitive information. ； in, The total number of samples in the dataset. This is a sample of a single private text message. The true labels for the samples.
9. The method for detecting and assessing privacy leakage risks of pre-trained language models according to claim 7, characterized in that, The formula for calculating the risk score in step S4 is as follows: ； in, The weights are specifically the weights assigned to the sensitive information types in step S2. The percentage of sensitive information; privacy leakage risk levels are classified according to risk scores, including: 。