Method and server for making a service resistant to personal privacy inference attacks

Abstract

The present application provides a computer-implemented method for preventing a chatbot from leaking personal privacy under a black-box personal attribute inference attack. The chatbot is provided via a neural network executed by a processor of a server. The method includes training, by the processor, a language model (LM) of the chatbot according to utility objectives; applying, by the processor, one or more defense objectives to the chatbot's target LM by fine-tuning using a fake attacker model and pre-defined attributes with annotated datasets with a personal attribute predictor; and defending, by the processor, against inference attacks using the chatbot's target LM such that the personal privacy of content inputted and sent to the chatbot cannot be predicted by an external predictor and the security level of the chatbot is guaranteed.

Description

Methods and servers for protecting services from privacy-based inference attacks Technical Field

[0001] This invention generally relates to privacy protection, and more specifically, to a method and server for protecting personal privacy when performing services provided via neural networks in order to prevent the service from leaking personal privacy under personal privacy inference attacks. Background Technology

[0002] Social chatbots have been widely used in many applications to answer real-world questions and even provide emotional companionship. With the continuous development of large-scale pre-trained language models, some attempts have been made to build chatbots based on large generative language models such as GPT-2 (Generative Pre-trained Transformer 2) and XLNet. However, because private conversations are collected to train these model-based (LM) chatbots, and large language models tend to memorize training data, and some private data can be recovered from the model through black-box training data extraction attacks, these LM-based chatbots pose a risk of leaking private information.

[0003] Recent research has proposed differential privacy and nonlikelihood training to address the aforementioned memory problems. Beyond these memory issues, implicit representations of machine learning models with simple objectives can also reveal sensitive properties of the input. However, few studies have considered the overlearning problem of these learning models.

[0004] Therefore, how to prevent chatbots from leaking private information under black-box private personal information inference attacks, and how to prevent chatbots from overlearning their logic without reducing chatbot performance, are technical problems that need to be solved in this field. Summary of the Invention

[0005] According to one aspect of the present invention, a computer implementation method is provided for preventing chatbots from leaking personal privacy under black-box personal attribute inference attacks. The chatbot is provided via a neural network executed by a server's processor. The method includes: training a language model (LM) for the chatbot by the processor according to a utility objective; fine-tuning the target LM of the chatbot by the processor using a personal attribute predictor and predefined attributes of a fake attacker model and an annotated dataset by applying one or more defense objectives; and using the target LM of the chatbot to defend against inference attacks, such that the personal privacy of the content input and sent to the chatbot cannot be predicted by an external predictor, and the security level of the chatbot is guaranteed.

[0006] According to another aspect of the present invention, a computer implementation method is provided for preventing a service from leaking personal privacy under a personal privacy inference attack. The service is provided via a neural network executed by a server's processor. The method includes: training a primary algorithmic model of the service by the processor according to a utility objective; fine-tuning the primary algorithmic model of the service by the processor using an attribute predictor with a fake attacker model and predefined attributes of an annotated dataset to apply one or more defense objectives; and using the primary algorithmic model of the service to defend against inference attacks, such that the personal privacy of the content input to and sent to the service cannot be predicted by an external predictor, and the security level of the service is guaranteed.

[0007] According to another aspect of the present invention, a server is provided, and the server includes one or more processors configured to execute machine instructions to implement the above-described method. Attached Figure Description

[0008] Embodiments of the present invention will be described in more detail below with reference to the accompanying drawings, in which:

[0009] Figure 1 depicts a block diagram illustrating an electronic device according to an embodiment of the present invention;

[0010] Figure 2 depicts a flowchart of the process of improving the training of the chatbot's language model to enable the chatbot to defend itself;

[0011] Figure 3 depicts a schematic diagram illustrating the generated attacker model used to infer personal attributes;

[0012] Figure 4 illustrates a schematic diagram of a chatbot defense scenario under a personal attribute reasoning attack;

[0013] Figure 5 depicts a schematic diagram comparing undefended chatbots with defended chatbots; and

[0014] Figure 6 depicts a flowchart illustrating how the service is defended by improving the training process of the main algorithmic model of the service. Detailed Implementation

[0015] In the following description, a method and server are set forth as preferred examples, the server being configured to implement the method for performing personal privacy protection of services (e.g., chatbots) provided via neural networks, to prevent the service from disclosing personal privacy under personal privacy inference attacks (e.g., black-box personal attribute inference attacks) and the like. Those skilled in the art will understand that modifications, including additions and / or substitutions, can be made without departing from the scope and spirit of the invention. Specific details may be omitted so as not to obscure the invention; however, this disclosure is prepared to enable those skilled in the art to practice the teachings herein without engaging in undue experimentation.

[0016] In the publicly presented content, the fine-tuned GPT-2 is a language model for chatbots. First, a black-box attack is provided / generated to infer the personal attributes of the corresponding speaker using the final hidden state before the LM head of GPT-2 (generating pre-trained converter 2). The LM head is also a projection matrix, and subsequently, the final hidden representation is multiplied using softmax activation to obtain the word distribution, making the decoding algorithm usable to output words for responding to initiated conversations. It should be mentioned that the language model used for chatbots could be other types of algorithmic models, such as RNNs (Recurrent Neural Networks), XLNet, and T5 (Text-to-Text Transfer Transformer).

[0017] Secondly, the examples demonstrate that LM-based chatbots may overlearn, thereby revealing the speaker's personal attributes (overlearning is a broad concept; successful inference attacks are often caused by overlearning. While the provided defenses are effective against inference attacks, whether they prevent overlearning remains unknown. In fact, overlearning frequently occurs for many machine learning tasks). A single external multilayer perceptron (MLP) attacker model achieves a 37.59% probability of correctly inferring the speaker's personal attributes from 4332 attributes. The high accuracy of the simple external model implies a potential vulnerability in the hidden state to disclose the speaker's personal privacy. Therefore, it is necessary to improve the training algorithm to prevent such overlearning problems in order to avoid successful personal attribute inference attacks.

[0018] Finally, a defensive learning strategy is applied to GPT-2 to prevent such black-box attacks. Providing KL loss combined with MI loss as an additional defense objective for training GPT-2 reduces the attacker's personal attribute inference accuracy to 0.53%. GPT-2 is constructed by stacking converter decoder blocks, and then the LM head takes the hidden representation of the final decoder block as input and output tokens (words). For training, the token representations are input to the GPT-2 model, and the cross-entropy loss between the current input token and the next input token is computed to facilitate the model's prediction of the next word given the current word. For the inference process (after training), the input prefix token (or a simple token indicating the "beginning of the sentence") is used. <bos>The token is then used, and GPT-2 is used to generate words based on the prefix. This word also becomes the next input for GPT-2. This process is repeated to generate sentences word by word until the end of the sentence token is reached. <eos>Until the length exceeds the model's limitations.

[0019] The effects of this invention can be summarized as follows:

[0020] This disclosure discloses and analyzes hidden personal attribute inference attacks on LM-based chatbots, and considers them as a privacy risk.

[0021] It provides an effective defense training algorithm to prevent the leakage of the speaker's personal attributes by minimizing interaction information and approximating uniform distribution.

[0022] Extensive experiments were conducted to quantify both the privacy and utility of the proposed defense mechanism. Aside from addressing the issue of personal attribute leakage, the experimental results demonstrate that the proposed training algorithm for enabling chatbots to defend themselves has virtually no negative impact on utility.

[0023] Referring to FIG1 in the following description, according to various embodiments of the present invention, a server 100 is provided, which includes a processor 110, a non-transitory memory circuit 120, and a data communication circuit 130.

[0024] Non-transitory memory circuitry 120 is configured to store machine instructions (or programs) 121 and host database 122. Database 122 can be used to store parameters / models for running the chatbot, query data QD, training data TD, and / or response data RD.

[0025] Data communication circuit 130 is configured to establish a network connection for receiving object data OD, and the network connection can be a wired or wireless data communication connection. Furthermore, data communication circuit 130 is configured to establish other network connections for sending result data RD. For example, query data QD can be sent by another electronic device (e.g., a mobile device, PC, electric vehicle), and the query data QD contains messages input by a user of the other electronic device, as well as auxiliary data, wherein the auxiliary data includes: an identification code of the electronic device and / or the user. Response data RD can contain response content generated based on the content of query data QD. Training data TD is configured to train a language model used by the chatbot on the server, and the training data TD contains content or conversations corresponding to the user, as well as attribute / tag information related to the corresponding user. Some details of the training data TD may refer to, for example, Table 1 listed below.

[0026]

[0027] Table 1: Statistics of the training dataset

[0028] Essentially, the training data TD is a dataset collected from open-domain conversations where speakers specify personal attributes. Furthermore, these conversations should reflect the given personal attributes of the speakers.

[0029] Processor 110 executes machine instructions 121 to implement the methods provided by the presented disclosure.

[0030] Referring to Figure 2, in step S210, the processor 110 trains the language model (LM) of the chatbot according to a utility objective (through training data TD). The utility objective includes the LM loss.

[0031] Next, in step S220, processor 110 applies one or more defense targets using a personal attribute predictor to fine-tune the chatbot's target LM by using a fake attacker model and a trained LM, along with predefined attributes from an annotated dataset. The fake attacker model is used in the trained LM (e.g., the fine-tuned target LM). Defense targets include one or a combination of the following: KL (Kourbeck-Leibler) loss; and Mutual Information (MI) loss. Because the details of the actual attacker model are unavailable during an actual attack to update the LM, a fake attacker (adversary) is used as a rehearsal for updating the LM.

[0032] For example, referring to Figure 3, processor 110 generates a fake attacker model AM, which includes a projection layer and a softmax activation function layer. The projection layer contains multiple fully connected layers. The input sentence (context / content) from the query data QD is fed into the chatbot's language model (LM), and then the hidden representation of the LM is fed into the fake attacker model to output one or more inferred personal attributes (outcome data).

[0033] Furthermore, referring to Figure 4 (the provided training process), as shown by arrow A41, the user's client terminal 420 sends first query data QD1 to the chatbot 410 (or service) maintained by the server 100. The first content of the first query data QD1 is obtained by the attacker's personal attribute predictor 430 (or another type of predictor) (as shown by arrow A42). The chatbot 410 generates response data RD1, which has response content corresponding to the content of the query data QD1 (as shown by arrow A43), and the response content is obtained by the attacker's personal attribute predictor 430 (as shown by arrow A44). Next, the client terminal 420 sends second query data QD2 to the chatbot 410 (as shown by arrow A45), and the second content of the second query data QD2 is obtained by the attacker's personal attribute predictor 430 (as shown by arrow A46). Finally, the attacker's personal attribute predictor 430 can be a fake attacker model generated by the processor 110, and the attacker's personal attribute predictor 430 can output predicted data PD1 of the predicted personal attributes of the user with the client terminal based on the first content, the response content and the second content (as shown by arrow A47).

[0034] Returning to Figure 2, in step S230, the processor 110 uses the chatbot's target LM to defend against inference attacks, ensuring that the personal privacy of the content input and sent to the chatbot cannot be predicted by an external predictor, and guaranteeing the chatbot's security level. It should be noted that personal privacy can be understood as a type of personal privacy information within the corresponding content.

[0035] More specifically, there exists a GPT-2-based chatbot f pre-trained for a private conversation D. Only an autoregressive language model (GPT-2 model and LN head) is used to train the chatbot according to the following formula (1):

[0036] (1)

[0037] in 'f' refers to the loss function of the LM model; 'f' refers to the LM model. This refers to the parameters of the LM; It refers to the i-th word in the sentence; This refers to the situation where, given the discourse U = {w0,w1,...,w...}, the discourse is... |U|−1 In the case of}, the probability distribution of LM f; c refers to the previous content in private session D.

[0038] Furthermore, negative log-likelihood and softmax are applied to estimate the likelihood of a given discourse U = {w0, w1, ..., w |U|−1 Given the prior context c in} and D, the probability distribution Pr(w) of the target dialogue language model f is... i |c,w0,w1,...,w i−1 The adversary possesses an external annotated dialogue dataset D with n sessions. a = {(bU1,bs1),(bU2,bs2),...,(bU n ,bs n )}, where bU i Indicates the list of utterances from the i-th session {u i1 ,u i2 ,...,u ini }, and bs i The list of sensitive personal attributes corresponding to the corresponding discourse {s i1 ,s i2 ,...,s ini }, where "sensitive" means private information that users do not wish to disclose.

[0039] Each person's attribute s kj It is an integer that can be mapped to its individual attributes according to a predefined dictionary and 0 ≤ s kj ≤ C−1, where C is the total number of predefined personal attributes. The opponent's goal is to infer the speaker's personal attributes s based on the embedding f(u) of their utterances, where u and s refer to any utterance and its personal attribute markers. Furthermore, u and s are not in D. a Furthermore, there is no fixed pattern or rule for u.

[0040] On the other hand, details of algorithms used by fake attackers (e.g., black-box personal attribute inference attacks) are described below.

[0041] Personal attribute inference attacks can be viewed as supervised classification tasks. In a black-box attack setup, the adversary can only query the target dialogue model f by accessing the embeddings of the adversary's input, but cannot access or modify the model parameters θ. f As shown in the left part of Figure 5 (undefended chatbot), the adversary attempts to establish a system that utilizes its external data D. a And establish a personal attribute predictor model for dialogue model f. Personal attribute predictor output (f(u kj The loss function L is the estimated probability distribution for individual attribute C. A Using the cross-entropy between the predicted distribution and the background fact distribution, it can be expressed as the following formula (2):

[0042] (2)

[0043] in It is the loss function of the fake attacker; CE refers to the personal attribute tag s kj With the output of the personal attribute predictor (f(u kj Cross-entropy between )). A well-performing individual attribute predictor. This could pose a significant privacy threat. This is especially true for Machine Learning as a Service (MLaaS). It can be directly used to perform man-in-the-middle attacks on application programming interfaces (APIs). Furthermore, even if the original data is protected and the transmission channel is secure, curious service providers may still train their attacker models. To collect personal attributes of service users.

[0044] Regarding KL loss, the goal of KL loss is to minimize... The Kohlbek-Leibler divergence between (f(u)) and a uniform distribution. It makes... The flattening of the distribution of (f(u)) makes it impossible for the adversary to train the attacker model. Afterwards, any useful information can be obtained. The KL divergence between the uniform distribution and can be expressed as the following formula (3):

[0045] (3)

[0046] Where UNI indicates a uniform distribution and k indicates the k-th individual attribute label. For optimization, the constant term is omitted, and the following loss function is obtained as expressed by the following formula (4):

[0047] (4)

[0048] However, from the defender's perspective, they lack a model for attackers. Access to its parameters. Alternatively, the defender can construct its own personal attribute predictor to resemble a fake attacker. More precisely, it can simulate an adversary's access to the dataset. and personal attribute predictor p Add a comment. Next, the KL loss becomes as expressed by the following formula (5):

[0049] (5)

[0050] in This refers to the loss function of KL loss; Here, θ refers to the parameters of the fake attacker; u refers to the utterance (sentence in the conversation); k refers to the personal attribute tag index (the kth personal attribute tag); C refers to the total number of predefined personal attributes; and f(u) refers to the hidden state of the chatbot. Furthermore, the chatbot's parameter θ... f and fake attacker θ Ap Updated via KL loss. The chatbot is trained with a fake attacker to prevent overlearning by flattening the attacker's model distribution.

[0051] Equations (3) to (5) describe how KL is converted into cross-entropy loss. During training, after calculating the loss, the GPT-2 model can be updated via backpropagation (chain rule).

[0052] Regarding the MI loss, the privacy constraint requires that the hidden representation should not disclose personal privacy. This constraint is similar to a defensive objective (traditional training only trains GPT-2 with LM heads to improve generative utility), which can be achieved by the improved training process method provided by this invention.

[0053] In other words, given any utterance u and the personal properties behind utterance u, the goal is to minimize the interactive information between f(u) and s, as expressed by the following formula (6):

[0054] (6)

[0055] The upper limit of the relevant export can be expressed by the following formula (7):

[0056] (7)

[0057] Where p(s) can be any distribution of s; q(x) refers to the distribution of x through θ. f The probability distribution of the parameterized model f, and it is assumed that f(u) is sampled from the conditional distribution q(f(u)|x,s) (the conditional probability distribution of Y, which is the probability distribution of Y given X when X is known to be a specific value); This refers to the expected value of a distribution q(x). A conditional distribution is the distribution of the values ​​of a variable given that other variables have specified values. It also satisfies all the properties of a probability distribution.

[0058] However, q(s|f(u)) is difficult to estimate. Instead, p... Ψ (s|f(u)) is expressed by minimizing its KL divergence (using a neural network, such as a personal attribute predictor, to represent p) Ψ (s|f(u))) estimates q(s|f(u)), and then the following lower bound can be obtained as expressed by the following formula (8):

[0059]

[0060] (8)

[0061] Therefore, the objective in formula (6) can be formulated as an opponent training objective as expressed by the following formula (9):

[0062] (9)

[0063] This refers to the attacker's model that attempts to infer s from f(u). Since log p(s) is independent of f(u), and it can be removed from equation (9) to obtain the following formula (10):

[0064] (10)

[0065] Therefore, formula (10) shows how to infer the opponent p of s from f(u). Ψ With modification θ f A game of adversarial competition between defenders protecting their targets from attacks based on personal attribute inference. A personal attribute predictor model with softmax activation is used. p Come and learn p Ψ In order to obtain the defender's ultimate objective as expressed by the following formula (11):

[0066] (11)

[0067] For fake adversaries (attackers) and chatbots, formula (11) can be rewritten as two losses: L mi1 (u kj ,s kj ;θ Ap ) = CE(A p (f(u kj )),s kj ) and L mi2 (u kj ,s kj ;θ f ) = −CE(A p (f(u kj )),s kj Then, the MI loss can be formulated as expressed by the following formula (12):

[0068] L mi = λ0L mi1 + L mi2 (12)

[0069] Where λ0 controls the fake attacker p The ratio between the defender and the target f. This ratio is a hyperparameter that controls the impact of adversarial games (such as tradeoffs). f is updated after training against the defensive target.

[0070] Referring to the right portion of Figure 5, it illustrates how a chatbot is trained to counter black-box attacks. The defender's loss function combines KL loss, MI loss, and LM loss. It should be noted that the false adversary target in the MI loss violates the attempt to make... p The KL loss is flattened. The proposed loss assigns more weight to the KL loss, as expressed in the following formula (13):

[0071] L = L f + λ1L kl + λ2L mi (13)

[0072] Where λ1 and λ2 are hyperparameters, where λ1 ≥ 10λ2 to flatten p The distribution. Although the chatbot trained with a total loss L during the black-box attack was still unable to interfere. The training process, but L aims to alleviate the problem of overlearning personal attributes in f in order to address such personal attribute inference attacks.

[0073] Referring to Figure 5, it illustrates the overall flow for both attack and defense scenarios. The arrows in Phase 1 indicate the target of GPT-2, while the arrows in Phase 2 are used by the adversary. For the left and right sections, GPT-2 is trained first, and its parameters are frozen after training. Subsequently, the adversary trains its attacker model based on the GPT-2 embeddings.

[0074] The LM header is used to generate the response content. The LM header is also a projection matrix. The final hidden representation uses softmax activation multiplication to obtain the word distribution. The subsequent decoding algorithm can then be used to output the words (response content).

[0075] The "Personal Attribute Predictor" is used to apply the defense target in Phase 1. In Phase 2, the GPT-2 model is frozen (no longer adjusted or updated), and the "Personal Attribute Predictor" becomes the attacker model for executing the attack (the dataset maintained by Phase 2 is different from that of Phase 1).

[0076] More specifically, Phase 1 refers to training. The GPT-2 needs to be tuned against defensive targets to make it immune to inference attacks. After training from Phase 1, Phase 2 instructs on actual attacks against the tuned GPT-2. Such attacks then become ineffective.

[0077] Phase 1 serves as an adjustment process to update GPT-2 to make it immune to inference attacks. To avoid attacks, the defender needs to simulate the attacker's behavior to update the GPT-2 model. Phase 2 is the actual attack. The defender has no way of knowing what data (or which attributes) will be inferred during the attack. Phases 1 and 2 can be viewed as different time steps. Phase 1 occurs earlier than Phase 2 to avoid the attack. Note that in the right part of Figure 5, the attacker incorrectly predicts the personal attribute "favorite color is blue" due to the effective defense applied to the chatbot.

[0078] The defense target updates the predictor and GPT-2 model via backpropagation. There are no updates to the LM head (the LM head is only updated during generation).

[0079] In the following description, numerous experiments are conducted to evaluate the performance of the proposed defense learning strategy in terms of privacy and utility. The Experimental Setup section describes the relevant experimental setups. The Privacy section demonstrates attacker performance to measure how privacy is protected. The Utility section uses automated metrics to evaluate the utility of the chatbot after the application of the defense mechanism. The Ablation Study section performs ablation experiments on the defense target.

[0080] Experimental setup

[0081] To train a GPT-2 model as a chatbot, a DialoGPT model pre-trained on Reddit comment chains was used. Subsequently, the PersonaChat dataset was used to fine-tune the GPT-2 LM for the chatbot. Annotated datasets of the adversaries were then obtained. a The algorithm aligns personal attributes with corresponding utterances using positive examples (utterances, personal attributes) provided in the Natural Language Inference (NLI) dataset. For utterances without annotated personal attributes, a label of -1 is assigned to them. The dataset is rearranged to balance the label distribution between the training / validation / test datasets and partitioned in an 82:9:9 ratio. First, attackers and defenders share the same training data. In subsequent sections, the annotated data for the adversary and defender are separated without overlap. a The summary statistics are shown in Table 1 below.

[0082]

[0083] Table 1: Statistics of the aligned dataset.

[0084] Regarding the fake attacker model, in the experiments, a 2-layer neural network with cross-entropy loss was used as the fake attacker model. The fake attacker model utilizes the last token from GPT-2. <endoftext>The final layer embedding of "" was used as the model input. In addition, other attacker model architectures (attackers based on transformer blocks) and input embeddings (the average of all embeddings in the final layer of GPT-2) were tried, but the attack performance was not as good as the above 2-layer model.

[0085] Regarding the training details, for each session (as in integration within each training data TD), the utterance is represented by a special token. <endoftext>"The GPT-2 was trained using a concatenation method. To decode the output from the GPT-2, a nucleus sampling method was applied. The top-p coefficient was set to 0.9, with a temperature coefficient of 0.9, to sample words from the GPT-2. For optimization, two AdamW optimizers were set, one for the chatbot and one for the personal attribute predictor. The learning rate was 3e-5, with linear warm-up and decay. The hyperparameters were set to λ0 = 1, λ1 = 10, and λ2 = 1."

[0086] Regarding evaluation metrics, they are based on privacy and utility. For privacy, attacker performance is evaluated using personal attribute inference accuracy and a weighted F1-score. For utility, BERTScore, Distinct, BLEU, and Perplexity (PPL) are applied as evaluation metrics. BERTScore and BLEU measure the similarity between the generated output and the background facts, while Distinct (Dist) focuses on diversity. Perplexity represents uncertainty as the LM model fits the data.

[0087] Privacy.

[0088] The fake attacker models are listed in several scenarios presented in Table 2 below. The attack performance is denoted by Acc, which refers to the accuracy of the personal attribute inference test; F1 is the weighted average F1 score using 4332 personal attribute labels; and the maximum ratio indicates the ratio of the most frequent predictions among all predictions. The worse the attack model performance, the better the privacy protection achieved.

[0089]

[0090] Table 2: Assessment of privacy.

[0091] To illustrate the overlearning problem of GPT-2, consider two baseline attacks. If the adversary is unaware of the distribution of personal attributes, it can randomly guess 4332 tags (random prediction). Otherwise, the adversary can perform the best guess by only guessing the most frequent personal attributes in the dataset. LM indicates that only the language model objective was used to train the chatbot without any defense mechanisms. According to the table, LM achieves 37.59% accuracy in test personal attribute inference, while guessing the most frequent tags has only 0.72% accuracy. That is, the black-box personal attribute inference attack achieves 52 times the guessing accuracy. The huge performance gap between the attacker's model and the baseline guessing method indicates that the simple language model objective has a serious overlearning problem, unintentionally capturing the speaker's personal privacy.

[0092] To avoid the problem of overlearning personal attributes, an additional defense target is used as mentioned above. LM+KL+MI utilizes the language model, the KL loss in Equation (13), and the MI loss to train GPT-2. As shown in Table 2, LM+KL+MI significantly reduces the attack accuracy from 37.59% to 0.53%, and the F1-score drops from 0.37 to near 0. In terms of privacy protection, this defense mechanism even outperforms the best guess. That is, even if the adversary adds annotations to its own dataset to train the attacker model, the attack performance is still lower than simply guessing the most frequent tags. Therefore, black-box personal attribute prediction attacks become useless after applying the overall loss of the chatbot. The adversary cannot obtain any speaker's personal attributes from the embedding f(u) by training its personal attribute predictor.

[0093] To understand why the proposed overall loss works well, we examined the ratio of the most frequently predicted labels across all predictions. Table 2 reveals that the ratio of the most frequently predicted labels in the test group was 0.72%. After applying KL and MI losses, the attacker's model tended to predict single labels. For LM+KL+MI, the highest ratio even reached 81.87% of the predictions. This means that the proposed defense strategy could potentially "fool" the attacker's model into making incorrect predictions for single time slots. In summary, the experiments demonstrate that the proposed defensive learning strategy effectively mitigates the problem of overlearning of personal attributes and avoids black-box personal attribute inference attacks.

[0094] utility.

[0095] Besides privacy, utility is another key objective in training chatbots. Several automated metrics are considered to evaluate generation performance. For generation, GPT-2 is used to generate the response of the second speaker (speaker B in Figure 1) with all previous rounds as context. The generated model output is then compared with the context fact response. Dist-1 and Dist-2 are used to count the ratios of different unary and bigram segments. BLEU-1, BLEU-2, and BLEU-4 are applied to evaluate the generation similarity with the context fact. Due to the one-to-many nature of chat, BLEU is insufficient to compare the generated response with the context fact. Therefore, BERTScore's precision, recall, and accuracy are suitable for measuring similarity in the embedding space.

[0096] The evaluation results are shown in Table 3 below.

[0097]

[0098] Table 3: Evaluation of the utility of 4332 personal attribute markers.

[0099] LM is the baseline GPT-2 without any defense mechanisms. Adding KL loss significantly increases the perplexity from 14.8 to 28.9. After combining KL loss with MI loss, the perplexity decreases to 19.674. A plausible explanation is that KL loss confuses the individual attribute predictor and indirectly increases the uncertainty of GPT-2. Due to the one-to-many mapping between context and response, all GPT-2 models have relatively low BLEU scores. For Distinct and BERTScore, there are only minor differences between the baseline LM with defense mechanisms and GPT-2. Although uncertainty increases after applying KL loss and MI loss, it has no impact on the quality of the generated data. In summary, applying the proposed defense strategy has almost no negative impact on utility.

[0100] Ablation experiment.

[0101] To demonstrate the effectiveness of the proposed KL and MI losses and how they affect the performance of black-box personal attribute inference attacks, inclusion and removal of the proposed defense targets are considered. The results are presented in Table 2. The LM+KL model is trained using the language model and the KL loss in Equations (1) and (5) to train GPT-2. The LM+MI model is trained using the language model and the MI loss in Equations (1) and (12) to train GPT-2. The LM+KL+MI model is trained using the language model, the KL loss in Equation (13), and the MI loss. As can be seen from the table, all three models are able to reduce the test accuracy of black-box attacks. From a defensive perspective, the KL loss is weaker, but it tends to flatten the estimated personal attribute distribution at a much smaller maximum ratio. The LM+MI model shares similar test accuracy and F1-score with the LM+KL+MI model, but almost all predictions are made at a ratio of 99.84% for a single personal attribute label. After applying the KL loss to the LM+KL+MI model, the maximum ratio drops to 81.87%. As previously discussed, higher maximum ratios can also lead to privacy breaches. If an adversary knows the individual attribute with the highest ratio, they can improve their guessing by not predicting this attribute, which is a threat to fewer individual attribute labels (e.g., binary classification). This result validates that KL loss introduces a flatter estimate and that MI loss is effective against overlearning of individual attributes.

[0102] Previous black-box attacks typically assumed that the dataset D was annotated. a A similar data distribution must be shared with the defender's training data. To examine the performance of defense strategies against unseen individual attributes, a dataset D is used for the adversary. a Assign markers that cannot be obtained by the defender. Split the data using markers based on 500 unique personal attributes held by the opponent.

[0103] Table 2 illustrates the attacker's performance on imbalanced data distribution settings. Personal attribute inference accuracy remains extremely low, and the attacker model tends to predict more individual attribute labels compared to balanced data distribution settings. These results demonstrate that the proposed overall loss also prevents black-box personal attribute inference attacks on unseen personal attributes and validate previous recommendations that combining LM and MI losses during training can fool the attacker model into making incorrect predictions.

[0104] The above experiments were conducted based on 4332 personal attribute tags. In reality, many personal attributes share similar meanings and can be further clustered. Furthermore, considering the defensive performance against a smaller tag space is also feasible. Sentence-BERT embeddings of all personal attribute sentences were used, and k-means clustering was performed on the embeddings to obtain 8 clusters. The embeddings are vector representations from the model used to represent words. These clusters were then manually examined and categorized into car, food, animals (pets), family information, hobbies, job, personal information, and music taste. Based on the above data split (data split with 500 personal attributes), only the data in the first 3 clusters were assigned to the opponent to create an imbalanced data distribution.

[0105] Attack performance is presented in the lower part of Table 2. For imbalanced data distributions with small label spaces, the provided defense achieves lower attack accuracy compared to random prediction and best guess. However, the F1-score for imbalanced distributions is slightly higher than that for random prediction. Furthermore, although the distribution of the 8 clusters is clearly denser, the maximum ratio of the 8 clusters is much smaller compared to 4332 labels. This means that for smaller label spaces, the provided defense structure is not as robust as before. Moreover, the defense method is effective.

[0106] To make predictions about individual attributes, the argmax function is used to estimate the distribution of the individual attribute predictor. However, the internal distribution conveys crucial information about how the individual attribute predictor estimates f(u). It then sets up an imbalanced data split across the aforementioned eight clusters to examine for attackers. and fake attackers p A personal attribute predictor.

[0107] Table 4 below lists the test sets and attackers. and fake attackers p The detailed average distribution of the 8 markers is shown in Table 4. data The KL divergence refers to the KL divergence with a data distribution, and KL... uni The indicator shows a uniformly distributed KL divergence. It should be noted that a fake attacker only has access rights to the last 5 tags, and their KL... uni The calculation is based solely on these 5 tags, while the attacker's KL... uni Consider all 8 tags.

[0108]

[0109] Table 4: Test set distribution and average estimated distribution of personal attribute predictors on 8 clusters.

[0110] Furthermore, it calculates its KL divergence against the background fact distribution and the uniform distribution. Fake attackers tend to significantly differ from the data and attempt to flatten their distribution within their own training set (the last 5 labels). This behavior aligns with the training objective of flattening the distribution and deviating from the background fact distribution. For the attacker, their distribution appears close to the background fact distribution. This indicates the attacker's model... It can still learn statistical information about individual attributes. However, its attack performance is poor in terms of the imbalance shown in Table 2. Poor performance means that the proposed defensive learning strategy can confuse attackers in estimating a single sample f(u) and ultimately lead to incorrect predictions.

[0111] To provide a visual view of the utility, a generated sample is shown in Table 5 below.

[0112]

[0113] Table 5: Dialogue generation example for speaker B. All previous utterances between A and B are used as context to generate the response.

[0114] Both LM and LM+KL+MI can generate fluent and appropriate responses. Furthermore, they tend to maintain consistency with the preceding context. For example, if the context states that speaker B is a vegetarian, both chatbots reply that they do not like meat. This generated instance demonstrates the effectiveness of the proposed defense learning objective in preserving the model's utility.

[0115] While the methods described above are used to defend chatbots against personal attribute inference attacks, the present invention is not limited thereto. The provided methods can be used to prevent services from leaking personal privacy under personal privacy inference attacks, wherein the service is provided via a neural network executed by a server's processor. For example, referring to Figure 6, in step S610, the processor trains a primary algorithmic model of the service according to a utility objective. Next, in step S620, the processor 110 applies one or more defense objectives using an attribute predictor to fine-tune the primary algorithmic model of the service using a fake attacker model, the trained primary algorithmic model, and predefined attributes of an annotated dataset. Next, in step S630, the processor 110 uses the primary algorithmic model of the service to defend against inference attacks, ensuring that the personal privacy of the content input and sent to the service cannot be predicted by an external predictor, and guaranteeing the security level of the service.

[0116] Building upon the above, this paper demonstrates that LM-based chatbots tend to disclose personal attributes of speakers and proposes an effective defense against black-box personal attribute inference attacks on GPT-2. Unlike other traditional domains that suffer utility degradation, the defensive learning strategy does not compromise the robust generative capabilities of LM-based chatbots. Furthermore, extensive experiments are conducted to evaluate both privacy and utility. Black-box personal attribute inference attacks are performed under various settings to demonstrate the robustness of the proposed defensive learning strategy. Additionally, automated metrics are used to demonstrate the sustained utility of the proposed defensive learning strategy.

[0117] The functional units of the devices and methods according to the embodiments disclosed herein may be implemented using computing devices, computer processors, or electronic circuit systems, including but not limited to application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), and other programmable logic devices configured or programmed according to the teachings of this disclosure. Computer instructions or software code that run in the computing device, computer processor, or programmable logic device can be readily created by those skilled in the art based on the teachings of this disclosure.

[0118] All or part of the methods according to the embodiments can be executed in one or more computing devices including server computers, personal computers, laptop computers, mobile computing devices (e.g., smartphones) and tablet computers.

[0119] The embodiments include computer storage media in which computer instructions or software code are stored, which can be used to program a computer or microprocessor to perform any of the processes of the present invention. The storage media may include, but is not limited to, floppy disks, optical disks, Blu-ray discs, DVDs, CD-ROMs and magneto-optical disks, ROMs, RAMs, flash memory devices, or any type of media or device suitable for storing instructions, code, and / or data.

[0120] Each of the functional units according to various embodiments may also be implemented in a distributed computing environment and / or cloud computing environment, wherein one or more processing devices interconnected via communication networks such as intranets, wide area networks (WANs), local area networks (LANs), the Internet, and other forms of data transmission media execute all or part of the machine instructions in a distributed manner.

[0121] The foregoing description of the invention has been provided for purposes of illustration and description. The invention is not intended to be exhaustive or to limit it to the precise forms disclosed. Many modifications and variations will be apparent to those skilled in the art.

[0122] The embodiments were chosen and described in order to best explain the principles of the invention and its practical application, thereby enabling others skilled in the art to understand the various embodiments of the invention and the various modifications suitable for the particular intended use.< / endoftext> < / endoftext> < / eos> < / bos>

Claims

1. A computer implementation method for preventing chatbots from leaking personal privacy under black-box personal attribute inference attacks, characterized in that, The chatbot is provided by a neural network executed by a server's processor. The method includes: training a language model (LM) of the chatbot by the processor according to a utility objective; applying one or more defense objectives by the processor using a personal attribute predictor, fine-tuning the target LM of the chatbot using a fake attacker model and predefined attributes of an annotated dataset; and using the target LM of the chatbot to defend against inference attacks, such that the personal privacy of the content input to and sent to the chatbot cannot be predicted by an external predictor, and the security level of the chatbot is guaranteed; wherein fine-tuning the target LM of the chatbot includes: inputting an input sentence from query data into the target LM of the chatbot; inputting a hidden representation of the target LM of the chatbot into the fake attacker model to output one or more inferred personal attributes as a pre-render of the fake attacker to update the target LM of the chatbot.

2. The method according to claim 1, characterized in that, The utility objective includes LM loss.

3. The method according to claim 2, characterized in that, The LM loss is the objective function of the LM, and the objective function is expressed by the following formula: in 'f' refers to the loss function of the LM model; 'f' refers to the LM model. This refers to the parameters of the LM; It refers to the i-th word in the sentence; This refers to the situation where, given the discourse U = {w0,w1,...,w...}, the discourse is... |U|−1 In the case of}, the probability distribution of LMf; c refers to the previous content in private session D.

4. The method according to claim 1, characterized in that, The defense targets include one or a combination of the following: KL loss; and MI loss.

5. The method according to claim 4, characterized in that, The objective function of the KL loss is expressed by the following formula: in This refers to the loss function of the KL loss; 'u' refers to the parameters of the fake attacker; 'u' refers to the utterance; 'k' refers to the personal attribute tag index; 'C' refers to the total number of predefined personal attributes; and 'f(u)' refers to the hidden state of the chatbot.

6. The method according to claim 4, characterized in that, The objective function of the MI loss is expressed by the following formula: in This refers to the loss function of the KL loss; p Ψ (s|f(u)) refers to the distribution function used to estimate q(s|f(u)), where q(s|f(u)) refers to the distribution function obtained through θ. f The probability distribution of the parameterized model f; This refers to an attacker model that attempts to infer s from f(u).

7. The method according to claim 1, characterized in that, The fake attacker includes: a projection layer, which comprises multiple fully connected layers; and a softmax activation function layer.

8. The method according to claim 7, characterized in that, The loss function of the fake attacker is expressed by the following formula: in It is the loss function of the fake attacker; CE refers to the personal attribute tag s kj With the output of the personal attribute predictor (f(u kj The cross-entropy between )).

9. A server for preventing chatbots from leaking personal privacy under black-box personal attribute inference attacks, characterized in that, The chatbot is provided via a neural network executed by a processor of the server, the server comprising: the processor configured to execute machine instructions to implement a computer-implemented method, the method comprising: training a language model (LM) of the chatbot according to a utility objective by the processor; applying one or more defensive objectives by the processor using a personal attribute predictor to fine-tune the target LM of the chatbot using a fake attacker model and the trained LM and predefined attributes of an annotated dataset; and using the target LM of the chatbot to defend against inference attacks such that the personal privacy of the content input to and sent to the chatbot cannot be predicted by an external predictor, and the security level of the chatbot is guaranteed; wherein fine-tuning the target LM of the chatbot comprises: inputting an input sentence from query data into the target LM of the chatbot; inputting a hidden representation of the target LM of the chatbot into the fake attacker model to output one or more inferred personal attributes as a pre-render of a fake attacker to update the target LM of the chatbot.

10. The server according to claim 9, characterized in that, The utility objective includes LM loss.

11. The server according to claim 10, characterized in that, The LM loss is the objective function of the LM, and the objective function is expressed by the following formula: in 'f' refers to the loss function of the LM model; 'f' refers to the LM model. This refers to the parameters of the LM; It refers to the i-th word in the sentence; This refers to the situation where, given the discourse U = {w0,w1,...,w...}, the discourse is... |U|−1 In the case of}, the probability distribution of LMf; c refers to the previous content in private session D.

12. The server according to claim 9, characterized in that, The defense targets include one or a combination of the following: KL loss; and MI loss.

13. The server according to claim 12, characterized in that, The objective function of the KL loss is expressed by the following formula: in This refers to the loss function of the KL loss; 'u' refers to the parameters of the fake attacker; 'u' refers to the utterance; 'k' refers to the personal attribute tag index; 'C' refers to the total number of predefined personal attributes; and 'f(u)' refers to the hidden state of the chatbot.

14. The server according to claim 12, characterized in that, The objective function of the MI loss is expressed by the following formula: in This refers to the loss function of the KL loss; p Ψ (s|f(u)) refers to the distribution function used to estimate q(s|f(u)), where q(s|f(u)) refers to the distribution function obtained through θ. f The probability distribution of the parameterized model f; This refers to an attacker model that attempts to infer s from f(u).

15. The server according to claim 9, characterized in that, The fake attacker includes: a projection layer, which comprises multiple fully connected layers; and a softmax activation function layer.

16. The server according to claim 15, characterized in that, The loss function of the fake attacker is expressed by the following formula: in It is the loss function of the fake attacker; CE refers to the personal attribute tag s kj With the output of the personal attribute predictor (f(u kj The cross-entropy between )).

17. A computer implementation method for preventing services from leaking personal privacy under personal privacy reasoning attacks, characterized in that, The service is provided via a neural network executed by a server's processor. The method includes: training a primary algorithmic model of the service by the processor according to a utility objective; applying one or more defense objectives by the processor using an attribute predictor to fine-tune the primary algorithmic model of the service using a fake attacker model and predefined attributes of an annotated dataset; and using the primary algorithmic model of the service to defend against inference attacks, such that the personal privacy of the content input to and sent to the service cannot be predicted by an external predictor, and the security level of the service is guaranteed; wherein fine-tuning the primary algorithmic model of the service includes: inputting an input sentence from query data into the primary algorithmic model of the service; inputting a hidden representation of the primary algorithmic model of the service into the fake attacker model to output one or more inferred personal attributes as a pre-render of the fake attacker to update the primary algorithmic model of the service.

Images