Privacy protection method and device based on large model, equipment and medium

Abstract

The embodiment of the invention relates to a privacy protection method and device based on a large model, equipment and a medium. The method comprises the steps of obtaining privacy information in a text input by a user and an entity type identifier of the privacy information; according to the user input text and the entity type identifier, performing context-based coding processing and type-based decoding processing on the privacy information to obtain alternative information corresponding to the privacy information; and based on the corresponding relationship between the privacy information and the alternative information, performing desensitization question answering based on a question answering model on the user input text, recovering the privacy information, and generating a response text of the user input text. Therefore, the contextual semantic coherence and integrity of the replacement information and the entity type consistency of the replacement information and the privacy information are improved, the influence of replacing the privacy information with the replacement information on the desensitization question and answer is reduced, the user privacy is protected in the desensitization question and answer process, and the user experience is improved. And the finally generated response text has better accuracy and text coherence.

Description

Technical Field

[0001] The present disclosure relates to the field of natural language processing technology, and in particular to a privacy protection method, device, equipment and medium based on a large model. Background Art

[0002] With the rapid development of artificial intelligence technology, Large Language Models (LLMs) are widely used in intelligent dialogue, text generation, machine translation and other fields due to their excellent natural language understanding and generation capabilities. When users interact with large language models, the user input text may contain private information. If the large language model directly uses this private information for reasoning, there is a risk of privacy leakage.

[0003] In related technologies, the risk of privacy leakage can be reduced through differential privacy mechanisms, data desensitization, semantic generalization, etc. However, it is difficult to ensure the accuracy and consistency of model reasoning results while protecting user privacy in the above methods. Summary of the invention

[0004] In order to solve the above technical problems or at least partially solve the above technical problems, the present disclosure provides a privacy protection method, apparatus, device and medium based on a large model.

[0005] The present disclosure provides a privacy protection method based on a large model, including:

[0006] Obtaining private information in the user input text and an entity type identifier of the private information;

[0007] According to the user input text and the entity type identifier, performing context-based encoding processing and type-based decoding processing on the private information to obtain replacement information corresponding to the private information;

[0008] Based on the correspondence between the private information and the replacement information, the user input text is subjected to desensitized question and answer based on a question and answer model and the private information is restored to generate a response text for the user input text.

[0009] The present disclosure also provides a privacy protection device based on a large model, including:

[0010] An acquisition module, used to acquire the private information in the text input by the user and the entity type identifier of the private information;

[0011] A processing module, configured to perform context-based encoding processing and type-based decoding processing on the private information according to the user input text and the entity type identifier, so as to obtain replacement information corresponding to the private information;

[0012] The question-and-answer module is used to perform desensitized question-and-answering based on a question-and-answer model on the user input text based on the corresponding relationship between the private information and the replacement information, and restore the private information to generate a response text for the user input text.

[0013] An embodiment of the present disclosure also provides an electronic device, comprising: a processor; a memory for storing executable instructions of the processor; the processor is used to read the executable instructions from the memory and execute the instructions to implement the large model-based privacy protection method provided in the embodiment of the present disclosure.

[0014] The embodiment of the present disclosure also provides a computer-readable storage medium, which stores a computer program, and the computer program is used to execute the privacy protection method based on a large model as provided in the embodiment of the present disclosure.

[0015] The technical solution provided by the embodiments of the present disclosure has the following advantages over the prior art: the privacy protection solution based on a large model provided in the embodiments of the present disclosure, the method comprising: obtaining the privacy information in the user input text and the entity type identifier of the privacy information; performing context-based encoding processing and type-based decoding processing on the privacy information according to the user input text and the entity type identifier, to obtain alternative information corresponding to the privacy information; based on the correspondence between the privacy information and the alternative information, performing desensitizing question and answering based on the question and answer model on the user input text and restoring the privacy information, to generate a response text for the user input text. By adopting the above technical scheme, after determining the privacy information and its type in the user input text, the privacy information is encoded based on the context information in the user input text, and then decoded based on the type of the privacy information to obtain the alternative information corresponding to the privacy information, and a desensitized question and answer is performed based on the alternative information, and the private information in the question and answer answer is restored to obtain the response text, thereby improving the coherence and integrity of the contextual semantics of the alternative information, and improving the consistency of the entity type corresponding to the alternative information and the privacy information, reducing the impact of replacing the private information with the alternative information on the desensitized question and answer, protecting the user's privacy in the subsequent desensitized question and answer process, and making the finally generated response text have better accuracy and text coherence. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0017] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0018] Figure 1 A flowchart of a privacy protection method based on a large model provided in an embodiment of the present disclosure;

[0019] Figure 2 A flowchart of another privacy protection method based on a large model provided in an embodiment of the present disclosure;

[0020] Figure 3 A schematic diagram of a privacy protection method based on a large model provided in an embodiment of the present disclosure;

[0021] Figure 4 A schematic diagram of the structure of a privacy protection device based on a large model provided in an embodiment of the present disclosure;

[0022] Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0023] In order to more clearly understand the above-mentioned objectives, features and advantages of the present disclosure, the scheme of the present disclosure will be further described below. It should be noted that the embodiments of the present disclosure and the features in the embodiments can be combined with each other without conflict.

[0024] In the following description, many specific details are set forth to facilitate a full understanding of the present disclosure, but the present disclosure may also be implemented in other ways different from those described herein; it is obvious that the embodiments in the specification are only part of the embodiments of the present disclosure, rather than all of the embodiments.

[0025] With the rapid development of artificial intelligence technology, Large Language Models (LLMs) are widely used in human-computer dialogue, text generation, machine translation and other fields due to their excellent natural language understanding and generation capabilities. When users interact with large language models, the user input text may contain private information. If the large language model directly uses this private information for reasoning, there is a risk of privacy leakage.

[0026] In related technologies, the risk of privacy leakage can be reduced through differential privacy mechanisms, data desensitization, and semantic generalization. Specifically, the differential privacy mechanism reduces the identifiability of data by adding noise to the data input to the model or the data output by the model. The differential privacy mechanism has a significant effect in protecting data privacy, but the added noise may affect the accuracy of the final result and the coherence of the text. In addition, the differential privacy mechanism is usually an optimization for the model training stage, and has weak support for real-time privacy protection in the reasoning process of the model application stage. Especially in precise tasks that are highly dependent on context, the responses generated based on the differential privacy mechanism may deviate from the user's intentions, reducing the reliability of practical applications.

[0027] Data desensitization replaces private information in the data with common expressions before the data is input into the model, for example, replacing "person A" with "someone". Although this method can reduce the direct exposure of private information, it uses simple word list replacement or regular expression replacement and lacks context awareness. This method may cause the loss of semantic consistency of the context, destroy the semantics of the input text, and lack the semantic information based on reasoning, making it difficult for the subsequent large language model to correctly understand the context and generate high-quality responses, affecting the reasoning effect of the large language model. For example, after desensitizing "person A at university B" to "someone at a certain university", the generated result may lack sufficient semantic information.

[0028] Semantic generalization converts specific information into abstract expressions to protect user privacy by blurring or semantically generalizing private information. For example, "person A" is replaced by "a person". This method can reduce privacy exposure and has a good effect on privacy protection, but it may oversimplify specific information and make the data overly abstract after generalization, resulting in the loss of key information in the context and incomplete semantic expression of the context, which in turn affects the accuracy of reasoning of large language models. Especially in question-answering tasks that require precise context relevance, this abstract expression may seriously affect the reasoning effect.

[0029] In addition, the real-time privacy protection capabilities of the above methods are insufficient. Specifically, privacy protection methods such as differential privacy mechanisms are usually applied to the training stage of large language models, and their ability to process the privacy information input by users during the reasoning process of the large language model application stage is relatively weak. In the scenario of real-time dynamic reasoning, there is a lack of efficient privacy detection and processing mechanisms, making it difficult to meet user privacy protection needs in real time.

[0030] In summary, the relevant technologies have deficiencies in privacy protection and semantic retention. It is difficult to balance the accuracy, semantic consistency, and coherence of reasoning results while protecting user privacy and security, and there are problems such as loss of key information.

[0031] In order to solve the above problems, the embodiments of the present disclosure provide a privacy protection method based on a large model, which is introduced below in conjunction with specific embodiments.

[0032] Figure 1 The present invention provides a flowchart of a privacy protection method based on a large model according to an embodiment of the present invention. The privacy protection method based on a large model can be applied to a privacy protection device based on a large model. The privacy protection device based on a large model can be implemented using software and / or hardware. The privacy protection device based on a large model can generally be integrated into an electronic device. Figure 1 As shown, the privacy protection method based on the large model includes:

[0033] Step 101: Obtain private information in a text input by a user and an entity type identifier of the private information.

[0034] The user input text may be the original text input by the user, and the user input text may be used for human-computer dialogue. The privacy information may be confidential information related to the user, and the privacy information may be understood as information that the user does not want to be disclosed. The entity type identifier may be used to identify the type of the privacy entity, and the privacy entity may be the entity recorded by the privacy information. This embodiment does not limit the entity type identifier, for example, the entity type identifier may include one or more of a name identifier, an address identifier, and a telephone number identifier.

[0035] In the disclosed embodiment, a user can enter user input text in a text box, and a privacy protection device based on a large model can obtain the user input text, perform privacy identification on the user input text, and obtain the privacy information contained in the user input text. In addition, the entity type of the privacy information is identified to determine the entity type identifier corresponding to the privacy information.

[0036] In some embodiments of the present disclosure, obtaining private information in user input text and an entity type identifier of the private information includes: inputting the user input text into a trained entity recognition model to obtain a location identifier and an entity type identifier of the private information output by the entity recognition model.

[0037] The entity recognition model may be a neural network model with a privacy information recognition function, and the entity recognition model may be a model built based on named entity recognition technology. This embodiment does not limit the entity recognition model. For example, the entity recognition model may be a pre-trained language named entity recognition (Bidirectional Encoder Representations from Transformers Named Entity Recognition, BERT-NER) model built based on a transformation architecture (Transformer). The location identifier may be used to mark the location of the privacy information in the entity recognition model.

[0038] In this embodiment, the privacy protection device based on the large model can input the user input text into the trained entity recognition model, and the entity recognition model performs entity detection on the user input text and outputs a privacy information set and an entity type identifier. The privacy information set EE = {ee1, ee2, ..., ee n}, where ee1, ee2, …, ee n For multiple private information.

[0039] Specifically, the private information set EE=NER(XX), where NER() represents an entity recognition model and XX represents user input text. The entity recognition model can be trained on training data corresponding to manually annotated private information.

[0040] In the above solution, automatic dynamic detection of privacy information in user input text is implemented, which creates a basis for automatic desensitization of user input text in the application stage of the model.

[0041] In some embodiments of the present disclosure, the location identifier includes a first location identifier and a second location identifier, the first character identifier is determined based on the first location identifier and the entity type identifier, the first character identifier is used to mark the first character of the private information in the user input text, and the second location identifier is used to mark the subsequent characters of the private information in the user input text.

[0042] Among them, the first position identifier can be used to mark the position of the first character of the private information, and the first character can be the first character in the private information according to the word order. This embodiment does not limit the first position identifier. For example, the first position identifier can be B. The second position identifier can be used to mark the position of subsequent characters of the private information. The subsequent characters can be characters that are located after the first character in the word order of the private information. This embodiment does not limit the second position identifier. For example, the second position identifier can be I. The first character identifier can be an identifier that corresponds to the first position identifier and the entity type identifier as a whole. This embodiment does not limit the first character identifier. For example, the first character identifier can be obtained by splicing the first position identifier and the entity type identifier.

[0043] In this embodiment, the entity recognition model can perform character identification on the user input text and mark the privacy information in the user input text. Specifically, the entity recognition model can identify the first character of the privacy information in the user input text through the first character identifier, and identify the subsequent characters of the privacy information in the user input text through the second position identifier. The privacy protection device based on the large model can determine the privacy information based on the first character identifier and the second position identifiers that are consecutive after the first character identifier, and parse the first character identifier to determine the entity type identifier of the privacy information. For example, the first position identifier can be B, the entity type identifier corresponding to the name type can be NAEM, and the first character identifier can be B_NAME.

[0044] In the above scheme, the beginning of the private information and the entity type identifier corresponding to the private information are marked by the first character identifier, and the specific scope of the private information is marked by the second position identifier. The first character identifier contains multiple important information at the same time, thereby improving the recognition efficiency of private information.

[0045] Step 102: Perform context-based encoding and type-based decoding on the private information according to the user input text and the entity type identifier to obtain replacement information corresponding to the private information.

[0046] Among them, the replacement information can be used to desensitize and replace the private information. The generation of the replacement information refers to the context of the private information in the user input text and the entity type corresponding to the private information. The replacement information can be semantically related to the private information and correspond to the same entity type. The context-based encoding process can be a natural language encoding process based on context-aware technology, and the context relevance between the replacement information and the private information can be improved through the context-based encoding process. The type-based decoding process can be a natural language decoding process based on the entity type, and the type-based decoding process can improve the replacement quality of the private information with the replacement information.

[0047] In the disclosed embodiment, the privacy protection device based on the large model can perform context-based encoding processing according to the position of the private information in the user input text, and perform entity type-based decoding processing based on the encoding processing result, and obtain the replacement information corresponding to the private information through the above encoding and decoding.

[0048] Figure 2 A flowchart of another privacy protection method based on a large model provided in an embodiment of the present disclosure is shown in FIG. Figure 2 As shown, in some embodiments of the present disclosure, according to the user input text and the entity type identifier, the private information is encoded based on the context and decoded based on the type to obtain the replacement information corresponding to the private information, including:

[0049] Step 201: input the user input text and private information into a model encoder of a trained information generation model to obtain an entity semantic vector output by the model encoder.

[0050] Among them, the information generation model can be used to determine the corresponding alternative information according to the contextual semantics of the private information and the entity type corresponding to the private information. The information generation model can be a pre-trained neural network model. The model encoder (Transformer Encoder) can be used to convert the data input into the model encoder into an intermediate representation in the form of a vector, which contains the feature information of the data. The entity semantic vector can be used to represent the semantic information of the private information itself and the contextual semantic information of the private information in the user input text. Through the entity semantic vector, the information generation model can be guided to pay attention to the context in which the private information is located.

[0051] In this embodiment, the privacy protection device based on the large model can input the user input text and the private information into the model encoder, wherein the private information can be input into the model encoder in the form of the position information corresponding to the private information in the user input text, and the position information can represent the character position of the private information in the user input text. The model encoder performs context encoding on the private information based on the user input text, and then obtains the vector representation of the private information in the user input text, and obtains the entity semantic vector, which contains the semantic information and context information of the private information, and is used to generate alternative information related to the private information.

[0052] For example, for the private information set EE = {ee1, ee2, ..., ee n}, each private information can be represented as ee i , the location information of the private information in the user input text is recorded as r i, the user input text is recorded as XX. The model encoder is used to encode the private information and its corresponding position information, and the entity semantic vector representation h of the private information in the user input text is extracted. i for:

[0053] h i =Encoder(XX,r i );

[0054] Among them, Encoder() can represent the model encoder.

[0055] Step 202: Input the entity semantic vector and entity type identifier into the model decoder of the information generation model to obtain the replacement information output by the model decoder.

[0056] Among them, the model decoder (Transformer Decoder) can be used to convert the intermediate representation generated by the model encoder into a target form output.

[0057] In this embodiment, the privacy protection device based on the large model can input the entity semantic vector and the entity type identifier into the model decoder, and the model decoder dynamically generates replacement information based on the entity semantic vector. The entity type corresponding to the replacement information has a high similarity with the entity type corresponding to the privacy entity, and the probability that the replacement information contains the privacy information is low. Specifically, the replacement information ee′ i Can be:

[0058] ee′ i =Decoder(h i ,t i );

[0059] Among them, Dncoder() can represent the model decoder, h i It can be entity semantic information, t i The entity type identifier can be one-hot encoded, and the entity type identifier can enable the model decoder to predict alternative information under the constraint of the entity type identifier.

[0060] In the above scheme, an information generation model is used to predict the detected private information based on the context of the user input text and the entity type identifier, and to generate alternative information that has a high semantic consistency with the private information and does not contain the private information.

[0061] In some embodiments of the present disclosure, the training process of the information generation model includes:

[0062] Step a1, obtaining sample privacy information in the sample input text and a sample type identifier of the sample privacy information.

[0063] The sample input text may be input text used for human-computer dialogue in training data. The sample private information may be private information in the sample input text. The sample type identifier may be used to identify the type of entity represented by the sample private information.

[0064] In the disclosed embodiment, the sample input text and the sample privacy information and sample type identifier corresponding to the sample input text may be predetermined, and the privacy protection device based on the large model may obtain the existing sample input text and the sample privacy information and sample type identifier corresponding to the sample input text. Alternatively, the sample input text may be predetermined, and the privacy protection device based on the large model may obtain the sample input text, perform privacy identification on the sample input text, and obtain the sample privacy information contained in the sample input text. In addition, the entity type of the sample privacy information may be identified to determine the sample type identifier corresponding to the sample privacy information.

[0065] Step a2: input the sample input text and the sample privacy information into the intermediate encoder of the intermediate model to obtain the sample semantic vector output by the intermediate encoder.

[0066] The intermediate model may be a model in the training process, and the model obtained after the intermediate model is trained may be an information generation model. The intermediate encoder may be an encoder that has not completed training, and the model encoder obtained after the intermediate encoder is trained may be a model encoder. The sample semantic vector may be used to represent the semantic information of the sample private information itself and the contextual semantic information of the sample private information in the sample input text, and the sample semantic vector may be used to guide the intermediate model to pay attention to the context in which the sample private information is located.

[0067] In this embodiment, the privacy protection device based on the large model can input the sample input text and the sample privacy information into the intermediate encoder, wherein the sample privacy information can be input into the intermediate encoder in the form of sample position information corresponding to the sample privacy information in the sample input text, and the sample position information can represent the character position of the sample privacy information in the sample input text. The intermediate encoder contextually encodes the sample privacy information based on the sample input text, and then obtains the vector representation of the sample privacy information in the sample input text, and obtains the sample semantic vector, which contains the semantic information and context information of the sample privacy information, and is used to generate sample replacement information related to the sample privacy information.

[0068] Step a3: input the sample semantic vector and the sample type identifier into the intermediate decoder of the intermediate model to obtain the sample replacement information output by the intermediate decoder.

[0069] The intermediate decoder may be an encoder in the training process, and the model decoder may be obtained after the intermediate decoder completes the training. The intermediate decoder may be used to convert the intermediate representation generated by the model encoder into a target form output. The sample replacement information may be information for desensitizing and replacing the sample privacy information.

[0070] In this embodiment, the privacy protection device based on the large model can input the sample semantic vector and the sample type identifier into the intermediate decoder, and the intermediate decoder dynamically generates sample replacement information based on the sample semantic vector, which is expected to be consistent with the entity type of the sample privacy entity and does not contain the sample privacy information.

[0071] Step a4, training the intermediate model according to the model loss function, sample privacy information, and sample replacement information until an information generation model is obtained; wherein the model loss function is constructed based on at least one of the semantic loss function, the privacy loss function, and the fluency loss function.

[0072] Among them, the model loss function can be used to determine the loss in one or more dimensions after replacing the sample privacy information. The semantic loss function can be used to determine the loss in the semantic accuracy dimension after replacing the sample privacy information. The privacy loss function can be used to determine the loss in the privacy protection dimension after replacing the sample privacy information. The fluency loss function can be used to determine the loss in the text flow dimension after replacing the sample privacy information.

[0073] In this embodiment, during the training process of the information generation model, the intermediate model can be trained in a supervised learning manner using training data. During the training process, the model is iteratively trained with the goal of minimizing the model loss function based on sample privacy information and sample replacement information until a fully trained information generation model is obtained.

[0074] For example, the model loss function may be determined based on two of the semantic loss function, the privacy loss function, and the fluency loss function, or the model loss function may be determined based on the semantic loss function, the privacy loss function, and the fluency loss function. Thus, the information generation model is trained based on multiple dimensions, so that the information generation model can balance the requirements of multiple dimensions and optimize the quality of the alternative information and the consistency of the alternative information with the context.

[0075] In some embodiments of the present disclosure, the semantic loss value in the semantic loss function is the vector distance between the vector of the sample private information and the vector of the sample replacement information.

[0076] The semantic loss value may represent the loss in the semantic accuracy dimension after replacing the sample private information, and the semantic loss value may be the semantic similarity between the sample private information and the sample replacement information. The vector of the sample private information may be an embedding vector obtained by converting the sample private information. The vector of the sample replacement information may be an embedding vector obtained by converting the sample replacement information. The vector distance may be the norm of the difference vector between the two embedding vectors.

[0077] In this embodiment, during the training process of the information generation model, the privacy protection device based on the large model can calculate the semantic loss value between the sample replacement information and the sample privacy information through a sentence-level bidirectional encoder (Sentence-BERT), and then based on the semantic loss function, the correlation between the replacement information and the privacy information can be improved.

[0078] For example, the semantic loss value L semantic Can be:

[0079] L semantic =∥Embed(e i ′ )-Embed(e i )∥;

[0080] Among them, e i ′ Indicates sample replacement information, e i Represents sample privacy information, where Embed() represents the model for calculating the vector.

[0081] In some embodiments of the present disclosure, the privacy loss value in the privacy loss function is the cross entropy value of the discriminant probability, and the discriminant probability represents the probability that the sample replacement information contains the sample privacy information. The discriminant probability is determined by a preset discriminator model.

[0082] The privacy loss value can represent the loss in the privacy protection dimension after replacing the sample privacy information. The privacy loss value can be the loss value corresponding to the probability that the sample privacy information is contained in the sample replacement information generated during the intermediate model training process. The discriminator model can be used to determine the probability that the sample privacy information is contained in the sample replacement information.

[0083] In this embodiment, during the training process of the intermediate model, the privacy protection device based on the large model can determine the discrimination probability corresponding to the sample replacement information through the discriminator model, and calculate the binary cross entropy loss value based on the discrimination probability to obtain the privacy loss value.

[0084] Specifically, the privacy loss value L privacy Can be:

[0085] L privacy=-S(e i ′ )logS(e i ′ );

[0086] Among them, e i ′ represents the sample replacement information, and S() represents the discriminator model. Thus, a noise scoring mechanism is added to adjust the probability of sample privacy information contained in the sample replacement information.

[0087] In some embodiments of the present disclosure, the model loss function may be constructed based on a semantic loss function and a privacy loss function, so as to achieve a balance between the semantic consistency of private information and the privacy protection of substitute information.

[0088] In some embodiments of the present disclosure, the fluency loss value in the fluency loss function is the perplexity of the sample desensitized input text corresponding to the sample replacement information.

[0089] Among them, the fluency loss value can characterize the text fluency of the sample desensitized input text obtained by replacing the sample privacy information in the sample input text with the sample replacement information.

[0090] In this embodiment, the privacy protection device based on the large model can replace the sample privacy information in the sample input text with the sample replacement information to obtain the sample desensitized input text corresponding to the sample input text, and input the sample desensitized input text into the perplexity calculation model to obtain the fluency loss value.

[0091] Specifically, the sample privacy information in the sample input text can be i Replace with sample replacement information i ′ , get the sample desensitized input text X corresponding to the sample input text ′ The sample desensitized input text X ′ The fluency loss value L perplexity Can be:

[0092] L perplexity =Perplexity(X ′ );

[0093] Among them, Perplexity() can be a perplexity calculation model. Therefore, training the information generation model based on the fluency loss function can improve the fluency of the text after desensitization.

[0094] In some embodiments of the present disclosure, the model loss value corresponding to the model loss function may be the sum of the semantic loss value, the privacy loss value, and the fluency loss value, that is, the model loss value L total Can be:

[0095] L total =L semantic +L privacy +L perplexity ;

[0096] Among them, L semantic represents the semantic loss value, L privacy represents the privacy loss value, L perplexity Indicates the fluency loss value.

[0097] In the above scheme, the privacy protection effect, semantic consistency, and text fluency of the desensitization process are quantified. While improving the privacy security of users, the text quality of the alternative information is improved, making the alternative information diverse and improving the integrity of the final generated sentence, so that the privacy protection method based on the large model can be applied in a variety of scenarios.

[0098] Step 103: Based on the correspondence between the private information and the alternative information, desensitize the user input text based on the question-answering model and restore the private information to generate a response text for the user input text.

[0099] The corresponding relationship may be a relationship in which the substitute information replaces the private information. The question-answering model may be a large language model with a dialogue function, and this embodiment does not limit the type of the question-answering model. The response text may be a text used to respond to the user input text, and the response text may contain the recovered private information.

[0100] In this embodiment, the privacy protection device based on the large model can replace the private information in the user input text with the alternative information according to the correspondence between the private information and the alternative information to obtain a desensitized question and answer text, and conduct an intelligent dialogue based on the desensitized question and answer text and restore the alternative information in the intelligent dialogue result to sensitive information to obtain a response text corresponding to the user input text.

[0101] In some embodiments of the present disclosure, based on the correspondence between private information and alternative information, desensitizing the user input text based on the question-answering model and restoring the private information, generating a response text for the user input text, including:

[0102] The private information in the user input text is replaced with the corresponding alternative information to obtain the desensitized input text corresponding to the user input text; the desensitized input is input into the question-answering model to obtain the desensitized output text output by the question-answering model; if the desensitized output text contains alternative information, the alternative information in the desensitized output text is replaced with the corresponding private information to obtain the answer text.

[0103] The desensitized input text may be a text obtained after desensitizing the user input text. The desensitized output text may be an answer text output by the question-answering model according to the desensitized input text, and the desensitized input text may be understood as an intermediate response result to the user input text.

[0104] In this embodiment, the privacy protection model based on the large model can replace the private information in the user input text with the substitute information corresponding to the private information, implement the desensitization processing of the user input text, and obtain the desensitized input text corresponding to the user input text. The desensitized input text is input into the question-answering model to obtain the desensitized output text output by the question-answering model. Furthermore, the privacy protection device based on the large model can determine whether the desensitized output text includes the substitute information. If so, the substitute information in the desensitized output text is replaced with the private information corresponding to the substitute information, implement the recovery processing of the private information, and obtain the answer text corresponding to the user input text.

[0105] In the above scheme, the user input text is first desensitized to obtain a desensitized input text corresponding to the user input text, and question and answer processing is performed based on the desensitized input text to obtain a desensitized output text. The replacement information in the desensitized output text is restored to private information to obtain a response text. While achieving a normal response to the user input text, it avoids inputting private information into the question and answer model, thereby avoiding the leakage of user privacy.

[0106] The privacy protection method based on a large model provided by the embodiment of the present disclosure includes: obtaining the private information in the user input text and the entity type identifier of the private information; according to the user input text and the entity type identifier, performing context-based encoding processing and type-based decoding processing on the private information to obtain the replacement information corresponding to the private information; based on the corresponding relationship between the private information and the replacement information, performing desensitized question and answer based on the question and answer model on the user input text and restoring the private information, and generating the answer text of the user input text. Using the above technical solution, after determining the private information and its type in the user input text, the private information is encoded based on the context information in the user input text, and then decoded based on the type of the private information to obtain the replacement information corresponding to the private information, and desensitized question and answer based on the replacement information and restore the private information in the question and answer answer to obtain the answer text, thereby improving the coherence and integrity of the context semantics of the replacement information, and improving the consistency of the entity type corresponding to the replacement information and the private information, reducing the impact of replacing the private information with the replacement information on the desensitized question and answer, protecting the user's privacy in the subsequent desensitized question and answer process, and making the finally generated answer text have better accuracy and text coherence.

[0107] In addition, this privacy protection method based on a large model meets the real-time requirements for privacy protection in dynamic interaction scenarios and is suitable for model application scenarios that provide real-time responses to user input text.

[0108] Next, the privacy protection method based on a large model in the embodiment of the present disclosure is further explained through a specific example. The privacy protection method based on a large model integrates the identification of private information, the replacement of private information to alternative information, and the model training based on a multi-dimensional semantic loss function into the reasoning process of the question-answering model. A balance between privacy protection and semantic consistency is achieved, and the privacy processing of private information is achieved through dynamic detection of private information and generation of alternative information to replace the private information based on contextual perception of private information. At the same time, the fluency and accuracy of the final answer text are ensured, which effectively improves the security and applicability of the question-answering model in privacy scenarios and achieves comprehensive protection of user privacy.

[0109] Figure 3 A schematic diagram of a privacy protection method based on a large model provided in an embodiment of the present disclosure is shown in FIG. Figure 3 As shown, in the privacy protection method based on the large model, first, the user input text is obtained. The user input text may contain highly sensitive privacy information, such as name, address, telephone number, etc.

[0110] Furthermore, the private information in the text input by the user is detected and determined. Specifically, the private information in the text input by the user is automatically identified through the named entity recognition technology and the private information is marked through the first character mark and the second position mark. The entity recognition model can be a Transformer-based named entity recognition (BERT-NER) model.

[0111] Furthermore, alternative information corresponding to the private information is generated. Specifically, alternative information semantically related to the private information is generated through context-aware technology, and the alternative information is made to achieve a good balance between context-based semantic consistency and privacy protection. To achieve this goal, the information generation model needs to be trained in advance.

[0112] The information generation model can be obtained by supervised learning training based on the model loss function, and the quality and context consistency of the alternative information can be optimized through the model loss function. total Can be:

[0113] L total =L semantic +L privacy +L perplexity ;

[0114] Among them, L semanticRepresents the semantic loss value, L privacy represents the privacy loss value, L perplexity Represents the fluency loss value.

[0115] The trained information generation model may include a model encoder and a model decoder. The model encoder may be used to obtain a vector representation of private information in a user entity text. The entity semantic vector output by the model encoder may include semantic information of the private information and contextual information of the private information in the user input text. The entity semantic vector is used to generate alternative information related to the private information. Based on the entity semantic vector, alternative information corresponding to the corresponding entity type of the private information but not containing the private information may be dynamically generated through the decoding process of the model decoder. After determining the alternative information, the privacy protection device based on the large model may replace the private information in the user input text with the corresponding alternative information according to the correspondence between the private information and the alternative information, and obtain a desensitized input text corresponding to the user input text.

[0116] Furthermore, the desensitized input text is input into the question-answering model, and the desensitized input text is inferred by the question-answering model to obtain a desensitized output text. According to the correspondence between the private information and the alternative information, the alternative information in the desensitized output text is restored to the private information to obtain a response text.

[0117] The privacy protection method based on a large model provided by the disclosed embodiment combines named entity recognition, information generation model, and recovery of private information to build a modular privacy protection reasoning process. It can protect the private information input by users in a variety of complex scenarios and optimize the generation effect of the question-answering model. Through the combination of named entity recognition and information replacement, user privacy protection is effectively achieved, while maintaining the consistency and integrity of contextual semantics, and improving the accuracy and naturalness of the reasoning results of the question-answering model.

[0118] In addition, it realizes the dynamic replacement of sensitive information, reduces the risk of directly exposing private information, improves the quality of the substitute information used in the reasoning process and its relevance to private information, improves the security of the response text generation process and the practicality of the result. In addition, through the effective combination of privacy protection and model loss function, it can flexibly determine the substitute information corresponding to private information in a variety of complex application scenarios, realizes efficient and privacy-preserving model reasoning, and significantly enhances the application breadth and user trust of the question-answering model.

[0119] Figure 4 The structure diagram of a privacy protection device based on a large model provided by an embodiment of the present disclosure is shown in FIG. 1 , which can be implemented by software and / or hardware, or can be integrated in an electronic device. Figure 4 As shown, the privacy protection device based on the large model includes:

[0120] An acquisition module 401 is used to acquire private information in a text input by a user and an entity type identifier of the private information;

[0121] A processing module 402 is used to perform context-based encoding processing and type-based decoding processing on the private information according to the user input text and the entity type identifier, so as to obtain replacement information corresponding to the private information;

[0122] The question-and-answer module 403 is used to perform desensitized question-and-answering based on a question-and-answer model on the user input text based on the corresponding relationship between the private information and the replacement information, and restore the private information to generate a response text for the user input text.

[0123] Optionally, the acquisition module 401 is used to:

[0124] The user input text is input into the trained entity recognition model to obtain the location identifier and entity type identifier of the private information output by the entity recognition model.

[0125] Optionally, the location identifier includes a first location identifier and a second location identifier, the first character identifier is determined based on the first location identifier and the entity type identifier, the first character identifier is used to mark the first character of the private information in the user input text, and the second location identifier is used to mark subsequent characters of the private information in the user input text.

[0126] Optionally, the processing module 402 is used to:

[0127] Inputting the user input text and the private information into a model encoder of a trained information generation model to obtain an entity semantic vector output by the model encoder;

[0128] The entity semantic vector and the entity type identifier are input into a model decoder of the information generation model to obtain the replacement information output by the model decoder.

[0129] Optionally, the privacy protection device based on the large model further includes a training module, and the training module is used to:

[0130] Acquire sample privacy information in the sample input text and a sample type identifier of the sample privacy information;

[0131] Inputting the sample input text and the sample privacy information into an intermediate encoder of an intermediate model to obtain a sample semantic vector output by the intermediate encoder;

[0132] Inputting the sample semantic vector and the sample type identifier into an intermediate decoder of the intermediate model to obtain sample replacement information output by the intermediate decoder;

[0133] The intermediate model is trained according to the model loss function, the sample privacy information, and the sample replacement information until the information generation model is obtained; wherein the model loss function is constructed based on at least one of the semantic loss function, the privacy loss function, and the fluency loss function.

[0134] Optionally, the semantic loss value in the semantic loss function is the vector distance between the vector of the sample privacy information and the vector of the sample replacement information; the privacy loss value in the privacy loss function is the cross entropy value of the discriminant probability, the discriminant probability represents the probability that the sample replacement information contains the sample privacy information, and the discriminant probability is determined by a pre-set discriminator model; the fluency loss value in the fluency loss function is the perplexity of the sample desensitized input text corresponding to the sample replacement information.

[0135] Optionally, the question-answering module 403 is used to:

[0136] Replacing the private information in the user input text with the corresponding replacement information to obtain a desensitized input text corresponding to the user input text;

[0137] Input the desensitized text into the question-answering model to obtain the desensitized output text output by the question-answering model;

[0138] If the desensitized output text contains the replacement information, the replacement information in the desensitized output text is replaced with the corresponding private information to obtain the response text.

[0139] It should be noted that Figure 4 The privacy protection device 400 based on a large model shown can execute each step in the above-mentioned privacy protection method embodiment based on a large model, and realize each process and effect in the above-mentioned privacy protection method embodiment based on a large model, which will not be described in detail here.

[0140] Figure 5 The structure diagram of an electronic device provided by the embodiment of the present disclosure is shown in FIG. Figure 5 As shown, the electronic device 500 includes one or more processors 501 and a memory 502 .

[0141] The processor 501 may be a central processing unit (CPU) or other forms of processing units having large model-based privacy protection capabilities and / or instruction execution capabilities, and may control other components in the electronic device 500 to perform desired functions.

[0142] The memory 502 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory (cache), etc. The non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 501 may run the program instructions to implement the privacy protection method based on the large model of the embodiment of the present disclosure described above and / or other desired functions. Various contents such as input signals, signal components, noise components, etc. may also be stored in the computer-readable storage medium.

[0143] In one example, the electronic device 500 may further include: an input device 503 and an output device 504 , and these components are interconnected via a bus system and / or other forms of connection mechanisms (not shown).

[0144] In addition, the input device 503 may also include, for example, a keyboard, a mouse, and the like.

[0145] The output device 504 can output various information to the outside, including the determined distance information, direction information, etc. The output device 504 can include, for example, a display, a speaker, a printer, a communication network and a remote output device connected thereto, and the like.

[0146] Of course, to simplify, Figure 5 Only some of the components related to the present disclosure in the electronic device 500 are shown, and components such as a bus, an input / output interface, etc. are omitted. In addition, according to specific application situations, the electronic device 500 may also include any other appropriate components.

[0147] In addition to the above-mentioned method and device, the embodiment of the present disclosure may also be a computer program product, which includes computer program instructions, which, when executed by a processor, enable the processor to execute the privacy protection method based on a large model provided by the embodiment of the present disclosure.

[0148] The computer program product may be written in any combination of one or more programming languages ​​to write program code for performing the operations of the disclosed embodiments, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0149] In addition, the embodiment of the present disclosure may also be a computer-readable storage medium on which computer program instructions are stored. When the computer program instructions are executed by a processor, the processor executes the privacy protection method based on a large model provided by the embodiment of the present disclosure.

[0150] The computer readable storage medium can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can include, for example, but is not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0151] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0152] The above description is only a specific embodiment of the present disclosure, so that those skilled in the art can understand or implement the present disclosure. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure will not be limited to the embodiments described herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A privacy protection method based on a large model, characterized in that: include: Obtaining private information in the user input text and an entity type identifier of the private information; According to the user input text and the entity type identifier, performing context-based encoding processing and type-based decoding processing on the private information to obtain replacement information corresponding to the private information; Based on the correspondence between the private information and the replacement information, the user input text is subjected to desensitized question and answer based on a question and answer model and the private information is restored to generate a response text for the user input text.

2. The method according to claim 1, characterized in that The step of obtaining the private information in the text input by the user and the entity type identifier of the private information includes: The user input text is input into the trained entity recognition model to obtain the location identifier and entity type identifier of the private information output by the entity recognition model.

3. The method according to claim 2, characterized in that The position identifier includes a first position identifier and a second position identifier, the first character identifier is determined based on the first position identifier and the entity type identifier, the first character identifier is used to mark the first character of the private information in the user input text, and the second position identifier is used to mark subsequent characters of the private information in the user input text.

4. The method according to claim 1, characterized in that: The performing context-based encoding processing and type-based decoding processing on the private information according to the user input text and the entity type identifier to obtain replacement information corresponding to the private information includes: Inputting the user input text and the private information into a model encoder of a trained information generation model to obtain an entity semantic vector output by the model encoder; The entity semantic vector and the entity type identifier are input into a model decoder of the information generation model to obtain the replacement information output by the model decoder.

5. The method according to claim 1, characterized in that The training process of the information generation model includes: Acquire sample privacy information in the sample input text and a sample type identifier of the sample privacy information; Inputting the sample input text and the sample privacy information into an intermediate encoder of an intermediate model to obtain a sample semantic vector output by the intermediate encoder; Inputting the sample semantic vector and the sample type identifier into an intermediate decoder of the intermediate model to obtain sample replacement information output by the intermediate decoder; The intermediate model is trained according to the model loss function, the sample privacy information, and the sample replacement information until the information generation model is obtained; wherein the model loss function is constructed based on at least one of the semantic loss function, the privacy loss function, and the fluency loss function.

6. The method according to claim 5, characterized in that The semantic loss value in the semantic loss function is the vector distance between the vector of the sample private information and the vector of the sample replacement information; The privacy loss value in the privacy loss function is a cross entropy value of a discriminant probability, wherein the discriminant probability represents a probability that the sample replacement information contains the sample privacy information, and the discriminant probability is determined by a preset discriminator model; The fluency loss value in the fluency loss function is the perplexity of the sample desensitized input text corresponding to the sample replacement information.

7. The method according to claim 1, characterized in that The method of performing desensitizing question and answering based on a question and answer model on the user input text and restoring the private information based on the correspondence between the private information and the replacement information, and generating a response text for the user input text, includes: Replacing the private information in the user input text with the corresponding replacement information to obtain a desensitized input text corresponding to the user input text; Input the desensitized input text into the question-answering model to obtain a desensitized output text output by the question-answering model; If the desensitized output text contains the replacement information, the replacement information in the desensitized output text is replaced with the corresponding private information to obtain the response text.

8. A privacy protection device based on a large model, characterized in that: include: An acquisition module, used to acquire the private information in the text input by the user and the entity type identifier of the private information; A processing module, configured to perform context-based encoding processing and type-based decoding processing on the private information according to the user input text and the entity type identifier, so as to obtain replacement information corresponding to the private information; The question-and-answer module is used to perform desensitized question-and-answering based on a question-and-answer model on the user input text based on the corresponding relationship between the private information and the replacement information, and restore the private information to generate a response text for the user input text.

9. An electronic device, characterized in that: The electronic device comprises: processor; a memory for storing instructions executable by the processor; The processor is used to read the executable instructions from the memory and execute the instructions to implement the privacy protection method based on a large model as described in any one of claims 1-7 above.

10. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and the computer program is used to execute the privacy protection method based on a large model as described in any one of claims 1 to 7.

Images