Natural language anonymization method, device and storage medium using large language model

Abstract

The present application discloses a natural language anonymization method, device and storage medium using a large language model, which relates to artificial intelligence technology, including: obtaining original text data and configuring prompt words; based on the original text data and the associated prompt words, using the LLM to extract relevant subject information and non-subject information in the original text data; using the LLM to remove the subject information contained in the non-subject information to complete the anonymization of the non-subject information; and anonymizing the subject information; based on the anonymized subject information and the non-subject information, using the LLM to merge the anonymized subject information and the non-subject information into natural language. The present application provides an effective method for desensitizing text data, which can retain information and semantics to the greatest extent.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a natural language anonymization method, device and storage medium using a large language model. Background Art

[0002] Natural language-based models such as big models are all built on the training of a large number of raw text data sets. The raw data will inevitably contain sensitive information, so the protection of the big language model can be bypassed by people with ulterior motives to obtain sensitive data in the training set. To ensure security, it is necessary to desensitize the raw data before training and modeling. However, natural language texts have the characteristics of vague and abstract meanings and flexible and changeable sentence structures. How to effectively remove sensitive information from them while retaining the original sentence structure and semantics for subsequent use has become a major challenge.

[0003] For example, if there is a sentence like this: "Yesterday when I was walking by the lake, I ran into my neighbor Xiao Li on Su Causeway. He had just left home." Then we can infer the address information of this person, which is Xihu District, Hangzhou City. However, this kind of reasoning cannot be completed automatically using traditional programs, and naturally targeted desensitization cannot be performed.

[0004] Data anonymization is a technology that protects privacy and security, and is an important development direction for data security. However, text data anonymization is mainly aimed at unstructured data, involving technologies such as natural language processing and information extraction, which is relatively difficult.

[0005] Traditional data anonymization routes mainly include generalization and differentiation, and the principles are as follows:

[0006] Generalizing data. Some data elements are more easily associated with specific individuals than others. To protect the privacy of these specific individuals, generalization techniques are used to remove some of the relevant data or replace it with a common value. For example, generalization techniques may be used to replace all area code or phone number entries with the same sequence of numbers. Through generalization, we can achieve K-anonymity. For example, suppose there is a data set with K of 50 and the attribute is zip code. If we look at the data related to anyone in the data set, we will find that there are 49 other people with the same zip code. Therefore, we cannot identify the identity of anyone in the data set based on the zip code alone.

[0007] If all people in a dataset have the same value for a sensitive attribute, then knowing that these people belong to the dataset may reveal that sensitive information. To mitigate this risk, we may employ L-diversity. L-diversity is also an industry-standard term for the degree of diversity in sensitive values. For example, suppose a group of people all searched for the same sensitive health topic at the same time. If we look at this dataset, we will not know who searched for this topic (thanks to K-anonymity). However, since all people in this dataset have the same sensitive attribute (i.e., the topic of the query), there may still be a risk of privacy leakage. L-diversity means that the anonymized dataset will not only contain flu queries, but will contain flu queries as well as other queries to further protect user privacy.

[0008] Adding noise to the data, Differential privacy is a technique that adds mathematical noise to the data. If this technique is used, it is impossible to determine whether any individual belongs to a dataset, because the output of a given algorithm will look basically the same, whether it includes or does not include information about the corresponding individual. For example, suppose you are measuring the overall flu query trend in a certain geographic area. To achieve differential privacy, you can add noise to the dataset. This means that the number of people in a community searching for flu-related content can be increased or decreased, but doing so will not affect the measurement of this trend over a larger geographic area. For text information, you can use word embedding vectors and add differential privacy noise to the vector to form new vocabulary. However, adding noise to the data may make the data less useful. Summary of the invention

[0009] The embodiments of the present application provide a natural language anonymization method, device and storage medium using a large language model, providing an effective method for desensitizing text data, retaining information and semantics to the greatest extent.

[0010] The embodiment of the present application provides a natural language anonymization method using a large language model, including:

[0011] Obtaining original text data and configuring prompt words, wherein the configured prompt words are associated with tasks that instruct the large language model LLM to perform;

[0012] Based on the original text data and the associated prompt words, extracting relevant subject information and non-subject information in the original text data using the LLM;

[0013] According to the associated prompt words, the subject information directly or indirectly included in the non-subject information is removed by using the LLM to anonymize the non-subject information; and

[0014] Anonymize the subject information;

[0015] Based on the anonymized subject information and the non-subject information, the anonymized subject information and the non-subject information are merged into natural language using the LLM.

[0016] Optionally, based on the original text data and the associated prompt words, extracting relevant subject information and non-subject information from the original text data using the LLM includes:

[0017] Based on the original text data and the associated prompt words, the LLM is instructed to extract relevant subject information and non-subject information in the original text data according to a set format.

[0018] Optionally, before anonymizing the subject information and the non-subject information, the method further includes:

[0019] Based on the non-subject information determined by the LLM, the LLM is used to perform reasoning based on the associated prompt words to supplement the subject information.

[0020] Optionally, anonymizing the subject information includes:

[0021] Desensitizing the subject information; and,

[0022] The desensitized subject information is anonymized using an anonymization algorithm.

[0023] Optionally, anonymizing the subject information further includes:

[0024] An outlier value is set for each attribute in the subject information to limit the attribute to a set range and eliminate abnormal attribute values.

[0025] The embodiment of the present application further proposes a natural language anonymization device using a large language model, including a processor configured to:

[0026] Obtaining original text data and configuring prompt words, wherein the configured prompt words are associated with tasks that instruct the large language model LLM to perform;

[0027] Based on the original text data and the associated prompt words, extracting relevant subject information and non-subject information in the original text data using the LLM;

[0028] According to the associated prompt words, the subject information directly or indirectly included in the non-subject information is removed by using the LLM to anonymize the non-subject information; and

[0029] Anonymize the subject information;

[0030] Based on the anonymized subject information and the non-subject information, the anonymized subject information and the non-subject information are merged into natural language using the LLM.

[0031] Optionally, the processor is specifically configured to:

[0032] Based on the non-subject information determined by the LLM, the LLM is used to perform reasoning based on the associated prompt words to supplement the subject information.

[0033] Optionally, the processor is specifically configured to:

[0034] Desensitizing the subject information; and,

[0035] The desensitized subject information is anonymized using an anonymization algorithm

[0036] Optionally, the processor is specifically configured to:

[0037] An outlier value is set for each attribute in the subject information to limit the attribute to a set range and eliminate abnormal attribute values.

[0038] The embodiment of the present application further proposes a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the natural language anonymization method using a large language model as described above are implemented.

[0039] The embodiment of the present application proposes an effective method for desensitizing text data through a large language model and an anonymization method, thereby retaining information and semantics to the greatest extent.

[0040] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present application. Also, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:

[0042] Figure 1 The following is a basic flow chart of the natural language anonymization method according to an embodiment of the present application;

[0043] Figure 2 The following is a schematic diagram of the overall process of the natural language anonymization method according to an embodiment of the present application. DETAILED DESCRIPTION

[0044] The exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0045] Traditional anonymization methods usually target tabular data, but tabular data can generate false data, while text data cannot generate false "words", and encryption methods may make text data difficult to apply.

[0046] Usually, text anonymization cannot completely defend against re-identification attacks, membership inference attacks, and other attacks because the output words are related to the original words.

[0047] Traditional rule-based and perturbation-based methods inevitably affect the original semantics and reduce the usability of the text.

[0048] The embodiment of the present application provides a natural language anonymization method using a large language model. The method of the embodiment of the present application uses the understanding and reasoning ability of the large language model to complete the parsing and anonymization of the subject sensitive information in the text. In the text database, there are sentences of various styles. After reading, the large model can be used to format the output. For example, the text content can be classified into a table, and then different columns can be anonymized separately. Specifically, Figure 1 , Figure 2 As shown, the method of the embodiment of the present application includes the following steps:

[0049] In step S101, the original text data is obtained, and prompt words are configured, wherein the configured prompt words are associated with the tasks that the large language model LLM is instructed to perform. In some specific examples, the configuration of prompt words is determined according to the tasks that the large language model LLM needs to be instructed to perform, that is, the prompt words are guiding texts provided when interacting with the LLM dialogue system. In the case where LLM is required to cooperate in performing different tasks, the corresponding prompt words can be configured. Write clear and unambiguous prompt words to accurately express your intentions and questions and obtain more accurate answers from the large model. Prompt words can be manually input, preset by the large language model, or generated by the corresponding extraction model.

[0050] In step S102, based on the original text data and the associated prompt words, the LLM is used to extract the relevant subject information and non-subject information in the original text data. In some embodiments, based on the original text data and the associated prompt words, the LLM is used to extract the relevant subject information and non-subject information in the original text data, including: based on the original text data and the associated prompt words, instructing the LLM to extract the relevant subject information and non-subject information in the original text data according to a set format.

[0051] In a specific example, the subject information may be a feature or information that can be associated with a specific individual. Non-subject information may be a feature or information that cannot be associated with a specific individual. LLM can be used to format and extract the original text data according to the instructions of the prompt words. For example, LLM can be required to summarize the text content and fill it into a specified table. The table column names include two categories: subject information and non-subject information. In some examples, the subject information may include: name, gender, zip code, address, mobile phone number, ID number, email address, xxx, etc. Non-subject information may be the content remaining after the subject information is removed from the original text data.

[0052] In a specific example, the format can be set to use a table, for example, and the main information column name needs to be selected in combination with the specific characteristics of the data set. Taking "Yesterday when I was walking by the lake, I ran into my neighbor Xiao Li at Sudi, who had just left home" as an example, the table extraction part outputs the following table table0 (the name Xiao Li is deleted from the non-main information column):

[0053]

[0054] In step S103, according to the associated prompt words, the subject information directly or indirectly included in the non-subject information is removed by using the LLM to anonymize the non-subject information. In a specific example, for the anonymization task of non-subject information, according to the prompt words in the aforementioned embodiment, the large model is prompted to delete the corresponding subject information or the content that can infer the subject information from the non-subject information, and the non-subject information table is obtained as follows (LLM will automatically perform deletion according to the content based on its own reasoning, for example, "just left home" is very important for address reasoning, so LLM chooses to delete the content):

[0055]

[0056] After the non-entity information column is anonymized, it can be extracted as table_non_entity.

[0057] In step S104, the subject information is also anonymized. Specifically, for the anonymization of the subject information, for example, the subject information column obtained by processing can be extracted, for example, as table_entity, and then the table_entity is anonymized. The execution order of this step and step S103 is not limited, and the subject information can be anonymized first.

[0058] In step S105, based on the anonymized subject information and the non-subject information, the LLM is used to merge the anonymized subject information and the non-subject information into natural language. For example, the anonymized table_entity and table_non_entity are simultaneously input into the LLM, and the subject information and non-subject information of the same row are merged to restore the natural language. In this way, completely anonymized text information can be obtained, and the sentence structure and semantics are also retained to the greatest extent, which can support related data applications and model training.

[0059] The embodiment of the present application proposes an effective method for desensitizing text data through a large language model and an anonymization method, thereby retaining information and semantics to the greatest extent.

[0060] In some embodiments, before anonymizing the subject information and the non-subject information, the method further includes:

[0061] Based on the non-subject information determined by the LLM, the LLM is used to perform reasoning again based on the associated prompt words to supplement the subject information.

[0062] Still taking the table0 in the aforementioned embodiment as an example, after obtaining the table0, part of the main information is hidden in the non-main information. In the embodiment of the present application, further, the associated prompt words are used to instruct LLM to perform the table supplement task. Specifically, the associated prompt words and the non-main information columns in table0 can be used to prompt LLM to infer the main information of each column. For example, for the table0, the name (gender / zip code / address / mobile phone number / ID number / email address / xxx) column is supplemented. After multiple inferences, the table table_complement is obtained as follows:

[0063]

[0064] Based on the table table_complement, anonymization of the non-subject information column in step S103 is performed again.

[0065] In some embodiments, anonymizing the subject information includes:

[0066] Desensitize the subject information. In a specific example, based on the table table_complement, the subject information column can be extracted to obtain table_entity. This part of information is directly related to the person's information and has high sensitivity. In the embodiment of the present application, these attributes are desensitized, for example, by using methods such as masking and hashing, which shall comply with relevant regulations.

[0067] The desensitized subject information is anonymized using an anonymization algorithm. Specifically, after the initial desensitization is completed, the subject information in table_entity is further anonymized using a method such as k-anonymity. The Mondrian algorithm based on the greedy anonymization algorithm can also be used. In addition, combined with the specific circumstances of the project, operations such as l-diversification and t-approximation can also be performed on table_entity as needed.

[0068] In some embodiments, anonymizing the subject information further includes:

[0069] Outliers are set for each attribute in the subject information to limit the attribute to a set range and eliminate abnormal attribute values. In a specific example, in response to the outlier problem, the attributes of each individual in the data set are limited to a specific range in the embodiment of the present application, thereby eliminating outliers in the data set.

[0070] The method of the embodiment of the present application is designed to extract and delete some information that is explicitly or implicitly present in the text and can point to a specific individual. The understanding ability and structured output ability of the large model are used to extract explicit sensitive information, the reasoning ability is used to complete the extraction of implicit sensitive information, and the text generation ability of the large model is used to keep the semantics of the sentence unchanged. The method of the present application has excellent anonymization performance, making the text information more secure, and will not seriously damage the usability of the text.

[0071] The embodiment of the present application further proposes a natural language anonymization device using a large language model, including a processor configured to:

[0072] Obtain the original text data and configure prompt words, wherein the configured prompt words are associated with the tasks that the large language model LLM is instructed to perform. In some specific examples, the configuration of prompt words is determined according to the tasks that the large language model LLM needs to be instructed to perform, that is, the prompt words are guiding texts provided when interacting with the LLM dialogue system. In the case where LLM is required to cooperate in performing different tasks, it is sufficient to configure the corresponding prompt words. Write clear and unambiguous prompt words to accurately express your intentions and questions and obtain more accurate answers from the large model. Prompt words can be manually input, preset by the large language model, or generated by the corresponding extraction model.

[0073] Based on the original text data and the associated prompt words, the LLM is used to extract the relevant subject information and non-subject information in the original text data. In a specific example, the subject information may be a feature or information that can be associated with a specific individual. Non-subject information may be a feature or information that cannot be associated with a specific individual. LLM may be used to format and extract the original text data according to the instructions of the prompt words. For example, LLM may be required to summarize the text content and fill it into a specified table. The table column names include two categories: subject information and non-subject information. In some examples, the subject information may include: name, gender, zip code, address, mobile phone number, ID number, email address, xxx, etc. Non-subject information may be the content remaining after the subject information is removed from the original text data. In a specific example, the set format may be, for example, a table, and the subject information column name needs to be selected in combination with the specific characteristics of the data set.

[0074] For the non-subject information, according to the associated prompt words, the LLM is used to remove the subject information directly or indirectly contained in the non-subject information to anonymize the non-subject information, and anonymize the subject information. In a specific example, for the anonymization task of the non-subject information, according to the prompt words in the aforementioned embodiment, the large model is prompted to delete the corresponding subject information from the non-subject information or to infer the content of the subject information.

[0075] Based on the anonymized subject information and the non-subject information, the LLM is used to merge the anonymized subject information and the non-subject information into natural language. The anonymized subject information and non-subject information are simultaneously input into the LLM, and the subject information and non-subject information of the same row are merged to restore the natural language. In this way, completely anonymized text information can be obtained, and the sentence structure and semantics are also retained to the greatest extent, which can support related data applications and model training.

[0076] In some embodiments, the processor is specifically configured to:

[0077] Based on the non-subject information determined by the LLM, the LLM is used to perform reasoning again based on the associated prompt words to supplement the subject information.

[0078] In some embodiments, the processor is specifically configured to:

[0079] Desensitizing the subject information; and,

[0080] The desensitized subject information is anonymized using an anonymization algorithm

[0081] In some embodiments, the processor is specifically configured to:

[0082] An outlier value is set for each attribute in the subject information to limit the attribute to a set range and eliminate abnormal attribute values.

[0083] The embodiment of the present application further proposes a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the natural language anonymization method using a large language model as described above are implemented.

[0084] It should be noted that in the various embodiments of the present application, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "includes a ..." does not exclude the presence of other identical elements in the process, method, article or device including the element.

[0085] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0086] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0087] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

Claims

1. A natural language anonymization method using a large language model, characterized in that: include: Obtaining original text data and configuring prompt words, wherein the configured prompt words are associated with tasks that instruct the large language model LLM to perform; Based on the original text data and the associated prompt words, extract the relevant subject information and non-subject information in the original text data using the LLM; wherein, extracting the relevant subject information and non-subject information in the original text data using the LLM includes: formatting and extracting the original text data according to the prompt words using the LLM, wherein the subject information is a feature or information that can be associated with a specific individual, and the non-subject information is a feature or information that cannot be associated with a specific individual; According to the associated prompt words, the subject information directly or indirectly included in the non-subject information is removed by using the LLM to anonymize the non-subject information; and Anonymize the subject information; Based on the anonymized subject information and the non-subject information, using the LLM to merge the anonymized subject information and the non-subject information into natural language; Before anonymizing the subject information and the non-subject information, the method further includes: Based on the non-subject information determined by the LLM, the LLM is used to perform reasoning again based on the associated prompt words to supplement the subject information; Wherein, using the LLM to remove the subject information directly or indirectly contained in the non-subject information to anonymize the non-subject information includes: deleting the corresponding subject information or the content that can be inferred from the subject information from the non-subject information according to the prompt word prompt large model; The anonymization of the subject information includes: Desensitizing the subject information; and, The desensitized subject information is anonymized using an anonymization algorithm.

2. The natural language anonymization method using a large language model as claimed in claim 1, characterized in that: Based on the original text data and the associated prompt words, extracting relevant subject information and non-subject information from the original text data using the LLM includes: Based on the original text data and the associated prompt words, the LLM is instructed to extract relevant subject information and non-subject information in the original text data according to a set format.

3. The natural language anonymization method using a large language model as claimed in claim 1, characterized in that: Anonymizing the subject information also includes: An outlier value is set for each attribute in the subject information to limit the attribute to a set range and eliminate abnormal attribute values.

4. A natural language anonymization device using a large language model, characterized in that: comprising a processor configured to: Obtaining original text data and configuring prompt words, wherein the configured prompt words are associated with tasks that instruct the large language model LLM to perform; Based on the original text data and the associated prompt words, extract the relevant subject information and non-subject information in the original text data using the LLM; wherein, extracting the relevant subject information and non-subject information in the original text data using the LLM includes: formatting and extracting the original text data according to the prompt words using the LLM, wherein the subject information is a feature or information that can be associated with a specific individual, and the non-subject information is a feature or information that cannot be associated with a specific individual; According to the associated prompt words, the subject information directly or indirectly included in the non-subject information is removed by using the LLM to anonymize the non-subject information; and Anonymize the subject information; Based on the anonymized subject information and the non-subject information, using the LLM, merging the anonymized subject information and the non-subject information into natural language; The processor is specifically configured to: Based on the non-subject information determined by the LLM, the LLM is used to perform reasoning again based on the associated prompt words to supplement the subject information; Wherein, using the LLM to remove the subject information directly or indirectly contained in the non-subject information to anonymize the non-subject information includes: deleting the corresponding subject information or the content that can be inferred from the subject information from the non-subject information according to the prompt word prompt large model; The anonymization of the subject information includes: Desensitizing the subject information; and, The desensitized subject information is anonymized using an anonymization algorithm.

5. The natural language anonymization device using a large language model as claimed in claim 4, characterized in that: The processor is specifically configured to: An outlier value is set for each attribute in the subject information to limit the attribute to a set range and eliminate abnormal attribute values.

6. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the natural language anonymization method using a large language model as claimed in any one of claims 1 to 3.

Images