Privacy protection text generation method and system based on large language model

Abstract

The invention discloses a privacy protection text generation method and system based on a large language model. The method comprises the following steps: presetting a sensitive word library with sensitive word scores; performing sensitive word matching on each word in the input text and the sensitive word library, calculating sensitivity scores of the matched sensitive words, and summing the sensitivity scores to obtain a total sensitivity score of the input text; combining the total sensitivity score with a grading threshold to obtain a sensitivity level of the input text; establishing a mapping function between the sensitivity level and a differential privacy parameter, and automatically generating the differential privacy parameter in a differential privacy mechanism according to the sensitivity level; and adaptively controlling the privacy protection intensity of the text generated by the large language model by using a differential privacy parameter corresponding to the sensitivity level. According to the method and the system disclosed by the invention, the privacy protection intensity can be effectively adjusted when different sensitive information is coped with, the generation fluency is ensured to be kept in the low-sensitive content, stronger privacy protection is provided in the high-sensitive content, and accurate privacy control is realized.

Description

Technical Field

[0001] The present invention belongs to the field of artificial intelligence technology, and in particular relates to a privacy-preserving text generation method and system based on a large language model. Background Art

[0002] With the increasing application of large language models in intelligent question-answering, information retrieval, and generative content, privacy protection has become an important concern. Current technical solutions mainly focus on three aspects: de-identification of private data in the data preprocessing stage, privacy protection technologies in the model training process (such as federated learning, differential privacy, and encryption methods), and data protection and access rights control measures in the reasoning stage. These methods attempt to reduce the risk of user sensitive information exposure from different angles, but in actual applications, many solutions only implement protection in the training or reasoning stage, and lack real-time protection of generated content.

[0003] Currently, privacy protection technology for large language models is gradually maturing, but it is still mainly focused on static protection measures such as data encryption, adversarial feature extraction, and permission management. As user privacy needs increase, single static protection is difficult to cope with complex privacy leakage scenarios, and the demand for dynamic and adaptive privacy protection technology increases. However, most existing patents still have deficiencies in the accuracy of privacy protection and flexible user control, and are difficult to adapt to different privacy needs in diverse application scenarios.

[0004] Chinese patent document CN 118296654 A discloses a privacy protection method based on knowledge retrieval and context screening. By searching the knowledge base for the context of user input before generating answers, direct sensitive data is excluded to avoid direct exposure of private information to the model. The core idea is to ensure the privacy of generated content by screening and replacing user input. However, this scheme has limitations in terms of the degree of privacy protection. This scheme relies on a pre-set knowledge base and permission management, and cannot flexibly adjust the privacy protection intensity during the generation process according to the sensitivity of the input content. For content with different sensitivities, this method is difficult to achieve differentiated protection. This scheme fails to distinguish between low, medium and high protection levels according to the sensitivity of the content, and it is difficult to meet diverse privacy needs. The permissions and data replacement methods used are relatively fixed, which is not suitable for scenarios where users can set privacy levels independently. This method is suitable for known sensitive information scenarios, but it is difficult to fully cover privacy protection in complex scenarios such as open-ended questions and answers and personalized generation.

[0005] Chinese patent document CN 118394889 A discloses a method and device for federated fine-tuning of a large language model based on gradient compression. By combining gradient compression technology with federated fine-tuning, the efficiency of data transmission during model training is optimized, aiming to achieve privacy protection by compressing gradient data. This solution reduces communication overhead while ensuring privacy security in federated learning scenarios, but there are still shortcomings: this method is mainly used in the training phase of the model, and reduces the risk of data leakage through gradient encryption and compression, but does not cover privacy protection in the content generation phase. It is impossible to effectively protect the privacy needs in real-time generated content. The gradient compression of this solution does not dynamically adjust the protection strength according to the sensitivity of the input content during processing, and cannot achieve detailed control of differential privacy. Therefore, in practical applications, this solution is difficult to meet the adaptive protection needs for content with different sensitivities. This solution is suitable for privacy protection needs in distributed environments, but is difficult to apply in centralized or single system applications, limiting its extensiveness.

[0006] Chinese patent document CN 118690409 A discloses a privacy protection method, device, equipment, medium and program product for a large language model. It uses privacy attribution score matrix technology to identify the privacy neurons of each layer of the large language model and perform activation repair to reduce privacy leakage. Its advantage is to adjust the internal structure of the model, but it also has the following defects: the scheme relies on direct operations on model layer parameters and neurons, and protects privacy through matrix calculation and privacy neuron adjustment. The calculation complexity is high and may affect system performance. Since the privacy protection of this method is based on model internal parameter repair rather than real-time adaptive protection of the generated content itself, it is difficult to respond quickly to changes in the sensitivity of the input content. Especially in open generation tasks, this solution is difficult to ensure detailed privacy protection in the generated content. This patent document is effective in specific scenarios, but it is difficult to expand to a wider range of application scenarios. Its protection measures are too dependent on the model architecture and privacy attribution matrix, and its applicability to large language models of other structures is limited. Summary of the invention

[0007] In order to solve the privacy protection problem of the large language model when generating content, the present invention provides a privacy-preserving text generation method and system based on a large language model, designs an integrated differential privacy protection system, and can adjust the differential privacy parameters in real time according to the sensitivity of the user input content, and perform appropriate privacy processing when generating content, thereby effectively protecting user privacy while ensuring the accuracy of the answer.

[0008] The technical solutions adopted are as follows:

[0009] On the one hand, the present invention provides a privacy-preserving text generation method based on a large language model, which pre-sets a sensitive word library with sensitive word scores; performs sensitive word matching on each word in the input text with the sensitive word library, calculates and sums the sensitivity score of each matched sensitive word, and obtains a total sensitivity score of the input text; combines the total sensitivity score and the grading threshold to obtain the sensitivity level of the input text; establishes a mapping function between the sensitivity level and the differential privacy parameter, and automatically generates the differential privacy parameter in the differential privacy mechanism according to the sensitivity level; and uses the differential privacy parameter corresponding to the sensitivity level to adaptively control the privacy protection strength of the text generated by the large language model.

[0010] Furthermore, the method also includes using a semantic similarity method to detect implicit sensitive information present in the input text; assigning a corresponding sensitivity score to each matching explicit sensitive word in the input text and the detected implicit sensitive information to obtain a total sensitivity score; and dividing the input text into three sensitivity levels of low, medium, and high according to the total sensitivity score and a set grading threshold.

[0011] Preferably, the semantic similarity method is used to detect the implicit sensitive information in the input text, and the specific method is:

[0012] Use the word vector model to transform each input word x in the input text i Convert to word vector representation

[0013] Use the word vector model to embed all the words in the sensitive word library j Convert to word vector representation

[0014] Calculate for each input word x in the input text i All words in the sensitive word library j The cosine similarity sim(x i ,s j );

[0015] Set the similarity threshold δ, if sim(x i ,s j )≥δ, then the input word x is considered i It is used to imply sensitive information and assign a corresponding sensitivity score.

[0016] Preferably, Laplace noise is added to each word feature in the text generated by the large language model, and the differential privacy parameters in the differential privacy mechanism are used to control the added noise intensity.

[0017] Further preferably, when the input text is judged to be of high sensitivity level, the sensitive features in the text generated by the large language model after differential privacy parameter processing are fuzzified, and the degree of replacement of sensitive information is controlled by the fuzzification factor β in the fuzzification process. The formula used is:

[0018] x″ i =β·x i +(1-β)·T

[0019] Where: x″ i is the fuzzy feature, x i is a sensitive feature;

[0020] T is a fuzzy replacement value, which is a common word or a synonym;

[0021] β∈[0, 1] is the fuzzification factor, which is used to control the degree of fuzzification.

[0022] Preferably, the mapping function between the sensitivity level and the differential privacy parameter is established as follows:

[0023]

[0024] Where: α is a constant, S is the total sensitivity score of the input text, and ∈ is the differential privacy parameter.

[0025] On the other hand, the present invention also provides a privacy-preserving text generation system based on a large language model, which includes: a sensitive word library, and a sensitivity score corresponding to each sensitive word is preset;

[0026] A sensitivity analysis module, which matches the input text with sensitive words in combination with the sensitive word library, and assigns a corresponding sensitivity score to each matched sensitive word to obtain a total sensitivity score of the input text, and determines the sensitivity level of the input text according to the total sensitivity score and a preset grading threshold;

[0027] A privacy parameter adaptive adjustment module, which obtains the corresponding differential privacy parameters according to the mapping function between the constructed total sensitivity score and the differential privacy parameters;

[0028] The differential privacy generation module performs privacy protection intervention on the text content generated by the big data model based on the differential privacy parameters obtained by the privacy parameter adaptive adjustment module and then outputs the result.

[0029] Furthermore, the differential privacy generation module includes a noise adding module and a fuzzification processing module; the noise adding module adds Laplace noise to each word feature in the text generated by the large language model, and uses the differential privacy parameters in the differential privacy mechanism to control the noise intensity;

[0030] When the input text is determined to be of a high sensitivity level, the fuzzification processing module performs fuzzification processing on the sensitive features in the text generated by the large language model, and uses the fuzzification factor β in the fuzzification processing to control the degree of replacement of sensitive information.

[0031] Furthermore, the system also includes a user privacy mode control module, in which selectable high privacy mode, balanced mode and low privacy mode are set. The differential privacy parameter ∈ set in the high privacy mode is 0.1, and the fuzzification factor β is 0.9; the differential privacy parameter ∈ set in the balanced mode is 0.5, and the fuzzification factor β is 0.5; the differential privacy parameter ∈ set in the low privacy mode is 1.0, and the fuzzification factor β is 0.1.

[0032] The technical solution of the present invention has the following advantages:

[0033] A. The method and system of the present invention realize dynamic adaptive adjustment of privacy protection of generated content through the combination of sensitivity analysis module, privacy parameter adaptive adjustment module and differential privacy generation module. Different from the static protection mechanism in the prior art, the present invention can automatically adjust the differential privacy parameters and fuzzification factors according to the sensitivity score of the user input text to achieve flexible changes in protection strength. This adaptability ensures that the system can effectively adjust the privacy protection strength when dealing with different sensitive information, ensures the smoothness of generation in low-sensitivity content, provides stronger privacy protection in high-sensitivity content, and realizes precise privacy control.

[0034] B. The method and system of the present invention allow users to independently select different privacy modes (such as high privacy mode, balanced mode and low privacy mode) through the user privacy mode control module. Different modes correspond to different privacy protection parameters, so that users can independently adjust the privacy protection strength of generated content according to actual needs. Different from the existing technology which mostly adopts a single privacy protection strategy, the present invention provides users with a personalized selection space of multi-layer privacy protection modes, which can adapt to different application scenarios and privacy needs, thereby enhancing the user's sense of control over the system and the user experience.

[0035] C. The present invention adopts a dual protection mechanism combining differential privacy noise addition and fuzzification processing in the differential privacy generation module. For medium-sensitive content, the system mainly relies on noise addition to achieve differential privacy; for highly sensitive content, the system not only increases the noise intensity, but also replaces specific sensitive information through fuzzification to further enhance protection. The dual protection strategy of differential privacy and fuzzification processing overcomes the limitations of single noise or feature extraction in the prior art, can implement more efficient privacy protection for generated content in complex scenarios, and enhances the concealment and security of sensitive information. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the specific embodiments of the present invention, the drawings required for use in the specific embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0037] Figure 1 is a flow chart of the privacy protection text generation method provided by the present invention;

[0038] Figure 2 It is a system composition diagram provided by the present invention. DETAILED DESCRIPTION

[0039] The technical solution of the present invention will be described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0040] like Figure 1 As shown, the present invention provides a privacy-preserving text generation method based on a large language model, which specifically includes the following steps:

[0041] [S01] A sensitive word library with sensitive word scores is preset.

[0042] Sensitive word library S = {s 1 ,s 2 , ..., s m}, where each sensitive word s j There is a corresponding sensitivity score P(s j ). The sensitive word library contains multiple categories of sensitive words, such as name, address, contact information, ID number, financial information, etc. The scores of these words are graded based on the sensitivity of the privacy information. For example, financial information and ID number have higher sensitivity scores, while general personal information has lower sensitivity scores.

[0043] [S02] Match each word in the input text with the sensitive word library, calculate the sensitivity score of each matching sensitive word (mainly explicit sensitive words) and sum them up to obtain the total sensitivity score of the input text.

[0044] First, the input text content X = {x 1 , x 2 , …, x n} is split into a set of words or phrases, denoted by x i , where each x iRepresents an independent word or phrase. This set is processed through word segmentation, stop word removal, punctuation removal and other preprocessing steps to obtain a structured text in order to improve the recognition accuracy of sensitive information.

[0045] Each word x of the input text X i Match the sensitive word library S one by one and calculate the sum of the sensitivity scores of direct matches:

[0046]

[0047] This score represents the total score of the explicit sensitive words matched in the input text. Each matched sensitive word has its corresponding score P(s j ) contributes a certain score, and the accumulation of all matching scores forms the basic score of sensitivity.

[0048] In addition to directly matching explicit sensitive words in the input text, in many cases, the content input by the user may not contain sensitive words in the sensitive word library, but may contain synonyms or semantically similar words. In order to detect such potential implicit sensitive information, the present invention further performs semantic similarity calculation to evaluate the semantic proximity between the input word and the words in the sensitive word library.

[0049] For each input word x i , first convert it into a word vector representation through a word vector model Similarly, all words in the sensitive word library are j Convert to word vector Next, calculate each x i and j The cosine similarity between:

[0050]

[0051] in, and Represents the vector representation of the input word and sensitive word respectively, and the cosine similarity sim(x i ,s j ) ranges from [-1,1]. The closer it is to 1, the more similar the semantics of the words are.

[0052] Set the similarity threshold δ to determine whether there is semantic similarity. i ,s j )≥δ, then x i is a potential sensitive word and is assigned a corresponding sensitivity score P(s j ). The total score of the semantic similarity part is calculated as follows:

[0053]

[0054] This score is used to assess the sensitivity of potential hidden sensitive information in the input content.

[0055] After completing the direct match score Score match and similarity score Score similarity After calculation of , these two scores are added together to form an overall sensitivity score for the input text:

[0056] TotalSensitiveScore=Score mathc +Score similarity

[0057] The design of this scoring method takes into account the dual impact of direct sensitive words and potential implicit sensitive information, making the sensitivity assessment more comprehensive and reasonable.

[0058] [S03] Combine the total sensitivity score and the grading threshold to obtain the sensitivity level of the input text.

[0059] The present invention divides the sensitivity of input text into three levels: low, medium, and high to meet different privacy protection requirements. The sensitivity level division is based on the set classification threshold T low , T medium and T high :

[0060] When Total Sensitivity Score <T low When , the input is judged as "low sensitivity", indicating that there is no significant risk of privacy leakage;

[0061] When T low ≤Total Sensitivity Score<T medium When the input is judged as "medium sensitive", the system is prompted to perform moderate privacy protection during the generation process;

[0062] When Total Sensitivity Score≥T high When the input is judged as "highly sensitive", the system needs to implement strict privacy protection measures.

[0063] Finally, based on the obtained total sensitivity score and the set grading threshold, the sensitivity level of the input content is classified into low, medium, and high for the subsequent setting of the differential privacy protection strength.

[0064]

S04

[0065] The present invention maps different sensitivity levels to specific privacy parameter values ​​by establishing a sensitivity-differential privacy parameter mapping table to ensure that the privacy protection strength is consistent with the sensitivity. For example, low-sensitivity content uses a smaller noise intensity to maintain content accuracy; while highly sensitive content uses a larger noise intensity to ensure the security of private data.

[0066] The goal of the adaptive adjustment of differential privacy parameters in the present invention is to use the differential privacy parameter ∈ in the differential privacy mechanism to control the noise intensity so as to adaptively adjust the protection intensity during the content generation process. According to the basic principle of differential privacy, when ∈ takes a larger value, there is less intervention in the output and the generated content is more accurate; when ∈ is smaller, the noise added to the generated content is larger, thereby enhancing privacy protection. Through regulation, automated privacy protection can be achieved at low, medium, and high sensitivities.

[0067] First, a mapping table between sensitivity level L and differential privacy parameter ∈ is defined to associate different sensitivities with specific privacy protection parameters. Assuming that the sensitivity level is divided into three levels: low sensitivity, medium sensitivity, and high sensitivity, the system sets corresponding differential privacy parameters for each sensitivity level. Specifically, the following relationship is defined:

[0068] L = {low sensitivity, medium sensitivity, high sensitivity}

[0069] ∈=f(L)

[0070] The mapping function f(·) is designed so that the higher the sensitivity, the smaller the differential privacy parameter ∈, in order to increase the noise intensity. The specific differential privacy parameter value can be determined through experiments and system configuration, and is set as:

[0071] ∈ 低 =1.0,∈ 中 =0.5,∈ 高 =0.1

[0072] This ensures stronger privacy protection for highly sensitive content.

[0073] In order to further realize adaptive differential privacy protection, the present invention calculates the corresponding differential privacy parameters based on the sensitivity score S of the input content. The sensitivity score S reflects the privacy of the content. The higher the score, the higher the privacy, and the stronger the privacy protection is required.

[0074] The adjustment formula of the differential privacy parameter is as follows:

[0075]

[0076] Among them, α is a constant used to control the influence of the sensitivity score S on the differential privacy parameter ∈. The higher the sensitivity score, the smaller the value of ∈, that is, the greater the protection. According to this formula, the method of the present invention can dynamically adjust the differential privacy parameter value for different sensitivity scores to ensure strict protection of highly sensitive content.

[0077] As a further preferred method of the present invention, in highly sensitive scenarios, in addition to noise intervention, the present invention also provides the calculation of a fuzzification factor β to further enhance the fuzzification of the content. In practical applications, the value of the fuzzification factor β can be calculated based on the differential privacy parameter ∈ as follows:

[0078] β=1-∈

[0079] When the sensitivity score is high, ∈ is small, and the fuzzification factor β is close to 1, indicating a strong degree of fuzzification. Based on the fuzzification factor β, explicit sensitive words or implicit information can be fuzzified to ensure the privacy of the generated content. For example, specific sensitive information can be replaced with a general expression to reduce the risk of privacy leakage.

[0080] Therefore, the differential privacy parameter ∈ and the fuzzification factor β are automatically generated according to the sensitivity level. For sensitivity scores in different intervals, appropriate differential privacy parameter values ​​are assigned respectively, forming the following rules:

[0081]

[0082] At each sensitivity level, a corresponding obfuscation factor β is also generated to ensure appropriate privacy protection and content clarity.

[0083]

S05

[0084] After determining the value of ∈, the present invention generates noise according to the Laplace distribution to interfere with the generated content. Assume that the generated answer content is represented by a vector X=(x 1 , x 2 , ..., x n ), where x i is the feature representation of the i-th word or phrase. Then the formula for adding Laplace noise is as follows:

[0085]

[0086] Among them, Δf represents the sensitivity of the query function f, that is, the influence on the generated content. By setting the size of ∈, the distribution intensity of the noise vector Y is controlled, and differential privacy intervention on the generated content is achieved.

[0087] The present invention intervenes in the privacy of the generated content by adjusting the differential privacy parameter ∈ and the fuzzification factor β, so that the generated text can significantly reduce the risk of privacy leakage when it contains sensitive information. By adaptively adjusting the noise and fuzzification degree, the module maintains the coherence and accuracy of the content while protecting privacy.

[0088] Based on the text content input by the user, a preliminary answer is first generated through a large language model. This preliminary answer is used for subsequent privacy intervention operations. The initially generated content will then be processed for privacy protection according to the privacy parameters to ensure that the generated content meets the protection standards of differential privacy.

[0089] After obtaining the preliminary answer X, the present invention selects an appropriate privacy intervention strategy based on the adaptively adjusted differential privacy parameters. Privacy protection is achieved through two methods: noise addition and fuzzification. The specific steps are as follows:

[0090] (1) Noise addition

[0091] In order to meet the requirements of differential privacy, the present invention adds Laplace noise to the generated content. The noise intensity is controlled by the privacy parameter ∈, and the distribution of the noise satisfies the Laplace distribution Lap(λ), where Here, Δf represents the sensitivity of the generated content, that is, the degree of influence of a small change in the input data on the generated content.

[0092] For each feature x i , the feature x′ after adding noise i It is expressed as:

[0093] x′ i =x i +Y i

[0094] in: Right now:

[0095]

[0096] Specifically, the generation of Laplace noise satisfies the following formula:

[0097]

[0098] Among them, U~Uniform(0,1) is a uniformly distributed random variable.

[0099] By i Add Laplace noise Y i , and the answer vector X′=(x′ 1 , x′ 2 , ..., x′ n), the vector is moderately noise-intervened, so that the privacy of the generated content is enhanced.

[0100] (2) Fuzzy processing

[0101] For content with a high sensitivity level, privacy may not be ensured by simply adding noise. Therefore, when content is marked as highly sensitive, the present invention further applies fuzzification. Fuzzification controls the degree of replacement of specific content by using a fuzzification factor β, making the sensitive information of the generated content more concealed.

[0102] Let the sensitive feature in the initial answer be x i , fuzzy processing will i Convert to fuzzy feature x″ i , the formula is:

[0103] x″ i =β·x i +(1-β)·T

[0104] Where: T is the fuzzy replacement value, which can be a common word or a synonym; β∈[0,1] is the fuzzification factor, which controls the degree of fuzzification. Usually, β for highly sensitive content takes a smaller value to ensure that sensitive information is fuzzy expressed or replaced.

[0105] Finally, for feature x i , its privacy-preserving output is determined based on the following formula:

[0106]

[0107] Where: x′ i It is the feature after adding noise, which is suitable for low or medium sensitivity content; while x″ i This is a blurred feature, suitable for highly sensitive content.

[0108] After completing the noise addition and blurring process, the final processed answer vector is output The vector is differentially private protected, and sensitive information has been effectively processed through noise or blurring, thereby ensuring that user privacy information is not directly leaked during the content generation process.

[0109] By adding noise and blurring, the method of the present invention can effectively reduce the risk of sensitive information leakage. Noise addition adaptively controls the privacy protection strength of the generated content according to the Laplace distribution, and blurring provides further protection for highly sensitive information, ensuring the security of user privacy. The method of the present invention effectively balances the privacy and accuracy of the generated content.

[0110] like Figure 2As shown, the present invention also provides a privacy-preserving text generation system based on a large language model, which includes: a sensitive word library, a sensitivity analysis module, a privacy parameter adaptive adjustment module and a differential privacy generation module, wherein in the sensitive word library, each sensitive word presets a corresponding sensitivity score; the sensitivity analysis module performs sensitive word matching on the input text in combination with the sensitive word library, and assigns a corresponding sensitivity score to each matched sensitive word to obtain a total sensitivity score of the input text, and determines the sensitivity level of the input text according to the total sensitivity score and a preset grading threshold; the privacy parameter adaptive adjustment module obtains corresponding differential privacy parameters and fuzzification factors according to a mapping function between the constructed total sensitivity score and the differential privacy parameters, which are used for intervention processing of the subsequently generated text content; the differential privacy generation module performs privacy protection intervention on the text content generated by the big data model based on the differential privacy parameters and fuzzification factors obtained by the privacy parameter adaptive adjustment module, and then outputs it.

[0111] Specifically, the differential privacy generation module includes a noise addition module and a fuzzification processing module; the noise addition module adds Laplace noise to each word feature in the text generated by the large language model, and uses the differential privacy parameters in the differential privacy mechanism to control the noise intensity; when the input text is judged to be a high sensitivity level, the fuzzification processing module fuzzifies the sensitive features in the text generated by the large language model, and uses the fuzzification factor β in the fuzzification processing to control the degree of replacement of sensitive information.

[0112] As a further preferred embodiment of the present invention, the system also includes a user privacy mode control module, in which selectable high privacy mode, balanced mode and low privacy mode are set. The differential privacy parameter ∈ set in the high privacy mode is 0.1, and the fuzzification factor β is 0.9; the differential privacy parameter ∈ set in the balanced mode is 0.5, and the fuzzification factor β is 0.5; the differential privacy parameter ∈ set in the low privacy mode is 1.0, and the fuzzification factor β is 0.1. Of course, the present invention is not limited to the three privacy modes set above, and the differential privacy parameters and fuzzification factors set in each privacy mode can also be changed to different values ​​according to specific circumstances, which will not be repeated here.

[0113] The sensitivity analysis module first matches the input text with sensitive words through the preset sensitive word library; then, it uses semantic similarity calculation to analyze the possible implicit sensitive information, and assigns a corresponding sensitivity score to each matched explicit sensitive word or detected implicit sensitive content; finally, the sensitivity analysis module divides the input content into three sensitivity levels: low, medium, and high according to the scoring results and the set grading threshold, so that the downstream module can decide the privacy protection strength.

[0114] The privacy parameter adaptive adjustment module is the core module of this system to achieve dynamic privacy protection. It aims to adjust the differential privacy parameters according to the results of the sensitivity analysis module, so as to apply appropriate privacy protection when generating content.

[0115] The differential privacy generation module implements differential privacy protection through Laplace noise and fuzzification strategies when generating content, which meets the user's needs for privacy data protection and is suitable for real-time privacy protection in various application scenarios.

[0116] Anything not described in the present invention is applicable to the prior art.

[0117] Obviously, the above embodiments are merely examples for the purpose of clear explanation, and are not intended to limit the implementation methods. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the implementation methods here. However, the obvious changes or modifications derived therefrom are still within the scope of protection of the present invention.

Claims

1. A privacy-preserving text generation method based on a large language model, characterized in that: A sensitive word library with sensitive word scores is preset; each word in the input text is matched with the sensitive word library, the sensitivity score of each matching sensitive word is calculated and summed up to obtain the total sensitivity score of the input text; The sensitivity level of the input text is obtained by combining the total sensitivity score and the grading threshold. A mapping function between the sensitivity level and the differential privacy parameters is established, and the differential privacy parameters in the differential privacy mechanism are automatically generated according to the sensitivity level. The differential privacy parameters corresponding to the sensitivity level are used to adaptively control the privacy protection strength of the text generated by the large language model.

2. The privacy-preserving text generation method based on a large language model according to claim 1, characterized in that: The method further includes detecting the implicit sensitive information in the input text using a semantic similarity method; assigning a corresponding sensitivity score to each matched explicit sensitive word in the input text and the detected implicit sensitive information to obtain a total sensitivity score; According to the total sensitivity score and the set grading threshold, the input text is divided into three sensitivity levels: low, medium, and high.

3. The privacy-preserving text generation method based on a large language model according to claim 2, characterized in that: The semantic similarity method is used to detect the implicit sensitive information in the input text. The specific method is: Use the word vector model to transform each input word x in the input text i Convert to word vector representation Use the word vector model to embed all the words in the sensitive word library j Convert to word vector representation Calculate for each input word x in the input text i All words in the sensitive word library j The cosine similarity sim(x i ,s j ); Set the similarity threshold δ, if sim(x i ,s j )≥δ, then the input word x is considered i It is used to imply sensitive information and assign a corresponding sensitivity score.

4. The privacy-preserving text generation method based on a large language model according to claim 2, characterized in that: Laplace noise is added to each word feature in the text generated by the large language model, and the differential privacy parameters in the differential privacy mechanism are used to control the intensity of the added noise.

5. The privacy-preserving text generation method based on a large language model according to claim 4, characterized in that: When the input text is judged as a high sensitivity level, the sensitive features in the text generated by the large language model after differential privacy parameter processing are fuzzified, and the fuzzification factor β in the fuzzification process is used to control the degree of replacement of sensitive information. The formula used is: x" i =β·x i +(1-β)·T Where: x″ i is the fuzzy feature, x i is a sensitive feature; T is a fuzzy replacement value, which is a common word or a synonym; β∈[0, 1] is the fuzzification factor, which is used to control the degree of fuzzification.

6. The privacy-preserving text generation method based on a large language model according to claim 1, characterized in that: The mapping function between the established sensitivity level and the differential privacy parameter is as follows: Where: α is a constant, S is the total sensitivity score of the input text, and ∈ is the differential privacy parameter.

7. A privacy-preserving text generation system based on a large language model, characterized in that: These include: sensitive The vocabulary, which presets the sensitivity score for each sensitive word; A sensitivity analysis module, which matches the input text with sensitive words in combination with the sensitive word library, and assigns a corresponding sensitivity score to each matched sensitive word to obtain a total sensitivity score of the input text, and determines the sensitivity level of the input text according to the total sensitivity score and a preset grading threshold; A privacy parameter adaptive adjustment module, which obtains the corresponding differential privacy parameters according to the mapping function between the constructed total sensitivity score and the differential privacy parameters; The differential privacy generation module performs privacy protection intervention on the text content generated by the big data model based on the differential privacy parameters obtained by the privacy parameter adaptive adjustment module and then outputs the result.

8. The privacy-preserving text generation system based on a large language model according to claim 7, characterized in that: The differential privacy generation module includes a noise adding module and a fuzzification processing module; the noise adding module adds Laplace noise to each word feature in the text generated by the large language model, and uses the differential privacy parameters in the differential privacy mechanism to control the noise intensity; When the input text is determined to be of a high sensitivity level, the fuzzification processing module performs fuzzification processing on the sensitive features in the text generated by the large language model, and uses the fuzzification factor β in the fuzzification processing to control the degree of replacement of sensitive information.

9. The privacy-preserving text generation system based on a large language model according to claim 8, characterized in that: The system also includes a user privacy mode control module, in which selectable high privacy mode, balanced mode and low privacy mode are set. The differential privacy parameter ∈ set in the high privacy mode is 0.1, and the fuzzification factor β is 0.9; the differential privacy parameter ∈ set in the balanced mode is 0.5, and the fuzzification factor β is 0.5; the differential privacy parameter ∈ set in the low privacy mode is 1.0, and the fuzzification factor β is 0.1.

Images