Sensitive data processing method and system combining privacy score and dynamic threshold

Abstract

The application relates to the technical field of large language model privacy protection, and discloses a sensitive data processing method and system combining privacy scores and dynamic thresholds. The method comprises the following steps: obtaining user input text in a scene dimension, determining each word segmentation and each privacy word of the input text; based on the attention mechanism of a large language model, calculating a basic privacy score of the input text according to the privacy word weight of each word segmentation, calculating an associated risk enhancement score of the input text according to the privacy word co-occurrence frequency of each privacy word pair, and determining a comprehensive privacy score of the input text according to the basic privacy score and the associated risk enhancement score; obtaining historical comprehensive privacy score statistical data in the scene dimension, determining a dynamic threshold according to the privacy word occurrence frequency of the input text and the historical comprehensive privacy score statistical data; and performing corresponding operations on the input text according to the comparison result of the comprehensive privacy score and the dynamic threshold. The privacy safety of a user in the interaction process with a large language model can be effectively protected.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of large language model privacy protection, which can be applied in medical and financial fields, and particularly relates to a sensitive data processing method and system combining privacy scoring and dynamic threshold. BACKGROUND

[0002] With the rapid development of large language models (LLM), they have shown strong text generation and understanding capabilities in many fields such as finance and medicine. However, in the interaction process between users and large language models, the input text of users may contain a large amount of sensitive private information, such as bank card numbers, ID card numbers, disease diagnosis results, medical detection indicators, etc. Once this sensitive private information is processed or stored by the large language model, it may lead to serious privacy leakage risks, causing economic losses or health privacy leakage to users.

[0003] Currently, privacy protection for large language models mainly focuses on the model training stage, such as using differential privacy, federated learning, etc. to protect the privacy of training data. However, in the model inference stage, i.e. the interaction process between user input text and large language models, there are relatively few privacy protection measures. Some existing methods mainly perform simple keyword filtering or fuzzing processing on input text, which has the following shortcomings.

[0004] Keyword filtering limitations: traditional keyword filtering can only identify pre-set sensitive words and cannot accurately identify the context association risks of sensitive words, for example, "Mr. Zhang lives in the Chaoyang District of the Guomao Building" has no obvious sensitive words, but can be associated to infer the first six digits of the ID card number;

[0005] Lack of dynamic evaluation mechanism: traditional methods mostly use fixed thresholds to judge privacy risks, and fixed thresholds are prone to false positives or false negatives, and cannot be adaptively adjusted according to different scene dimensions and user historical behaviors;

[0006] Not making full use of the attention mechanism of large language models, which cannot evaluate privacy risks from the perspective of the attention degree of the model to the text.

[0007] Therefore, there is an urgent need for a method and system that can comprehensively consider the direct exposure risk, context association risk and model attention focus degree of sensitive data before the user input text enters the large language model, and realize dynamic threshold adjustment and hierarchical privacy protection. SUMMARY

[0008] To solve the above problems, the embodiments of the present application provide a sensitive data processing method and system combining privacy scoring and dynamic threshold.

[0009] The embodiments of the present application adopt the following technical solutions:

[0010] In a first aspect, the application provides a sensitive data processing method combining privacy score and dynamic threshold, comprising:

[0011] obtaining user input text in a scene dimension, performing word segmentation on the input text to obtain each word segment, and identifying each privacy word from each word segment according to a predefined privacy dictionary in the scene dimension;

[0012] calculating a basic privacy score of the input text according to the privacy word weight of each word segment based on the attention mechanism of a large language model, calculating an associated risk enhancement score of the input text according to the privacy word co-occurrence frequency of each privacy word pair, and determining a comprehensive privacy score of the input text according to the basic privacy score and the associated risk enhancement score;

[0013] obtaining historical comprehensive privacy score statistical data in the scene dimension, and determining a dynamic threshold according to the privacy word occurrence frequency of the input text and the historical comprehensive privacy score statistical data;

[0014] performing a corresponding operation on the input text according to the comparison result of the comprehensive privacy score and the dynamic threshold.

[0015] In a second aspect, the application also provides a sensitive data processing system combining privacy score and dynamic threshold, comprising:

[0016] a text splitting unit configured to obtain user input text in a scene dimension, perform word segmentation on the input text to obtain each word segment, and identify each privacy word from each word segment according to a predefined privacy dictionary in the scene dimension;

[0017] a privacy scoring unit configured to calculate a basic privacy score of the input text according to the privacy word weight of each word segment based on the attention mechanism of a large language model, calculate an associated risk enhancement score of the input text according to the privacy word co-occurrence frequency of each privacy word pair, and determine a comprehensive privacy score of the input text according to the basic privacy score and the associated risk enhancement score;

[0018] a threshold calculation unit configured to obtain historical comprehensive privacy score statistical data in the scene dimension, and determine a dynamic threshold according to the privacy word occurrence frequency of the input text and the historical comprehensive privacy score statistical data;

[0019] a processing operation unit configured to perform a corresponding operation on the input text according to the comparison result of the comprehensive privacy score and the dynamic threshold.

[0020] In a third aspect, the application also provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the sensitive data processing method combining privacy score and dynamic threshold when executing the computer program.

[0021] In a fourth aspect, the present application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the sensitive data processing method combining a privacy score and a dynamic threshold.

[0022] The above at least one technical solution adopted by the embodiments of the present application can achieve the following beneficial effects:

[0023] The present application first acquires the user input text in the scene dimension, performs word segmentation processing on the input text to obtain each segmented word, and identifies each privacy word from each segmented word according to the predefined privacy dictionary in the scene dimension; then, based on the attention mechanism of the large language model, the basic privacy score of the input text is calculated according to the privacy word weight of each segmented word, the associated risk enhancement score of the input text is calculated according to the privacy word co-occurrence frequency of each privacy word pair, and the comprehensive privacy score of the input text is determined according to the basic privacy score and the associated risk enhancement score; then, the historical comprehensive privacy score statistical data in the scene dimension is acquired, and the dynamic threshold is determined according to the privacy word appearance frequency of the input text and the historical comprehensive privacy score statistical data; finally, the corresponding operation is performed on the input text according to the comparison result of the comprehensive privacy score and the dynamic threshold.

[0024] The present application performs privacy scoring on the input text from the two dimensions of sensitive data exposure and privacy inference risk, which can comprehensively and accurately assess the privacy risk of the user input text, avoiding the limitations of single-dimensional evaluation.

[0025] The present application makes full use of the attention mechanism of the large language model to evaluate the privacy risk from the perspective of the model's attention to the text, which can more accurately identify the sensitive information that the model really pays attention to.

[0026] The present application calculates the dynamic threshold in combination with the historical data in the scene dimension and the privacy word appearance frequency of the input text, which can be adaptively adjusted according to different scenes and real-time behaviors of users, improving the accuracy and flexibility of privacy protection.

[0027] The present application takes different processing operations according to the comparison result of the privacy score and the dynamic threshold, realizes the hierarchical protection of sensitive data, and at the same time protects privacy as much as possible while ensuring the usability of the large language model.

[0028] The method proposed by the present application is suitable for high-privacy-sensitive scenes such as finance and medical treatment, and can effectively protect the privacy and security of users in the process of interacting with the large language model. BRIEF DESCRIPTION OF DRAWINGS

[0029] The drawings described herein are used to provide further understanding of the present application, and form a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application, and do not constitute an improper limitation on the present application. In the drawings:

[0030] Figure 1 A flowchart of a sensitive data processing method combining privacy score and dynamic threshold is shown according to an embodiment of the present application;

[0031] Figure 2 A structural diagram of a sensitive data processing system combining privacy score and dynamic threshold is shown according to an embodiment of the present application;

[0032] Figure 3 A result diagram of an electronic device is shown according to an embodiment of the present application. DETAILED DESCRIPTION

[0033] To make the objectives, technical solutions and advantages of the present application clearer, the technical solutions of the present application will be described below in detail with the embodiments of the present application and the corresponding drawings. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work fall within the scope of protection of the present application.

[0034] The purpose of the present application is to provide a sensitive data processing method combining privacy score and dynamic threshold, so as to evaluate the privacy risk of input text before the input text of the user is input into a large language model, and to take different processing operations according to the evaluation result to realize the hierarchical protection of sensitive data. To achieve the above purpose, Figure 1 A sensitive data processing method combining privacy score and dynamic threshold is shown according to an embodiment of the present application, from Figure 1 It can be seen that the present embodiment includes steps S110-S140:

[0035] Step S110, obtaining a user input text of a scene dimension, performing word segmentation processing on the input text to obtain each word, and identifying each privacy word from each word according to a predefined privacy dictionary of the scene dimension.

[0036] In this step, the scene dimension refers to the field to which the user input text belongs, such as a financial scene, a medical scene, etc. The predefined privacy dictionary is constructed in advance according to the sensitive information features of different scene dimensions, and contains common privacy words under each scene dimension. For example, the predefined privacy dictionary of the financial scene includes bank card number (specific digits), credit card number (specific digits), ID number (specific digits), etc.; the predefined privacy dictionary of the medical scene includes disease name, prescription drug name, etc.

[0037] The input text of the user in the scene dimension is processed by a natural language processing tool (such as jieba) to split the continuous input text into individual tokens.

[0038] For example, the input text is "I use the credit card (card number xxxxxxxxxxxx) of the Construction Bank to pay 19800 yuan of monthly mortgage", and the split tokens are "I", "use", "Construction Bank", "credit card", "card number", "xxxxxxxxxxxxx", "pay", "monthly", "19800 yuan", and "mortgage".

[0039] The tokens are traversed and matched with the predefined privacy dictionary in the scene dimension to identify the privacy words in the tokens.

[0040] For example, the privacy words "xxxxxxxxxxxxxxxx", "19800 yuan", and "mortgage" are identified from the tokens.

[0041] In step S120, the attention mechanism of the large language model is used to calculate the basic privacy score of the input text according to the privacy word weight of each token, the associated risk enhancement score of the input text is calculated according to the privacy word co-occurrence frequency of each privacy word pair, and the comprehensive privacy score of the input text is determined according to the basic privacy score and the associated risk enhancement score.

[0042] In the Transformer architecture of the large language model, the attention mechanism can reflect the attention degree of the model to different tokens. The higher the attention weight of a token, the more likely the model is to capture and remember the token information, and the higher the risk of leakage of the token information.

[0043] The privacy word weight is a quantitative measure of the sensitivity of each token. The basic privacy score is a combination of the privacy word weight of the token and the attention weight of the model to the token, which comprehensively evaluates the risk of direct exposure of the token.

[0044] The appearance of a privacy word pair may leak more sensitive information, and the privacy word co-occurrence frequency is used to measure the probability of the simultaneous appearance of two privacy words. The associated risk enhancement score is based on the privacy word co-occurrence frequency and the attention intensity of the model to the sensitive privacy word pair, which quantifies the privacy inference risk.

[0045] The comprehensive privacy score is obtained by combining the basic privacy score and the associated risk enhancement score, and the global risk value of the input text is determined.

[0046] The implementation of step S120 is described in detail as follows.

[0047] The attention weight matrix of the dimensions of "Transformer layer x attention head x token" is constructed by obtaining the attention weight of each token of each Transformer layer and each attention head in the large language model through the interface of the large language model.

[0048] An element in the attention weight matrix is represented as , wherein represents a Transformer layer index, represents an attention head index, represents a token index. That is represents the attention weight of the th Transformer layer, the th attention head to the th token, wherein the value range of the Transformer layer index is [1, L], the value range of the attention head index is [1, H], L represents the total number of Transformer layers in the large language model, and H represents the total number of attention heads in a single Transformer layer.

[0049] In the basic privacy dimension, the privacy word weight is a quantification of the sensitivity of the token, including a basic weight and a pattern compensation. The basic weight is determined according to whether the token is in the predefined privacy dictionary of the scene dimension and whether it belongs to the sensitive category set. The pattern compensation is determined according to the confidence of the token matching the sensitive pattern.

[0050] The basic weight of each token is calculated according to the following basic weight formula:

[0051] (Formula 1);

[0052] wherein represents the basic weight, represents the th token, represents the predefined privacy dictionary of the scene dimension, represents the preset privacy weight in the privacy dictionary, represents the sensitive category set, represents the word vector, represents the word vector similarity function, represents the sensitive category reference weight, represents the minimum value.

[0053] In the basic weight formula, represents the predefined privacy dictionary of the scene dimension, which is the same as the predefined privacy dictionary of the scene dimension used when identifying each privacy word. When when the predefined privacy dictionary belongs to the scenario dimension, . wherein, represents the preset privacy weight in the privacy dictionary.

[0054] In the basic weight formula, represents a sensitive category set (such as a financial category, a medical category, etc.). When does not belong to the predefined privacy dictionary of the scenario dimension but belongs to the sensitive category set, . wherein, represents a word vector, which can be generated based on a word vector model trained according to the sensitive category corpus; calculates the similarity of with the sensitive category benchmark word vector; is an artificially preset sensitive category benchmark weight related to the sensitive category, such as a financial category benchmark weight preset as 0.75 and a medical category benchmark weight preset as 0.8.

[0055] In the basic weight formula, represents a minimum value, used to assign a value to a segmented word that is not a privacy word and does not belong to the sensitive category set.

[0056] The mode compensation of each segmented word is calculated according to the following mode compensation formula:

[0057] ; (Formula 2);

[0058] wherein, represents the mode compensation, represents the sensitive mode confidence.

[0059] In the mode compensation formula, represents the sensitive mode confidence, which is obtained by a mode matching algorithm. For example completely matches the bank card number mode, then .

[0060] Based on the basic weight and the mode compensation, the privacy word weight of each segmented word is calculated according to the following privacy word weight function:

[0061] ; (Formula 3)

[0062] wherein, represents the privacy word weight, represents the mode compensation coefficient.

[0063] In the privacy word weight function, represents the mode compensation coefficient, which is determined by artificial presetting, such as 0.3.

[0064] After calculating the privacy word weight of each segmented word, the local privacy sensitivity score is calculated in combination with the attention weight matrix. The local privacy sensitivity score refers to the privacy sensitivity score under one Transformer layer and one attention head.

[0065] Based on the attention weight matrix and the privacy word weight of each segmented word, the privacy sensitivity score of the th Transformer layer and the th attention head is calculated according to the following privacy sensitivity score formula:

[0066] ; (Formula 4)

[0067] Wherein, represents the privacy sensitivity score of the th Transformer layer and the th attention head, represents the input text, represents the total number of segmented words.

[0068] The privacy sensitivity scores of all Transformer layers and all attention heads are summarized to obtain the basic privacy score of the input text.

[0069] The basic privacy score can be calculated according to the following summary formula:

[0070] ; (Formula 5).

[0071] In the correlation risk enhancement dimension, the co-occurrence frequency of the privacy word pair is detected by the co-occurrence detection function.

[0072] The privacy word co-occurrence frequency is calculated by the following co-occurrence detection function:

[0073] ; (Formula 6);

[0074] Wherein, represents the privacy word co-occurrence frequency, represents the th privacy word, represents the th privacy word, represents the privacy correlation training data set of the scene dimension, represents the statistical number, represents the maximum.

[0075] In the co-occurrence detection function, represents the privacy correlation training data set of the scene dimension, which is usually composed of a large number of historical sensitive texts, and is stored after artificial labeling and preprocessing.

[0076] The meaning of "counting the number of texts in which The meaning of "counting the number of texts in which The meaning of "counting the number of texts in which and The meaning of "counting the number of texts in which For example, there are 1000 financial texts in which "bank card number (specific number)" and "mortgage" are contained, and there are 350 texts in which both "bank card number (specific number)" and "mortgage" are contained, then .

[0077] The meaning of "counting the number of texts in which The meaning of "counting the number of texts in which The meaning of "counting the number of texts in which For example, there are 1000 financial texts in which "bank card number (specific number)" and "mortgage" are contained, and there are 350 texts in which both "bank card number (specific number)" and "mortgage" are contained, then .

[0078] The meaning of "counting the number of texts in which The meaning of "counting the number of texts in which The meaning of "counting the number of texts in which For example, there are 1000 financial texts in which "bank card number (specific number)" and "mortgage" are contained, and there are 350 texts in which both "bank card number (specific number)" and "mortgage" are contained, then .

[0079] After calculating the privacy word co-occurrence frequency, the correlation risk enhancement score is calculated in combination with the attention weight matrix.

[0080] Based on the attention head weight matrix and the privacy word co-occurrence frequency of each privacy word pair, the correlation risk enhancement score is calculated according to the following correlation risk enhancement score formula:

[0081] ; (Formula 7);

[0082] Wherein, represents the correlation risk enhancement score, represents the set of attention heads sensitive to the privacy word pair, represents the global attention value of the input text of the th attention head, represents the indicator function, represents the co-occurrence risk threshold.

[0083] In the correlation risk enhancement score formula, represents the set of attention heads sensitive to the privacy word pair. That is, if it is found through manual screening that the 3rd Transformer layer, the 2nd attention head and the 5th Transformer layer, the 2nd attention head pay high attention to the financial privacy word pair, then the two attention weights are included in .

[0084] In the correlation risk enhancement scoring formula, represents the first attention head to the input text of all segmented words. When calculating, sum all elements of the first attention head.

[0085] In the correlation risk enhancement scoring formula, represents the indicator function. If the condition in the parentheses is true, it takes the value 1, and if the condition in the parentheses is not true, it takes the value 0.

[0086] In the correlation risk enhancement scoring formula, represents the co-occurrence risk threshold, which is an artificial preset value. For example, the financial scenario dimension is set to 0.7, and the medical scenario dimension is set to 0.6.

[0087] After calculating the basic privacy score and the correlation risk enhancement score, based on the basic privacy score and the correlation risk enhancement score, the comprehensive privacy score is calculated according to the following comprehensive privacy score formula:

[0088] ; (Formula 8)

[0089] wherein, represents the comprehensive privacy score, represents the correlation privacy coefficient.

[0090] Step S130, obtain the historical comprehensive privacy score statistical data of the scenario dimension, and determine the dynamic threshold value according to the privacy word appearance frequency of the input text and the historical comprehensive privacy score statistical data.

[0091] According to the historical comprehensive privacy score statistical data of the scenario dimension and the situation of the input text, the threshold value of risk judgment is dynamically generated to avoid errors caused by fixed threshold value.

[0092] In some optional embodiments, step S130, obtaining the historical comprehensive privacy score statistical data of the scenario dimension, and determining the dynamic threshold value according to the privacy word appearance frequency of the input text and the historical comprehensive privacy score statistical data, comprises: obtaining each historical comprehensive privacy score of the scenario dimension, calculating the historical comprehensive privacy score mean and the historical comprehensive privacy score standard deviation of each historical comprehensive privacy score; based on the historical comprehensive privacy score mean, the historical comprehensive privacy score standard deviation, the privacy word appearance frequency of the input text, and according to the following dynamic threshold value calculation formula, the dynamic threshold value is determined: ; wherein, represents the dynamic threshold value, represents the historical comprehensive privacy score mean of the scenario dimension, represents the sensitivity adjustment factor, represents the historical comprehensive privacy score standard deviation of the scenario dimension,​ Represents the natural logarithm. This indicates the frequency of private words appearing in the input text.

[0093] Obtain historical comprehensive privacy scores for each scenario dimension, and calculate the average historical comprehensive privacy score based on each historical comprehensive privacy score. .

[0094] Obtain historical comprehensive privacy scores for each scenario dimension, and calculate the standard deviation of the historical comprehensive privacy scores based on each historical comprehensive privacy score. .

[0095] Calculate the frequency of private words in the input text . This is used to indicate the total number of times each privacy word appears in the input text.

[0096] The dynamic threshold is then determined according to the following dynamic threshold calculation formula:

[0097] ;(Formula 9);

[0098] In the dynamic threshold calculation formula, This represents the sensitivity adjustment factor, which is preset manually, such as 0.2.

[0099] In the dynamic threshold calculation formula Represents the natural logarithm, used to smooth the effect of the frequency of privacy words in the input text.

[0100] Step S140: Perform corresponding operations on the input text based on the comparison results of the comprehensive privacy score and the dynamic threshold.

[0101] The final comprehensive privacy score is compared with a dynamic threshold, and different actions are performed based on the comparison results.

[0102] In some optional implementations, step S140, performing corresponding operations on the input text based on the comparison result of the comprehensive privacy score and the dynamic threshold, includes: comparing the comprehensive privacy score with the dynamic threshold; if It allows input text to be directly fed into the large language model; if The privacy-related words in the input text are blurred before being input into the large language model; if It intercepts input text and outputs a privacy protection warning message.

[0103] if If the input text is not properly transmitted, it will be directly transmitted to the large language model for a normal response.

[0104] if If yes, the privacy words in the input text are replaced, and the replaced text is transmitted to the large language model. The replacement method of the privacy words can refer to the prior art, which is not limited here.

[0105] If yes , the input text is rejected to be transmitted to the large language model, and a privacy protection warning information is output to the user.

[0106] Regardless of which processing operation is performed, a privacy audit log is recorded.

[0107] The method proposed in the present application is described below through two specific embodiments.

[0108] Embodiment 1, financial scenario dimension embodiment:

[0109] User input text: I use the construction bank credit card (card number xxxxxxxxxxxx) to pay 19800 yuan of monthly mortgage.

[0110] Each word: "I", "uses", "construction bank", "credit card", "card number", "xxxxxxxxxxxxxxxx", "pay", "every month", "19800 yuan", "mortgage".

[0111] Each privacy word: "xxxxxxxxxxxxxxxx", "19800 yuan", "mortgage".

[0112] The attention mechanism of the large language model is called to obtain the attention weights of the tokens corresponding to the above word pieces for each Transformer layer and each attention head. For example, if the large language model has 12 layers (L=12) and each layer has 12 attention heads (H=12), the attention matrix constructed contains the weight information of 144 attention heads for each word piece.

[0113] The privacy word weight of each word piece is calculated, for example, "bank card number (specific number)" belongs to the predefined privacy word dictionary of the financial scenario dimension, according to , the sensitive pattern confidence (complete matching of the bank card number pattern) is calculated. ; therefore, the privacy word weight .​

[0114] "19800 yuan", "mortgage" also belong to the predefined privacy dictionary of the financial scenario dimension, and the calculation method is the same as "bank card number (specific number)", which will not be repeated here.

[0115] For example, "construction bank" belongs to the financial sensitive category set, according to The calculation is ; the sensitive pattern confidence (complete match of financial institution pattern), then ; therefore, the privacy word weight .

[0116] "credit card", "card number", "payment" also belong to the financial sensitive category set, and the calculation method is the same as "construction bank", which will not be repeated here.

[0117] For example, "I", "use", "monthly" all belong to other, so the basic weight and pattern compensation are very small.

[0118] According to the privacy sensitivity score formula, the privacy sensitivity score of each Transformer layer and each attention head is calculated, and then the privacy sensitivity score is summarized to calculate the basic privacy score of the input text. For example, the basic privacy score of the input text .

[0119] The privacy word co-occurrence frequency of the privacy word pair is calculated. For example, when calculating the co-occurrence frequency of "bank card number (specific number)" and "mortgage", in the financial privacy correlation training data set , the number of texts containing both "bank card number (specific number)" and "mortgage" is , the number of texts containing "bank card number (specific number)" is , and the number of texts containing "mortgage" is , then . The financial scenario co-occurrence risk threshold is 0.7, so the indicator function .

[0120] Based on the global attention value of the attention head pair to the input text, the correlation risk enhancement score is calculated.

[0121] Then the comprehensive privacy score .

[0122] The historical comprehensive privacy score mean of the financial scenario dimension is , the historical comprehensive privacy score standard deviation is , and the privacy word appearance frequency of the input text is .

[0123] Then the dynamic threshold is .

[0124] Therefore, the interception and warning operation is performed, and the user is output with "Your input contains high-sensitive financial information. To protect your property safety, the request has been intercepted. Please remove the sensitive information and try again."

[0125] Record the privacy audit log.

[0126] Embodiment 2, medical scenario dimension embodiment:

[0127] User input text: I have type II diabetes, and the recent fasting blood glucose value is 7.8 mmol / L.

[0128] Each segmented word: "I", have", "Type II diabetes", "recently", "fasting blood glucose value", "7.8 mmol / L".

[0129] Each privacy word: "Type II diabetes", "fasting blood glucose value", "7.8 mmol / L".

[0130] Call the attention mechanism of the large language model to obtain the attention weight of each Transformer layer and each attention head on the token corresponding to the above segmented words. For example, the large language model has 12 layers (L=12) and 12 attention heads (H=12) per layer. Therefore, the attention matrix contains 144 weight information of each segmented word.

[0131] Calculate the privacy word weight of each segmented word, and then obtain the basic privacy score according to the attention weight matrix and the privacy word weight of each segmented word .

[0132] Calculate the co-occurrence frequency of the privacy word pair, and then obtain the correlation risk enhancement score according to the attention head weight matrix and the privacy word co-occurrence frequency of each privacy word pair .

[0133] Then, the comprehensive privacy score is .

[0134] Dynamic threshold .

[0135] Therefore, the privacy word fuzzification processing operation is performed, and "I have a chronic metabolic disease, and the recent detection value is within the attention range" is input to the large language model.

[0136] record a privacy audit log.

[0137] As can be seen from the above two embodiments, the application can accurately identify sensitive information in user input text in high privacy sensitive scenarios such as finance and medical treatment, comprehensively evaluate multi-dimensional privacy risks, and take corresponding privacy protection measures according to dynamic thresholds, thereby effectively protecting user privacy security.

[0138] Figure 2 A sensitive data processing system combining privacy scores and dynamic thresholds according to an embodiment of the application is shown, from Figure 2 As can be seen, the sensitive data processing system 200 combining privacy scores and dynamic thresholds includes:

[0139] The text splitting unit 210 is configured to obtain a user input text in a scene dimension, perform word segmentation processing on the input text to obtain each word, and identify each privacy word from each word according to a predefined privacy dictionary in the scene dimension;

[0140] The privacy score unit 220 is configured to calculate a basic privacy score of the input text according to the privacy word weight of each word based on the attention mechanism of the large language model, calculate an associated risk enhancement score of the input text according to the privacy word co-occurrence frequency of each privacy word pair, and determine a comprehensive privacy score of the input text according to the basic privacy score and the associated risk enhancement score;

[0141] The threshold calculation unit 230 is configured to obtain historical comprehensive privacy score statistical data in the scene dimension, and determine a dynamic threshold according to the privacy word occurrence frequency of the input text and the historical comprehensive privacy score statistical data;

[0142] The processing operation unit 240 is configured to perform a corresponding operation on the input text according to the comparison result of the comprehensive privacy score and the dynamic threshold.

[0143] In some optional embodiments, in the above system, the privacy score unit 220 is configured to obtain the attention weight of each word for each Transformer layer and each attention head in the large language model, and construct an attention weight matrix; wherein the element in the attention weight matrix is the attention weight of the i-th word for the j-th Transformer layer and the k-th attention head ; wherein the value range of the Transformer layer index is [1, L], the value range of the attention head index is [1, H], L represents the total number of Transformer layers in the large language model, and H represents the total number of attention heads in a single Transformer layer. ​​​

[0144] In some optional implementations, in the above system, the privacy word weight includes a base weight and pattern compensation; the privacy scoring unit 220 is used to: calculate the base weight of each word segment according to the following base weight formula: ;in, Indicates the basic weight. Indicates the first Each word, A predefined privacy dictionary representing the scene dimension. This indicates the preset privacy weights in the privacy dictionary. Represents a set of sensitive categories. Representing word vectors, Represents the word vector similarity function. Indicates the baseline weight of the sensitive category. This represents the minimum value; the pattern compensation for each word segmentation is calculated according to the following pattern compensation formula: ;in, Representation pattern compensation, Indicates the confidence level of sensitive patterns; based on the base weights and pattern compensation, the privacy word weights of each word segment are calculated according to the following privacy word weight function: ;in, Indicates the weight of privacy terms. The mode compensation coefficient is represented.

[0145] In some optional implementations, in the above system, the privacy scoring unit 220 is used to: calculate the privacy sensitivity scoring formula based on the attention weight matrix and the privacy word weights of each word segment. The first Transformer layer, the... Privacy sensitivity score for each user's attention: ;in, Indicates the first The first Transformer layer, the... Privacy sensitivity score for each user's attention. This indicates the input text. This represents the total number of word segments; by summing the privacy sensitivity scores of all Transformer layers and all attention heads, a basic privacy score for the input text is obtained. .

[0146] In some alternative implementations, in the above system, the privacy scoring unit 220 is used to: calculate the co-occurrence frequency of privacy words using the following co-occurrence detection function: ;in, Indicates the co-occurrence frequency of privacy words. Indicates the first A private word, Indicates the first a privacy word, a privacy-related training data set representing a scene dimension, representing a statistical quantity, representing taking the maximum.

[0147] In some optional embodiments, in the above system, the privacy scoring unit 220 is configured to calculate an association risk enhancement score according to the following association risk enhancement score formula based on the attention head weight matrix and the privacy word co-occurrence frequency of each privacy word pair: ; wherein, represents the association risk enhancement score, represents a set of attention heads sensitive to the privacy word pair, represents the global attention value of the th attention head on the input text, represents an indicator function, represents a co-occurrence risk threshold.

[0148] In some optional embodiments, in the above system, the privacy scoring unit 220 is configured to calculate a comprehensive privacy score according to the following comprehensive privacy score formula based on the basic privacy score and the association risk enhancement score: ; wherein, represents the comprehensive privacy score, represents an association privacy coefficient.

[0149] In some optional embodiments, in the above system, the threshold calculation unit 230 is configured to: obtain each historical comprehensive privacy score of the scene dimension, calculate the historical comprehensive privacy score mean and the historical comprehensive privacy score standard deviation of each historical comprehensive privacy score; determine a dynamic threshold according to the following dynamic threshold calculation formula based on the historical comprehensive privacy score mean, the historical comprehensive privacy score standard deviation, and the privacy word occurrence frequency of the input text: ; wherein, represents the dynamic threshold, represents the historical comprehensive privacy score mean of the scene dimension, represents a sensitivity adjustment factor, represents the historical comprehensive privacy score standard deviation of the scene dimension, represents a natural logarithm, represents the privacy word occurrence frequency of the input text.

[0150] In some optional embodiments, in the above system, the processing operation unit 240 is configured to: compare the comprehensive privacy score with the dynamic threshold; if , allow the input text to be directly input to the large language model; if , perform fuzzification processing on each privacy word in the input text and then input the processed text to the large language model; if intercept the input text and output a privacy protection warning message.

[0151] In the risk assessment mechanism, the application combines the basic privacy score with the associated risk enhancement score to form a hierarchical and progressive evaluation mode. The basic privacy score is responsible for quantifying explicit privacy features, while the associated risk enhancement score accurately captures distributed privacy information and potential reasoning risks by analyzing contextual semantic associations, privacy word co-occurrence patterns, and other deep features. This evaluation system enables the system to identify associated privacy and addresses the fundamental limitations of traditional methods that can only detect isolated sensitive words. Meanwhile, the application further utilizes the attention mechanism of large language models to analyze the attention weight distribution within the model, enabling dynamic tracking and precise positioning of privacy risks. This analysis method based on the internal mechanisms of deep learning models can identify high-risk attention patterns and uncover potential transmission paths of privacy information in conversations.

[0152] In the decision mechanism, the application adopts a dynamic threshold mechanism with scene adaptation capability. This mechanism can intelligently calibrate according to the conversation field sensitivity, historical behavior patterns, and real-time risk level, automatically increasing the protection level in high-sensitive scenarios such as finance and medicine, while maintaining a smooth experience in daily conversations. This dynamic adjustment mechanism ensures strict protection in high-risk scenarios while effectively avoiding excessive interception, achieving a balance between precision and adaptability.

[0153] It should be noted that the sensitive data processing system 200 combining privacy scores and dynamic thresholds described above can implement the sensitive data processing method combining privacy scores and dynamic thresholds described above, and thus will not be described again.

[0154] Figure 3 The structural schematic diagram of an electronic device according to an embodiment of the application is shown. As shown in Figure 3 The electronic device includes a processor, a memory, a network interface, and a database connected by a system bus. The processor of the electronic device is used to provide computing and control capabilities. The memory of the electronic device includes non-volatile and / or volatile storage media, internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the electronic device is used to communicate with the outside through network connection. The computer program is executed by the processor to function or step of the sensitive data processing method combining privacy scores and dynamic thresholds.

[0155] In one embodiment, the electronic device provided in this application includes a memory and a processor. The memory stores a database and a computer program that can run on the processor. When the processor executes the computer program, it implements the steps of the aforementioned sensitive data processing method that combines privacy scoring and dynamic thresholds.

[0156] The above is as stated in this application. Figure 2 The method for sensitive data processing system combining privacy scoring and dynamic thresholds disclosed in the illustrated embodiments can be applied to a processor or implemented by a processor. During implementation, each step of the above method can be completed by integrated logic circuits in the processor's hardware or by instructions in software form. The processor can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it can also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The steps of the method disclosed in the embodiments of this application can be directly implemented by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. The software modules can reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. This storage medium is located in memory, and the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method.

[0157] In one embodiment, a computer-readable storage medium is also provided, on which a computer program is stored, which, when executed by a processor, implements the steps of the aforementioned sensitive data processing method combining privacy scoring and dynamic thresholds.

[0158] It should be noted that the functions or steps that the above-mentioned electronic devices or computer-readable storage media can achieve can be referred to the relevant descriptions in the foregoing method embodiments. To avoid repetition, they will not be described one by one here.

[0159] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer readable storage medium, and when executed, can include the processes of the above-mentioned embodiment methods. Any reference to memory, storage, database or other medium used in the embodiments provided by the present application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration but not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0160] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the above-mentioned division of functional units and modules is exemplified. In actual application, the above-mentioned functions can be completed by different functional units or modules according to needs, that is, the internal structure of the system is divided into different functional units or modules to complete all or part of the above-described functions.

[0161] The above-described embodiments are only used to illustrate the technical solutions of the present application, but not limit them. Although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacements for part of the technical features. Such modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. A sensitive data processing method combining privacy scoring and dynamic thresholds, characterized in that, include: The system acquires user input text from the context dimension, performs word segmentation on the input text to obtain each word, and identifies each privacy word from each word segmentation based on a predefined privacy dictionary for the context dimension. Based on the attention mechanism of the large language model, the basic privacy score of the input text is calculated according to the privacy word weight of each segment, the association risk enhancement score of the input text is calculated according to the privacy word co-occurrence frequency of each privacy word pair, and the comprehensive privacy score of the input text is determined based on the basic privacy score and the association risk enhancement score. Obtain historical comprehensive privacy score statistics for the scenario dimension, and determine dynamic thresholds based on the frequency of privacy words in the input text and historical comprehensive privacy score statistics; The input text is processed according to the comparison between the comprehensive privacy score and the dynamic threshold. The attention mechanism based on the large language model includes: Obtain the attention weights of each Transformer layer and each attention head for each word segmentation in the large language model, and construct an attention weight matrix; The elements in the attention weight matrix are the first... The first Transformer layer, the... The first attention head to the first Attention weights for each word segmentation ; Among them, the Transformer layer index The value range is [1, L], and the attention head index is... The value range is [1, H], where L represents the total number of Transformer layers in the large language model and H represents the total number of attention heads in a single Transformer layer; The privacy term weights include base weights and pattern compensation; The basic weight of each word segmentation is calculated based on the following basic weight formula: ； in, Indicates the basic weight. Indicates the first Each word, A predefined privacy dictionary representing the scene dimension. This indicates the preset privacy weights in the privacy dictionary. Represents a set of sensitive categories. Representing word vectors, Represents the word vector similarity function. Indicates the baseline weight of the sensitive category. Indicates the minimum value; The pattern compensation for each word segment is calculated according to the following pattern compensation formula: ； in, Representation pattern compensation, Indicates the confidence level of the sensitive pattern; Based on the base weights and pattern compensation, the privacy word weights of each word segment are calculated according to the following privacy word weight function: ； in, Indicates the weight of privacy terms. Representation mode compensation coefficient; The calculation of the basic privacy score of the input text based on the privacy word weights of each segment includes: Based on the attention weight matrix and the privacy word weights of each word segment, the privacy sensitivity scoring formula is used to calculate the first... The first Transformer layer, the... Privacy sensitivity score for each user's attention: ； in, Indicates the first The first Transformer layer, the... Privacy sensitivity score for each user's attention. This indicates the input text. Indicates the total number of word segments; By summing the privacy sensitivity scores of all Transformer layers and all attention heads, a basic privacy score for the input text is obtained. ; The co-occurrence frequency of the privacy word is calculated using the following co-occurrence detection function: ； in, Indicates the co-occurrence frequency of privacy words. Indicates the first A private word, Indicates the first A private word, This represents a privacy-related training dataset representing the scene dimension. Indicates the statistical quantity. This indicates taking the maximum value; The calculation of the association risk enhancement score of the input text based on the co-occurrence frequency of privacy words in each privacy word pair includes: Based on the attention head weight matrix and the co-occurrence frequency of privacy words for each privacy word pair, the association risk enhancement score is calculated according to the following association risk enhancement scoring formula: ； in, Indicates an enhanced risk score. This represents the set of attention heads sensitive to privacy-related words. Indicates the first The global attention value of each attention head on the input text. Indicates an indicator function, Indicates the co-occurrence risk threshold; The determination of the comprehensive privacy score of the input text based on the basic privacy score and the associated risk enhancement score includes: Based on the basic privacy score and the associated risk enhancement score, the comprehensive privacy score is calculated according to the following comprehensive privacy score formula: ； in, Indicates the overall privacy score, Indicates the associated privacy coefficient; The acquisition of historical comprehensive privacy score statistics for the scene dimension, and the determination of dynamic thresholds based on the frequency of privacy words in the input text and the historical comprehensive privacy score statistics, include: Obtain the historical comprehensive privacy scores for each scenario dimension, and calculate the mean and standard deviation of the historical comprehensive privacy scores for each historical comprehensive privacy score. Based on the historical average comprehensive privacy score, the historical standard deviation of the comprehensive privacy score, and the frequency of privacy words in the input text, the dynamic threshold is determined according to the following dynamic threshold calculation formula: ； in, Indicates dynamic threshold. This represents the historical average comprehensive privacy score across different scenarios. Indicates the sensitivity modulator. This represents the standard deviation of the historical comprehensive privacy score across different scenarios. Represents the natural logarithm. This indicates the frequency of private words appearing in the input text.

2. The method according to claim 1, characterized in that, The step of performing corresponding operations on the input text based on the comparison result of the comprehensive privacy score and the dynamic threshold includes: Compare the overall privacy score with dynamic thresholds; like It allows input text to be directly fed into a large language model; like The privacy words in the input text are blurred before being input into the large language model; like It intercepts input text and outputs a privacy protection warning message.

3. A sensitive data processing system combining privacy scoring and dynamic thresholds, characterized in that, include: The text segmentation unit is used to acquire user input text from the scene dimension, perform word segmentation on the input text to obtain each word, and identify each privacy word from each word segment according to the predefined privacy dictionary of the scene dimension. The privacy scoring unit is used to calculate the basic privacy score of the input text based on the attention mechanism of the large language model, calculate the association risk enhancement score of the input text based on the privacy word weight of each segment, calculate the association risk enhancement score of the input text based on the privacy word co-occurrence frequency of each privacy word pair, and determine the comprehensive privacy score of the input text based on the basic privacy score and the association risk enhancement score. The threshold calculation unit is used to obtain historical comprehensive privacy score statistics for the scene dimension, and to determine the dynamic threshold based on the frequency of privacy words in the input text and the historical comprehensive privacy score statistics. The processing unit is used to perform corresponding operations on the input text based on the comparison result of the comprehensive privacy score and the dynamic threshold; The privacy scoring unit is used to: obtain the attention weights of each Transformer layer and each attention head for each word segmentation in the large language model, and construct an attention weight matrix; wherein, the elements in the attention weight matrix are the... The first Transformer layer, the... The first attention head to the first Attention weights for each word segmentation ; where Transformer layer index The value range is [1, L], and the attention head index is... The value range is [1, H], where L represents the total number of Transformer layers in the large language model and H represents the total number of attention heads in a single Transformer layer; Privacy word weights include base weights and pattern compensation; the privacy scoring unit is used to calculate the base weight of each word segment according to the following base weight formula: ;in, Indicates the basic weight. Indicates the first Each word, A predefined privacy dictionary representing the scene dimension. This indicates the preset privacy weights in the privacy dictionary. Represents a set of sensitive categories. Representing word vectors, Represents the word vector similarity function. Indicates the baseline weight of the sensitive category. This represents the minimum value; the pattern compensation for each word segmentation is calculated according to the following pattern compensation formula: ;in, Representation pattern compensation, Indicates the confidence level of sensitive patterns; based on the base weights and pattern compensation, the privacy word weights of each word segment are calculated according to the following privacy word weight function: ;in, Indicates the weight of privacy terms. Representation mode compensation coefficient; The privacy scoring unit is used to: calculate the privacy sensitivity scoring formula based on the attention weight matrix and the privacy word weights of each segmented word. The first Transformer layer, the... Privacy sensitivity score for each user's attention: ;in, Indicates the first The first Transformer layer, the... Privacy sensitivity score for each user's attention. This indicates the input text. This represents the total number of word segments; by summing the privacy sensitivity scores of all Transformer layers and all attention heads, a basic privacy score for the input text is obtained. ; The privacy scoring unit is used to calculate the co-occurrence frequency of privacy words using the following co-occurrence detection function: ;in, Indicates the co-occurrence frequency of privacy words. Indicates the first A private word, Indicates the first A private word, This represents a privacy-related training dataset representing the scene dimension. Indicates the statistical quantity. This indicates taking the maximum value; The privacy scoring unit is used to calculate the association risk enhancement score based on the attention head weight matrix and the co-occurrence frequency of privacy words in each privacy word pair, according to the following association risk enhancement scoring formula: ;in, Indicates an enhanced risk score. This represents the set of attention heads sensitive to privacy-related words. Indicates the first The global attention value of each attention head on the input text. Indicates an indicator function, Indicates the co-occurrence risk threshold; The privacy scoring unit is used to calculate a comprehensive privacy score based on the basic privacy score and the associated risk enhancement score, according to the following comprehensive privacy score formula: ;in, Indicates the overall privacy score, Indicates the associated privacy coefficient; The threshold calculation unit is used to: obtain historical comprehensive privacy scores for each scene dimension, calculate the mean and standard deviation of the historical comprehensive privacy scores for each score; and determine the dynamic threshold based on the mean, standard deviation, and frequency of privacy words in the input text, according to the following dynamic threshold calculation formula: ;in, Indicates dynamic threshold. This represents the historical average comprehensive privacy score across different scenarios. Indicates the sensitivity modulator. This represents the standard deviation of the historical comprehensive privacy score across different scenarios. Represents the natural logarithm. This indicates the frequency of private words appearing in the input text.

Images