Self-adaptive personal information desensitization method and system based on availability evaluation

Abstract

The invention discloses a self-adaptive personal information desensitization method and system based on availability evaluation. The method comprises the following steps: receiving a to-be-desensitized original data set and identifying a privacy field; a corresponding desensitization strategy is formulated for the privacy fields, at least two fields are appointed to use a generalization algorithm, and for the privacy fields appointed with the generalization algorithm, one field enabling the availability of a corresponding field set to be the highest is circularly selected for generalization until K anonymity is met; for other privacy fields, performing desensitization processing on the corresponding privacy fields according to the formulated desensitization strategy; and combining the desensitization results to obtain a desensitized data set. According to the scheme, the process of using the availability index to guide K anonymity is provided, so that the desensitization process pays attention to the availability and privacy at the same time, the availability of the data set can be ensured to the maximum extent under the condition that the privacy meets the requirement, the finally output desensitized data set has higher availability, and the desensitization efficiency is improved. And a certain degree of guarantee is provided for the subsequent data analysis effect.

Description

Technical Field

[0001] The present invention belongs to the technical field of personal information desensitization, and in particular, to an adaptive personal information desensitization method and system based on usability evaluation. Background Art

[0002] Existing personal information desensitization systems usually include the following main steps: inputting original data, identifying sensitive information, desensitizing data, and outputting desensitized data. These systems often only focus on the effectiveness of data desensitization, that is, desensitizing the desensitized data through various pre-selected desensitization algorithms, so that the desensitized data will not leak sensitive personal information contained in the original data as much as possible. However, the desensitized data will also be used for subsequent data analysis. If the original data loses a lot of valuable information after desensitization, it is also inappropriate. Therefore, for data desensitization, attention should also be paid to the amount of valuable information contained in the desensitized data, that is, the availability of the desensitized data. Most of the existing personal information desensitization systems lack the availability evaluation and feedback mechanism for desensitized data, and therefore cannot strike a good balance between data privacy and data availability. Summary of the invention

[0003] The purpose of the present invention is to address the above-mentioned problems and to propose an adaptive personal information desensitization method and system based on usability evaluation.

[0004] An adaptive personal information desensitization method based on usability evaluation, the method comprising:

[0005] S1. Sensitive information identification: receiving the original data set to be desensitized and identifying the privacy fields;

[0006] S2. Desensitization plan formulation: formulate corresponding desensitization strategies for privacy fields, and at least two fields are specified to use generalized algorithms, usually specifying the desensitization method of the generalized algorithm for the quasi-identifier;

[0007] S3. Execution of the desensitization scheme: for the privacy fields for which the generalization algorithm is specified, a field that makes the corresponding field set most available is cyclically selected for generalization until K anonymity is satisfied;

[0008] For the remaining privacy fields, desensitize the corresponding privacy fields according to the formulated desensitization strategy;

[0009] S4. Combine the desensitization results of the remaining privacy fields and the privacy fields of the specified generalization algorithm to obtain the desensitized dataset.

[0010] In the above-mentioned adaptive personal information desensitization method based on usability evaluation, in step S3, each cycle includes the following process:

[0011] S31. Treat each privacy field d1, d2, ... d of the generalized field set [D] separatelyn Perform one-step generalization, [D] = d1, d2, ... d n , represents the original field set of the specified generalization algorithm or the field set after generalization processing in the previous cycle, n represents the number of privacy fields of the specified generalization algorithm, if the current cycle is the first cycle, then [D] is the original field set, if not, it is the field set after generalization processing in the previous cycle;

[0012] S32. Each generalized privacy field is combined with the remaining non-generalized privacy fields to form a new field set [D i ’ ], i = 1, 2, ... n;

[0013] [D i ’ ] represents a new field set in which the i-th privacy field in the [D] field set is generalized and the remaining privacy fields are not generalized;

[0014] This process will get n new field sets [D i ’ ];

[0015] S33. Calculate each new field set by the availability calculation method [D i ’ ]’s availability indicators;

[0016] S34. Select the new field set with the highest usability index [D i ’ ] and determine the field d in the new field set i For this round of generalization objects, the new field set with the highest availability index [D i ’ ] will be the field set after this round of generalization [D i ’ ];

[0017] S35. If the new field set after this round of generalization [D i ’ ] satisfies K anonymity, the desensitization process ends, otherwise, the new field set [D i ’ ] is the field set to be generalized [D], and steps S31-S35 are repeated.

[0018] The fields mentioned above are for the privacy fields that specify the generalization algorithm. This solution mainly makes adaptive adjustments to the desensitization of the generalization algorithm, so that the desensitized data can not only meet the K anonymity of the generalization algorithm, but also have high availability. The desensitization strategy for other fields is consistent with the existing technology. In addition, the generalization and non-generalization mentioned here refer to the generalization or non-generalization of this round of cycles.

[0019] In the above-mentioned adaptive personal information desensitization method based on usability evaluation, in step S3, if the fields after this round of generalization do not meet K anonymity, the generalization results of each privacy field are saved for subsequent cyclic use;

[0020] In this cycle, the generalized object d in the previous cycle i The generalization results of the previous field are further generalized, and the remaining fields use the saved generalization results. In this way, it is possible to avoid repeating the same generalization processing on the same field in each cycle. From the second round onwards, only one field needs to be calculated in each cycle.

[0021] In the above-mentioned adaptive personal information desensitization method based on usability evaluation, in each round of the cycle, by evaluating each new field set [D i ’ ] is used to calculate the availability index based on its impact on the performance index of the ensemble learner on downstream machine learning tasks.

[0022] In the above-mentioned adaptive personal information desensitization method based on usability evaluation, the usability index is a comprehensive usability index obtained by weighting multiple partial usability indexes;

[0023] And by setting multiple different machine learning tasks, multiple one-sided availability indicators are obtained.

[0024] In the above-mentioned adaptive personal information desensitization method based on usability evaluation, in step S3, in each cycle, the field that makes the corresponding field set most usable is selected for generalization in the following manner:

[0025] Set m machine learning tasks T1, T2, ..., T m , m is a natural number greater than 2;

[0026] For each task T j Specify the corresponding target field and feature field, j = 1, 2, ... m;

[0027] For each task T j Specify x base learners B j1 ,B j2 ,…,B jx , forming an integrated learner C j ;

[0028] In step S3, each new field set [D i ’ ]Through multi-fold cross validation, we can j Train an ensemble learner instance C ij, represents an instance obtained by training based on the jth machine learning task using a new field set in which the i-th privacy field is generalized and the other privacy fields are not generalized, and testing the ensemble learner instance C ij To obtain the corresponding new field set [D i ’ ]The corresponding performance index is used as a one-sided indicator of the availability of task j;

[0029] According to the set weighted parameters, the m availability-specific indicators are weighted and summed to obtain a new field set [D i ’ ]Corresponding comprehensive availability index;

[0030] Compare the new field sets [D i ’ ], select the new field set with the highest comprehensive availability index [D i ’ ]Field d i It is the generalized object of this round.

[0031] In the above-mentioned adaptive personal information desensitization method based on usability evaluation, in step S1, the original data set to be desensitized is a structured data table obtained from a data source;

[0032] After obtaining the structured data table, classify the fields therein and identify the privacy fields containing personal information.

[0033] In the above-mentioned adaptive personal information desensitization method based on usability evaluation, in step S1, the privacy field is identified by an identifier identification method and a field meaning identification method;

[0034] Identify direct identifiers, quasi-identifiers, and non-identifiers by means of identifier identification;

[0035] Direct identifiers and quasi-identifiers are identified as privacy fields;

[0036] The alignment identifier uses a generalized algorithm, and the direct identifier can use the corresponding desensitization algorithm as needed, which is not limited here.

[0037] In the above-mentioned adaptive personal information desensitization method based on usability evaluation, in step S2,

[0038] The number of machine learning tasks m = the number of quasi-identifier fields. The machine learning tasks are set as follows:

[0039] For each task T j , the jth field is used as the target field of the task, and the remaining fields are used as the feature fields of the task.

[0040] An adaptive personal information desensitization system based on usability evaluation based on the method, the system comprising:

[0041] A data input module, used to receive the original data set to be desensitized;

[0042] Sensitive information identification module, used to identify privacy fields in data;

[0043] A desensitization scheme formulation module is used to formulate a desensitization strategy and specify the use of a generalization algorithm for at least two fields;

[0044] The desensitization execution module is used to execute the desensitization strategy for the remaining privacy fields; loop through the privacy fields for which the generalization algorithm is specified, and select a field that makes the corresponding field set most available according to the availability index to generalize until the K anonymous detection conditions are met;

[0045] Usability evaluation module, used to calculate usability indicators and provide feedback to the desensitization execution module;

[0046] The data output module is used to merge the anonymization results and output the anonymized data set.

[0047] The advantages of the present invention are:

[0048] 1. This scheme proposes to use the availability index to guide the K-anonymity process, so that the desensitization process focuses on both availability and privacy. It can maximize the availability of the data set while ensuring that privacy meets the requirements, so that the final output desensitized data set has higher availability, which provides a certain degree of guarantee for the effect of subsequent data analysis;

[0049] 2. This scheme proposes a method to characterize the availability of desensitized data. This method quantitatively evaluates the availability of the desensitized data set and calculates a comprehensive availability index through the performance indicators of multiple machine learning tasks to intuitively reflect the availability of the desensitized data set. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 A diagram of an adaptive personal information desensitization system based on usability evaluation according to an embodiment of the present invention;

[0051] Figure 2 This is a flow chart of an adaptive personal information desensitization method based on usability evaluation according to an embodiment of the present invention;

[0052] Figure 3 The present invention is a flowchart of a method in which the usability index guides the generalization action in the adaptive personal information desensitization method based on usability evaluation in an embodiment of the present invention. DETAILED DESCRIPTION

[0053] The present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0054] like Figure 1 As shown, this solution provides an adaptive personal information desensitization system based on usability evaluation, including:

[0055] The data input module is used to receive the original data set to be desensitized. It can also include a data preprocessing module for preprocessing the data. For example, if the data to be desensitized is a structured data table, the preprocessing methods include filling empty values ​​and detecting empty fields in the original data table.

[0056] A sensitive information identification module, used to identify privacy fields in the data, which may specifically include a personal information identifier identification module, used to identify identifier information of each field in the data table, and a field meaning identification module, used to identify privacy fields in the data table;

[0057] The desensitization scheme formulation module is used to specify the desensitization algorithm for each field of the data table, and specifies the use of a generalization algorithm for at least two fields; it also includes a generalization hierarchy tree generation module, which is used to generate a generalization hierarchy tree for each quasi-identifier field, that is, it sets how the field specified by the generalization algorithm should be generalized.

[0058] The desensitization execution module is used to execute the desensitization strategy on the remaining privacy fields, and to cyclically select the field with the highest availability among the privacy fields for which the generalization algorithm is specified according to the availability index for generalization until the K anonymous detection conditions are met.

[0059] Availability evaluation module, used to calculate each new field set in each cycle [D i ’ ] and provide feedback to the desensitization execution module;

[0060] Data output module, used to output the anonymized data table.

[0061] Specifically, the adaptive personal information desensitization method based on usability evaluation proposed in this scheme is as follows:

[0062] S1. Data input, obtaining the structured data table to be desensitized from the data source;

[0063] S2. Sensitive information identification: classify the various fields of the input data set and identify the privacy fields containing personal information; specifically, it includes the identification of personal information identifiers and the identification of field meanings. Identify direct identifiers, quasi-identifiers, and non-identifiers through identifier identification; direct identifiers and quasi-identifiers are privacy fields; a generalized algorithm is used for quasi-identifiers, and direct identifiers can use corresponding desensitization algorithms as needed, which are not limited here.

[0064] The specific method of identifier identification can be:

[0065] By matching the identifier library, the direct identifier and the quasi-identifier in the identifier library are identified;

[0066] Identify quasi-identifiers in the remaining fields based on functional dependencies. If the direct identifier function depends on a field set, each field in the field set is used as a quasi-identifier.

[0067] The remaining fields are treated as non-identifiers.

[0068] Different types of privacy data may be desensitized using different desensitization algorithms. Therefore, for direct identifiers, the meaning of the fields can be further identified, and then these direct identifiers can be desensitized using the corresponding desensitization algorithms:

[0069] Rule matching: Identify fields with a uniform format, such as ID card number, mobile phone number, bank card number, email address, license plate number, and IP address, by matching regular expressions;

[0070] Field classification based on named entity recognition: For other unclassified fields, the text classification model is used to classify them and identify names, addresses, etc.

[0071] Combining regular expression-based rule matching and deep learning model-based text classification methods to identify sensitive data in the original data is not a simple regular expression, which can improve the ability to identify personal information. At the same time, it is not a simple deep learning model. Regular expressions can be used to quickly identify some personal information with a fixed format, reducing the amount of text input into the deep learning model, thereby speeding up the identification and extraction of personal information.

[0072] S3. Desensitization scheme formulation: formulate corresponding desensitization strategies for privacy fields, and quasi-identifier fields are specified to use generalized algorithms;

[0073] Specifically, it includes desensitization algorithm matching and generalization hierarchical tree generation. The former formulates corresponding desensitization strategies for each privacy field, and the latter generates a generalization hierarchical tree for the generalized desensitized fields.

[0074] Quasi-identifiers are mostly numeric fields and categorical fields. The following describes the generation methods of generalized hierarchical trees for numeric fields and categorical fields respectively:

[0075] For numeric fields, a field value will be generalized to an interval containing the value; merging the interval X = [x1, x2] and the interval Y = [y1, y2] will result in the interval Z = [min(x1, y1), max(x2, y2)]. Initially, each different field value v corresponds to an interval [v, v] of length 0, which serves as the leaf node of the generalization hierarchy tree, and all intervals are sorted. Next, each loop traverses all two adjacent intervals, selects the two adjacent intervals with the shortest length after merging, and merges them. The new interval serves as the parent node of the two intervals, until all intervals are merged into one interval (that is, the root node of the generalization hierarchy tree).

[0076] For categorical fields, the field value is first encoded as a word embedding vector (for example, using Word2Vec, GloVe, or obtained through a pre-trained model), and then a distance measurement function is set based on semantic similarity (the higher the semantic similarity, the shorter the distance, such as using cosine similarity). A hierarchical clustering algorithm is then used to obtain a hierarchical clustering tree of these field values. Non-leaf node values ​​are replaced by a set of their child node values ​​to obtain a generalized hierarchical tree for the field.

[0077] S4. Execution of the desensitization scheme: For privacy fields with a specified generalization algorithm, a field with the highest availability of the corresponding field set is selected cyclically for generalization until K anonymity is met. This includes availability evaluation and data desensitization execution. The former calculates the availability index of the field set, and the latter performs generalization operations based on the availability index.

[0078] For the remaining privacy fields, the corresponding privacy fields are desensitized according to the established desensitization strategy.

[0079] S5. Combine the desensitization results of the remaining privacy fields and the privacy fields of the specified generalization algorithm, as well as the parts of the original data set that do not need to be desensitized, to obtain a desensitized data set, and save the desensitized data set in a specified location for the convenience of further processing by the user.

[0080] At present, some common desensitization algorithms include rearrangement, masking, relationship mapping, generalization, anonymization, data replacement, etc. A desensitization algorithm can also be applied to multiple field types. For example, generalization can be applied to numerical and categorical types of quasi-identifiers. Numerical fields contain numerical data that can be mathematically operated, such as integers or floating-point numbers, such as age, height, weight, price, temperature, grades, etc.; categorical fields contain qualitative data, usually used to represent categories or labels, such as gender, country, occupation, product type, color, etc.

[0081] In the design of traditional desensitization strategies, technical personnel or set rules determine an available desensitization algorithm for each field type to be desensitized. For example, a generalized desensitization algorithm is used for numeric fields and categorical fields, and other desensitization algorithms are used for other fields. Then, when the field type is identified, the determined desensitization algorithm is enabled to perform the desensitization operation. In other words, for each field type, a specific operation is performed. This method is faster, but it is impossible to find a balance between desensitization and availability. It is easy to have problems such as excessive desensitization resulting in unavailable data, or data being available but with too low a degree of desensitization.

[0082] This embodiment specifies the use of a generalized algorithm for fields identified as privacy fields, such as quasi-identifiers. The remaining fields, such as direct identifiers, are still similar to general technologies, that is, the user, the technician, or the set rules determine an available desensitization algorithm for each type of field to be desensitized, such as the use of shielding, pseudonymization, or randomization algorithms. For example, shielding or pseudonymization is used for names. In general, quasi-identifiers are mostly categorical fields and numerical fields. Specifying a generalized algorithm for sensitive categorical fields and numerical fields in the data set will have the best effect. When put into use, some quasi-identifiers can also be specified to use a generalized algorithm, and the remaining quasi-identifiers still use the traditional method. The specifics are not limited here. As long as at least two fields are specified to use a generalized algorithm, the basic purpose desired can be achieved through the method proposed in this solution.

[0083] After the desensitization algorithm is formulated, the desensitization action is performed according to the formulated desensitization strategy. Specifically, for the remaining privacy fields, that is, the privacy fields that do not belong to the quasi-identifier, the corresponding privacy fields are desensitized according to the formulated desensitization strategy. This step is consistent with the existing technology and will not be repeated here. For the privacy fields of quasi-identifiers, this solution combines the availability index to guide the generalization action, so as to maximize the availability of data while ensuring privacy.

[0084] Finally, the desensitization results of the remaining privacy fields are combined with the desensitization results of the quasi-identifiers to obtain the final desensitized data set.

[0085] Specifically, the usability indicators are used to guide generalization actions, including:

[0086] The identifier field is cyclically selected to generalize a field that makes the corresponding dataset most usable until K anonymity is met. Specifically, each cycle includes the following process:

[0087] S41. Treat each privacy field d1, d2, ... d of the generalized field set [D] separately n Perform one-step generalization respectively, [D] = d1, d2...d n, represents the original field set of the quasi-identifier of the specified generalization algorithm or the field set after generalization processing in the previous cycle, n represents the number of quasi-identifiers. If the current cycle is the first cycle, then it is the original field set, if not, it is the field set after generalization processing in the previous cycle.

[0088] S42. Each generalized privacy field is combined with the remaining non-generalized privacy fields to form a new field set [D i ’ ], i = 1, 2, ... n;

[0089] [D i ’ ] indicates that the i-th privacy field in the [D] field set is generalized and the other privacy fields are not generalized into a new field set, resulting in n new field sets [D i ’ ];

[0090] D43. Calculate each new field set using the availability calculation method [D i ’ ]’s availability indicators;

[0091] S44. Select the new field set with the highest usability index [D i ’ ] and determine the field d in the new field set i is the generalized object of this round;

[0092] S45. If the new field set after this round of generalization [D i ’ ] satisfies K anonymity, the desensitization process ends, otherwise, the new field set [D i ’ ] is the field set to be generalized [D], and steps S31-S35 are repeated.

[0093] Preferably, if the fields after generalization in this round do not meet K anonymity, the generalization results of each privacy field are saved for subsequent cyclic use;

[0094] In this cycle, the generalized object d in the previous cycle i The generalization results of the are further generalized, and the remaining fields use the saved generalization results.

[0095] For example, the three fields A, B, and C are designated to be desensitized using the generalization algorithm. That is, at this time, the privacy fields [D] = d1, d2, d3 = A, B, C. In the first round of loop, the three fields are generalized in one step to obtain A1, B1, and C1 respectively. Each generalized privacy field is combined with the remaining non-generalized privacy fields to form a new field set, and three generalization results are obtained: [D1 ’ ]=(A1,B,C),[D2 ’]=(A,B1,C),[D3 ’ ]=(A,B,C1), each generalization result corresponds to a new field set. Assuming that (A1,B,C) has the highest availability index, A is selected for generalization in this cycle. After the first cycle, the current data becomes [D1 ’ ]=(A1,B,C), and save the generalization results B1 and C1 of B and C at the same time. The second cycle [D]=[D1 ’ ]=(A1,B,C) generalize the three fields A1,B,C in one step to get A2,B1,C1 (B1 and C1 have been calculated in the first round of loop, so from the second round onwards, only one field needs to be calculated in each round of loop), and three generalization results [D1 ’ ]=(A2,B,C),[D2 ’ ]=(A1,B1,C),[D3 ’ ]=(A1,B,C1), assuming [D2 ’ ]=(A1,B1,C) has the highest availability, so this round of loop selects B for generalization. After the second round of loop, the current data becomes [D2 ’ ]=(A1,B1,C), and save the generalization results A2, C1, etc. of A1 and C at the same time. This process is repeated until the field after a certain round of generalization satisfies K anonymity, and then the desensitization action of this part of the field designated to use the generalization algorithm is completed.

[0096] Furthermore, in each round of the loop, by evaluating each new field set [D i ’ ] The availability index is calculated based on the impact of the performance index of the ensemble learner on the downstream machine learning tasks. This scheme describes the method of data availability after desensitization, and evaluates the availability by calculating the comprehensive availability index through the performance index of multiple machine learning tasks, which can intuitively and accurately reflect the availability of the desensitized data set. However, it should be noted that when it is put into use, if the technicians in this field use the basic idea of ​​using the availability index to guide the K-anonymity process of this scheme, even if other methods are proposed to conduct availability evaluation, it should still be within the scope of protection of this scheme.

[0097] Specifically, in each cycle, the field with the highest availability is selected for generalization in the following way:

[0098] Set m machine learning tasks (classification tasks or regression tasks) T1, T2, ..., T m , the number of machine learning tasks m = the number of quasi-identifier fields.

[0099] For each task T j Specify the corresponding target field and feature field, j = 1, 2, ... m;

[0100] Specifically, quasi-identifiers generally include numeric fields and categorical fields. For each categorical sensitive attribute field, a classification task can be used, and it can be used as the target field of a classification task, and the remaining fields can be used as feature fields of the task; for each numeric sensitive attribute field, a regression task can be used, and it can be used as the target field of a regression task, and the remaining fields can be used as feature fields of the task.

[0101] For each task T j Specify x base learners B j1 ,B j2 ,…,B jx , forming an integrated learner C j ;

[0102] Use each new field set [D i ’ ]Through multi-fold cross validation, we can j Train an ensemble learner instance C ij (Use the i-th privacy field to be generalized and the rest of the privacy fields to be not generalized, and use the new field set trained based on the j-th machine learning task to obtain the instance), and test the ensemble learner instance C ij To obtain the corresponding new field set [D i ’ ]The corresponding performance index is used as a one-sided indicator of the availability of task j;

[0103] According to the set weighted parameters, the m availability-specific indicators are weighted and summed to obtain a new field set [D i ’ ]Corresponding comprehensive availability index;

[0104] Through the above method, each new field set [D i ’ ], comparing the availability of each new field set [D i ’ ], select the new field set with the highest comprehensive availability index [D i ’ ]Field d i It is the generalized object of this round.

[0105] The weighting parameters are determined by those skilled in the art according to the requirements. For example, [D i ’ The comprehensive availability index of ] is determined as [D i ’ ]The average value of all availability indicators, not limited to this.

[0106] Further, in another embodiment, during the usability evaluation process, each new field set [D i ’ ] is combined with other data to train and test the learner to evaluate each new field set [D i ’ ] availability, other data refers to the data in the original dataset that does not need to be desensitized and the data that has been desensitized by a non-generalized algorithm.

[0107] This embodiment uses hospital patient data as an example for illustration. Assume that there is a patient data set of a hospital, which contains the patient's personal information and medical records. The system goal is to desensitize these data so that they can be used for medical research and data analysis while protecting the patient's privacy.

[0108] The data set fields include patient ID, name, gender, age, ID number, mobile phone number, email address, home address, medical history, drug allergy history, treatment plan, treatment cost, admission date, discharge date, diagnosis results and other information.

[0109] The goal of desensitization is to protect the patient's sensitive personal information, such as name, gender, age, ID number, mobile phone number, email address, and home address, while retaining the analytical value of the data, such as medical history, drug allergy history, treatment plan, etc.

[0110] In this scenario, the fields of quasi-identifiers include: age, treatment cost, admission and discharge date, gender, diagnosis result (name or status of the disease, each name belongs to a category), treatment plan (each treatment method belongs to a category), drug allergy history (yes, no), home address, etc. Different machine learning tasks can be set to evaluate the usability of desensitized data, such as:

[0111] Task 1: Predicting treatment options

[0112] Target field: Treatment options

[0113] Feature fields: age, treatment costs, admission and discharge dates, gender, diagnosis results, history of drug allergies (yes, no), home address, etc.

[0114] Task 2: Classification of diagnostic results

[0115] Target field: Diagnosis result

[0116] Feature fields: age, treatment costs, admission and discharge dates, gender, treatment plan, drug allergy history, home address, etc.

[0117] Task 3: Drug allergy prediction

[0118] Target field: Drug allergy history

[0119] Feature fields: age, treatment costs, admission and discharge dates, gender, diagnosis results, treatment plan, home address, etc.

[0120] The processing flow is as follows:

[0121] Data input: Obtain patient datasets from hospital systems.

[0122] Sensitive information identification: Use regular expressions to identify ID numbers, mobile phone numbers, and email addresses; use text classification models to identify names and home addresses.

[0123] Desensitization plan formulation: Apply masking or encryption desensitization algorithms to ID card numbers, mobile phone numbers, and email addresses. Apply generalization algorithms to align identifiers.

[0124] Desensitization scheme execution: Apply the desensitization algorithm to generate a desensitized data set. For fields that require the use of a generalization algorithm, a field that makes the corresponding field set most usable is cyclically selected for generalization until K anonymity is met.

[0125] Availability calculation and feedback: In each cycle, for each new field set [D i ’ ], and evaluate the performance of these instances (such as accuracy, recall, F1 score, etc.) on the test set. Calculate the partial usability index of each task, and sum them up to get the comprehensive usability index.

[0126] From multiple new field sets [D i ’ ], select [D] with the highest comprehensive availability index. i ’ ], determine the field d in the new field set i is the generalization object of this round. i True generalization, and then the next round of cycles until K anonymity is met.

[0127] Data output: The desensitized results of quasi-identifiers and other sensitive information are combined with the data that does not need to be desensitized to output the desensitized data set.

[0128] In this way, not only the patient's sensitive information is protected, but also the anesthetized data is ensured to have high usability when performing tasks such as treatment plan analysis, diagnosis result classification, and drug allergy prediction.

[0129] As used in this application, the terms "component", "module", "system", etc. are intended to refer to a computer-related entity, which can be hardware, firmware, a combination of hardware and software, software, or software in operation. For example, a component can be, but is not limited to: a process running on a processor, a processor, an object, an executable file, a thread in execution, a program, and / or a computer. As an example, an application running on a computing device and the computing device can both be components. One or more components can exist in a process and / or thread in execution, and a component can be located in a computer and / or distributed between two or more computers. In addition, these components can be executed from various computer-readable media having various data structures thereon. These components can communicate in a local and / or remote process manner, such as according to a signal having one or more data packets (e.g., data from a component that interacts with another component in a local system, a distributed system, and / or interacts with other systems in a signal manner through a network such as the Internet).

[0130] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. An adaptive personal information desensitization method based on usability evaluation, characterized in that: The method includes: S1. Sensitive information identification: receiving the original data set to be desensitized and identifying the privacy fields; S2. Desensitization plan formulation: formulate corresponding desensitization strategies for privacy fields, and at least two fields are specified to use generalized algorithms; S3. Execution of the desensitization scheme: for the privacy fields for which the generalization algorithm is specified, a field that makes the corresponding field set most available is cyclically selected for generalization until K anonymity is satisfied; For the remaining privacy fields, desensitize the corresponding privacy fields according to the formulated desensitization strategy; S4. Combine the desensitization results of the remaining privacy fields and the privacy fields of the specified generalization algorithm to obtain the desensitized dataset.

2. The method for self-adaptive personal information desensitization based on usability evaluation according to claim 1, characterized in that: In step S3, each cycle includes the following process: S31. Treating each privacy field d1, d2, ... d of the generalized field set [D] n Perform one-step generalization respectively, [D] represents the original field set of the specified generalization algorithm or the field set after generalization processing in the previous cycle, and n represents the number of privacy fields of the specified generalization algorithm; S32. Each generalized privacy field is combined with the remaining non-generalized privacy fields to form a new field set [D i ’ ], i = 1, 2, ... n; [D i ’ ] indicates a new field set in which the i-th privacy field in the [D] field set is generalized and the remaining privacy fields are not generalized; S33. Calculate each new field set by the availability calculation method [D i ’ ]’s availability indicators; S34. Select the new field set with the highest usability index [D i ’ ] and determine the field d in the new field set i is the generalized object of this round; S35. If the new field set after this round of generalization [D i ’ ] satisfies K anonymity, the desensitization process ends, otherwise, the new field set [D i ’ ] is the field set to be generalized [D], and steps S31-S35 are repeated.

3. The method for self-adaptive personal information desensitization based on usability evaluation according to claim 2, characterized in that: In step S3, if the fields after generalization in this round do not meet K anonymity, the generalization results of each privacy field are saved for subsequent cyclic use; In this cycle, the generalized object d in the previous cycle i The generalization results of the are further generalized, and the remaining fields use the saved generalization results.

4. The method for self-adaptive personal information desensitization based on usability evaluation according to claim 3, characterized in that: In each round of the loop, by evaluating each new set of fields [D i ’ ] is used to calculate the availability index based on its impact on the performance index of the ensemble learner on downstream machine learning tasks.

5. The method for self-adaptive personal information desensitization based on usability evaluation according to claim 4, characterized in that: The availability index is a comprehensive availability index obtained by weighting multiple partial availability indicators; And by setting multiple different machine learning tasks, multiple one-sided availability indicators are obtained.

6. The method for self-adaptive personal information desensitization based on usability evaluation according to claim 5, characterized in that: In step S3, in each cycle, the field that makes the corresponding field set most usable is selected for generalization in the following manner: Set m machine learning tasks T1, T2, ..., T m , m is a natural number greater than 2; For each task T j Specify the corresponding target field and feature field, j = 1, 2, ... m; For each task T j Specify x base learners B j1 ,B j2 ,…,B jx , forming an integrated learner C j ; In step S3, each new field set [D i ’ ]Through multi-fold cross validation, we can j Train an ensemble learner instance C ij , and test the ensemble learner instance C ij To obtain the corresponding new field set [D i ’ ]The corresponding performance index is used as a one-sided indicator of the availability of task j; According to the set weighted parameters, the m availability-specific indicators are weighted and summed to obtain a new field set [D i ’ ]Corresponding comprehensive availability index; Compare the new field sets [D i ’ ], select the new field set with the highest comprehensive availability index [D i ’ ]Field d i It is the generalized object of this round.

7. The method for self-adaptive personal information desensitization based on usability evaluation according to claim 1, characterized in that: In step S1, the original data set to be desensitized is a structured data table obtained from a data source; After obtaining the structured data table, classify the fields therein and identify the privacy fields containing personal information.

8. The method for self-adaptive personal information desensitization based on usability evaluation according to claim 6, characterized in that: In step S1, direct identifiers, quasi-identifiers and non-identifiers are identified from the original data set through an identifier identification method, and direct identifiers and quasi-identifiers are identified as privacy fields; In step S2, the generalization algorithm is specified for the alignment identifier.

9. The method for self-adaptive personal information desensitization based on usability evaluation according to claim 8, characterized in that: In step S2, the number of machine learning tasks m = the number of quasi-identifier fields, and the machine learning tasks are set as follows: For each task T j , the jth field is used as the target field of the task, and the remaining fields are used as the feature fields of the task.

10. An adaptive personal information desensitization system based on usability evaluation based on the method according to any one of claims 1 to 9, characterized in that: The system includes: A data input module, used to receive the original data set to be desensitized; Sensitive information identification module, used to identify privacy fields in data; A desensitization scheme formulation module is used to formulate a desensitization strategy and specify the use of a generalization algorithm for at least two fields; The desensitization execution module is used to execute the desensitization strategy for the remaining privacy fields; loop through the privacy fields for which the generalization algorithm is specified, and select a field that makes the corresponding field set most available according to the availability index to generalize until the K anonymous detection conditions are met; Usability evaluation module, used to calculate usability indicators and provide feedback to the desensitization execution module; The data output module is used to merge the anonymization results and output the anonymized data set.

Images