A method and system for user sensitive data desensitization for local databases
By constructing a generalized hierarchical tree in a local database and dynamically setting the initial K value of the K-anonymity algorithm, the problem of insufficient handling of data sensitivity differences in existing technologies is solved, achieving efficient data desensitization and ensuring the privacy and security of highly sensitive data and the availability of low-sensitivity data.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- SICHUAN COMMERCIAL INVESTMENT INFORMATION TECH CO LTD
- Filing Date
- 2026-04-28
- Publication Date
- 2026-07-07
AI Technical Summary
Existing data anonymization technologies cannot effectively handle the varying degrees of sensitivity among different technical terms, resulting in insufficient protection of highly sensitive information or excessive anonymization of less sensitive information, thus disrupting the balance between data privacy protection and data utility.
By acquiring structured data from a local database, segmentation and word segmentation are performed to extract semantic vectors of keywords and sub-words. A generalized hierarchical tree is constructed using clustering algorithms and open-source knowledge graphs. Keyword path similarity is calculated to select representative keywords for desensitization. Based on the desensitization index, the initial K value of the K-anonymity algorithm is dynamically set to divide the data into high and low sensitivity groups for differentiated desensitization.
It achieves the best balance between data privacy protection and data utility by reducing system overhead, accurately identifying keyword associations, and dynamically matching the desensitization intensity while ensuring semantic consistency.
Smart Images
Figure CN122113172B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of data anonymization technology, specifically to a method and system for anonymizing user-sensitive data in a local database. Background Technology
[0002] The local database stores a large amount of structured data from enterprise customers. During data transmission, in order to avoid leaking customers' sensitive information and privacy data, the system needs to perform anonymization processing on the data to be stored to ensure that the system can effectively protect customers' privacy rights.
[0003] Existing data anonymization techniques, such as the K-anonymity algorithm, typically rely on a preset initial K value to perform uniform anonymization on target data. However, in real-world scenarios, customer data to be stored often contains various technical terms with significantly different levels of sensitivity, requiring varying degrees of anonymization. Traditional uniform processing methods fail to adequately consider the heterogeneity in sensitivity among different target data, easily leading to insufficient protection of highly sensitive information and the risk of leakage, or excessive anonymization of less sensitive information rendering it unusable. This severely disrupts the balance between data privacy protection and data utility. Summary of the Invention
[0004] To address the aforementioned technical problems, the purpose of this application is to provide a method and system for de-identifying user-sensitive data in a local database. The specific technical solution adopted is as follows:
[0005] In a first aspect, embodiments of this application provide a method for de-identifying user-sensitive data in a local database, the method comprising the following steps:
[0006] Obtain the structured data to be stored in the local database, divide the structured data into data to be de-identified based on the metadata information of the data table, perform segmentation and word segmentation on the data to be de-identified, and extract keywords and corresponding sub-words and semantic vectors of the sub-words;
[0007] The semantic feature vectors of keywords are determined based on the semantic vectors of sub-words; the keywords are clustered based on the similarity of the semantic feature vectors between keywords, and a generalized hierarchical tree of each cluster is constructed by combining open-source knowledge graphs.
[0008] In the generalized hierarchical tree, the path similarity between any two keywords is calculated to determine whether there is a relationship between the two keywords, and representative de-identified keywords are selected based on the relationship.
[0009] Based on the node depth of the generalization hierarchy tree, the number of keywords with related relationships, the number of keywords representing desensitization, and the uncertainty of keyword distribution, a desensitization index is constructed.
[0010] Based on the aforementioned desensitization index, all generalized hierarchical trees are divided into high-sensitivity groups and low-sensitivity groups. Initial K values in the K-anonymity algorithm of different magnitudes are dynamically set for the high-sensitivity groups and low-sensitivity groups to perform desensitization processing on the data to be desensitized.
[0011] Preferably, the process for determining the semantic feature vector of the keyword is as follows:
[0012] The frequency of each sub-word corresponding to the statistical keyword in the data to be de-identified is used as the weight of the semantic vector of each sub-word. The result of the weighted average of the semantic vectors of all sub-words is used as the semantic feature vector of the keyword.
[0013] Preferably, the distance metric used in the keyword clustering process is the Euclidean distance between the semantic feature vectors of the keywords.
[0014] Preferably, the construction process of the generalization hierarchical tree of each cluster is as follows:
[0015] For each cluster, the preset top-level root node of the domain is used as the root node of the generalization hierarchy tree, and the keywords in the cluster are used as leaf nodes. Based on the entity hierarchy relationship in the open-source knowledge graph, the higher-level concept paths from each keyword to the root node are merged to obtain a tree structure, which serves as the generalization hierarchy tree.
[0016] Preferably, determining whether any two keywords have a correlation includes:
[0017] In the generalized hierarchical tree, the mean of the path similarity between all keywords is calculated and denoted as the similarity threshold.
[0018] If the path similarity between two keywords is greater than the preset similarity threshold, then the nodes on the path from the two keywords to their corresponding common parent node are counted respectively, forming the path node sequence of each keyword. Based on the similarity between the path node sequences of the two keywords and the difference in the length of the path node sequences, a similarity coefficient is constructed between the two keywords. Otherwise, no similarity coefficient is constructed.
[0019] In the path node sequences of two keywords, starting from the starting element of the longest path node sequence, all elements with the same length as the shortest path node sequence are obtained to form a reference sequence of the shortest path node sequence. If the similarity coefficient between the two keywords is greater than or equal to a preset threshold, and the shortest path node sequence is exactly the same as its reference sequence, then it is determined that there is a correlation between the two keywords; otherwise, there is no correlation.
[0020] Preferably, the step of filtering de-identified representative keywords based on the association relationship includes:
[0021] In the generalized hierarchical tree, keywords with related relationships are grouped into related keyword pairs. The common parent node with the largest hierarchical depth among all common parent nodes corresponding to any related keyword pair is used as the representative keyword for desensitization.
[0022] Preferably, the process of constructing the desensitization index is as follows:
[0023] Desensitization index of generalized hierarchical tree The expression is: In the formula, This indicates the node depth of the generalized hierarchical tree; This indicates the number of keywords with related relationships in the generalization hierarchy tree; This indicates the number of desensitized keywords in the generalization hierarchy tree; This represents the quantitative result of the uncertainty in the distribution of keywords in the generalized hierarchical tree.
[0024] Preferably, the quantification process of the uncertainty in the keyword distribution in the generalized hierarchical tree is as follows:
[0025] Calculate the mean of the semantic feature vectors of all keywords with related relationships in the generalization hierarchy tree, and use it as the baseline vector;
[0026] The information entropy of the similarity between the semantic feature vectors of all related keywords and the baseline vector is calculated separately, which serves as a quantification of the uncertainty of keyword distribution in the generalization hierarchical tree.
[0027] Preferably, the step of dividing all generalized hierarchical trees into high-sensitivity groups and low-sensitivity groups based on the desensitization index includes:
[0028] The desensitization index of all generalized hierarchical trees is used as input to a threshold segmentation algorithm, which outputs a segmentation threshold. Generalized hierarchical trees with desensitization indices greater than the segmentation threshold are classified as high-sensitivity groups, and the remaining generalized hierarchical trees are classified as low-sensitivity groups. Secondly, embodiments of this application also provide a user-sensitive data desensitization system for a local database, including a memory, a processor, and a computer program stored in the memory and running on the processor. When the processor executes the computer program, it implements the steps of any one of the above-described user-sensitive data desensitization methods for a local database.
[0029] This application has at least the following beneficial effects:
[0030] This application first constructs keyword semantic feature vectors using weighted averaging, and then uses clustering algorithms and open-source knowledge graphs to build a generalized hierarchical tree, transforming discrete semantic information into structured hierarchical relationships. This not only provides a unified semantic basis for subsequent desensitization rules and generalization operations, but also effectively solves the problem of cumbersome computation when processing large-scale keywords individually, helping to reduce system overhead while ensuring semantic consistency. Furthermore, by calculating path similarity and similarity coefficients, the application accurately identifies the associations between keywords and selects the highest-level representative keywords for desensitization, achieving unified reuse of desensitization rules within a group. This process effectively avoids redundant operations of independently calculating desensitization paths for each keyword, helping to reduce system computational overhead and improve desensitization efficiency while ensuring semantic consistency. Furthermore, this application integrates node depth and association... By considering multiple dimensions such as the number of keywords, semantic distribution uncertainty, and the number of keywords representing desensitization, a desensitization index was constructed to quantify the comprehensive sensitivity risk of each group. This index not only accurately reflects the impact of data hierarchy span, cluster density, and semantic dispersion on privacy risks, but also provides a scientific basis for the dynamic setting of the initial K value. This effectively solves the problem of insufficient protection for highly sensitive data or impaired utility of low-sensitive data caused by uniform desensitization intensity. Finally, this application divides the generalized hierarchical tree into high and low sensitivity groups through a threshold segmentation algorithm, and dynamically sets differentiated initial K values based on the ranking within each group. This achieves a precise match between desensitization intensity and data sensitivity. High-intensity generalization ensures the privacy and security of highly sensitive data, while moderate processing preserves the usability of low-sensitive data, thus achieving the best balance between data privacy protection and data utility. Attached Figure Description
[0031] To more clearly illustrate the technical solutions and advantages in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0032] Figure 1 A flowchart illustrating the steps of a method for de-identifying user-sensitive data in a local database, as provided in one embodiment of this application;
[0033] Figure 2 This is a flowchart of a generalized hierarchical tree grouping process provided in one embodiment of this application. Detailed Implementation
[0034] To further illustrate the technical means and effects adopted by this application to achieve the intended purpose of the invention, the following, in conjunction with the accompanying drawings and preferred embodiments, details the specific implementation, structure, features, and effects of a user-sensitive data desensitization method and system for a local database proposed in this application. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. Furthermore, specific features, structures, or characteristics in one or more embodiments can be combined in any suitable form.
[0035] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains.
[0036] The following description, in conjunction with the accompanying drawings, details a specific scheme for a method and system for desensitizing user-sensitive data in a local database provided in this application.
[0037] Please see Figure 1 The diagram illustrates a flowchart of a method for de-identifying user-sensitive data in a local database according to an embodiment of this application. The method includes the following steps:
[0038] Step S1: Obtain the structured data to be stored in the local database, divide the structured data into data to be de-identified based on the metadata information of the data table, segment and word-segment the data to be de-identified, and extract keywords and corresponding sub-words and semantic vectors of the sub-words.
[0039] Databases are typically built upon distributed system infrastructure, SQL database management systems, or database management systems. This embodiment uses an SQL database management system as an example, implementing data transmission through preset SQL commands and automatically acquiring structured data during the transmission process.
[0040] This embodiment uses a company's database as an example. It employs SQL commands to retrieve structured data to be stored in the company's local database. Based on metadata information such as field names and field types, the collected structured data is identified and classified.
[0041] Highly sensitive data such as customer personal information, company profile, and amounts involved are marked as absolutely anonymized data;
[0042] Text data such as content outlines and specific project details are marked as data to be de-identified.
[0043] Among them, the data table is a pre-established storage structure in the database.
[0044] Furthermore, the text content of the data to be anonymized is sequentially subjected to segmentation, word segmentation, vectorization, and keyword extraction operations, as follows:
[0045] (1) Text segmentation processing.
[0046] Since the original text is quite long, it needs to be segmented first. The data to be de-identified is used as input, and text segmentation parameters are set (the value range of the text segmentation parameter is [200-500] characters; in this embodiment, the text segmentation parameter is set to 300). The HanLP algorithm is used to segment the input data, resulting in several text data segments. The process of segmenting text using the HanLP algorithm is a well-known technique and will not be described in detail here.
[0047] (2) Word segmentation and semantic feature vector acquisition.
[0048] To accurately calculate semantic features, each text data segment is segmented into words. These text data segments are then used as input to the BPE (Byte Pair Encoding) algorithm. The vocabulary size (range [2000, 100000], set to 5000 in this embodiment) and minimum word frequency (range [1, 10], set to 2 in this embodiment) are set. Finally, all sub-words are output. Sub-words refer to semantic units that are finer-grained than traditional words. Based on the segmentation results, semantic vectors corresponding to each sub-word are obtained through word embedding methods.
[0049] The process of using the BPE algorithm to segment text and the process of using word embedding to obtain the semantic vectors of each sub-word are well-known techniques and will not be described in detail here.
[0050] (3) Keyword extraction.
[0051] Based on the word segmentation results, core keywords are extracted. All text data segments are used as input to the TextRank algorithm. The window size (the value range of the window size is [2, 20], and the window size is set to 5 in this embodiment) and the damping coefficient (the value range of the damping coefficient is [0, 1], and the damping coefficient is set to 0.85 in this embodiment) are set. The output is the keywords in the data to be de-identified extracted from the word segmentation results.
[0052] The process of extracting keywords from text using the TextRank algorithm is a well-known technique and will not be elaborated further.
[0053] This completes the data collection and preprocessing.
[0054] Step S2: Determine the semantic feature vector of the keyword based on the semantic vector of the sub-word; cluster the keywords based on the similarity of the semantic feature vectors between the keywords, and construct a generalized hierarchical tree for each cluster by combining the open-source knowledge graph.
[0055] Directly desensitizing a large number of discrete keywords leads to low processing efficiency and makes it difficult to guarantee semantic consistency. Constructing a generalized hierarchical tree can organize semantically similar keywords into a hierarchical tree structure, thereby clarifying the hierarchical semantic relationship between each keyword and providing a unified structural basis for subsequent grouping based on semantic paths, formulating desensitization rules, and implementing generalization operations.
[0056] The original keyword set contains a large amount of data. Constructing a generalized hierarchical tree for each keyword individually would result in a cumbersome computational process and a loose structure, significantly increasing system overhead. Therefore, this embodiment determines the semantic feature vector of keywords based on the semantic vectors of sub-words; it then clusters keywords based on the similarity of their semantic feature vectors and, in conjunction with an open-source knowledge graph, constructs a generalized hierarchical tree for each cluster. The specific process is as follows:
[0057] First, the semantic feature vector of the keyword is determined based on the semantic vector of the sub-word. Specifically:
[0058] The frequency of each sub-word corresponding to the statistical keyword in the data to be de-identified is used as the weight of the semantic vector of each sub-word. The result of the weighted average of the semantic vectors of all sub-words is used as the semantic feature vector of the keyword.
[0059] Since the semantic feature vectors of each keyword are quite complex when constructing the generalized hierarchical tree, directly performing subsequent calculations would increase computational overhead. Therefore, this embodiment performs dimensionality reduction processing on the semantic feature vectors of each keyword. The semantic feature vectors used in subsequent calculations and analyses are all dimensionality-reduced semantic feature vectors. The specific dimensionality reduction process is as follows: the semantic feature vectors of each keyword are used as input to the dimensionality reduction algorithm. In this embodiment, the principal component analysis (PCA) algorithm is used to reduce the dimensionality of the semantic feature vectors. In this embodiment, the target dimensionality reduction dimension is set to 50% of the original dimension. Note that the dimension after dimensionality reduction must be a positive number, which can be achieved by rounding down. Finally, the dimensionality-reduced semantic feature vector is output.
[0060] The process of using principal component analysis (PCA) to reduce the dimensionality of vectors is a well-known technique and will not be elaborated further.
[0061] Furthermore, this embodiment clusters keywords based on the similarity of their semantic feature vectors. Specifically:
[0062] In this embodiment, all keywords in the data to be de-identified are used as input to the clustering algorithm. The k-means clustering algorithm is used to cluster the data. The distance metric is set as the Euclidean distance between the semantic feature vectors of the keywords. The number of clusters is determined by the elbow rule. Finally, all clusters are output.
[0063] The process of clustering using the k-means clustering algorithm, the process of determining the number of clusters using the elbow rule, and the Euclidean distance are all well-known techniques and will not be elaborated further.
[0064] It should be noted that since the dimension of the semantic vector of the sub-words obtained in step S1 is fixed, the dimension of the semantic feature vector of the keywords obtained after weighted averaging is also fixed. Therefore, the length of the semantic feature vectors of different keywords is the same, and the cosine similarity can be calculated directly.
[0065] Furthermore, this embodiment constructs a generalized hierarchical tree for each cluster based on the clustering results and in conjunction with an open-source knowledge graph. Specifically:
[0066] For each cluster, the preset top-level root node of the domain is used as the root node of the generalization hierarchy tree, and the keywords in the cluster are used as leaf nodes. Based on the entity hierarchy relationship in the open-source knowledge graph, the higher-level concept paths from each keyword to the root node are merged to obtain a tree structure, which serves as the generalization hierarchy tree.
[0067] To facilitate understanding, let's illustrate with a concrete example: Suppose the keyword extracted from the data to be anonymized is "small cell lung cancer". First, the pre-defined root node of the medical field, "disease", is used as the root node of the generalized hierarchical tree, and "small cell lung cancer" in the cluster is used as the leaf node. Next, the path of its higher-level concept is retrieved based on the open-source knowledge graph, resulting in "small cell lung cancer → lung cancer → respiratory system diseases → disease". Subsequently, the nodes on this path are connected from bottom to top, and the same nodes are merged (if there is also "lung adenocarcinoma" in the cluster, the "lung cancer" node in its path will be shared with "small cell lung cancer"), thus constructing a generalized hierarchical tree with "disease" as the root node, "small cell lung cancer" as the leaf node, and a clear hierarchical relationship.
[0068] It should be noted that the open-source knowledge graph used in this embodiment is CN-DBpedia. In actual applications, as other implementation methods, implementers may also use Hownet depending on the specific circumstances. This embodiment does not impose any special restrictions.
[0069] The process of using open-source knowledge graphs to assist in building a generalized hierarchical tree is a well-known technique and will not be elaborated further.
[0070] Thus, this embodiment constructs keyword semantic feature vectors through weighted averaging and uses clustering algorithms and open-source knowledge graphs to construct generalized hierarchical trees, transforming discrete semantic information into structured hierarchical relationships. This not only provides a unified semantic basis for subsequent desensitization rules and implementation of generalization operations, but also effectively solves the problem of cumbersome calculations when processing large-scale keywords individually, helping to reduce system overhead while ensuring semantic consistency.
[0071] Step S3: In the generalized hierarchical tree, calculate the path similarity between any two keywords to determine whether there is a relationship between the two keywords, and select representative keywords for desensitization based on the relationship.
[0072] While constructing a generalized hierarchical tree clarifies the hierarchical structure of each keyword, in actual desensitization, performing desensitization calculations independently for each keyword in the tree still results in a large number of redundant operations, leading to low system processing efficiency. Analysis revealed that within the same generalized hierarchical tree, some keywords exhibit strong semantic connections, and their desensitization rules often have consistency or inclusion relationships, eliminating the need for repeated calculations. Therefore, to reduce computational overhead and improve desensitization efficiency, this embodiment calculates the path similarity between any two keywords within the generalized hierarchical tree to determine whether there is a correlation between them. Based on this correlation, representative keywords for desensitization are selected, laying the foundation for subsequently developing efficient desensitization rules. The specific process is as follows:
[0073] First, in the generalized hierarchical tree, the path similarity between any two keywords is calculated to determine whether there is a correlation between the two keywords. Specifically:
[0074] In the generalized hierarchical tree, the mean of the path similarity between all keywords is calculated and denoted as the similarity threshold.
[0075] If the path similarity between two keywords is greater than a preset similarity threshold, then the nodes on the paths leading to their corresponding common parent nodes are counted separately, forming separate path node sequences for each keyword. Based on the similarity between the path node sequences and the difference in path node sequence lengths, a similarity coefficient is constructed between the two keywords. Otherwise, no similarity coefficient is constructed. The expression for the similarity coefficient between two keywords is as follows: In the formula, This represents the similarity coefficient between keyword i and keyword j; This represents the cosine similarity between the path node sequences of keyword i and keyword j. This represents the absolute difference in length between the path node sequences of keyword i and keyword j; This represents a preset constant greater than 0, used to prevent the denominator from being 0. In this embodiment... The value of is 0.01. Under the premise of ensuring that the denominator is not 0 and does not excessively affect the calculation result, the implementer can also set it according to the specific situation. This embodiment does not impose any special restrictions. The similarity threshold is 0.4 in this embodiment. The implementer can also set it according to the specific situation. This embodiment does not impose any special restrictions. The calculation process of cosine similarity is a well-known technology and will not be described in detail.
[0076] In the path node sequences of two keywords, starting from the starting element of the longest path node sequence, all elements with the same length as the shortest path node sequence are obtained to form a reference sequence of the shortest path node sequence. If the similarity coefficient between the two keywords is greater than or equal to a preset threshold, and the shortest path node sequence is exactly the same as its reference sequence, then it is determined that there is a correlation between the two keywords; otherwise, there is no correlation.
[0077] It should be noted that the method for calculating the path similarity between key nodes in this embodiment adopts the Lowest Common Ancestor (LCA) algorithm. The process of calculating path similarity using the LCA algorithm is a well-known technique and will not be described in detail here.
[0078] It should be noted that in this embodiment, the preset threshold is the mean of all similarity coefficients.
[0079] It should be noted that, since different keywords have different depths in the generalization hierarchy tree, the lengths of their path node sequences are also inconsistent, making it impossible to directly calculate the similarity between sequences. Therefore, this embodiment uses a one-hot encoding algorithm to vectorize the path node sequences: First, all nodes appearing in the generalization hierarchy tree are counted, and a complete set of nodes is constructed as the encoding dictionary; Second, for the path node sequence of each keyword, it is mapped to a binary vector with the same length as the encoding dictionary; If a node exists in the path of the keyword, the corresponding position in the vector is 1, otherwise it is 0. After the above processing, the path node sequences of all keywords are transformed into feature vectors with uniform dimensions, so that the similarity between sequences can be calculated.
[0080] To facilitate understanding, a specific calculation example of the one-hot coding algorithm is given: Assume that all nodes in the generalized hierarchical tree have only 4 elements: {root node, traffic item, bridge, road}; the path of keyword A (bridge) is: root node → traffic item → bridge, encoding vector: [1, 1, 1, 0] (root present, traffic present, bridge present, no road present); the path of keyword B (road) is: root node → traffic item → road, encoding vector: [1, 1, 0, 1] (root present, traffic present, no bridge present, road present).
[0081] Furthermore, based on the aforementioned association, this embodiment filters representative keywords for de-identification, specifically:
[0082] In this embodiment, in the generalization hierarchy tree, keywords with related relationships are grouped into related keyword pairs. The common parent node with the highest level depth among all common parent nodes corresponding to any related keyword pair is used as the de-identification representative keyword. The de-identification representative keyword is used to represent the core semantic features and the highest level of security coverage within the same related keyword group. Its role in subsequent content is to achieve unified de-identification management at the group level. Specifically, since there is a hierarchical semantic inclusion relationship between related keywords, the de-identification representative keyword, as the highest level (coarsest granularity) node within the group, naturally covers the semantic range of all related keywords within the group in its generalization result. When performing K-anonymization processing later, all keywords within the group directly reuse the generalization rules of the de-identification representative keyword, avoiding redundant operations of repeatedly calculating the de-identification path for each keyword. Thus, while ensuring data semantic consistency, the computational overhead of the system is greatly reduced and the de-identification efficiency is improved.
[0083] Thus, this embodiment accurately identifies the association between keywords by calculating path similarity and similarity coefficient, and selects the highest-level representative keywords for desensitization, realizing the unified reuse of desensitization rules within the group. This process effectively avoids the redundant operation of calculating the desensitization path independently for each keyword, and helps to reduce system computational overhead and improve desensitization efficiency while ensuring semantic consistency.
[0084] Step S4: Construct a desensitization index based on the node depth of the generalization hierarchy tree, the number of keywords with related relationships, the number of desensitization representative keywords, and the uncertainty of keyword distribution.
[0085] Although keywords were categorized into different groups using desensitization rules, and representative desensitized keywords were selected, enabling rule reuse within each group, significant differences still exist in the data sensitivity and desensitization requirements of different groups when actually implementing the K-anonymity algorithm. Applying a uniform desensitization strength to all groups may still result in insufficient protection for highly sensitive groups or impaired data utility for low-sensitive groups. Therefore, to quantify the overall sensitivity risk of each keyword group, this embodiment constructs a desensitization index based on the node depth of the generalization hierarchy tree, the number of related keywords, the number of representative desensitized keywords, and the uncertainty of keyword distribution. This index guides the dynamic setting of the initial K value in the K-anonymity algorithm. The specific process is as follows:
[0086] As one implementation method, in this embodiment, the desensitization index of the generalized hierarchical tree is... The expression is: In the formula, This indicates the node depth of the generalized hierarchical tree; This indicates the number of keywords with related relationships in the generalization hierarchy tree; This indicates the number of desensitized keywords in the generalization hierarchy tree; This represents the quantitative result of the uncertainty in the distribution of keywords in the generalized hierarchical tree.
[0087] It should be noted that if no desensitized representative keywords are selected from the generalization hierarchy tree, that is... At that time, The value is set to 1 for correction.
[0088] The quantification process of the uncertainty in keyword distribution in the generalized hierarchical tree is as follows:
[0089] Calculate the mean of the semantic feature vectors of all keywords with related relationships in the generalization hierarchy tree, and use it as the baseline vector;
[0090] Since cosine similarity may have negative values and the sum is not equal to 1, all calculated cosine similarities need to be normalized first. In this embodiment, the Softmax function is used to map the similarity to a probability distribution, so that its value range is in the interval (0,1) and the sum is 1. The mapped probability values are substituted into the Shannon entropy formula for calculation. The information entropy of the similarity between the semantic feature vectors of all related keywords and the baseline vector is calculated respectively, which is used as the quantification result of the uncertainty of keyword distribution in the generalization hierarchical tree.
[0091] It should be noted that in this embodiment, the cosine similarity between the semantic feature vectors of all related keywords and the baseline vector is used as the similarity between the semantic feature vectors of all related keywords and the baseline vector; the calculation process of Shannon entropy is a well-known technique and will not be described in detail here.
[0092] Based on the desensitization index, it can be understood that the desensitization index is used to characterize the comprehensive sensitivity risk of each keyword group in terms of semantic structure and distribution characteristics. Its calculation is mainly affected by node depth, the number of keywords with related relationships, the number of keywords representing desensitization, and the uncertainty of keyword distribution; specifically, node depth... The number of keywords that are related to each other As a molecule, a larger value directly leads to a larger desensitization index, reflecting a large data hierarchy span within the group or a high keyword cluster density, thus increasing the sensitivity risk; the quantitative result of keyword distribution uncertainty. The larger the value, the higher the anonymization index, reflecting a high degree of dispersion and inconsistency in the semantic features of keywords within the group, increasing the risk of privacy leakage; while anonymization represents the number of keywords. As the denominator, a smaller value leads to a larger desensitization index, reflecting that a large number of related keywords within the group are clustered in a very small number of representative nodes, forming a high-density semantic aggregation. The size of the anonymized set of the data is too small, resulting in excessive exposure of privacy details, which is a high-risk situation. Therefore, it is necessary to increase the desensitization intensity for key protection. Conversely, the smaller the node depth and the number of related keywords, the smaller the quantitative result T of the uncertainty of keyword distribution, and the larger the number of desensitized representative keywords, the smaller the desensitization index will be. This reflects that the data hierarchy span within the group is small, the keyword cluster density is low, the semantic feature consistency is strong, and the risk is dispersed, which is a low-risk situation. Therefore, the desensitization intensity should be reduced to retain the usability of the data to the greatest extent.
[0093] Thus, this embodiment constructs a desensitization index that quantifies the comprehensive sensitivity risk of each group by integrating multi-dimensional features such as node depth, number of associated keywords, semantic distribution uncertainty, and number of desensitized representative keywords. This not only accurately reflects the impact of data hierarchy span, cluster density, and semantic dispersion on privacy risks, but also provides a scientific basis for the dynamic setting of the initial K value, thereby effectively solving the problem of insufficient protection of highly sensitive data or impaired utility of low-sensitive data caused by uniform desensitization intensity.
[0094] Step S5: Based on the desensitization index, divide all generalized hierarchical trees into high-sensitivity groups and low-sensitivity groups, and dynamically set the initial K value of the K-anonymity algorithm of different magnitudes for the high-sensitivity groups and low-sensitivity groups, so as to perform desensitization processing on the data to be desensitized.
[0095] While the desensitization index quantifies the comprehensive sensitivity of each keyword group in terms of semantic structure and distribution characteristics, it is merely a numerical evaluation result and cannot directly affect the data desensitization process. The core parameter of the K-anonymity algorithm—the initial K value—directly determines the strength of data generalization and the final security of the data. Therefore, to transform the desensitization index into actual desensitization capability, a mapping mechanism between the desensitization index and the initial K value needs to be established. Based on the desensitization index, all generalization hierarchical trees are divided into high-sensitivity and low-sensitivity groups. Initial K values of different magnitudes in the K-anonymity algorithm are dynamically set for both the high-sensitivity and low-sensitivity groups to perform desensitization processing on the data to be desensitized. The specific process is as follows:
[0096] First, in this embodiment, based on the desensitization index, all generalized hierarchical trees are divided into high-sensitivity groups and low-sensitivity groups, specifically:
[0097] In this embodiment, the desensitization index of all generalized hierarchical trees is used as the input of the threshold segmentation algorithm, and the output is the segmentation threshold. Generalized hierarchical trees with desensitization index greater than the segmentation threshold are classified as high-sensitivity groups, and the remaining generalized hierarchical trees are classified as low-sensitivity groups.
[0098] Preferably, the flowchart of the generalized hierarchical tree grouping process provided in this embodiment is as follows: Figure 2 As shown.
[0099] It should be noted that there are many commonly used threshold segmentation algorithms. In this embodiment, the Otsu threshold segmentation algorithm is used to obtain the segmentation threshold. In practical applications, as other implementation methods, implementers may also use other threshold segmentation algorithms according to specific circumstances. This embodiment does not impose any special restrictions.
[0100] The process of obtaining the segmentation threshold using the Otsu threshold segmentation algorithm is a well-known technique and will not be described in detail here.
[0101] Initial K value optimization value in the K-anonymous algorithm The expression is: In the formula, , These represent the preset upper and lower limits of the initial K value, respectively; , They are respectively represented as Set upper limits for the maximum and minimum values; X represents the order value of the generalization hierarchy tree i in the corresponding group; X represents the high-sensitivity group; Y represents the low-sensitivity group; max() represents the maximum value function.
[0102] It should be noted that, in this embodiment, , The values are 8 and 3 respectively; , The values are 15 and 7 respectively.
[0103] Specifically, the generalization hierarchical trees in the high-sensitivity group are arranged in ascending order according to their corresponding desensitization indices, and the arrangement results are numbered sequentially starting from 0. This yields the order value of each generalization hierarchical tree in the high-sensitivity group. Similarly, for the generalization hierarchical trees in the low-sensitivity group, the same numbering rules as those for the high-sensitivity group are used to number the generalization hierarchical trees in the low-sensitivity group. The specific numbering process will not be elaborated further.
[0104] Furthermore, based on the optimized initial K value, for the highly sensitive group, the system performs a deeper node-up operation in the generalization hierarchy tree according to its larger K value parameter, generalizing the keywords into broader and coarser-grained higher-level concepts to ensure enhanced privacy protection by increasing the size of the anonymity set; for the low-sensitive group, the system performs a shallower node-up operation in the generalization hierarchy tree according to its smaller K value parameter, only generalizing the keywords into semantically similar fine-grained higher-level nodes, preserving the original semantic features of the data to the greatest extent while meeting the anonymity requirements; finally, the generalized keywords replace the original data to generate a de-anonymized dataset that meets the K-anonymity constraint and completes the transmission.
[0105] The process of using the K-anonymity algorithm to anonymize the data is a well-known technique and will not be elaborated further.
[0106] Thus, this embodiment divides the generalized hierarchical tree into high and low sensitivity groups using a threshold segmentation algorithm, and dynamically sets differentiated initial K values based on the sorting within each group. This achieves a precise match between the desensitization intensity and the data sensitivity level. It ensures the privacy and security of highly sensitive data through high-intensity generalization, while retaining the usability of low-sensitive data through appropriate processing, thereby achieving the best balance between data privacy protection and data utility.
[0107] Based on the same inventive concept as the above methods, this application also provides a user-sensitive data de-identification system for a local database, including a memory, a processor, and a computer program stored in the memory and running on the processor. When the processor executes the computer program, it implements the steps of any one of the above-described user-sensitive data de-identification methods for a local database.
[0108] It should be noted that the order of the embodiments described above is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. Furthermore, specific embodiments of this specification have been described above. Additionally, the processes depicted in the accompanying drawings do not necessarily require a specific or sequential order to achieve the desired results. In some implementations, multitasking and parallel processing are possible or may be advantageous.
[0109] The various embodiments in this specification are described in a progressive manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.
[0110] The above description is only a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the principles of this application should be included within the protection scope of this application.
Claims
1. A method for desensitizing user-sensitive data in a local database, characterized in that, The method includes the following steps: Obtain the structured data to be stored in the local database, divide the structured data into data to be de-identified based on the metadata information of the data table, perform segmentation and word segmentation on the data to be de-identified, and extract keywords and corresponding sub-words and semantic vectors of the sub-words; The semantic feature vectors of keywords are determined based on the semantic vectors of sub-words; the keywords are clustered based on the similarity of the semantic feature vectors between keywords, and a generalized hierarchical tree of each cluster is constructed by combining open-source knowledge graphs. In the generalized hierarchical tree, the path similarity between any two keywords is calculated to determine whether there is a relationship between the two keywords, and representative de-identified keywords are selected based on the relationship. Based on the node depth of the generalization hierarchy tree, the number of keywords with related relationships, the number of keywords representing desensitization, and the uncertainty of keyword distribution, a desensitization index is constructed. Based on the aforementioned desensitization index, all generalized hierarchical trees are divided into high-sensitivity groups and low-sensitivity groups. Initial K values in the K-anonymity algorithm of different magnitudes are dynamically set for the high-sensitivity groups and low-sensitivity groups respectively, so as to perform desensitization processing on the data to be desensitized. The process of constructing the desensitization indicators is as follows: Desensitization index of generalized hierarchical tree The expression is: In the formula, This indicates the node depth of the generalized hierarchical tree; This indicates the number of keywords with related relationships in the generalization hierarchy tree; This indicates the number of desensitized keywords in the generalization hierarchy tree; This represents the quantitative result indicating the uncertainty of keyword distribution in the generalization hierarchy tree; The process of dividing all generalized hierarchical trees into high-sensitivity and low-sensitivity groups based on the desensitization index includes: The desensitization index of all generalized hierarchical trees is used as the input of the threshold segmentation algorithm, and the output is the segmentation threshold. Generalized hierarchical trees with desensitization index greater than the segmentation threshold are classified as high-sensitivity groups, and the remaining generalized hierarchical trees are classified as low-sensitivity groups.
2. The method for desensitizing user-sensitive data in a local database as described in claim 1, characterized in that, The process of determining the semantic feature vector of the keyword is as follows: The frequency of each sub-word corresponding to the statistical keyword in the data to be de-identified is used as the weight of the semantic vector of each sub-word. The result of the weighted average of the semantic vectors of all sub-words is used as the semantic feature vector of the keyword.
3. The method for desensitizing user-sensitive data in a local database as described in claim 1, characterized in that, The distance metric used in the keyword clustering process is the Euclidean distance between the semantic feature vectors of the keywords.
4. The method for desensitizing user-sensitive data in a local database as described in claim 1, characterized in that, The process of constructing the generalization hierarchical tree for each cluster is as follows: For each cluster, the preset top-level root node of the domain is used as the root node of the generalization hierarchy tree, and the keywords in the cluster are used as leaf nodes. Based on the entity hierarchy relationship in the open-source knowledge graph, the higher-level concept paths from each keyword to the root node are merged to obtain a tree structure, which serves as the generalization hierarchy tree.
5. The method for desensitizing user-sensitive data in a local database as described in claim 4, characterized in that, Determining whether any two keywords have a correlation includes: In the generalized hierarchical tree, the mean of the path similarity between all keywords is calculated and denoted as the similarity threshold. If the path similarity between two keywords is greater than the preset similarity threshold, then the nodes on the path from the two keywords to their corresponding common parent node are counted respectively, forming the path node sequence of each keyword. Based on the similarity between the path node sequences of the two keywords and the difference in the length of the path node sequences, a similarity coefficient is constructed between the two keywords. Otherwise, no similarity coefficient is constructed. In the path node sequences of two keywords, starting from the starting element of the longest path node sequence, all elements with the same length as the shortest path node sequence are obtained to form a reference sequence of the shortest path node sequence. If the similarity coefficient between the two keywords is greater than or equal to a preset threshold, and the shortest path node sequence is exactly the same as its reference sequence, then it is determined that there is a correlation between the two keywords; otherwise, there is no correlation.
6. The method for desensitizing user-sensitive data in a local database as described in claim 4, characterized in that, The process of filtering de-identified representative keywords based on the aforementioned association relationships includes: In the generalized hierarchical tree, keywords with related relationships are grouped into related keyword pairs. The common parent node with the largest hierarchical depth among all common parent nodes corresponding to any related keyword pair is used as the representative keyword for desensitization.
7. The method for desensitizing user-sensitive data in a local database as described in claim 1, characterized in that, The quantification process of the uncertainty in the keyword distribution in the generalized hierarchical tree is as follows: Calculate the mean of the semantic feature vectors of all keywords with related relationships in the generalization hierarchy tree, and use it as the baseline vector; The information entropy of the similarity between the semantic feature vectors of all related keywords and the baseline vector is calculated separately, which serves as a quantification of the uncertainty of keyword distribution in the generalization hierarchical tree.
8. A user-sensitive data anonymization system for a local database, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the user-sensitive data desensitization method for a local database as described in any one of claims 1-7.