Knowledge graph-based database metadata sensitive identification method and system

Abstract

The present disclosure provides a knowledge graph-based database metadata sensitive identification method and system, relating to the technical field of data security, obtaining metadata of a database to be identified, and translating the metadata; labeling feature tags for description information and data sample value information in the translated metadata; identifying sensitive information for the feature tags, formulating a sensitive information target, constructing a sensitive information identification rule, extracting and modeling the feature tags using a deep learning model, and extracting sensitive information; establishing multiple-dimensional knowledge graphs in combination with the metadata, feature tags, and sensitive information, and performing reasoning analysis using the knowledge graphs to complete missing content or content with missing tags and improve the knowledge graphs. The present disclosure can accurately identify sensitive information and address the problems of insufficient rules and manual labeling.

Description

Technical Field

[0001] This disclosure relates to the field of data security technology, specifically to a method and system for sensitive identification of database metadata based on knowledge graphs. Background Technology

[0002] The statements in this section are merely background information relating to this disclosure and do not necessarily constitute prior art.

[0003] In the era of big data, sensitive data identification is the foundation of data security governance. Its main function is to accurately identify sensitive data in the system and protect the information security of individuals, enterprises and even the country. This includes personal privacy information, corporate privacy information, human resources privacy information, sales privacy information, distributor privacy information, etc., and at a higher level, it may involve important information such as work secrets and state secrets.

[0004] The premise of protecting data security is to correctly distinguish which data is important and sensitive. Sensitive data identification, as an important process in data security governance, can effectively identify important data, improve data management efficiency, and ensure data security. Then, downstream technical means and protection measures are applied to classify, protect, and manage these data resources.

[0005] Currently, most enterprises and organizations use rule-based matching, such as regular expressions and fuzzy matching, to match sensitive information in data using predefined rules and keywords. Others utilize machine learning algorithms, such as support vector machines and neural networks, to classify and identify sensitive data by training models. Alternatively, a hybrid approach combines rule-based matching and machine learning to improve the accuracy and completeness of sensitive data identification. Rule-based matching requires predefined rules, and while simple to implement, it can only identify known sensitive information and is difficult to extend to newly emerging sensitive data. In contrast, rule-based machine learning methods can adapt to new types of sensitive data, but require a large amount of training data and manual annotation costs.

[0006] However, the inventors discovered that although the above technologies can all identify sensitive data, they cannot predict the sensitivity of missing data or infer implicit sensitive content. This is because important data with incorrect or unknown names is often encountered during data scanning, which greatly increases the difficulty of identifying sensitive data. The advantages and limitations of different technologies will affect their application effect and applicability in specific fields. Summary of the Invention

[0007] To address the aforementioned issues, this disclosure proposes a knowledge graph-based method and system for identifying sensitive database metadata. It employs a complementary approach combining rules and model algorithms, constructing a graph knowledge network using knowledge graph methods. This leverages the functional characteristics of knowledge graphs to enhance data correlation, providing a broader perspective on sensitive data identification. By associating different types of data, it discovers similarities and connections between data, thereby more accurately identifying sensitive data and enabling prediction, reasoning, completion, and tracing of unknown data.

[0008] According to some embodiments, the present disclosure adopts the following technical solutions:

[0009] Knowledge graph-based methods for identifying sensitive database metadata include:

[0010] Obtain the metadata of the database to be identified and translate the metadata;

[0011] Feature labels are added to the descriptive information and data sample value information in the translated metadata;

[0012] Sensitive information is identified from feature labels, sensitive information targets are defined, sensitive information identification rules are constructed, and deep learning models are used to extract and model feature labels to extract sensitive information.

[0013] By combining metadata, feature tags, and sensitive information, a multi-dimensional knowledge graph is established. The knowledge graph is then used for reasoning and analysis, and missing or missing content or tags are filled in to improve the knowledge graph.

[0014] According to some embodiments, the present disclosure adopts the following technical solutions:

[0015] A knowledge graph-based database metadata sensitivity identification system includes:

[0016] The data acquisition module is used to acquire metadata from the database to be identified and to translate the metadata.

[0017] The tagging module is used to annotate the descriptive information and data sample value information in the translated metadata with feature tags;

[0018] The identification module is used to identify sensitive information from feature labels, define sensitive information targets, construct sensitive information identification rules, and use deep learning models to extract and model feature labels to extract sensitive information.

[0019] The knowledge reasoning module is used to build a multi-dimensional knowledge graph by combining metadata, feature tags, and sensitive information. It uses the knowledge graph for reasoning and analysis, fills in missing content or tags, and improves the knowledge graph.

[0020] According to some embodiments, the present disclosure adopts the following technical solutions:

[0021] A non-transitory computer-readable storage medium is provided for storing computer instructions, which, when executed by a processor, implement the knowledge graph-based database metadata sensitive identification method.

[0022] According to some embodiments, the present disclosure adopts the following technical solutions:

[0023] An electronic device includes a processor, a memory, and a computer program; wherein the processor is connected to the memory, the computer program is stored in the memory, and when the electronic device is running, the processor executes the computer program stored in the memory to enable the electronic device to perform the aforementioned knowledge graph-based database metadata sensitive identification method.

[0024] Compared with the prior art, the beneficial effects of this disclosure are as follows:

[0025] This disclosed knowledge graph-based method for sensitive database metadata identification combines rule-based and model-based algorithms with knowledge graph methods to construct a graph knowledge network. Leveraging the functional characteristics of knowledge graphs enhances data correlation, providing a broader perspective on sensitive data identification. It can associate different types of data, discover similarities and connections between them, and thus more accurately identify sensitive data. It also predicts, infers, completes, and traces the source of unknown data. Furthermore, it supports real-time updates, enabling rapid adaptation and application of newly emerging sensitive data types, ensuring the identification and protection of sensitive data and maintaining its up-to-date state. This fundamentally overcomes the challenges of insufficient rules and manual annotation, thereby providing better data value identification and business competitiveness.

[0026] This disclosure effectively addresses the problem of sensitive data identification in databases, from data acquisition, cleaning, matching, and understanding to recognition, and can also identify and reason about missing data. It boasts high recognition efficiency; verified, it can complete the identification of 1000 data entries within 1000 milliseconds, maintaining an accuracy rate above 80% and a coverage rate exceeding 90%. Furthermore, this disclosure supports manual review and iterative correction, improving the system's accuracy, flexibility, and efficiency. It is well-suited to the future needs of data security governance and can be continuously expanded and rapidly adapted to new industries. Attached Figure Description

[0027] The accompanying drawings, which form part of this disclosure, are used to provide a further understanding of this disclosure. The illustrative embodiments of this disclosure and their descriptions are used to explain this disclosure and do not constitute an undue limitation of this disclosure.

[0028] Figure 1This is a flowchart illustrating the sensitive data identification process according to an embodiment of the present disclosure.

[0029] Figure 2 This is a schematic diagram illustrating the system operation of an embodiment of this disclosure. Detailed Implementation

[0030] The present disclosure will be further described below with reference to the accompanying drawings and embodiments.

[0031] It should be noted that the following detailed descriptions are illustrative and intended to provide further explanation of this disclosure. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure pertains.

[0032] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the exemplary embodiments according to this disclosure. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms “comprising” and / or “including” are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0033] Example 1

[0034] One embodiment of this disclosure provides a database metadata sensitivity identification method based on knowledge graphs, including:

[0035] Step 1: Obtain the metadata of the database to be identified and translate the metadata;

[0036] Step 2: Label the descriptive information and data sample value information in the translated metadata with feature tags;

[0037] Step 3: Identify sensitive information in feature labels, define sensitive information targets, construct sensitive information identification rules, and use deep learning models to extract and model feature labels to extract sensitive information;

[0038] Step 4: Combine metadata, feature tags, and sensitive information to build a multi-dimensional knowledge graph. Use the knowledge graph for reasoning and analysis, fill in missing content or tags, and improve the knowledge graph.

[0039] As one embodiment, the specific implementation of the knowledge graph-based database metadata sensitivity identification method disclosed herein is as follows:

[0040] Step 1: Scan the enterprise database to collect its metadata, including: database name, table name, data field name, and stored data content, and translate it (mostly in English, such as: data_base, table_name, TableNname, ZRZTYSJ, etc.). If the translation fails and results in missing information (such as misspelled words, unknown spellings, unknown abbreviations, initials of pinyin, etc.), proceed to step 7.

[0041] Step 2: If official metadata description information is provided (official dictionary, official Chinese description of this database and data table information), proceed to step 4 and use the tag annotation module to directly add feature tags to the official metadata description information.

[0042] Step 3: If you provide data sample value information (such as sample values ​​stored in the database, such as 1390990****, Technical Director, Male, Bachelor's Degree, etc.), go to Step 3 and use the labeling module to label specific sample values ​​such as 1390990****, Technical Director, Male, Bachelor's Degree with tags such as phone number, job title, gender, and education level.

[0043] Step 4: Label the descriptive information and data sample value information in the translated metadata with feature tags;

[0044] Step 5: Identify sensitive information in the feature labels, define sensitive information targets, construct sensitive information identification rules, and use a deep learning model to extract and model the feature labels to extract sensitive information;

[0045] Step 6: Construct a knowledge graph by combining multiple dimensions such as database name, table name, field name, stored sample values, translation results, feature tags, and sensitive information tags.

[0046] Step 7: Construct knowledge graph reasoning rules and models to perform reasoning and prediction on data information that has no translation results, no field descriptions, and meets sensitive types.

[0047] Step 8: Use knowledge graphs to infer and complete missing content or tags.

[0048] Step 9: The iterative knowledge graph can be manually reviewed, and the results of steps 4, 5, 6, and 7 can be corrected.

[0049] Step 10: End.

[0050] Specifically, in step 4, feature labels are added to the descriptive information and data sample value information in the translated metadata. This includes:

[0051] Feature labels are generated using certain rules, mainly through regular expressions and keyword matching, to identify data types such as strings, Chinese characters, English characters, mixed Chinese and English characters, integers, and floating-point numbers.

[0052] The deep learning model primarily utilizes NLP technology, employing a Tinybert pre-trained model to train an event state classification model. The trained model then undergoes model distillation, retaining the essence learned by the original model while reducing its size from 50MB to 20MB – a reduction of more than half. Distillation also improves the model's recognition speed by 30%. This model can label table-level dimensions of event description information (contract events, sales events, procurement events, meeting events, R&D events, HR events, etc., more than 10 types), and can be supplemented and adjusted according to business needs.

[0053] By employing NLP technology and a NER named entity recognition model or multi-classification model pre-trained with Tingbert, data such as 'person's name', 'occupation', 'address', 'company name', 'organization name', 'organization', and 'university' can be identified and categorized as sensitive data. Due to the scalability of the AI ​​model, it can be expanded later based on training data as needed, such as adding the recognition of proprietary entities like 'industry', 'disease', and 'drug name', demonstrating strong scalability.

[0054] Official mathematical algorithms can identify sensitive data such as social credit codes, business license numbers, organization codes, ID cards, and bank cards. The algorithm strictly adheres to official rules and is highly accurate.

[0055] Some data can be matched using regular expressions (emphasizing industry-standard regular expression rules), such as phone numbers, IP addresses, and URLs. Due to the algorithmic nature of regular expressions, the matching speed is relatively high.

[0056] It can also tag keywords that businesses are interested in, such as 'distributor', 'supplier', 'order', and 'contract'.

[0057] In step 5, sensitive information is identified from the feature labels, sensitive information targets are defined, sensitive information identification rules are constructed, and a deep learning model is used to extract and model the feature labels to extract sensitive information.

[0058] Sensitive data targets can be customized according to business needs. After data analysis, identification rules are constructed, and sensitive data can be extracted and modeled using rules or deep learning models (which may include database names, data table names, field names, sample values, official description information, etc.).

[0059] In step 6, a knowledge graph is constructed by combining multiple dimensions such as database name, table name, field name, stored sample values, translation results, feature tags, and sensitive information tags.

[0060] Using Neo4j batch import technology, a knowledge graph is constructed from tens of millions of data points. Nodes are defined by database names, table names, field names and translation results, data dictionaries, event tags, and sensitive data tags, with hierarchical or inclusion relationships forming the knowledge graph. This achieves a mapping relationship of 'entity-relationship-entity' (e.g., Database A - Storage - Table Aa, Database A - Storage - Table Ab, Table Ab - Description - Event, Table Ab - Storage - Data Type, Field Name - Storage - Sample Value, Sample Value - Regular Expression - Sensitive Tag, etc.), which is then used by the knowledge graph reasoning module.

[0061] In step 7, knowledge graph reasoning rules and models are constructed to perform reasoning and prediction on data information that has no translation results, no field descriptions, and conforms to sensitive types.

[0062] On the completed knowledge graph, models are deployed to perform functions such as knowledge reasoning, implicit sensitivity mining, and node relationship completion, in order to complete the lineage analysis, contextual reasoning, and unknown information completion of unknown data, and to achieve key data label completion and label result optimization.

[0063] This invention discloses a data completion method based on knowledge graph relation inference and a method for mining 'cluster data':

[0064] 1. Data completion for relation inference:

[0065] 1) Add tags for correlation analysis: If data table A and data table B have more than 50% (customizable) common correlation tags, and data table A has an official data dictionary, then data table B can also use the official data dictionary information of data table A.

[0066] 2) Event completion for correlation analysis: If data tables A and B have more than 50% (customizable) common correlation tags, and data table A has sensitive event tags, then data table B can use the sensitive event tags from data table A; similarly, if data table A does not have event tags but data table B does, then data table A can use the event tags from data table B.

[0067] 3) Translation completion: If the sample values ​​of data field A and data field B use the same sensitive data label (such as bank card number), and data field A has a translation result (such as bank account information), while data field B does not have a translation result (translation failed), the translation of data field B can be completed using the translation result of data field A, and vice versa.

[0068] 4) Translation completion: For example, in the same data table, if data field A is 'GWY_info' and data field A has the official data dictionary translation result 'civil servant information', while data field B 'GWY' does not have a translation result, the translation of data field B can be completed using the translation result of data field A.

[0069] 5) Sensitive Tag Completion: If 90% (customizable) of the information in tables A and B are associated with common sensitive tags and event tags, and the content stored in field A1 of table A and field B2 of table B is of the same string type and highly similar, then if field A1 of table A has a sensitive tag, then field B2 of table B can also use this sensitive tag.

[0070] 2. Data mining using 'Group Data':

[0071] 1) If a certain 'data group A' has no correlation or low correlation with other 'data groups', the reasons can be analyzed separately to determine if it is a new data category and to construct a new label to represent it.

[0072] 2) Multiple 'data clusters' form a 'cluster', indicating that these 'data clusters' must have a certain correlation. After analysis, they can be labeled with a situation tag to describe the situation of the entire 'data cluster'.

[0073] Once the knowledge graph is constructed, mature graph model algorithms from the industry can be used for practice and analysis. Here are a few simple examples:

[0074] TransE: A distance-based model for computing relationships between nodes in a knowledge graph. This model represents each relationship as a vector from one node to another and attempts to minimize their distances within the knowledge graph to infer the underlying relationships between nodes.

[0075] ConvE: A model employing an image convolutional neural network (CNN) to infer relationships between nodes in a knowledge graph. This model applies image convolution operations to the matrix representations of nodes and relationships, resulting in a vector representation that allows it to distinguish between different relationship types.

[0076] ComplEx: A tensor decomposition-based model used to compute relationships between nodes in a knowledge graph. Unlike TransE, ComplEx represents nodes and relationships as complex vectors to better handle multiple relationships between nodes and relationships.

[0077] KGAT: A model based on Graph Attention Network (GAT) that can perform reasoning at both the node and relation levels simultaneously. This model uses an attention mechanism to capture the associations between nodes and relations, making it more adaptable to the complex relationships in knowledge graphs when predicting potential relationships between nodes and relations.

[0078] In step 8, inaccurate translation results, labeling system annotations, and sensitivity identification results are manually updated and corrected. The Snowball concept from Bootstrap is used to improve knowledge graph reasoning. Snowball adopts the idea of ​​'snowballing', collecting translation, annotation, and sensitivity identification results (including knowledge graph completion results). For the translation and annotation results of each method, a confidence score is given. Methods with high confidence scores are retained, and methods with low confidence scores are corrected. This process is iterated multiple times until it stabilizes, and corrections and completions are performed to update and optimize the reasoning effect, generating a snowball effect.

[0079] Example 2

[0080] One embodiment of this disclosure provides a database metadata sensitivity identification system based on knowledge graphs, including:

[0081] The data acquisition module is used to acquire metadata from the database to be identified and to translate the metadata.

[0082] The tagging module is used to annotate the descriptive information and data sample value information in the translated metadata with feature tags;

[0083] The identification module is used to identify sensitive information from feature labels, define sensitive information targets, construct sensitive information identification rules, and use deep learning models to extract and model feature labels to extract sensitive information.

[0084] The knowledge reasoning module is used to build a multi-dimensional knowledge graph by combining metadata, feature tags, and sensitive information. It uses the knowledge graph for reasoning and analysis, fills in missing content or tags, and improves the knowledge graph.

[0085] The data acquisition module includes a data collection module, which mainly contains three functions:

[0086] 1) The text translation module mainly translates and stores database names, table names, field names, and data content (e.g., data_base: database, table_name: table name, CompanyName: company name, ZRZTYSJ: data from the Department of Natural Resources, etc.).

[0087] 2) Sample acquisition module, which mainly stores the data sample values ​​(such as sample values ​​stored in the database, such as 1390990****, technical director, male, undergraduate, etc.).

[0088] The field description acquisition module primarily collects and stores official descriptions of the stored information (also known as the official data dictionary, which mainly describes the specific content of the stored data). Throughout the acquisition and storage process, the relational structure must be maintained, preserving relationships and their sources to facilitate the storage of inclusion relationships when constructing the knowledge graph.

[0089] Example 3

[0090] One embodiment of this disclosure provides a non-transitory computer-readable storage medium for storing computer instructions, which, when executed by a processor, implement the knowledge graph-based database metadata sensitive identification method.

[0091] Example 4

[0092] One embodiment of this disclosure provides an electronic device, including a processor, a memory, and a computer program; wherein the processor is connected to the memory, the computer program is stored in the memory, and when the electronic device is running, the processor executes the computer program stored in the memory to enable the electronic device to perform the aforementioned knowledge graph-based database metadata sensitive identification method.

[0093] This disclosure is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create a machine for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0094] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0095] While the specific embodiments of this disclosure have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of this disclosure. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of this disclosure are still within the scope of protection of this disclosure.

Claims

1. A knowledge graph-based database metadata sensitive identification method, characterized in that, The method comprises the following steps: Obtain the metadata of the database to be identified, and translate the metadata; Label feature tags in the description information and data sample value information in the translated metadata; Perform sensitive information identification on the feature tags, formulate sensitive information targets, construct sensitive information identification rules, and extract sensitive information using the rules or use a deep learning model to extract and model the information to extract sensitive information; Establish multiple-dimensional knowledge graphs by combining the metadata, feature tags, and sensitive information, use the knowledge graphs for reasoning analysis, complete the content that is missing or has missing labels, and perfect the knowledge graphs; The method comprises the following steps: Label feature tags in the description information and data sample value information in the translated metadata, which comprises the following steps: use regular expressions and keyword matching methods to label data types for string types, Chinese, English, mixed English and Chinese, integer types, and floating-point types; use a tinybert pre-training model to train an event state classification model, and perform event state classification model distillation; the event state classification model labels table event description information and table-level dimensions, and adjusts and supplements the labels according to business requirements; The method comprises the following steps: Relationship inference data completion: if A data table and B data table have common association labels above a custom threshold, and the A data table has an official data dictionary, the B data table also uses the official data dictionary information of the A data table; 2.The knowledge graph based database metadata sensitive identification method of claim 1, wherein, Event completion: if A data table and B data table have common association labels above a custom threshold, and the A data table has a sensitive event label, the B data table can use the sensitive event label of the A data table; similarly, if the A data table does not have an event label and the B data table does, the A table can use the event label of the B table. 3.The knowledge graph based database metadata sensitive identification method of claim 1, wherein, The metadata comprises data_base, table_name, TableNname, and ZRZTYSJ, and the metadata is translated into a database name, a data table name, a data field name, and stored data content. 4.The knowledge graph based database metadata sensitive identification method of claim 1, wherein, If the translation fails and results in a lack, build a knowledge graph reasoning rule and a model, and use the knowledge graph to infer and complete the missing content or labels for data without translation results, field descriptions, and sensitive types. The method comprises the following steps: 5.The knowledge graph based database metadata sensitive identification method of claim 1, wherein, Translation completion: if A data field sample value and B data field sample value use a common sensitive data label, and the A data field has a translation result, the B data field does not have a translation result, and the B data field translation can be completed using the translation result of the A data field. The method comprises the following steps:

6. The knowledge graph based database metadata sensitive identification system, realizing the knowledge graph based database metadata sensitive identification method according to any one of claims 1-5, characterized in that, Sensitive label completion: if A and B tables have common sensitive labels and event labels associated with a custom threshold, and the contents stored in A1 field of the A table and B2 field of the B table are of the same string type and are highly similar, the A1 field of the A table has a sensitive label, and the B2 field of the B table also uses the sensitive label. The method comprises the following steps: A data acquisition module is configured to obtain the metadata of the database to be identified, and translate the metadata; A label module is configured to label feature tags in the description information and data sample value information in the translated metadata; The identification module is configured to perform sensitive information identification on the feature label, formulate a sensitive information target, construct a sensitive information identification rule, extract and model the feature label by using a deep learning model, and extract sensitive information. The knowledge reasoning module is configured to combine metadata, feature labels, and sensitive information to establish a plurality of dimensional knowledge graphs, perform reasoning analysis by using the knowledge graphs, complete content with missing content or labels, and perfect the knowledge graphs.

7. A non-transitory computer-readable storage medium, comprising: The non-transitory computer readable storage medium is configured to store computer instructions, and the computer instructions are executed by a processor to implement the database metadata sensitive identification method based on the knowledge graph according to any one of claims 1-5.

8. An electronic device, comprising: The electronic device comprises a processor, a memory, and a computer program; the processor is connected with the memory, and the computer program is stored in the memory; when the electronic device is running, the processor executes the computer program stored in the memory, so that the electronic device performs the database metadata sensitive identification method based on the knowledge graph according to any one of claims 1-5. ​

Images