Sensitive data automatic identification method and system, medium and electronic equipment

Abstract

The invention relates to the technical field of data security, in particular to an automatic sensitive data identification method and system, a medium and electronic equipment, and the automatic sensitive data identification method comprises the following steps: obtaining data of a related data table, and carrying out preliminary screening and entity identification on the data table; for different entity types, corresponding features are extracted, a basic bottom-taking model is constructed, and an initial sensitivity level is calculated by adopting a rule method; training samples are optimized in combination with manual annotation, an improved model is constructed by using a decision tree algorithm, and the final sensitivity level of the data table is determined through the improved model. The problems of efficiency and accuracy of identifying sensitive information in large-scale data in a traditional method are effectively solved, the risk of sensitive information leakage is reduced, and intelligent and automatic identification of the sensitive data is achieved.

Description

Technical Field

[0001] The present invention relates to the field of data security technology, and in particular to a method, system, medium and electronic equipment for automatically identifying sensitive data. Background Art

[0002] In the digital age, the storage and management of government data has become a crucial component of information security. As data volumes continue to expand and data types become increasingly complex, the need to protect and manage sensitive data is growing. Sensitive data is often stored in databases, and its value and importance necessitate effective protection measures. However, identifying and managing sensitive data presents numerous challenges.

[0003] Existing data protection technologies typically include access control, audit logging, and encrypted storage. While these technologies can prevent the illegal acquisition and exploitation of sensitive data, they have limitations. For example, rule-based automatic identification methods struggle to fully cover all potentially sensitive data. In particular, information in hidden data tables (such as temporary tables, intermediate tables, and backup tables) is often overlooked, resulting in ineffective identification and protection of sensitive data.

[0004] Furthermore, the increasing complexity and dynamic nature of data systems further increases the difficulty of identifying sensitive data. Traditional methods, which rely on manual operations or simple system rules, struggle to cope with the large and constantly changing number of data tables. For example, in some systems, the number of data tables may grow from a small number of formal tables to tens of thousands, containing a large amount of sensitive information. Existing technologies are inefficient in handling such complex situations and are unable to promptly and accurately identify and protect all sensitive data. This limitation not only increases the risk of data leakage but also poses a potential threat to the continuity and credibility of government services.

[0005] Therefore, how to accurately identify and protect sensitive data while ensuring data openness and sharing has become an important research topic in the current data security field. Summary of the Invention

[0006] In order to solve the problem of the above-mentioned prior art that sensitive data is difficult to effectively identify, the embodiments of the present invention provide a method, system, medium and electronic device for automatic identification of sensitive data to effectively realize automatic identification of sensitive data.

[0007] In a first aspect, an embodiment of the present invention provides a method for automatically identifying sensitive data, comprising the following steps: Data acquisition step: acquiring a data table, wherein the data table includes at least one of metadata, data content, and data operation log; Entity identification step: performing entity identification on the data table to identify natural person entities and organizational entities based on key fields; determining the natural person entity identifier and organizational entity identifier of each data table based on the ratio between the number of records in the data table containing natural person entities and organizational entities and the number of key field types; A feature engineering construction step: for a data table containing the natural person entity or organizational entity, extracting feature information related to the natural person entity or organizational entity, performing feature processing to obtain feature values of each feature data, and then constructing a natural person entity feature engineering and an organizational entity feature engineering in combination with the natural person entity identifier and the organizational entity identifier; A preliminary sensitivity level determination step: constructing a natural person basic underlay model and an organizational basic underlay model based on the natural person entity feature engineering and organizational entity feature engineering based on preset sensitivity level evaluation rules; determining the preliminary sensitivity level of the data table based on the natural person basic underlay model and organizational basic underlay model; Early warning step: triggering an early warning for a data table of a determined sensitivity level according to the early warning rules, manually processing the early warning and simultaneously correcting the sensitivity level of the data table; Optimize the model step: combine the data generated by the natural person basic backup model and the organizational basic backup model with the data of the manually corrected data table's sensitivity level as training samples, and train the natural person optimization model and organizational optimization model based on the decision tree algorithm; Final sensitivity level assessment steps: Use the natural person optimization model and organizational structure optimization model to perform the final sensitivity level assessment of the data table.

[0008] In a second aspect, the present invention further provides a system for automatically identifying sensitive data, comprising: A data acquisition module; used to acquire a data table, wherein the data table includes at least one of metadata, data content, and data operation log; An entity recognition module is configured to perform entity recognition on the data table to identify natural person entities and organizational entities based on key fields; and then determine the natural person entity identifier and organizational entity identifier of each data table based on the ratio between the number of records in the data table containing natural person entities and organizational entities and the number of key field types; A feature engineering construction module is used to extract feature information related to the natural person entity or organizational entity from a data table containing the natural person entity or organizational entity, perform feature processing to obtain feature values of each feature data, and then construct a natural person entity feature engineering and an organizational entity feature engineering based on the natural person entity identifier and the organizational entity identifier; A preliminary sensitivity level determination module is configured to construct a natural person basic backup model and an organizational basic backup model based on the natural person entity feature engineering and organizational entity feature engineering, based on preset sensitivity level evaluation rules; and to determine the preliminary sensitivity level of the data table based on the natural person basic backup model and the organizational basic backup model. An early warning module; used to trigger an early warning for a data table of a determined sensitivity level according to an early warning rule, manually process the early warning, and simultaneously correct the sensitivity level of the data table; Optimization model module: used to combine the data generated by the natural person basic backup model and the organizational basic backup model with the data of the manually corrected sensitivity level of the data table as training samples, and train the natural person optimization model and organizational optimization model based on the decision tree algorithm; Final sensitivity level assessment module; used to assess the final sensitivity level of data tables using the natural person optimization model and organizational optimization model.

[0009] In a third aspect, the present invention further provides a computer-readable storage medium, which stores computer instructions and, when executed by a computer processor, implements the method for automatically identifying sensitive data as described in the above embodiment.

[0010] In a fourth aspect, the present invention also provides an electronic device comprising at least one processor and a memory communicatively connected to the processor, wherein the memory stores instructions executable by at least one processor, and the instructions are executed by at least one processor to enable the processor to execute the method for automatic identification of sensitive data as described in the above embodiment.

[0011] Compared with the existing technology, the automatic sensitive data identification method provided by the present invention obtains data from relevant data tables, performs preliminary screening and entity identification on the data tables; extracts corresponding features and constructs a basic backup model for different entity types, and uses a rule-based method to calculate the initial sensitivity level; combines manual annotation to optimize training samples, uses a decision tree algorithm to build an improved model, and determines the final sensitivity level of the data table through the improved model. It effectively solves the efficiency and accuracy problems of traditional methods in identifying sensitive information in large-scale data, and realizes intelligent and automated identification of sensitive data. This method can effectively improve the level of data security management, reduce the risk of sensitive information leakage, and provide strong support for the data protection work of enterprises and organizations. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following is a brief introduction to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0013] Figure 1 A flowchart of a method for automatically identifying sensitive data provided in Example 1 of the present invention; Figure 2 A flowchart of the multi-group model optimization process provided in the first embodiment of the present invention; Figure 3 A flowchart of the sensitivity level assessment process provided in the first embodiment of the present invention; Figure 4 This is a structural block diagram of the sensitive data automatic identification system provided in Example 3 of the present invention; Figure 5 This is a structural diagram of an electronic device provided in Example 5 of the present invention. DETAILED DESCRIPTION

[0014] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0015] In the description of the present invention, it should be noted that the terms "center," "longitudinal," "lateral," "upper," "lower," "front," "back," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," "outer," and the like, indicating orientations or positional relationships, are based on the orientations or positional relationships shown in the accompanying drawings and are intended only to facilitate the description of the present invention and simplify the description. They do not indicate or imply that the devices or components referred to must have a specific orientation, be constructed, or operate in a specific orientation. Therefore, they should not be construed as limiting the present invention. Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0016] Example 1 See also Figure 1 , an embodiment of the present invention provides a method for automatically identifying sensitive data, comprising the following steps: Step S10, data acquisition step: acquiring a data table, wherein the data table includes at least one of metadata, data content, and data operation log.

[0017] Among them, the metadata includes information such as the data table name, data table comments, field name, field type, field comments, the code set corresponding to the field, the correspondence between the code value and its code name, the number of records, and the amount of data in the data table; the data content is composed of field values, and the data in the data table is collected and obtained on demand; the operation log includes event time, client IP, client tool, running time, data tables involved, SQL query statements, number of returned records, etc.

[0018] Preferably, after performing the data acquisition step, this embodiment further includes a rating condition determination step. If the number of records in the data table exceeds a preset number or the amount of data in the data table exceeds a preset storage value, an entity recognition step is performed on the data table; otherwise, the data table is discarded and the entity recognition step is not performed, or the sensitivity level of the data table is set to an insensitive level.

[0019] This is because a large number of data tables in the database are empty or contain very little information, which essentially poses no sensitivity risk and is generally classified as non-sensitive data. Therefore, evaluating these tables is meaningless. Therefore, the above steps set rating criteria. By excluding data tables that do not meet the rating criteria, computing power can be effectively saved for subsequent identification and judgment, while also improving the reliability of the model algorithm. This prevents the impact of irrelevant data tables with small amounts of information on model training. The preset number of entries and the preset storage value can be reasonably limited based on actual needs. For example, this embodiment sets the preset number of entries to 100 and the preset storage value to 1MB. When the number of records in a data table exceeds 100 and the data volume exceeds 1MB, the rating process begins. Furthermore, since the number of records and data volume in a data table may fluctuate over time, once these changes meet the aforementioned rating criteria, the data table can be included in subsequent sensitivity ratings without worrying about omissions.

[0020] Step S20, entity identification step: performing entity identification on the data table to identify natural person entities and organizational entities based on key fields; then determining the natural person entity identifier and organizational entity identifier of each data table based on the ratio between the number of records in the data table containing natural person entities and organizational entities and the number of key field types; Specifically, sensitive data shared in daily government affairs generally only involves natural persons and organizations, as well as other private information directly related to them. Therefore, the purpose of performing entity identification for natural persons and organizations in this embodiment is to eliminate unnecessary data identification, thereby effectively saving computing power and improving identification efficiency.

[0021] In step S20, entity recognition of the data table includes: extracting records from each data table and inputting them into the recognition model to determine whether there are key fields of natural persons or organizations in the records; if there are key fields, judging whether the data table belongs to a natural person entity, an organizational entity, or a non-unique entity based on the key fields; if there are no key fields, judging whether the data table belongs to an entity to be determined.

[0022] Among them, the keywords for identifying natural person entities can be document numbers (such as ID cards, personal work numbers, passport numbers, etc.), names, codes, etc., and the keywords for identifying organizational entities can be organizational names, organizational codes, etc.

[0023] Specifically, a machine deep learning algorithm can be used to train a name recognition model, and then the recognition model can be used to perform keyword recognition on the field value to determine whether the data table contains the name of a natural person or the name of an organization. The following takes the "Organization Name Recognition Model" as an example to introduce in detail the steps of keyword recognition: (1) Prepare a training set containing 200,000 samples of real organization names and non-organization names; annotate the fields containing organization names in each text; (2) Segment and tag the text in the original data set; use BERT's tokenizer to convert the text into the corresponding token sequence for subsequent model processing; (3) Load the pre-trained "recognition model": choose to use the pre-trained bert-base-chinese model and load the corresponding tokenizer; (4) Use the training set to load the pre-trained "recognition model" (5) Prepare a test set containing about 10,000 data items to evaluate the performance of the "recognition model"; and use the trained model to predict the test set; (6) Calculate the accuracy, recall and F1 score of the test set based on the prediction results; If all of these indicators reach more than 90%, the model is considered to have good performance; and use the model as an effective organization name recognition model to determine whether the text contains an organization name; (7) In the process of using the organization name recognition model to recognize data text, if more than 90% of the prediction results of the data content of a field are organization names, then the field is determined to be an organization name field.

[0024] Furthermore, the natural composition rules of ID numbers or organization codes are used to identify the presence of natural person ID numbers or organization codes. Specifically, multiple ID number composition rules can be used for verification. If a string of numbers meets one of the composition rules, the number is considered an ID number. During verification using the composition rules, if more than 90% of the verification results for a field are ID numbers, the field is considered an ID number field. For example, the regular expression for an ID number can be expressed as: ^[1-9]\ d{5}(19|20) \ d{2} (0[1-9]|1[0-2]) (0[1-9]|[1-2] \ d|3[0-1]) \ d{3} (\d|X|x)$. This expression describes the format of an ID card number: it begins with a digit from 1 to 9, followed by five digits, then a four-digit year (19 or 20 followed by two digits), followed by a two-digit month (01-12), a two-digit day (01-31), and finally a three-digit sequence number and a check digit (a number or X or x). For example, electronic passport numbers also have specific rules. The number composition can be pure eight-digit Arabic numerals starting with the letter E, or E followed by an English letter other than I or O, followed by seven Arabic numerals. Similarly, other ID card codes can be designed based on actual encoding rules to effectively select the required ID numbers.

[0025] Next, the natural person entity identifier and organizational entity identifier of each data table are determined based on the ratio between the number of records and the number of key field types in the data table having the natural person entity and the organizational entity, including the following steps: When the data table has key fields for a natural person entity, if the ratio of the number of records to the number of key field types is less than a first preset ratio, the data table is determined to belong to natural person entity identification 1, and its natural person entity identification is recorded as 1; if the ratio of the number of records to the number of key field types is greater than or equal to the first preset ratio and less than or equal to the second preset ratio, the data table is determined to belong to natural person entity identification 2, and its natural person entity identification is recorded as 2; if the ratio of the number of records to the number of key field types is greater than or equal to a third preset ratio or the ratio between the number of different key field types for the same natural person is less than a fourth preset ratio, the data table is determined to belong to an entity to be determined.

[0026] It should be noted that the ratio of different key fields for the same natural person can be, for example, the ratio of name type to ID number type, or the ratio of name type to work number type, and can be appropriately set based on actual needs. Furthermore, by comparing the ratio of different key fields for the same natural person with a fourth preset ratio, the purpose is to exclude natural persons unrelated to the natural person or organization corresponding to the data table content. For example, some data may record the name and ID number of the person handling the data in a data table, but the person handling the data is actually unrelated to the entity in this data table. Therefore, this person's information needs to be excluded to prevent the person handling the data from misjudging the sensitive data level. More specifically, when handling household registration transfers, a data table stores change data, including the outgoing household registration address, the incoming household registration address, and the ID number and name of the household registration person handling the data, but not the ID number and name of the household registration person. In this case, since the person handling the data has no direct relationship to the data in the data table, it is not considered sensitive data and should be classified as a table where the entity identity cannot be determined. Therefore, here, by comparing the fourth preset ratio, it is extracted and classified into the entity to be determined category.

[0027] Among them, the first preset ratio is greater than 1 and less than 1.5, the second preset ratio is greater than 1.5 and less than 100, the third preset ratio is greater than 100, and the fourth preset ratio is 0.4. Among them, the specific data of the first preset ratio should be adjusted according to whether the same natural person in the data table contains historical records, the specific data of the second preset ratio should be set according to the proportion of unrelated persons in the data table, and the specific data of the third preset ratio and the fourth preset ratio should be set according to the matching requirements of the key field information of the same natural person in the data table. That is, when the ratio of the number of records in the data table to the number of key field types is greater than 1 and less than 1.5, it means that the data information corresponding to the natural person in the data table does not contain historical records, and the natural person entity identifier corresponding to the data table is recorded as 1; when the ratio of the number of records in the data table to the number of key field types is greater than or equal to 1.5 and less than 100, it means that the data information corresponding to the natural person in the data table contains historical records, and the natural person entity identifier corresponding to the data table is recorded as 2. When the ratio of the number of records to the number of key field types in a data table is greater than 100 or the ratio between the number of different key field types for the same natural person is less than 0.4, it means that the entity identifier of the data table cannot be determined and the data table belongs to the entity pending.

[0028] When there are multiple ID numbers or names, the above logic is looped until the entity identifier is clear or there is no entity identifier at the end of the loop, then the data table is determined to be entity pending.

[0029] As an example, using the basic information of registered residents as an example, we randomly sample 10,000 records from a data table for keyword recognition. We then eliminate duplicates based on the correspondence between ID numbers and names. This yields data where each name and ID number corresponds to only one natural person. We then select no more than 1,000 unique ID numbers from these 10,000 records (i.e., 1,000 key field types represent 1,000 natural persons) and extract the corresponding data. If these 1,000 natural persons have only 1,100 records in the data table, then the number of records is 1,100 and the number of key field types is 1,000. The ratio of these two is 1,100 / 1,000, which is less than 150%, indicating that on average, each natural person has fewer than 1.5 records. This entity does not contain historical data, and the natural person entity identifier is recorded as 1. For example, taking the train ticket purchase information of the past three years as an example, although there is a person who only took the train once in three years, more people will take the train many times in the past three years. Therefore, among the 1,000 natural persons selected, there may be 3,100 records of their data in the data table. The ratio of the number of records to the number of key field types is 3,100 / 1,000 ≥ 150%, indicating that the data table contains historical data, and the natural person entity identifier is recorded as 2.

[0030] By distinguishing the natural person entity identifiers described above, it is possible to determine whether the data table contains multiple different records for the same natural person. Different records represent the historical trajectory of that natural person. Identifying a natural person with a historical trajectory indicates a higher level of sensitivity, and if this information is leaked, the impact on that natural person will be greater. Therefore, distinguishing the natural person entity identifiers described above facilitates subsequent sensitivity level determinations.

[0031] When the data table has a key field of an organizational entity, if the key field in the data table is unique, the data table is determined to belong to organizational entity identification 1, and its organizational entity identification is recorded as 1; if the data table contains multiple organizational entities and the ratio of the number of key field types in the data table to the number of records is judged according to the record quantity classification and meets the specified ratio, the data table is determined to belong to organizational entity identification 2, and its organizational entity identification is recorded as 2; if the data table contains multiple organizational entities and does not meet the specified ratio, the data table is determined to belong to entity to be determined; the specified ratio is greater than 0 and less than 2%.

[0032] In this case, if the ratio of the number of key field types to the number of records in a data table meets a specified ratio, the conditions for determining that the data table belongs to organizational entity identification 2 may be, for example: when the number of records is less than 500, the ratio of the number of key field types (i.e., the number of organizational types) to the number of records is greater than 2%; when the number of records is between 500 and 5,000, the ratio is greater than 1%; and when the number of records is greater than 5,000, the ratio is greater than 0.1%. If the data table meets these conditions, the organizational entity identification of the data table is determined to be 2. If the data table does not meet these conditions, those fields are determined to belong to government affairs processing units, and the entity status of the data table is pending. The purpose of determining whether the ratio of the number of key field types to the number of records in a data table meets a specified ratio based on the number of records is to prevent irrelevant information, such as government affairs processing units, from being identified as organizational entities. The reason is that government affairs business processing units are only responsible for handling the specific information business of certain organizations. For example, there is only one government affairs business processing unit for 200 enterprises, but this government affairs business processing unit has nothing to do with the specific information business content handled in the data itself. Therefore, it is necessary to distinguish them through the above-mentioned grading evaluation method.

[0033] It should be noted that how to classify by the number of records and how to set the limit ratio can be reasonably set according to historical information in actual applications, and this embodiment does not limit them here.

[0034] In step S20, since some data tables do not contain natural person or organizational entities, they are not inherently sensitive. However, during data manipulation and access, if there are mutual access links between data tables containing natural person or organizational entities, these data tables may become sensitive data. These data tables require real-time monitoring and, when necessary, early warning alerts. For example, a data table containing only names or ID cards may not be sensitive data. However, if this data table is accessed in conjunction with another data table containing address information, the combined ID and address information could be leaked. Based on this information, for example, it could be possible to determine the high number of elderly people in a particular building, potentially leading to targeted marketing, fraud, and other unjustified gains. Therefore, for sensitive data whose combined leakage could pose a threat to others or society, timely sensitivity level assessment and precise early warning monitoring are necessary to detect unauthorized access.

[0035] In view of the above phenomenon, this embodiment also provides an entity identification determination step for the associated data table in the entity identification step. Specifically, in the entity identification step, it also includes: judging the association between the data table belonging to the pending entity and the data table with the natural person entity identification of 1 and the data table with the natural person entity identification of 2 based on the data operation log. Similarly, judging the association between the data table belonging to the pending entity and the data table with the organizational entity identification of 1 and the data table with the organizational entity identification of 2 based on the data operation log. For example, when two data tables are associated in SQL, and the number of SQL executions exceeds the preset number, it indicates that the two data tables have an association relationship. When the data table with the pending entity is associated with multiple data tables with clear entity identifications, the data table with the clear entity identification that has the largest total number of SQL executions is used as the object for subsequent calculations.

[0036] In specific implementation, if the ratio of the number of records in the data table of the entity to be determined and the number of records in the data table with the associated natural person entity identifier of 1 is less than the first set ratio, then the data table of the entity to be determined is determined to belong to the natural person entity identifier 3, and its natural person entity identifier is recorded as 3; if the ratio of the number of records in the data table of the entity to be determined and the number of records in the data table with the associated natural person entity identifier of 1 is greater than or equal to the first set ratio, then the data table of the entity to be determined is determined to belong to the natural person entity identifier 4, and its natural person entity identifier is recorded as 4; if the data table of the entity to be determined is associated with the data table with the natural person entity identifier of 2, then the data table of the entity to be determined is determined to belong to the natural person entity identifier 4, and its natural person entity identifier is recorded as 4; the first set ratio is set to 1.5.

[0037] For example, assume that the first set ratio is 150%; the number of records in the data table of the entity to be determined is A, and the number of records in the data table of the associated natural person entity is B, then the ratio of the two is X=A / B.

[0038] If the associated data table has a "Natural Person Entity Identifier" of 1 and X < 150%, it indicates that the data table for the pending entity does not contain historical records and entity keywords, so the entity identifier of the data table for the pending entity is 3; If the associated data table has a "Natural Person Entity Identifier" of 1 and X ≥ 150%, it indicates that the data table for the pending entity contains historical records but does not contain entity keywords. In this case, the entity identifier of the data table for the pending entity is 4. If the associated data table has a "natural person entity identifier" of 2, then the entity identifier of the data table for the entity to be determined is 4.

[0039] Through the above settings, the entity identification of the data table associated with the natural person entity can be effectively distinguished, which is conducive to the subsequent sensitive data level assessment.

[0040] Similarly, if the ratio of the number of records in the entity's pending data table to the number of records in the data table with the associated organizational entity identifier of 1 is less than the second set ratio, then the entity's pending data table is determined to belong to organizational entity identifier 3, and its organizational entity identifier is recorded as 3; if the ratio of the number of records in the entity's pending data table to the number of records in the data table with the associated organizational entity identifier of 1 is greater than or equal to the second set ratio, then the entity's pending data table is determined to belong to organizational entity identifier 4, and its organizational entity identifier is recorded as 4; if the entity's pending data table is associated with the data table with the organizational entity identifier of 2, then the entity's pending data table is determined to belong to organizational entity identifier 4, and its organizational entity identifier is recorded as 4; the second set ratio is set to 1.5.

[0041] For example, assuming the second set ratio is 150%; the number of records in the data table of the entity to be determined is E, and the number of records in the data table of the associated organizational entity is F, then the ratio of the two is Y=E / F.

[0042] If the associated data table has an "Organization Entity ID" of 1 and Y < 150%, it indicates that the data table for the pending entity does not contain historical records and entity keywords, so the entity ID of the data table for the pending entity is recorded as 3; If the associated data table has an "Organization Entity ID" of 1 and Y ≥ 150%, it indicates that the data table for the pending entity contains historical records but does not contain entity keywords. In this case, the entity ID of the data table for the pending entity is 4. If the associated data table has an "Organization Entity ID" of 2, then the entity ID of the data table for the entity to be determined is 4.

[0043] Through the above settings, the entity identification of the data tables of the associated organizational entities can be effectively distinguished, which is beneficial to the subsequent sensitive data level assessment.

[0044] The specific values of the first set ratio and the second set ratio can be reasonably selected according to the actual correlation degree, and are not limited in this embodiment.

[0045] Step S30, feature engineering construction step; for the data table having the natural person entity or organizational entity, extract the feature information related to the natural person entity or organizational entity, and perform feature processing to obtain the feature value of each feature data, and then combine the natural person entity identifier and the organizational entity identifier to construct the natural person entity feature engineering and the organizational entity feature engineering.

[0046] Specifically, features are composed of fields, and different field identifications can be reasonably set using existing technologies based on actual needs. Among them, different field identifications mainly include data table comments, field comments, field types, code sets corresponding to fields, and field values. Field identification is performed by judging whether field comments or data table comments contain keywords, whether the relevant code sets corresponding to the fields contain keywords, statistically analyzing the rationality of field value distribution, and cross-checking with other field values in the record. Based on the field value to be judged, it is checked whether it can be correctly converted to the relevant data type, such as date type, number type, etc. to obtain the field.

[0047] The characteristic information related to the natural person entity includes at least one of basic information, communication information, location information, health data, financial data, biometric information, judicial information, administrative law enforcement information, and the number of data table records; the characteristic information related to the organizational entity includes at least one of organizational entity identification type, basic information, basic tax information, utility fee information, insurance information, provident fund information, real estate registration information, financial data, other financial data, and the number of data table records. New characteristic information can be reasonably selected and added based on actual needs, and this embodiment is not limited here.

[0048] Based on the above characteristic information, we can select appropriate identification methods to effectively identify fields according to actual information requirements. For example, to determine whether it is a nationality field, we can check whether the field value contains the country abbreviation or English letters; whether the field conforms to the 18-digit ID card code rule; and whether the code name contains keywords and the number of codes is correct.

[0049] For example, extracting feature information can leverage Chinese word segmentation technology and custom standard lexicons to effectively identify relevant content. Custom standard lexicons can be rationally constructed based on actual feature information for comparison and verification. Examples include address standard lexicons, administrative division code lexicons, airline code lexicons, train number lexicons, private disease lexicons (keywords related to diseases that may be discriminated against, primarily including mental illnesses and sexually transmitted diseases), and common diseases (excluding private diseases).

[0050] The implementation process is illustrated by taking the recognition of "address location accuracy" as an example: (1) Segment the field value and determine whether it is an address text. Specifically, use HanLP to segment the text. Use HanLP's Nature to determine whether the segmented text has address features. If it has address features, the text is an address text; otherwise, it is not an address text. (2) If it is an address text, the segmentation result is compared with the standard address vocabulary to obtain the location accuracy. The standard address vocabulary contains information such as city, district, street, house number, room number, community name, building name, and coordinates. The segmentation result is compared with the content in the vocabulary level by level to effectively determine the location accuracy (such as house number or room number).

[0051] For example, feature information can be effectively identified by building a recognition model. Taking the "hotel name recognition model" as an example, its implementation steps are detailed as follows: (1) Preparation: Prepare a training set containing 100,000 samples of real hotel names and non-hotel names (including more than 10,000 other organizational names); annotate each text to determine whether it is an organizational name; (2) Data preprocessing: Segment and tag the text in the original dataset; use BERT's tokenizer to convert the text into a corresponding token sequence for subsequent model processing; (3) Load the pre-trained "recognition model": Select the pre-trained bert-base-chinese model and load the corresponding tokenizer; (4) Use the training set to train the loaded pre-trained "recognition model" to adapt to the task of hotel name recognition; (5) Prepare a test set containing approximately 10,000 data items to evaluate the performance of the "recognition model". Use the trained model to predict the test set; (6) Calculate the accuracy, recall and F1 score of the test set based on the prediction results; If these indicators all reach above 90%, the model can be considered to have good performance and saved to obtain an effective hotel name recognition model; (7) Apply the hotel name recognition model to extract feature information from the data table to determine whether the text is a hotel name; When the text content of a field in the data sampling record has more than 90% prediction results of the hotel name, then the field can be determined to be a hotel name field.

[0052] After identifying the above-mentioned feature information, the identified feature information can be processed to obtain the corresponding feature value. Specifically, the feature value assignment rules are reasonably set according to the actual sensitivity of the feature information; and the corresponding feature value is obtained according to the assignment rules for the identified and extracted feature information.

[0053] To effectively illustrate the process of processing feature information to obtain feature values for each feature data, Table 1 and Table 2 below provide examples of feature information assignment rules for a natural person entity and an organizational entity, respectively: Table 1 Rules for assigning characteristic information of natural person entities

[0054] Table 2 Rules for assigning characteristic information of organizational entities

[0055] It should be noted that the above assignment rules serve only as an example and can be adjusted appropriately based on specific sensitive circumstances during actual application. For example, the information on organizational structures in this example covers a wide range of data, but much of this data is non-sensitive and is typically excluded from feature analysis. For example, lists of abnormal business operations, trial case closures, enforcement case closures, lists of dishonest debtors, and environmental protection credit ratings are all non-sensitive data and are not subject to feature extraction or assignment in this example.

[0056] The above assignment rules can be used to effectively assign feature values to the feature information extracted from the data table to construct the natural person entity feature engineering and organizational entity feature engineering.

[0057] Step S40, preliminary sensitivity level determination step; according to the natural person entity feature engineering and organizational entity feature engineering, construct a natural person basic backup model and an organizational basic backup model based on preset sensitivity level judgment rules; determine the preliminary sensitivity level of the data table based on the natural person basic backup model and the organizational basic backup model.

[0058] Among them, the construction of the natural person basic safety net model and the organizational basic safety net model is to provide a preliminary data sensitivity level assessment when the sensitivity level of all data tables has not yet been determined or the number of data tables is relatively small (for example, about 100 tables), so as to lay the foundation for subsequent model optimization and improvement.

[0059] Specifically, it is constructed based on preset sensitivity level assessment rules, which are reasonably set based on the importance of sensitive data. To effectively illustrate the setting of sensitivity level assessment rules, the following Tables 3 and 4 provide examples of sensitivity level assessment rules for natural persons and organizational entities respectively: Table 3 Sensitivity Level Evaluation Rules for Natural Person Entities

[0060] Table 4 Sensitivity Level Evaluation Rules for Organizational Entities

[0061] It should be noted that the “ / ” in the above table indicates that there is no restriction on the content of the data of the corresponding information in the table, that is, the data content of the corresponding information will not affect the determination of its sensitivity level.

[0062] The above-mentioned sensitivity level evaluation rules are used to construct a basic safety net model for natural persons and a basic safety net model for organizations. The above-mentioned safety net model can be used to effectively determine the preliminary sensitivity level of a data table. In this embodiment, the sensitivity levels are divided into five levels: level 1: highly sensitive, level 2: relatively sensitive, level 3: generally sensitive, level 4: low sensitive, and level 5: insensitive. Of course, depending on the actual application, this embodiment can also design the sensitivity levels to other levels, and the sensitivity level evaluation rules can also be designed to other rules that conform to actual applications.

[0063] Step S50, an early warning step; triggering an early warning for the data table with the determined sensitivity level according to the early warning rules, manually processing the early warning and synchronously correcting the sensitivity level of the data table.

[0064] During specific implementation, warning rules are set based on the sensitivity level of the data table determined by the basic safety net model and combined with SQL execution logs (recording information such as client tools, IP, users, runtime periods, and number of returned records). Based on this, warning information is triggered to indicate that if unauthorized persons access data with a sensitive level, a warning warning is required.

[0065] Step S60, model optimization step: combine the data generated by the natural person basic backup model and the organizational basic backup model with the data of the manually corrected sensitivity level of the data table as training samples, and train the natural person optimization model and the organizational optimization model based on the decision tree algorithm.

[0066] In specific implementation, the sensitivity level generated by the basic safety net model is used as a benchmark. When manually processing warnings and annotating the sensitivity level of the data table, if the manually annotated level is inconsistent with the level generated by the basic safety net model, the manually annotated level shall prevail, and a training set shall be formed accordingly. Next, based on the basic safety net model for natural persons and the basic safety net model for organizations, the training samples are divided into multiple training sets and validation sets. Each training set is trained using a decision tree algorithm to obtain a corresponding natural person optimization model and organizational optimization model. The decision tree model selects the optimal partitioning feature at each node using the Gini coefficient. The multiple natural person optimization models and multiple organizational optimization models are then validated using the validation set.

[0067] The steps of selecting the optimal partitioning features by the Gini algorithm include: (1) Calculate the Gini coefficient corresponding to each feature using the following formula: ; Where, represents the Gini coefficient of set P; Indicates the number of categories; Indicates that the set P belongs to the category The proportion of samples in the set P; (2) Calculate the Gini index of the entire feature set using the following formula: ; Where, represents the weighted Gini coefficient of the entire feature set, Indicates the feature set The proportion of the number of samples in the subset to the total sample, Indicates the first The Gini coefficient of the subset; In the above calculation process, the feature and threshold with the smallest Gini index are selected for division to obtain the final optimization model.

[0068] Taking the health data feature value as an example, the relevant Gini coefficient calculation process is illustrated and a decision tree is generated. Assuming 30 samples, the sensitivity level distribution of the health feature data is as follows, and the calculation process is illustrated:

[0069] 1. Calculate the number of each health data type at each sensitivity level: ; ; ; 2. Calculate the Gini index (weighted average according to the number of samples):

[0070] 3. Through layer-by-layer calculation, the features and thresholds with the smallest Gini coefficient are selected for division to select the final optimization model.

[0071] Specifically, how to use the decision tree algorithm to train and generate the natural person optimization model and the organizational structure optimization model can be reasonably set according to the actual conventional training method, and this embodiment will not be repeated here.

[0072] Based on the optimization model generated by the above training set, this embodiment can use multiple verification sets to verify the generated optimization model to obtain the accuracy of the model.

[0073] Then, based on the verification results, check whether the various indicators in the model are underfitting or overfitting. If so, adjust the model parameters to obtain the ideal model.

[0074] For example, when randomly sampling a validation set for sensitivity level assessment, it was found that the accuracy of sensitivity level three was low. Based on this problem, this embodiment discovered that the uneven distribution of samples caused the prediction results to be unbalanced in categories, resulting in low accuracy. Therefore, this embodiment preferably readjusts the number of samples in the validation set and adjusts the proportion of samples for each sensitivity level in the training set and validation set through stratified sampling. For example, 80% of the samples at each sensitivity level are sampled as the training set and 20% of the samples are used as the validation set. For example, if there are only 100 samples at sensitivity level one, 80 are sampled as the training set and 20 as the validation set. If there are 1,000 samples at sensitivity level two, 800 are sampled as the training set and 200 as the validation set. By using the above-mentioned stratified sampling method for samples of different sensitivity levels, the sample proportions of each sensitivity level are ensured to be roughly the same, effectively avoiding the problem of inaccurate assessment of a certain sensitivity level due to sample imbalance, and improving the accuracy of the model.

[0075] In one embodiment, during training, it was found that the number of samples at sensitivity level 5 (not sensitive) was excessive, far exceeding the number of samples at other sensitivity levels, resulting in an imbalance in the weights predicted by the model. Based on this, this embodiment automatically balances the weights by setting sensitivity level weight parameters in the model to avoid this imbalance.

[0076] Furthermore, during the model parameter tuning process, this embodiment discovered that while the training set accuracy exceeded 90%, the validation set exhibited overfitting. To address this issue, this embodiment preferably employs RandomizedSearchCV (randomized search cross-validation), which searches through random sampling in the hyperparameter space, validating the prediction results with different parameters each time. This effectively avoids overfitting of the validation set. This method randomly selects different parameter combinations for cross-validation, targeting key parameters in the decision tree algorithm, such as min_samples_split (the minimum number of samples required for node splitting) and min_samples_leaf (the minimum number of samples for leaf nodes). After multiple rounds of iterative training and validation of these parameter combinations, the model's evaluation metrics on the validation set were significantly improved, with accuracy, recall, and F1 scores all exceeding 85%. This method effectively avoids overfitting of the validation set caused by validation using fixed parameter combinations while ensuring effective parameter tuning.

[0077] Step S70, final sensitivity level assessment step: use the natural person optimization model and the organizational structure optimization model to perform the final sensitivity level assessment of the data table.

[0078] During specific implementation, a data table to be assessed is obtained, and the sensitivity level of the data table is assessed using multiple groups of natural person optimization models or multiple groups of organizational optimization models, so as to obtain the assessment results of each natural person optimization model or multiple groups of organizational optimization models; According to the evaluation results of each natural person optimization model or multiple groups of organizational optimization models, the number of evaluations for each sensitivity level is counted, and the sensitivity level with the largest number of evaluations is used as the final sensitivity level of the data table to be evaluated.

[0079] That is, this embodiment trains the training set by adopting a decision tree algorithm, generates a corresponding decision tree after the training is completed, and uses multiple training sets to generate multiple optimization models.

[0080] See also Figure 2 、 Figure 3 , this embodiment preferably obtains three decision trees by training three training sets, and obtains three optimization models by training three validation sets. When a new data set is transmitted in, it is evaluated by the three optimization models to obtain the evaluation results, and then the number of predictions for each sensitivity level is counted, and the sensitivity level with the most evaluations is used as the final sensitivity level. For example, the second-level sensitivity is obtained twice and the third-level sensitivity is obtained once through the three optimization models respectively, so the final sensitivity level can be obtained as the second level. In the actual training process, too few optimization models are likely to lead to insufficient accuracy, and too many optimization models are likely to lead to increased computational costs and reduced efficiency. Therefore, this embodiment preferably uses three optimization models to evaluate the final sensitivity level.

[0081] Through the above settings, the optimization model of multiple decision trees can be effectively utilized to improve the robustness and generalization ability of the model.

[0082] Example 2 The method for automatically identifying sensitive numbers provided in the embodiment of the present invention can be effectively applied to government shared data to automatically identify sensitive data in government shared data. Government shared data refers to data resources shared between government departments in accordance with laws and regulations to achieve cross-departmental collaboration, improve public service efficiency, and optimize social governance. In this specific embodiment 2, the solution of the embodiment of the present invention is described in detail using a data table from a government data sharing center as an example.

[0083] Currently, the government data sharing center has 16,184 data tables, covering a wide range of detailed information on natural persons and organizational entities. Preliminary analysis revealed that 8,623 data tables (including temporary tables and intermediate tables generated during the calculation process) have not yet been registered. To ensure data security, the automatic sensitive data identification method provided in this embodiment is adopted to accurately identify sensitive data from massive data tables and implement effective management. The core of the automatic sensitive data identification method provided in this embodiment lies in optimizing and improving the sensitivity level assessment through entity recognition, feature engineering construction, the application of a safety net model, and model optimization and improvement. The specific steps are as follows: (1) Data acquisition step: Obtain a total of 24,807 data tables (including registered formal tables and unregistered hidden and temporary tables) from the government data sharing center. The data tables include at least one of metadata, data content, and data operation logs.

[0084] (2) Entity Identification Step: Identify 24,807 data tables and use key field screening to accurately distinguish natural person entities, organizational entities, and entities unrelated to either. This step is the foundation of the entire identification process and ensures the accuracy of subsequent sensitivity level assessments.

[0085] (3) Feature engineering construction step: construct feature engineering for each data table containing the natural person entity or organizational entity to extract feature information of each natural person entity or organizational entity.

[0086] (4) Preliminary sensitivity level determination step: After completing the feature engineering, we used the rules of the natural person and organizational model to conduct a preliminary sensitivity level assessment of the natural person and organizational entity data. This step enables us to quickly understand the sensitivity of the natural person and organizational entity data in the government data sharing center.

[0087] (5) Early warning step: During the initial sensitivity level assessment process, combined with SQL execution logs (recording client tools, IP addresses, users, runtimes, number of returned records, etc.), early warning information is triggered according to the preset early warning rules. When manually processing the early warning, the sensitivity level of the data table is simultaneously marked and corrected.

[0088] (6) Model optimization step: Using the data table with the corrected sensitivity level as a training sample, a natural person optimization model and an organizational optimization model are generated based on the decision tree algorithm to replace the underlying model as the sensitivity level assessment model. In this process, on the one hand, the algorithm can be improved based on the content of Example 1, and on the other hand, feature engineering can be improved based on actual problems encountered to improve the accuracy of the model.

[0089] For example, using the natural person optimization model as an example, this embodiment, building on the existing feature engineering, directly incorporates the "date of birth" field as one of the feature information. Specifically, the data is divided into the following categories: 0 represents multiple age groups, 1 represents children / adolescents (0-18 years old), 2 represents young adults / early adulthood (18-35 years old), 3 represents middle adulthood (35-50 years old), 4 represents late adulthood (50-60 years old), and 5 represents seniors (over 60 years old). This design is based on the observation that, in practical applications, when the natural person entity identification tag is 1 or 3 (natural person entity without historical information), the location information is 1 (located to the room or building number), and the date of birth is 0 (multiple age groups) or 5 (senior citizens), the data sensitivity level should be set to level 1 (highly sensitive). Similarly, when the entity identification tag is 1 or 3, the location information is 2 or 3 (coordinates or house number, etc.), and the date of birth is 0 or 5, the data sensitivity level should be set to level 2 (relatively sensitive). (Analysis suggests that this may be because data on senior citizens is easily used for dangerous activities such as door-to-door product recommendations, thus requiring a higher sensitivity level.) However, in the original natural person protection model, the sensitivity level of this type of data is typically classified as level three, namely, moderately sensitive. However, through improvements in feature engineering and analysis of practical applications, we found that the sensitivity of this data is actually higher, which does not meet the requirements of the intended application. Therefore, by directly defining the "date of birth" field as feature information in the feature engineering mentioned above, we can effectively ensure that the sensitivity level classification meets the preset requirements.

[0090] It should be noted that, according to the inventive concept, this embodiment is not limited to the above-mentioned improvement methods for feature engineering. There are other methods for adjusting feature engineering based on actual problems discovered, all of which fall within the scope of protection of the embodiments of the present invention.

[0091] After a period of operation and optimization based on the above model, the latest natural person optimization model and organizational optimization model achieved an accuracy rate of 99% in predicting sensitivity levels for the 100 newly added data tables.

[0092] In summary, by adopting the automatic identification method of sensitive data provided by the embodiment of the present invention, the government data sharing center has successfully achieved accurate identification of sensitive data in massive data tables, which not only effectively reduces the risk of data leakage, but also significantly improves the efficiency and quality of data processing, providing strong guarantees for the safe sharing and utilization of government data.

[0093] Example 3 The present invention also provides a sensitive data automatic identification system, such as Figure 4 Shown, including: A data acquisition module; used to acquire a data table, wherein the data table includes at least one of metadata, data content, and data operation log; Entity recognition module; used to perform entity recognition on the data table to identify natural person entities and organizational entities based on key fields; determine the natural person entity identifier and organizational entity identifier of each data table based on the ratio between the number of records in the data table containing natural person entities and organizational entities and the number of key field types; A feature engineering construction module is used to extract feature information related to the natural person entity or organizational entity from a data table containing the natural person entity or organizational entity, perform feature processing to obtain feature values of each feature data, and then construct a natural person entity feature engineering and an organizational entity feature engineering based on the natural person entity identifier and the organizational entity identifier; A preliminary sensitivity level determination module is configured to construct a natural person basic backup model and an organizational basic backup model based on the natural person entity feature engineering and organizational entity feature engineering, based on preset sensitivity level evaluation rules; and to determine the preliminary sensitivity level of the data table based on the natural person basic backup model and the organizational basic backup model. An early warning module; used to trigger an early warning for a data table of a determined sensitivity level according to an early warning rule, manually process the early warning, and simultaneously correct the sensitivity level of the data table; Optimization model module: used to combine the data generated by the natural person basic backup model and the organizational basic backup model with the data of the manually corrected data table sensitivity level as training samples, and train the natural person optimization model and organizational optimization model based on the decision tree algorithm; Final sensitivity level assessment module; used to assess the final sensitivity level of data tables using the natural person optimization model and organizational optimization model.

[0094] The specific design of each module can be referred to the description of the first embodiment, which will not be repeated here.

[0095] Example 4 The present invention also provides a computer-readable storage medium, which stores computer instructions and, when executed by a computer processor, implements the method for automatically identifying sensitive data as described in Example 1.

[0096] In a specific implementation, the computer-readable storage medium is a magnetic disk, an optical disk, a read-only memory (ROM), a random access memory (RAM), a flash memory (Flash Memory), a hard disk drive (HDD) or a solid-state drive (SSD), etc.; the computer-readable storage medium may also include a combination of the above types of memory.

[0097] Example 5 The present invention also provides an electronic device, such as Figure 5 As shown, it includes at least one processor and a memory communicatively connected to the processor, wherein the memory stores instructions that can be executed by at least one processor, and the instructions are executed by at least one processor to enable the processor to perform the method for automatic identification of sensitive data as described in Example 1.

[0098] In a specific implementation, the number of processors may be one or more, and the processor may be a central processing unit (CPU). The processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or a combination of the above chips. The general-purpose processor may be a microprocessor or any conventional processor.

[0099] The memory and the processor can be communicatively connected via a bus or other means. The memory stores instructions that can be executed by at least one processor. The instructions are executed by at least one processor to enable the processor to execute the sensitive data automatic identification method introduced in the above method embodiment.

[0100] Although this document frequently uses terms such as data table, natural person entity, organizational entity, natural person entity identifier, organizational entity identifier, natural person entity feature engineering, organizational entity feature engineering, natural person basic backup model, organizational basic backup model, natural person optimization model, and organizational optimization model, the use of other terms is not excluded. These terms are used solely to more conveniently describe and explain the essence of the present invention; interpreting them as any additional limitations is contrary to the spirit of the present invention.

[0101] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for automatically identifying sensitive data, characterized in that: The steps include: Data acquisition step: acquiring a data table, wherein the data table includes at least one of metadata, data content, and data operation log; Entity identification step: performing entity identification on the data table to identify natural person entities and organizational entities based on key fields; then determining the natural person entity identifier and organizational entity identifier of each data table based on the ratio between the number of records in the data table containing natural person entities and organizational entities and the number of key field types; A feature engineering construction step: for a data table containing the natural person entity or organizational entity, extracting feature information related to the natural person entity or organizational entity, performing feature processing to obtain feature values of each feature data, and then constructing a natural person entity feature engineering and an organizational entity feature engineering in combination with the natural person entity identifier and the organizational entity identifier; A preliminary sensitivity level determination step: constructing a natural person basic underlay model and an organizational basic underlay model based on the natural person entity feature engineering and organizational entity feature engineering based on preset sensitivity level evaluation rules; determining the preliminary sensitivity level of the data table based on the natural person basic underlay model and organizational basic underlay model; Early warning step: triggering an early warning for a data table of a determined sensitivity level according to the early warning rules, manually processing the early warning and simultaneously correcting the sensitivity level of the data table; Optimize model steps; The data generated by the natural person basic safety net model and the organizational basic safety net model are combined with the data of the manually corrected sensitivity level data table as training samples, and the natural person optimization model and organizational optimization model are trained based on the decision tree algorithm respectively; Final sensitivity level assessment steps: Use the natural person optimization model and organizational structure optimization model to perform the final sensitivity level assessment of the data table.

2. The method for automatically identifying sensitive data according to claim 1, wherein: Before the entity recognition step, a rating condition determination step is also included; if the number of records in the data table exceeds a preset number or the amount of data in the data table exceeds a preset storage value, the entity recognition step is performed on the data table; Otherwise, the data table is discarded and the entity recognition step is not performed.

3. The method for automatically identifying sensitive data according to claim 1, wherein: In the entity recognition step, performing entity recognition on the data table includes: Extract records from each data table and input them into the recognition model to determine whether there are key fields of natural persons or organizations in the records; if there are key fields, determine whether the data table belongs to a natural person entity, an organizational entity, or a non-unique entity based on the key fields; if there are no key fields, determine whether the data table belongs to an entity to be determined.

4. The method for automatically identifying sensitive data according to claim 1, wherein: The method of determining the natural person entity identifier and organizational entity identifier of each data table based on the ratio between the number of records and the number of key field types in the data table having the natural person entity and the organizational entity comprises the following steps: When the data table has key fields of a natural person entity, if the ratio of the number of records to the number of key field types is less than a first preset ratio, the data table is determined to belong to natural person entity identification 1, and its natural person entity identification is recorded as 1; if the ratio of the number of records to the number of key field types is greater than or equal to the first preset ratio and less than or equal to the second preset ratio, the data table is determined to belong to natural person entity identification 2, and its natural person entity identification is recorded as 2; if the ratio of the number of records to the number of key field types is greater than or equal to a third preset ratio or the ratio between different key fields of the same natural person is less than a fourth preset ratio, the data table is determined to belong to an entity to be determined; wherein, the first preset ratio is greater than 1 and less than 1.5, the second preset ratio is greater than or equal to 1.5 and less than 100, the third preset ratio is greater than 100; and the fourth preset ratio is 0.4; When the data table has a key field of an organizational entity, if the key field in the data table is unique, the data table is determined to belong to organizational entity identification 1, and its organizational entity identification is recorded as 1; if the data table contains multiple organizational entities and the ratio of the number of key field types in the data table to the number of records is judged according to the record quantity classification and meets the specified ratio, the data table is determined to belong to organizational entity identification 2, and its organizational entity identification is recorded as 2; if the data table contains multiple organizational entities and does not meet the specified ratio, the data table is determined to belong to entity to be determined; the specified ratio is greater than 0 and less than 2%.

5. The method for automatically identifying sensitive data according to claim 4, characterized in that: The entity identification step further includes: determining, based on the data operation log, the association between the data table belonging to the pending entity and the data table with the natural person entity identifier 1 and the data table with the natural person entity identifier 2; If the ratio of the number of records in the data table for the entity to be determined and the number of records in the data table with the associated natural person entity identifier of 1 is less than the first set ratio, the data table for the entity to be determined is determined to belong to the natural person entity identifier 3, and its natural person entity identifier is recorded as 3; if the ratio of the number of records in the data table for the entity to be determined and the number of records in the data table with the associated natural person entity identifier of 1 is greater than or equal to the first set ratio, the data table for the entity to be determined is determined to belong to the natural person entity identifier 4, and its natural person entity identifier is recorded as 4; if the data table for the entity to be determined is associated with the data table with the natural person entity identifier of 2, the data table for the entity to be determined is determined to belong to the natural person entity identifier 4, and its natural person entity identifier is recorded as 4; the first set ratio is set to 1.5; Determine, based on the data operation log, the association between the data table belonging to the pending entity and the data table with the organizational entity identifier 1 and the data table with the organizational entity identifier 2; If the ratio of the number of records in the entity's pending data table to the number of records in the data table with the associated organizational entity identifier of 1 is less than the second set ratio, then the entity's pending data table is determined to belong to organizational entity identifier 3, and its organizational entity identifier is recorded as 3; if the ratio of the number of records in the entity's pending data table to the number of records in the data table with the associated organizational entity identifier of 1 is greater than or equal to the second set ratio, then the entity's pending data table is determined to belong to organizational entity identifier 4, and its organizational entity identifier is recorded as 4; if the entity's pending data table is associated with the data table with the organizational entity identifier of 2, then the entity's pending data table is determined to belong to organizational entity identifier 4, and its organizational entity identifier is recorded as 4; the second set ratio is set to 1.

5.

6. The method for automatically identifying sensitive data according to claim 1, wherein: In the feature engineering construction step, the feature information related to the natural person entity includes at least one feature data of basic information, communication information, location information, health data, financial data, biometric information, judicial information, administrative law enforcement information, and the number of data table records; the feature information related to the organizational entity includes at least one feature data of organizational entity identification type, basic information, basic tax information, water, electricity, gas fee information, insurance information, provident fund information, real estate registration information, financial data, other financial data, and the number of data table records.

7. The method for automatically identifying sensitive data according to claim 1, wherein: In the model optimization step, the data generated by the natural person basic backup model and the organizational basic backup model are combined with the data of the manually corrected data table sensitivity level as training samples, and the natural person optimization model and organizational optimization model are trained based on the decision tree algorithm, including the following steps: Relying on the natural person basic support model and the organizational basic support model, the training samples are divided into multiple training sets and validation sets, and each training set is trained using a decision tree algorithm to obtain a corresponding natural person optimization model and organizational optimization model. The decision tree algorithm selects the optimal partitioning feature at each node using the Gini coefficient; the validation set is then used to validate the multiple natural person optimization models and the multiple organizational optimization models. In the final sensitivity level assessment step, the natural person optimization model and the organizational structure optimization model are used to perform the final sensitivity level assessment of the data table, including the following steps: Obtain a data table to be assessed, and assess the sensitivity level of the data table using multiple groups of natural person optimization models or multiple groups of organizational optimization models to obtain an assessment result for each natural person optimization model or organizational optimization model; According to the evaluation results of each natural person optimization model or multiple groups of organizational optimization models, the number of evaluations for each sensitivity level is counted, and the sensitivity level with the largest number of evaluations is used as the final sensitivity level of the data table to be evaluated.

8. A sensitive data automatic identification system, characterized in that: include: Data acquisition module; Used to obtain a data table, where the data table includes at least one of metadata, data content, and data operation log; Entity recognition module; Used to perform entity recognition on the data table to identify natural person entities and organizational entities based on key fields; then determine the natural person entity identifier and organizational entity identifier of each data table based on the ratio between the number of records in the data table containing natural person entities and organizational entities and the number of key field types; Feature engineering building blocks; For extracting feature information related to the natural person entity or organizational entity from a data table containing the natural person entity or organizational entity, performing feature processing to obtain feature values of each feature data, and then constructing natural person entity feature engineering and organizational entity feature engineering in combination with the natural person entity identifier and the organizational entity identifier; Preliminary sensitivity level determination module; Used to construct a natural person basic underlay model and an organizational basic underlay model based on the natural person entity feature engineering and organizational entity feature engineering, based on preset sensitivity level assessment rules; and determine the preliminary sensitivity level of the data table based on the natural person basic underlay model and organizational basic underlay model; Early warning module; Used to trigger an alert for a data table with a determined sensitivity level according to the alert rules, manually process the alert and simultaneously correct the sensitivity level of the data table; Optimize model modules; It is used to combine the data generated by the natural person basic safety net model and the organizational basic safety net model with the data of the manually corrected sensitivity level of the data table as training samples, and train the natural person optimization model and organizational optimization model respectively based on the decision tree algorithm; Final sensitivity rating module; Used to assess the final sensitivity level of data tables using the natural person optimization model and organizational optimization model.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and when the computer is executed by a processor, the method for automatically identifying sensitive data according to any one of claims 1 to 7 is implemented.

10. An electronic device, characterized in that: It includes at least one processor and a memory communicatively connected to the processor, wherein the memory stores instructions that can be executed by at least one processor, and the instructions are executed by at least one processor to enable the processor to perform the method for automatic identification of sensitive data as described in any one of claims 1-7.

Images