A data desensitization method based on bloodline analysis and its system

Abstract

This invention relates to the field of data anonymization technology, and in particular to a data anonymization method and system based on lineage analysis. The lineage analysis-based data anonymization method includes: establishing a first table and a second table in a database; establishing a third table and storing it in the system, configuring anonymization settings for the third table in the system, wherein the third table includes a combined table and a chart; storing the anonymization settings for the third table in the first table; when the system starts, scanning the third table and recording the scan results in the second table, and recording fields in the third table that cannot be anonymized in the first table; when establishing a fourth table or modifying the third table, determining whether the user's current operation, if continued, would lead to the leakage of the original data on the anonymized fields; if so, prohibiting the operation; if not, allowing the establishment of the fourth table or modification of the third table, and inheriting the anonymization settings from the first table to the establishment of the fourth table or modification of the third table.

Description

Technical Field

[0001] This invention relates to the field of data desensitization technology, and in particular to a data desensitization method based on bloodline analysis and a system thereof. Background Technology

[0002] With technological advancements, users are generating increasingly more data online, making their personal information more and more sensitive. Therefore, while conducting reasonable data analysis, businesses need to anonymize key user information to protect it from leakage.

[0003] Most existing data masking solutions focus on masking the actual stored data. While this approach offers the advantage of more secure masked data, it has disadvantages: firstly, the configuration and full effectiveness of masking rules can be delayed depending on the data volume and the solution's principles; secondly, it cannot meet the masking requirements for generating intermediate results and creating necessary analytical charts based on those intermediate results during data source processing.

[0004] For the reasons mentioned above, there is an urgent need for a data anonymization method that can anonymize data only during display without performing anonymization during data processing. Summary of the Invention

[0005] This invention provides a data anonymization method and system based on lineage analysis, which solves the problems of delays caused by different data volumes and scheme principles in the prior art, and the inability to meet the anonymization requirements for generating intermediate results and creating necessary analytical charts based on intermediate results when processing data sources.

[0006] The first aspect of this application provides a data desensitization method based on bloodline analysis, characterized in that it includes:

[0007] Create a first table and a second table in the database. The first table represents the de-identification configuration and includes a field for identifying the unique ID of the work table and the merged table, a field for storing the user's unique ID, a field for determining whether de-identification is required, a field for editing the de-identification configuration, and a field for storing the de-identification configuration. The second table represents the lineage relationship between the fields in the storage table, including calculating the unique ID of the source table, calculating the unique ID of the source field, calculating the unique ID of the result work table, and calculating the unique ID of the result field.

[0008] A third table is created and stored in the system. An anonymization configuration is configured for the third table in the system. The third table includes a combined table and a chart.

[0009] Store the de-identification configuration of the third table in the first table;

[0010] When the system is started, the third table is scanned, and the scan results are recorded in the second table. Fields in the third table that cannot be de-identified are also recorded in the first table.

[0011] When creating a fourth table or modifying a third table, determine whether continuing the user's current operation will lead to the leakage of the original data in the masked fields. The fourth table refers to a combined table or chart created by the user.

[0012] If applicable, the operation is prohibited; otherwise, the creation of the fourth table or modification of the third table is allowed, and the de-identification configuration in the first table is inherited when creating the fourth table or modifying the third table.

[0013] In one feasible approach, the step of creating the first table and the second table in the database includes:

[0014] The field storing the user's unique ID indicates that each user's permission to access the first table is limited to one record in the first table.

[0015] The field for editing the de-identification configuration indicates whether the user can edit the field for the de-identification configuration of the first table;

[0016] The fields used to store the de-identification configuration are those used to store the de-identification configuration.

[0017] The unique ID of the calculation source table represents the unique ID of the worksheet composed of third-party data sources;

[0018] The unique ID of the calculation result worksheet represents the unique ID of the intermediate result table generated by the third-party data source.

[0019] In one feasible approach, the step of creating a third table, storing the third table in the system, and configuring de-identification settings for the third table in the system includes:

[0020] Create a de-identification configuration for the third table that conforms to the rules of the first table.

[0021] In one possible implementation, the step of storing the de-identification configuration of the third table in the first table includes:

[0022] If the third table is displayed, retrieve the de-identification configuration of the third table from the first table;

[0023] Based on the anonymization configuration corresponding to the third table in the first table, the third table is displayed.

[0024] In one possible implementation, the steps of scanning the third table and recording the scan results in the second table when the system is started, and recording fields in the third table that cannot be de-identified in the first table, include:

[0025] Establish and / or update the lineage relationships between the fields in the worksheet;

[0026] Determine whether the fields in each table in the chain that constitutes the merged table will lead to the leakage of de-identified data;

[0027] If a field exists that could lead to data leakage due to data masking, mark it as a field that cannot be masked and store it in the first table. Also, add a flag to the configuration of the third table that uses the field in the first table to prohibit data masking for that field.

[0028] In one possible implementation, after the steps of scanning the third table and recording the scan results in the second table when the system is started, and recording the fields in the third table that cannot be de-identified in the first table, the method further includes:

[0029] When a user adjusts the desensitization configuration corresponding to a field in the third table, the desensitization configuration is inherited from the first table to all downstream tables that use the field, based on the calculated lineage relationship recorded in the second table. The adjustment of the desensitization configuration includes at least one of the following adjustment methods: creating, modifying, and deleting the desensitization configuration.

[0030] In one feasible approach, the first table, the second table, and the third table are configured in the big data analysis tool program, and the first table, the second table, and the third table are converted into Spark SQL statements and provided to Spark for execution to generate charts or combined tables that meet user requirements.

[0031] Based on the ANTLR parser, Spark is used to parse the data tables, fields, and computational logic used by Spark SQL. The data tables refer to worksheets or merged tables.

[0032] A second aspect of this application provides a data desensitization system based on kinship analysis, applied to the aforementioned data desensitization method based on kinship analysis, the system comprising:

[0033] The first creation unit is used to create a first table and a second table in the database. The first table represents the de-identification configuration and includes a field for identifying the unique ID of the work table and the merged table, a field for storing the user's unique ID, a field for determining whether de-identification is required, a field for editing the de-identification configuration, and a field for storing the de-identification configuration. The second table represents the lineage relationship between the fields in the storage table and includes calculating the unique ID of the source table, calculating the unique ID of the source field, calculating the unique ID of the result work table, and calculating the unique ID of the result field.

[0034] The second creation unit is used to create a third table and store the third table in the system, and configure the desensitization configuration for the third table in the system, wherein the third table includes a combined table and a chart;

[0035] A storage unit is used to store the de-identification configuration of the third table in the first table;

[0036] The scanning unit is used to scan the third table when the system is started, record the scan results in the second table, and record the fields in the third table that cannot be de-identified in the first table.

[0037] The first judgment unit is used to determine whether the user's current operation will lead to the leakage of the original data on the de-identified fields if the user continues to perform the operation when creating the fourth table or modifying the third table. The fourth table refers to the combined table or chart created by the user.

[0038] The second judgment unit is used to prohibit the operation if the condition is met, and to allow the creation of the fourth table or the modification of the third table if the condition is not met, and to inherit the desensitization configuration in the first table to the creation of the fourth table or the modification of the third table.

[0039] A third aspect of this application provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the aforementioned data desensitization method based on lineage analysis.

[0040] A fourth aspect of this application provides a computer storage medium storing a computer program thereon, characterized in that the computer program, when executed by a processor, implements the steps of the aforementioned data desensitization method based on lineage analysis.

[0041] Beneficial effects of this invention:

[0042] This application provides a data anonymization method and system based on lineage analysis. First, a first table and a second table are created in the database to store anonymization and permission configurations, as well as the lineage relationships between table fields. Next, a third table is created in the system to represent user-created tables or charts, and corresponding anonymization configurations are configured for the third table. This allows the anonymization configuration of the third table to be stored in the first table for later display. Additionally, upon system startup, the third table is scanned, and the scan results are recorded in the second table, along with fields in the third table that cannot be anonymized. Finally, when creating a fourth table or modifying the third table, it is determined whether continuing the user's current operation would lead to the leakage of the original data in the anonymized fields. If so, the operation is prohibited; otherwise, creation or modification is allowed, and the anonymization configurations in the first table are inherited when creating the fourth table or modifying the third table. The above approach establishes masking configurations for the newly created third and fourth tables. Each time a masking configuration is created, modified, or deleted in the chain between the third and fourth tables, the system traces back to the initial configuration location through fields in the second table or similar related record tables to determine how to mask the data. Furthermore, for each third or fourth table that is determined to inherit the masking configuration through tracing, a corresponding masking configuration is generated. This speeds up high-frequency operations (queries) and avoids the need for recursive queries of other table configurations. Attached Figure Description

[0043] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0044] Figure 1 This is a flowchart of a data desensitization method based on bloodline analysis according to the present invention;

[0045] Figure 2 This is a flowchart illustrating the process of the third table in the present invention, which demonstrates a data desensitization method based on bloodline analysis.

[0046] Figure 3 This is a flowchart illustrating the determination of the combined table link in a data desensitization method based on bloodline analysis according to the present invention.

[0047] Figure 4 This is a flowchart illustrating the data desensitization process for users viewing a table and the configuration of permissions for editing the desensitization settings in a data desensitization method based on lineage analysis according to the present invention.

[0048] Figure 5 This is a flowchart illustrating the computer calculation process of ANTLR parsing SQL, which is used in a data desensitization method based on lineage analysis according to the present invention. Detailed Implementation

[0049] The technical solution of the present invention will be clearly and completely described below with reference to the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0050] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the stated features. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified. Furthermore, the terms "installed," "connected," and "linked" should be interpreted broadly; for example, they may refer to a fixed connection, a detachable connection, or an integral connection; they may refer to a mechanical connection or an electrical connection; they may refer to a direct connection or an indirect connection through an intermediate medium; and they may refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0051] The following explanations of some of the names appearing in this application are provided to aid in understanding the technical solution of this application:

[0052] Data sources are connected to the system, bringing in data from various third-party data sources for processing and analysis.

[0053] A worksheet is the name of a data source within a big data analytics tool.

[0054] "Table merging" refers to processing worksheets to generate intermediate results in big data analytics tools. For example, table merging involves operations such as joining, aggregating, and combining worksheets. These operations facilitate clarifying the data logic within the worksheets, reusing analysis results, and generating charts. This operation is used to stratify data in big data analytics, which is beneficial for handling complex data analysis needs and is widely used in data analysis.

[0055] A chart is a graphical representation of the results of an analysis, created using worksheets or intermediate results generated through spreadsheet processing.

[0056] Bloodline relationship refers to the calculated relationship between the fields in the storage table of the second table.

[0057] The results table refers to the final table formed after combining the calculations from the tables.

[0058] A chain refers to the specific calculation process by which worksheets and merged tables, through set calculations in the data source, ultimately produce the merged table or chart required by the user.

[0059] Big data analytics tools refer to enterprise-level big data visualization and analysis tools based on the Hadoop ecosystem.

[0060] ANTLR, or Another Tool for Language Recognition, is a cross-language parser that can be used to read, process, execute, or translate structured text or binary files. It is widely used to build languages, tools, and frameworks. ANTLR can generate a parser from a grammatical perspective that can build and traverse a parse tree.

[0061] Relatedly, in big data analytics tool systems, while conducting reasonable analysis of third-party data sources, it is necessary to anonymize key user information from these sources to protect users' personal information from leakage. Big data analytics tool systems typically contain the following types of functions:

[0062] 1. Data source access: Access data from various third-party data sources into the system for processing and analysis.

[0063] 2. Process the data source to generate intermediate results.

[0064] 3. Use the data source or the intermediate results generated by processing in Function 2 to create the required analysis charts.

[0065] 4. Implement access control for generated worksheets, charts, and merged tables, controlling viewing and editing permissions. The granularity of permissions can be precise down to individual worksheets, charts, etc.

[0066] There are several challenges in de-identifying data within big data analytics tools and systems:

[0067] 1. When a worksheet or merge table has a data masking rule configured, if a new merge table or chart is created using this rule, it may use fields in the original worksheet that have been masked. This rule needs to be correctly inherited into the new merge table to avoid leakage of the original data. Similarly, when configuring a new rule, if there are already child tables / charts created based on this worksheet or merge table, the rule also needs to be inherited into the child tables immediately.

[0068] 2. Users typically want data anonymization to minimize its impact on data calculations. For example, when performing a join query between two tables using fields like ID card number or mobile phone number. This requires that the original data be used for data storage and calculation, with anonymization only applied during display. Therefore, a comprehensive review and coverage of functions that display data is necessary.

[0069] Many existing solutions focus on anonymizing the actual stored data. While this approach offers the advantage of enhanced data security, it also has drawbacks: firstly, the configuration and full effectiveness of the anonymization rules can be delayed depending on the data volume and the solution's underlying principles; secondly, it fails to meet the requirements of the aforementioned challenge 2.

[0070] For the reasons mentioned above, the first aspect of this application provides a data desensitization method based on bloodline analysis, which can be implemented by three functional modules, namely the first functional module, the second functional module and the third functional module.

[0071] The first functional module is responsible for reading and writing the de-identification scheme, as well as implementing the de-identification behavior, including implementing different de-identification effects (whether to de-identify) for users with different permissions.

[0072] The second functional module parses user-input table merging and chart creation operations, and decides whether to allow the operation based on whether it would lead to the leakage of the original anonymized data. If allowed, the anonymization configuration is inherited according to the data lineage.

[0073] The third functional module maintains the changes in the lineage relationships between table fields, and updates the lineage relationships between table fields each time a calculation method changes, or when a calculation method is generated or deleted.

[0074] like Figure 1 As shown, the method includes:

[0075] S100: Create a first table and a second table in the database, and configure the first table and the second table to form a de-identification configuration.

[0076] The first table represents the data masking configuration. It includes fields for identifying the unique IDs of the worksheet and merged table, storing the user's unique ID, determining whether data masking is required, editing the data masking configuration, and storing the data masking configuration. The second table represents the lineage relationship between the fields in the storage table, including the unique ID of the source table, the unique ID of the source field, the unique ID of the result worksheet, and the unique ID of the result field.

[0077] Specifically, the field that stores the user's unique ID indicates that each user's permission to access one record in the first table is limited to that record.

[0078] The field for editing the de-identification configuration indicates whether the user can edit the field configured for de-identification in the first table.

[0079] The field for storing de-identification configurations indicates the field used to store de-identification configurations.

[0080] Calculate the unique ID of the source table. This can be the unique ID of a worksheet composed of third-party data sources, or it can be the unique ID of a worksheet composed of other combined tables in the system.

[0081] The unique ID of the calculation results worksheet represents the unique ID of the intermediate result table generated by the third-party data source, or it can be the unique ID of the intermediate result table generated by other intermediate results in the system through calculation.

[0082] It should be noted that each worksheet and merged table has a corresponding de-identification configuration for each user, stored in the first table. The "Fields for Editing De-identification Configuration" field in the first table indicates whether the user has the permission to edit the de-identification configuration of this table as a whole. Based on this, the "Fields for Storage of De-identification Configuration" field stores whether each field in this table is allowed to be configured with de-identification rules. If S401-S403 detects that a certain field in the third table is not allowed to be configured with de-identification rules, it will be stored in this field in the first table. If configuration is allowed, this field in the first table stores the specific de-identification rules. The content of the de-identification rules includes: (1) the fields that need to be de-identified; (2) the specific de-identification method, for example, starting from a certain character and continuing for a certain number of characters.

[0083] It should also be noted that the database is for big data analysis tools, and the unique IDs facilitate locating and searching in the first and second tables.

[0084] S200: Create a third table and store it in the system. Configure the data masking settings for the third table in the system.

[0085] The third table includes a combined table and charts.

[0086] Specifically, create a desensitization configuration for the third table that conforms to the rules of the first table.

[0087] The third table is a table created by the user in the big data analysis tool program according to their needs. It can be a combined table or a chart formed from the combined table for analysis.

[0088] It should be noted that the system is composed of big data analysis tools and programs.

[0089] Configure the anonymization settings for the third table in the big data analytics tool program. This anonymization settings will determine how the third table is displayed.

[0090] S300: Store the desensitization configuration of the third table in the first table.

[0091] like Figure 2As shown, the process of displaying the third table includes steps S301 and S302.

[0092] S301: If the third table is displayed, retrieve the desensitization configuration of the third table from the first table.

[0093] S302: Display the third table based on the desensitization configuration corresponding to the third table in the first table.

[0094] Each merged table has a corresponding desensitization configuration. When it is necessary to desensitize the results displayed in the third table, the desensitization can be performed directly by reading the configuration from the first table, which is a desensitization configuration stored in the first table.

[0095] It's important to note that multiple duplicate masking configurations (repeated masking configurations) may exist in the tables along the table merging chain that constitute the third table. This is because a table configured with masking rules may be used multiple times, generating different sub-tables or merged tables. In other words, if these sub-tables or merged tables all use the masked fields under the masking rules, it will result in masking configurations existing in these sub-tables or merged tables, thus creating multiple duplicate masking configurations. Therefore, when masking configurations exist in each sub-table or merged table, queries only need to check if the fields in the sub-table or merged table have masking rules. It's no longer necessary to trace back through lineage to the table that initially set the rules. For example, a sub-table or merged table might trace back to the third table initially accessed from a third-party data source in the computation chain via the second table, thus avoiding recursive query steps. In other words, multiple duplicate masking configurations can speed up high-frequency operations (queries) by avoiding the need to recursively query the configurations of other tables.

[0096] S400: When the system starts, it scans the third table and records the scan results in the second table, as well as records the fields in the third table that cannot be de-identified in the first table.

[0097] When the system starts, the third functional module scans all existing worksheets and merged tables in the system and records the results in the second table. During the scanning process, the second functional module determines which fields in each table within the merged table chain can no longer be anonymized due to specific calculations. These specific calculations may include calculations in the table that could potentially leak data. For example, if a user wants to attack this anonymization system, they can deliberately design special calculations to extract the original data. For instance, if a user wants to obtain the original data of the anonymized ID card field, they can use another field besides the ID card field, perform an aggregation operation, and sum the ID card field data. The summation method is as follows:

[0098] SELECT colA,sum(ID)AS idNum FROM someTable GROUP BY colA.

[0099] If colA is a field that corresponds exactly to the content of an ID card, then the sum of the ID number fields (ID) will still result in the ID number itself. This would lead to data leakage. Therefore, this kind of calculation is what step S401 needs to detect and prohibit.

[0100] like Figure 3 As shown, specifically, the determination of the table link includes steps S401 to S403.

[0101] S401: Establish and / or update lineage relationships between worksheet fields.

[0102] It should be noted that when a worksheet is created, it may not immediately decide whether to configure data masking rules. Therefore, when the decision to configure data masking is made, this worksheet may have already been used to create multiple merged tables and charts, and some corresponding configuration operations may have occurred during this process. Thus, if data masking rules are configured on certain fields in an upstream table, the original data may be leaked because specific calculation rules have already been used downstream in the calculation chain for that field. Therefore, it is necessary to establish and / or update the lineage of worksheet fields to determine which fields may lead to the leakage of masked data.

[0103] S402: Based on blood relations, determine whether the fields in each table in the chain that constitutes the merged table may lead to the leakage of anonymized data.

[0104] The process involves determining whether fields in each table within the merged table's computational chain could potentially lead to data leakage due to masking issues. Specifically, based on lineage, all fields from third-party data sources accessing each table (the original worksheet) are traced downstream to obtain the computational chain for all these fields (the chain is a field-level computational chain), and the computational functions used at each step of the calculation in each table. Next, these computational functions are checked for any that could potentially leak masked data downstream. If a function is found that could lead to downstream data leakage, the entire computational chain containing that function is deemed to have the potential to leak masked data. For example, a field might use necessary data such as a user's phone number or ID card number for calculation, but a specific computational function might be used that could lead to data leakage. In this case, masking rules can no longer be configured for that computational chain.

[0105] S403: If there is a situation that may lead to the leakage of de-identified data, mark the field as prohibited from de-identification configuration, store it in the first table, and add the mark of prohibiting de-identification of the field to the configuration of all third tables that use the field in the first table.

[0106] Specifically, if a field in a merged table is identified as potentially causing data leakage due to masking issues based on the field's lineage, it indicates that the field has already undergone specific calculations in the upstream table. For such fields, not only will a flag prohibiting masking be added to the configuration of that field in the third table, but this flag will also be applied to the configurations of all tables using that field. In other words, these fields will be marked as prohibiting masking configuration in the first table's field configuration. In this case, users will no longer be able to configure masking rules for these fields to prevent data leakage caused by user configuration of masking rules.

[0107] In addition, when a user adjusts the desensitization configuration using a field, the desensitization configuration is inherited from the first table to all downstream tables that use the field, based on the calculated lineage relationship recorded in the second table. Adjusting the desensitization configuration includes at least one of the following adjustment methods: creating, modifying, and deleting the desensitization configuration.

[0108] Specifically, when a user adjusts the data masking configuration for a field in the third table—for example, when creating a new data masking configuration—the configuration for the third table being operated on by the current user is first created and then stored in the first table. Furthermore, based on the lineage records in the second table, the data masking rules in the first table are inherited by all downstream tables that use that field when a user creates a new data masking configuration for a field. In other words, a new data masking configuration record corresponding to the downstream third table is created in the first table, and this data masking configuration is stored in that record. When a user creates a table, if this user uses a field created by a previous user, and the second table records that the field uses that data masking configuration, the table created by this user will inherit the data masking configuration for that field. Based on the lineage, content in the table that has a data masking configuration corresponding to that field will be inherited.

[0109] S500: When creating the fourth table or modifying the third table, determine whether continuing the user's current operation will lead to the leakage of the original data on the masked fields.

[0110] The fourth table represents a combined table or chart created by the user, preferably a combined table.

[0111] S600: If present, operation is prohibited; otherwise, the creation of the fourth table or modification of the third table is permitted, and the desensitization configuration in the first table is inherited when creating the fourth table or modifying the third table.

[0112] Specifically, when a user attempts to create a fourth table or modify existing rules in a third table, module two checks whether the calculations performed during this creation or modification would lead to the leakage of the original data of the anonymized fields. The fourth table is used to distinguish it from the third table and for ease of description; the table name is not limited in this application. If any calculations would lead to the leakage of the original data of the anonymized fields, the creation or modification operation is prohibited; otherwise, it is allowed. Then, based on the analysis results, the third functional module inherits the anonymization configuration (if any) to the third table. Additionally, when creating a new fourth table, the anonymization configuration (if any) must be inherited to the fourth table.

[0113] Relatedly, in big data analytics tools, users often want specific users or user groups to view raw data. This necessitates upgrading the existing access control logic to display different anonymized results to different users or user groups. Based on this, access control needs to be implemented for generated worksheets, charts, and merged tables, controlling viewing and editing permissions. The granularity of permissions can be precise down to the individual worksheet or chart. For example... Figure 4 As shown, this application provides a data anonymization method based on lineage analysis, which also includes configuring the anonymization effect when a user views the table and configuring the permissions for editing the anonymization settings. Specifically, it includes:

[0114] S1000: If the first user imports data from the worksheet, such as configuring the data source and uploading, the user's permissions include: (1) having the permission to view the original data of this worksheet; (2) having the permission to edit the worksheet's desensitization configuration; and (3) being able to decide whether the authorized user has the aforementioned (1) and (2) permissions when this table is authorized to the second user.

[0115] S2000: When the first user authorizes the table he / she introduced to the second user, if he / she chooses not to authorize the permission to edit the masking configuration, the permissions that the second user does not have include: (1) He / she cannot edit this worksheet, as well as the masking configuration of all merged tables created based on this worksheet; (2) When the second user authorizes this table to others, or authorizes the merged tables created based on this worksheet to others, he / she cannot choose to allow other authorized users to edit the masking configuration.

[0116] S3000: Masking rules can only be modified (modification includes at least deletion) in the table where they were created, and cannot be modified in subsequent inherited tables. In other words, if you do not have permission to mask the table where it was created, then even if you have the permission to edit the masking rules of the subsequently inherited tables, you will not be able to edit or delete the inherited masking rules.

[0117] In this context, the first user is the user who created the worksheet, while the second and third users represent users other than the user who created the worksheet.

[0118] In addition, the worksheet created by the first user supports a whitelist mechanism. If the whitelist mechanism is required, a new table needs to be created in the database. The new table must have at least two fields: (1) the unique ID corresponding to the worksheet covered by the whitelist; and (2) the unique user ID of the whitelisted user for the new table. Users in the whitelist can view the original data of the tables configured in the whitelist and have the permission to edit the de-identification configuration of the tables configured in the whitelist.

[0119] The new table is stored in the database identical to the first and second tables, so that tables created in subsequent actions can be used as records in this new table. Users on the whitelist can view the original data of the tables configured for them in the whitelist and have permission to edit the data masking configurations for these tables. In other words, a record in the whitelist contains at least a worksheet ID and a user ID. When this user ID views the worksheet corresponding to the worksheet ID, it is unaffected by the data masking configuration of that worksheet and is unaffected by other permission controls; they have the permission to edit the worksheet's data masking configuration.

[0120] Steps S100 to S600 are performed within a big data analytics tool program, such as a BDP (Business Data Platform). The configuration of the first, second, and third tables within the big data analytics tool program involves converting these tables into Spark SQL statements, which are then provided to Spark for execution. After execution, Spark generates charts or merged tables that meet user requirements, such as a modified third table or a newly created fourth table.

[0121] Based on the ANTLR parser, Spark is parsed to obtain the data tables, fields, and computational logic used by Spark SQL. The data tables refer to worksheets or merged tables. Additionally, parsing Spark SQL retrieves the execution plan, which outlines how the actual data computations will be performed.

[0122] Specifically, the big data analytics tool program includes a first functional module, a second functional module, and a third functional module. The table merging configurations performed by the user on various interfaces within the first functional module are ultimately converted into Spark SQL statements, which are then executed by Spark. The second functional module parses the Spark SQL statements, analyzing whether masked fields are used, whether these masked fields involve inheritance, and whether any calculations could potentially leak the original data.

[0123] The second functional module uses ANTLR, which provides the ability to parse syntax trees. Using this ability to parse the execution plan generated by Spark for SQL, we can obtain the data tables and fields used by SQL, as well as the logic involved in the entire calculation.

[0124] like Figure 5 As shown, ANTLR's SQL parsing is specifically manifested as follows:

[0125] Input: SQL for calculation.

[0126] The output includes: the mapping between the upstream tables and fields that the SQL depends on and the fields of the final SQL output result, and a list of functions used in the upstream table fields during a series of calculations.

[0127] SQL is used as input to the Antlr parser. The Spark SQL syntax tree is constructed. By traversing the syntax tree, the project, relation and other tree nodes of the SQL are obtained, and then the source tables, fields and function calls on the fields are obtained. Through layer-by-layer parsing, the lineage relationship chain of field transformation is constructed.

[0128] The system will automatically analyze the lineage relationship chain constructed by Antlr to determine whether the final output field is related to the sensitive field in the source table. If there is a relationship, the desensitization rules will be applied.

[0129] The process of the data desensitization method based on bloodline analysis provided in this application is as follows:

[0130] When the system starts up:

[0131] 1. Scan all calculation paths for all fields in all upstream (initial) tables (first table and second table), and trace back to the last calculation step of each path, and record all calculation functions used in these paths.

[0132] 2. If a computational function is found that could potentially leak the original data in downstream de-identified data, the entire computational chain containing that function will be identified and recorded as not allowing the configuration of de-identification rules.

[0133] 3. Record and update all scanned calculated relationships (blood relations).

[0134] When a user wants to create a new combined table or chart (the third table), or modify an existing chart or combined table (the fourth table):

[0135] 1. Check if the user has used any calculation functions on fields that have been configured to be de-identified, which could lead to the leakage of de-identified data. If so, block the operation.

[0136] 2. Check if users have used calculation functions on un-masked fields that could lead to the leakage of masked data. If so, allow the operation and mark this field as prohibited from masking, as well as on all tables that may subsequently use this field in the future.

[0137] 3. After a user successfully creates / modifies a bloodline, update the bloodline relationships stored in the bloodline table according to the new calculation logic.

[0138] When a user wants to configure de-identification rules:

[0139] The execution steps are S1000, S2000, and S3000. A second aspect of this application provides a data desensitization system based on bloodline analysis, applied to the aforementioned data desensitization method based on bloodline analysis. The system includes:

[0140] The first creation unit is used to create a first table and a second table in the database. The first table represents the de-identification configuration and includes a field for identifying the unique ID of the work table and the merged table, a field for storing the user's unique ID, a field for determining whether de-identification is required, a field for editing the de-identification configuration, and a field for storing the de-identification configuration. The second table represents the lineage relationship between the fields in the storage table, including the unique ID of the source table, the unique ID of the source field, the unique ID of the result work table, and the unique ID of the result field.

[0141] The second creation unit is used to create the third table and store the third table in the system. The system configures the desensitization configuration for the third table, which includes a combined table and a chart.

[0142] A storage unit is used to store the de-identification configuration of the third table in the first table;

[0143] The scanning unit is used to scan the third table when the system starts, record the scan results in the second table, and record the fields in the third table that cannot be de-identified in the first table.

[0144] The first judgment unit is used to determine whether the user's current operation will lead to the leakage of the original data on the masked fields if the user continues to create the fourth table or modify the third table. The fourth table refers to the combined table or chart created by the user.

[0145] The second judgment unit is used to prohibit the operation if the condition is met, and to allow the creation of the fourth table or the modification of the third table if the condition is not met, and to inherit the desensitization configuration in the first table to the creation of the fourth table or the modification of the third table.

[0146] A third aspect of this application provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the aforementioned data desensitization method based on lineage analysis.

[0147] A fourth aspect of this application provides a computer storage medium storing a computer program thereon, characterized in that the computer program, when executed by a processor, implements the steps of the aforementioned data desensitization method based on lineage analysis.

[0148] In the description of the embodiments of the present invention, those skilled in the art should understand that the embodiments of the present invention can be implemented as methods, apparatuses, electronic devices, and computer-readable storage media. Therefore, the embodiments of the present invention can be specifically implemented in the following forms: entirely hardware, entirely software (including firmware, resident software, microcode, etc.), or a combination of hardware and software. Furthermore, in some embodiments, the embodiments of the present invention can also be implemented as a computer program product in one or more computer-readable storage media, the computer-readable storage media containing computer program code.

[0149] The aforementioned computer-readable storage medium may be any combination of one or more computer-readable storage media. Computer-readable storage media include: electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any combination thereof. More specific examples of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, optical fiber, optical disc read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any combination thereof. In embodiments of the present invention, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0150] The computer program code contained in the aforementioned computer-readable storage medium may be transmitted using any suitable medium, including wireless, wire, optical fiber, radio frequency (RF), or any suitable combination thereof.

[0151] Computer program code for performing the operations of the embodiments of the present invention can be written in assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, integrated circuit configuration data, or in one or more programming languages ​​or a combination thereof. Programming languages ​​include object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The computer program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer or an external computer via any type of network, including a local area network (LAN) or a wide area network (WAN).

[0152] The embodiments of the present invention describe the provided methods, apparatus, and electronic devices through flowcharts and / or block diagrams.

[0153] It should be understood that each block of a flowchart and / or block diagram, as well as combinations of blocks in a flowchart and / or block diagram, can be implemented by computer-readable program instructions. These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine that, when executed by a computer or other programmable data processing apparatus, creates means for implementing the functions / operations specified in the blocks of the flowchart and / or block diagram.

[0154] These computer-readable program instructions may also be stored in a computer-readable storage medium that enables a computer or other programmable data processing device to function in a particular manner. In this way, the instructions stored in the computer-readable storage medium produce an instruction apparatus product that includes the functions / operations specified in the blocks of a flowchart and / or block diagram.

[0155] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus or other device to produce a computer-implemented process, such that the instructions that execute on the computer or other programmable data processing apparatus provide a process for implementing the functions / operations specified in the blocks of the flowchart and / or block diagram.

[0156] In the specification and claims of this invention, the terms "first" and "second," etc., are used to distinguish different objects, not to describe a specific order of objects. For example, "first target object" and "second target object," etc., are used to distinguish different target objects, not to describe a specific order of target objects.

[0157] In embodiments of the present invention, the terms "exemplary" or "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or design described as "exemplary" or "for example" in embodiments of the present invention should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of the terms "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0158] In the description of the embodiments of the present invention, unless otherwise stated, "multiple" means two or more. For example, multiple processing units means two or more processing units; multiple systems means two or more systems.

[0159] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A data desensitization method based on kinship analysis, characterized in that, include: Create a first table and a second table in the database. The first table represents the de-identification configuration and includes a field for identifying the unique ID of the work table and the merged table, a field for storing the user's unique ID, a field for determining whether de-identification is required, a field for editing the de-identification configuration, and a field for storing the de-identification configuration. The second table represents the lineage relationship between the fields in the storage table, including calculating the unique ID of the source table, calculating the unique ID of the source field, calculating the unique ID of the result work table, and calculating the unique ID of the result field. A third table is created and stored in the system. An anonymization configuration is configured for the third table in the system. The third table includes a combined table and a chart. Store the de-identification configuration of the third table in the first table; When the system is started, the third table is scanned, and the scan results are recorded in the second table. Fields in the third table that cannot be de-identified are also recorded in the first table. When creating a fourth table or modifying a third table, determine whether continuing the user's current operation will lead to the leakage of the original data in the masked fields. The fourth table refers to a combined table or chart created by the user. If applicable, the operation is prohibited; otherwise, the creation of the fourth table or modification of the third table is allowed, and the de-identification configuration in the first table is inherited when creating the fourth table or modifying the third table.

2. The data desensitization method based on bloodline analysis according to claim 1, characterized in that, The steps of creating the first table and the second table in the database include: The field storing the user's unique ID indicates that each user's permission to access the first table is limited to one record in the first table. The field for editing the de-identification configuration indicates whether the user can edit the field for the de-identification configuration of the first table; The fields used to store the de-identification configuration are those used to store the de-identification configuration. The unique ID of the calculation source table represents the unique ID of the third table that serves as the data source in the calculation; The unique ID of the calculation result worksheet represents the unique ID of the fourth table generated as the result after the calculation source table has undergone the specified calculation.

3. The data desensitization method based on bloodline analysis according to claim 1, characterized in that, The steps of creating a third table, storing the third table in the system, and configuring de-identification settings for the third table in the system include: Create a de-identification configuration for the third table that conforms to the rules of the first table.

4. The data desensitization method based on bloodline analysis according to claim 1, characterized in that, The step of storing the de-identification configuration of the third table in the first table includes: If the third table is displayed, retrieve the de-identification configuration of the third table from the first table; Based on the anonymization configuration corresponding to the third table in the first table, the third table is displayed.

5. The data desensitization method based on bloodline analysis according to claim 1, characterized in that, The steps of scanning the third table and recording the scan results in the second table when the system is started, and recording fields in the third table that cannot be de-identified in the first table, include: Establish and / or update the lineage relationships between the fields in the worksheet; Based on blood relations, determine whether the fields in each table in the chain that constitutes the merged table will lead to data leakage due to de-identification; If a field exists that could lead to data leakage due to data masking, mark it as prohibited from being masked and store it in the first table. Also, add a flag prohibiting data masking for that field to the configuration of all fields in the third table that use that field in the first table.

6. The data desensitization method based on bloodline analysis according to claim 1, characterized in that, After the steps of scanning the third table and recording the scan results in the second table when starting the system, and recording the fields in the third table that cannot be de-identified in the first table, the system further includes: When a user adjusts the desensitization configuration corresponding to a field in the third table, the desensitization configuration is inherited from the first table to all downstream tables that use the field, based on the calculated lineage relationship recorded in the second table. The adjustment of the desensitization configuration includes at least one of the following adjustment methods: creating, modifying, and deleting the desensitization configuration.

7. The data desensitization method based on lineage analysis according to any one of claims 1-6, characterized in that, The first table, the second table, and the third table are configured in the big data analysis tool program. The first table, the second table, and the third table are converted into Spark SQL statements and provided to Spark for execution to generate charts or combined tables that meet the user's requirements. Based on the ANTLR parser, Spark is used to parse the data tables, fields, and computational logic used by Spark SQL. The data tables refer to worksheets or merged tables.

8. A data desensitization system based on lineage analysis, characterized in that, The data desensitization method based on bloodline analysis, applicable to any one of claims 1-7, comprises: The first creation unit is used to create a first table and a second table in the database. The first table represents the de-identification configuration and includes a field for identifying the unique ID of the work table and the merged table, a field for storing the user's unique ID, a field for determining whether de-identification is required, a field for editing the de-identification configuration, and a field for storing the de-identification configuration. The second table represents the lineage relationship between the fields in the storage table and includes calculating the unique ID of the source table, calculating the unique ID of the source field, calculating the unique ID of the result work table, and calculating the unique ID of the result field. The second creation unit is used to create a third table and store the third table in the system, and configure the desensitization configuration for the third table in the system, wherein the third table includes a combined table and a chart; A storage unit is used to store the de-identification configuration of the third table in the first table; The scanning unit is used to scan the third table when the system is started, record the scan results in the second table, and record the fields in the third table that cannot be de-identified in the first table. The first judgment unit is used to determine whether the user's current operation will lead to the leakage of the original data on the de-identified fields if the user continues to perform the operation when creating the fourth table or modifying the third table. The fourth table refers to the combined table or chart created by the user. The second judgment unit is used to prohibit the operation if the condition is met, and to allow the creation of the fourth table or the modification of the third table if the condition is not met, and to inherit the desensitization configuration in the first table to the creation of the fourth table or the modification of the third table.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the data desensitization method based on lineage analysis as described in any one of claims 1 to 7.

10. A computer storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the data desensitization method based on lineage analysis as described in any one of claims 1 to 7.

Images