Data protection method and device
By embedding data protection devices in the execution engine of big data databases and leveraging distributed processing capabilities, the problem of difficult data protection operations in big data environments is solved, achieving efficient and transparent data filtering and desensitization, reducing costs and improving security.
Patent Information
- Application Number
- CN201810411644.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2018-05-02
- Publication Date
- 2024-11-08
- Estimated Expiration
- 2039-02-24
AI Technical Summary
In the context of big data, existing technologies for data protection and operation are difficult to implement. They require specialized de-identification middleware, which increases the cost and complexity of hardware and software, resulting in high maintenance costs, high network bandwidth and computing resource consumption, poor performance, and an inability to meet the requirements for high-performance dynamic real-time de-identification. Furthermore, there are risks of data leakage and a large workload for system migration.
The big data database execution engine has a built-in data protection device. By obtaining the client's identity identifier and corresponding preset policies, it performs data filtering and de-identification operations. It uses distributed processing capabilities to achieve high-performance data protection, avoiding the need for external de-identification servers and keeping the user request logic and original data values unchanged.
It achieves high-performance data protection functions, reduces costs, improves efficiency, ensures data security, transparently and seamlessly protects sensitive data, meets the dynamic and real-time protection needs of big data, and reduces the risk of data leakage.
Smart Images

Figure CN110443059B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of communications, and in particular, to a data protection method and device. Background Art
[0002] In related technologies, data desensitization is a technology that uses desensitization rules to deform data for certain sensitive information, thereby achieving reliable protection of sensitive privacy data.
[0003] In the related art, the technologies, solutions and products for implementing database desensitization often perform desensitization through an external desensitization server or a desensitization proxy gateway by means of middleware conversion.
[0004] Figure 1 It is a flowchart of data protection through a desensitizing server in the relevant technology, such as Figure 1 As shown, the following steps are included:
[0005] Step 1: Send a query request. Specifically, the application sends the request to the data desensitization server;
[0006] Step 2: Rewrite the query request. Specifically, the data desensitization server receives and parses the data access request to determine whether the access involves sensitive data.
[0007] Step 3: Execute the query request. Specifically, the data desensitizing server modifies the data access request to obtain a desensitized data access request, and sends the modified desensitized data access request to the database;
[0008] Step 4: The database returns the query request. Specifically, the database obtains the data query result according to the desensitized data access request, and replaces the sensitive data in the query result data with preset information to obtain the desensitized data.
[0009] Step 5: The desensitizing server returns the query request. Specifically, the desensitizing server receives the desensitized data returned by the database and outputs the desensitized data to the application.
[0010] The above-mentioned related technical solutions have the following major defects and shortcomings:
[0011] ① Requires special desensitization middleware, that is, an external data desensitization server, which increases the cost and complexity of software and hardware, and the operation and maintenance cost is high;
[0012] ② Both query requests and result returns need to go through the desensitization middleware, which consumes a lot of network bandwidth and computing and storage resources, has poor performance, and low concurrency. Big data is generally a large data set with a large volume of data. Many practical applications have already formed a PB-level data volume. When the data volume is very large, the middleware method cannot meet the response time requirements of high-performance dynamic real-time desensitization of massive data in big data scenarios;
[0013] ③The result data returned by the database needs to be cached in the desensitizing server, which brings the security risk of data leakage;
[0014] ④ The relevant technology has changed the usage mode of the application system and cannot achieve complete transparency of the application. It is necessary to change the database access address and port, which changes the usage habits of the application users and introduces a lot of workload for the migration of the existing system.
[0015] There is currently no effective solution to the problem that it is difficult to implement data protection operations in a big data environment in related technologies. Summary of the invention
[0016] The embodiments of the present application provide a data protection method and device to at least solve the problem in the related art that it is difficult to implement data protection operations in a big data environment.
[0017] According to an embodiment of the present application, a data protection method is provided, including: a database execution engine receives data request information sent by a client; obtains a preset policy corresponding to an identity identifier of the client carried in the data request information, and the database execution engine performs a data protection operation on the requested data according to the preset policy to obtain target data.
[0018] According to another embodiment of the present application, a data protection device is provided, including: a receiving module for receiving data request information sent by a client; a data protection module for obtaining a preset policy corresponding to an identity of the client carried in the data request information, and the database execution engine performs a data protection operation on the requested data according to the preset policy to obtain target data.
[0019] According to another embodiment of the present application, a storage medium is provided, in which a computer program is stored, wherein the computer program is configured to execute the steps of any of the above method embodiments when running.
[0020] According to another embodiment of the present application, an electronic device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.
[0021] Through this application, the database execution engine receives the data request information sent by the client; obtains the preset policy corresponding to the client's identity carried in the data request information, and the database execution engine performs data protection operations on the requested data according to the preset policy to obtain the target data; the database execution engine sends the target data to the client. This solves the problem of difficulty in implementing data protection operations in a big data environment in related technologies. This application abandons the external desensitizing server solution and builds a data protection device into the big data database execution engine. Under the premise of not changing the user request logic and the original value of the data, it uses the distributed processing capabilities of the big data database engine itself to achieve high-performance data protection functions. BRIEF DESCRIPTION OF THE DRAWINGS
[0022] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:
[0023] Figure 1 It is a flowchart of data protection through a desensitizing server according to the relevant technology;
[0024] Figure 2 It is a module diagram of a big data sensitive data protection device according to an embodiment of the present application;
[0025] Figure 3 is a schematic diagram of supporting data protection functions of multiple big data database engines simultaneously according to another embodiment of the present application;
[0026] Figure 4 is a data protection strategy model diagram according to another embodiment of the present application;
[0027] Figure 5 is a schematic diagram of data protection policy configuration according to another embodiment of the present application;
[0028] Figure 6 is a desensitization algorithm management model diagram according to another embodiment of the present application;
[0029] Figure 7 This is a data protection flow chart of a SQL big data database engine according to another embodiment of the present application;
[0030] Figure 8 This is a data protection flow chart of a non-SQL big data database engine according to another embodiment of the present application;
[0031] Fig. 9 is a flowchart for setting a data protection policy implementation through a policy management interface according to another embodiment of the present application;
[0032] Fig.10 is a flowchart for setting a data protection policy implementation through an API interface according to another embodiment of the present application;
[0033] Fig.11 This is a flow chart of data protection during data import and export according to another embodiment of the present application. DETAILED DESCRIPTION
[0034] The present application will be described in detail below with reference to the accompanying drawings and in combination with embodiments. It should be noted that the embodiments and features in the embodiments of the present application can be combined with each other without conflict.
[0035] It should be noted that the terms "first", "second", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.
[0036] An embodiment of the present application provides a solution for a user device to obtain data from a database execution engine. The database execution engine is different from the related art and has a built-in data protection device. After receiving the data request information from the client, it performs data protection internally. The data protection may include data desensitization, data filtering, etc.
[0037] The present application embodiment can be run on Figure 2 In the network architecture shown, Figure 2 is a module diagram of a big data sensitive data protection device according to an embodiment of the present application, such as Figure 2 As shown, the network architecture includes at least three parts: a big data database client, a big data database execution engine, and a unified policy service. The specific description of the device module of each part can be found in the subsequent embodiments of this application.
[0038] In this embodiment, a data protection method running on the above network architecture is provided, and the process includes the following steps:
[0039] Step 1: The database execution engine receives the data request information sent by the client; this is equivalent to Figure 2 The big data database execution engine in ;
[0040] Step 2: obtaining a preset policy corresponding to the client identity carried in the data request information, and the database execution engine performs a data protection operation on the requested data according to the preset policy to obtain the target data;
[0041] Optionally, there is also a step three, in which the database execution engine sends the target data to the client.
[0042] Using the above steps, the database execution engine receives the data request information sent by the client; obtains the preset policy corresponding to the client's identity carried in the data request information, and the database execution engine performs data protection operations on the requested data according to the preset policy to obtain the target data; the database execution engine sends the target data to the client. The problem of difficulty in implementing data protection operations in a big data environment in related technologies is solved. This application abandons the external desensitizing server solution and builds a data protection device into the big data database execution engine. Under the premise of not changing the user request logic and the original value of the data, it uses the distributed processing capabilities of the big data database engine itself to achieve high-performance data protection functions.
[0043] Optionally, the database execution engine performs data protection operations on the requested data according to the preset policy, including: the database execution engine performs data filtering operations and / or data desensitization operations on the requested data according to the preset policy, wherein the preset policy includes filtering rules and / or desensitization rules.
[0044] Optionally, the database execution engine performs data filtering operations and / or data desensitizing operations on the requested data, including one of the following: performing a data desensitizing operation after performing a data filtering operation on the requested data; embedding a data filtering operation process in the process of performing a data desensitizing operation; performing a data filtering operation on the requested data; performing a data desensitizing operation on the requested data.
[0045] Optionally, when the database receives a structured query language SQL statement, the database execution engine performs the data desensitization operation on the requested data according to a preset strategy, including: generating an abstract syntax tree AST according to the SQL statement; deeply traversing the AST to obtain the table names and field names that the outermost multiple output fields of the SQL statement depend on layer by layer; and desensitizing the output fields corresponding to the field names in the outermost output fields of the SQL statement. The outermost layer of the SQL statement is also called the top layer.
[0046] The table name in this application document may also be an alias of the table, and the field name may also be an alias of the field.
[0047] Optionally, after obtaining the table names and field names on which the outermost multiple output fields depend layer by layer, the method further includes at least one of the following:
[0048] It is prohibited to desensitize the output fields in the clauses related to intermediate calculations in the SQL statement;
[0049] It is prohibited to mask the output fields in the Where and / or Limit conditional statements in the SQL statement.
[0050] The clauses related to intermediate calculations in the above SQL statements include: GROUP BY, ORDER BY, SORT BY, DISTRBUTE BY, JOIN, UNION, LIKE, BETWEEN, function and other clauses.
[0051] Optionally, the derived table in the select clause of the associated derived table in the SQL statement is data-massaged by one of the following methods: obtaining the table and the preset masking rule corresponding to the field name on which the output field name of the select clause query of the associated derived table depends, using the preset masking rule as the masking rule of the output field, and performing data masking on the derived table according to the masking rule; after the database executes the select clause of the associated derived table, the derived table is a derived table after data masking according to the preset masking rule. The select clause includes clauses such as create table as select or insert into select.
[0052] Optionally, after the database receives the structured query language SQL statement, it is determined that the current SQL statement has not been rewritten, and the AST tree is deeply traversed to obtain at least one of the following information:
[0053] Obtaining the internal structure of the SQL statement, generating a dependency table of the output fields queried by the SQL statement, wherein the dependency table records the tables and fields that each output field depends on layer by layer in the form of a tree;
[0054] Get the name of each outermost output field of the select statement query in the SQL statement, and generate the outermost select query output field list;
[0055] Get the names of all newly created tables and inserted tables in the SQL statement. All newly created tables or inserted tables in the SQL statement can be tables in the create table as select or insert into select clauses.
[0056] Optionally, the following steps are performed for each output field in the outermost output field list queried by the select statement: select the first output field from the outermost output field list in a preset order; deeply traverse the dependency table to obtain the table name and field name of the deepest dependency corresponding to the first output field name; obtain the desensitization rules preset for the table name and field name of the deepest dependency; after determining that the SQL statement is a select statement, process the partial SQL statement corresponding to the first output field according to the desensitization rules to form a first SQL statement, and use the first SQL statement to rewrite the original value of the AST node corresponding to the first output field.
[0057] Optionally, the desensitization rules of the tables and fields of the deepest dependency are obtained, and after determining that the SQL statement is a createtable as select or insert into select statement, the corresponding desensitization rules are inserted into all newly created tables and current output fields in the inserted table of the database. The deepest dependency in the present application document may include, in the multi-layer correspondence between a certain outermost output field and a table, excluding the correspondence between the middle layers, the table and field corresponding to the outermost output field at the deepest level.
[0058] Optionally, after rewriting the AST node value corresponding to each output field in the outermost output field list, traverse the rewritten AST to resynthesize the second SQL statement, and determine that the second SQL statement is the SQL statement that is finally executed.
[0059] Optionally, after traversing the rewritten AST to resynthesize the second SQL statement and determining that the second SQL statement is the final SQL statement to be executed, the database determines a physical plan to be executed based on the final SQL statement to be executed; executes the physical plan, and feeds back the processing result to the client.
[0060] Optionally, when the database receives a structured query language SQL statement, the database execution engine performs the data filtering operation on the requested data including:
[0061] Generate an abstract syntax tree AST based on the SQL statement;
[0062] Traversing the AST executes the following steps:
[0063] When a leaf node in the AST is a physical table, obtain a table filtering rule of the physical table;
[0064] In the AST, a subquery node is added, wherein the original parent node of the leaf node is the parent node of the subquery node, and the leaf node is the child node of the subquery node;
[0065] Apply the table filtering rule to the subquery node;
[0066] After traversing the AST and executing the above steps, a reconstructed AST is obtained, a reconstructed SQL statement is obtained according to the reconstructed AST, and the reconstructed SQL statement is executed.
[0067] Optionally, when the database receives a non-SQL access request, the requested data is protected according to a preset strategy, including: the database obtains the data requested by the non-SQL access request; obtains the preset strategy corresponding to the data, and executes at least one of the following for each data set corresponding to the data according to the preset strategy: filters the data according to the filtering rules included in the preset strategy; parses the data, calls the desensitization rules corresponding to each column one by one to desensitize each column, and reassembles the columns after the desensitization processing according to the original data format to obtain the desensitized data.
[0068] Optionally, the database is connected to a policy device, wherein the policy device is allowed to provide a policy model for one or more databases, and the policy device establishes a policy model based on at least one of the following: identity identification, resource entity, protection rules, wherein the protection rules include desensitization rules and / or filtering rules.
[0069] Optionally, the policy device is externally connected to an API interface for receiving input signals for configuring the policy model.
[0070] Optionally, the policy device is also used to store a desensitization rule model, wherein the policy device determines the desensitization rule model of the sensitive data domain based on at least one of the following: a data desensitization operator, a data desensitization rule; wherein the data desensitization operator includes a basic function for implementing data desensitization, and the data desensitization rule includes a rule for performing data desensitization based on the data desensitization operator.
[0071] Another embodiment of the present application proposes a method and device for protecting sensitive data of big data to solve the problems existing in related technologies, such as high deployment cost, imperfect security mechanism, lack of complete transparency to users, poor performance, and inability to meet the requirements of dynamic real-time protection of big data.
[0072] The embodiment of the present application adopts the following technical solution: the present application realizes high-performance data desensitization and data filtering functions by building a data protection device into the execution engine of the big data database without changing the user request logic and the original value of the data and utilizing the distributed processing capability of the big data database engine itself. The present application abandons the external desensitization server solution and can protect the privacy data of big data efficiently and in real time; at the same time, it is completely transparent to the application and user use, and the user can be completely unaware of the data protection process, thus realizing seamless and transparent protection of sensitive data.
[0073] The device for protecting sensitive data of big data described in the embodiment of the present application is composed of the following parts, which can correspond to Figure 2 The network architecture in:
[0074] 1. Big data database client: The client can be an application, program, command or script that accesses big data services. It directly initiates access requests to the big data database execution engine through the network and receives processing results.
[0075] In particular, the big data database client will carry the user name that currently submits the access request as the visitor's identity in the request by default.
[0076] The client can realize this by adopting relevant existing technologies.
[0077] 2. Big Data Database Execution Engine: The Big Data Database Execution Engine is responsible for completing the business logic processing of client requests;
[0078] The big data database execution engine includes a request parsing module, an execution module, a result processing module and a data protection device;
[0079] The present application adds a data protection device to the big data database execution engine. The device includes a data protection module and a policy agent module to implement the data protection function of the big data database engine.
[0080] The data protection module is built into the big data database engine and is responsible for completing real-time data filtering and / or data desensitization functions.
[0081] The policy agent module is responsible for providing a protection policy query interface for the data protection module. The policy agent module is deployed together with the big data database engine as a pluggable plug-in, and the big data database engine is responsible for loading and starting it.
[0082] 3. Unified policy service: Provide unified data protection policy service for multiple big data database engines at the same time, which includes a policy database, a policy management module, and an algorithm management module. These three modules constitute the policy device of this application.
[0083] The policy database is responsible for storing configuration data related to data protection policies and algorithms.
[0084] The policy management module has a built-in unified big data protection policy model, which abstracts access users, protected resource entities, and sensitive data protection rules. The policy model consists of three parts:
[0085] ① Identity: including user name, user group name, role name, etc. The same user can belong to different user groups or different roles. The identity represents the object to be managed;
[0086] ② Resource entity: represents the big data resources needed, and is described using a hierarchical model of service name-database-table-column. It can manage the resources of multiple different big data components at the same time;
[0087] ③ Protection rules: including column desensitization rules and row filtering rules, which specifically implement the conversion algorithm for desensitization and data filtering.
[0088] There is a many-to-many relationship between identity identifiers and resource entities, and there is also a many-to-many relationship between resource entities and protection rules. Desensitization rules and filtering rules can be used separately or in combination.
[0089] Furthermore, based on the model, the policy management module provides a set of policy configuration-related API interfaces for secondary development. It also provides a management interface through which system administrators or compliance administrators can configure and manage data protection policies for various big data engine components.
[0090] The algorithm management module includes a unified framework for the development, deployment, and management of big data masking operators and masking rules. It can manage built-in and third-party developed masking operators in a unified manner, and provides an algorithm management API interface and management interface.
[0091] In order to uniformly manage the desensitization algorithms of various big data database engines, the present application embodiment proposes a big data desensitization algorithm management model, which includes three parts:
[0092] ① Data desensitization operator: The desensitization operator is the function that actually performs the data desensitization conversion, and implements various desensitization algorithms and functions, including but not limited to replacement, randomization, encryption, etc. The data desensitization operator resides in the big data database execution engine in the form of an algorithm library;
[0093] ② Data desensitization rules: Based on the capabilities of the data desensitization operator, data desensitization rules that meet the compliance requirements are generated according to the compliance requirements of the industry and are used to transform data of specific data types;
[0094] ③Sensitive data domain: Describe the functional meaning of the column based on the column data or column name, and classify sensitive data. For example, "ID number" is a type of sensitive data. In this category, all ID number data fields in the database are managed, and unified desensitization rules can be implemented. Each sensitive data domain has a default desensitization rule.
[0095] The same desensitizing operator can create multiple different desensitizing rules. The same desensitizing rule can be distributed in different sensitive data domains. A sensitive data domain includes multiple desensitizing rules.
[0096] The embodiment of the present application abandons the data desensitizing server, and the big data database client directly initiates a data access request to the big data database execution engine, and the user name is carried in the request as the user's identity identification; when the big data database execution engine receives the access request from the client, the request parsing module parses it, and the data protection module analyzes the content of the request access based on the user identification information built into the current request session and combines the policy data obtained from the policy agent to determine whether the request needs to perform data filtering and data desensitization. If data filtering needs to be performed, the data is screened according to the data filtering conditions in the policy. If data desensitization needs to be performed, the data is converted according to the corresponding desensitization rules in the desensitization policy. The data filtering and desensitization conversion process is completed in parallel by the execution module using the distributed computing power of the cluster, and the filtered and desensitized results are directly returned to the client by the result processing module.
[0097] For SQL-type big data database engines (including but not limited to SparkSQL / Impala / Hive / Kylin, etc.), the data desensitization method of the embodiment of the present application will not destroy the business logic of the original SQL access request. The present application deeply traverses and parses the abstract syntax tree generated after parsing the SQL statement to find the dependencies between the tables and fields involved in the SQL statement, and finally only desensitizes the sensitive fields in the outermost output field of the select query; the embodiment of the present application does not desensitize sensitive fields appearing in SQL statements such as GROUP BY / ORDER BY / SORT BY / DISTRBUTE BY / JOIN / UNION / LIKE / BETWEN / function clauses, and does not desensitize sensitive fields appearing in WHERE and or limit conditions.
[0098] In particular, in the case where there are sensitive fields in the select clauses of create table as select and insert into select and desensitization rules are set for the fields, the embodiments of the present application provide two processing modes to automatically desensitize and convert such SQL to protect the sensitive data of the derived table.
[0099] There are two processing modes for derived table desensitization:
[0100] Mode 1: Derived mode, the select clauses of create table as select and insert into select query the table field that the output field name depends on as the desensitization rule of the output field, insert the desensitization rule into the protection policy library, and when querying the derived table, perform desensitization conversion according to the inserted desensitization rule. This mode does not modify the original data of the derived table and is the default processing mode of the embodiment of this application;
[0101] Mode 2: Rewrite mode, rewrite the select clause of create table as select and insert into select. After executing create table as select and insert into select, the data in the derived table is anonymized data.
[0102] The data filtering described in the embodiment of the present application is achieved by deeply traversing and analyzing the AST nodes generated by the from clause of the abstract syntax tree. If the current AST node is a leaf node and a physical table, the policy proxy interface is called to obtain the table filtering rules. If the filtering rules for the table exist, the AST tree is reconstructed, and a sub-query node is added. The original parent node of the leaf node becomes the parent node of the sub-query node, and the leaf node is the child node of the sub-query node. The obtained filtering rules are applied to the sub-query node to complete the reconstruction of the AST tree.
[0103] The data protection process of the embodiment of the present application can first execute the data filtering process to generate a reconstructed AST, and then execute the data desensitization process based on this AST; the data filtering process can also be embedded in the data desensitization process. Regardless of the method used, the final execution result of the data protection process is the same.
[0104] For SQL-based big data database engines, the real-time protection process of sensitive data described in the embodiment of this application is as follows:
[0105] Step 1: The client directly initiates an SQL access request to the big data database execution engine through the network, and the request carries the user name as the user's identity;
[0106] Step 2: The SQL parser of the big data database engine performs lexical analysis on the SQL statement to obtain a lexical analysis general symbol stream, and then performs syntax analysis on the symbol stream to construct a syntax tree;
[0107] Step 3: The SQL analyzer of the big data database engine performs in-depth analysis on the unparsed relations in the syntax tree to obtain the abstract syntax tree AST;
[0108] Step 4: The AST parsing module of the big data database engine reversely parses the AST syntax tree to obtain a decomposable object tree structure;
[0109] Step 5: The AST reconstruction module of the big data database engine reconstructs the AST, further abstracts and structures the SQL, and generates a reconstructed AST tree;
[0110] Step 6: The big data database engine determines whether the data protection function switch of the engine has been turned on and the current SQL statement has not been rewritten. If both of the above conditions are met, step 7 enters the data protection process, otherwise, step 9 is executed;
[0111] Step 7: Enter the data protection process, through deep traversal and analysis of the AST tree, rewrite the SQL statement involving sensitive fields and reconstruct the AST tree. Step 7 includes the following 11 steps, as follows:
[0112] Step 7.1: Obtain the SQL internal structure by deeply traversing the AST tree and generate a dependency table of the query output fields. The dependency table records the tables and fields that each query output field depends on layer by layer in the form of a tree.
[0113] In the process of traversing and parsing the AST nodes generated by the from clause, if the current AST node is a leaf node and is a physical table, call the policy proxy interface to obtain the table filtering rules. If there are filtering rules for the table, reconstruct the AST tree and add a subquery node. The original parent node of the leaf node becomes the parent node of the subquery node, and the leaf node becomes the child node of the subquery node. Apply the obtained filtering rules to the subquery node to complete the reconstruction of the AST tree.
[0114] Step 7.2: Obtain the name of each output field of the outermost layer of the select query by deeply traversing the AST tree, and generate the output field list of the outermost layer of the select query;
[0115] In particular, when the output field of the outermost query is *, it means that the output fields are all the output fields of the next layer of the dependency table, and the next layer of fields are used as the final output fields of the outermost query, and so on;
[0116] Step 7.3: Obtain all newly created tables or inserted table names (or aliases) in the create table or insert clause by deeply traversing the AST tree;
[0117] In particular: The purpose of step 7.1 to step 7.3 is to deeply analyze the AST nodes to obtain the internal structure of SQL. The execution order of step 7.1, step 7.2, and step 7.3 can be adjusted. No matter which step is executed first, the generated SQL internal structure result is the same.
[0118] Step 7.4: Get an output field name (or alias) from the outermost output field list of the select query in order;
[0119] Step 7.5: Deeply traverse the dependency table to find the final dependent table (or table alias) and field name (or field alias) corresponding to the current output field name;
[0120] Step 7.6: Call the policy proxy module interface (with the user name) to obtain the desensitization rules of the table and field that the current output field ultimately depends on;
[0121] Step 7.7: If the desensitization rule is not obtained, jump to step 7.10;
[0122] If the current SQL statement is a select statement, jump to step 7.8 to rewrite the SQL;
[0123] If the current statement is a create table / insert statement, obtain the derived table desensitization mode configured for the big data database engine;
[0124] If it is in rewrite mode, execute step 7.8 to rewrite the SQL. If it is in derive mode, execute step 7.9 to insert a new desensitization policy.
[0125] Step 7.8: Rewrite the value of the AST node (a SQL statement) corresponding to the current output field, and replace the original value of the AST node with the new SQL statement spelled using the desensitization rule;
[0126] Step 7.9: Call the policy proxy interface to insert the desensitization policy corresponding to the current output field into the data protection policy database. The resource table name corresponding to the desensitization policy is the table name (or alias) obtained in step 7.3, and the desensitization rule is the desensitization rule obtained in step 7.6. If there are multiple tables, the corresponding desensitization policy is inserted into each table;
[0127] Step 7.10: Determine whether all the outermost query output fields of the select have been processed. If not, process the remaining output fields one by one according to steps 7.4-7.9;
[0128] Step 7.11: If the SQL is rewritten, traverse the rewritten abstract syntax tree, resynthesize the abstract syntax tree into a new SQL statement, generate the final executed SQL statement, and set the SQL rewrite mark;
[0129] In particular, the SQL rewrite mark is a control variable parameter described in the embodiment of the present application.
[0130] Step 8: The big data database execution engine determines whether the data protection switch is turned on and the SQL has been rewritten. If the above two conditions are met at the same time, execute step 2 to submit the rewritten SQL statement to the SQL parsing interface to re-analyze the SQL statement; otherwise, execute step 9 to continue the subsequent process;
[0131] Step 9: The SQL optimizer of the big data database engine optimizes most of the logic layer by transforming the manipulation tree, merging operators, reducing the number of jobs, and improving performance;
[0132] Step 10: The executable physical plan module of the big data database engine generates an executable physical plan and generates an executable big data job;
[0133] Step 11: The execution module of the big data database engine submits the executable big data job to the distributed execution environment of the big data database engine for distributed execution;
[0134] Step 12: After execution, the big data database engine result processing module will return the result set data directly to the client.
[0135] In particular, steps 1 to 5 and steps 9 to 12 are existing functions and processes of the big data database engine, steps 6 to 8 are newly added data protection processes in the embodiment of the present application, and the rewritten SQL statement executed in step 11 is executed concurrently on each distributed processing node.
[0136] In particular, the data filtering process described in the embodiment of the present application can be integrated into the data desensitization process, and the data filtering process can be embedded when analyzing the from clause in step 7.1. At the same time, it can also be further adjusted by not executing the data filtering process in step 7.1, adding data filtering process processing before step 7.11, generating a reconstructed AST, and the subsequent process remains unchanged.
[0137] Or the data filtering process and the data desensitization process are called successively as two independent sub-processes of data protection:
[0138] First, execute the data filtering process, insert the data filtering process before the original step 7.1, generate the reconstructed AST, and then execute the original steps 7.1-7.11 in sequence (the data filtering process in the original step 7.1 is no longer called).
[0139] Regardless of the execution order, the final execution result is the same.
[0140] The data protection process described in the embodiment of this application for non-SQL big data database engines (including but not limited to KUDU, etc.) is as follows:
[0141] Step 1: The client application directly initiates a data request to the big data database engine server through the network through commands, scripts, applications, interface calls, etc. The client will carry the visitor's own user name information in the request message;
[0142] Step 2: The server accepts the client's access request and parses it;
[0143] Step 3: According to the distributed scheduling framework of the big data database engine, the request scheduling is distributed to each processing node to execute specific logic;
[0144] Step 4: The processing node receives the data processing request, queries the data that meets the requirements according to the request business logic, and obtains the query result set;
[0145] In particular, the present application performs data filtering and desensitization functions on the result set at each distributed processing node after the result set is obtained;
[0146] Step 5: The processing node determines whether the data protection function of the big data database engine is turned on. If it is turned on, it goes to step 6, otherwise it goes to step 9;
[0147] Step 6: Filter and desensitize the query result data set obtained in step 4;
[0148] Step 6.1: Call the policy proxy interface to obtain the data protection policy of the current access user;
[0149] Step 6.2: Based on the acquired data desensitization strategy and the resource entity information of the user's access request, determine whether the current requested access data set needs to be desensitized;
[0150] Step 6.3: Get a piece of result data from the result set in order;
[0151] Step 6.4: According to the data filtering rules, determine whether the record meets the filtering conditions. If it does not meet the filtering conditions, execute step 6.5, otherwise execute step 6.7;
[0152] Step 6.5: According to the judgment result of step 6.2, determine whether the current record needs to perform data desensitization. If desensitization is required, execute step 6.6; otherwise, execute step 6.7;
[0153] Step 6.6: parse the current result data, find the columns (possibly multiple) that need to be desensitized, call the corresponding data desensitization rules one by one to perform desensitization conversion, and reassemble the converted results into desensitized result data according to the original data format;
[0154] Step 6.7: Determine whether all current data sets have been processed. If not, execute step 6.3 until all the obtained result set data has been filtered and anonymized;
[0155] Step 7: The processing node encapsulates the result data and returns the result directly to the client;
[0156] Step 8: The client directly obtains the return results from each distributed processing node and summarizes them.
[0157] In particular, steps 1 to 4 and steps 7 to 8 are existing functions and processes of the big data database engine in the related technology, step 6 is the functional process of the data protection module, and steps 4 to 7 are executed in parallel on each processing node.
[0158] The following is further explained in conjunction with specific embodiments of the present application.
[0159] like Figure 2 As shown, the device for protecting sensitive data of big data described in the embodiment of the present application is composed of the following parts:
[0160] 1. Big data database client: The client is an application, program, command or script that accesses big data services. It initiates access requests to the big data database execution engine through the network and receives processing results. By default, the request of the big data database client carries the user name as the user's identity. The client uses existing technology and does not need to make any changes.
[0161] 2. Big Data Database Execution Engine: The Big Data Database Execution Engine is responsible for completing the business logic processing of client requests;
[0162] The big data database execution engine includes a request parsing module, an execution module, a result processing module and a data protection device;
[0163] The present application adds a data protection device to the big data database execution engine. The device includes a data protection module and a policy agent module to implement the data protection function of the big data database engine.
[0164] The data protection module is built into the big data database engine and is responsible for completing real-time data filtering and / or data desensitization functions.
[0165] The policy agent module is responsible for providing a protection policy query interface for the data protection module. The policy agent module is deployed together with the big data database engine as a pluggable plug-in, which is loaded and started by the big data database engine. The plug-in method realizes the separation of policy management and execution logic and facilitates function expansion.
[0166] The policy proxy module obtains the data protection policy configured by the execution engine from the policy management module of the present application and caches it in the local memory, thereby obtaining the policy and providing a policy interface to the outside world. In this way, the policy management module of the present application can simultaneously manage the data protection policies of multiple big data database engines, which is the default implementation of the policy proxy module plug-in.
[0167] When the big data database execution engine receives an access request from the client, it is parsed by the request parsing module. The data protection module analyzes the content of the request based on the user identification information built into the request session and the policy data obtained from the policy agent to determine whether the request needs to perform data filtering and data desensitization. If data filtering is required, the data is screened according to the data filtering conditions in the policy. If data desensitization is required, the data is converted according to the corresponding desensitization rules in the desensitization policy. The data filtering and desensitization conversion process is completed in parallel by the execution module using the distributed computing power of the cluster, and the filtered and desensitized results are returned directly to the client by the result processing module.
[0168] 3. Unified policy service: Provide unified data protection policy service for each big data database engine, which includes a policy database, a policy management module, and an algorithm management module. These three modules constitute the policy device of this application.
[0169] The policy database is responsible for uniformly storing configuration data related to the policy management and algorithm management modules.
[0170] The policy management module has a built-in unified big data protection policy model, which abstracts services, resources and access entities, creates a resource model that can meet the needs of various components of big data, and can be flexibly expanded to support new big data database engine filtering and desensitization policy management; further, based on the model, a set of policy configuration-related API interfaces are provided to the outside world for secondary development, and a management interface is also provided. System administrators or compliance administrators can configure and manage data protection policies for various components of big data through the interface.
[0171] The algorithm management module includes a unified framework for the development, deployment, and management of big data masking operators and masking rules. It can manage built-in and third-party developed masking operators in a unified manner, and provides an API interface and management interface for algorithm management.
[0172] Figure 3 is a schematic diagram of supporting data protection functions of multiple big data database engines simultaneously according to another embodiment of the present application, such as Figure 3 As shown, the method of the present application can support various big data database engines to implement data protection functions. The policy device provides a unified protection strategy, and a set of data protection devices is built in each big data database engine (including but not limited to Hive, SparkSQL, Kylin, etc.), and data protection can be implemented according to the method described in the embodiment of the present application. The policy device can include policy management, algorithm management, and policy database.
[0173] In view of the inconsistency of resource models of different big data database engines, this application has built a set of resource policy models in the policy management module. Figure 4 is a data protection strategy model diagram according to another embodiment of the present application, such as Figure 4 As shown in the figure, the model abstracts access users, protected resource entities, and sensitive data protection rules. The policy model consists of three parts:
[0174] ① Identity: including user name, user group name, role name, etc. The same user can belong to different user groups or different roles. The identity represents the object to be managed;
[0175] ② Resource entity: represents the big data resources needed, and is described using a hierarchical model of service name-database-table-column. It can manage the resources of multiple different big data components at the same time;
[0176] ③ Protection rules: including column desensitization rules and row filtering rules, and specifically implement the conversion algorithm for desensitization and data filtering;
[0177] There is a many-to-many relationship between identity identifiers and resource entities, and there is also a many-to-many relationship between resource entities and protection rules. Desensitization rules and filtering rules can be used separately or in combination.
[0178] After the policy model is instantiated, as shown in Table 1, Table 1 is a schematic table according to specific embodiment 2 of the present application.
[0179] Table 1
[0180]
[0181] There is a many-to-many relationship between identity identifiers and resource entities, and there is also a many-to-many relationship between resource entities and processing rules.
[0182] For example:
[0183] User Zhang San is a low-privilege user. The full masking rule Mask_Last_64 is used for the F1 field of the T1 table in the DB1 database of the SparkSQL service.
[0184] User Li Si is a high-authority user. For the F1 field of the T1 table in the DB1 database of the SparkSQL service, the masking rule Mask_Last_8 is used to mask the last 8 bits.
[0185] The user with the role of director is a high-authority user. For the F1 field of the T1 table in the DB1 database of the SparkSQL service, the masking rule Mask_Last_2 with two digits after masking is used;
[0186] Through the model of this application, various fine-grained desensitization policy management can be supported, and different desensitization rules can be configured for different access entity types such as users / user groups / roles, etc.
[0187] In addition, flexible data filtering rules can be implemented for different users, user groups, and roles:
[0188] User Zhang San belongs to project group 1 and can only obtain records belonging to project group 1 in table T1 in database DB1 of the SparkSQL service;
[0189] User Li Si belongs to project group 1 and can only obtain records belonging to project group 1 in table T1 in database DB1 of the SparkSQL service;
[0190] Column desensitization rules and row filtering rules can be used in combination flexibly. You can set column desensitization rules or row filtering rules separately for different identity identifiers, or set column desensitization rules and row filtering rules at the same time.
[0191] For example, user Zhang San is a low-privilege user. For the F1 field of the T1 table in the DB1 database of the SparkSQL service, the full desensitization rule is adopted. At the same time, user Zhang San is a low-privilege user. For the F1 field of the T1 table in the DB1 database of the SparkSQL service, the full desensitization rule is adopted. When Zhang San queries the F1 field of the T1 table in the DB1 database of the SparkSQL service, only the record set of project group 1 will be output, and the F1 field will be fully desensitized.
[0192] As described above, the policy model of the present application can flexibly implement the data protection policy configuration function of providing fine-grained control for multiple big data database engines.
[0193] Schematic diagram of the interface for configuring a single protection policy for a big data service. Figure 5 is a schematic diagram of data protection policy configuration according to another embodiment of the present application, such as Figure 5 As shown in the figure, the specific steps of configuring the policy include:
[0194] Step 1: The compliance administrator logs in to the system and enters the data protection policy configuration interface corresponding to the big data database engine;
[0195] 1. Big data service name: represents the service name of the big data database engine for which the protection policy is to be configured, such as SparkSQL, Hive, etc.
[0196] 2. Identity: including user name, user group name, role name, etc. The same user can belong to different user groups or different roles. The identity represents the object to be managed;
[0197] 3. Resource entity: represents the big data resources needed, and is described in a hierarchical model of service name-database-table-column. For Hbase, the database is the Hbase Namespace, and the column is represented by the column family: qualifier. Other databases such as Hive and SparkSql can correspond to the database-table-column hierarchical model. Resource entities support fuzzy query, wildcard and other configuration methods.
[0198] 4. Protection rules: including column desensitization rules and row filtering rules, which specifically implement the conversion algorithm for desensitization and data filtering.
[0199] There is a many-to-many relationship between identity identifiers and resource entities, and there is also a many-to-many relationship between resource entities and processing rules. Desensitization rules and filtering rules can be set at the same time.
[0200] Step 2: First, select the resource entity as the protection object, select or fill in the database, table, and column name, and assign the column to the corresponding sensitive data domain. You can select linkage, for example, after entering the database name, the table of the database can be automatically associated, or you can fill in a wildcard, and the table that meets the wildcard condition will be protected according to the unified rules;
[0201] Step 3: Select the user / user group / role etc. that has limited access to the protected object;
[0202] Step 4: Select the sensitive field column and select the corresponding desensitization rule, such as random, replacement, offset, etc. If you do not set a desensitization rule, the default desensitization rule of the sensitive data domain is used;
[0203] Step 5: Set the corresponding table data filtering strategy;
[0204] Step 6: After saving the data protection policy, the policy is saved to the policy database, and the single policy configuration is completed.
[0205] In order to uniformly manage the desensitization algorithms of various big data database engines, this application proposes a desensitization algorithm management model, which includes three parts: Figure 6 is a desensitization algorithm management model diagram according to another embodiment of the present application, such as Figure 6 As shown, the description of sensitive data domain, desensitization rules, and desensitization operators are as follows:
[0206] ① Data desensitization operator: The desensitization operator is the function that actually performs the data desensitization conversion, and implements various desensitization algorithms and functions, including but not limited to replacement, randomization, encryption, etc. The data desensitization operator resides in the big data database execution engine in the form of an algorithm library;
[0207] ② Data desensitization rules: Based on the capabilities of the data desensitization operator, data desensitization rules that meet the compliance requirements are generated according to the compliance requirements of the industry and are used to transform data of specific data types;
[0208] ③Sensitive data domain: Describe the functional meaning of the column based on the column data or column name, and classify sensitive data. For example, "ID number" is a type of sensitive data. In this category, all ID number data fields in the database are managed, and unified desensitization rules can be implemented. Each sensitive data domain has a default desensitization rule.
[0209] The same desensitizing operator can create multiple different desensitizing rules. The same desensitizing rule can be distributed in different sensitive data domains. A sensitive data domain includes multiple desensitizing rules.
[0210] Table 2 is a list of desensitization rule names, desensitization operators, and desensitization operator actual parameter parameters of specific embodiment 4, as shown in Table 2:
[0211] Table 2
[0212] Desensitization rule name Desensitization operator Desensitizing operator actual parameter list Mask_Last_64 Mask_Last_N col,64,'*' Mask_Last_8 Mask_Last_N col,8,'*' Mask_Last_2 Mask_Last_N col,2,'*'
[0213] As shown in Table 2, the same masking operator Mask_Last_N generates three different masking rules, and different masking rules have different actual parameters for the operator: Mask_Last_64, Mask_Last_8, and Mask_Last_2. Among them, the parameter of Mask_Last_64 is col,64, and '*' masks the last 64 characters of the current column col with '*'; the parameter of Mask_Last_8 is col,8, and '*' masks the last 8 characters of the current column col with '*'; the parameter of Mask_Last_2 is col,2, and '*' masks the last 2 characters of the current column col with '*'. In this way, different masking rules can be generated based on the same operator, and these rules can be flexibly applied to different user entities to generate different masked data.
[0214] Each sensitive data field has a default desensitization rule. Table 3 is a correspondence table between sensitive data fields and desensitization rules in specific embodiment 4, as shown in Table 3:
[0215] Table 3
[0216] Sensitive data domain name Default desensitization rule name ID card field IDHidingUDF_Full Bank card domain CARDMaskHidingUDF_First_2 Name field ChinaNameMaskHidingUDF_First_1
[0217] This type of sensitive data uses the masking rules of the sensitive data domain by default to reduce configuration workload. If the default rules do not meet the actual masking requirements of the field, you can select a suitable rule from the masking rule list.
[0218] For SQL-type big data database engines (including but not limited to SparkSQL / Impala / Hive / Kylin, etc.), the data desensitization method of the present application will not destroy the business logic of the original SQL access request. The present application deeply traverses and parses the abstract syntax tree generated after parsing the SQL statement to find the dependencies between the tables and fields involved in the SQL statement, and finally only desensitizes the sensitive fields in the outermost output field of the select query; the present application does not desensitize sensitive fields appearing in SQL statements such as GROUP BY / ORDER BY / SORT BY / DISTRBUTE BY / JOIN / UNION / LIKE / BETWEN / function clauses, and does not desensitize sensitive fields appearing in WHERE and / or limit conditions.
[0219] For example: select zip, sum(salary) as salary from stuff group by zip having sum(salary)>15000;
[0220] If zip is a sensitive field, a desensitization conversion algorithm is set for it. Through the method of this application, the zip in the underlying statement fromstuff group by zip having sum(salary)>15000 will not be desensitized, and only the outermost select output field zip will be desensitized.
[0221] Through the method of this application, the desensitization capability of various complex SQL statements can be supported, and the desensitization capability of VIEW and CTE statements can be supported at the same time.
[0222] In particular, when there are sensitive fields in the select clause of create table as select and insert into select and desensitization rules are set for the fields, this application provides two processing modes to automatically desensitize such SQL and protect sensitive data in derived tables.
[0223] There are two processing modes for derived table desensitization:
[0224] Mode 1: Derived mode, the desensitization rule corresponding to the table field name (alias) that the output field name (alias) of the select clause query of create table as select and insert into select depends on is used as the desensitization rule of the output field, and the desensitization rule is inserted into the protection policy library. When querying the derived table, the desensitization conversion is performed according to the inserted desensitization rule. This mode does not modify the original data of the derived table and is the default processing mode of this application;
[0225] Mode 2: Rewrite mode, rewrite the select clause of create table as select and insert into select. After executing create table as select and insert into select, the data in the derived table is anonymized data.
[0226] like:
[0227] Statement 1: create table stuff_1as select*from stuff;
[0228] Statement 2: insert into table stuff_2select*from stuff;
[0229] Some fields in the stuff table contain sensitive fields, and desensitization rules are set;
[0230] After executing statement 1 and statement 2 respectively:
[0231] In mode 1, the data in the newly created table stuff_1 is the original data of stuff; the data inserted into the table stuff_2 is the original data of stuff; when querying stuff_1 and stuff_2, desensitization conversion is performed according to the derived desensitization rules;
[0232] In mode 2, the data in the newly created table stuff_1 and the inserted table stuff_2 are all the data after the stuff table is desensitized;
[0233] The data filtering described in the embodiment of the present application is achieved by deeply traversing and analyzing the AST nodes generated by the from clause of the abstract syntax tree. If the current AST node is a leaf node and a physical table, the policy proxy interface is called to obtain the table filtering rules. If the filtering rules for the table exist, the AST tree is reconstructed, and a sub-query node is added. The original parent node of the leaf node becomes the parent node of the sub-query node, and the leaf node is the child node of the sub-query node. The obtained filtering rules are applied to the sub-query node to complete the reconstruction of the AST tree.
[0234] The data protection process of this application can first execute the data filtering process to generate a reconstructed AST, and then execute the data desensitization process based on this AST; the data filtering process can also be embedded in the data desensitization process. Regardless of the method, the final execution result of the data protection process is the same.
[0235] Figure 7 This is a data protection flow chart of a SQL big data database engine according to another embodiment of the present application, such as Figure 7 As shown, for SQL-type big data database engines, the real-time protection process of sensitive data described in the embodiment of the present application is as follows:
[0236] This process takes the embedding of data filtering process in data masking as an example.
[0237] Step 1: Client SQL request. Specifically, the client directly initiates an SQL access request to the big data database execution engine through the network, and the request carries the user name as the user's identity.
[0238] Step 2: For the SQL parser, specifically, the SQL parser of the big data database engine performs lexical analysis on the SQL statement to obtain a lexical analysis general symbol stream, and then performs syntax analysis on the symbol stream to construct a syntax tree;
[0239] Step 3: For the SQL analyzer, specifically, the SQL analyzer of the big data database engine deeply analyzes the unparsed relations in the syntax tree to obtain the abstract syntax tree AST;
[0240] Step 4: Parse the abstract syntax tree. Specifically, the AST parsing module of the big data database engine reversely parses the AST syntax tree to obtain a decomposable object tree structure.
[0241] Step 5: Reconstruct the abstract syntax tree. Specifically, the AST reconstruction module of the big data database engine reconstructs the AST, further abstracts and structures the SQL, and generates a reconstructed AST tree.
[0242] Step 6: The protection switch is turned on &&SQL has not been rewritten. Specifically, the big data database engine determines that the data protection function switch of this engine has been turned on and the current SQL statement has not been rewritten. If the above two conditions are met at the same time, execute step 7 to enter the data protection process, such as Figure 7 The data protection process on the right, otherwise go to step 9;
[0243] Step 7: Enter the data protection process, deeply traverse and parse the AST tree, rewrite the SQL statements involving sensitive fields, and reconstruct the AST tree:
[0244] In this specific embodiment, the abstract syntax tree parsing process generated by HIVE is taken as an example:
[0245] Step 7.1: Deeply traverse and parse the AST to obtain the internal structure of the from clause and generate the dependency table fromTables of the query output fields. The dependency table records the tables and fields that each query output field depends on layer by layer in the form of a tree.
[0246] In the process of traversing and parsing the AST of the from clause, if the current AST node is a leaf node and a physical table, call the policy proxy interface to obtain the table filtering rules. If the filtering rules for the table exist, reconstruct the AST tree, add a subquery node, and the original parent node of the leaf node becomes the parent node of the subquery node. The leaf node is the child node of the subquery node, and the obtained filtering rules are applied to the subquery node to complete the reconstruction of the AST tree. In this way, when the rewritten SQL statement is generated based on the AST, the SQL statement assembled with the filtering rules will be used as the subquery of the table, and the original physical table will be used as the alias of the subquery.
[0247] Step 7.2: Obtain the name of each output field of the outermost layer of the select query by deeply traversing the AST, and generate the output field list of the outermost layer of the select query;
[0248] In particular: when the output field of the outermost query is '*', it means that the output fields are all the output fields of the next layer of the dependency table, and the next layer of fields are used as the final output fields of the outermost query, and so on;
[0249] Step 7.3: Obtain all newly created tables or inserted table names (or table aliases) in the create table or insert clause by deeply traversing the AST tree;
[0250] In particular: The purpose of step 7.1 to step 7.3 is to deeply analyze the AST nodes to obtain the internal structure of SQL. The execution order of step 7.1, step 7.2, and step 7.3 can be adjusted. No matter which step is executed first, the generated SQL internal structure result is the same.
[0251] Step 7.4: Get an output field name from the outermost output field list of the select query in order;
[0252] Step 7.5: Deeply traverse the dependency table to find the final dependent table name (or table alias) and field name (or field alias) corresponding to the current output field name;
[0253] Step 7.6: Call the policy proxy module interface (with the user name) to obtain the final dependent table and field desensitization rules of the current output field;
[0254] Step 7.7: If no desensitization rules are obtained, jump to step 7.10; if the current SQL statement is a select statement, jump to step 7.8 to rewrite the SQL; if the current statement is a create table / insert statement, obtain the derived table desensitization mode configured by the big data database engine. If it is a rewrite mode, execute step 7.8 to rewrite the SQL; if it is a derived mode, execute step 7.9 to insert a new desensitization policy;
[0255] Step 7.8: Rewrite the value of the AST node (a SQL statement) corresponding to the current output field, and replace the original value of the AST node with the new SQL statement spelled according to the desensitization rule. If the output field does not contain the desensitization rule, no change will be made.
[0256] Step 7.9: Call the policy proxy interface to insert the desensitization policy corresponding to the current output field into the data protection policy database. The resource table name corresponding to the desensitization policy is the table name (or table alias) obtained in step 7.3, and the desensitization rule is the desensitization rule obtained in step 7.6. If there are multiple tables, the corresponding desensitization policy is inserted into each table;
[0257] Step 7.10: Determine whether all the outermost query output fields of the select have been processed. If not, process the remaining output fields one by one according to steps 7.4-7.9;
[0258] Step 7.11: If the SQL is rewritten, traverse the rewritten abstract syntax tree, resynthesize the abstract syntax tree into a new SQL statement, generate the final executed SQL statement, and set the SQL rewrite mark as rewritten to prevent it from being modified again next time;
[0259] In particular, the SQL rewrite mark is a control variable parameter described in the embodiment of the present application.
[0260] Step 8: After executing step 7.11, the big data database execution engine determines whether the data protection switch is turned on and the SQL has been rewritten. If the above two conditions are met at the same time, execute step 2 to submit the rewritten SQL statement to the SQL parsing interface to re-analyze the SQL statement; otherwise, execute step 9 to continue the subsequent process;
[0261] Step 9: SQL optimizer. Specifically, the SQL optimizer of the big data database engine optimizes most of the logic layer by transforming the manipulation tree, merging operators, reducing the number of jobs, and improving performance.
[0262] Step 10: Generate an executable physical plan. Specifically, the executable physical plan module of the big data database engine generates an executable physical plan and generates an executable big data job.
[0263] Step 11: big data cluster distributed execution, specifically, the execution module of the big data database engine submits the executable big data job to the distributed execution environment of the big data database engine for distributed execution;
[0264] Step 12: Return the execution result to the client. Specifically, after the execution is completed, the big data database engine result processing module will return the result set data directly to the client.
[0265] In particular, steps 1 to 5 and steps 9 to 12 are existing functions and processes of the big data database engine, steps 6 to 8 are new data protection processes added to this application, and the rewritten SQL statement executed in step 11 is executed concurrently on each distributed processing node.
[0266] In particular: the data filtering process described in the embodiment of the present application can be integrated into the data desensitization process, and the data filtering process is embedded when analyzing the from clause in step 7.1; at the same time, it can also be further adjusted, not executing the data filtering process in step 7.1, adding data filtering process processing before step 7.11, generating a reconstructed AST, and the subsequent process remains unchanged;
[0267] Or the data filtering process and the data desensitization process are called successively as two independent sub-processes of data protection:
[0268] First, execute the data filtering process, insert the data filtering process before the original step 7.1, generate the reconstructed AST, and then execute the original steps 7.1-7.11 in sequence (the data filtering process in the original step 7.1 is no longer called).
[0269] Regardless of the execution order, the final execution result is the same.
[0270] Data protection process for non-SQL big data database engines (including but not limited to KUDU, etc.), Figure 8 This is a data protection flow chart of a non-SQL big data database engine according to another embodiment of the present application, such as Figure 8 As shown:
[0271] Step 1: Client request. Specifically, the client application directly initiates a data request to the big data database engine server through the network through commands, scripts, applications, interface calls, etc. The client will carry the visitor's own username information in the request message;
[0272] Step 2: Request parsing. Specifically, the server accepts the client's access request and parses it.
[0273] Step 3: Scheduling execution. Specifically, according to the distributed scheduling framework of the big data database engine, the request scheduling is distributed to each processing node to execute specific logic;
[0274] Step 4: Each processing node obtains the result data set. Specifically, the processing node receives the data processing request, queries the data that meets the requirements according to the request business logic, and obtains the query result set;
[0275] In particular, the present application performs data filtering and desensitization functions on the result set at each distributed processing node after the result set is obtained;
[0276] Step 5: Turn on the data protection switch. Specifically, the processing node determines whether the data protection function of the big data database engine is turned on. If it is turned on, go to step 6, otherwise go to step 9;
[0277] Step 6: Data protection process, specifically, filtering and desensitizing the query result data set obtained in step 4;
[0278] Step 6.1: Obtain the protection policy. Specifically, call the policy proxy interface to obtain the data protection policy of the current access user.
[0279] Step 6.2: Determine whether the current result set meets the desensitization strategy. Specifically, determine whether the current requested access data set needs to be desensitized based on the acquired data desensitization strategy and the resource entity information of the user access request;
[0280] Step 6.3: Get a piece of result data from the result set in sequence. Specifically, get a piece of result data from the result set in sequence, also called a record;
[0281] Step 6.4: Determine whether the current record meets the filtering policy? Specifically, according to the data filtering rules, determine whether the record meets the filtering conditions. If it does not meet the filtering conditions, execute step 6.5, otherwise execute step 6.7;
[0282] Step 6.5: According to the judgment result of step 6.2, determine whether the current record needs to perform data desensitization. If desensitization is required, execute step 6.6; otherwise, execute step 6.7;
[0283] Step 6.6: Parse the data, call the desensitization rules one by one, perform data desensitization conversion, and assemble to generate a desensitized record. Specifically, parse the current result data, find the columns (possibly multiple) that need to be desensitized, call the corresponding data desensitization rules one by one to perform desensitization conversion, and reassemble the converted results into the desensitized result data according to the original data format;
[0284] Step 6.7: Determine whether all result sets have been processed. Specifically, determine whether all current data sets have been processed. If not, execute step 6.3 until all the obtained result set data have been filtered and desensitized.
[0285] Step 7: Group the filtered and desensitized result set. Specifically, the processing node encapsulates the result data.
[0286] Step 8: The result set is returned to the client. Specifically, the client directly obtains the returned results from each distributed processing node and summarizes them.
[0287] The data masking and data filtering processes described in this flow can be used in combination or separately.
[0288] In particular, steps 1 to 4 and steps 7 to 8 are existing functions and processes of the big data database engine, step 6 is the functional process of the data protection module, and steps 4 to 7 are executed in parallel on each processing node.
[0289] The policy agent module described in the embodiment of the present application is deployed together with the big data database engine as a plug-in package, and the big data database engine is responsible for loading and starting it. The policy agent module obtains the data protection policy configured by the execution engine from the policy management module of the present application and caches it. In this way, the policy is obtained and the policy interface is provided to the outside. This is the default implementation method of the policy agent module plug-in.
[0290] The default plugin implementation steps are as follows:
[0291] Step 1: The policy agent module is deployed together with the big data database engine as a plug-in package. The big data database engine is responsible for loading and starting. Only one policy agent plug-in can be selected for loading;
[0292] Step 2: Obtain the data protection policy configured by the execution engine from the policy management module;
[0293] Step 3: When the execution engine receives the access request from the client, it parses the request. The data protection module analyzes the access request content based on the user identification information built into the request session and combines it with the policy data obtained from the policy agent.
[0294] Step 4: The policy code module provides protection strategies for the data protection module, audits access requests, and records user access logs to sensitive data to make sensitive data traceable.
[0295] Step 5: The data protection module determines whether data filtering and desensitization are required for this request. If data filtering or desensitization is required, the data is filtered and converted according to the corresponding filtering rules and desensitization rules in the policy. The data filtering and desensitization conversion process is completed in parallel using the distributed computing capabilities of the cluster.
[0296] Step 6: The data desensitization and filtering results are returned directly to the client by the big data database engine.
[0297] The compliance administrator sets sensitive data protection policies for multiple big data database engines through the policy management interface. Each big data database engine obtains the data protection policy corresponding to the engine and transparently protects the data in real time during the client access process. Fig. 9 According to another embodiment of the present application, a data protection policy implementation flow chart is set through a policy management interface, such as Fig. 9 shown.
[0298] Step 1: Log in. Specifically, the compliance administrator uses his / her own account and password to log in to the policy management interface;
[0299] Step 2: Set the protection policy. The compliance administrator enters the data protection policy configuration interface corresponding to the big data component, sets the protection object through the interface, selects the big data database engine service, database, table, column in turn, selects the user / user group / role that has limited access to the protection object, and then selects the corresponding desensitization rules for sensitive field columns, such as random, replacement, offset, etc. You can also set the corresponding table data filtering policy. After submitting the configuration, the data protection policy will be saved to the policy database;
[0300] Step 3: The policy agent module obtains the policy from the policy database in step 2. Specifically, the policy agent module obtains the data protection policy;
[0301] Step 4: The big data client initiates an access request, specifically, directly initiates a query request to the big data database engine;
[0302] Step 5: Data protection. Specifically, when the big data execution engine receives the access request from the client, it parses the request;
[0303] Step 6: The data protection module obtains the policy from the policy agent module. Specifically, the data protection module analyzes the content of the requested access based on the user identification information built into the current request session and combines the policy data obtained from the policy agent.
[0304] Step 7: Determine whether data filtering and desensitization are required for this request. If data filtering or desensitization is required, filter and convert the data according to the corresponding filtering rules and desensitization rules in the policy. The conversion process of data filtering and desensitization is completed in parallel by fully utilizing the distributed computing power of the cluster.
[0305] Step 8: The big data database execution engine returns the execution results directly to the client.
[0306] Secondary developers or applications call the data protection policy interface through the API interface to set the data protection policies of multiple big data database engines. Each big data database engine obtains the data protection policy corresponding to the engine, and transparently protects the data in real time during the client access process. Fig.10 This is a flowchart for implementing a data protection policy through an API interface according to another embodiment of the present application.
[0307] Step 1: API login. Specifically, the secondary developer or application performs user authentication through the API authentication interface. The interface has built-in accounts and passwords, and the policy management module authenticates the user identity of the API caller.
[0308] Step 2: API sets protection strategy. Specifically, after identity authentication is passed, the secondary developer or application calls the policy setting API interface provided by the policy management module, sets the protection object through the interface, selects the big data database engine service, database, table, column in turn, sets the user / user group / role that has limited access to the protection object, and then sets the corresponding desensitization rules for sensitive field columns, such as random, replacement, offset, etc. You can also set the corresponding table data filtering strategy. After the interface call is successful, the data protection strategy will be saved to the strategy database;
[0309] Step 3: The policy agent module obtains the data protection policy from the policy database;
[0310] Step 4: The big data client sends an access request, specifically, directly initiates a query request to the big data database engine;
[0311] Step 5: When the big data execution engine receives the access request from the client, it parses the request;
[0312] Step 6: The data protection module obtains the policy from the policy agent module. Specifically, the data protection module analyzes the content of the requested access based on the user identification information built into the current request session and combines the policy data obtained from the policy agent.
[0313] Step 7: The data protection module determines whether the request needs to be filtered and desensitized. If data filtering or desensitization is required, the data is filtered and converted according to the corresponding filtering rules and desensitization rules in the policy. The data filtering and desensitization conversion process is completed in parallel using the distributed computing power of the cluster.
[0314] Step 8: The big data database execution engine returns the execution results directly to the client.
[0315] Sqoop (including Sqoop1 and Sqoop2) is a tool for transferring data between Hadoop and relational databases. It can export data from large databases such as Hive to relational databases (such as MySQL, Oracle, etc.). After setting the desensitization strategy for the big data database engine, the Sqoop tool can automatically implement the data desensitization function during the data export process and export the desensitized data to the relational database. Taking Hive as an example, Fig.11 This is a flowchart of data protection during data import and export according to another embodiment of the present application, such as Fig.11 shown.
[0316] Step 1: Set up protection policies. Specifically, the compliance administrator sorts out sensitive information according to the compliance requirements of the industry and sets up desensitization policies for Hive sensitive fields. Different policies are applied to different identity users.
[0317] Step 2: The client issues an export command. Specifically, the Hive data is exported to the relational database through the Sqoop client command. The Sqoop export command can specify the source and target databases, tables, and conditional expressions.
[0318] Step 3: Export data and perform data desensitization. Specifically, Sqoop sends a data query request to the Hive execution engine. When the Hive execution engine receives the data query request, it desensitizes the result data according to the desensitization policy set in step 1 and the user's identity information, and writes the desensitized data into the relational database.
[0319] Step 4: The execution result is returned to the client. Specifically, Sqoop returns the command execution result to the client.
[0320] This specific implementation process can be used as the implementation process of the big data static desensitization program.
[0321] Compared with the related art, the method and device described in the embodiment of the present application firstly do not require a dedicated desensitizing server, saving additional software and hardware costs; secondly, the execution efficiency is the highest, and the distributed computing capabilities of the big data database execution engine are fully utilized to complete data filtering and desensitization functions with high performance, which can simultaneously meet the requirements of efficient static and dynamic data desensitization of big data; thirdly, it is completely transparent to applications and users, does not change the business logic, does not destroy the original data, and is unaware of the application; at the same time, the method and device provided by the present application have a high security factor, the intermediate data is not landed, and the risk of data leakage is reduced. In particular, the present application can also automatically desensitize and convert tables with inheritance relationships to comprehensively protect data security; in addition, it has a high scalability and a unified policy management and algorithm management framework. The data protection method of the present application can be extended to the big data field, and can realize the sensitive data protection capabilities of various big data database engines, including mainstream offline, interactive, online computing, and other SQL or non-SQL big data database engines.
[0322] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in each embodiment of the present application.
[0323] In the embodiments of the present application, a data protection device is also provided, which is used to implement the above embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware of a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.
[0324] According to another embodiment of the present application, a data protection device is provided, including:
[0325] A receiving module, used for receiving data request information sent by a client;
[0326] A data protection module is used to obtain a preset policy corresponding to the identity of the client carried in the data request information, and the database execution engine performs a data protection operation on the requested data according to the preset policy to obtain the target data;
[0327] Optionally, the device may further include a sending module, configured to send the target data to the client.
[0328] Optionally, the data protection module further includes a policy agent module, which is used to obtain the data protection policy corresponding to the policy device and provide a protection policy query interface for the data protection module.
[0329] Optionally, the data protection module is also used to perform data filtering operations and / or data desensitization operations on the requested data according to a preset policy, wherein the preset policy includes filtering rules and / or desensitization rules.
[0330] Optionally, the data protection module performs data filtering operations and / or data desensitizing operations on the requested data, including one of the following: performing a data desensitizing operation after performing a data filtering operation on the requested data; embedding a data filtering operation process in the process of performing a data desensitizing operation; performing a data filtering operation on the requested data; performing a data desensitizing operation on the requested data.
[0331] Optionally, when the database receives a structured query language SQL statement, the data protection module is also used to generate an abstract syntax tree AST based on the SQL statement; deeply traverse the AST to obtain the table names and field names that the outermost (or top-level) multiple output fields of the SQL statement depend on layer by layer; and desensitize the output fields corresponding to the field names in the outermost output fields of the SQL statement.
[0332] The table name in this application document is also the alias of the table, and the field name can be the alias of the field.
[0333] Optionally, after obtaining the table names and field names on which the outermost multiple output fields depend layer by layer, the device further includes at least one of the following:
[0334] It is prohibited to desensitize the output fields in the clauses related to intermediate calculations in the SQL statement;
[0335] It is prohibited to mask the output fields in the Where and / or Limit conditional statements in the SQL statement.
[0336] The clauses related to intermediate calculations in the above SQL statements include: GROUP BY, ORDER BY, SORT BY, DISTRBUTE BY, JOIN, UNION, LIKE, BETWEEN, function and other clauses.
[0337] Optionally, data desensitization is performed on the derived table in the select clause of the associated derived table in the SQL statement (the select clause includes clauses such as create table as select or insert into select) in one of the following ways: obtaining the preset desensitization rule corresponding to the table and field name on which the output field name of the select clause query of the associated derived table depends, using the preset desensitization rule as the desensitization rule for the output field, and performing data desensitization on the derived table according to the desensitization rule; after the database executes the select clause of the associated derived table, the derived table is a derived table after data desensitization is performed according to the preset desensitization rule.
[0338] Optionally, after the database receives the structured query language SQL statement, it is determined that the current SQL statement has not been rewritten, and the AST tree is deeply traversed to obtain at least one of the following information:
[0339] Obtaining the internal structure of the SQL statement, generating a dependency table of the output fields queried by the SQL statement, wherein the dependency table records the tables and fields that each output field depends on layer by layer in the form of a tree;
[0340] Get the name of each outermost output field of the select statement query in the SQL statement, and generate the outermost select query output field list;
[0341] Get the names of all newly created tables and inserted tables in the SQL statement (optionally, the tables in the create table or insert clause).
[0342] Optionally, the following steps are performed for each output field in the outermost output field list queried by the select statement: select the first output field from the outermost output field list in a preset order; deeply traverse the dependency table to obtain the table name and field name of the deepest dependency corresponding to the first output field name; obtain the desensitization rules preset for the table name and field name of the deepest dependency; after determining that the SQL statement is a select statement, process the partial SQL statement corresponding to the first output field according to the desensitization rules to form a first SQL statement, and use the first SQL statement to rewrite the original value of the AST node corresponding to the first output field.
[0343] Optionally, the desensitization rules of the deepest dependent table and field are obtained, and after determining that the SQL statement is a create table as select or insert into select statement, the corresponding desensitization rules are inserted into the current output fields of all newly created tables and inserted tables in the database.
[0344] Optionally, after rewriting the AST node value corresponding to each output field in the outermost output field list, traverse the rewritten AST to resynthesize the second SQL statement, and determine that the second SQL statement is the SQL statement that is finally executed.
[0345] Optionally, after traversing the rewritten AST to resynthesize the second SQL statement and determining that the second SQL statement is the final SQL statement to be executed, the database determines a physical plan to be executed based on the final SQL statement to be executed; executes the physical plan, and feeds back the processing result to the client.
[0346] Optionally, when the database receives a structured query language SQL statement, the database execution engine performs the data filtering operation on the requested data including:
[0347] Generate an abstract syntax tree AST based on the SQL statement;
[0348] Traversing the AST executes the following steps:
[0349] When a leaf node in the AST is a physical table, obtain a table filtering rule of the physical table;
[0350] In the AST, a subquery node is added, wherein the original parent node of the leaf node is the parent node of the subquery node, and the leaf node is the child node of the subquery node;
[0351] Apply the table filtering rule to the subquery node;
[0352] After traversing the AST and executing the above steps, a reconstructed AST is obtained, a reconstructed SQL statement is obtained according to the reconstructed AST, and the reconstructed SQL statement is executed.
[0353] Optionally, when the database receives a non-SQL access request, data protection is performed on the requested data according to a preset strategy, including: the database obtains the data requested by the non-SQL access request; obtains the preset strategy corresponding to the data, and executes at least one of the following for each data set corresponding to the data according to the preset strategy: filters the data according to the filtering rules included in the preset strategy; parses the data, calls the desensitization rules corresponding to each column one by one to desensitize each column, and reassembles the columns after the desensitization processing according to the original data format to obtain the desensitized data.
[0354] Optionally, the database is connected to a policy device, wherein the policy device is allowed to provide a policy model for one or more databases, and the policy device establishes a policy model based on at least one of the following: identity identification, resource entity, protection rules, wherein the protection rules include desensitization rules and / or filtering rules.
[0355] Optionally, the policy device is externally connected to an API interface for receiving input signals for configuring the policy model.
[0356] Optionally, the policy device is also used to store a desensitization rule model, wherein the policy device determines the desensitization rule model based on at least one of the following: a data desensitization operator, a data desensitization rule; wherein the data desensitization operator includes a basic function for implementing data desensitization, and the data desensitization rule includes a rule for performing data desensitization based on the data desensitization operator.
[0357] It should be noted that the above modules can be implemented by software or hardware. For the latter, it can be implemented in the following ways, but not limited to: the above modules are all located in the same processor; or the above modules are located in different processors in any combination.
[0358] According to another embodiment of the present application, a storage medium is provided, in which a computer program is stored, wherein the computer program is configured to execute the method described in any one of the above embodiments when running.
[0359] An embodiment of the present application further provides an electronic device, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.
[0360] Obviously, those skilled in the art should understand that the above modules or steps of the present application can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, and optionally, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order from that herein, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present application is not limited to any specific combination of hardware and software.
[0361] The above description is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.
Claims
1. A data protection method, characterized in that: include: The database execution engine receives data request information sent by the client; Acquire a preset policy corresponding to the identity of the client carried in the data request information, and the database execution engine performs a data protection operation on the requested data according to the preset policy to obtain target data; The database execution engine performs a data protection operation on the requested data according to the preset policy, including: the database execution engine performs a data desensitization operation on the requested data according to the preset policy, wherein the preset policy includes a desensitization rule; The database execution engine performs a data desensitization operation on the requested data according to the preset strategy, including: converting the data in the sensitive data domain in the requested data according to the desensitization rules corresponding to the identity identifier of the client, wherein the desensitization rules are used to convert the data in the sensitive data domain in the requested data.
2. The method according to claim 1, characterized in that: The database execution engine performs a data protection operation on the requested data according to the preset strategy, including: The database execution engine performs data filtering operations and data desensitization operations on the requested data according to a preset strategy, wherein the preset strategy includes filtering rules and desensitization rules.
3. The method according to claim 2, characterized in that The database execution engine performs a data filtering operation and a data desensitization operation on the requested data, including one of the following: After performing data filtering operations on the requested data, perform data desensitization operations; Embed data filtering operation process in the process of performing data desensitization operation.
4. The method according to claim 1, characterized in that: When the database receives a structured query language SQL statement, the database execution engine performs the data desensitization operation on the requested data according to a preset strategy, including: Generate an abstract syntax tree AST according to the SQL statement; Deeply traverse the AST to obtain the table names and field names that the outermost multiple output fields of the SQL statement depend on layer by layer; The output field corresponding to the field name in the outermost output field of the SQL statement is data-massaged.
5. The method according to claim 4, characterized in that After obtaining the table names and field names to which the outermost multiple output fields depend layer by layer, the method further includes at least one of the following: It is prohibited to desensitize the output fields in the clauses related to the intermediate calculation in the SQL statement; It is prohibited to desensitize the output fields in the Where and / or Limit conditional statements in the SQL statement.
6. The method according to claim 4, characterized in that Desensitize the data of the derived table in the select clause of the associated derived table in the SQL statement by one of the following methods: Obtaining the table and the preset desensitization rule corresponding to the field name on which the output field name of the select clause query of the associated derived table depends, using the preset desensitization rule as the desensitization rule of the output field, and performing data desensitization on the derived table according to the desensitization rule; After the database executes the select clause of the associated derived table, the derived table is a derived table after data desensitization is performed according to the preset desensitization rule.
7. The method according to claim 4, characterized in that After the database receives the structured query language SQL statement, it is determined that the current SQL statement has not been rewritten, and the AST tree is deeply traversed to obtain at least one of the following information: Obtaining the internal structure of the SQL statement, generating a dependency table of the output fields queried by the SQL statement, wherein the dependency table records the tables and fields that each output field depends on layer by layer in the form of a tree; Obtain the name of each outermost output field of the select statement query in the SQL statement, and generate an outermost select query output field list; Get the names of all newly created tables and inserted tables in the SQL statement.
8. The method according to claim 7, characterized in that For each output field in the outermost output field list of the select statement query, the following steps are performed: Selecting a first output field from the outermost output field list in a preset order; Deeply traverse the dependency table to obtain the deepest dependent table name and field name corresponding to the first output field name; Obtaining the desensitization rules preset for the table name and field name of the deepest dependency; After determining that the SQL statement is a select statement, the partial SQL statement corresponding to the first output field is processed according to the desensitization rule to form a first SQL statement, and the original value of the AST node corresponding to the first output field is rewritten using the first SQL statement.
9. The method according to claim 8, characterized in that Obtain the desensitization rules of the most deeply dependent tables and fields, and after determining that the SQL statement is a create table as select or insert into select statement, insert the corresponding desensitization rules into all newly created tables and current output fields in the inserted table of the database.
10. The method according to claim 8, characterized in that After rewriting the AST node value corresponding to each output field in the outermost output field list, the method further includes: The rewritten AST is traversed to resynthesize the second SQL statement, and the second SQL statement is determined to be the SQL statement to be finally executed.
11. The method according to claim 10, characterized in that After traversing the rewritten AST and resynthesizing the second SQL statement, and determining that the second SQL statement is the SQL statement to be finally executed, the method further includes: The database determines a physical plan to be executed according to the finally executed SQL statement; The physical plan is executed, and the processing result is fed back to the client.
12. The method according to claim 2, characterized in that: When the database receives a structured query language SQL statement, the database execution engine performs the data filtering operation on the requested data, including: Generate an abstract syntax tree AST according to the SQL statement; Traversing the AST performs the following steps: When a leaf node in the AST is a physical table, obtaining a table filtering rule of the physical table; In the AST, a subquery node is added, wherein the original parent node of the leaf node is the parent node of the subquery node, and the leaf node is the child node of the subquery node; Applying the table filter rule to the subquery node; After traversing the AST and executing the above steps, a reconstructed AST is obtained, a reconstructed SQL statement is obtained according to the reconstructed AST, and the reconstructed SQL statement is executed.
13. The method according to claim 2, characterized in that When the database receives a non-SQL access request, it protects the requested data according to the preset strategy, including: The database obtains the data requested by the non-SQL access request; Obtain a preset strategy corresponding to the data, and perform at least one of the following on each piece of data in the data set corresponding to the data in sequence according to the preset strategy: Filtering the data according to the filtering rules included in the preset strategy; Parse the data, call the desensitization rules corresponding to each column one by one to desensitize the data in each column, and reassemble the desensitized columns according to the original data format to obtain the desensitized data.
14. The method according to claim 1, characterized in that The database is connected to a policy device, wherein the policy device is allowed to provide a policy model for one or more databases, and the policy device establishes a policy model based on at least one of the following contents: identity identification, resource entity, protection rules, wherein the protection rules include desensitization rules and filtering rules.
15. The method according to claim 14, characterized in that The policy device is externally connected to an API interface for receiving input signals for configuring a policy model.
16. The method according to claim 14, characterized in that The policy device is also used to store a desensitization rule model, wherein the policy device determines the desensitization rule model according to at least one of the following: a data desensitization operator, a data desensitization rule; The data desensitization operator includes a basic function for implementing data desensitization. The data desensitization rules include rules for performing data desensitization based on the data desensitization operator.
17. A data protection device, characterized in that: include: A receiving module, used for receiving data request information sent by a client; A data protection module, used to obtain a preset policy corresponding to the identity of the client carried in the data request information, and perform a data protection operation on the requested data according to the preset policy to obtain target data; The data protection module is used to perform a data desensitization operation on the requested data according to the preset policy, wherein the preset policy includes a desensitization rule; The data protection module is used to convert the data in the sensitive data domain in the requested data according to the desensitization rules corresponding to the identity identifier of the client, wherein the desensitization rules are used to convert the data in the sensitive data domain in the requested data.
18. A storage medium, characterized in that: The storage medium stores a computer program, wherein the computer program is configured to execute the method according to any one of claims 1 to 16 when executed.
19. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to run the computer program to perform the method according to any one of claims 1 to 16.
