Data processing method and device

Abstract

The present application provides a data processing method and device, the method comprising obtaining a first instruction and a first data table; wherein the first instruction is used to process the first data table and the second data table, and the first instruction is analyzed using an abstract syntax tree to obtain an instruction structure tree; when it is determined that each node branch in the instruction structure tree meets the data security processing condition, and there is no node with a value marked as a group in the instruction structure tree, a node branch of a single column operation is selected from the instruction structure tree as an alternative branch to obtain multiple alternative branches; at least one target branch is selected from the multiple alternative branches, and the values ​​of the nodes with values ​​marked as column in the instruction structure tree are counted, and the first data table and the first instruction in a plaintext state are processed according to the values ​​of the at least one target branch and the nodes with values ​​marked as column in the instruction structure tree to obtain a third data table and a second instruction, thereby reducing the amount of data processing in a ciphertext state and improving data processing efficiency.

Description

Technical Field

[0001] The present application relates to the field of financial technology (Fintech), and in particular to a data processing method and device. Background Art

[0002] With the development of computer technology, more and more technologies are being applied in the financial field. The traditional financial industry is gradually transforming into financial technology (Fintech), and privacy computing is no exception. However, due to the security and real-time requirements of the financial industry, higher requirements are also placed on privacy computing.

[0003] Data preprocessing will be performed before privacy computing is performed. However, the preprocessing of privacy computing usually involves data cleaning, format legitimacy detection, or data quality verification of data tables. It still requires processing a large amount of data in encrypted state, resulting in low data processing efficiency. Summary of the invention

[0004] The present application provides a data processing method and device, which aims to process data and instructions in plain text state in advance, reduce the amount of data processing in cipher text state, and thus improve data processing efficiency.

[0005] An embodiment of the present application provides a data processing method, which is applied to a first node and includes the following steps:

[0006] Obtain a first instruction and a first data table; wherein the first instruction is used to process the first data table and the second data table, the first data table is stored in the first node, and the second data table is stored in the second node;

[0007] The first instruction is analyzed using an abstract syntax tree to obtain an instruction structure tree; when it is determined that each node branch in the instruction structure tree meets the data security processing condition and there is no node whose value is a grouping mark in the instruction structure tree, a node branch of a single column operation is selected from the instruction structure tree as an alternative branch to obtain multiple alternative branches;

[0008] Select at least one target branch from multiple alternative branches, and count the values ​​of the nodes whose values ​​are column marks in the instruction structure tree. According to the values ​​of at least one target branch and the nodes whose values ​​are column marks in the instruction structure tree, process the first data table and the first instruction in the plain text state to obtain a third data table and a second instruction.

[0009] In one embodiment, after the first data table and the first instruction in the plain text state are processed according to at least one target branch to obtain the third data table and the second instruction, the method further includes:

[0010] generating a first processing request according to the second instruction and the encrypted third data table, and sending the first processing request to the third node;

[0011] Among them, the third node is also used to receive the second processing request sent by the second node, and to perform data processing according to the first processing request and the second processing request, and send the processing results to the first node and the second node; the second processing request is generated by the second node according to the third instruction and the encrypted fourth data table, and the third instruction and the fourth data table are obtained by processing the second data table and the first instruction.

[0012] In one embodiment, determining whether each node branch in the instruction structure tree meets the data security processing condition specifically includes any one of the following:

[0013] There is no node branch representing full table query in the instruction structure tree;

[0014] There is no node branch representing full column query in the instruction structure tree;

[0015] The instruction structure tree includes a first node branch representing a single-column query and other second node branches belonging to result branches; wherein the second node branch is not a node branch representing a name.

[0016] In one embodiment, a node branch of a single column operation is selected from the instruction structure tree as an alternative branch to obtain multiple alternative branches, specifically including:

[0017] Each node branch in the instruction structure tree is traversed. When the expression corresponding to the node branch is a filter statement, an operation statement or an aggregation statement, and the node branch contains only one node whose value is a column mark, the node branch is used as an alternative branch.

[0018] In one embodiment, selecting at least one target branch from a plurality of candidate branches specifically includes:

[0019] When the expression corresponding to the alternative branch is a filter statement, the alternative branch is used as the target branch;

[0020] When the expression corresponding to the alternative branch is an operation statement or an aggregate statement, and the nodes whose values ​​are column-marked contained in the alternative branch are different from the nodes whose values ​​are column-marked in the remaining node branches belonging to the result branch in the instruction structure tree, the alternative branch is used as the target branch.

[0021] In one embodiment, the first data table and the first instruction in the plain text state are processed according to at least one target branch and the value of the node whose value is a column mark of the instruction structure tree to obtain the third data table and the second instruction, specifically including:

[0022] Repeat the process of processing the corresponding column data in the first data table in the plain text state according to the expression corresponding to one of the target branches to obtain the first intermediate data table, and update the first instruction according to the target branch, and use the first intermediate data table as the first data table, until all target branches are traversed, and use the first intermediate data table obtained in the last cycle as the second intermediate data table, and use the updated first instruction obtained in the last cycle as the intermediate instruction;

[0023] According to the value of the node whose value is the column mark in the instruction structure tree, the column data operated by the intermediate instruction is selected from the second intermediate data table to obtain the third data table; and the column mark in the intermediate instruction is updated to obtain the second instruction.

[0024] In one embodiment, the traversal order of the target branches is the target branch whose expression is a filter statement, the target branch whose expression is a calculation statement, and the target branch whose expression is an aggregation statement.

[0025] In one embodiment, the method further comprises:

[0026] When it is determined that each node in the instruction structure tree cannot meet the data security processing conditions, and there are nodes whose values ​​are group marks in the instruction structure tree, the first data table in the plain text state is grouped to obtain a third data table, and the first instruction is processed according to the nodes whose values ​​are group marks to obtain a second instruction.

[0027] Another embodiment of the present application provides an electronic device, comprising: a processor, and a memory communicatively connected to the processor;

[0028] The memory stores computer-executable instructions;

[0029] The processor executes the computer-executable instructions stored in the memory to implement the data processing method involved in any of the above embodiments.

[0030] Another embodiment of the present application provides a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, they are used to implement the data processing method involved in any of the above embodiments.

[0031] Another embodiment of the present application provides a computer program product, including a computer program, which implements the data processing method involved in any of the above embodiments when executed by a processor.

[0032] The embodiment of the present application provides a data processing method and device, wherein a first data table is stored in plain text in a first node, and a second data table is stored in plain text in a second node. The first node obtains a first instruction for processing the first data table and the second data table, and uses an abstract syntax tree to analyze the first instruction, so as to process the first instruction and the first data table in plain text based on the instruction structure. Compared with processing the first data table in cipher text, the first data table in plain text state is processed in the first node, and the processing efficiency is higher. When the third data table obtained by simplifying the first data table is transmitted to the third node in cipher text state for processing, the amount of data processed by the third node is significantly reduced, and the data processing efficiency of the third node is improved. In addition, when processing the first data table, a security processing check is first performed to avoid data leakage and increase the security of data processing. And a single-column operation instruction is selected to process the first data table according to the data column operated in the first instruction, which can reduce the data processing complexity of the first node. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0034] Figure 1 A structural diagram of a data processing system provided in one embodiment of the present application;

[0035] Figure 2 A flowchart of a data processing method provided by another embodiment of the present application;

[0036] Figure 3 A schematic diagram of an instruction structure tree provided for an embodiment of the present application;

[0037] Figure 4 A flowchart of a data processing method provided in yet another embodiment of the present application;

[0038] Figure 5 A structural diagram of a data processing device provided in yet another embodiment of the present application;

[0039] Figure 6 A structural diagram of an electronic device provided in yet another embodiment of the present application.

[0040] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0041] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0042] Privacy computing refers to the calculation and analysis of data by multiple parties without exposing their data, so as to break the data silos. When performing privacy computing, it is necessary to have the ability to strengthen privacy protection and data security compliance. Commonly used technical means include Trusted Execution Environment (TEE), federated learning, secure multi-party computing, etc.

[0043] Among them, SQL-oriented privacy computing is one of the most common application scenarios. The user inputs standard SQL instructions and implements joint secure queries of multi-party data tables through the privacy computing platform. The privacy computing platform includes private privacy computing platforms and public privacy computing platforms held by both parties. The usual execution process is that each party uses its own client to input SQL instructions and the data table to be queried into its own private privacy computing platform, and then each private privacy computing platform converts the SQL instructions into an algorithm within the public privacy computing platform and inputs it into the public privacy computing platform. The public privacy computing platform also receives the encrypted data table input by the private privacy computing platform, and then the public privacy computing platform executes the algorithm for the encrypted data table based on the secure multi-party computing framework to implement multi-party joint privacy computing.

[0044] Before converting SQL into a public privacy platform algorithm, the SQL and input data tables can be simplified in advance, and operations that do not rely on data from other institutions can be performed in advance as much as possible to reduce the complexity of SQL and the size of the data table. However, the preprocessing of privacy computing is usually mainly data cleaning, format legitimacy detection, or data quality verification of data tables, which still requires processing of a large amount of data in ciphertext state, resulting in low data processing efficiency.

[0045] Based on this, the embodiment of the present application provides a data processing method and device, which aims to optimize the privacy computing process. The factors that affect the efficiency of privacy computing on the public privacy computing platform mainly include four aspects: the size of the data table, that is, the number of rows and columns of the data table, the format of the data stored in the data table, for example: the floating point overhead is much greater than the integer overhead, the number of atomic ciphertext calculations, that is, the number of addition, multiplication, and comparison operations, and the type of underlying cryptographic protocol. The present application pre-processes the data table in the plaintext state in terms of the size of the data table and the number of atomic ciphertext calculations, optimizes the privacy computing process, and improves the efficiency of privacy computing.

[0046] For example, if Party A and Party B each have 100,000 pieces of data and need to calculate the maximum value of the two, without any processing, the overhead of directly comparing the ciphertext of 200,000 pieces of data is very huge, usually taking hundreds of seconds. If the two parties first calculate the maximum value of each party's data in plain text, and then compare the two maximum values ​​in ciphertext through the public privacy computing platform to obtain the final result, the calculation time can be shortened to milliseconds.

[0047] like Figure 1 As shown, an embodiment of the present application provides a data processing system, which includes a first node, a second node and a third node. The first node and the third node are connected in communication, and the second node and the third node are connected in communication. The first node processes the first data table and the first instruction in the plain text state to obtain the third data and the second instruction. The data size of the third data is smaller, and the calculation process in the second instruction is less. The first node also sends a first processing request to the third node. The second node processes the second data table and the first instruction in the plain text state to obtain the fourth data and the third instruction. The data size of the fourth data is smaller, and the calculation process in the third instruction is less. The second node also sends a second processing request to the third node. After receiving the first processing request and the second processing request, the third node parses the encrypted third data table and the second instruction from the first processing request, parses the encrypted fourth data table and the third instruction from the second processing request, and merges the second instruction and the third instruction to obtain the fourth instruction, and then calculates the encrypted third data table and the encrypted fourth data table in the ciphertext state based on the fourth instruction to obtain the processing result. The specific process of data processing of the data processing system can refer to the following method embodiment, which will not be repeated here.

[0048] like Figure 2 As shown, an embodiment of the present application provides a data processing method, which is applied to a first node and specifically includes the following steps:

[0049] S201: Obtain a first instruction and a first data table.

[0050] In this step, the first instruction is used to process the first data table and the second data table, the first data table is stored in the first node, and the second data table is stored in the second node. That is, the first instruction is used to jointly process the first data table stored in the first node and the second data table stored in the second node.

[0051] S202: Use the abstract syntax tree to analyze the first instruction to obtain an instruction structure tree.

[0052] In this step, the first instruction is subjected to lexical analysis and grammatical analysis using an Abstract Syntax Tree (AST) method to obtain an instruction structure tree.

[0053] AST is a tree-like representation of the grammatical structure of a programming language, and is generally used for target code generation or source code parsing. In this embodiment, an abstract syntax tree method is used to convert the first instruction into an instruction structure tree to implement parsing of the first instruction.

[0054] Each node in the instruction structure tree represents a token, and each node includes a type and a value. The type includes keywords, identifiers, operators, etc., and the value of each node can be select, sum, max, >, <, etc. Usually AST is saved in data structures such as JSON or multidimensional arrays.

[0055] The instruction structure tree can be divided into multiple node branches, which can be divided into result branches and conditional branches according to classification. The result branch is mainly used to process data, such as summing, finding the maximum value, filtering, etc. The conditional branch is used to limit the data to be processed. When the conditional branch has a grouping instruction, the grouping instruction is given priority, that is, the grouping instruction has the highest priority.

[0056] Correspondingly, the instruction structure tree obtained also contains result branches and conditional branches. When the value of a node is from, join, group by or where, and the type is a keyword, the node and its child nodes are all conditional branches. Other nodes and their child nodes at the same level as the conditional branch are result branches.

[0057] For example, the instruction structure tree is described by taking the first instruction as an SQL instruction:

[0058] create table powerfl as

[0059] select sum(t1.push_count) as sum from t2 join t1 on t2.biz=t1.bizgroup by t2.name.

[0060] t0 is a data table, t1 is another data table, and the instruction structure tree of the above SQL instruction is as follows: Figure 3 As shown in the figure, the node with value From and its child nodes all belong to the conditional branch, the node with value Group and its child nodes also belong to the conditional branch, and the node with value Select and its child nodes form multiple node branches as result branches.

[0061] S203, judging whether each node branch in the instruction structure tree meets the data security processing conditions, if so, proceeding to S204, otherwise ending the process.

[0062] The data security processing condition is used to determine whether the first instruction has the risk of leaking the first data table. Both full table query and full column query have the risk of data leakage. By traversing each node branch in the instruction structure tree, it is determined whether each node branch meets the data security processing condition. When traversing each node branch in the instruction structure tree, a recursive method is used to perform depth-first or breadth-first traversal.

[0063] In one embodiment, satisfying the data security processing condition means that there is no node branch representing full table query in the instruction structure tree. Not satisfying the data security processing condition means that there is a node branch representing full table query in the instruction structure tree.

[0064] In one embodiment, satisfying the data security processing condition means that there is no node branch representing a full-column query in the instruction structure tree. Satisfying the data security processing condition may also mean that the instruction structure tree contains a first node branch representing a single-column query and other second node branches belonging to the result branch. Among them, the second node branch is not a node branch representing a name. Not satisfying the data security processing condition means that the node branches belonging to the result branch in the instruction structure tree only include node branches representing a single-column query and node branches representing a name. Not satisfying the data security processing condition may also mean that the node branches belonging to the result branch in the instruction structure tree only include node branches representing a single-column query.

[0065] S204, determine whether there is a node with a value of a grouping mark in the instruction structure tree, if not, go to S205, if yes, go to S208.

[0066] In this step, when there is a node whose value is a grouping mark in the instruction structure tree, grouping processing is performed first, that is, the grouping instruction has the highest priority.

[0067] S205: Select a node branch of a single column operation as an alternative branch from the instruction structure tree to obtain multiple alternative branches.

[0068] In this step, the node processing of the single-column operation is relatively simple, and the first data table can be processed accordingly in advance. Based on this, when determining whether each node branch in the instruction structure tree meets the data security processing condition, and when there is no node whose value is a grouping mark in the instruction structure tree, the node branch of the single-column operation is selected from the instruction structure tree as an alternative branch to obtain multiple alternative branches.

[0069] In one embodiment, in order to determine alternative branches from the instruction structure tree, each node branch in the instruction structure tree is traversed. When the expression corresponding to the node branch is a filter statement, an operation statement or an aggregation statement, and the node branch only contains one node whose value is a column mark, the node branch is used as an alternative branch.

[0070] Traverse the child nodes of each parent node in the instruction structure tree. When the parent node contains the first child node, the second child node and the third child node, and the first child node is a column tag, the second child node is an operation tag, and the third child node is a constant tag, the parent node is used as an alternative branch. Or

[0071] In one embodiment, in order to determine alternative branches from the instruction structure tree, the child nodes of each parent node in the instruction structure tree are traversed. When the value of the parent node is max or min, the parent node contains a fourth child node, and the type of the fourth child node is a column mark, the parent node is used as an alternative branch.

[0072] In the above technical solution, by identifying the child nodes under the parent node, it is determined whether the parent node is a single-column operation node.

[0073] S206: Select at least one target branch from the multiple candidate branches, and count the values ​​of the nodes marked with columns with values ​​in the instruction structure tree.

[0074] In this step, when the expression corresponding to the alternative branch is a filter statement, the alternative branch is used as the target branch. When the expression corresponding to the alternative branch is an operation statement or an aggregation statement, and the nodes with column labels contained in the alternative branch are different from the nodes with column labels in the remaining node branches belonging to the result branch in the instruction structure tree, the alternative branch is used as the target branch, that is, when the target branch is the result branch, the column data operated is only operated once, and is not operated again in other node branches belonging to the result branch. Through such a setting, the data processing process in the first node is not too complicated, and the hardware resource requirements for the first node can be reduced.

[0075] By counting the values ​​of the nodes whose type is column mark in the instruction structure tree, the column data that has not been operated on in the first data table can be deleted to reduce the amount of data.

[0076] S207: Process the first data table and the first instruction in plain text according to at least one target branch and the value of the node whose column is marked by the value of the instruction structure tree to obtain a third data table and a second instruction.

[0077] In this step, the first data table and the first instruction in the plain text state are first processed according to at least one target branch, and then the processed first data table and the processed first instruction are processed according to the value of the node marked with the column value of the instruction structure tree to obtain the third data table and the second instruction.

[0078] First, processing the first data table and the first instruction in the plain text state according to at least one target branch specifically includes the following sub-steps:

[0079] S01. Process corresponding column data in a first data table in a plain text state according to an expression corresponding to one of the target branches to obtain a first intermediate data table.

[0080] The column data operated by the target branch is selected from the first data table, and the column data is processed according to the expression corresponding to the target branch to obtain the first intermediate data table.

[0081] S02. Update the intermediate instructions according to the target branch.

[0082] The instruction corresponding to the target branch in the intermediate instruction is replaced with the column identifier of the column data operated by the target branch, so as to realize the update of the intermediate instruction.

[0083] S03. Use the first intermediate data table as the first data table.

[0084] The first intermediate data table is used as the first data table so that subsequent target nodes continue to perform processing on the first intermediate data table.

[0085] S04, determine whether all target branches have been traversed, if so, enter S05, otherwise enter S01.

[0086] If all target branches have been traversed, the result of the last loop is directly output; if not all target branches have been traversed, the next loop is continued.

[0087] The traversal order of the target branch is the target branch of the expression as a filter statement, the target branch of the expression as a calculation statement, and the target branch of the expression as an aggregation statement. Since the target branch of the expression as a filter statement can reduce the amount of data processing, data processing is performed preferentially according to the target branch of the expression as a filter statement, which can reduce the amount of data for subsequent data processing and improve data processing efficiency. The target branch of the expression as a calculation statement and the target branch of the expression as an aggregation statement can reduce the number of atomic calculations in the third node.

[0088] S05: Use the first intermediate data table obtained in the last cycle as the second intermediate data table, and use the updated intermediate instructions obtained in the last cycle as the intermediate instructions.

[0089] After obtaining the intermediate instructions and the second intermediate data table through multiple cycles, the column data operated by the intermediate instruction is selected from the second intermediate data table according to the value of the node whose value is the column mark in the instruction structure tree to obtain the third data table. And the column identifier in the intermediate instruction is updated to obtain the second instruction. That is, the column data that does not need to be processed is deleted from the second intermediate data table to obtain the third data table. Due to the deletion of the column data, the column identifier of the retained column data is updated, and it is necessary to update the column identifier in the intermediate instruction to obtain the second instruction. By deleting the column data, when the first node generates the first processing request, the data processing amount can be reduced and the data processing efficiency can be improved.

[0090] S208 , performing grouping processing on the first data table in plain text to obtain a third data table, and removing the grouping instructions in the first instruction to obtain a second instruction.

[0091] In this step, when the first instruction includes a grouping instruction, the first data table is grouped according to the grouping instruction in the first instruction to obtain a third data table, and the grouping instruction in the first instruction is removed to obtain a second instruction.

[0092] In the above technical solution, the first data table is stored in plain text in the first node, and the second data table is stored in plain text in the second node. The first node obtains a first instruction for processing the first data table and the second data table, and uses an abstract syntax tree to analyze the first instruction, so as to process the first instruction and the first data table in plain text based on the instruction structure. Compared with processing the first data table in cipher text, the data processing process is processed to improve the data processing efficiency. In addition, when processing, a security processing check is first performed to avoid data leakage and increase the security of data processing. And the first data table is processed according to the data column operated in the instruction and the single column operation instruction is selected, which can reduce the data processing complexity of the first node.

[0093] like Figure 4 As shown, another embodiment of the present application provides a data processing method, which is applied to a data processing system. The data processing method specifically includes the following steps:

[0094] S301. The first node generates a first processing request according to the second instruction and the encrypted third data table.

[0095] Among them, the first node is based on Figure 2The embodiment shown processes the first instruction and the first data table to obtain the second instruction and the third data table, encrypts the third data table, and then generates a first processing request based on the second instruction and the encrypted third data table.

[0096] S302: The first node sends a first processing request to the third node.

[0097] S303: The second node generates a second processing request according to the third instruction and the encrypted fourth data table.

[0098] The second node processes the first instruction and the second data table to obtain the third instruction and the fourth data table. The processing of the first instruction and the second data table is the same as Figure 2 The embodiments shown are the same and will not be described again here.

[0099] After encrypting the fourth data table, a second processing request is generated based on the third instruction and the encrypted fourth data table.

[0100] S304: The second node sends a second processing request to the third node.

[0101] S305: The third node processes data according to the first processing request and the second processing request to obtain a processing result.

[0102] In this step, the encrypted third data table and the second instruction are parsed from the first processing request, the encrypted fourth data table and the third instruction are parsed from the second processing request, and the second instruction and the third instruction are merged to obtain the fourth instruction, and then based on the fourth instruction, the encrypted third data table and the encrypted fourth data table are operated in a ciphertext state to obtain the processing result.

[0103] S305a: The third node sends the processing result to the first node.

[0104] S305b: The third node sends the processing result to the second node.

[0105] In the above technical solution, the first node has processed the first data table in plain text to obtain the third data with a smaller data volume. The second node has processed the second data table in plain text to obtain the fourth data with a smaller data volume. The third data table and the fourth data table processed by the third node in the ciphertext state have smaller data volumes, which can reduce the data processing volume of the third node and reduce the resource loss of the third node.

[0106] Another embodiment of the present application provides a data processing method, which is applied to a first node and specifically includes the following steps:

[0107] S401: Obtain a first instruction and a first data table.

[0108] In this step, the first instruction is used to jointly process the first data table and the second data table.

[0109] S402: Use the abstract syntax tree to analyze the first instruction to obtain an instruction structure tree.

[0110] This step has been described in detail in the above embodiment and will not be repeated here.

[0111] S403, judging whether each node branch in the instruction structure tree meets the data security processing conditions, if so, proceeding to S404, otherwise ending the process.

[0112] In this step, the instruction structure tree is traversed to find a node with a value of select and a type of keyword, and then all branches under it are traversed with the node as the parent node, and the parent node and its child nodes with a value of from, join, group by or where and a type of keyword are ignored, and the remaining nodes are all result branches. Then, each node in the result branch is traversed, and if a node with a value of * and a type of identifier appears, it is determined that a full table query appears in the first instruction, and it is determined that the instructions represented by each node in the instruction structure tree do not meet the data security processing conditions.

[0113] If in the result branch, in addition to the node with a value of as and a type of keyword and its child nodes, there is only one node with a value of Xn.Ym and a type of identifier, then it is determined that a single-column query appears in the first instruction. These query operations will directly expose the original data, and the instructions represented by each node in the instruction structure tree are determined to not meet the data security processing conditions.

[0114] Example 1: The first instruction is a SQL instruction, and specifically the first instruction is select t0.column1, t0.column1>t1.column1 from t0, t1 where t0.id=t1.id.

[0115] Among them, the node branches belonging to the result branch include the node branch select t0.column1 and the node branch t0.column1>t1.column1. The node branch t0.column1>t1.column1 is a sub-branch of the node branch selectt0.column1. The node branch select t0.column1 indicates a full-column query, and the node branch t0.column1>t1.column1 indicates a comparison operation on two columns of data. That is, the node branches belonging to the result branch include the node branch of the full-column query and the node branch of the comparison operation. Therefore, the instruction structure tree of this instruction meets the data security processing conditions.

[0116] Example 2: The first instruction is an SQL instruction, specifically select t0.column1 from t0, t1 where t0.id = t1.id. The node branches belonging to the result branch include the node branch select t0.column1, that is, the node branches belonging to the result branch only include the node branches of the full column query, therefore, the instruction structure tree of this instruction does not meet the data security processing conditions.

[0117] S404, determine whether there is a node with a value of a grouping mark in the instruction structure tree, if not, go to S405, if yes, go to S408.

[0118] In this step, it is determined whether there is a node with a value of group by and a type of keyword in the instruction structure tree. If so, it is determined that the first instruction includes a grouping instruction.

[0119] S405: Select a node branch of a single column operation as an alternative branch from the instruction structure tree to obtain multiple alternative branches.

[0120] In this step, the instruction structure tree is traversed, nodes whose value is as and whose type is keyword and their child nodes are removed, the values ​​of the remaining nodes are pieced together into an expression, and then the expression is separated into multiple sub-expressions that can be executed concurrently. More specifically, all nodes whose node type is keyword are obtained, and the expression is split using the nodes whose type is keyword to obtain multiple sub-expressions that can be executed concurrently. Each sub-expression contains only one node of keyword type. If only one column mark is involved in the sub-expression, the common parent node of each node corresponding to the sub-expression is used as an alternative branch.

[0121] In one embodiment, the instruction structure tree is traversed to find a node whose value is where and whose type is a keyword, and then all child nodes under the branch are traversed, all nodes of each child node are parsed, and the values ​​of nodes whose type is an operator, nodes whose type is an identifier and whose value is a column tag, and nodes whose type is a constant are obtained. These three nodes constitute an instruction for filtering column data, and the three nodes can be used as alternative branches.

[0122] S406: Select at least one target branch from the multiple candidate branches, and count the values ​​of the nodes marked with columns with values ​​in the instruction structure tree.

[0123] In this step, when the expression corresponding to the alternative branch is a filter statement, the alternative branch is used as the target branch. When the expression corresponding to the alternative branch is an operation statement or an aggregation statement, and the nodes with column-marked values ​​contained in the alternative branch are different from the nodes with column-marked values ​​in the remaining node branches belonging to the result branch in the instruction structure tree, the alternative branch is used as the target branch.

[0124] S407: Process the first data table and the first instruction according to at least one target branch and the column label of the data operated by each node in the instruction structure tree to obtain a third data table and a second instruction.

[0125] In this step, the first data table and the first instruction in the plain text state are first processed according to at least one target branch, and then the processed first data table and the processed first instruction are processed according to the value of the node marked with the column value of the instruction structure tree to obtain the third data table and the second instruction.

[0126] More specifically, repeatedly process the corresponding column data in the first intermediate data table according to the expression corresponding to one of the target branches to obtain the second intermediate data table, update the intermediate instruction according to the target branch, and use the second intermediate data table as the first intermediate data table until all target branches are traversed, and use the second intermediate data table obtained in the last loop as the third data table, and use the intermediate instruction obtained in the last loop as the second instruction.

[0127] When the sub-expression represents a filter statement, the instructions corresponding to the node and the sub-node are executed first, and the first data table can be filtered, which can greatly reduce the number of rows in the first data table. When compiling the privacy calculation algorithm, the size of the data table will be used as a parameter, the amount of data in the data table will be greatly reduced, the compilation process will be faster, and the privacy calculation process will take less time. In addition, if the number of data items in the original table is very large, but the number of data items after actual filtering is very small, it can not only increase the speed of algorithm compilation, but also greatly reduce the computational overhead of ID alignment (that is, privacy intersection).

[0128] When the sub-expression is an operation statement, the column data involved is calculated according to the sub-expression, and the original column data is replaced by the calculation result. The column data in the first data table in the first instruction can be regarded as constants. If the sub-expression only involves the calculation of a single column of data in the first data table, and the single column of data is only calculated once, the calculation can be completed according to the operator priority.

[0129] If there is a node whose type is a keyword and whose value is an aggregate function, get the value of the child node of the node, that is, the name of the column involved in the aggregation. Then traverse all nodes in the instruction structure tree. If the column only appears in the result branch of the aggregate function, complete the aggregation operation on the data of the column in advance and write the result back to the column.

[0130] After the above loop operation is completed and the second intermediate data table and the intermediate instruction are obtained, the column data operated by the intermediate instruction is selected from the second intermediate data table according to the value of the node with the column mark in the instruction structure tree to obtain the third data table. The column mark in the intermediate instruction is updated to obtain the second instruction.

[0131] To obtain the third data table and the second instruction, traverse the instruction structure tree and record all nodes in the instruction structure tree whose values ​​are column tags and whose types are identifiers, that is, record the nodes whose data formats are Xm.Yn and whose types are identifiers. These nodes represent the column data involved in the first data table and the second data table.

[0132] By counting the nodes in the first data table whose values ​​are column tags and whose types are identifiers, and putting the nodes into a set, all the columns of the first data table are traversed. If the column tag of the column data is not in the set, the column is deleted in the second intermediate data table. Then the remaining column tags of the second intermediate data table and the column tags in the intermediate instructions are changed to increase in sequence from 0 according to the original order, and the third data table and the second instruction are obtained. If the amount of data in the original table is very large, and the column data required for calculation in the first instruction is very small, the number of columns in the data table can be greatly reduced by deleting the redundant columns, and the efficiency of data processing is significantly improved.

[0133] In the above technical solution, deleting redundant column data and performing data filtering when the sub-expression corresponding to the target branch is a filter statement can significantly reduce the amount of data in the first data table. Performing data operations or aggregation operations when the sub-expression corresponding to the target branch is an operation statement or an aggregation statement can reduce the number of atomic ciphertext calculations.

[0134] S408 , performing grouping processing on the first data table to obtain a third data table, and removing the grouping instruction in the first instruction to obtain a second instruction.

[0135] In this step, the instruction structure tree is traversed to find a node whose value is the grouping tag group by and whose type is a keyword, and the value of the child node of the node is obtained, that is, the column name used for grouping. If the column data is in the first data table, grouping is performed. Then the grouped elements and the number of each element are added as two columns of new elements to the data table. This avoids the need to traverse the grouping twice in the ciphertext form, which introduces a large amount of computational overhead.

[0136] After completing the above data processing steps, the third node can start privacy calculation based on the final instructions and data table.

[0137] The following example illustrates the processing of the first instruction and the first data table:

[0138] Example 3: The first instruction is: select sum(t0.column2)+sum(t1.column1)+3, min(selectt0.column1)-min(t1.column3)from t0, t1 where t0.id=t1.id and t0.Column3>2. The first data table t0 is shown in Table 1, and the second data table t1 is shown in Table 2.

[0139] Table 1 The first data table t0

[0140] ID column0 column1 column2 column3 0 0 1 2 1 1 0 1 2 1 2 0 1 2 3 3 0 1 2 3

[0141] Table 2 Second data table t1

[0142] ID column0 column1 column2 column3 0 0.1 1.1 2.1 3.1 1 0.1 1.1 2.1 3.1 2 0.1 1.1 2.1 3.1 3 0.1 1.1 2.1 3.1

[0143] By traversing the instruction structure tree, it is determined that the value of the third child node in the node branch t0.Column3>2 is 2, indicating a constant mark, the value of the first child node is t0.Column3, indicating a column identifier in the first data table, and the value of the second child node is >, indicating an operation symbol. Then, the node branch can be used as an alternative branch. Since the expression of the node branch t0.Column3>2 is a filter statement, the node branch t0.Column3>2 can be used as the target branch. The first data table is processed according to the expression corresponding to the target branch to obtain the first intermediate data table, as shown in Table 3.

[0144] Table 3 The first intermediate data table

[0145] ID column0 column1 column2 column3 2 0 1 2 3 3 0 1 2 3

[0146] Accordingly, the first instruction is updated to: select sum(t0.column2)+sum(t1.column1)+3, min(select t0.column1)-min(t1.column3)from t0, t1 where t0.id=t1.id

[0147] The column data involved in the first instruction after the statistics update are t0.column1 and t0.column0, then the two columns of data t0.column0 and t0.column3 in the first data table are deleted, and the remaining column data are renamed to obtain the third data table, as shown in Table 4.

[0148] Table 4 The third data table

[0149] ID column1 column2 2 1 2 3 1 2

[0150] The second instruction obtained is: select sum(t0.column1)+sum(t1.column1)+3, min(select t0.column0)-min(t1.Column3) from t0, t1 where t0.id=t1.id.

[0151] Example 4: The first instruction is select (t0.column0+5)*(t1.column0-6)*2+20 from t0, t1 where t0.id=t1.id. The first data table t0 is shown in Table 5, and the second data table t1 is shown in Table 6.

[0152] Table 5 The first data table t0

[0153] ID column0 0 5 1 5 2 5 3 5

[0154] Table 6 Second data table t1

[0155] ID column0 0 11 1 11 2 11 3 11

[0156] In the first node, the first node determines that the value of the third child node of the branch node t0.column0+5 is 5, which is a constant mark, and the value of the first child node is t0.Column0, which indicates the column identifier in the first data table. The branch node can be used as an alternative branch. The expression of the branch node t0.column0+5 is an operation statement. The instruction structure tree also includes branch nodes t1.column0-6. The column data operated is different. The branch node t0.column0+5 can be used as the target node, and the operation is performed according to the expression corresponding to the branch node t0.column0+5. The first data table only contains one column of data, so the second intermediate data table obtained after the operation can be used as the third data table, as shown in Table 7. The second instruction obtained is select t0.column0*(t1.column0-6)*2+20 from t0, t1 where t0.id=t1.id.

[0157] Table 7 The third data table

[0158] ID column0 0 10 1 10 2 10 3 10

[0159] In the second node, the second node determines that the value of the third child node of the branch node t1.column0-6 is 6, which is a constant mark, the value of the second child node is -, which is an operation symbol, and the value of the first child node is t1.column0, which indicates the column identifier in the first data table. The branch node can be used as an alternative branch. The expression of the branch node t1.column0-6 is an operation statement. The instruction structure tree also includes a branch node t0.column0+5. The column data operated is different, so the branch node t1.column0-6 can be used as the target node, and the operation is performed according to the expression corresponding to the branch node t1.column0-6. The second data table only contains one column of data, so the intermediate data table obtained after the operation can be used as the fourth data table, as shown in Table 8. The third instruction obtained is select (t0.column0+5)*t1.column0+20 from t0, t1 where t0.id=t1.id.

[0160] Table 8 Fourth data table

[0161] ID column0 0 10 1 10 2 10 3 10

[0162] In the third node, after receiving the first processing request sent by the first node and the second processing request sent by the second node, the third node parses and obtains the second instruction and the third instruction, merges the second instruction and the third instruction to obtain the fourth instruction, and processes the third data table and the fourth data table in ciphertext form according to the fourth instruction. The fourth instruction is: select t0.column0*t1.column0+20 from t0, t1 where t0.id=t1.id.

[0163] Example 5: The first instruction is select max(t0.column0)-min(t1.column0) from t0, t1. The first data table t0 is shown in Table 9, and the second data table t1 is shown in Table 10.

[0164] Table 9 First data table

[0165] ID column0 0 5 1 6 2 7 3 8

[0166] Table 10 Second data table

[0167] ID column0 0 1 1 1 2 2 3 3

[0168] In the first node, by traversing the instruction structure tree, it is determined that the value of the child node in the node branch max (t0.column0) is t0.Column0, which represents the column identifier in the first data table. Then the node branch can be used as an alternative branch. The expression of the node branch max (t0.column0) is an aggregation statement. The instruction structure tree also includes the branch node min (t1.column0). The column data operated is different. Then the node branch max (t0.column0) can be used as the target branch. The first data table is processed according to the expression corresponding to the target branch to obtain the second intermediate data table. Since the first data table only contains one column of data, the second intermediate data table obtained after the operation can be used as the third data table, as shown in Table 11.

[0169] Table 11 The third data table

[0170] ID column0 0 8

[0171] In the second node, by traversing the instruction structure tree, it is determined that the value of the child node in the node branch min(t1.column0) is t1.column0, which represents the column identifier in the second data table. Then the node branch can be used as an alternative branch. The expression of the node branch is the aggregate statement min(t1.column0). The instruction structure tree also includes the branch node max(t0.column0). The column data operated on is different. Then the node branch min(t1.column0) can be used as the target branch. The second data table is processed according to the expression corresponding to the target branch to obtain an intermediate data table. Since the second data table only contains one column of data, the intermediate data table obtained after the operation can be used as the fourth data table, as shown in Table 12.

[0172] Table 12 Fourth data table

[0173] ID column0 0 1

[0174] The fourth instruction obtained by the third node after the fusion processing is: select t0.column0-t1.column0from t0, t1.

[0175] Example 6: The first instruction is: select max(t0.column0)from t0 join t1 on t0.id=t1.id group by t1.column0.

[0176] After completing the ID alignment, the first node calculates the grouping situation based on t1.column0, obtains auxiliary information for subsequent privacy calculation, and adds it to the first data table, as shown in Table 13.

[0177] Table 13 First data table

[0178]

[0179]

[0180] like Figure 5 As shown, an embodiment of the present application provides a data processing device, the device comprising:

[0181] The acquisition module 501 is used to acquire a first instruction and a first data table; wherein the first instruction is used to process the first data table and the second data table, the first data table is stored in the first node, and the second data table is stored in the second node;

[0182] The processing module 502 is used to analyze the first instruction using an abstract syntax tree to obtain an instruction structure tree; when it is determined that each node branch in the instruction structure tree meets the data security processing condition and there is no node whose value is a grouping mark in the instruction structure tree, a node branch of a single column operation is selected from the instruction structure tree as an alternative branch to obtain multiple alternative branches;

[0183] The processing module 502 is used to select at least one target branch from multiple alternative branches, and to count the values ​​of the nodes whose values ​​are column marks in the instruction structure tree. According to the values ​​of at least one target branch and the nodes whose values ​​are column marks in the instruction structure tree, the first data table and the first instruction in the plain text state are processed to obtain a third data table and a second instruction.

[0184] In one embodiment, the processing module 502 is further configured to:

[0185] generating a first processing request according to the second instruction and the encrypted third data table, and sending the first processing request to the third node;

[0186] Among them, the third node is also used to receive the second processing request sent by the second node, and to perform data processing according to the first processing request and the second processing request, and send the processing results to the first node and the second node; the second processing request is generated by the second node according to the third instruction and the encrypted fourth data table, and the third instruction and the fourth data table are obtained by processing the second data table and the first instruction.

[0187] In one embodiment, determining whether each node branch in the instruction structure tree meets the data security processing condition specifically includes any one of the following:

[0188] There is no node branch representing full table query in the instruction structure tree;

[0189] There is no node branch representing full column query in the instruction structure tree;

[0190] The instruction structure tree includes a first node branch representing a single-column query and other second node branches belonging to result branches; wherein the second node branch is not a node branch representing a name.

[0191] In one embodiment, the processing module 502 is specifically configured to:

[0192] Each node branch in the instruction structure tree is traversed. When the expression corresponding to the node branch is a filter statement, an operation statement or an aggregation statement, and the node branch contains only one node whose value is a column mark, the node branch is used as an alternative branch.

[0193] In one embodiment, the processing module 502 is specifically configured to:

[0194] When the expression corresponding to the alternative branch is a filter statement, the alternative branch is used as the target branch;

[0195] When the expression corresponding to the alternative branch is an operation statement or an aggregate statement, and the nodes whose values ​​are column-marked contained in the alternative branch are different from the nodes whose values ​​are column-marked in the remaining node branches belonging to the result branch in the instruction structure tree, the alternative branch is used as the target branch.

[0196] In one embodiment, the processing module 502 is specifically configured to:

[0197] Repeat the process of processing the corresponding column data in the first data table in the plain text state according to the expression corresponding to one of the target branches to obtain the first intermediate data table, and update the first instruction according to the target branch, and use the first intermediate data table as the first data table, until all target branches are traversed, and use the first intermediate data table obtained in the last cycle as the second intermediate data table, and use the updated first instruction obtained in the last cycle as the intermediate instruction;

[0198] According to the value of the node whose value is the column mark in the instruction structure tree, the column data operated by the intermediate instruction is selected from the second intermediate data table to obtain the third data table; and the column mark in the intermediate instruction is updated to obtain the second instruction.

[0199] In one embodiment, the traversal order of the target branches is the target branch whose expression is a filter statement, the target branch whose expression is a calculation statement, and the target branch whose expression is an aggregation statement.

[0200] In one embodiment, the processing module 502 is further configured to:

[0201] When it is determined that each node in the instruction structure tree cannot meet the data security processing conditions, and there are nodes whose values ​​are group marks in the instruction structure tree, the first data table in the plain text state is grouped to obtain a third data table, and the first instruction is processed according to the nodes whose values ​​are group marks to obtain a second instruction.

[0202] like Figure 6 As shown, an embodiment of the present application provides an electronic device 600 , which includes a memory 601 and a processor 602 .

[0203] The memory 601 is used to store computer instructions executable by the processor;

[0204] The processor 602 implements each step of the method in the above embodiment when executing the computer instruction. For details, please refer to the relevant description in the above method embodiment.

[0205] Optionally, the memory 601 can be independent or integrated with the processor 602. When the memory 601 is independently provided, the electronic device further includes a bus for connecting the memory 601 and the processor 602.

[0206] An embodiment of the present application further provides a computer-readable storage medium, in which computer instructions are stored. When a processor executes the computer instructions, each step of the method in the above embodiment is implemented.

[0207] The embodiment of the present application also provides a computer program product, including computer instructions, which implement the various steps of the method in the above embodiment when executed by a processor.

[0208] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention disclosed herein. The present application is intended to cover any modification, use or adaptation of the present application, which follows the general principles of the present application and includes common knowledge or customary techniques in the art that are not disclosed in the present application. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present application are indicated by the following claims.

[0209] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A data processing method, characterized in that: The method is applied to a first node, and the method comprises: Obtain a first instruction and a first data table; wherein the first instruction is used to process the first data table and the second data table, the first data table is stored in the first node, and the second data table is stored in the second node; The first instruction is analyzed using an abstract syntax tree to obtain an instruction structure tree; when it is determined that each node branch in the instruction structure tree meets the data security processing condition and there is no node whose value is a grouping mark in the instruction structure tree, a node branch of a single column operation is selected from the instruction structure tree as an alternative branch to obtain multiple alternative branches; Selecting at least one target branch from the multiple candidate branches, and counting the values ​​of the nodes whose values ​​are column marks in the instruction structure tree, and processing the first data table and the first instruction in a plain text state according to the at least one target branch and the values ​​of the nodes whose values ​​are column marks in the instruction structure tree, to obtain a third data table and a second instruction; Generate a first processing request according to the second instruction and the encrypted third data table, and send the first processing request to the third node; The third node is further used to receive a second processing request sent by the second node, and to perform data processing according to the first processing request and the second processing request, and to send the processing result to the first node and the second node; the second processing request is generated by the second node according to a third instruction and an encrypted fourth data table, and the third instruction and the fourth data table are obtained by processing the second data table and the first instruction; The first data table and the first instruction in a plaintext state are processed according to the at least one target branch and the value of the node whose value is a column mark of the instruction structure tree to obtain the third data table and the second instruction, specifically including: Repeat the process of processing the corresponding column data in the first data table in the plain text state according to the expression corresponding to one of the target branches to obtain a first intermediate data table, and update the first instruction according to the target branch, and use the first intermediate data table as the first data table, until all target branches are traversed, and use the first intermediate data table obtained in the last cycle as the second intermediate data table, and use the updated first instruction obtained in the last cycle as the intermediate instruction; According to the value of the node whose value is the column mark in the instruction structure tree, the column data operated by the intermediate instruction is selected from the second intermediate data table to obtain a third data table; and the column mark in the intermediate instruction is updated to obtain the second instruction.

2. The data processing method according to claim 1, characterized in that: Determining whether each node branch in the instruction structure tree meets the data security processing conditions specifically includes any of the following: There is no node branch representing full table query in the instruction structure tree; There is no node branch representing a full-column query in the instruction structure tree; The instruction structure tree includes a first node branch representing a single-column query and other second node branches belonging to result branches; wherein the second node branch is not a node branch representing a name.

3. The data processing method according to claim 2, characterized in that: Selecting a node branch of a single column operation as an alternative branch from the instruction structure tree to obtain multiple alternative branches specifically includes: Each node branch in the instruction structure tree is traversed, and when the expression corresponding to the node branch is a filter statement, an operation statement or an aggregation statement, and the node branch contains only one node whose value is a column mark, the node branch is used as the alternative branch.

4. The data processing method according to claim 3, characterized in that: Selecting at least one target branch from the multiple candidate branches; specifically comprising: When the expression corresponding to the alternative branch is a filter statement, the alternative branch is used as the target branch; When the expression corresponding to the alternative branch is an operation statement or an aggregation statement, and the nodes with column-marked values ​​contained in the alternative branch are different from the nodes with column-marked values ​​in the remaining node branches belonging to the result branch in the instruction structure tree, the alternative branch is used as the target branch.

5. The data processing method according to claim 1, characterized in that: The traversal order of the target branches is the target branch whose expression is a filter statement, the target branch whose expression is a calculation statement, and the target branch whose expression is an aggregation statement.

6. The data processing method according to claim 1, characterized in that: The method further comprises: When it is determined that each node in the instruction structure tree cannot meet the data security processing conditions, and there are nodes with values ​​as group marks in the instruction structure tree, the first data table in the plain text state is grouped to obtain a third data table, and the first instruction is processed according to the nodes with values ​​as group marks to obtain the second instruction.

7. An electronic device, characterized in that: include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory to implement the data processing method according to any one of claims 1 to 6.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which are used to implement the data processing method according to any one of claims 1 to 6 when executed by a processor.

Images