Data processing method, system, engine, device and storage medium

Abstract

Embodiments of the present application provide a data processing method, system, engine, device and storage medium. The method includes: processing the data processing logic in the data processing engine to obtain an execution plan. The execution plan contains multiple basic operators. In order to ensure the data security of the operator during the execution of the data calculation process, a trusted execution plan composed of trusted operators is generated based on the calling relationship between the basic operator and the trusted execution environment. Then, the trusted operator calls the trusted execution environment, and performs corresponding data processing on the data to be processed in the trusted execution environment. The trusted operator is used to call the trusted execution environment, and the trusted operator performs calculations in the trusted execution environment to ensure data security. Using the trusted operator as the basic unit for calling the trusted execution environment can not only meet data security, but also effectively reduce the number of calls to the trusted execution environment, thereby effectively reducing the workload of the data processing engine and improving the working performance of the data processing engine.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to data processing methods, systems, engines, devices, and storage media. Background Art

[0002] With the rapid development of computer technology, the security of data processing has received more and more attention.

[0003] The development of trusted execution environment (TEE) technology has made full-chain encryption of data possible. Some data processing engines also invoke TEEs to ensure data security and perform data processing tasks such as calculations within them. During data processing, invoking and encrypting the data within the TEE results in significant system overhead. This can severely impact processing efficiency, especially when processing large amounts of data. Summary of the Invention

[0004] To solve or improve the problems existing in the prior art, the embodiments of the present application provide data processing methods, systems, engines, devices and storage media.

[0005] In one aspect, in one embodiment of the present application, a data processing method is provided. The method includes:

[0006] Determine an execution plan corresponding to the data processing logic; the execution plan includes basic operators;

[0007] generating a trusted execution plan composed of trusted operators according to the calling relationship between the basic operators and the trusted execution environment;

[0008] The received data to be processed is processed based on at least one of the trusted operators in the trusted execution plan and the trusted execution environment called by the trusted operator.

[0009] In a second aspect, in one embodiment of the present application, a data processing system is provided, the system comprising:

[0010] The client is used to send data processing requests to the data processing system;

[0011] A data processing engine is used to implement a data processing method described in the first aspect.

[0012] In a third aspect, in one embodiment of the present application, a data processing engine is provided, which includes: an execution plan production module, a trusted execution plan production module and a data processing module; the engine is used to implement a data processing method described in the first aspect.

[0013] In a fourth aspect, in one embodiment of the present application, an electronic device is provided, including a memory and a processor; wherein,

[0014] The memory is used to store programs;

[0015] The processor is coupled to the memory and is used to execute the program stored in the memory to implement the data processing method described in the first aspect.

[0016] In the fifth aspect, in one embodiment of the present application, a non-transitory machine-readable storage medium is provided, on which executable code is stored. When the executable code is executed by a processor of an electronic device, the processor executes a data processing method as described in the first aspect.

[0017] The technical solution provided by the embodiment of the present application processes the data processing system pre-processed in the data processing engine to obtain an execution plan. Among them, the execution plan contains multiple basic operators. In order to ensure the data security of the operator when executing the data calculation process, it is necessary to generate a trusted operator and a trusted execution plan composed of the trusted operators based on the calling relationship between the basic operator and the trusted execution environment. Then, the trusted operator calls the trusted execution environment, and performs corresponding data processing on the data to be processed in the trusted execution environment. Through the above solution, the trusted operator is used to call the trusted execution environment, which can ensure that the trusted operators are executing the calculation process in the trusted execution environment and ensure data security. In addition, using the trusted operator as the basic unit for calling the trusted execution environment can not only meet data security, but also effectively reduce the number of calls to the trusted execution environment, thereby effectively reducing the workload of the data processing engine and improving the working performance of the data processing engine. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0019] Figure 1 The process intention of the data processing method provided in the embodiment of the present application;

[0020] Figure 2 A schematic diagram of the process of generating a trusted execution plan for an example embodiment of the present application;

[0021] Figure 3A schematic diagram of a trusted execution plan including multiple operators, illustrated in an embodiment of the present application;

[0022] Figure 4 A schematic diagram illustrating the process of executing a computational process by a fused trusted operator, as exemplified in an embodiment of the present application;

[0023] Figure 5 A schematic diagram illustrating a trusted operator performing a computational process according to an embodiment of the present application;

[0024] Figure 6 A schematic diagram of the data processing process of multiple trusted operators provided in an embodiment of the present application;

[0025] Figure 7 A schematic diagram of the structure of a data processing system provided in an embodiment of the present application;

[0026] Figure 8 A schematic diagram of the structure of the data processing engine provided in an embodiment of the present application;

[0027] Figure 9 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0028] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.

[0029] In some processes described in the specification, claims and the above-mentioned figures of this application, multiple operations appearing in a specific order are included. These operations may not be executed in the order in which they appear in this document or may be executed in parallel. The serial numbers of the operations, such as 101, 102, etc., are only used to distinguish between different operations, and the serial numbers themselves do not represent any order of execution. In addition, these processes may include more or fewer operations, and these operations may be executed in sequence or in parallel. It should be noted that the descriptions of "first", "second", etc. in this document are used to distinguish different messages, devices, modules, etc., and do not represent a sequence, nor do they limit "first" and "second" to different types. In addition, the embodiments described below are only some of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without making creative work are within the scope of protection of this application.

[0030] With the rapid development of database technology, users have increasingly higher requirements for database security when using databases. For some data with relatively high encryption requirements, a fully encrypted database can be used to achieve comprehensive encryption protection for all aspects of data transmission, calculation, and storage. In actual applications, although more comprehensive security protection can be achieved for the data in the user database, if the data is securely processed in a trusted execution environment, the performance of the data processing engine will be greatly reduced, seriously affecting its normal working efficiency. Therefore, there is a need for a technical solution that can improve the working performance of the data processing engine when securely processing data. In the technical solution of this application, the specific working process will be described in the following embodiments.

[0031] Explanation of terms:

[0032] Trusted Execution Environment (TEE) is a secure area within the processor that ensures the confidentiality and integrity of the code and data loaded within it. It provides an isolated execution space (enclave) where trusted programs running within it are protected from intrusion by standard applications, the operating system (OS), and even the hypervisor.

[0033] Data computing engine: A general term for a type of specialized or general-purpose data processing system. Users typically call it by writing SQL or scripts. The system will complete the task logic analysis and task processing, as well as data processing.

[0034] Execution plan: The data processing engine converts a user-defined task into a series of task flows for internal calculations. This task flow can be a series of simple tasks, a tree, or a directed acyclic graph.

[0035] Operator: The basic unit of execution plan in a computing engine. For example, in databases, there are operators such as aggregation, join, and filter, while in AI systems, there are operators such as matrix multiplication and matrix transposition.

[0036] like Figure 1 The process intention of the data processing method provided in the embodiment of the present application. The execution subject of the method can be a data processing engine, and the method is executed in a trusted execution environment (TEE). The execution environment usually protects the code and data therein from being leaked or maliciously tampered by providing a secure execution environment isolated from the outside world. The data processing method specifically includes the following steps:

[0037] Step 101: Determine an execution plan corresponding to the data processing logic; the execution plan includes basic operators.

[0038] Step 102: Generate a trusted execution plan consisting of trusted operators based on the calling relationship between the basic operators and the trusted execution environment.

[0039] Step 103: Based on at least one of the trusted operators in the trusted execution plan and the trusted execution environment called by the trusted operator, data processing is performed on the received data to be processed.

[0040] In a data processing system, users write SQL statements, operator logic, and dependency graphs as needed. Based on these SQL statements, operator logic, dependency graphs, and other data processing logic, corresponding execution plans are generated. For example, users can use the EXPLAIN command to view the logical execution plan generated by the optimizer for a given SQL statement.

[0041] The trusted execution plan includes trusted operators, which serve as the basic unit for invoking the trusted environment in subsequent data processing tasks. Generally speaking, a trusted execution plan includes multiple trusted operators, each of which has dependencies and an execution order. It should be noted that the dependencies between trusted operators can be linear, tree-like, or directed acyclic graph-like.

[0042] In practical applications, operators (basic operators and trusted operators) can only perform computations on plaintext data when executing computational tasks. If the data being processed is not plaintext, it must be decrypted to obtain the plaintext before the corresponding computational processing can be performed. Since data is displayed in plaintext during the operator's computational tasks, to ensure data security during the data processing process, the corresponding computational tasks must be executed in a trusted execution environment (TEE). Existing solutions typically call the TEE on a function-by-function basis, ensuring the security of the plaintext data during the execution of each function. If an operator contains multiple functions, each function must call the TEE separately during execution. Each call requires performing operations such as data parsing, serialization, computation, encryption, and compression. The more frequent and frequent calls to the TEE, the greater the system overhead. Excessively frequent and intensive calls can even impact system performance. In this application, a trusted execution plan consisting of trusted operators is generated by determining the calling relationship between basic operators and the trusted execution environment. The trusted execution environment is then called using the trusted operators as the basic unit. Upon receiving a data processing request or receiving data to be processed, the data is processed according to the trusted execution plan. Specifically, based on the execution order of the trusted operators in the trusted execution plan, each trusted operator will call the trusted execution environment as needed to complete the relevant data processing task.

[0043] Unlike the existing technology that uses trusted operator internal functions as the basic unit for calling the trusted execution environment, this method uses trusted operators as the basic unit for calling the trusted execution environment, which can effectively reduce the number of calls to the trusted execution environment, reduce system overhead, and improve system performance when the system securely processes data in the trusted execution environment.

[0044] In one or more embodiments of the present application, generating a trusted execution plan composed of trusted operators based on the calling relationship between the basic operators and the trusted execution environment includes: obtaining the basic operators contained in the execution plan; generating trusted operators based on the calling relationship between the basic operators and the trusted execution environment; generating the trusted execution plan based on the trusted operators and operator dependencies.

[0045] For example, Figure 2 The following is a schematic diagram of the process of generating a trusted execution plan for the embodiment of the present application. Figure 2As shown in the figure, the basic operators in the execution plan include: Aggregation, Filter, and Scan. To ensure secure and reliable data processing, these basic operators can be converted into trusted operators, resulting in the trusted operators Agg, Filter, and Scan shown in the figure. When generating a trusted execution plan, the conversion is performed on an operator-by-operator basis, without affecting the dependencies between operators. The dependencies between operators remain unchanged during the conversion process. This results in a trusted execution plan that uses operators as the basic unit for invoking the trusted execution environment. This ensures secure and reliable data processing while effectively reducing the number of calls to the trusted execution environment, thereby avoiding excessive system performance overhead caused by calls to the trusted execution environment.

[0046] In one or more embodiments of the present application, after generating a trusted operator based on the calling relationship between the basic operator and the trusted execution environment, it also includes: judging whether the trusted operators are allowed to be fused based on the operator dependency relationship and / or operator type between the trusted operators; if fusion is allowed, generating a fused trusted operator.

[0047] Operator dependencies refer to the execution order of trusted operators when executing data processing tasks using a trusted execution plan. This refers to the fixed execution order of several trusted operators, or the close relationship between the input and output data of several trusted operators. Some operator types require other trusted operators to complete the computational task, indicating that these trusted operators can be integrated with other trusted operators.

[0048] The idea behind operator fusion is to merge at least two dependent operators into a single, integrated operator whenever possible. This not only maximizes CPU continuity, improving CPU utilization and efficiency, but also allows the fused trusted operator, consisting of multiple trusted operators, to call the trusted execution environment as a single, integrated operator. This effectively reduces the number of calls to the trusted execution environment and avoids excessive system overhead.

[0049] It should be noted that when performing operator fusion, each basic operator must first be converted into a trusted operator, and then at least two trusted operators must be fused to form a fused trusted operator. Furthermore, due to the limited space of the trusted execution environment, unlimited fusion is not possible. The number of fused operators or the number of functions contained in the fused operator must be limited.

[0050] In one or more embodiments of the present application, generating the trusted execution plan based on the trusted operator and the operator dependency includes: generating the trusted execution plan based on the operator dependency between the trusted operator and the fused trusted operator, and / or based on the operator dependency between the trusted operator and the trusted operator, and / or based on the operator dependency between the fused trusted operator and the fused trusted operator.

[0051] like Figure 3 This is a schematic diagram of a trusted execution plan including multiple operators, as illustrated in the embodiment of this application. Figure 3 As can be seen in the figure, the trusted execution plan includes multiple trusted operators and a fused trusted operator. The dependencies between trusted operators and fused trusted operators are determined based on the dependencies between the operators before fusion. When a trusted execution plan includes a fused trusted operator, each operator is executed sequentially according to their execution order. When a fused trusted operator includes multiple operators, they are executed sequentially according to the execution order or dependencies between the operators. When calling the trusted execution environment, the fused trusted operator is called once as a basic unit, while each unfused trusted operator is called individually. The fused trusted execution plan requires fewer calls to the trusted execution environment than the unfused execution plan. As a result, while the trusted execution plan and the execution plan have the same data security protection, the trusted execution plan has higher data processing efficiency than the execution plan.

[0052] For example, if there is a query that needs to obtain the number of rows in a certain column of a table with a value greater than 5, you can use Figure 2 The operator shown in the figure can first use the trusted operator scan to scan the data on disk. Then, the filter operator Filter can be used to filter the data in the scan results, for example, filtering for values ​​greater than 5. Finally, the trusted operator Agg can be used to calculate the number of remaining rows. During the execution of these trusted operators, each requires a call to the trusted execution environment. This ensures that the trusted execution plan completes after three calls to the trusted execution environment, ensuring data security while effectively reducing the number of calls to the trusted execution environment.

[0053] It's important to note that the execution order of trusted operators in a trusted execution plan can be understood as the order in which a row of data is processed by trusted operator A and then by trusted operator B. Once the execution order is clarified, the dependencies between trusted operators can be further clarified, which helps determine whether trusted operator fusion is permitted. If the number of functions contained in the fused trusted operator exceeds a threshold, other trusted operators are prohibited from participating in the fusion.

[0054] In one or more embodiments of the present application, the processing of the received data to be processed based on at least one of the trusted operators in the trusted execution plan and the trusted execution environment called by the trusted operator includes:

[0055] Inputting the data to be processed into the trusted operator and / or the fused trusted operator;

[0056] The trusted operator and / or the fused trusted operator calls the trusted execution environment to perform secure processing on the data to be processed, thereby obtaining a plurality of data elements or a plurality of rows of data;

[0057] The trusted operator and / or the fused trusted operator is used to perform data processing on the encrypted data block to obtain an encryption processing result.

[0058] It should be noted that the secure processing of the data to be processed mentioned here refers to parsing and decrypting the data to be processed in a trusted execution environment, thereby obtaining data elements, multiple lines of data, etc. in plain text.

[0059] As mentioned above, when performing data calculations, trusted operators and / or fused trusted operators can only perform calculations and processing on data elements, row data, etc. in plain text within a trusted execution environment. After receiving the data to be processed, decryption processing is performed within the trusted execution environment, and after data processing is completed, encryption processing is required. Some traditional solutions use data elements as the basic unit for encryption. Therefore, when decrypting encrypted data within a trusted execution environment, it is also necessary to decrypt each data element one by one. If the number of data elements is large, it will take a lot of time to decrypt each one, as well as the subsequent encryption work, which will affect the performance of the data processing system. Therefore, in the solution of the present application, the data to be processed can be a whole block of data containing multiple data elements and row data. When decrypting, the whole block of data can be decrypted to obtain multiple data elements and row data, avoiding decryption of each data element and row data one by one. The above solution can effectively improve the efficiency of secure data processing in a trusted execution environment and enhance the system performance of the data processing system.

[0060] In one or more embodiments of the present application, the fused trusted operator includes: a first trusted operator and a second trusted operator;

[0061] The fusion trusted operator calls the trusted execution environment to securely process the data to be processed to obtain multiple data elements or multiple rows of data, including: decrypting and parsing the data to be processed in the trusted execution environment to obtain multiple data elements or multiple rows of data.

[0062] The fused trusted operator is used to perform data processing on the data to be processed to obtain an encrypted processing result, including: using the first basic operator in the fused trusted operator to perform calculation processing on the multiple data elements or multiple rows of data to obtain a first intermediate result; sending the first intermediate result to the second basic operator to perform calculation processing on the first intermediate result to generate an encrypted processing result.

[0063] For example, if Figure 4 This is a schematic diagram of the process of performing calculation processing by the fusion trusted operator as an example of the embodiment of this application. Figure 4 As can be seen in the figure, the fused trusted operator includes a first trusted operator and a second trusted operator. After the data to be processed is input into the trusted operator, the data to be processed is decrypted and parsed in the trusted execution environment to obtain a first intermediate result containing multiple data elements and / or multiple row data. Without exiting the trusted execution environment, the second trusted operator that has a dependency relationship with the first trusted operator continues to perform corresponding calculations on the first intermediate result. The calculation result obtained is also composed of multiple data elements and / or row data. When all the trusted operators in the fused trusted operator have completed the calculation tasks, the encrypted processing results will be output. Before output, multiple data elements and / or row data need to be lightweight encrypted. Among them, the specific process of lightweight encryption processing is as follows:

[0064] Lightweight serialization and compression are performed on the multiple data elements or multiple rows of data included in the calculation result of the fused trusted operator to generate encrypted data blocks.

[0065] For example, after the second trusted operator calculates and processes, data element A1, data element B1, and data element C1 are obtained. In order to reduce the workload of subsequent trusted operators or fused trusted operators in decrypting and parsing data element A1, data element B1, and data element C1, data element A1, data element B1, and data element C1 are lightweight serialized, compressed, and assembled to obtain block data, and then encryption processing is performed on the block data to obtain a whole encrypted data block D1 (it should be noted that in actual applications, the encrypted data block obtained can be one or more, and the generation of an encrypted database D1 here is only for illustration and does not constitute a limitation on the technical solution of the present application). Then, the whole encrypted data block in an encrypted state is sent to the next trusted operator. Since the whole encrypted data block is in an encrypted state, it will not affect the data security protection effect even when it is transmitted.

[0066] In one or more embodiments of the present application, the fused trusted operator calls the trusted execution environment to securely process the data to be processed to obtain multiple data elements or multiple rows of data, including: decrypting and parsing the data to be processed in the trusted execution environment to obtain multiple data elements or multiple rows of data.

[0067] The fused trusted operator is used to perform data processing on the multiple data elements or multiple rows of data to obtain an encryption processing result, including: using the trusted operator to perform calculation processing on the multiple data elements or multiple rows of data to generate an encryption processing result.

[0068] When computing a trusted operator that has not been fused, the processing process is as follows: Figure 5 As shown. Figure 5 This is a schematic diagram of the process of a trusted operator performing a calculation process as illustrated in the embodiment of this application. Figure 5 As can be seen in the figure, the trusted operator performs data decryption and data parsing in the trusted execution environment, and then uses the trusted operator to perform calculations on the obtained multiple data elements or multiple rows of data.

[0069] The multiple data elements or multiple rows of data included in the calculation result of the trusted operator are lightweight serialized, compressed, and encrypted to generate encrypted data blocks.

[0070] For example, after computational processing by a trusted operator, data elements A2, B2, and C2 are obtained. To reduce the workload of subsequent trusted operators or fused trusted operators in decrypting and parsing data elements A2, B2, and C2, data elements A2, B2, and C2 are serialized, compressed, and assembled into blocks of data. These blocks of data are then encrypted to obtain a complete encrypted data block D2. This encrypted block is then sent to the next trusted operator. Because the encrypted block remains encrypted, data security protection is not compromised even during transmission.

[0071] In one or more embodiments of the present application, after obtaining the encryption result, it also includes: sending the encryption processing result to the corresponding trusted operator or the corresponding fusion trusted operator according to the operator dependency relationship; so that the corresponding trusted operator or the corresponding fusion trusted operator performs data processing on the encryption processing result in the called trusted execution environment.

[0072] like Figure 6 Schematic diagram of the data processing process of multiple trusted operators provided in the embodiment of this application. Figure 6 As can be seen in the figure, after trusted operator A / fused trusted operator A completes the computational task, it outputs the encrypted processing result. Since the encryption processing result is an encrypted data block, it should be emphasized that the encryption processing result can be a whole encrypted data block or multiple encrypted data blocks. The encrypted data block contains multiple data elements and row data. Therefore, trusted operator B / fused trusted operator B can parse and decrypt the whole encrypted data block to obtain multiple data elements and / or row data. Trusted operator B / fused trusted operator B then performs computation on the multiple data elements and / or row data, serializes and compresses the computational processing result to obtain a whole block (which can be one or more blocks) of encrypted data, and continues to send it to the next trusted operator or fused trusted operator according to the above steps until the entire trusted execution plan is completed. Based on the above scheme, in the trusted execution plan, using a trusted operator or a fusion trusted operator containing multiple trusted operators as the basic unit for calling the trusted execution environment can effectively reduce the number of calls to the trusted execution environment during the execution of data processing tasks, as well as the number of repeated parsing, deserialization, encryption, serialization and other data processing in the trusted execution environment, thereby reducing the workload of the data processing system and improving data processing performance and efficiency.

[0073] Based on the same idea, the embodiment of the present application also provides a data processing system. Figure 7 This is a schematic diagram of the structure of the data processing system provided in the embodiment of the present application. Figure 7 As can be seen in the figure, the system includes:

[0074] The client 71 is used to send a data processing request to the data processing system.

[0075] The data processing engine 72 is used to determine the execution plan corresponding to the data processing logic in response to the data processing request provided by the client; the execution plan includes basic operators; based on the calling relationship between the basic operators and the trusted execution environment, a trusted execution plan composed of trusted operators is generated; based on at least one of the trusted operators in the trusted execution plan and the trusted execution environment called by the trusted operator, data processing is performed on the received data to be processed.

[0076] In addition, the data processing engine 72 is also used to obtain the basic operators included in the execution plan;

[0077] generating a trusted operator based on a calling relationship between the basic operator and the trusted execution environment;

[0078] The trusted execution plan is generated according to the trusted operators and operator dependencies.

[0079] The data processing engine 72 is further configured to determine whether the trusted operators are allowed to be merged based on the operator dependency relationship and / or operator type between the trusted operators;

[0080] If fusion is allowed, a fusion trusted operator is generated.

[0081] The data processing engine 72 is further configured to: based on the operator dependency relationship between the trusted operator and the fused trusted operator, and / or

[0082] Based on the operator dependency relationship between the trusted operators, and / or,

[0083] Based on the operator dependency relationship between the fused trusted operator and the fused trusted operator,

[0084] Generate the trusted execution plan.

[0085] The data processing engine 72 is further configured to input the data to be processed into the trusted operator and / or the fused trusted operator;

[0086] The trusted operator and / or the fused trusted operator calls the trusted execution environment to perform secure processing on the data to be processed, thereby obtaining a plurality of data elements or a plurality of rows of data;

[0087] The trusted operator and / or the fused trusted operator is used to perform data processing on the multiple data elements or multiple rows of data to obtain an encryption processing result.

[0088] Optionally, the fused trusted operator includes: a first trusted operator and a second trusted operator;

[0089] The fusion trusted operator calls the trusted execution environment to perform secure processing on the data to be processed, and obtains multiple data elements or multiple rows of data, including:

[0090] Decrypting and parsing the data to be processed in the trusted execution environment to obtain multiple data elements or multiple lines of data;

[0091] The fused trusted operator is used to perform data processing on the data to be processed to obtain an encryption processing result, including:

[0092] Using the first basic operator in the fused trusted operator to perform computation on the multiple data elements or multiple rows of data to obtain a first intermediate result;

[0093] The first intermediate result is sent to the second basic operator to perform calculation processing on the first intermediate result to generate an encrypted processing result.

[0094] The data processing engine 72 is further configured to decrypt and parse the data to be processed in the trusted execution environment to obtain multiple data elements or multiple rows of data;

[0095] Using the trusted operator to perform data processing on the multiple data elements or multiple rows of data to obtain an encryption processing result, including:

[0096] The trusted operator is used to perform calculation processing on the multiple data elements or multiple rows of data to generate an encryption processing result.

[0097] The data processing engine 72 is also used to send the encryption processing result to the corresponding trusted operator or the corresponding fusion trusted operator according to the operator dependency relationship; so that the corresponding trusted operator or the corresponding fusion trusted operator can perform data processing on the encryption processing result in the called trusted execution environment.

[0098] The data processing engine 72 is further configured to perform lightweight serialization, compression, and encryption on the multiple data elements or multiple rows of data included in the fused trusted operator or the trusted operator calculation processing result to generate encrypted data blocks.

[0099] Based on this system, the data processing system pre-installed in the data processing engine is processed to obtain an execution plan. The execution plan includes multiple basic operators. In order to ensure data security when operators execute data calculation processes, it is necessary to generate trusted operators and a trusted execution plan composed of trusted operators based on the calling relationship between the basic operators and the trusted execution environment. Furthermore, the trusted operators call the trusted execution environment, and perform corresponding data processing on the data to be processed in the trusted execution environment. Through the above-mentioned scheme, by using trusted operators to call the trusted execution environment, it is possible to ensure that all trusted operators execute calculation processes in the trusted execution environment, thereby ensuring data security. Furthermore, by using trusted operators as the basic unit for calling the trusted execution environment, while ensuring data security, the number of calls to the trusted execution environment can be effectively reduced, thereby effectively reducing the workload of the data processing engine and improving the performance of the data processing engine.

[0100] Based on the same idea, the embodiment also provides a data processing engine. Figure 8 This is a schematic diagram of the structure of the data processing engine provided in the embodiment of the present application. Figure 8 As can be seen in the figure, the engine includes:

[0101] The execution plan generation module 81 is used to determine the execution plan corresponding to the data processing logic; the execution plan includes basic operators.

[0102] The trusted execution plan generating module 82 is configured to generate a trusted execution plan composed of trusted operators according to the calling relationship between the basic operators and the trusted execution environment.

[0103] The data processing module 83 is configured to process the received data to be processed based on at least one of the trusted operators in the trusted execution plan and the trusted execution environment called by the trusted operator.

[0104] A trusted execution plan generation module 82 is used to obtain basic operators included in the execution plan;

[0105] generating a trusted operator based on a calling relationship between the basic operator and the trusted execution environment;

[0106] The trusted execution plan is generated according to the trusted operators and operator dependencies.

[0107] Optionally, the trusted execution plan generating module 82 is further configured to determine whether the trusted operators are allowed to be merged based on the operator dependency relationship and / or operator type between the trusted operators;

[0108] If fusion is allowed, a fusion trusted operator is generated.

[0109] Optionally, the trusted execution plan generating module 82 is further configured to generate a trusted execution plan based on the operator dependency relationship between the trusted operator and the fused trusted operator, and / or

[0110] Based on the operator dependency relationship between the trusted operators, and / or,

[0111] Based on the operator dependency relationship between the fused trusted operator and the fused trusted operator,

[0112] Generate the trusted execution plan.

[0113] Optionally, the trusted execution plan generating module 82 is further configured to input the data to be processed into the trusted operator and / or the fused trusted operator;

[0114] The trusted operator and / or the fused trusted operator calls the trusted execution environment to perform secure processing on the data to be processed, thereby obtaining a plurality of data elements or a plurality of rows of data;

[0115] The trusted operator and / or the fused trusted operator is used to perform data processing on the multiple data elements or multiple rows of data to obtain an encryption processing result.

[0116] Optionally, the trusted execution plan generating module 82 is further configured to: the fused trusted operator includes: a first trusted operator and a second trusted operator;

[0117] The fusion trusted operator calls the trusted execution environment to perform secure processing on the data to be processed, and obtains multiple data elements or multiple rows of data, including:

[0118] Decrypting and parsing the data to be processed in the trusted execution environment to obtain multiple data elements or multiple lines of data;

[0119] The fused trusted operator is used to perform data processing on the data to be processed to obtain an encryption processing result, including:

[0120] Using the first basic operator in the fused trusted operator to perform computation on the multiple data elements or multiple rows of data to obtain a first intermediate result;

[0121] The first intermediate result is sent to the second basic operator to perform calculation processing on the first intermediate result to generate an encrypted processing result.

[0122] Optionally, the trusted execution plan generating module 82 is further configured to decrypt and parse the data to be processed in the trusted execution environment to obtain multiple data elements or multiple rows of data;

[0123] Using the trusted operator to perform data processing on the multiple data elements or multiple rows of data to obtain an encryption processing result, including:

[0124] The trusted operator is used to perform calculation processing on the multiple data elements or multiple rows of data to generate an encryption processing result.

[0125] Optionally, the trusted execution plan generation module 82 is also used to send the encryption processing result to the corresponding trusted operator or the corresponding fusion trusted operator based on the operator dependency relationship; so that the corresponding trusted operator or the corresponding fusion trusted operator can perform data processing on the encryption processing result in the called trusted execution environment.

[0126] Optionally, the trusted execution plan generation module 82 is further configured to perform lightweight serialization, compression, and encryption on the multiple data elements or multiple rows of data included in the fused trusted operator or the trusted operator calculation processing result to generate encrypted data blocks.

[0127] An embodiment of the present application also provides an electronic device. The electronic device is a master node electronic device in a computing unit. Figure 9 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. The electronic device includes a memory 901, a processor 902 and a communication component 903; wherein,

[0128] The memory 901 is used to store programs;

[0129] The processor 902 is coupled to the memory and is configured to execute the program stored in the memory to:

[0130] Determine an execution plan corresponding to the data processing logic; the execution plan includes basic operators;

[0131] generating a trusted execution plan composed of trusted operators according to the calling relationship between the basic operators and the trusted execution environment;

[0132] The received data to be processed is processed based on at least one of the trusted operators in the trusted execution plan and the trusted execution environment called by the trusted operator.

[0133] Optionally, the processor 902 is further configured to obtain a basic operator included in the execution plan;

[0134] generating a trusted operator based on a calling relationship between the basic operator and the trusted execution environment;

[0135] The trusted execution plan is generated according to the trusted operators and operator dependencies.

[0136] Optionally, the processor 902 is further configured to determine whether the trusted operators are allowed to be merged based on the operator dependency relationship and / or operator type between the trusted operators;

[0137] If fusion is allowed, a fusion trusted operator is generated.

[0138] Optionally, the processor 902 is further configured to:

[0139] Based on the operator dependency relationship between the trusted operators, and / or,

[0140] Based on the operator dependency relationship between the fused trusted operator and the fused trusted operator,

[0141] Generate the trusted execution plan.

[0142] Optionally, the processor 902 is further configured to input the data to be processed into the trusted operator and / or the fused trusted operator;

[0143] The trusted operator and / or the fused trusted operator calls the trusted execution environment to perform secure processing on the data to be processed, thereby obtaining a plurality of data elements or a plurality of rows of data;

[0144] The trusted operator and / or the fused trusted operator is used to perform data processing on the multiple data elements or multiple rows of data to obtain an encryption processing result.

[0145] The fused trusted operator includes: a first trusted operator and a second trusted operator;

[0146] The processor 902 is further configured to decrypt and parse the data to be processed in the trusted execution environment to obtain multiple data elements or multiple lines of data;

[0147] The processor 902 is further configured to use the first basic operator in the fused trusted operator to perform calculation processing on the multiple data elements or multiple rows of data to obtain a first intermediate result;

[0148] The first intermediate result is sent to the second basic operator to perform calculation processing on the first intermediate result to generate an encrypted processing result.

[0149] Optionally, the processor 902 is further configured to decrypt and parse the data to be processed in the trusted execution environment to obtain multiple data elements or multiple lines of data;

[0150] Using the trusted operator to perform data processing on the multiple data elements or multiple rows of data to obtain an encryption processing result, including:

[0151] The trusted operator is used to perform calculation processing on the multiple data elements or multiple rows of data to generate an encryption processing result.

[0152] Optionally, the processor 902 is also used to send the encryption processing result to the corresponding trusted operator or the corresponding fusion trusted operator according to the operator dependency relationship; so that the corresponding trusted operator or the corresponding fusion trusted operator performs data processing on the encryption processing result in the called trusted execution environment.

[0153] Optionally, the processor 902 is further configured to perform lightweight serialization, compression, and encryption processing on the multiple data elements or multiple rows of data included in the fused trusted operator or the result of the trusted operator calculation to generate an encrypted data block.

[0154] The memory 901 can be configured to store various other data to support operations on the electronic device. Examples of such data include instructions for any application or method operating on the electronic device. The memory can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0155] Furthermore, the processor 902 in this embodiment may specifically be: a programmable switching processing chip, which is configured with a data replication engine and can replicate received data.

[0156] When the processor 902 executes the program in the memory, in addition to the above functions, it can also realize other functions. For details, please refer to the description of the above embodiments. Figure 9 As shown, the electronic device further includes: a power supply component 904 and other components.

[0157] The embodiment of the present application further provides a non-transitory machine-readable storage medium, wherein the non-transitory machine-readable storage medium stores executable code, and when the executable code is executed by a processor of an electronic device, the processor executes Figure 1 The method described in the corresponding embodiment.

[0158] Based on the above embodiment, the data processing system pre-processed in the data processing engine is processed to obtain an execution plan. Among them, the execution plan contains multiple basic operators. In order to ensure the data security of the operator when executing the data calculation process, it is necessary to generate a trusted operator and a trusted execution plan composed of the trusted operators based on the calling relationship between the basic operator and the trusted execution environment. Then, the trusted operator calls the trusted execution environment, and performs corresponding data processing on the data to be processed in the trusted execution environment. Through the above scheme, the trusted operator is used to call the trusted execution environment, which can ensure that the trusted operators are executing the calculation process in the trusted execution environment and ensure data security. In addition, using the trusted operator as the basic unit for calling the trusted execution environment can not only meet data security, but also effectively reduce the number of calls to the trusted execution environment, thereby effectively reducing the workload of the data processing engine and improving the working performance of the data processing engine.

[0159] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one location or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0160] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, or of course, by hardware. Based on this understanding, the essence of the above technical solution or the part that contributes to the existing technology can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or certain parts of the embodiments.

[0161] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A data processing method, applied to a data processing engine, comprising: Determine the execution plan corresponding to the data processing logic; The execution plan includes basic operators; According to the calling relationship between the basic operator and the trusted execution environment, a trusted execution plan composed of trusted operators is generated, including: Obtaining basic operators included in the execution plan; generating a trusted operator based on a calling relationship between the basic operator and the trusted execution environment; Based on the operator dependency relationship and / or operator type between the trusted operators, determining whether the trusted operators are allowed to be fused; if fusion is allowed, generating a fused trusted operator; Generate the trusted execution plan according to the trusted operators and operator dependencies; Based on at least one of the trusted operators in the trusted execution plan and the trusted execution environment called by the trusted operator, data processing is performed on the received data to be processed, including: Inputting the data to be processed into the trusted operator and / or the fused trusted operator, wherein the fused trusted operator includes: a first trusted operator and a second trusted operator; The trusted operator and / or the fused trusted operator calls the trusted execution environment to decrypt and parse the data to be processed, thereby obtaining a plurality of data elements or a plurality of rows of data; Using the trusted operator and / or the fused trusted operator to perform data processing on the multiple data elements or multiple rows of data to obtain a lightweight encryption processing result; The step of using the fused trusted operator to process the multiple data elements or multiple rows of data to obtain a lightweight encryption processing result includes: Using the first trusted operator in the fused trusted operator to perform computation on the multiple data elements or multiple rows of data to obtain a first intermediate result; Without exiting the trusted execution environment, the first intermediate result is sent to the second trusted operator to perform calculation processing on the first intermediate result to generate a lightweight encryption processing result.

2. The method according to claim 1, wherein generating the trusted execution plan based on the trusted operator and the operator dependency relationship comprises: Based on the operator dependency relationship between the trusted operator and the fused trusted operator, and / or, Based on the operator dependency relationship between the trusted operators, and / or, Based on the operator dependency relationship between the fused trusted operator and the fused trusted operator, Generate the trusted execution plan.

3. The method according to claim 1, wherein the trusted operator calls the trusted execution environment to perform secure processing on the data to be processed to obtain multiple data elements or multiple rows of data, comprising: Decrypting and parsing the data to be processed in the trusted execution environment to obtain multiple data elements or multiple lines of data; The trusted operator is used to perform data processing on the multiple data elements or multiple rows of data to obtain a lightweight encryption processing result, including: The trusted operator is used to perform computational processing on the multiple data elements or multiple rows of data to generate a lightweight encryption processing result.

4. The method according to claim 1 or 3, after obtaining the lightweight encryption result, further comprising: According to the operator dependency relationship, the encryption processing result is sent to the corresponding trusted operator or the corresponding fusion trusted operator; So that the corresponding trusted operator or the corresponding fused trusted operator can perform data processing on the encryption processing result in the called trusted execution environment.

5. The method according to claim 1 or 3, wherein generating a lightweight encryption result comprises: Serializing and compressing the multiple data elements or multiple rows of data included in the fused trusted operator or the trusted operator calculation result to generate a complete encrypted data block; The entire encrypted data block is encrypted to generate the encryption result.

6. A data processing system, comprising: The client is used to send data processing requests to the data processing system; A data processing engine, configured to implement the method according to any one of claims 1 to 5.

7. A data processing engine, comprising: Execution plan production module, trusted execution plan generation module and data processing module; The engine is used to implement the method according to any one of claims 1 to 5.

8. An electronic device comprising a memory and a processor; wherein: The memory is used to store programs; The processor is coupled to the memory and is configured to execute the program stored in the memory to implement the method according to any one of claims 1 to 5.

9. A non-transitory machine-readable storage medium having executable code stored thereon, wherein when the executable code is executed by a processor of an electronic device, the processor is caused to execute the method according to any one of claims 1 to 6.

Images