Data processing method and system, server, storage medium and program product

Abstract

The invention relates to the technical field of privacy computing, and discloses a data processing method and system, a server, a storage medium and a program product. The method comprises the following steps: receiving a first credible report and a second credible report reported by a cluster server; and comparing the first credible report with the second credible report, and sending a generated report comparison result to the client, so that the client determines whether the second credible report is credible according to the report comparison result, and when the second credible report is credible, the client establishes a credible channel with the cluster server. The cluster server performs ciphertext read-write operation on the external data source based on the trusted channel; a structured query language engine is deployed in the cluster server, and the structured query language engine is used for performing ciphertext reading and writing on external data source data. By implementing the technical scheme of the invention, secure communication between the client and the cluster server can be ensured so as to realize secure ciphertext read-write operation on an external data source.

Description

Technical Field

[0001] The present invention relates to the field of privacy computing technology, and specifically to a data processing method, system, server, storage medium and program product. Background Art

[0002] Privacy computing is a computing technology designed to protect data privacy. It uses cryptography and secure computing technology to avoid exposing original data during data processing and calculation. It allows data owners to share data securely while ensuring that the data is fully protected during calculation and analysis.

[0003] However, in privacy applications based on Spark clusters, there is a challenge of insufficient credibility of the accessed Spark clusters, and there is a risk that cluster nodes may be tampered with. In addition, the current Spark cluster lacks support for data encryption reading and writing when accessing third-party storage systems through Java Database Connectivity (JDBC), resulting in the risk of data security leakage cannot be effectively prevented. Summary of the invention

[0004] In view of this, the present invention provides a data processing method, system, server, storage medium and program product to solve the problems of cluster node tampering and data security leakage in privacy applications based on Spark clusters.

[0005] In the first aspect, the present invention provides a data processing method, which is applied to a privacy computing platform, including: receiving a first trusted report and a second trusted report reported by a cluster server; the first trusted report is generated by the master node of the cluster server during initial registration, and the second trusted report is generated by the cluster server based on a trusted report request sent by a client; comparing the first trusted report and the second trusted report, generating a report comparison result, and sending the report comparison result to the client, so that the client determines whether the second trusted report is credible based on the report comparison result, and when the second trusted report is credible, the client establishes a trusted channel with the cluster server, so that the cluster server performs ciphertext reading and writing operations on an external data source based on the trusted channel; a structured query language engine is deployed in the cluster server, and the structured query language engine is an engine for performing ciphertext reading and writing of external data source data.

[0006] The data processing method provided by the embodiment of the present invention ensures that the client can verify the credibility of the cluster server by comparing the first trusted report and the second trusted report reported by the cluster server, thereby preventing the risk of cluster nodes being tampered with. When the second trusted report is verified to be credible, the client can establish a trusted channel with the cluster server. This channel ensures secure data transmission, especially when performing ciphertext reading and writing operations on external data sources. By using trusted reports and trusted channels, the risk of data security leakage is effectively prevented, and the security and reliability of the overall data processing process are improved.

[0007] In an optional embodiment, receiving a first trusted report reported by a cluster server includes: starting multiple slave nodes of the cluster server so that the multiple slave nodes generate first summary information, and sending the first summary information to a master node, the master node generates a first trusted report based on the first summary information, and registers the first trusted report to the privacy computing platform.

[0008] In the data processing method provided by the embodiment of the present invention, the master node generates a first trusted report based on summary information sent by multiple slave nodes. Even if a slave node fails or is tampered with, the integrity and reliability of the overall trusted report can still be guaranteed. This distributed generation method significantly reduces the impact of single point failures. The distributed generated first trusted report can more effectively prevent the risk of data tampering and security leakage, because the report generation process itself relies on the collaboration and verification of multiple nodes, which improves the security of the overall data processing process.

[0009] In an optional implementation, node change information reported by the cluster server is received, where the node change information is sent by each slave node when the collected first summary information changes, and is used to characterize that the slave node whose first summary information has changed is untrustworthy; based on the node change information, the task corresponding to the slave node whose first summary information has changed is stopped.

[0010] The data processing method provided by the embodiment of the present invention can more effectively verify the integrity of data and enhance the credibility of the cluster by monitoring the changes of summary information. Node change information can help quickly identify faults or anomalies, so that timely measures can be taken to reduce the impact of faults on data processing. Timely monitoring of changes in summary information helps to discover potential security threats and prevent data tampering or leakage.

[0011] In an optional implementation, information of a newly added slave node reported by a cluster server is received, and the report of the newly added slave node is approved; the information of the newly added slave node is generated when the cluster server is to be expanded; when it is determined that the report is passed, the cluster server is instructed to add the newly added slave node.

[0012] The data processing method provided by the embodiment of the present invention can flexibly expand the computing capacity of the cluster to meet the growing processing needs by approving the information of the newly added slave nodes. The approval mechanism can ensure that only verified slave nodes are added, thereby preventing the access of unauthorized nodes and protecting security.

[0013] In an optional implementation, information of the slave node to be removed reported by the cluster server is received, where the information of the slave node to be removed is generated when the cluster server is to be scaled down; and the slave node to be removed is removed from the cluster server according to the information of the slave node to be removed.

[0014] The data processing method provided by the embodiment of the present invention helps to optimize resource allocation and improve overall performance by removing unnecessary slave nodes, and effectively avoids malicious use of removed slave nodes after being attacked.

[0015] In the second aspect, the present invention provides a data processing method, which is applied to a cluster server, wherein the cluster server includes a master node and multiple slave nodes, including: controlling the master node to generate a first trusted report during initial registration, and reporting the first trusted report to a privacy computing platform; receiving a trusted report request sent by a client; sending the trusted report request to the master node, so that the master node generates a second trusted report based on the trusted report request, and reporting the second trusted report to the privacy computing platform through the client, so that the privacy computing platform compares the first trusted report and the second trusted report to generate a report comparison result; based on a trusted channel between the client and the cluster server, receiving a database operation statement sent by the client, the database operation statement includes encryption parameters; the trusted channel is generated when it is determined that the second trusted report is trusted based on the report comparison result; and performing ciphertext reading and writing on the specified external data source according to the encryption parameters.

[0016] The data processing method provided by the embodiment of the present invention is based on the trusted channel established between the client and the cluster, and performs ciphertext read and write operations on the external data source according to the encryption parameters provided by the client. This method ensures the security of data during transmission and processing, and effectively prevents the risk of data leakage and tampering. The privacy computing platform can compare the first trusted report with the second trusted report and generate a report comparison result. This process ensures the credibility of the cluster server and the accuracy of data processing. The client can determine the credibility of the second trusted report based on the comparison result, thereby safely establishing and using a trusted channel. By integrating the comparison mechanism of the first trusted report and the second trusted report, the overall consistency and reliability of the data processing operation can be maintained.

[0017] In an optional embodiment, the master node is controlled to generate a first trusted report during initial registration, including: when multiple slave nodes are in a startup state, the multiple slave nodes are controlled to collect corresponding node information according to a first preset period, and the first summary information generated according to the node information is sent to the master node, so that the master node generates a first trusted report based on the first summary information, and the node information includes at least one of hardware information, kernel information and immutable information.

[0018] In the data processing method provided by the embodiment of the present invention, the master node generates a first trusted report based on the node information collected by multiple slave nodes. This centralized generation method simplifies the management and verification process of the report and ensures the consistency and accuracy of the report. Only the first summary information generated by the slave node needs to be sent to the master node instead of a large amount of original data, which effectively reduces the cost of data transmission and network traffic. This optimization can significantly reduce the burden of data processing and transmission in a large-scale cluster environment. Through the collection of information at a preset period and the generation of centralized reports, information can be responded to and processed more quickly.

[0019] In an optional embodiment, each slave node is controlled to collect the first summary information according to a second preset period, so that when each slave node determines that the corresponding first summary information has changed, it sends the change information to the master node, so that the master node will eliminate the changed slave nodes and report to the privacy computing platform that the changed slave nodes are untrustworthy.

[0020] The data processing method provided by the embodiment of the present invention can monitor the changes of data in real time by controlling each slave node to collect the first summary information through the second preset period. This real-time detection can timely detect changes in node status, which helps to respond quickly and make necessary adjustments and processing. When the changed nodes are removed in time and untrusted information is reported to the privacy computing platform, it helps to maintain the stability and security of the system. In this way, the impact of bad nodes on the normal operation of the system can be prevented, and the reliability of data processing and computing tasks can be guaranteed.

[0021] In an optional implementation, information of newly added slave nodes is obtained; when the cluster server is to be expanded, the master node is controlled to report the information of the newly added slave nodes to the privacy computing platform; when the privacy computing platform's report is approved, the newly added slave nodes are added to the cluster server.

[0022] In the data processing method provided by the embodiment of the present invention, the master node reports the information of the newly added slave nodes to the privacy computing platform, and adds them to the cluster only after the report is passed, which helps to ensure the orderliness and compliance of the expansion process. Through the reporting and verification process, the newly added slave nodes can be reviewed to ensure that they meet the security and performance requirements of the cluster. This reduces the risk of introducing potential untrusted nodes and improves the security of the cluster.

[0023] In an optional implementation, information about the slave nodes to be removed is obtained; when the cluster server is to be scaled down, the master node is controlled to report the information about the slave nodes to be removed to the privacy computing platform.

[0024] The data processing method provided by the embodiment of the present invention enables the master node to obtain the information of the slave node to be removed in a timely manner, so that the state change of the node can be monitored in real time. This real-time monitoring can quickly discover node failures or failures, which helps to take necessary capacity reduction measures in a timely manner.

[0025] In an optional implementation, the status information of the cluster server is obtained; when the cluster server recovers from an abnormal state, the control master node collects the first summary information from the cluster server that recovers to normal.

[0026] According to the data processing method provided by the embodiment of the present invention, after the cluster server recovers from an abnormal state, the master node can collect the first summary information from the cluster server that has recovered to normal, which ensures continuous monitoring and updating of the cluster status and helps to timely understand the health status of the recovery node.

[0027] In an optional implementation, the control master node collects second summary information of multiple slave nodes based on the trusted report request, and generates a second trusted report based on the second summary information.

[0028] In the data processing method provided by the embodiment of the present invention, the second summary information is collected based on a trusted report request, which means that the acquisition and generation of the information are authorized and authenticated. This security measure ensures the credibility of the data and prevents unauthorized access or tampering, thereby improving the security of the system.

[0029] In an optional implementation, ciphertext reading and writing are performed on a designated external data source according to encryption parameters, including: identifying encryption parameters in a database operation statement of a client, and encapsulating the encryption parameters into a first operator of a syntax parsing phase; converting the first operator into a second operator of a logical planning phase using a preset conversion rule; if the second operator includes a read statement, converting the read statement into a read operation operator of a physical planning phase, decrypting and calculating the external data source using the read operation operator, and returning the calculation result to the client through a trusted channel; if the second operator includes a write statement, converting the write statement into a write operation operator of a physical planning phase, decrypting, calculating, and encrypting the external data source using the write operation operator to obtain encrypted data to be written, and writing the encrypted data into the external data source.

[0030] The data processing method provided by the embodiment of the present invention protects sensitive data and prevents unauthorized access and data leakage by using encryption in the reading and writing process. It supports flexible reading and writing of external data sources, allows necessary calculations to be performed in an encrypted state, and improves the efficiency of data processing. The calculation results are transmitted through a trusted channel to ensure the security of data during transmission and enhance the trust of the overall system. Encapsulating encryption parameters into the syntax parsing stage ensures that security is considered in the initial stage of data operations and reduces the complexity of subsequent stages. By dividing operations into logical plans and physical plans, the query execution process is optimized and the overall performance and maintainability are improved. Calculations are performed in a ciphertext state, which reduces plaintext operations and reduces potential security risks.

[0031] In the third aspect, the present invention provides a data processing method, which is applied to a client, including: sending a trusted report request to a cluster server, so that the master node in the cluster server generates a second trusted report based on the trusted report request; reporting the second trusted report to a privacy computing platform, so that the privacy computing platform compares the first trusted report and the second trusted report to generate a report comparison result; based on a trusted channel between the client and the cluster server, sending a database operation statement to the cluster server, so that the cluster server reads and writes ciphertext to a specified external data source according to the encryption parameters included in the database operation statement; the trusted channel is generated when the second trusted report is determined to be trustworthy based on the report comparison result.

[0032] The data processing method provided in the embodiment of the present invention ensures the accuracy and credibility of information by generating a second trusted report and comparing it with the first trusted report, which helps to improve the trust of the overall system. The report comparison mechanism can effectively detect potential malicious behavior and prevent data from being tampered with during transmission and processing. The use of trusted channels for database operations ensures the secure transmission of data between the client and the cluster server, reducing the risk of data leakage. It supports ciphertext reading and writing according to encryption parameters, allowing flexible operations on external data sources while ensuring data security.

[0033] In a fourth aspect, the present invention provides a data processing system, comprising a client, a cluster server and a privacy computing platform, wherein the client, the cluster server and the privacy computing platform are communicatively connected; the cluster server is used to obtain a first trusted report, and is used to report the first trusted report to the privacy computing platform; the client is used to send a trusted report request to the cluster server; the cluster server is used to send the trusted report request to the master node, so that the master node generates a second trusted report based on the trusted report request, and sends the second trusted report to the privacy computing platform through the client; the privacy computing platform is used to compare the first trusted report and the second trusted report, and is used to send the report comparison result to the client; when the client determines that the second trusted report is trustworthy based on the report comparison result, the client is also used to establish a trusted channel with the cluster server; the cluster server is also used to obtain encryption parameters provided by the client, and is used to read and write ciphertext to the specified external data source based on the trusted channel and in accordance with the encryption parameters.

[0034] The data processing system provided by the embodiment of the present invention ensures the credibility and integrity of the data by comparing the first trusted report with the second trusted report. This comparison mechanism helps to ensure that the data has not been tampered with or modified during the data transmission and processing process, thereby improving the security and reliability of data processing. When the client determines that the second trusted report is credible, a trusted channel can be established with the cluster server. These secure channels ensure the confidentiality and integrity of the data during transmission, and prevent the risk of data leakage and unauthorized access. As an intermediate node, the cluster server is responsible for processing requests and forwarding of trusted reports, and also assumes the function of establishing trusted channels and performing ciphertext read and write operations. This distributed processing capability enables the system to handle large-scale data and complex computing tasks, thereby improving the scalability and efficiency of the system.

[0035] In a fifth aspect, the present invention provides a data processing device, comprising: a first receiving module, used to receive a first trusted report and a second trusted report reported by a cluster server; the first trusted report is generated by the master node of the cluster server at the time of initial registration, and the second trusted report is generated by the cluster server based on a trusted report request sent by a client; a comparison module, used to compare the first trusted report and the second trusted report, generate a report comparison result, and send the report comparison result to the client, so that the client determines whether the second trusted report is credible based on the report comparison result, and when the second trusted report is credible, the client establishes a trusted channel with the cluster server, so that the cluster server performs ciphertext reading and writing operations on an external data source based on the trusted channel; a structured query language engine is deployed in the cluster server, and the structured query language engine is an engine for performing ciphertext reading and writing of data from an external data source.

[0036] In the sixth aspect, the present invention provides a data processing device, including: a first control module, used to control the master node to generate a first trusted report during initial registration, and report the first trusted report to a privacy computing platform; a second receiving module, used to receive a trusted report request sent by the client; a first sending module, used to send the trusted report request to the master node, so that the master node generates a second trusted report based on the trusted report request, and reports the second trusted report to the privacy computing platform through the client, so that the privacy computing platform compares the first trusted report and the second trusted report to generate a report comparison result; a third receiving module, used to receive a database operation statement sent by the client based on a trusted channel between the client and the cluster server, the database operation statement including encryption parameters; the trusted channel is generated when it is determined that the second trusted report is trustworthy based on the report comparison result; a reading and writing module, used to read and write ciphertext to a specified external data source according to the encryption parameters.

[0037] In the seventh aspect, the present invention provides a data processing device, including: a second sending module, used to send a trusted report request to a cluster server, so that the master node in the cluster server generates a second trusted report based on the trusted report request; a third sending module, used to report the second trusted report to the privacy computing platform, so that the privacy computing platform compares the first trusted report and the second trusted report to generate a report comparison result; a fourth sending module, used to send a database operation statement to the cluster server based on a trusted channel between the client and the cluster server, so that the cluster server reads and writes ciphertext to the specified external data source according to the encryption parameters included in the database operation statement; the trusted channel is generated when the second trusted report is determined to be credible based on the report comparison result.

[0038] In an eighth aspect, the present invention provides a server, comprising: a memory and a processor, the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the data processing method of the first aspect or any corresponding embodiment thereof, the data processing method of the second aspect or any corresponding embodiment thereof, and the data processing method of the third aspect by executing the computer instructions.

[0039] In a ninth aspect, the present invention provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute the data processing method of the first aspect or any corresponding embodiment thereof, the data processing method of the second aspect or any corresponding embodiment thereof, and the data processing method of the third aspect.

[0040] In the tenth aspect, the present invention provides a computer program product, comprising computer instructions, wherein the computer instructions are used to enable a computer to execute the data processing method of the first aspect or any corresponding embodiment thereof, execute the data processing method of the second aspect or any corresponding embodiment thereof, and execute the data processing method of the third aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0042] Figure 1 is a flow chart of a data processing method according to an embodiment of the present invention;

[0043] Figure 2 is a flow chart of another data processing method according to an embodiment of the present invention;

[0044] Figure 3 is a flow chart of another data processing method according to an embodiment of the present invention;

[0045] Figure 4 is a flow chart of yet another data processing method according to an embodiment of the present invention;

[0046] Figure 5 is a flow chart of yet another data processing method according to an embodiment of the present invention;

[0047] Figure 6 is a flow chart of yet another data processing method according to an embodiment of the present invention;

[0048] Figure 7 is a structural block diagram of a data processing system according to an embodiment of the present invention;

[0049] Figure 8 is a structural block diagram of a data processing device according to an embodiment of the present invention;

[0050] Fig. 9 is a structural block diagram of another data processing device according to an embodiment of the present invention;

[0051] Fig.10 is a structural block diagram of another data processing device according to an embodiment of the present invention;

[0052] Fig.11 It is a schematic diagram of the hardware structure of the server according to an embodiment of the present invention. DETAILED DESCRIPTION

[0053] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0054] Privacy computing is an advanced technology focused on protecting data privacy. It combines cryptography and secure computing techniques to make it possible to process and compute data without leaking the original data. This technology enables data owners to share data more securely while ensuring that the data is fully protected during the computing and analysis process. With the rapid increase in the amount of data, the need for large-scale data computing while ensuring data security is becoming increasingly urgent. Therefore, ensuring the security of computing infrastructure and the security of processing massive amounts of data have become two challenges that cannot be ignored.

[0055] In this context, the present invention selects Spark cluster as the basic computing facility, making full use of its support for Secure Sockets Layer (SSL) encrypted transmission, master-worker ciphertext communication and intermediate shuffle data ciphertext storage and other features to cope with the needs of massive data processing. At present, since the JDBC access method of the Spark cluster does not yet support encryption and decryption functions, it is necessary to develop data definition language (DDL) related syntax and functions to achieve encryption and decryption support for data. In addition, in order to ensure the credibility of the cluster, the user must confirm the credibility of the cluster before accessing the data to prevent the data information from being tampered with. Therefore, a solution that can support privacy computing in a Spark cluster environment is needed to ensure that data privacy and security are effectively protected.

[0056] In view of this, the technical solution of the present invention improves the reliability of certificates and the transparency of reports by deploying clusters and developing remote authentication services to ensure the credibility of Spark clusters and prevent the clusters from being tampered with and causing data leakage of privacy applications. The privacy computing platform uses an image template to start on each Spark cluster host and divides the system path into a variable area and an immutable area. The variable area is used to temporarily store data during the Spark calculation process and is empty at startup; while the immutable area includes Spark service-related code, necessary dependent files, and the operating system's own content. The content of the immutable area has been reviewed and cannot be changed, which is the basis of credibility. During the startup process, each host communicates with the privacy computing platform to generate a summary based on the hardware, kernel, Linux initial RAM disk (initrd), and immutable area. Users can use this summary to confirm the integrity of the cluster to ensure that it has not been tampered with. In order to ensure the credibility of the cluster, key measures include generating certificate reports and supporting cluster expansion and reduction functions. These measures not only ensure the security and integrity of the data, but also provide a trusted operating environment and stable expansion capabilities for privacy computing.

[0057] According to an embodiment of the present invention, a data processing method embodiment is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0058] In this embodiment, a data processing method is provided, which can be used in a privacy computing platform. Figure 1 is a flow chart of a data processing method according to an embodiment of the present invention. Figure 1 As shown, the process includes the following steps:

[0059] Step S101, receiving a first trusted report and a second trusted report reported by a cluster server; the first trusted report is generated by the master node of the cluster server during initial registration, and the second trusted report is generated by the cluster server based on a trusted report request sent by a client.

[0060] A privacy computing platform refers to an overall environment or system for performing computing tasks and data processing, which includes multiple servers and related software infrastructure. Specifically, a privacy computing platform can be a centralized management system or a trusted root server.

[0061] A cluster server refers to a computing node in a distributed computing environment that works together to provide the computing power of the cluster. Specifically, a cluster server includes a master node and slave nodes, which jointly run computing tasks and data processing tasks. A master node refers to a core node or server in a cluster server, which is responsible for managing and controlling the work of other nodes or servers.

[0062] The client refers to the system or device that runs the privacy application, such as a personal computer, smartphone or other terminal device.

[0063] The first trusted report is a report generated by the master node of the cluster server during initial registration, which may include trusted status information of the hardware, firmware, and software of the cluster server to ensure that the cluster server is in a trusted state during registration. The first trusted report provides a trusted basis for the cluster server in its initial state, and this report can be used as a baseline for subsequent credibility verification. Specifically, when the master node of the cluster server is initially registered, the privacy computing platform will receive the first trusted report, which occurs during the registration phase of the server, where the master node reports its initial status to the privacy computing platform. This process can be completed through network communication, registration API, or other mechanisms, which are not limited here. Optionally, after receiving the first trusted report sent by the master node, the privacy computing platform can notify the authority to sign the first trusted report to generate a trusted certificate and file it.

[0064] The second trusted report is a report generated by the cluster server after receiving a trusted report request from the client. The second trusted report is information about the current status of the cluster server, which may include real-time security status and health status, etc. It provides the trusted status information of the cluster server at the current moment, which is used to verify whether the cluster server is still in a trusted state. Specifically, the second trusted report is generated after the cluster server receives the trusted report request from the client. The privacy computing platform needs to have a mechanism for receiving such a report, for example, it can be received through network communication or API interface. The cluster server will send the second trusted report to the privacy computing platform, and the privacy computing platform can receive this report through a specific endpoint or interface.

[0065] Step S102, compare the first trusted report and the second trusted report, generate a report comparison result, and send the report comparison result to the client, so that the client determines whether the second trusted report is credible based on the report comparison result. When the second trusted report is credible, the client establishes a trusted channel with the cluster server, so that the cluster server performs encrypted reading and writing operations on the external data source based on the trusted channel; a structured query language engine is deployed in the cluster server, and the structured query language engine is an engine for performing encrypted reading and writing of external data source data.

[0066] The report comparison result refers to the conclusion or result obtained after comparing and analyzing the first trusted report and the second trusted report. For example, the comparison process may include comparing the content of the report, data consistency, possible differences or errors, etc. The comparison result can indicate whether the second trusted report is consistent with the first trusted report or there are differences, thereby helping the client to evaluate the credibility of the second trusted report.

[0067] A trusted channel is a secure communication path or connection that uses encryption technology and other security measures to ensure the confidentiality, integrity, and reliability of communications. The client needs to establish a trusted channel with the cluster server so that the cluster server can safely perform ciphertext read and write operations with external data sources to ensure that data will not be tampered with or leaked during transmission.

[0068] External data sources refer to data from outside the cluster that the cluster server needs to access or process. The Structured Query Language (SQL) engine refers to a database engine used to process data from external data sources. Specifically, the SQL engine can process data from external data sources, which are encrypted when transmitted to the SQL engine, and the SQL engine is responsible for decryption and encryption operations internally.

[0069] Specifically, the privacy computing platform compares the content and data of the first trusted report and the second trusted report to check their consistency or difference. A conclusion is generated based on the comparison results, indicating the credibility of the second trusted report. The comparison results are sent to the client, allowing the client to decide how to handle the second trusted report based on these results. If the second trusted report is confirmed to be credible, the client needs to establish a secure communication channel (i.e., a trusted channel) to the cluster server so that subsequent data operations are safe.

[0070] The data processing method provided by the embodiment of the present invention ensures that the client can verify the credibility of the cluster server by comparing the first trusted report and the second trusted report reported by the cluster server, thereby preventing the risk of cluster nodes being tampered with. When the second trusted report is verified to be credible, the client can establish a trusted channel with the cluster server. This channel ensures secure data transmission, especially when performing ciphertext reading and writing operations on external data sources. By using trusted reports and trusted channels, the risk of data security leakage is effectively prevented, and the security and reliability of the overall data processing process are improved.

[0071] In this embodiment, a data processing method is provided, which can be used in a privacy computing platform. Figure 2 is a flow chart of a data processing method according to an embodiment of the present invention. Figure 2As shown, the process includes the following steps:

[0072] Step S201, receiving a first trusted report and a second trusted report reported by a cluster server; the first trusted report is generated by the master node of the cluster server during initial registration, and the second trusted report is generated by the cluster server based on a trusted report request sent by a client.

[0073] Specifically, the above step S201 includes:

[0074] Step S2011, start multiple slave nodes of the cluster server so that the multiple slave nodes generate first summary information and send the first summary information to the master node. The master node generates a first trusted report based on the first summary information and registers the first trusted report to the privacy computing platform.

[0075] Multiple Worker Nodes refer to multiple machines or servers in a cluster server, which are connected to the master node through cluster management software (such as Hadoop, Spark, Kubernetes, etc.) and perform distributed computing tasks. The first summary information refers to the summary collected from multiple slave nodes of the cluster server and generated based on the system status information. Specifically, ensure that all slave nodes are configured and ready to run cluster services and system status monitoring programs. Run the system status monitoring program on the slave node, which is responsible for collecting the status information of the slave nodes and generating the first summary information of multiple slave nodes based on the status information. Multiple slave nodes send the first summary information to the master node, for example, through a network communication protocol (such as TCP / IP), and the master node can have a dedicated receiving program to wait for the information from the slave nodes. After the master node receives the first summary information sent by all slave nodes, it summarizes and processes the information, generates a first trusted report, and registers the first trusted report to the relevant storage or database of the privacy computing platform for subsequent trusted verification.

[0076] Step S2012, receiving node change information reported by the cluster server, the node change information is sent by each slave node when the collected first summary information changes, and is used to indicate that the slave node whose first summary information has changed is untrustworthy; according to the node change information, stop executing the task corresponding to the slave node whose first summary information has changed.

[0077] The node change information is used to characterize that each slave node determines that the corresponding first summary information has changed, that is, the changed slave node is untrustworthy. Specifically, when each slave node determines that the corresponding first summary information has changed, it sends the change information to the master node, and the master node removes the changed slave node and reports to the privacy computing platform that the changed slave node is untrustworthy. When the privacy computing platform receives information about an untrustworthy slave node (that is, a slave node whose first summary information has changed), it will immediately stop executing the task of the untrustworthy slave node.

[0078] The data processing method provided by the embodiment of the present invention can more effectively verify the integrity of data and enhance the credibility of the cluster by monitoring the changes of summary information. Node change information can help quickly identify faults or anomalies, so that timely measures can be taken to reduce the impact of faults on data processing. Timely monitoring of changes in summary information helps to discover potential security threats and prevent data tampering or leakage.

[0079] Step S2013, receiving the information of the newly added slave node reported by the cluster server, and approving the report of the newly added slave node; the information of the newly added slave node is generated when the cluster server is to be expanded; when it is determined that the report is passed, instruct the cluster server to add the newly added slave node.

[0080] A newly added slave node refers to a newly added slave node in the cluster server. It is a node added to expand the computing or storage capacity of the cluster server. Specifically, after the cluster server detects the addition of a new slave node, it will begin to collect relevant information about the node. The master node will send a report to the privacy computing platform to apply for the new node to join the cluster. After receiving the report, the privacy computing platform will review and confirm it. If the review is passed, the addition of the new slave node will be officially recognized, and the cluster server will be instructed to add the new slave node.

[0081] The data processing method provided by the embodiment of the present invention can flexibly expand the computing capacity of the cluster to meet the growing processing needs by approving the information of the newly added slave nodes. The approval mechanism can ensure that only verified slave nodes are added, thereby preventing the access of unauthorized nodes and protecting security.

[0082] Step S2014, receiving information of the slave node to be removed reported by the cluster server, where the information of the slave node to be removed is generated when the cluster server is to be scaled down; according to the information of the slave node to be removed, removing the slave node to be removed from the cluster server.

[0083] The slave nodes to be removed refer to the slave nodes that need to be removed when the cluster server is to be scaled down. Specifically, when the cluster server is to be scaled down, the privacy computing platform instructs the master node to remove the slave nodes to be removed from the cluster server or perform other cleanup operations.

[0084] Furthermore, when a slave node is determined to be faulty or unavailable, the cluster server will automatically collect detailed information about the slave node and send a report to the privacy computing platform through the master node, indicating that the node needs to be processed to prevent the failed or unavailable slave node from being attacked and used maliciously. After receiving the report, the privacy computing platform will mark the slave node as unavailable and stop executing its tasks.

[0085] The data processing method provided by the embodiment of the present invention helps to optimize resource allocation and improve overall performance by removing unnecessary slave nodes, and effectively avoids malicious use of removed slave nodes after being attacked.

[0086] Step S2015: Receive the second trusted report reported by the cluster server. Figure 1 Step S101 of the illustrated embodiment will not be described in detail here.

[0087] Step S202, compare the first trusted report and the second trusted report, generate a report comparison result, and send the report comparison result to the client, so that the client can determine whether the second trusted report is credible according to the report comparison result. If the second trusted report is credible, the client establishes a trusted channel with the cluster server, so that the cluster server can perform ciphertext reading and writing operations on the external data source based on the trusted channel; the cluster server is deployed with a structured query language engine, which is an engine for performing ciphertext reading and writing on the external data source data. For details, please refer to Figure 1 Step S102 of the illustrated embodiment will not be described in detail here.

[0088] In the data processing method provided by the embodiment of the present invention, the master node generates a first trusted report based on summary information sent by multiple slave nodes. Even if a slave node fails or is tampered with, the integrity and reliability of the overall trusted report can still be guaranteed. This distributed generation method significantly reduces the impact of single point failures. The distributed generated first trusted report can more effectively prevent the risk of data tampering and security leakage, because the report generation process itself relies on the collaboration and verification of multiple nodes, which improves the security of the overall data processing process.

[0089] In this embodiment, a data processing method is provided, which can be used in a cluster server. Figure 3 is a flow chart of a data processing method according to an embodiment of the present invention. Figure 3 As shown, the process includes the following steps:

[0090] Step S301, control the master node to generate a first trusted report during initial registration, and report the first trusted report to the privacy computing platform.

[0091] Specifically, when the cluster server is first started or initialized for registration, the master node will perform a series of system status checks and data collection operations. The master node will collect key system information, and based on the collected data, the master node uses a security algorithm to generate a unique trusted report, namely the first trusted report. The obtained first trusted report is sent to the privacy computing platform for storage and subsequent comparative analysis.

[0092] Step S302: receiving a trust report request sent by the client.

[0093] A trusted report request is a request from a client to a cluster server to obtain a trusted report of the current system. The purpose of this request is to obtain security verification information about the current state of the cluster server to ensure that the cluster server maintains its expected security and integrity status during operation. Specifically, the cluster server receives a request from a client, and the client wants to obtain a trusted report of the system.

[0094] Step S303: Send the trusted report request to the master node so that the master node generates a second trusted report based on the trusted report request, and reports the second trusted report to the privacy computing platform through the client, so that the privacy computing platform compares the first trusted report and the second trusted report to generate a report comparison result.

[0095] The second trusted report is generated based on the second summary information provided by the slave node, and the second summary information is collected from each slave node during the trusted report request process. Specifically, the second summary information refers to the summary generated based on the system status information collected by the master node from multiple slave nodes of the cluster server after receiving the trusted report request.

[0096] Specifically, the client's trusted report request is forwarded to the master node, which is responsible for processing these requests. After receiving the request, the master node will generate a second trusted report based on the request content and the information of the slave node. The generated second trusted report is reported to the privacy computing platform through the client. The privacy computing platform will use this report for comparison and analysis.

[0097] Step S304, based on the trusted channel between the client and the cluster server, receiving the database operation statement sent by the client, the database operation statement including the encryption parameter; the trusted channel is generated when the second trusted report is determined to be credible according to the report comparison result.

[0098] A trusted channel is a secure communication path established between a client and a cluster server to ensure the confidentiality and integrity of data transmission. For example, encryption protocols (such as TLS / SSL) can be used to encrypt communication content, and digital certificates can be used for authentication to ensure the security of communication.

[0099] Encryption parameters refer to the specific configuration information about data encryption and protection provided by the client, which may include encryption columns, encryption algorithms, encryption methods, keys, etc., as shown in Table 1. Specifically, before interacting with the cluster service, the client sends a database operation statement in which the corresponding encryption parameters are configured and specified. The client sends the database operation statement with encryption parameters to the cluster server through a secure communication channel.

[0100] Table 1

[0101]

[0102] Step S305: Read and write ciphertext to the specified external data source according to the encryption parameters.

[0103] Specifically, according to the encryption parameters provided by the client, when accessing external data sources, the data is read or written in ciphertext form. This ensures that even if the data is intercepted during storage or transmission, it cannot be read or understood by unauthorized persons.

[0104] The data processing method provided by the embodiment of the present invention is based on the trusted channel established between the client and the cluster, and performs ciphertext read and write operations on the external data source according to the encryption parameters provided by the client. This method ensures the security of data during transmission and processing, and effectively prevents the risk of data leakage and tampering. The privacy computing platform can compare the first trusted report with the second trusted report and generate a report comparison result. This process ensures the credibility of the cluster server and the accuracy of data processing. The client can determine the credibility of the second trusted report based on the comparison result, thereby safely establishing and using a trusted channel. By integrating the comparison mechanism of the first trusted report and the second trusted report, the overall consistency and reliability of the data processing operation can be maintained.

[0105] In this embodiment, a data processing method is provided, which can be used in a cluster server. Figure 4 is a flow chart of a data processing method according to an embodiment of the present invention. Figure 4 As shown, the process includes the following steps:

[0106] Step S401, control the master node to generate a first trusted report during initial registration, and report the first trusted report to the privacy computing platform.

[0107] Specifically, the above step S401 includes:

[0108] Step S4011, when multiple slave nodes are in the startup state, control the multiple slave nodes to collect corresponding node information according to the first preset period, and send the first summary information generated according to the node information to the master node, so that the master node generates a first trusted report based on the first summary information, and the node information includes at least one of hardware information, kernel information and immutable information.

[0109] The first preset period refers to the time interval for the slave node to collect information after startup. This is a predetermined time period, such as every few minutes or hours, the node will collect data. Specifically, when the slave nodes are started, they will be configured to perform information collection operations regularly according to the first preset period. In each period, the slave node will collect its hardware information (such as CPU model, memory capacity), kernel information (operating system version, kernel version) and immutable information (system configuration file, firmware version, etc.). The collected information will be used to generate the first summary information, for example, a hash value can be generated. This process ensures that the results of each collection are consistent unless the node information is tampered with. The generated first summary information will be sent to the master node after each first preset period. The master node receives the first summary information from all slave nodes. Based on these summary information, the master node generates a first trusted report, which contains the preliminary security status of all nodes in the cluster.

[0110] In the data processing method provided by the embodiment of the present invention, the master node generates a first trusted report based on the node information collected by multiple slave nodes. This centralized generation method simplifies the management and verification process of the report and ensures the consistency and accuracy of the report. Only the first summary information generated by the slave node needs to be sent to the master node instead of a large amount of original data, which effectively reduces the cost of data transmission and network traffic. This optimization can significantly reduce the burden of data processing and transmission in a large-scale cluster environment. Through the collection of information at a preset period and the generation of centralized reports, information can be responded to and processed more quickly.

[0111] Step S4012: Report the first trusted report to the privacy computing platform. Figure 3 Step S301 of the illustrated embodiment will not be described in detail here.

[0112] Step S4013, control each slave node to collect the first summary information according to the second preset period, so that when each slave node determines that the corresponding first summary information has changed, it sends the change information to the master node, so that the master node will eliminate the changed slave nodes and report to the privacy computing platform that the changed slave nodes are untrustworthy.

[0113] The second preset period refers to the time interval at which the slave node regularly monitors the first summary information, and this period is different from the period (the first preset period) for collecting information for the first time. The second preset period is to periodically check whether the first summary information of the node is consistent to ensure that the slave node has not been tampered with or attacked. The specific time interval (such as every hour, every day) depends on the security requirements and performance requirements of the system. Change information refers to the situation where the first summary information detected by the slave node is inconsistent with the previously recorded summary information. Specifically, a timer task or a scheduler is set on each slave node to automatically collect the first summary information according to the second preset period, for example, periodic collection can be achieved by starting an additional thread on each slave node. At the end of each second preset period, the slave node will re-collect new first summary information and compare the new summary information with the previously recorded summary information. If the newly collected first summary information is inconsistent with the previously recorded information, the slave node will detect the change. The slave node will send change information to the master node to report the change of the first summary information. After receiving the change information from the slave node, the master node will check the change information. If it is found that the state of a slave node has changed unreliably, the master node will remove the node from the cluster. The master node marks the changed nodes as untrusted and reports the status changes of these nodes to the privacy computing platform. After receiving the report from the master node, the privacy computing platform will record the untrusted slave nodes and take further measures as needed, such as blocking the operation of the node, updating the cluster status, etc.

[0114] The data processing method provided by the embodiment of the present invention can monitor the changes of data in real time by controlling each slave node to collect the first summary information through the second preset period. This real-time detection can timely detect changes in node status, which helps to respond quickly and make necessary adjustments and processing. When the changed nodes are removed in time and untrusted information is reported to the privacy computing platform, it helps to maintain the stability and security of the system. In this way, the impact of bad nodes on the normal operation of the system can be prevented, and the reliability of data processing and computing tasks can be guaranteed.

[0115] Step S4014, obtain the information of the newly added slave node; when the cluster server is to be expanded, control the master node to report the information of the newly added slave node to the privacy computing platform; when the privacy computing platform reports and approves, the newly added slave node will be added to the cluster server.

[0116] Specifically, the cluster server will detect that a new slave node has joined. Once a new node is discovered, the cluster server will start to collect relevant information about the node and control the master node to send a report to the privacy computing platform, indicating that a new slave node needs to be added to the cluster. After receiving the report from the master node, the privacy computing platform will review and confirm it. If the privacy computing platform confirms that the report is correct, it will approve the addition of the new node to the cluster.

[0117] In the data processing method provided by the embodiment of the present invention, the master node reports the information of the newly added slave nodes to the privacy computing platform, and adds them to the cluster only after the report is passed, which helps to ensure the orderliness and compliance of the expansion process. Through the reporting and verification process, the newly added slave nodes can be reviewed to ensure that they meet the security and performance requirements of the cluster. This reduces the risk of introducing potential untrusted nodes and improves the security of the cluster.

[0118] Step S4015, obtain the information of the slave nodes to be removed; when the cluster server is to be scaled down, control the master node to report the information of the slave nodes to be removed to the privacy computing platform.

[0119] Specifically, the cluster server has a monitoring system or health check mechanism to regularly check the availability of itself and each node. If a slave node is detected to be unavailable, the monitoring system will record the relevant fault information. Once a slave node is detected to be unavailable, the cluster server will automatically collect relevant information about the node. Based on the fault information detected by the monitoring system, the master node is controlled to send a report to the privacy computing platform, indicating that a slave node is unavailable and needs to be processed. After receiving the report from the master node, the privacy computing platform will perform corresponding processing. If a scaling-down operation is required, the privacy computing platform will instruct the master node to remove the unavailable node (i.e., the slave node to be removed) from the cluster or perform other cleanup and processing operations.

[0120] The data processing method provided by the embodiment of the present invention enables the master node to obtain the information of the slave node to be removed in a timely manner, so that the state change of the node can be monitored in real time. This real-time monitoring can quickly discover node failures or failures, which helps to take necessary capacity reduction measures in a timely manner.

[0121] Step S4016, obtaining status information of the cluster server; when the cluster server returns to normal from the abnormal state, the control master node collects the first summary information from the cluster server that has returned to normal.

[0122] Specifically, the cluster server monitors its own status in real time according to the monitoring system or health check mechanism it has. For example, the monitoring system can generate logs to record the running status and events of the cluster server. Once the cluster server notifies the master node that the status of the cluster server has returned to normal from the abnormal state, the master node can continue to collect the first summary information.

[0123] According to the data processing method provided by the embodiment of the present invention, after the cluster server recovers from an abnormal state, the master node can collect the first summary information from the cluster server that has recovered to normal, which ensures continuous monitoring and updating of the cluster status and helps to timely understand the health status of the recovery node.

[0124] Step S402: Receive a trust report request sent by the client. Figure 3 Step S302 of the illustrated embodiment will not be described in detail here.

[0125] Step S403: Send the trusted report request to the master node so that the master node generates a second trusted report based on the trusted report request, and reports the second trusted report to the privacy computing platform through the client, so that the privacy computing platform compares the first trusted report and the second trusted report to generate a report comparison result.

[0126] In some optional implementations, the control master node collects second summary information of the plurality of slave nodes based on the trusted report request, and generates a second trusted report based on the second summary information.

[0127] Specifically, the master node obtains the second summary information from the plurality of slave nodes according to the existing trusted report request, and generates a second trusted report using the second summary information to ensure the reliability and security of data processing.

[0128] Step S404: Based on the trusted channel between the client and the cluster server, a database operation statement sent by the client is received, and the database operation statement includes encryption parameters; the trusted channel is generated when the second trusted report is determined to be credible based on the report comparison result. For details, please refer to Figure 3 Step S304 of the illustrated embodiment will not be described in detail here.

[0129] Step S405: Read and write ciphertext to the specified external data source according to the encryption parameters. Figure 3 Step S305 of the illustrated embodiment will not be described in detail here.

[0130] In this embodiment, a data processing method is provided, which can be used in a cluster server. Figure 5 is a flow chart of a data processing method according to an embodiment of the present invention. Figure 5 As shown, the process includes the following steps:

[0131] Step S501: Control the master node to generate a first trusted report during initial registration, and report the first trusted report to the privacy computing platform. Figure 4 Step S401 of the illustrated embodiment will not be described in detail here.

[0132] Step S502: Receive a trust report request sent by the client. Figure 4 Step S402 of the illustrated embodiment will not be described in detail here.

[0133] Step S503: Send the trusted report request to the master node, so that the master node generates a second trusted report based on the trusted report request, and reports the second trusted report to the privacy computing platform through the client, so that the privacy computing platform compares the first trusted report and the second trusted report to generate a report comparison result. Figure 4 Step S403 of the illustrated embodiment will not be described in detail here.

[0134] Step S504: Based on the trusted channel between the client and the cluster server, a database operation statement sent by the client is received, and the database operation statement includes encryption parameters; the trusted channel is generated when the second trusted report is determined to be credible based on the report comparison result. For details, please refer to Figure 4 Step S404 of the illustrated embodiment will not be described in detail here.

[0135] Step S505: read and write ciphertext to the specified external data source according to the encryption parameters.

[0136] Specifically, the above step S505 includes:

[0137] Step S5051, identifying the encryption parameters in the database operation statement of the client, and encapsulating the encryption parameters into the first operator of the syntax parsing stage.

[0138] Specifically, the database operation statement may include, for example, SQL. The operation of SQL involves the stages of syntax analysis, logical planning, logical optimization, physical planning, and physical execution.

[0139] The first operator refers to the newly defined parsing operator InitAnalyzerPlan. Specifically, during the syntax parsing phase, the SQL parser is modified to recognize encryption parameters additionally defined in the user SQL, such as encryption columns, encryption algorithms, encryption methods (private keys or public keys), private keys, and other information. These encryption parameters are encapsulated into the newly defined parsing operator InitAnalyzerPlan, i.e., the first operator. The first operator can be used to encapsulate data.

[0140] It should also be noted that if the database operation statement does not contain encryption parameters, it will be processed according to the normal process.

[0141] Step S5052: Convert the first operator into a second operator in the logic planning stage by using a preset conversion rule.

[0142] The second operator refers to the newly defined operator EncryInfoPlan. Specifically, in the logical plan phase, the SQL conversion rule is modified to convert the encrypted parameters to the newly defined operator EncryInfoPlan (the second operator) after extracting the encrypted parameters from the parsing operator InitAnalyzerPlan. The second operator contains conversion characteristics, and the processing mechanism inside the cluster server can be applied to convert the second operator to the operator in the physical plan phase.

[0143] Step S5053: If the second operator includes a read statement, the read statement is converted into a read operation operator in the physical planning phase, the read operation operator is used to decrypt and calculate the external data source, and the calculation result is returned to the client through a trusted channel.

[0144] The read statement can be, for example, a select statement, and the read operation operator is the JDBCDataSourceScanExec physical operator for the select statement. Specifically, the attribute information of the read operation operator is read, each piece of data is first decrypted, then calculated, and then the calculation result is returned to the client through a trusted channel. JDBCDataSourceScanExec can support predicate push-down optimization and specified column reading optimization. For example, the data in the data source can be filtered before reading, and only qualified data can be read, thereby reducing the amount of data read from the data source, achieving the effects of shortening data retrieval time and reducing unnecessary memory cache, and constant overlay optimization to reduce unnecessary calculations.

[0145] Step S5054: If the second operator includes a write statement, the write statement is converted into a write operation operator in the physical planning stage, and the write operation operator is used to perform decryption, calculation and encryption operations on the external data source to obtain the encrypted data to be written, and the encrypted data is written to the external data source.

[0146] The write statement may be, for example, an Insert statement, and the write operator is the InsertDataSourceExec physical operator for the insert statement. Specifically, the attribute information of the write operator is read, each piece of data is first decrypted, and the result data is calculated and encrypted to obtain the encrypted data to be written. The encrypted data is written to the external data source in batches, thereby reducing the number of writes and improving throughput.

[0147] The data processing method provided by the embodiment of the present invention protects sensitive data and prevents unauthorized access and data leakage by using encryption in the reading and writing process. It supports flexible reading and writing of external data sources, allows necessary calculations to be performed in an encrypted state, and improves the efficiency of data processing. The calculation results are transmitted through a trusted channel to ensure the security of data during transmission and enhance the trust of the overall system. Encapsulating encryption parameters into the syntax parsing stage ensures that security is considered in the initial stage of data operations and reduces the complexity of subsequent stages. By dividing operations into logical plans and physical plans, the query execution process is optimized and the overall performance and maintainability are improved. Calculations are performed in a ciphertext state, which reduces plaintext operations and reduces potential security risks.

[0148] In this embodiment, a data processing method is provided, which can be used in a client. Figure 6 is a flow chart of a data processing method according to an embodiment of the present invention. Figure 6 As shown, the process includes the following steps:

[0149] Step S601: Send a trusted report request to a cluster server, so that a master node in the cluster server generates a second trusted report based on the trusted report request.

[0150] Specifically, the client initiates a trusted report request, requesting the master node in the cluster server to generate a second trusted report. The trusted report request triggers the processing flow of the master node to generate a new report reflecting the current cluster status.

[0151] Step S602: Report the second trusted report to the privacy computing platform so that the privacy computing platform compares the first trusted report and the second trusted report to generate a report comparison result.

[0152] Specifically, the master node generates a second trusted report based on the received trusted report request. The second trusted report will be sent to the privacy computing platform, which will compare it with the first trusted report previously reported. The comparison results will be used to analyze the credibility and status of the cluster server to ensure that the system has not been tampered with or damaged.

[0153] Step S603, based on the trusted channel between the client and the cluster server, send a database operation statement to the cluster server, so that the cluster server reads and writes ciphertext to the specified external data source according to the encryption parameters included in the database operation statement; the trusted channel is generated when the second trusted report is determined to be credible based on the report comparison result.

[0154] Specifically, the client sends database operation statements to the cluster server through a secure and trusted channel. These statements contain necessary encryption parameters. The cluster server will read and write external data sources based on these parameters to ensure privacy protection during data storage and transmission.

[0155] The data processing method provided in the embodiment of the present invention ensures the accuracy and credibility of information by generating a second trusted report and comparing it with the first trusted report, which helps to improve the trust of the overall system. The report comparison mechanism can effectively detect potential malicious behavior and prevent data from being tampered with during transmission and processing. The use of trusted channels for database operations ensures the secure transmission of data between the client and the cluster server, reducing the risk of data leakage. It supports ciphertext reading and writing according to encryption parameters, allowing flexible operations on external data sources while ensuring data security.

[0156] In this embodiment, a data processing system is provided. Figure 7 As shown, the data processing system includes: a client 1, a cluster server 2 and a privacy computing platform 3; wherein the client 1, the cluster server 2 and the privacy computing platform 3 are communicatively connected.

[0157] The cluster server 2 is used to obtain the first trusted report, and to report the first trusted report to the privacy computing platform 3.

[0158] Specifically, the cluster server 2 obtains the first trusted report generated by the master node and sends it to the privacy computing platform 3.

[0159] Client 1 is used to send a trusted report request to cluster server 2 .

[0160] Specifically, client 1 initiates a request to cluster server 2 to obtain a trusted report.

[0161] The cluster server 2 is used to send a trusted report request to the master node, so that the master node generates a second trusted report based on the trusted report request, and sends the second trusted report to the privacy computing platform 3 through the client 1.

[0162] Specifically, cluster server 2 sends a trusted report request to the master node to generate a second trusted report. The generated second trusted report is reported to privacy computing platform 3 through client 1, and computing platform 3 will use this report for comparison and analysis.

[0163] The privacy computing platform 3 is used to compare the first trusted report and the second trusted report, and to send the report comparison result to the client 1.

[0164] The privacy computing platform 3 receives the second trusted report sent by the cluster server 2, compares it with the first trusted report received previously, and returns the comparison result to the client 1.

[0165] When the client 1 determines that the second trusted report is credible based on the report comparison result, the client 1 is further configured to establish a trusted channel with the cluster server 2 .

[0166] Specifically, when client 1 determines that the second trusted report is credible based on the report comparison result, client 1 needs to establish a trusted channel with cluster server 2 so that cluster server 2 can safely perform encrypted reading and writing operations with external data sources to ensure that data will not be tampered with or leaked during transmission.

[0167] The cluster server 2 is further used to obtain the encryption parameters provided by the client 1 and to read and write ciphertext to a designated external data source based on the trusted channel and in accordance with the encryption parameters.

[0168] The cluster server 2 receives the encryption parameters from the client 1 and performs ciphertext reading and writing operations on the specified external data source based on the established trusted channel.

[0169] The data processing system provided by the embodiment of the present invention ensures the credibility and integrity of the data by comparing the first trusted report with the second trusted report. This comparison mechanism helps to ensure that the data has not been tampered with or modified during the data transmission and processing process, thereby improving the security and reliability of data processing. When the client determines that the second trusted report is credible, a trusted channel can be established with the cluster server. These secure channels ensure the confidentiality and integrity of the data during transmission, and prevent the risk of data leakage and unauthorized access. As an intermediate node, the cluster server is responsible for processing requests and forwarding of trusted reports, and also assumes the function of establishing trusted channels and performing ciphertext read and write operations. This distributed processing capability enables the system to handle large-scale data and complex computing tasks, thereby improving the scalability and efficiency of the system.

[0170] In the present embodiment, a data processing device is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware of a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceived.

[0171] This embodiment provides a data processing device, such as Figure 8 As shown, including:

[0172] The first receiving module 701 is used to receive a first trusted report and a second trusted report reported by the cluster server; the first trusted report is generated by the master node of the cluster server during initial registration, and the second trusted report is generated by the cluster server based on a trusted report request sent by the client;

[0173] The comparison module 702 is used to compare the first trusted report and the second trusted report, generate a report comparison result, and send the report comparison result to the client, so that the client determines whether the second trusted report is credible based on the report comparison result. When the second trusted report is credible, the client establishes a trusted channel with the cluster server, so that the cluster server performs encrypted reading and writing operations on the external data source based on the trusted channel; a structured query language engine is deployed in the cluster server, and the structured query language engine is an engine for performing encrypted reading and writing of external data source data.

[0174] In some optional implementations, the first receiving module 701 includes:

[0175] The startup submodule is used to start multiple slave nodes of the cluster server so that the multiple slave nodes generate first summary information and send the first summary information to the master node. The master node generates a first trusted report based on the first summary information and registers the first trusted report to the privacy computing platform.

[0176] In some optional implementations, the first receiving module 701 further includes:

[0177] A first receiving submodule is used to receive node change information reported by the cluster server, where the node change information is sent by each slave node when the collected first summary information changes, and is used to indicate that the slave node whose first summary information changes is untrustworthy;

[0178] The stop submodule is used to stop executing the task corresponding to the slave node whose first summary information has changed according to the node change information.

[0179] In some optional implementations, the first receiving module 701 further includes:

[0180] The second receiving submodule is used to receive the information of the newly added slave node reported by the cluster server and to approve the report of the newly added slave node; the information of the newly added slave node is generated when the cluster server is to be expanded;

[0181] The first indication submodule is used to instruct the cluster server to add a new slave node when it is determined that the report is passed.

[0182] In some optional implementations, the first receiving module 701 further includes:

[0183] The third receiving submodule is used to receive information about the slave nodes to be removed reported by the cluster server, where the information about the slave nodes to be removed is generated when the cluster server is to be scaled down;

[0184] The removal submodule is used to remove the slave node to be removed from the cluster server according to the information of the slave node to be removed.

[0185] This embodiment also provides a data processing device, such as Fig. 9 As shown, including:

[0186] The first control module 801 is used to control the master node to generate a first trusted report during initial registration, and report the first trusted report to the privacy computing platform;

[0187] The second receiving module 802 is used to receive a trusted report request sent by the client;

[0188] A first sending module 803 is used to send a trusted report request to the master node, so that the master node generates a second trusted report based on the trusted report request, and reports the second trusted report to the privacy computing platform through the client, so that the privacy computing platform compares the first trusted report with the second trusted report and generates a report comparison result;

[0189] The third receiving module 804 is used to receive a database operation statement sent by the client based on a trusted channel between the client and the cluster server, the database operation statement including an encryption parameter; the trusted channel is generated when the second trusted report is determined to be credible based on the report comparison result;

[0190] The reading and writing module 805 is used to read and write ciphertext of a specified external data source according to encryption parameters.

[0191] In some optional implementations, the first control module 801 includes:

[0192] The first control submodule is used to control the multiple slave nodes to collect corresponding node information according to a first preset period when the multiple slave nodes are in a startup state, and send first summary information generated according to the node information to the master node, so that the master node generates a first trusted report based on the first summary information, and the node information includes at least one of hardware information, kernel information and immutable information.

[0193] In some optional implementations, the first control module 801 further includes:

[0194] The second control submodule is used to control each slave node to collect the first summary information according to the second preset period, so that when each slave node determines that the corresponding first summary information has changed, it sends the change information to the master node, so that the master node will eliminate the changed slave nodes and report to the privacy computing platform that the changed slave nodes are untrustworthy.

[0195] In some optional implementations, the first control module 801 further includes:

[0196] The first acquisition submodule is used to obtain information of the newly added slave node;

[0197] The third control submodule is used to control the master node to report the information of the newly added slave nodes to the privacy computing platform when the cluster server is to be expanded;

[0198] A submodule is added to add new slave nodes to the cluster server after the privacy computing platform report is passed.

[0199] In some optional implementations, the first control module 801 further includes:

[0200] The second acquisition submodule is used to obtain information of the slave node to be removed;

[0201] The fourth control submodule is used to control the master node to report the information of the slave nodes to be removed to the privacy computing platform when the cluster server is to be scaled down.

[0202] In some optional implementations, the first control module 801 further includes:

[0203] The third acquisition submodule is used to obtain the status information of the cluster server;

[0204] The fifth control submodule is used to control the master node to collect the first summary information from the cluster server that has returned to normal when the cluster server returns to normal from the abnormal state.

[0205] In some optional embodiments, the above device further includes:

[0206] The second control module is used to control the master node to collect second summary information of multiple slave nodes based on the trusted report request, and generate a second trusted report based on the second summary information.

[0207] In some optional implementations, the read-write module 805 includes:

[0208] The identification submodule is used to identify the encryption parameters in the database operation statement of the client and encapsulate the encryption parameters into the first operator of the syntax parsing stage;

[0209] A conversion submodule, used to convert the first operator into a second operator in the logic planning stage by using a preset conversion rule;

[0210] A first processing submodule is used for converting the read statement into a read operation operator in the physical planning phase if the second operator includes a read statement, decrypting and calculating the external data source by using the read operation operator, and returning the calculation result to the client through the trusted channel;

[0211] The second processing submodule is used to convert the write statement into a write operation operator in the physical planning stage if the second operator includes a write statement, use the write operation operator to decrypt, calculate and encrypt the external data source to obtain the encrypted data to be written, and write the encrypted data into the external data source.

[0212] This embodiment also provides a data processing device, such as Fig.10 As shown, including:

[0213] The second sending module 901 is used to send a trusted report request to the cluster server, so that the master node in the cluster server generates a second trusted report based on the trusted report request;

[0214] A third sending module 902 is used to report the second trusted report to the privacy computing platform, so that the privacy computing platform compares the first trusted report with the second trusted report and generates a report comparison result;

[0215] The fourth sending module 903 is used to send database operation statements to the cluster server based on the trusted channel between the client and the cluster server, so that the cluster server reads and writes ciphertext on the specified external data source according to the encryption parameters included in the database operation statements; the trusted channel is generated when the second trusted report is determined to be credible based on the report comparison result.

[0216] The further functional description of each of the above modules and units is the same as that of the above corresponding embodiments and will not be repeated here.

[0217] The data processing device in this embodiment is presented in the form of a functional unit, where the unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.

[0218] The data processing device provided by the embodiment of the present invention ensures that the client can verify the credibility of the cluster server by comparing the first trusted report and the second trusted report reported by the cluster server, thereby preventing the risk of cluster nodes being tampered with. When the second trusted report is verified to be credible, the client can establish a trusted channel with the cluster server. This channel ensures secure data transmission, especially when performing ciphertext reading and writing operations on external data sources. By using trusted reports and trusted channels, the risk of data security leakage is effectively prevented, and the security and reliability of the overall data processing process are improved.

[0219] The embodiment of the present invention also provides a server having the above Figure 8 , Fig. 9 as well as Fig.10 The data processing device shown.

[0220] See also Fig.11 , Fig.11 is a schematic diagram of the structure of a server provided by an optional embodiment of the present invention, such as Fig.11 As shown, the server includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. The various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process instructions executed in the server, including instructions stored in or on the memory to display graphical information of the GUI on an external input / output device (such as a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple servers can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Fig.11 A processor 10 is taken as an example.

[0221] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.

[0222] The memory 20 stores instructions executable by at least one processor 10, so that at least one processor 10 executes the method shown in the above embodiment.

[0223] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required by at least one function; the data storage area may store data created according to the use of the server, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the server via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0224] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 20 may also include a combination of the above types of memory.

[0225] The server further comprises a communication interface 30 for the server to communicate with other devices or a communication network.

[0226] The embodiment of the present invention also provides a computer-readable storage medium. The method according to the embodiment of the present invention can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium through a network download, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state hard disk, etc.; further, the storage medium can also include a combination of the above types of memories. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.

[0227] A part of the present invention may be applied as a computer program product, such as a computer program instruction, which, when executed by a computer, can call or provide the method and / or technical solution according to the present invention through the operation of the computer. Those skilled in the art should understand that the existence of the computer program instruction in a computer-readable medium includes, but is not limited to, a source file, an executable file, an installation package file, etc., and accordingly, the way in which the computer program instruction is executed by the computer includes, but is not limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding installed program. Here, the computer-readable medium may be any available computer-readable storage medium or communication medium accessible to the computer.

[0228] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A data processing method, characterized in that: Applied to a privacy computing platform, the method includes: Receive a first trusted report and a second trusted report reported by a cluster server; the first trusted report is generated by a master node of the cluster server during initial registration, and the second trusted report is generated by the cluster server based on a trusted report request sent by a client; The first trusted report and the second trusted report are compared to generate a report comparison result, and the report comparison result is sent to the client, so that the client determines whether the second trusted report is credible according to the report comparison result. When the second trusted report is credible, the client establishes a trusted channel with the cluster server, so that the cluster server performs ciphertext reading and writing operations on the external data source based on the trusted channel. A structured query language engine is deployed in the cluster server, and the structured query language engine is an engine for performing ciphertext reading and writing on external data source data.

2. The method according to claim 1, characterized in that Receiving a first credible report reported by the cluster server, including: Start multiple slave nodes of the cluster server so that the multiple slave nodes generate first summary information and send the first summary information to the master node. The master node generates the first trusted report based on the first summary information and registers the first trusted report to the privacy computing platform.

3. The method according to claim 2, characterized in that Also includes: receiving node change information reported by the cluster server, wherein the node change information is sent by each slave node when the collected first summary information changes, and is used to indicate that the slave node whose first summary information changes is untrustworthy; According to the node change information, the execution of tasks corresponding to the slave nodes whose first summary information has changed is stopped.

4. The method according to claim 3, characterized in that Also includes: Receive information of a newly added slave node reported by the cluster server, and approve the report of the newly added slave node; The information of the newly added slave node is generated when the cluster server is to be expanded; When it is determined that the report is passed, the cluster server is instructed to add the newly added slave node.

5. The method according to claim 4, characterized in that Also includes: receiving information of slave nodes to be removed reported by the cluster server, where the information of slave nodes to be removed is generated when the cluster server is to be scaled down; According to the information of the slave node to be removed, the slave node to be removed is removed from the cluster server.

6. A data processing method, characterized in that: Applied to a cluster server, the cluster server includes a master node and multiple slave nodes, the method includes: Control the master node to generate a first trusted report upon initial registration, and report the first trusted report to the privacy computing platform; Receive a trusted report request sent by a client; Sending the trusted report request to the master node, so that the master node generates a second trusted report based on the trusted report request, and reporting the second trusted report to the privacy computing platform through the client, so that the privacy computing platform compares the first trusted report and the second trusted report to generate a report comparison result; Based on the trusted channel between the client and the cluster server, receiving a database operation statement sent by the client, the database operation statement including an encryption parameter; the trusted channel is generated when it is determined that the second trusted report is credible according to the report comparison result; The encrypted data is read and written to the specified external data source according to the encryption parameters.

7. The method according to claim 6, characterized in that The control master node generates a first trusted report during initial registration, including: When the multiple slave nodes are in the startup state, the multiple slave nodes are controlled to collect corresponding node information according to a first preset period, and the first summary information generated according to the node information is sent to the master node, so that the master node generates the first trusted report based on the first summary information, and the node information includes at least one of hardware information, kernel information and immutable information.

8. The method according to claim 7, characterized in that Also includes: Control each of the slave nodes to collect the first summary information according to a second preset period, so that when each slave node determines that the corresponding first summary information has changed, it sends the change information to the master node, so that the master node will eliminate the changed slave nodes and report to the privacy computing platform that the changed slave nodes are untrustworthy.

9. The method according to claim 8, characterized in that Also includes: Get the information of the newly added slave node; When the cluster server is to be expanded, controlling the master node to report the information of the newly added slave node to the privacy computing platform; When the privacy computing platform reports successfully, the newly added slave node is added to the cluster server.

10. The method according to claim 9, characterized in that Also includes: Get the information of the slave node to be removed; When the cluster server is to be scaled down, the master node is controlled to report the information of the slave node to be removed to the privacy computing platform.

11. The method according to claim 10, characterized in that Also includes: Obtaining status information of the cluster server; When the cluster server returns to normal from the abnormal state, the master node is controlled to collect first summary information from the cluster server that returns to normal.

12. The method according to claim 6, characterized in that Also includes: The master node is controlled to collect second summary information of the plurality of slave nodes based on the trusted report request, and to generate the second trusted report based on the second summary information.

13. The method according to claim 6, characterized in that The ciphertext reading and writing of the specified external data source according to the encryption parameters includes: Identifying the encryption parameter in the database operation statement of the client, and encapsulating the encryption parameter into the first operator of the syntax parsing stage; Converting the first operator into a second operator in a logical planning phase using a preset conversion rule; If the second operator includes a read statement, convert the read statement into a read operation operator in the physical plan phase, use the read operation operator to perform decryption and calculation operations on the external data source, and return the calculation result to the client through the trusted channel; If the second operator includes a write statement, the write statement is converted into a write operation operator in the physical planning stage, and the write operation operator is used to perform decryption, calculation and encryption operations on the external data source to obtain encrypted data to be written, and the encrypted data is written into the external data source.

14. A data processing method, characterized in that: Applied to a client, the method comprises: Sending a trusted report request to a cluster server, so that a master node in the cluster server generates a second trusted report based on the trusted report request; Reporting the second trusted report to the privacy computing platform, so that the privacy computing platform compares the first trusted report and the second trusted report to generate a report comparison result; Based on the trusted channel between the client and the cluster server, a database operation statement is sent to the cluster server so that the cluster server reads and writes ciphertext on the designated external data source according to the encryption parameters included in the database operation statement; the trusted channel is generated when the second trusted report is determined to be credible based on the report comparison result.

15. A data processing system, characterized in that: It includes a client, a cluster server and a privacy computing platform, and the client, the cluster server and the privacy computing platform are communicatively connected with each other; The cluster server is used to obtain a first trusted report, and to report the first trusted report to the privacy computing platform; The client is used to send a trusted report request to the cluster server; The cluster server is used to send the trusted report request to the master node, so that the master node generates a second trusted report based on the trusted report request, and sends the second trusted report to the privacy computing platform through the client; The privacy computing platform is used to compare the first trusted report and the second trusted report, and to send the report comparison result to the client; When the client determines that the second credible report is credible based on the report comparison result, the client is further configured to establish a credible channel with the cluster server; The cluster server is further used to obtain the encryption parameters provided by the client, and to perform ciphertext reading and writing on a designated external data source based on the trusted channel and in accordance with the encryption parameters.

16. A server, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the data processing method according to any one of claims 1 to 5, the data processing method according to any one of claims 6 to 13, and the data processing method according to claim 14 by executing the computer instructions.

17. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the data processing method described in any one of claims 1 to 5, the data processing method described in any one of claims 6 to 13, and the data processing method described in claim 14.

18. A computer program product, characterized in that The method comprises computer instructions for causing a computer to execute the data processing method according to any one of claims 1 to 5, the data processing method according to any one of claims 6 to 13, and the data processing method according to claim 14.

Images