A data processing method and device

Abstract

The embodiment of the present application provides a data processing method, device, equipment and computer storage medium, wherein the method includes: sending a data acquisition request to a data node; receiving target data and an operation account book returned by the data node, wherein the target data is data generated by performing a preprocessing operation on the source data, and the operation account book is used to record the operation information of the preprocessing operation; using the operation account book to audit and verify the target data; if the target data passes the audit and verification, the target data is added to the aggregate data set. The embodiment of the present application can improve the security of the data processing process.

Description

Technical Field

[0001] The present invention relates to the field of Internet technology, specifically to the field of data processing technology, and in particular to a data processing method and a data processing device. Background Art

[0002] Many Internet application scenarios (such as insurance purchase scenarios, bank lending scenarios, advertising scenarios, etc.) involve data processing. Since the processed data usually contains some private data, such as the user's deposit data (such as the specific deposit amount), some private social data of the user (such as personal address, some private pictures), etc., the data processing process needs to set up a protection mechanism to protect the private data from being leaked during the processing process.

[0003] The protection mechanism commonly used in the prior art is the pre-code review mechanism, which is to review all the code programs used in the data processing process manually or with the help of professional tools before executing the data processing process to see if they are reliable. If they are reliable, these code programs are allowed to execute the data processing process. However, this pre-code review mechanism has limited protection for data, and it is difficult to predict the security of the code program during the actual execution process. Summary of the invention

[0004] The embodiments of the present application provide a data processing method, apparatus, device and computer storage medium, which can improve the security of the data processing process.

[0005] In a first aspect, an embodiment of the present application provides a data processing method, including:

[0006] Send a data acquisition request to the data node;

[0007] Receiving target data and an operation ledger returned by the data node, wherein the target data is data generated by performing a preprocessing operation on the source data, and the operation ledger is used to record operation information of the preprocessing operation;

[0008] Using the operation account book to audit and verify the target data;

[0009] If the target data passes the audit verification, the target data is added to the aggregated data set.

[0010] In a second aspect, the embodiment of the present application also provides another data processing method, including:

[0011] Receive a data acquisition request sent by a processing node;

[0012] Performing a preprocessing operation on the source data according to the data acquisition request to generate target data;

[0013] Using an operation book to record operation information of the pre-processing operation;

[0014] The target data and the operation ledger are returned to the processing node, so that the processing node uses the operation ledger to audit and verify the target data, and when the target data passes the audit and verification, the target data is added to the aggregated data set.

[0015] In a third aspect, an embodiment of the present application provides a data processing device, including:

[0016] A sending unit, used for sending a data acquisition request to a data node;

[0017] A receiving unit, configured to receive target data and an operation ledger returned by the data node, wherein the target data is data generated by performing a preprocessing operation on the source data, and the operation ledger is used to record operation information of the preprocessing operation;

[0018] An audit verification unit, used to audit and verify the target data using the operation account book;

[0019] A processing unit is configured to add the target data to the aggregated data set if the target data passes the audit verification.

[0020] In a fourth aspect, the embodiment of the present application also provides another data processing device, including:

[0021] A receiving unit, used for receiving a data acquisition request sent by a processing node;

[0022] A preprocessing operation unit, used to perform a preprocessing operation on the source data according to the data acquisition request to generate target data;

[0023] A recording unit, used to record the operation information of the pre-processing operation in an operation account book;

[0024] A sending unit is used to return the target data and the operation ledger to the processing node, so that the processing node uses the operation ledger to audit and verify the target data, and when the target data passes the audit and verification, the target data is added to the aggregated data set.

[0025] In a fifth aspect, an embodiment of the present application provides a data processing device, including an input interface and an output interface, and further including:

[0026] a processor adapted to implement one or more instructions; and,

[0027] A computer storage medium storing one or more first instructions, wherein the one or more first instructions are suitable for being loaded by the processor and executing the data processing method of the first aspect; or, the computer storage medium storing one or more second instructions, wherein the one or more second instructions are suitable for being loaded by the processor and executing the data processing method of the second aspect.

[0028] In a sixth aspect, an embodiment of the present application provides a computer storage medium, wherein the computer storage medium stores one or more first instructions, and the one or more instructions are suitable for being loaded by a processor and executing the data processing method of the first aspect above; or, the computer storage medium stores one or more second instructions, and the one or more second instructions are suitable for being loaded by the processor and executing the data processing method of the second aspect above.

[0029] In an embodiment of the present application, an operation ledger is used to perform a secure and reliable audit verification on the target data provided by the data node, and the target data is data generated by performing a preprocessing operation on the source data; through this audit verification process, it can be ensured that the preprocessing operation is performed in accordance with the processing rules jointly recognized by the source data owner (such as the data node) and the processing node, and it can be ensured that the target data can be successfully added to the aggregated data set for use by subsequent processes, and at the same time, the privacy data in the source data will not be leaked; at the same time, it can also be ensured that all data in the aggregated data set are reliable data, which is beneficial to ensure the security of subsequent processes using the aggregated data set, thereby improving the security of the data processing process. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0031] Figure 1 A diagram of a blockchain infrastructure provided by an exemplary embodiment of the present application is shown;

[0032] Figure 2 A schematic diagram of the structure of a blockchain provided by an exemplary embodiment of the present application is shown;

[0033] Figure 3 A schematic diagram of the architecture of a blockchain network provided by an exemplary embodiment of the present application is shown;

[0034] Figure 4 A schematic diagram of the architecture of a typical data processing system provided by an exemplary embodiment of the present application is shown;

[0035] Figure 5 A flowchart of a data processing method provided by an exemplary embodiment of the present application is shown;

[0036] Figure 6 A schematic diagram of storage of an operation account book provided by an exemplary embodiment of the present application is shown;

[0037] Figure 7a A schematic diagram of an audit smart contract provided by an exemplary embodiment of the present application is shown;

[0038] Figure 7b A schematic diagram of another audit smart contract provided by an exemplary embodiment of the present application is shown;

[0039] Figure 8 A flowchart of a data processing method provided by an exemplary embodiment of the present application is shown;

[0040] Fig. 9 A data flow diagram of a data processing method provided by an exemplary embodiment of the present application is shown;

[0041] Fig.10 A schematic diagram of the structure of a data processing device provided by an exemplary embodiment of the present application is shown;

[0042] Fig.11 A schematic diagram showing the structure of another data processing device provided by an exemplary embodiment of the present application is shown;

[0043] Fig.12 A structural schematic diagram of a data processing device provided by an exemplary embodiment of the present application is shown. DETAILED DESCRIPTION

[0044] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0045] The embodiments of this application involve blockchain. Blockchain refers to a decentralized infrastructure with distributed storage characteristics. Specifically, it is a data structure that organizes data blocks in a time sequence in a manner similar to a linked list. It can safely store data that has a sequential relationship and can be verified within the system, and cryptographically ensure that the data cannot be tampered with or forged.

[0046] Figure 1A diagram of the blockchain infrastructure provided by an exemplary embodiment of the present application is shown; Figure 1 As shown in the figure, the blockchain infrastructure mainly includes 5 hierarchical structures from 101 to 105 in order from bottom to top. Among them:

[0047] (1) Information data and Merkle trees are located at the bottom layer 101. Information data here refers to the original data that has been requested to be published to the blockchain network but has not yet formed a block, such as loan data, transaction data, etc. These original data need to be further processed (for example, verified by each node in the blockchain network, hashed, etc.) before they can be written into the block. The Merkle tree is an important part of blockchain technology. The blockchain will not directly save the original data in plain text. The original data needs to be hashed and stored in the form of hash values. The Merkle tree is used to organize the hash values ​​formed by hashing multiple original data according to a binary tree structure and save them in the block body of the block.

[0048] (2) Blocks are located at level 102. Blocks are data blocks. The information data in the bottom layer 101 is further processed and written into the blocks in level 102. Multiple blocks are connected in sequence into a chain structure, forming a blockchain. Figure 2 A schematic diagram of the structure of a blockchain provided by an exemplary embodiment of the present application is shown; Figure 2 As shown, block 201, block 202 and block 203 are sequentially connected into a chain structure. Block 202 is divided into two parts: a block header and a block body. The block header contains the summary value of the previous block 201, the summary value of the current block 202 and the Merkle root of the current block. The block body contains the complete data of the current block 202 and is organized in the form of a Merkle tree.

[0049] (3) The protocols and mechanisms followed by the blockchain are located at level 103. These protocols may include: P2P (Peer-to-Peer) protocol; mechanisms may include but are not limited to: broadcast mechanism, consensus mechanism (including PoW (Proof Of Work) mechanism, POS (Proof Of Stake) mechanism and other core mechanisms).

[0050] (4) The blockchain network is located at level 104. The blockchain network consists of multiple nodes; devices that can serve as nodes may include but are not limited to: PC (Personal Computer), server, smart phone, tablet computer, mobile computer, etc. Figure 3A schematic diagram of the architecture of a blockchain network provided by an exemplary embodiment of the present application is shown; the figure takes 7 nodes as an example for explanation, and the nodes in the blockchain network are networked in a P2P manner, and the nodes communicate with each other according to the P2P protocol; the nodes jointly follow the broadcast mechanism and consensus mechanism (including core mechanisms such as the PoW mechanism and the POS mechanism) to jointly ensure that the data on the blockchain cannot be tampered with or forged, and at the same time realize the decentralization and trustlessness of the blockchain.

[0051] (5) Smart contracts are located in the upper layer 105. Smart contracts are a set of scenario-responsive programmatic rules and logic, and are decentralized, information-sharing program codes deployed on the blockchain. The parties to the contract reach an agreement on the content of the contract, which is deployed in the blockchain in the form of a smart contract, and can automatically execute the contract on behalf of the signatories without relying on any central agency.

[0052] Due to the characteristics of blockchain such as decentralization, distributed storage, and data immutability and non-forgeability, more and more business activities (such as lending activities and financial transaction activities) are carried out based on blockchain technology, using the characteristics of blockchain to ensure the fairness and openness of business activities.

[0053] The embodiments of the present application involve aggregate calculation. The so-called aggregate calculation refers to the calculation process of aggregating multiple data into one data. In the data processing process of many Internet application scenarios, the process of aggregate calculation is usually involved; for example: in the insurance purchase scenario, the verification of the premium payable by the user is based on the basic insurance data of the user, and the basic insurance data of the user is obtained by aggregate calculation through multiple historical behavior data of the user, and the multiple historical behavior data here can be the historical diagnosis and treatment data of the user in multiple medical institutions within the set historical time period, etc. For another example: in the bank loan scenario, the assessment of the loan amount allowed to the user is based on the loan qualification evaluation data of the user, and the loan qualification evaluation data of the user is obtained by aggregate calculation of multiple historical asset data of the user, and the multiple historical asset data here can be the historical deposit data or historical loan data of the user in multiple banks. For another example: in the advertising delivery scenario, the decision on what type of advertisement to deliver to the user is based on the user's interest data, and the user's interest data is obtained by aggregate calculation of multiple historical social data of the user, and the multiple historical social data here can be the historical social data of the user in multiple social platforms.

[0054] Figure 4 FIG. 1 shows a typical data processing system architecture diagram provided by an exemplary embodiment of the present application; Figure 4As shown, the data processing system includes a processing node 402, a plurality of data nodes 401 connected to the processing node 402, and a service node 403 connected to the processing node.

[0055] in:

[0056] Data node 401 refers to a device that can provide target data suitable for use in a data processing process (such as an aggregate computing process). In a specific implementation, the data node may include but is not limited to: PC (Personal Computer), PDA (Tablet Computer), mobile phone, smart wearable device, server and other devices. In one embodiment, data node 401 may be the owner of the source data, and the data node 401 has a preprocessing capability, can perform preprocessing operations on the source data to obtain target data, and provide the target data to the aggregate computing process. In another embodiment, data node 401 may be a device independent of the owner of the source data, and the data node 401 can obtain the source data from the owner of the source data, and perform preprocessing operations on the source data to obtain the target data. Here, the owner of the source data may be a device that stores the source data. For example, if the source data is the user's historical medical data, the owner of the source data may be a service device of each medical institution that the user has visited in the past, which is used to store the user's historical medical data. For another example, if the source data is the user's historical deposit data or historical loan data, the owner of the source data may be a service device of each bank that the user has visited in the past, which is used to store the user's historical deposit data or historical loan data. For another example, if the source data is the user's historical social data, the owner of the source data may be a service device of each social platform system that the user has visited in the past, which is used to store the user's historical social data.

[0057] The service node 403 is a requesting device that initiates a data processing request to obtain aggregated response data; the service node 403 may include but is not limited to: PC, PDA (tablet computer), mobile phone, smart wearable device, server and other devices; for example: in the insurance purchase scenario, the insurance company employee initiates a data processing request to the processing node through the terminal device based on the need to verify the user's payable premium, so as to request the processing node 402 to aggregate the historical diagnosis and treatment data of the user in multiple medical institutions within the set historical time period to obtain the user's insurance basic data, then the terminal device used by the insurance company employee is the service node 403. For another example: in the bank loan scenario, the bank staff uses the terminal device to initiate a data processing request based on the need to assess the user's allowed loan amount, so as to request the processing node 402 to aggregate the user's multiple historical asset data to obtain the user's loan qualification assessment data, then the terminal device used by the insurance company employee is the service node 403. For another example, in an advertising delivery scenario, an advertiser uses the advertiser's server to initiate a data processing request based on the need to decide what type of advertisement to deliver to a user, requesting aggregation calculation of multiple historical social data of the user to obtain the user's interest data. In this case, the server used by the advertiser is business node 403.

[0058] The processing node 402 can be used to perform data processing (such as intelligent computing) processes, and the service node 402 can include but is not limited to: PC, PDA (tablet computer), mobile phone, smart wearable device, server and other devices. Specifically, the processing node 402 can receive a data processing request from the service node 403, determine the relevant multiple data nodes 401 according to the data processing request, trigger the multiple data nodes 401 to provide target data for aggregate calculation; then perform aggregate calculation on these target data to obtain the response data required by the service node; and finally return the response data to the service node 403. For example: in the insurance purchase scenario, the processing node 402 receives a data processing request sent by the service node 403 (the terminal device used by insurance company practitioners), analyzes the data processing request, and can determine that the service devices of multiple medical institutions are data nodes, and triggers these data nodes 401 to provide the user's historical diagnosis and treatment data, and performs aggregate calculation on these historical diagnosis and treatment data to obtain the user's insurance basic data and return it to the service node 403. For another example: in a bank lending scenario, processing node 402 receives a data processing request sent by business node 403 (a terminal device used by bank staff), and by analyzing the data processing request, it can be determined that service devices of multiple banks are data nodes 401, and these data nodes 401 are triggered to provide the user's historical deposit data or historical loan data, and these historical deposit data or historical loan data are aggregated and calculated to obtain the user's loan qualification evaluation data and returned to business node 403. For another example: in an advertising delivery scenario, processing node 401 receives a data processing request sent by business node 403 (a server used by advertisers), and by analyzing the data processing request, it can be determined that service devices of multiple social platforms are data nodes 401, and these data nodes 401 are triggered to provide the user's historical social data, and these historical social data are aggregated and calculated to obtain the user's interest data and returned to business node 403. It should be noted that processing node 402 can be an independent device or a combination of multiple devices; specifically, the data processing process performed by processing node 402 can be divided into multiple sub-processes. For example, according to the above description, the data processing process performed by processing node 402 can include an aggregation computing process, and a process of receiving and responding to data processing requests sent by a business node; then, if a device has both aggregation computing capabilities and communication capabilities with business nodes, then this device can act as processing node 402 to independently execute the data processing process.Of course, if one device only has aggregation computing capability and the other device has communication capability with the business node, then the combination of these two devices can be used as a processing node 402, and the two devices collaborate to execute the data processing process, for example: the device with communication capability receives the data processing request sent by the business node, and transmits it to the device with aggregation computing capability, triggering the device with aggregation computing capability to perform aggregation computing, and when the device with aggregation computing capability completes the aggregation computing and obtains the response data, it transmits it back to the device with communication capability, and the device with communication capability returns the response data to the business node.

[0059] In the data processing process, the target data required for aggregate calculation comes from the source data, which usually contains private data, such as: the user's diagnosis and treatment results (such as detailed information of the disease diagnosed by the user), the user's deposit data (such as the specific deposit amount), and some private social data of the user (such as personal address, some private pictures), etc. Therefore, the data processing process needs to set up a protection mechanism to protect the private data from being leaked during the processing process. The related art of this application mentions that the protection mechanism commonly used in the prior art is the prior code review mechanism, which is specifically: before executing the data processing process, it is required to obtain all the code programs used in the data processing process, including the code programs for preprocessing the source data, the code programs for aggregate calculation, and the code programs for other operations involved in the data processing process (such as request operations, interface operations, etc.); manually or with the help of professional tools to review whether these code programs are reliable. If they are reliable, it is confirmed that the data processing process will not steal private data, so that these code programs will be allowed to execute the data processing process. However, the prior art code review mechanism has limited protection for data. For example, if part of the code in the code program uses microcode (a type of code that is not open source), it is difficult to confirm whether these code programs have backdoor programs during the review process, and it is also difficult to predict whether these code programs will have abnormal operations during the actual execution process, which may leave security risks for the data processing process. In addition, these code programs also need to be protected, and it is impossible for them to be fully open for review in actual applications, so the security of the data processing process cannot be protected.

[0060] In order to improve the security in the data processing process, the embodiment of the present application proposes a data processing solution, which mainly includes the following technical improvements: ① Before executing the data processing process, no prior code review operation is performed, and the data processing process is directly executed. The data processing process includes two sub-processes, namely the preprocessing process and the aggregation calculation process. The two sub-processes are performed separately, but a security audit process is introduced between the two sub-processes; ② The preprocessing process is executed by the data node, which is used to perform preprocessing operations on the source data to obtain the target data. The preprocessing operation can only be performed in accordance with the processing rules jointly recognized by the source data owner (such as the data node) and the processing node to ensure that the target data can be used in the aggregation calculation process, and at the same time, the privacy data in the source data will not be leaked; ③ The concept of operating ledger is proposed , an operation ledger is used to record the operation information of the preprocessing operation; the operation ledger here is a vector ledger. The difference between the vector ledger and the conventional distributed ledger is that: first, although both the distributed ledger and the vector ledger are used to record factual data, the factual data recorded in the distributed ledger is a single data; while the vector ledger records the data flow based on mutual verification by multiple parties (parties involved in the operation, such as the business demander, the data owner, the data processor, etc.); for example, the operation information (or operation flow) recorded by the operation ledger includes the operation content of each operator (the physical device on the source data side, the interface device, and the physical device on the target data side) recorded in sequence according to the time sequence of the data operation. Any tampering of the data by any party may cause the operation flow of the vector ledger to be discontinuous, thereby ensuring the tamper-proof feature of the vector ledger. Secondly, although conventional distributed ledgers also have the property of being tamper-proof, their tamper-proof property is guaranteed by a large number of nodes backing up the same factual data; the tamper-proof property of the vector ledger is guaranteed by the correlation of the operation information between multiple operators, that is, the nodes involved in the vector ledger are related to each other based on the correlation of the operation information, and the nodes can verify each other without the need for a large number of nodes to back up, thus saving costs to a certain extent. In addition, the deployment forms of the vector ledger are relatively rich. For example, in the early stage of the formation of the vector ledger, there may be a form of interconnection between the vector ledger and the distributed ledger, and the reference facts and reference time of the vector ledger can be based on the existing timestamp nodes; when more and more mutually verifiable data streams are recorded using the vector ledger, driven by cost reduction, the vector ledger will continue to extend and cover all walks of life. Within a certain time frame, due to the causal relationship verification based on time series in the vector ledger, when false data is recorded in the vector ledger, these false data can be discovered and marked. For example, the vector ledger can be combined with big data processing and artificial intelligence reasoning to mark the false data in the data stream recorded by the vector ledger using methods such as big data processing and artificial intelligence processing.It can be seen that the combination of vector ledger, big data processing and artificial intelligence reasoning will further strengthen the tamper-proof characteristics of vector ledger. ④ In the newly added security audit process, a solution for reliable audit verification based on the operation ledger is proposed. According to the operation information of the preprocessing operation recorded in the vector ledger, each operation link of the preprocessing operation can be traced and audited and verified; if it is found that the operation record in the operation ledger has violated the audit rules, the preprocessing operation can be identified as an illegal operation, and then the target data obtained by the preprocessing operation can be intercepted, and the target data is not allowed to participate in the aggregation calculation process. The security audit process effectively connects the preprocessing process with the aggregation calculation process. Through the security audit process, it can be ensured that the preprocessing operation is executed in accordance with the processing rules jointly recognized by the source data owner (such as the data node) and the processing node, and the target data can be used by the aggregation calculation process. At the same time, the privacy data in the source data will not be leaked; at the same time, it can also ensure that all target data participating in the aggregation calculation process are reliable data, thereby ensuring the security of the aggregation calculation process and improving the overall security of the data processing process. ⑤ The audit rules used in the security audit process can be published and executed through trusted smart contracts, which can improve the efficiency and intelligence of the security audit process. ⑥ The data nodes, processing nodes, and business nodes involved in the data processing process can all be node devices in the blockchain network, and transactions are carried out in the form of transaction ledgers during the data processing process. The hierarchical relationship between transaction ledgers and the correlation between transaction ledgers and operation ledgers are proposed, ensuring the high credibility of the data processing process.

[0061] Figure 5 A flowchart of a data processing method provided by an exemplary embodiment of the present application is shown; the method can be Figure 4 The data node 401 shown interacts with the processing node 402 to achieve this; the method may include the following steps S501-S509:

[0062] S501: The processing node sends a data acquisition request to the data node.

[0063] S502: The data node receives a data acquisition request sent by the processing node.

[0064] The data acquisition request sent by the processing node is used to trigger the data node to perform preprocessing operations on the source data.

[0065] S503: The data node performs a preprocessing operation on the source data according to the data acquisition request to generate target data.

[0066] The preprocessing operation may include at least one of the following: a format conversion operation and a desensitization operation. Among them, the format conversion operation is used to perform conversion processing on the format of the source data according to the format requirements of the aggregate calculation. The purpose of the format conversion operation is to convert the source data that does not meet or does not fully meet the format requirements of the aggregate calculation into the target data that fully meets the format requirements of the aggregate calculation and is suitable for the aggregate calculation; for example: the historical diagnosis and treatment data of each medical institution is stored according to the format strategy of the medical institution, and the format of these historical diagnosis and treatment data does not necessarily meet the format requirements of the aggregate calculation. For example, if the source data (i.e., the original stored historical diagnosis and treatment data) is a description text in a natural sentence format, it needs to be converted into a digital text in a binary format for aggregate calculation. The desensitization operation is used to perform shielding processing on the privacy data in the source data; privacy data is data that the owner of the source data cannot or does not want to disclose, for example: according to the requirements of laws and regulations, medical institutions shall not disclose some privacy of patients (such as the diagnosis and treatment results of patient users); or medical institutions do not want to disclose some privacy of patients (such as the diagnosis and treatment expenses of patient users) based on their own operational needs, then these privacy data that cannot or do not want to be disclosed need to be desensitized. The purpose of the desensitization operation is to protect the privacy data in the source data from being leaked without affecting the aggregate calculation. It should be noted that the preprocessing operation is not limited to the format conversion operation and / or the desensitization operation, but may also include other operations, such as: tokenization operation.

[0067] S504: The data node uses an operation ledger to record operation information of the preprocessing operation.

[0068] S505: The data node returns the target data and the operation ledger to the processing node.

[0069] The operational ledger is a vector ledger. Figure 6 A schematic diagram of storage of an operation account book provided by an exemplary embodiment of the present application is shown; Figure 6As shown, the operation information recorded in the operation account book includes operation codes and operation parameters; wherein, the operation code includes at least one of the following: operation instructions and operation functions; the operation parameters include source data, the address of source data, the address of target data, target data and data changes caused by the operation. When the address of the source data points to the source entity device (including but not limited to: PC, PDA, mobile phone, smart wearable device, server and other devices), the address of the target data points to the target entity device (including but not limited to: PC, PDA, mobile phone, smart wearable device, server and other devices), and the source entity device and the target entity device are interconnected through an interface, the operation information also includes an operation flow; the operation flow includes: the operation time and operation content of the source entity device, the operation time and operation content of the interface operation, and the operation time and operation content of the target entity device. The operation time here can be represented by a timestamp. The operation content may include but is not limited to the following: the identification of the operator, the identification of the data being operated, the interface data flow (such as where the data being operated is transmitted from and to), the changes in the data caused by the operation (such as what format the data being operated changes from, or what value the data being operated changes from, etc.), etc. It can be seen that the operation ledger is a vector ledger based on the time sequence of operations. In one embodiment, the operation information is encrypted into a receipt and stored in the operation ledger; the encryption process here can be implemented based on various encryption algorithms, which may include any of the following: symmetric encryption algorithm, asymmetric encryption algorithm and hash algorithm.

[0070] S506: The processing node receives the target data and operation ledger returned by the data node.

[0071] S507: The processing node uses the operation account book to audit and verify the target data.

[0072] Audit verification is performed based on the operation book. Since the operation book is a vector book based on the operation time sequence, each operation link of the preprocessing operation can be traced back according to the operation information of the preprocessing operation recorded in the operation book, and then the matching audit rules can be used to audit and verify these operation links. In one embodiment, step S507 specifically includes the following sub-steps s1-s3:

[0073] s1, the processing node obtains the target audit rule that matches the operation ledger;

[0074] s2, the processing node reviews whether the operation information in the operation account book complies with the target audit rules;

[0075] s3, if it is compliant, the processing node confirms that the target data has passed the audit verification; if it is not compliant, the processing node confirms that the target data has not passed the audit verification.

[0076] Among them, the target audit rules are matched with the operation ledger, and are pre-formulated according to the actual situation and recognized by the data owner (such as data node) and the processing node. The so-called matching means that the target audit rules are formulated based on the attributes (including but not limited to type and field) corresponding to the operations recorded in the operation ledger, and are applicable to auditing and verifying the operations recorded in the operation ledger; for example: for the preprocessing operation of the user's historical diagnosis and treatment data, the matching audit rules can be formulated according to the format requirements of aggregate computing, the privacy requirements of medical institutions, and medical-related laws and regulations. For another example: for the preprocessing operation of the user's historical deposit data and historical loan data, the matching audit rules can be formulated according to the format requirements of aggregate computing, the privacy requirements of banks or financial institutions, and financial-related laws and regulations. For another example: for the preprocessing operation of the user's historical social data, the matching audit rules can be formulated according to the format requirements of aggregate computing, the privacy requirements of social platforms, and Internet-related laws and regulations. If it is found that an operation that violates the audit rules is recorded in the operation ledger, the preprocessing operation can be determined as an illegal operation, and then it is confirmed that the target data has not passed the audit verification, and the target data is not suitable for participating in the aggregate computing process. If it is found that all operations recorded in the operation ledger comply with the audit rules, the preprocessing operation can be deemed a legal operation, and then the target data can be confirmed to have passed the audit verification and can participate in the aggregate calculation process.

[0077] In another embodiment, the target audit rules may be published to the blockchain network in the form of an audit smart contract; then, sub-step s2 specifically includes the following sub-steps s21-s22:

[0078] s21, calling the audit smart contract in the blockchain network.

[0079] s22, run the execution program corresponding to the target audit rule declared in the audit smart contract, and review whether the operation information in the operation account book complies with the target audit rule.

[0080] In one implementation, an audit smart contract contains only one audit rule, and one audit rule matches one operation ledger; Figure 7a A schematic diagram of an audit smart contract provided by an exemplary embodiment of the present application is shown; see Figure 7aAs shown in the figure, operation book 1 matches audit rule 1, and audit rule 1 corresponds to audit smart contract 1; operation book 2 matches audit rule 2, and audit rule 2 corresponds to audit smart contract 2, and so on. Then, for multiple operation books, multiple audit smart contracts need to be called separately to perform audit verification.

[0081] In another embodiment, an audit smart contract may include multiple audit rules, each audit rule matching an operation ledger; Figure 7b A schematic diagram of another audit smart contract provided by an exemplary embodiment of the present application is shown; see Figure 7b As shown, operation ledger 1 matches audit rule 1, operation ledger 2 matches audit rule 2, and audit rule 1 and audit rule 2 together correspond to audit smart contract 1. Then, for multiple operation ledgers, the same audit smart contract can be called to perform audit verification.

[0082] It can be understood that audit rules are rules pre-formulated according to actual conditions and recognized by the data owner (such as data node) and the processing node; an audit rule usually contains multiple rules, which may include but are not limited to: privacy protection rules recognized by the data owner (such as data node) and the processing node, data quality rules recognized by the data owner (such as data node) and the processing node, data format rules recognized by the data owner (such as data node) and the processing node, etc. In a feasible implementation, these rules can be stored in the same device (such as stored in the processing node), or can be distributed and stored in different devices; and when used, multiple rules can be flexibly assembled to obtain audit rules as needed, for example: audit rule one includes rule 1 and rule 2, then rules 1 and rule 2 are assembled into audit rule one; audit rule two includes rule 1 and rule 3, then rules 1 and rule 3 are assembled into audit rule two; this can improve the reusability of the rules (such as the above-mentioned rule 1).

[0083] S508: If the target data passes the audit verification, the processing node adds the target data to the aggregated data set.

[0084] As mentioned above, the target data has passed the audit verification, which means that all operations recorded in the operation ledger comply with the audit rules, the preprocessing operation is a legal operation, and the target data can participate in the aggregate calculation process; therefore, the target data can be added to the aggregate data set. Here, the aggregate data set includes multiple data that have passed the audit verification, that is, all data in the aggregate data set are data that have passed the audit verification. The aggregate data set is the basis of the aggregate calculation process and is used to provide the required data for the aggregate calculation process.

[0085] In a feasible implementation, the method may further include step S505: if the target data fails to pass the audit verification, the target data is intercepted. As mentioned above, if the target data fails to pass the audit verification, it means that an operation that violates the audit rules is recorded in the operation account book, and the preprocessing operation is deemed to be an illegal operation. If the target data is used to participate in the aggregate calculation process, it may cause security risks in the aggregate calculation process. Therefore, the target data is not suitable for participating in the aggregate calculation process. The target data can be intercepted and prohibited from being added to the aggregate data set, thereby prohibiting the target data from participating in the aggregate calculation process.

[0086] In this embodiment, the processing node can be an independent device or a combination of multiple devices; specifically, if a device has data storage capabilities, audit verification capabilities, aggregate computing capabilities, etc., then the device can be used independently as a processing node, and the target data and operation ledger sent by the data node can be sent to the device together, and the device independently executes the storage process, audit verification process and aggregate computing process of the target data. Of course, it can be understood that if one device has data storage capabilities, another device has aggregate computing capabilities, and another device has an audit verification process, then the combination of these three devices can be used as a processing node 402, then the target data returned by the data node to the processing node 402 will be sent to the device with data storage capabilities, the operation ledger returned by the data node to the processing node 402 will be sent to the device with audit verification capabilities, and the aggregate computing process can be performed by the device with aggregate computing capabilities, and the three devices work together to complete the data processing process.

[0087] In the embodiment of the present application, the operation ledger is used to perform a secure and reliable audit verification on the target data provided by the data node. This can ensure that the preprocessing operation is performed in accordance with the processing rules jointly recognized by the source data owner (such as the data node) and the processing node, and ensure that the target data can be used by the aggregation calculation process. At the same time, the privacy data in the source data will not be leaked; at the same time, it can also ensure that all data participating in the aggregation calculation process are reliable data, which is conducive to ensuring the security of the subsequent aggregation calculation process, thereby improving the security of the entire data processing process.

[0088] Figure 8 A flowchart of a data processing method provided by an exemplary embodiment of the present application is shown; the method can be Figure 4 The data node 401, the processing node 402 and the service node 403 shown interact to achieve this; the method may include the following steps S801-S812:

[0089] S801, the service node sends a data processing request to the processing node.

[0090] S802: The processing node receives a data processing request sent by the service node.

[0091] The data processing request of the business node can be initiated on a data processing trading platform. The data processing trading platform here can be any of the following platforms: website, APP (Application), some small programs or subprograms connected to APP. After the business demander (such as insurance company practitioners, bank staff or advertisers) enters the data processing trading platform through the business node, they can perform data processing request operations (such as clicking the data processing request button or selecting the data processing request option) on the service page of the data processing trading platform, and then the business node will send a data processing request to the processing node.

[0092] S803: The processing node sends a data acquisition request to the data node.

[0093] S804: The data node receives a data acquisition request sent by the processing node.

[0094] S805: The data node performs a preprocessing operation on the source data according to the data acquisition request to generate target data.

[0095] S806: The data node uses an operation ledger to record operation information of the preprocessing operation.

[0096] S807: The data node returns the target data and the operation ledger to the processing node.

[0097] S808: The processing node receives the target data and operation ledger returned by the data node.

[0098] S809: The processing node uses the operation account book to audit and verify the target data.

[0099] S810: If the target data passes the audit verification, the processing node adds the target data to an aggregated data set, wherein the aggregated data set includes a plurality of data that pass the audit verification.

[0100] S811, the processing node performs aggregation calculation on multiple data in the aggregate data set to obtain response data.

[0101] Aggregate calculation can be implemented based on aggregation algorithm, and the aggregation algorithm here may include but is not limited to: clustering algorithm, merging algorithm, maximum and minimum value algorithm, average value calculation method, etc., which are not limited in the embodiments of this application. Response data is the result of aggregate calculation, and its type depends on the actual needs of the business node. For example, in the insurance purchase scenario, the response data refers to the user's basic insurance data; in the bank lending scenario, the response data is the user's lending qualification assessment data; and in the advertising delivery scenario, the response data is the user's interest data.

[0102] S812: The processing node sends the response data to the service node.

[0103] Fig. 9A data flow diagram of a data processing method provided by an exemplary embodiment of the present application is shown. In one embodiment, each node in the data processing process can jointly maintain the same operation ledger. Specifically, the operation ledger can be sent by the data node to the processing node. In addition to recording the operation information of the pre-processing operations performed by the data node, the operation ledger can also be used to record the operation information of other operations performed by the processing node. For example, the operation ledger can also be used to record the operation information of the security audit operations performed by the processing node; in this way, the legality of the security audit process can also be verified by using the operation ledger. For another example, the operation ledger can also record the operation information of the aggregate computing operations performed by the processing node, so that the operation ledger can be used to trace and verify the legality of the aggregate computing operations, such as verifying which data is used in the aggregate computing operations, or verifying what kind of algorithm or computing model is used for the aggregate computing, etc. The operation ledger can also be sent from the processing node to the business node, so that the operation ledger can also be used to record the operation information of the business node; that is, the operation ledger can interact between the nodes involved in the data processing process (business node, data node, processing node), and is used to record the operation information of the operations performed by each node in the data processing process. In this way, the operation ledger can be used to trace and verify all operations involved in the data processing process. In addition, the same operation ledger maintained by each node is a vectorized ledger, in which vector blocks (Vectorized Block) can be used to store the operation information of each node, for example: the operation ledger contains vector block 1, vector block 2, vector block 3, and vector block 4, wherein vector block 1 is used to store the operation information of the pre-processing operation performed by the data node (including operation time, operation data flow, etc.), vector block 2 is used to store the operation information of the secure computing operation performed by the processing node, vector block 3 is used to store the operation information of the aggregate computing operation performed by the processing node, and vector block 4 is used to store the operation information of the operation performed by the business node, and each vector block is associated according to the operation time recorded by each node and presents connectivity. It can be seen that the vector ledger is a collection of vector blocks, that is, a collection of ledger data composed of continuous and mutually verifiable operation data streams of multiple nodes.

[0104] In another embodiment, each node in the data processing process can maintain its own operation account book, but the operation account books of each node are interrelated. Specifically, the data node can maintain an operation account book, which is used to record the operation information of the data node performing preprocessing operations. The processing node can also maintain an operation account book, which is used to record the operation information of the security audit operation performed by the processing node and the operation information of the aggregate computing operation. The business node can also maintain an operation account book, which can be used to record the subsequent series of processing of the response data by the business node (such as processing sent to other devices, etc.). Since the operation account books of each node serve the same data processing process, these operation account books are interrelated; in this way, the operation account books of each node and the association relationship between the operation account books of each node are also a vector account book. The operation account books of each node can verify the legitimacy of all operations in the data processing process, and the operation account books of each node can also verify each other.

[0105] A transaction usually starts with a request and ends with a response; to simplify, a transaction can be composed of a request and a response. In this embodiment, the purpose of the business node sending a data processing request is to obtain response data, so the data processing request and the response data constitute a transaction, and both the data processing request and the response data can be recorded in the secondary transaction ledger. Similarly, the purpose of the processing node sending a data acquisition request to the data node is to obtain the target data, so the data acquisition request and the target data constitute a transaction, and both the data acquisition request and the target data can be recorded in the primary transaction ledger. Among them, the primary transaction ledger and the secondary transaction ledger are used to reflect the hierarchical relationship between the transaction ledgers. This hierarchical relationship is based on the aggregate computing process. The primary transaction ledger is used to record the upstream transactions of the aggregate computing process, and the secondary transaction ledger is used to record the downstream transactions of the aggregate computing process. Specifically: Since the transaction consisting of data processing request and response data is completed after the aggregation computing process is completed, the transaction is a downstream transaction of the aggregation computing process and is therefore recorded in the secondary transaction ledger; while the transaction consisting of data acquisition request and target data is completed before the aggregation computing process begins, the transaction is an upstream transaction of the aggregation computing process and is therefore recorded in the primary transaction ledger.

[0106] In one implementation, the present application embodiment may conduct transactions in the form of an account book, such as Fig. 9As shown, specifically: the data acquisition request sent by the processing node is sent to the data node through the primary transaction ledger, that is, the processing node sends the primary transaction ledger (the data acquisition request is recorded in the primary transaction ledger) to the data node; the target data is returned by the data node to the processing node through the primary transaction ledger, that is, the data node sends the primary transaction ledger to the processing node (the data acquisition request and the target data are recorded in the primary transaction ledger at the same time), and the processing node uses the primary transaction ledger sent by the data node to update the primary transaction ledger stored locally in the processing node, that is, after the transaction is completed, the content of the primary transaction ledger record on the data node side is consistent with the content of the primary transaction ledger record on the processing node side. Similarly, the data processing request is sent by the business node to the processing node through the secondary transaction account book, that is, the business node sends the secondary transaction account book (the data processing request is recorded in the secondary transaction account book) to the processing node; the response data is sent by the processing node to the business node through the secondary transaction account book, that is, the processing node sends the secondary transaction account book (the data processing request and response data are recorded in the secondary transaction account book at the same time) to the business node, and the business node uses the secondary transaction account book sent by the processing node to update the secondary transaction account book stored locally in the business node, that is, after the transaction is completed, the content of the secondary transaction account book record on the business node side is consistent with the content of the secondary transaction account book record on the processing node side. It can be understood that the primary transaction account book and the secondary transaction account book are associated with each other, specifically: the data acquisition request in the primary transaction account book is triggered by the data processing request in the secondary transaction account book, and the response data in the secondary transaction account book is obtained by aggregating and calculating the target data in the primary transaction account book. Furthermore, both the primary transaction ledger and the secondary transaction ledger are associated with the operation ledger; specifically: the data processing request in the secondary transaction ledger triggers the generation of the target data in the operation ledger and the primary transaction ledger, and the operation ledger can be used as the basis for auditing and verifying the target data in the primary transaction ledger. Furthermore, the audit verification process performed based on the operation ledger will affect the results of the response data in the secondary transaction ledger. In other words, the various ledgers involved in the data processing process of the embodiment of the present application have both hierarchical relationships and association relationships. In a macro sense, the hierarchical relationship and association relationship between the ledgers is itself a vector ledger, so the various ledgers can also verify each other.

[0107] In one implementation, since the ledgers can verify each other, when there is missing operation information in the operation ledger, the data recorded in the primary transaction ledger and / or the data recorded in the secondary transaction ledger can be set as reference fact data for the missing operation information in the operation ledger, that is, the operation ledger is verified and supplemented by the data recorded in the primary transaction ledger and / or the data recorded in the secondary transaction ledger.

[0108] In another embodiment, if Fig. 9 As shown, the data node can use a professional preprocessing computing engine to perform preprocessing operations on the source data, and the processing node can use a professional aggregation computing engine to perform aggregation computing on multiple data in the aggregate data set. Fig. 9 Where N is a positive integer. The preprocessing computing engine and the aggregation computing engine can be provided by a third-party service organization. Before the data processing process is executed, the preprocessing computing engine and the aggregation computing engine need to be registered with the processing node in advance. The registration process here requires the engine to be registered to provide the identifier of the engine to be registered. The identifier here may include but is not limited to the URI (User Registration Interface) of the engine to be registered, the identity (identification number) or other identifiers that can be addressed to the engine. Only a successfully registered preprocessing computing engine can be used to perform preprocessing operations. Similarly, a successfully registered aggregation computing engine can be used to perform aggregation computing operations. The registration mechanism can ensure that only successfully registered computing engines can be involved in the data processing process, thereby further ensuring the security of the data processing process.

[0109] In another embodiment, the data nodes, service nodes and processing nodes can all be node devices in the blockchain network (e.g. Figure 3 Node device shown). The blockchain network here includes any of the following: private chain network, alliance chain network and public chain network. This is equivalent to executing the data processing process of the embodiment of the present application based on the blockchain network. It can be understood that the data processing process of the present embodiment can be fully executed in the blockchain network, for example: the preprocessing operation of the data node, the generation process of the operation account book, the security audit process, the aggregation calculation process and the transactions executed by the transaction account book can all be executed in the blockchain network; in this way, with the help of the fairness and openness of the blockchain, the whole process of data processing is more credible, and the security of the data processing process is further improved. Of course, the data processing process of the present embodiment can also be partially executed in the blockchain network, for example: the preprocessing operation of the data node, the generation process of the operation account book can be executed off-chain, the security audit process can be executed in the blockchain network, the aggregation calculation process can be executed off-chain, and the transactions executed by the transaction account book can be executed in the blockchain network. In this way, both the scalable characteristics of the off-chain operation and the fair and open characteristics of the blockchain can be utilized to make the data processing process more flexible, while also ensuring the security of the data processing process.

[0110] In the embodiment of the present application, firstly, the data node is triggered by the data processing request of the business node to perform preprocessing operations on the source data to obtain the target data and the operation account book, and the target data provided by the data node is audited and verified securely and reliably by using the operation account book, so that the preprocessing operation can be performed in accordance with the processing rules jointly recognized by the source data owner (such as the data node) and the processing node, and the target data can be used by the aggregated computing process, and at the same time, the privacy data in the source data will not be leaked; secondly, the target data that passes the audit verification is added to the aggregated data set, and the multiple data that pass the audit verification in the aggregated data set are aggregated and calculated to obtain the response data and return it to the business node. In this way, all data participating in the aggregated computing process are reliable data, thereby ensuring the security of the aggregated computing process; thereby improving the security of the entire data processing process. Thirdly, transactions are carried out in the form of account books during the data processing process, and multiple account books have hierarchical relationships and associations, and multiple account books can be verified to jointly maintain the reliability of the data processing process; the data processing process can also be implemented based on the blockchain network, further improving the security of the data processing process.

[0111] Fig.10 A schematic diagram of the structure of a data processing device provided by an exemplary embodiment of the present application is shown; the data processing device may be a computer program (including program code) running in the processing node 402, for example, it may be an application software in the processing node 402; the data processing device may be used to execute Figure 5 or Figure 8 See the corresponding steps in the method shown. Fig.10 , the data processing device comprises the following units:

[0112] The sending unit 1001 is used to send a data acquisition request to a data node;

[0113] A receiving unit 1002 is used to receive target data and an operation ledger returned by the data node, wherein the target data is data generated by performing a preprocessing operation on the source data, and the operation ledger is used to record operation information of the preprocessing operation;

[0114] An audit verification unit 1003 is used to audit and verify the target data using the operation account book;

[0115] The processing unit 1004 is configured to add the target data to the aggregated data set if the target data passes the audit verification.

[0116] In one implementation, the processing unit 1004 is further configured to: intercept the target data if the target data fails the audit verification.

[0117] In another embodiment, the operation book is a vector book; the operation information includes an operation code and an operation parameter; wherein the operation code includes at least one of the following: an operation instruction and an operation function; the operation parameter includes source data, an address of source data, an address of target data, target data, and data changes caused by the operation;

[0118] The operation information is encrypted into a receipt and stored in the operation ledger.

[0119] In another embodiment, when the address of the source data points to the source entity device, the address of the target data points to the target entity device, and the source entity device and the target entity device are interconnected via an interface, the operation information further includes an operation flow;

[0120] The operation flow includes: the operation time and operation content of the source entity device, the operation time and operation content of the interface operation, and the operation time and operation content of the target entity device.

[0121] In another implementation, the audit verification unit 1003 is specifically used to:

[0122] Obtaining target audit rules that match the operational ledger;

[0123] Review whether the operation information in the operation account book complies with the target audit rules;

[0124] If it is in compliance, it is confirmed that the target data has passed the audit verification; if it is not in compliance, it is confirmed that the target data has not passed the audit verification.

[0125] In another embodiment, the target audit rules are published to the blockchain network in the form of an audit smart contract; the audit verification unit 1003 is specifically used to:

[0126] Calling the audit smart contract in the blockchain network;

[0127] Run the execution program corresponding to the target audit rule declared in the audit smart contract, and review whether the operation information in the operation account book complies with the target audit rule.

[0128] In another embodiment, the data acquisition request is recorded in a primary transaction ledger; the data acquisition request is sent to the data node via the primary transaction ledger;

[0129] The target data is recorded in the primary transaction ledger; the target data is returned by the data node through the primary transaction ledger;

[0130] The primary transaction ledger is associated with the operation ledger.

[0131] In another embodiment, the aggregated data set includes multiple data that have passed the review and verification; the processing unit 1004 is further used to: perform aggregate calculations on the multiple data in the aggregated data set to obtain response data.

[0132] In another implementation, the receiving unit 1002 is further configured to: receive a data processing request sent by a service node;

[0133] The sending unit 1001 is further used to: send the response data to the service node.

[0134] In another embodiment, the data processing request is recorded in a secondary transaction ledger; the data processing request is sent by the service node through the secondary transaction ledger;

[0135] The response data is recorded in the secondary transaction ledger; the response data is sent to the business node through the secondary transaction ledger;

[0136] The secondary transaction ledger is associated with the operational ledger.

[0137] In another implementation, the processing unit 1004 is further configured to:

[0138] When there is missing operation information in the operation account book, the data recorded in the primary transaction account book is set as reference fact data of the missing operation information in the operation account book.

[0139] In another implementation, the processing unit 1004 is further configured to:

[0140] When there is missing operation information in the operation account book, the data recorded in the secondary transaction account book is set as reference fact data of the missing operation information in the operation account book.

[0141] In another embodiment, the data node and the business node are both node devices in a blockchain network; the blockchain network includes any one of the following: a private chain network, a consortium chain network, and a public chain network.

[0142] According to one embodiment of the present invention, Fig.10The various units in the data processing device shown can be individually or completely combined into one or several other units to form, or one (some) of the units can be further divided into multiple functionally smaller units to form, which can achieve the same operation without affecting the realization of the technical effects of the embodiments of the present invention. The above-mentioned units are divided based on logical functions. In practical applications, the function of one unit can also be implemented by multiple units, or the functions of multiple units can be implemented by one unit. In other embodiments of the present application, the data processing device may also include other units. In practical applications, these functions can also be implemented with the assistance of other units and can be implemented by the collaboration of multiple units. According to another embodiment of the present application, it can be executed by running on a general computing device such as a computer including processing elements and storage elements such as a central processing unit (CPU), a random access storage medium (RAM), and a read-only storage medium (ROM). Figure 5 or Figure 8 A computer program (including program code) for each step involved in the corresponding method shown in Fig.10 The data processing device shown in the embodiment of the present invention is used to implement the data processing method based on blockchain. The computer program can be recorded on a computer-readable recording medium, for example, and loaded into the above-mentioned computing device through the computer-readable recording medium and run therein.

[0143] In the embodiment of the present application, firstly, the data node is triggered by the data processing request of the business node to perform preprocessing operations on the source data to obtain the target data and the operation account book, and the target data provided by the data node is audited and verified securely and reliably by using the operation account book, so that the preprocessing operation can be performed in accordance with the processing rules jointly recognized by the source data owner (such as the data node) and the processing node, and the target data can be used by the aggregated computing process, and at the same time, the privacy data in the source data will not be leaked; secondly, the target data that passes the audit verification is added to the aggregated data set, and the multiple data that pass the audit verification in the aggregated data set are aggregated and calculated to obtain the response data and return it to the business node. In this way, all data participating in the aggregated computing process are reliable data, thereby ensuring the security of the aggregated computing process; thereby improving the security of the entire data processing process. Thirdly, transactions are carried out in the form of account books during the data processing process, and multiple account books have hierarchical relationships and associations, and multiple account books can be verified to jointly maintain the reliability of the data processing process; the data processing process can also be implemented based on the blockchain network, further improving the security of the data processing process.

[0144] Fig.11The schematic diagram of the structure of another data processing device provided by an exemplary embodiment of the present application is shown. The data processing device may be a computer program (including program code) running in the data node 401, for example, it may be an application software in the data node 401; the data processing device may be used to execute Figure 5 or Figure 8 See the corresponding steps in the method shown. Fig.11 , the data processing device comprises the following units:

[0145] A receiving unit 1101 is used to receive a data acquisition request sent by a processing node;

[0146] The preprocessing operation unit 1102 is used to perform a preprocessing operation on the source data according to the data acquisition request to generate target data.

[0147] The recording unit 1103 is used to record the operation information of the pre-processing operation in an operation account book.

[0148] The sending unit 1104 is used to return the target data and the operation ledger to the processing node, so that the processing node uses the operation ledger to audit and verify the target data, and when the target data passes the audit and verification, the target data is added to the aggregated data set.

[0149] In one embodiment, the preprocessing operation includes at least one of the following: a format conversion operation and a desensitizing operation; the format conversion operation is used to perform conversion processing on the format of the source data according to the format requirements of the aggregate calculation; the desensitizing operation is used to perform shielding processing on the privacy data in the source data.

[0150] According to one embodiment of the present invention, Fig.11 The various units in the data processing device shown can be individually or completely combined into one or several other units to form, or one (some) of the units can be further divided into multiple functionally smaller units to form, which can achieve the same operation without affecting the realization of the technical effects of the embodiments of the present invention. The above-mentioned units are divided based on logical functions. In practical applications, the function of one unit can also be implemented by multiple units, or the functions of multiple units can be implemented by one unit. In other embodiments of the present application, the data processing device may also include other units. In practical applications, these functions can also be implemented with the assistance of other units and can be implemented by the collaboration of multiple units. According to another embodiment of the present application, it can be executed by running on a general computing device such as a computer including processing elements and storage elements such as a central processing unit (CPU), a random access storage medium (RAM), and a read-only storage medium (ROM). Figure 5 or Figure 8 A computer program (including program code) for each step involved in the corresponding method shown in Fig.11 The data processing device shown in the embodiment of the present invention is used to implement the data processing method based on blockchain. The computer program can be recorded on a computer-readable recording medium, for example, and loaded into the above-mentioned computing device through the computer-readable recording medium and run therein.

[0151] In the embodiment of the present application, the operation ledger is used to perform a secure and reliable audit verification on the target data provided by the data node. This can ensure that the preprocessing operation is performed in accordance with the processing rules jointly recognized by the source data owner (such as the data node) and the processing node, and ensure that the target data can be used by the aggregation calculation process. At the same time, the privacy data in the source data will not be leaked; at the same time, it can also ensure that all data participating in the aggregation calculation process are reliable data, which is conducive to ensuring the security of the subsequent aggregation calculation process, thereby improving the security of the entire data processing process.

[0152] Fig.12 FIG. 1 shows a schematic diagram of a data processing device provided by an exemplary embodiment of the present application. Fig.12 , the data processing device at least includes a processor 1201, an input device 1202, an output device 1203 and a computer storage medium 1204. The processor 1201, the input device 1202, the output device 1203 and the computer storage medium 1204 can be connected via a bus or other means. The computer storage medium 1204 can be stored in the memory of the terminal, the computer storage medium 1204 is used to store a computer program, the computer program includes program instructions, and the processor 1201 is used to execute the program instructions stored in the computer storage medium 1104. The processor 1201 (or CPU (Central Processing Unit)) is the computing core and control core of the data processing device, which is suitable for implementing one or more instructions, specifically suitable for loading and executing one or more instructions to realize the corresponding method flow or corresponding function.

[0153] The embodiment of the present invention also provides a computer storage medium (Memory), which is a memory device in a data processing device for storing programs and data. It is understandable that the computer storage medium here can include both built-in storage media in the data processing device and, of course, extended storage media supported by the data processing device. The computer storage medium provides a storage space, which stores the operating system of the data processing device. In addition, one or more instructions suitable for being loaded and executed by the processor 1201 are also stored in the storage space, and these instructions can be one or more computer programs (including program codes). It should be noted that the computer storage medium here can be a high-speed RAM memory, or a non-volatile memory (non-volatile memory), such as at least one disk storage; optionally, it can also be at least one computer storage medium located away from the aforementioned processor.

[0154] In one embodiment, the data processing device may be Figure 4 The processing node 402 shown in the figure; the computer storage medium stores one or more first instructions; the processor 1201 loads and executes the one or more first instructions stored in the computer storage medium to implement the corresponding steps in the above data processing method embodiment; in a specific implementation, the one or more first instructions in the computer storage medium are loaded by the processor 1201 and execute the following steps:

[0155] Send a data acquisition request to the data node;

[0156] Receiving target data and an operation ledger returned by the data node, wherein the target data is data generated by performing a preprocessing operation on the source data, and the operation ledger is used to record operation information of the preprocessing operation;

[0157] Using the operation account book to audit and verify the target data;

[0158] If the target data passes the audit verification, the target data is added to the aggregated data set.

[0159] In one embodiment, the one or more first instructions in the computer storage medium are loaded by the processor 1201 and further perform the following steps:

[0160] If the target data fails the audit verification, the target data is intercepted.

[0161] In another embodiment, the operation book is a vector book; the operation information includes an operation code and an operation parameter; wherein the operation code includes at least one of the following: an operation instruction and an operation function; the operation parameter includes source data, an address of source data, an address of target data, target data, and data changes caused by the operation;

[0162] The operation information is encrypted into a receipt and stored in the operation ledger.

[0163] In another embodiment, when the address of the source data points to the source entity device, the address of the target data points to the target entity device, and the source entity device and the target entity device are interconnected via an interface, the operation information further includes an operation flow;

[0164] The operation flow includes: the operation time and operation content of the source entity device, the operation time and operation content of the interface operation, and the operation time and operation content of the target entity device.

[0165] In another embodiment, when one or more first instructions in the computer storage medium are loaded by the processor 1201 and execute the step of using the operation account book to audit and verify the target data, the following steps are specifically performed:

[0166] Obtaining target audit rules that match the operational ledger;

[0167] Review whether the operation information in the operation account book complies with the target audit rules;

[0168] If it is in compliance, it is confirmed that the target data has passed the audit verification; if it is not in compliance, it is confirmed that the target data has not passed the audit verification.

[0169] In another embodiment, the target audit rule is published to the blockchain network in the form of an audit smart contract; when one or more first instructions in the computer storage medium are loaded by the processor 1201 and execute the step of reviewing whether the operation information in the operation ledger complies with the target audit rule, the following steps are specifically performed:

[0170] Calling the audit smart contract in the blockchain network;

[0171] Run the execution program corresponding to the target audit rule declared in the audit smart contract, and review whether the operation information in the operation account book complies with the target audit rule.

[0172] In another embodiment, the data acquisition request is recorded in a primary transaction ledger; the data acquisition request is sent to the data node via the primary transaction ledger;

[0173] The target data is recorded in the primary transaction ledger; the target data is returned by the data node through the primary transaction ledger;

[0174] The primary transaction ledger is associated with the operation ledger.

[0175] In another embodiment, the aggregated data set includes a plurality of data that have passed the audit verification; one or more first instructions in the computer storage medium are loaded by the processor 1201 and further perform the following steps:

[0176] Aggregate calculation is performed on multiple data in the aggregate data set to obtain response data.

[0177] In another embodiment, before the one or more first instructions in the computer storage medium are loaded by the processor 1201 and the step of sending a data acquisition request to the data node is executed, the following steps are further executed: receiving a data processing request sent by a service node;

[0178] One or more first instructions in the computer storage medium are loaded by the processor 1201 and execute the step of performing aggregation calculation on multiple data in the aggregate data set to obtain response data, and then the following step is performed: sending the response data to the business node.

[0179] In another embodiment, the data processing request is recorded in a secondary transaction ledger; the data processing request is sent by the service node through the secondary transaction ledger;

[0180] The response data is recorded in the secondary transaction ledger; the response data is sent to the business node through the secondary transaction ledger;

[0181] The secondary transaction ledger is associated with the operational ledger.

[0182] In another embodiment, the one or more first instructions in the computer storage medium are loaded by the processor 1201 and further perform the following steps:

[0183] When there is missing operation information in the operation account book, the data recorded in the primary transaction account book is set as reference fact data of the missing operation information in the operation account book.

[0184] In another embodiment, the one or more first instructions in the computer storage medium are loaded by the processor 1201 and further perform the following steps:

[0185] When there is missing operation information in the operation account book, the data recorded in the secondary transaction account book is set as reference fact data of the missing operation information in the operation account book.

[0186] In another embodiment, the data node and the business node are both node devices in a blockchain network; the blockchain network includes any one of the following: a private chain network, a consortium chain network, and a public chain network.

[0187] In another embodiment, the data processing device may be Figure 4 The data node 401 shown in FIG. 4 is a computer storage medium storing one or more second instructions; the processor 1201 loads and executes the one or more second instructions stored in the computer storage medium to implement the corresponding steps in the above data processing method embodiment; in a specific implementation, the one or more second instructions in the computer storage medium are loaded by the processor 1201 and the following steps are executed:

[0188] Receive a data acquisition request sent by a processing node;

[0189] Performing a preprocessing operation on the source data according to the data acquisition request to generate target data;

[0190] Using an operation book to record operation information of the pre-processing operation;

[0191] The target data and the operation ledger are returned to the processing node, so that the processing node uses the operation ledger to audit and verify the target data, and when the target data passes the audit and verification, the target data is added to the aggregated data set.

[0192] In one embodiment, the preprocessing operation includes at least one of the following: a format conversion operation and a desensitizing operation; the format conversion operation is used to perform conversion processing on the format of the source data according to the format requirements of the aggregate calculation; the desensitizing operation is used to perform shielding processing on the privacy data in the source data.

[0193] In the embodiment of the present application, firstly, the data node is triggered by the data processing request of the business node to perform preprocessing operations on the source data to obtain the target data and the operation account book, and the target data provided by the data node is audited and verified securely and reliably by using the operation account book, so that the preprocessing operation can be performed in accordance with the processing rules jointly recognized by the source data owner (such as the data node) and the processing node, and the target data can be used by the aggregated computing process, and at the same time, the privacy data in the source data will not be leaked; secondly, the target data that passes the audit verification is added to the aggregated data set, and the multiple data that pass the audit verification in the aggregated data set are aggregated and calculated to obtain the response data and return it to the business node. In this way, all data participating in the aggregated computing process are reliable data, thereby ensuring the security of the aggregated computing process; thereby improving the security of the entire data processing process. Thirdly, transactions are carried out in the form of account books during the data processing process, and multiple account books have hierarchical relationships and associations, and multiple account books can be verified to jointly maintain the reliability of the data processing process; the data processing process can also be implemented based on the blockchain network, further improving the security of the data processing process.

[0194] A person skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above-mentioned methods. The storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0195] The above disclosure is only the preferred embodiment of the present invention, which certainly cannot be used to limit the scope of the present invention. Therefore, equivalent changes made according to the claims of the present invention are still within the scope of the present invention.

Claims

1. A data processing method, characterized in that: The method comprises: Receive a data processing request sent by a business node; Determining a data node related to the data processing request; Acquire target data and an operation ledger from the data node, wherein the target data is data generated by the data node performing a preprocessing operation on the source data, and the operation ledger is used to record operation information of the preprocessing operation, and the operation ledger is a vector ledger, which refers to a ledger used to record data flows based on mutual verification between parties involved in the operation; Auditing and verifying the target data using the operation account book and the target audit rules matching the operation account book, wherein the target audit rules are rules jointly recognized by the owner of the source data and the processing node; If the operation information of the preprocessing operation complies with the target audit rule, confirming that the target data has passed the audit verification, and adding the target data as data for aggregate calculation to the aggregate data set; Aggregate calculation is performed on the data used for aggregate calculation to obtain response data required by the service node.

2. The method according to claim 1, characterized in that The obtaining target data and operating ledger from the data node includes: Sending a data acquisition request to the data node; Receive the target data and operation ledger returned by the data node.

3. The method according to claim 1, characterized in that The method further comprises: If the target data fails the audit verification, the target data is intercepted.

4. The method according to claim 1, characterized in that The operation information includes an operation code and operation parameters; wherein the operation code includes at least one of the following: an operation instruction and an operation function; the operation parameters include source data, the address of source data, the address of target data, target data and data changes caused by the operation; The operation information is encrypted into a receipt and stored in the operation ledger.

5. The method according to claim 4, characterized in that When the address of the source data points to the source entity device, the address of the target data points to the target entity device, and the source entity device and the target entity device are interconnected via an interface, the operation information further includes an operation flow; The operation flow includes: the operation time and operation content of the source entity device, the operation time and operation content of the interface operation, and the operation time and operation content of the target entity device.

6. The method according to claim 3, characterized in that The auditing and verifying the target data by using the operation account book and the target audit rules matching the operation account book includes: Obtaining target audit rules that match the operational ledger; Review whether the operation information in the operation account book complies with the target audit rules; If the operation information in the operation account book does not comply with the target audit rules, it is confirmed that the target data has not passed the audit verification.

7. The method according to claim 6, characterized in that The target audit rules are published to the blockchain network in the form of audit smart contracts; The review of whether the operation information in the operation account book complies with the target audit rules includes: Calling the audit smart contract in the blockchain network; Run the execution program corresponding to the target audit rule declared in the audit smart contract, and review whether the operation information in the operation account book complies with the target audit rule.

8. The method according to claim 6, characterized in that The rules jointly recognized by the data node and the processing node include multiple rules, and the multiple rules are stored in the same device, or the multiple rules are distributed and stored in multiple devices; The target audit rule is generated by selecting at least one rule from the multiple rules and assembling them.

9. The method according to claim 2, characterized in that The data acquisition request is recorded in the primary transaction ledger; the data acquisition request is sent to the data node through the primary transaction ledger; The target data is recorded in the primary transaction ledger; The target data is returned by the data node through the primary transaction ledger; The primary transaction ledger is associated with the operation ledger.

10. The method according to claim 1, characterized in that The aggregate data set includes a plurality of data for aggregate calculation; the aggregate calculation is performed on the data for aggregate calculation to obtain the response data required by the service node, including: Aggregate calculation is performed on multiple data in the aggregate data set to obtain the response data.

11. The method according to claim 1, characterized in that The method further comprises: The response data is sent to the service node.

12. The method according to claim 1, characterized in that The data processing request is recorded in the secondary transaction ledger; the data processing request is sent by the business node through the secondary transaction ledger; The response data is recorded in the secondary transaction ledger; The response data is sent to the business node through the secondary transaction ledger; The secondary transaction ledger is associated with the operational ledger.

13. The method according to claim 9 or 12, characterized in that The method further comprises: When there is missing operation information in the operation account book, the data recorded in the primary transaction account book or the secondary transaction account book is set as reference fact data of the missing operation information in the operation account book.

14. The method according to claim 1, wherein: The data processing process involves a processing node, the data node and the service node; Each node involved in the data processing process jointly maintains the same operation ledger; or, Each node involved in the data processing process maintains its own operation ledger, and the operation ledgers of each node are interrelated.

15. The method according to claim 1, wherein: The data processing process involves a processing node, the data node and the service node; Each node in the data processing process is a node device in the blockchain network; The blockchain network includes any of the following: a private chain network, a consortium chain network, and a public chain network.

16. A data processing method, characterized in that: The method comprises: receiving a data acquisition request sent by a processing node, wherein the data acquisition request is sent by the processing node after determining a data node related to the data processing request according to the data processing request sent by the service node; Performing a preprocessing operation on the source data according to the data acquisition request to generate target data; An operation book is used to record the operation information of the preprocessing operation, and the operation book is a vector book. The vector book refers to a book used to record data flows based on mutual verification between parties involved in the operation; The target data and the operation ledger are returned to the processing node, so that the processing node uses the operation ledger and the target audit rules matching the operation ledger to audit and verify the target data, and after the target data passes the audit verification, the target data is added to the aggregate data set as data for aggregate calculation, and the data for aggregate calculation is aggregated to obtain the response data required by the business node; the target audit rules are rules jointly recognized by the owner of the source data and the processing node.

17. The method according to claim 16, characterized in that The pre-processing operation includes at least one of the following: a format conversion operation and a desensitization processing operation; The format conversion operation is used to perform conversion processing on the format of the source data according to the format requirement of the aggregate calculation; The desensitization operation is used to perform shielding processing on the privacy data in the source data.

18. A data processing device, characterized in that: include: A receiving unit, used for receiving a data processing request sent by a service node; A processing unit, configured to determine a data node associated with the data processing request; The receiving unit is further used to obtain target data and an operation ledger from the data node, wherein the target data is data generated by the data node performing a preprocessing operation on the source data, and the operation ledger is used to record operation information of the preprocessing operation, and the operation ledger is a vector ledger, which refers to a ledger used to record data flows based on mutual verification between parties involved in the operation; An audit verification unit, configured to audit and verify the target data using the operation account book and a target audit rule matching the operation account book, wherein the target audit rule is a rule jointly recognized by the owner of the source data and the processing node; The processing unit is further configured to confirm that the target data passes the audit verification if the operation information of the pre-processing operation conforms to the target audit rule, and add the target data as data for aggregate calculation to the aggregate data set; The processing unit is further used to perform aggregate calculation on the data used for aggregate calculation to obtain response data required by the service node.

19. A data processing device, characterized in that: include: A receiving unit, configured to receive a data acquisition request sent by a processing node, wherein the data acquisition request is sent by the processing node after determining a data node related to the data processing request according to the data processing request sent by the service node; A preprocessing operation unit, used to perform a preprocessing operation on the source data according to the data acquisition request to generate target data; A recording unit, configured to record operation information of the preprocessing operation in an operation book, wherein the operation book is a vector book, which refers to a book for recording data flows based on mutual verification between parties involved in the operation; A sending unit is used to return the target data and the operation ledger to the processing node, so that the processing node adopts the operation ledger and the target audit rules matching the operation ledger to audit and verify the target data, and after the target data passes the audit verification, the target data is added to the aggregate data set as data for aggregate calculation, and the data for aggregate calculation is aggregated to obtain the response data required by the business node; the target audit rules are rules jointly recognized by the owner of the source data and the processing node.

20. A data processing device, comprising an input interface and an output interface, characterized in that: Also includes: a processor adapted to implement one or more instructions; as well as, A computer storage medium, wherein the computer storage medium stores one or more first instructions, wherein the one or more first instructions are suitable for being loaded by the processor and executing the data processing method as described in any one of claims 1 to 15; or, wherein the computer storage medium stores one or more second instructions, wherein the one or more second instructions are suitable for being loaded by the processor and executing the data processing method as described in claim 16 or 17.

21. A computer storage medium, characterized in that The computer storage medium stores one or more first instructions, and the one or more first instructions are suitable for being loaded by a processor and executing the data processing method as described in any one of claims 1-15; or, the computer storage medium stores one or more second instructions, and the one or more second instructions are suitable for being loaded by a processor and executing the data processing method as described in any one of claims 16 or 17.

Images