Data operation method, device, storage medium and blockchain system

Abstract

The present disclosure relates to a data operation method, apparatus, storage medium, and blockchain system. The method is applied to a blockchain node in a blockchain network and includes: in response to an operation request initiated for a database corresponding to the blockchain node, verifying whether the initiator of the operation request has the corresponding database operation permission; in the case where the permission verification is passed, performing a corresponding database operation on the database; after the database operation is completed, encapsulating the operation record of the current database operation into a transaction, and broadcasting the transaction to other blockchain nodes in the blockchain network, so that the other blockchain nodes record the transaction on the blockchain after consensus on the transaction. The present disclosure combines the characteristics of blockchain consensus, trustworthiness, decentralization, and anti-tampering, stores all operation records of the database on the blockchain, and achieves the purpose of data traceability and non-tampering.

Description

Technical Field

[0001] The present disclosure relates to the technical field of data processing, and in particular, to a data operation method, apparatus, storage medium, and blockchain system. Background Art

[0002] By building a data middle platform, various existing information resources can be effectively integrated to form more centralized, orderly, and shared data, breaking information silos, so that information can be effectively shared between industries and departments, thereby effectively improving the collaboration efficiency and service level. In terms of data management, the data middle platform can centralize the data of its subordinate departments and uniformly provide services to upper-layer applications. However, the upper-layer applications can easily read, modify, and operate the data managed by the data middle platform, and the data security is difficult to guarantee. Summary of the Invention

[0003] The purpose of the present disclosure is to provide a data operation method, apparatus, storage medium, and blockchain system to solve the above technical problems.

[0004] To achieve the above purpose, in a first aspect, the present disclosure provides a data operation method, which is applied to a blockchain node in a blockchain network. The method includes:

[0005] In response to an operation request initiated for the database corresponding to the blockchain node, verify whether the initiator of the operation request has the corresponding database operation permission;

[0006] When the permission verification is passed, perform the corresponding database operation on the database;

[0007] After the database operation is completed, encapsulate the operation record of the current database operation into a transaction, and broadcast the transaction to other blockchain nodes in the blockchain network, so that other blockchain nodes record the transaction on the blockchain after consensus on the transaction.

[0008] Optionally, the method further includes: when the consensus on the transaction fails, roll back the database operation performed on the database.

[0009] Optionally, the performing the corresponding database operation on the database includes:

[0010] Determine a preset conflict detection rule library, where the conflict detection rule library includes a plurality of prohibition rules, and the prohibition rules are used to indicate prohibited operations corresponding to data tables in the database;

[0011] Decompose the database operation into multiple atomic-level operations on the data tables in the database;

[0012] Perform conflict detection on the multiple atomic-level operations according to the conflict detection rule library;

[0013] Generate a linear operation flow based on the atomic-level operations among the multiple atomic-level operations that do not conflict with the prohibited rules in the conflict detection rule library;

[0014] Execute each atomic-level operation in the linear operation flow on the database corresponding to the blockchain node in sequence.

[0015] Optionally, the method further includes:

[0016] Determine a preset operation transfer rule library, where the operation transfer rule library includes multiple transfer rules, and each transfer rule is used to indicate a processing strategy when the corresponding operation fails;

[0017] The step of executing each atomic-level operation in the linear operation flow on the database corresponding to the blockchain node in sequence includes:

[0018] When the execution of the atomic-level operation in the linear operation flow fails, determine the target transfer rule corresponding to the atomic-level operation in the operation transfer rule library;

[0019] Respond to the processing strategy in the target transfer rule.

[0020] Optionally, the method further includes:

[0021] Determine a data aggregation plan, where the data aggregation plan is used to indicate the trigger condition for the data aggregation operation, at least one source data table corresponding to the data aggregation operation, and the target relational schema corresponding to the data aggregation operation;

[0022] When it is determined that the trigger condition is satisfied, extract the operation records corresponding to the source data table from the blockchain, and restore the corresponding source data table according to the operation records;

[0023] Aggregate the restored at least one source data table under the target relational schema to obtain an aggregated table;

[0024] Broadcast the aggregated table to other blockchain nodes in the blockchain network, so that other blockchain nodes record the aggregated table on the blockchain after reaching a consensus on the aggregated table.

[0025] Optionally, the step of aggregating the restored at least one source data table under the target relational schema to obtain an aggregated table includes:

[0026] Construct an aggregated table with the target relational schema, where the target relational schema includes a table structure and table fields, and the table fields include a primary key and non-primary key attributes;

[0027] Establish a mapping relationship between the non-primary key attributes of each source data table and the non-primary key attributes of the aggregation table;

[0028] Determine a primary key mediation table, which is used to map the primary keys of each source data table to the primary key of the primary key mediation table respectively;

[0029] For each source data table, perform a natural join on the source data table and the primary key mediation table to obtain a corresponding intermediate data table;

[0030] According to the set first projection condition and the intermediate data table, perform a corresponding projection operation on the source data table, and according to the mapping relationship between the source data table and the aggregation table, map the result of the projection operation to the table entries of the aggregation table to obtain the target data;

[0031] Perform a union operation on the target data corresponding to each source data table, and write the result of the union operation into the aggregation table.

[0032] Optionally, the method further includes:

[0033] Construct a historical table having the same relational schema as the aggregation table;

[0034] According to the set second projection condition, perform a corresponding projection operation on the aggregation table to obtain the data to be migrated;

[0035] Migrate the data to be migrated to the historical table;

[0036] Broadcast the historical table to other blockchain nodes in the blockchain network, so that other blockchain nodes record the historical table on the blockchain after consensus on the historical table.

[0037] In a second aspect, the present disclosure provides a data operation device configured in a blockchain node in a blockchain network. The device includes:

[0038] A permission verification module, configured to verify whether the initiator of the operation request has the corresponding database operation permission in response to an operation request initiated for the database corresponding to the blockchain node;

[0039] A database operation module, configured to perform a corresponding database operation on the database when the permission verification is passed;

[0040] A record-on-chain module, configured to encapsulate the operation record of the current database operation into a transaction after the database operation is completed, and broadcast the transaction to other blockchain nodes in the blockchain network, so that other blockchain nodes record the transaction on the blockchain after consensus on the transaction.

[0041] In a third aspect, the present disclosure provides a computer-readable storage medium having a computer program stored thereon, and when the program is executed by a processor, the method described in the first aspect is implemented.

[0042] In a fourth aspect, the present disclosure provides a blockchain system, including a blockchain network composed of multiple blockchain nodes, a data middle platform service module is provided on each of the blockchain nodes, a smart contract is deployed in the data middle platform service module, and the blockchain nodes are used to execute the smart contract in the data middle platform service module to implement the method described in the first aspect.

[0043] Through the above technical solution, since the permission verification of the corresponding database operation permissions of the initiator of the operation request is performed, the control of data access is realized, and all operations performed on the database are recorded on the blockchain, so that the upper-layer application cannot easily tamper with the data in the database, and the data in the data middle platform is traceable and cannot be tampered with. Moreover, the traditional database has the disadvantage of poor disaster recovery ability. In the above solution, since all operations on the database are stored on the blockchain, all blockchain nodes can query the operation records of all data tables in each database from the blockchain, and all the incoming data tables can be obtained according to the operation records, so that each blockchain node can obtain the globally consistent incoming data tables, thereby improving the disaster tolerance ability of the database, and this method can effectively be compatible with heterogeneous databases.

[0044] In addition, considering that the efficiency of the blockchain network to consensus on transactions is relatively low, the method adopted in this solution is to first store in the database and then perform consensus. Priority is given to ensuring that the blockchain nodes have the performance at the database level, so that the database operations can be completed in a timely manner. On this basis, the operation records are then consensus-linked to ensure that other external business systems interacting with the database will not be affected.

[0045] Other features and advantages of the present disclosure will be described in detail in the subsequent specific implementation section. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] The drawings are used to provide a further understanding of the present disclosure, and constitute a part of the specification, and are used to explain the present disclosure together with the following specific implementation, but do not constitute a limitation to the present disclosure. In the drawings:

[0047] Figure 1 The flowchart of the data operation method provided by an exemplary embodiment is shown;

[0048] Figure 2 It shows Figure 1 The flowchart of the specific implementation of performing the corresponding database operation on the database in step S120 in

[0049] Figure 3 The flowchart of a data operation method provided by an exemplary embodiment is shown;

[0050] Figure 4 shows Figure 3 the flowchart of the specific implementation manner of obtaining the aggregation table in step S330;

[0051] Figure 5 The flowchart of a data operation method provided by an exemplary embodiment is shown;

[0052] Figure 6 The schematic diagram of implementing the data middle - platform service based on the blockchain system is shown;

[0053] Figure 7 The block diagram of a data operation device provided by an exemplary embodiment is shown. Specific implementation manner

[0054] The following details the specific implementation manner of the present disclosure in conjunction with the accompanying drawings. It should be understood that the specific implementation manner described herein is only for explaining and understanding the present disclosure, and is not used to limit the present disclosure.

[0055] It should be noted that all actions of obtaining signals, information, or data in the present disclosure are carried out on the premise of complying with the corresponding data protection regulations and policies of the country where the location is located, and with the authorization given by the owner of the corresponding device.

[0056] The embodiments of the present disclosure provide a data operation method. By introducing the characteristics of blockchain consensus, trustworthiness, decentralization, and anti - tampering into the data middle - platform, all operation records of the database are stored on the blockchain, achieving the purpose of data traceability and non - tampering. To achieve the above - mentioned purpose, first, a blockchain network is constructed. The blockchain network includes multiple blockchain nodes, and the following data operation method provided by the embodiments of the present disclosure is implemented according to the constructed blockchain network.

[0057] Figure 1 The flowchart of a data operation method provided by an exemplary embodiment is shown. This method is applied to a blockchain node in the blockchain network, as Figure 1 shown, and the method includes:

[0058] S110, in response to an operation request initiated for the database corresponding to the blockchain node, verify whether the initiator of the operation request has the corresponding database operation permission.

[0059] Among them, each blockchain node in the blockchain network has its own corresponding database for storing data. The databases used by different blockchain nodes may be the same or different. For example, the database used by a certain blockchain node is MySQL, and the database used by another blockchain node is a document database. In the above steps, in response to an operation request initiated for the database local to the blockchain node, first verify whether the initiator of the operation request has the corresponding database operation permission.

[0060] In specific implementation, a smart contract is deployed on the blockchain node, and the operation of verifying whether the initiator of the operation request has the corresponding database operation permission is completed through the smart contract.

[0061] Optionally, the corresponding database operation permission includes: whether the initiator has the service permission to perform the corresponding operation on the database, and whether the initiator has the data permission to operate on the data table to be operated. When the initiator has both the corresponding service permission and data permission, it is determined that the initiator has the corresponding database operation permission for the database.

[0062] Exemplarily, the operation request is used to request to write a piece of data into data table A in the database. In the above steps, it is necessary to verify whether the initiator has the service permission to perform a write operation on the database, and whether it has the data permission to operate on data table A.

[0063] In specific implementation, a service permission verification smart contract and a data permission verification smart contract are deployed on the blockchain node. The service permission verification smart contract is used to verify whether the initiator has the service permission to perform the corresponding operation on the database, and the data permission verification smart contract is used to verify whether the initiator has the data permission to operate on the data table to be operated.

[0064] In the present disclosure, the method of permission verification needs to be combined with the blockchain. A permission verification method applicable to the blockchain network can be used. For example, a permission verification method based on CA (Certificate Authority), and / or a permission verification method based on distributed digital identity.

[0065] In the permission verification method based on CA, the public key certificate of the authorizing party is registered on the blockchain, and the received operation request will be attached with the private key signature result. The corresponding permission verification smart contract performs permission verification according to the private key signature result.

[0066] Based on the permission verification method of distributed digital identity, the digital identity information of the authorized party is saved on the blockchain. The initiator initiates an operation request to the database in the form of a distributed digital identity certificate, which includes the request content, the identity information of the initiator, the timestamp, the signature, and the signature verification algorithm information. After receiving the operation request, the corresponding permission verification smart contract finds the signature verification public key to be used from the distributed digital identity information on the chain according to the identity information of the initiator and the signature verification algorithm information, verifies the signature, and performs permission verification in combination with the timestamp.

[0067] The service permission verification smart contract can use any of the above permission verification methods to verify the service permissions of the initiator. Correspondingly, the data permission verification smart contract can also use any of the above permission verification methods to verify the service permissions of the initiator.

[0068] S120: If the authority check is passed, perform corresponding database operations on the database.

[0069] When it is verified that the initiator of the operation request has the corresponding database operation authority, the database operation requested by the operation request is executed on the database corresponding to the blockchain node.

[0070] S130, after the database operation is completed, the operation record of this database operation is encapsulated into a transaction, and the transaction is broadcast to other blockchain nodes in the blockchain network, so that other blockchain nodes can record the transaction on the blockchain after reaching consensus on the transaction.

[0071] After the above database operation is completed, the operation record of this database operation is encapsulated into a transaction, and the transaction is broadcast to other blockchain nodes in the blockchain network. All blockchain nodes in the blockchain network reach a consensus on the transaction, and after consensus, the transaction is recorded on the blockchain, thus completing the chaining of the operation record. After the operation record is chained, all blockchain nodes can query all operation records of each blockchain node on their respective databases from the blockchain.

[0072] For example, suppose that the database operation of blockchain node 1 is to write a piece of data "name is Zhang San, ID number is xxxx" to data table A in the local database. After the database operation is completed, a new piece of data "Zhang San, xxxx" is added to data table A in the database. At the same time, the operation record of this database operation will be recorded on the blockchain, and other blockchain nodes can read the operation record of "writing a piece of data in data table A, which is 'name is Zhang San, ID number is xxxx'" from the blockchain.

[0073] It should be noted that in the event that the transaction consensus fails, the above database operations performed on the database need to be rolled back to ensure the consistency of the operations on the database with the operation records saved on the blockchain.

[0074] It can be understood that in the above scheme, since the corresponding database operation permissions of the initiator of the operation request are verified, data access control is achieved, and all operations performed on the database are recorded on the blockchain, so that the upper-level application cannot easily tamper with the data in the database, and the data in the data center is traceable and cannot be tampered with. In addition, traditional databases have the disadvantage of poor disaster recovery capabilities. In the above scheme, since all operations on the database are stored on the blockchain, all blockchain nodes can query the operation records of all data tables in each database from the blockchain, and all stored data tables can be obtained according to the operation records. In this way, each blockchain node can obtain globally consistent stored data tables, thereby improving the disaster recovery capabilities of the database, and this method can effectively be compatible with heterogeneous databases.

[0075] In addition, considering that the blockchain network is less efficient in reaching consensus on transactions, this solution adopts the approach of first storing data in the database and then reaching consensus, giving priority to ensuring that blockchain nodes have database-level performance so that database operations can be completed in a timely manner. On this basis, the operation records are then put on the chain for consensus, thereby ensuring that other external business systems that interact with the database are not affected.

[0076] Optionally, Figure 2 FIG. 1 is a flowchart showing a specific implementation of performing corresponding database operations on the database in the above step S120. Figure 2 As shown, step S120 includes:

[0077] S210, determining a preset conflict detection rule base, wherein the conflict detection rule base includes a plurality of prohibition rules, each prohibition rule indicating a prohibited operation corresponding to a data table in a database.

[0078] A conflict detection rule base is established in advance, which includes multiple prohibition rules. Each prohibition rule indicates a prohibited operation corresponding to a data table in the database. The prohibited operation represents an atomic-level operation on the data table. The atomic-level operations include: table creation, table connection, projection operation, union operation, data writing, data deletion, data updating, and data reading.

[0079] Exemplarily, a conflict detection rule base P={p1, p2, ..., pn} is defined, where pi represents a prohibition rule. For example, the prohibition rule corresponding to p1 is "data table A is prohibited from being written".

[0080] S220. Decompose the database operation into multiple atomic operations on the data tables in the database.

[0081] Exemplarily, the corresponding database operation requested by the operation request is: writing one piece of data in data table B to data table A. Among them, decomposing the database operation into multiple atomic operations on the data tables in the database, the obtained multiple atomic operations include: atomic operation 1, reading one piece of data in data table B; atomic operation 2, writing the read data in data table B to data table A.

[0082] S230. Perform conflict detection on the multiple atomic operations according to the conflict detection rule library.

[0083] S240. Generate a linear operation flow according to the atomic operations among the multiple atomic operations that do not conflict with the prohibited rules in the conflict detection rule library.

[0084] Among them, when performing conflict detection on the decomposed multiple atomic operations according to the conflict detection rule library, when a certain decomposed atomic operation belongs to the operation prohibited by a certain prohibited rule in the conflict detection rule library, it is determined that the atomic operation conflicts with the prohibited rule in the conflict detection rule library. After completing the conflict detection of each atomic operation, filter out the atomic operations among the multiple atomic operations that conflict with the prohibited rules in the conflict detection rule library, and combine the remaining atomic operations that do not conflict with the prohibited rules in the conflict detection rule library into a linear operation flow.

[0085] Exemplarily, the prohibited rule p1 in the conflict detection rule library is "data table A is prohibited from being written", and a certain decomposed atomic operation includes "writing the read data in data table B to data table A", then it is determined that the atomic operation conflicts with the prohibited rule p1 in the conflict detection rule library.

[0086] S250. Sequentially execute each atomic operation in the linear operation flow on the database.

[0087] After obtaining the linear operation flow, sequentially execute each atomic operation in the linear operation flow on the database corresponding to the blockchain node.

[0088] Optionally, considering that in the above steps, there may be a situation where an operation fails during the execution of each atomic operation, an operation transfer rule library is established in advance. The operation transfer rule library includes multiple transfer rules, and each transfer rule respectively indicates the processing strategy when the corresponding operation fails.

[0089] Exemplarily, the transfer rules in the operation transfer rule library include but are not limited to: (1) Table establishment failure → Interrupt operation; (2) Table connection failure → Interrupt operation; (3) Projection operation failure → Save the operation record as a pending state, skip this step of operation and proceed to the next step in the linear operation flow; (4) Union operation failure → Save the operation record as a pending state, skip this step of operation and proceed to the next step in the linear operation flow; (5) Data write failure → Roll back this operation and proceed to the next step in the linear operation flow; (6) Data deletion failure → Roll back this operation; (7) Data update failure → Roll back this operation; (8) Data read failure → Skip this step of operation and proceed to the next step in the linear operation flow. Of course, the above-listed transfer rules are only examples and do not mean that they must be set according to such rules in actual situations.

[0090] During the process of sequentially executing each atomic-level operation in the linear operation flow on the database corresponding to the blockchain node, when the atomic-level operation in the linear operation flow is executed successfully, continue to execute the next atomic-level operation in the linear operation flow. When the atomic-level operation in the linear operation flow fails, according to the above operation transfer rule library, determine the target transfer rule in the operation transfer rule library corresponding to the failed atomic-level operation, and then respond to the processing strategy in the target transfer rule. For example, when executing the data read operation on data table B in the linear operation flow, if the data read operation fails, respond to its corresponding processing strategy, skip this step of operation and proceed to the next step in the linear operation flow.

[0091] After the linear operation flow is executed, the blockchain node encapsulates the operation record of this database operation into a transaction and stores it on the blockchain.

[0092] Optionally, after receiving an operation request, the blockchain node first checks whether the local data table is consistent with the data table recorded on the blockchain. If not, it is necessary to restore the data table through the corresponding block on the blockchain. If the corresponding block is missing locally on the blockchain node, synchronize the block from other blockchain nodes in the blockchain network.

[0093] Furthermore, considering that the owners of different blockchain nodes may have significant business differences, that is, different blockchain nodes use different databases. For example, one blockchain node uses a MySQL database, and another blockchain node uses a document-based database, which will lead to multi-source heterogeneity of data, making it difficult to achieve unified operation and management of data and causing difficulties in the development of upper-layer applications.

[0094] To solve the problem of multi-source heterogeneity of data, Figure 3 shows a flowchart of a data operation method provided by an exemplary embodiment, asFigure 3 As shown, the method includes:

[0095] S310. Determine a data aggregation plan, which is used to indicate the trigger condition for the data aggregation operation, at least one source data table corresponding to the data aggregation operation, and the target relational schema corresponding to the data aggregation operation.

[0096] Pre - establish a data aggregation plan, which is used to indicate the trigger condition for the data aggregation operation, at least one source data table corresponding to the data aggregation operation, and the target relational schema corresponding to the data aggregation operation. Among them, the trigger condition can be an event - type trigger condition or a time - type trigger condition.

[0097] For the time - type trigger condition, for example, a certain data aggregation plan is: every week (trigger condition), perform a data aggregation operation on source data table A and source data table B.

[0098] For the event - type trigger condition, for example, a certain data aggregation plan is: whenever source data table A is updated or every time 100,000 new data are added to source data table A (trigger condition), perform a data aggregation operation on source data table A and source data table B.

[0099] S320. When it is determined that the trigger condition is satisfied, extract the operation records of the corresponding source data table from the blockchain, and restore the corresponding source data table according to the extracted operation records.

[0100] The following takes the at least one source data table in the data aggregation plan being source data table A and source data table B respectively as an example for illustration. Since the blockchain stores the operation records of all data tables, including several operation records of source data table A, several operation records of source data table B, several operation records of source data table C, etc., when it is determined that the trigger condition in the data aggregation plan is satisfied, extract the operation records of source data table A and source data table B from the blockchain, restore the corresponding source data table A according to the operation records of source data table A, and restore the corresponding source data table B according to the operation records of source data table B.

[0101] S330. Aggregate the restored source data tables under the target relational schema to obtain an aggregated table.

[0102] S340. Broadcast the aggregated table to other blockchain nodes in the blockchain network so that other blockchain nodes record the aggregated table on the blockchain after consensus.

[0103] It can be understood that before this solution is adopted, the data tables are stored in different databases respectively. Due to the differences in databases, these data tables cannot be operated on uniformly. After adopting this solution, according to the data aggregation plan, the relevant source data tables are aggregated into a unified target relational schema to obtain an aggregated table. Therefore, all blockchain nodes can operate on the aggregated table by using a unified operation instruction, achieving the unity of operation.

[0104] Optionally, Figure 4 The flowchart showing the specific implementation manner of obtaining the aggregated table in step S330 provided by an exemplary embodiment is as follows Figure 4 shown, step S330 includes:

[0105] S410, construct an aggregated table with a target relational schema, where the target relational schema includes a table structure and table fields, and the table fields include a primary key and non-primary key attributes.

[0106] Construct an aggregated table with a target relational schema. At this time, the constructed aggregated table is an initial aggregated table. The target relational schema includes a table structure and table fields, and the table fields include a primary key and non-primary key attributes. Each attribute field is defined according to the business services required by the data middle platform.

[0107] Optionally, the table fields of the aggregated table include a primary key, non-primary key attributes, and a time field. The time field can be used to represent the last update time or other time information of the corresponding data. Similarly, the table fields of each source data table also include a primary key and non-primary key attributes. Optionally, a time field can also be included.

[0108] Denote the aggregated table as current, the source data table A as sourceA, and the source data table B as sourceB, then there is:

[0109] sourceA = {akey, z1, z2, z3, …, time};

[0110] sourceB = {bkey, g1, g2, g3, …, time};

[0111] current = {key, att1, att2, att3 … time};

[0112] where, akey is the primary key of the source data table A, z1, z2, z3 are the non-primary key attributes of the source data table A, bkey is the primary key of the source data table B, g1, g2, g3 are the non-primary key attributes of the source data table B, key is the primary key of the aggregated table, att1, att2, att3 are the non-primary key attributes of the aggregated table, and time is the time field.

[0113] S420. Establish the mapping relationship between the non-primary key attributes of each source data table and the non-primary key attributes of the aggregation table.

[0114] It can be understood that it is necessary to establish the mapping relationship between the non-primary key attributes of each source data table and the non-primary key attributes of the aggregation table in order to determine which entry in the source data table should be filled into which entry in the aggregation table. According to the above example, the following mapping relationship can be established:

[0115] att1 ← (z1, g1);

[0116] att2 ← (z2, g2);

[0117] att3 ← (z3, g3).

[0118] S430. Determine the primary key mediation table, which is used to map the primary key of each source data table to the primary key of the primary key mediation table respectively.

[0119] The different primary keys extracted from source data table A and source data table B will affect the unification of the relational schema, and data mediation is the process of unifying the form of the primary keys of the data. Therefore, it is necessary to establish a primary key mediation table for each source data table. By defining the primary key mediation table, the primary keys of the original relational schema of each source data table are mapped to the primary keys of the target relational schema of the aggregation table.

[0120] Exemplarily, according to the above example, the primary key mediation table is:

[0121] conc = {ckey, akey, bkey};

[0122] Among them, ckey is the primary key of the primary key mediation table.

[0123] S440. For each source data table, perform a natural join of the source data table and the primary key mediation table to obtain the corresponding intermediate data table.

[0124] Exemplarily, according to the above example, for source data table A, perform a natural join of source data table A and the primary key mediation table to obtain the intermediate data table corresponding to source data table A, denoted as TempA; for source data table B, perform a natural join of source data table B and the primary key mediation table to obtain the intermediate data table corresponding to source data table B, denoted as TempB.

[0125] Among them, TempA ← sourceA ∞ conc;

[0126] Among them, TempB ← sourceB ∞ conc;

[0127] Among them, ∞ is the natural join operator.

[0128] S450, perform the corresponding projection operation on the source data table according to the set first projection condition and the corresponding intermediate data table.

[0129] S460, map the result of the projection operation to the table entries of the aggregation table according to the mapping relationship between the source data table and the aggregation table to obtain the target data.

[0130] For each source data table, perform a projection operation on the source data table according to the set first projection condition and the intermediate data table corresponding to the source data table, and map the result of the projection operation to the table entries of the aggregation table according to the mapping relationship between the source data table and the aggregation table to obtain the target data corresponding to the source data table.

[0131] Exemplarily, according to the above example, denote the target data corresponding to the source data table A as dataA, and the target data corresponding to the source data table B as dataB, then there is:

[0132] dataA ← π(sourceA)(TempA);

[0133] dataB ← π(sourceB)(TempB);

[0134] where π is the projection operator.

[0135] As an example, the projection operation represented by dataA ← π(sourceA)(TempA) is:

[0136] Determine the fields in TempA that meet the conditions according to the first projection condition, and determine the data projection in sourceA for the fields in TempA that meet the conditions to obtain the required data.

[0137] S470, perform a union operation on the target data corresponding to each source data table, and write the result of the union operation into the aggregation table.

[0138] After the above steps, the obtained target data dataA and dataB are data in the same relational schema. Therefore, only need to perform a simple union operation on the target data dataA and dataB, and then write the result of the union operation into the aggregation table. The obtained aggregation table is:

[0139] current ← dataA ∪ dataB

[0140] Or, the obtained aggregation table is:

[0141] current ← current ∪ dataA ∪ dataB

[0142] In an alternative embodiment, in the above steps S410 to S470, when only performing a data aggregation operation for a new target relationship pattern, a new aggregation table is newly constructed. If multiple data aggregation operations are initiated for the same target relationship pattern, subsequent data aggregation operations do not need to construct a new aggregation table, but directly update the aggregation result to the originally constructed aggregation table.

[0143] As the data aggregation operation continues, the data in the aggregation table will become more and more. Therefore, it is necessary to timely migrate the old data with low value in the aggregation table to other tables for data archiving.

[0144] Optionally, Figure 5 shows a flowchart of a data operation method provided by an exemplary embodiment, as Figure 5 shown, the method includes:

[0145] S510, construct a historical table having the same relationship pattern as the aggregation table.

[0146] Among them, the same relationship pattern means that the historical table and the aggregation table have the same table structure and the same table fields. Step S510 can be executed simultaneously with step S410, that is, an aggregation table and a historical table having the same target relationship pattern are constructed simultaneously.

[0147] S520, perform a corresponding projection operation on the aggregation table according to the set second projection condition to obtain the data to be migrated.

[0148] Among them, perform a corresponding projection operation on the aggregation table according to the set second projection condition to obtain the data to be migrated dataOld.

[0149] Among them, dataOld←π(current)(time).

[0150] It can be understood that which data is defined as the old data to be migrated is defined by the second projection condition. For example, data one year ago is old data, or data one month ago is old data.

[0151] As an example, the projection operation represented by dataOld←π(current)(time) is:

[0152] According to the second projection condition and the time field time in current, determine the data in current whose time information meets the second projection condition to obtain the required data.

[0153] S530, migrate the data to be migrated to the historical table.

[0154] After determining the data to be migrated dataOld, migrate the data to be migrated dataOld from the aggregation table current to the historical table history, then there is:

[0155] current ← current – dataOld;

[0156] history ← dataOld;

[0157] Or, then there is:

[0158] history ← history ∪ dataOld.

[0159] S540, broadcast the historical table to other blockchain nodes in the blockchain network so that other blockchain nodes record the historical table on the blockchain after consensus on the historical table.

[0160] Through the above steps, the old data can be migrated to the historical table history in a timely manner, playing the role of data archiving.

[0161] In an exemplary embodiment of the present disclosure, a blockchain system is further provided. The blockchain system includes a blockchain network composed of multiple blockchain nodes. A data middle platform service module is provided on each blockchain node, and a smart contract is deployed in the data middle platform service module. Each blockchain node is used to execute the smart contract in the data middle platform service module to implement each step of the data operation method provided in the foregoing method embodiment.

[0162] It can be understood that the services of the data middle platform are implemented by smart contracts, that is, through the smart contracts deployed on each blockchain node, the services of the data middle platform can be implemented on the blockchain system.

[0163] Figure 6 Shows a schematic diagram of implementing the data middle platform service based on the above blockchain system. As Figure 6 shown, the blockchain system includes a blockchain network composed of multiple blockchain nodes. Each blockchain node has its own corresponding database for storing data. The databases used between different blockchain nodes may be the same or different. For example, the database used by a certain blockchain node is MySQL, and the database used by another blockchain node is a document-based database.

[0164] The services of the data middle platform include but are not limited to: data authorization management, service authorization management, data development plan, and data aggregation plan.

[0165] In specific implementation, a data middle platform service module is provided on each blockchain node, and corresponding data permission verification smart contracts, service permission verification smart contracts, data development smart contracts, and data aggregation smart contracts are deployed in the data middle platform service module.

[0166] The service permission verification smart contract is used to verify whether the initiator of the operation request has the service permission to perform the corresponding operation on the database after receiving the operation request. The data permission verification smart contract is used to verify whether the initiator has the data permission to operate on the data table to be operated after receiving the operation request. The data development smart contract is used to manage the conflict detection rule library and the operation transfer rule library. After determining that the permission verification of the service permission verification smart contract and the data permission verification smart contract passes, corresponding database operations are performed on the database according to the conflict detection rule library and the operation transfer rule library. The data aggregation smart contract is used to manage the data aggregation plan, and after determining that the trigger condition in the data aggregation plan is met, corresponding data aggregation operations are performed on the relevant source data tables in the data aggregation plan.

[0167] It should be noted that in the above blockchain system, blockchain technology is used to achieve data aggregation, the blockchain is combined with the database, operations on the database are stored on the chain, and smart contracts are used to implement the middle platform services. Data access control and scheduled tasks are performed on the chain, and the characteristics of blockchain consensus, trustworthiness, decentralization, and immutability are introduced into the data middle platform, effectively improving data security.

[0168] Regarding the blockchain system in the above embodiments, the detailed steps of the data operation method implemented by each blockchain node executing the smart contract have been described in detail in the embodiments related to the method, and will not be elaborated here.

[0169] In an exemplary embodiment of the present disclosure, a computer-readable storage medium is further provided, on which a computer program is stored, and when the program is executed by a processor, the steps of the data operation method provided in the foregoing embodiments are implemented.

[0170] In an exemplary embodiment of the present disclosure, a data operation device is further provided, and the device is configured in a blockchain node in a blockchain network. Figure 7 The block diagram of the data operation device is shown, as Figure 7 shown, the device 600 includes:

[0171] A permission verification module 610, configured to verify whether the initiator of the operation request has the corresponding database operation permission in response to an operation request initiated for the database corresponding to the blockchain node;

[0172] A database operation module 620, configured to perform corresponding database operations on the database when the permission verification passes;

[0173] A record uploading module 630, configured to, after the database operation is completed, encapsulate the operation record of the current database operation into a transaction, and broadcast the transaction to other blockchain nodes in the blockchain network, so that the other blockchain nodes record the transaction on the blockchain after reaching a consensus on the transaction.

[0174] Optionally, the device 600 further includes: a consensus failure handling module, configured to roll back the database operation performed on the database in the case of a consensus failure of the transaction.

[0175] Optionally, the database operation module 620 includes:

[0176] A conflict rule determination module, configured to determine a preset conflict detection rule library, where the conflict detection rule library includes a plurality of prohibition rules, and the prohibition rules are used to indicate prohibited operations corresponding to data tables in the database;

[0177] An operation decomposition module, configured to decompose the database operation into a plurality of atomic-level operations on data tables in the database;

[0178] A conflict detection module, configured to perform conflict detection on the plurality of atomic-level operations according to the conflict detection rule library;

[0179] An operation flow generation module, configured to generate a linear operation flow according to the atomic-level operations that do not conflict with the prohibition rules in the conflict detection rule library among the plurality of atomic-level operations;

[0180] An operation execution module, configured to sequentially execute each atomic-level operation in the linear operation flow on the database corresponding to the blockchain node.

[0181] Optionally, the device 600 further includes: a transfer rule determination module, configured to determine a preset operation transfer rule library, where the operation transfer rule library includes a plurality of transfer rules, and each transfer rule is used to indicate a processing strategy when the corresponding operation fails.

[0182] Wherein, the operation execution module is configured to, in the case of a failure in executing the atomic-level operation in the linear operation flow, determine a target transfer rule in the operation transfer rule library corresponding to the atomic-level operation; and respond to the processing strategy in the target transfer rule.

[0183] Optionally, the device 600 further includes:

[0184] An aggregation plan determination module, configured to determine a data aggregation plan, where the data aggregation plan is used to indicate a trigger condition for a data aggregation operation, at least one source data table corresponding to the data aggregation operation, and a target relational schema corresponding to the data aggregation operation;

[0185] A data table restoration module, configured to, when it is determined that the trigger condition is satisfied, extract operation records corresponding to the source data table from the blockchain, and restore the corresponding source data table according to the operation records;

[0186] A data aggregation module, configured to aggregate the restored at least one source data table under the target relational schema to obtain an aggregated table;

[0187] An aggregated table uploading module, configured to broadcast the aggregated table to other blockchain nodes in the blockchain network, so that other blockchain nodes record the aggregated table on the blockchain after consensus on the aggregated table.

[0188] Optionally, the data aggregation module includes:

[0189] An aggregated table construction module, configured to construct an aggregated table with the target relational schema, where the target relational schema includes a table structure and table fields, and the table fields include a primary key and non-primary key attributes;

[0190] An attribute mapping module, configured to establish a mapping relationship between non-primary key attributes of each source data table and non-primary key attributes of the aggregated table;

[0191] A primary key mediation module, configured to determine a primary key mediation table, where the primary key mediation table is used to map the primary keys of each source data table to the primary key of the primary key mediation table respectively;

[0192] A data mediation module, configured to perform a natural join on each source data table and the primary key mediation table to obtain a corresponding intermediate data table;

[0193] A first projection module, configured to perform a corresponding projection operation on the source data table according to set first projection conditions and the intermediate data table, and map the result of the projection operation to table entries of the aggregated table according to the mapping relationship between the source data table and the aggregated table to obtain target data;

[0194] An aggregated table obtaining module, configured to perform a union operation on the target data corresponding to each source data table, and write the result of the union operation into the aggregated table.

[0195] Optionally, the apparatus 600 further includes:

[0196] A historical table construction module, configured to construct a historical table having the same relational schema as the aggregated table;

[0197] A second projection module, configured to perform corresponding projection operations on the aggregation table according to set second projection conditions to obtain data to be migrated;

[0198] A data migration module, configured to migrate the data to be migrated to the historical table;

[0199] A historical table uploading module, configured to broadcast the historical table to other blockchain nodes in the blockchain network, so that after other blockchain nodes consensus on the historical table, record the historical table on the blockchain.

[0200] Regarding the device in the above embodiments, the specific manners in which each module performs operations have been described in detail in the embodiments related to the method, and will not be elaborated herein.

[0201] The preferred embodiments of the present disclosure have been described in detail above in conjunction with the accompanying drawings. However, the present disclosure is not limited to the specific details in the above embodiments. Within the scope of the technical concept of the present disclosure, various simple modifications can be made to the technical solutions of the present disclosure, and these simple modifications all fall within the protection scope of the present disclosure.

[0202] In addition, it should be noted that, in the above specific embodiments, the various specific technical features described can be combined in any suitable manner without conflict. To avoid unnecessary repetition, the present disclosure will not separately describe various possible combination manners.

[0203] Furthermore, any combination can be made between various different embodiments of the present disclosure as long as it does not violate the idea of the present disclosure, and it should also be regarded as the content disclosed by the present disclosure.

Claims

1. A data operation method, characterized in that, A blockchain node applied to a blockchain network, the method comprising: In response to an operation request initiated for the database corresponding to the blockchain node, verifying whether the initiator of the operation request has the corresponding database operation permission; When the permission verification is passed, performing the corresponding database operation on the database; After the database operation is completed, encapsulating the operation record of the current database operation into a transaction, and broadcasting the transaction to other blockchain nodes in the blockchain network, so that other blockchain nodes record the transaction on the blockchain after consensus on the transaction; Determining a data aggregation plan, the data aggregation plan being used to indicate a trigger condition for a data aggregation operation, at least one source data table corresponding to the data aggregation operation, and a target relational schema corresponding to the data aggregation operation; When it is determined that the trigger condition is satisfied, extracting the operation records corresponding to the source data table from the blockchain, and restoring the corresponding source data table according to the operation records; Aggregating the restored at least one source data table into the target relational schema to obtain an aggregated table; The aggregating the restored at least one source data table into the target relational schema to obtain an aggregated table includes: Constructing an aggregated table having the target relational schema, the target relational schema including a table structure and table fields, the table fields including a primary key and non-primary key attributes; Establishing a mapping relationship between the non-primary key attributes of each source data table and the non-primary key attributes of the aggregated table; Determining a primary key mediation table, the primary key mediation table being used to map the primary key of each source data table to the primary key of the primary key mediation table respectively; For each source data table, performing a natural join of the source data table and the primary key mediation table to obtain a corresponding intermediate data table; According to the set first projection condition and the intermediate data table, performing a corresponding projection operation on the source data table, and mapping the result of the projection operation to the table entries of the aggregated table according to the mapping relationship between the source data table and the aggregated table to obtain target data; Performing a union operation on the target data corresponding to each source data table, and writing the result of the union operation into the aggregated table.

2. The method according to claim 1, characterized in that, The method further comprises: When the transaction consensus fails, rolling back the database operation performed on the database.

3. The method according to claim 1, characterized in that, The performing the corresponding database operation on the database includes: Determining a preset conflict detection rule library, the conflict detection rule library including a plurality of prohibition rules, the prohibition rules being used to indicate prohibited operations corresponding to data tables in the database; Decomposing the database operation into a plurality of atomic-level operations on the data tables in the database; Performing conflict detection on the plurality of atomic-level operations according to the conflict detection rule library; Generating a linear operation flow according to the atomic-level operations that do not conflict with the prohibition rules in the conflict detection rule library among the plurality of atomic-level operations; Sequentially performing each atomic-level operation in the linear operation flow on the database corresponding to the blockchain node.

4. The method according to claim 3, characterized in that, The method further comprises: Determine a preset operation transfer rule library, which includes multiple transfer rules, and each transfer rule is used to indicate the processing strategy when the corresponding operation fails; Performing each atomic-level operation in the linear operation stream on the database corresponding to the blockchain node in sequence, includes: In the case where the execution of the atomic-level operation in the linear operation stream fails, determine the target transfer rule in the operation transfer rule library corresponding to the atomic-level operation; Respond to the processing strategy in the target transfer rule.

5. The method according to claim 1, characterized in that, The method further includes: Broadcast the aggregation table to other blockchain nodes in the blockchain network, so that other blockchain nodes record the aggregation table on the blockchain after reaching a consensus on the aggregation table.

6. The method according to claim 1, characterized in that, The method further includes: Construct a historical table with the same relational schema as the aggregation table; Perform corresponding projection operations on the aggregation table according to the set second projection condition to obtain the data to be migrated; Migrate the data to be migrated into the historical table; Broadcast the historical table to other blockchain nodes in the blockchain network, so that other blockchain nodes record the historical table on the blockchain after reaching a consensus on the historical table.

7. A data operation device, characterized in that, A blockchain node configured in a blockchain network, the device includes: A permission verification module, which is used to respond to an operation request initiated for the database corresponding to the blockchain node, and verify whether the initiator of the operation request has the corresponding database operation permission; A database operation module, which is used to perform corresponding database operations on the database when the permission verification is passed; A record-on-chain module, which is used to encapsulate the operation record of the current database operation into a transaction after the database operation is completed, and broadcast the transaction to other blockchain nodes in the blockchain network, so that other blockchain nodes record the transaction on the blockchain after reaching a consensus on the transaction; An aggregation plan determination module, which is used to determine a data aggregation plan, and the data aggregation plan is used to indicate the trigger condition of the data aggregation operation, at least one source data table corresponding to the data aggregation operation, and the target relational schema corresponding to the data aggregation operation; A data table restoration module, which is used to extract the operation records corresponding to the source data table from the blockchain when it is determined that the trigger condition is met, and restore the corresponding source data table according to the operation records; A data aggregation module, which is used to aggregate the restored at least one source data table under the target relational schema to obtain an aggregation table; The data aggregation module includes: An aggregation table construction module, which is used to construct an aggregation table with the target relational schema, the target relational schema includes a table structure and table fields, and the table fields include a primary key and non-primary key attributes; An attribute mapping module, which is used to establish a mapping relationship between the non-primary key attributes of each source data table and the non-primary key attributes of the aggregation table; A primary key mediation module, which is used to determine a primary key mediation table, and the primary key mediation table is used to map the primary keys of each source data table to the primary key of the primary key mediation table respectively; A data mediation module, which is used to perform a natural join on each source data table and the primary key mediation table to obtain a corresponding intermediate data table; A first projection module, which is used to perform a corresponding projection operation on the source data table according to the set first projection condition and the intermediate data table, and map the result of the projection operation to the table items of the aggregation table according to the mapping relationship between the source data table and the aggregation table to obtain target data; An aggregation table obtaining module, which is used to perform a union operation on the target data corresponding to each source data table and write the result of the union operation into the aggregation table.

8. A computer-readable storage medium, on which a computer program is stored, characterized in that, When the program is executed by a processor, it implements the method described in any one of claims 1-6.

9. A blockchain system, characterized in that, It includes a blockchain network composed of multiple blockchain nodes. A data middle platform service module is provided on each blockchain node, and a smart contract is deployed in the data middle platform service module. The blockchain node is used to execute the smart contract in the data middle platform service module to implement the method described in any one of claims 1-6.

Images