Data processing method, device and electronic device based on distributed database

Abstract

The present invention provides a data processing method, device and electronic device based on a distributed database, which can be applied to the field of database data flashback technology. The method includes: the main execution node sends the acquired operation request to the sub-execution node, and the operation request performs an operation on a target table; the sub-execution node sends the operation execution result to the main execution node, wherein the operation execution result is obtained by operating the target table according to the operation request; when the operation execution result indicates successful execution, the main execution node obtains the operation version identifier corresponding to the operation execution result, and sends the operation version identifier to the sub-execution node; the sub-execution node generates a bitmap record file according to the operation version identifier and the operation record information related to the target table; according to the bitmap record file and the operation version identifier, the data in the target table is flashed back and restored.

Description

Technical Field

[0001] The present invention relates to the technical field of database data flashback, and in particular to a data processing method, device and electronic equipment based on a distributed database. Background Art

[0002] FlashBack is an advanced technology for recovering abnormal data in a database. It can restore the data in the database to a previous point in time, thereby eliminating problems caused by misoperation, erroneous data or system failures.

[0003] Existing methods for implementing the flashback function for distributed analytical databases generally rely on distributed transaction log technology. In the process of implementing the flashback function, it is necessary to record multiple versions of transaction logs, etc. Therefore, the cost is high and the server and database overhead is large. Summary of the invention

[0004] In view of the above problems, the present invention provides a data processing method, device and electronic device based on a distributed database.

[0005] According to a first aspect of the present invention, a data processing method based on a distributed database is provided, which is applied to a cluster server, wherein the cluster server includes a main execution node and a sub-execution node, and the data processing method includes:

[0006] The main execution node sends the acquired operation request to the sub-execution node, and the operation request executes the operation on the target table; the sub-execution node sends the operation execution result to the main execution node, wherein the operation execution result is obtained by operating the target table according to the operation request; when the operation execution result indicates a successful execution, the main execution node obtains the operation version identifier corresponding to the operation execution result, and sends the operation version identifier to the sub-execution node; the sub-execution node generates a bitmap record file according to the operation version identifier and the operation record information related to the target table; and performs flashback query and flashback recovery operations on the data in the target table according to the bitmap record file and the operation version identifier.

[0007] Optionally, the cluster server includes multiple sub-execution nodes; the main execution node sends the acquired operation request to the sub-execution node, including:

[0008] In the case where the structure of the target table represents a distributed structure, the main execution node determines at least one benchmark sub-execution node and at least one reference sub-execution node from multiple sub-execution nodes; and sends an operation request to the at least one benchmark sub-execution node.

[0009] Optionally, the child execution node generates a bitmap record file according to the operation version identifier and the operation record information related to the target table, including:

[0010] The benchmark sub-execution node generates benchmark bitmap record information based on the operation record information of the target table stored on the benchmark sub-execution node, wherein the benchmark bitmap record information represents the data change status after the operation is performed on the target table based on the operation request; generates a benchmark file name based on the benchmark sub-execution node name and the operation version identifier; generates a benchmark bitmap record file corresponding to the operation version identifier based on the benchmark bitmap record information and the benchmark file name.

[0011] Optionally, the sub-execution node generates a bitmap record file according to the operation version identifier and the operation record information related to the target table, and further includes:

[0012] The reference sub-execution node generates a reference file name according to the reference sub-execution node name and the operation version identifier; copies the bitmap record information in the previous version bitmap record file corresponding to the target table stored on the reference sub-execution node to obtain the reference bitmap record information; and generates a reference bitmap record file corresponding to the operation version identifier according to the reference bitmap record information and the reference file name.

[0013] Optionally, the data processing method further includes: the main execution node adding the table name identifier, the operation version identifier and the operation request time of the target table to the flashback management table.

[0014] Optionally, based on the bitmap record file and the operation version identifier, perform flashback query and flashback recovery operations on the data in the target table, including:

[0015] The main execution node obtains a flashback query request for a target table, wherein the flashback query request includes target table attributes and query conditions; when the target table attributes include flashback attributes, the main execution node sends a first historical operation version identifier to the sub-execution node, wherein the first historical operation version identifier is obtained by matching the query conditions in the flashback management table, and the first historical operation version identifier represents an identifier for operating the target table at a historical target moment; the sub-execution node sends a bitmap record file corresponding to the first historical operation version identifier to the main execution node; the main execution node sends a data query result corresponding to the bitmap record file to the client.

[0016] Optionally, matching is performed in the flashback management table based on the query condition to obtain the first historical operation version identifier, including:

[0017] In the case where the query conditions include the target table name information, the target database name information, and the historical target time of the target table, a match is performed in the flashback management table based on the target table name information, the target database name information, and the historical target time information to obtain a first historical operation version identifier corresponding to the historical target time of the target table; in the case where the query conditions include the target table name information, the target database name information, and the preset operation version identifier of the target table, a match is performed in the flashback management table based on the target table name information, the target database name information, and the preset operation version identifier to obtain a matching result; in the case where the matching result indicates a successful match, the preset operation version identifier is determined as the first historical operation version identifier.

[0018] Optionally, performing flashback query and flashback recovery operations on the data in the target table according to the bitmap record file and the operation version identifier, further comprising:

[0019] The main execution node obtains a flashback recovery request for a target table, wherein the flashback recovery request includes target table attributes, target table name information, target database name information, and a deletion operation request time; in the case where the target table attributes include flashback attributes, the main execution node sends a second historical operation version identifier to the sub-execution node, wherein the second historical operation version identifier is obtained by matching the target table name information, the target database name information, and the deletion operation request time in a flashback management table, and the second historical operation version identifier represents an identifier of the latest operation before the deletion operation is performed on the target table; the sub-execution node sends a bitmap record file corresponding to the second historical operation version identifier to the main execution node; the main execution node recovers data on the target table according to the bitmap record file.

[0020] A second aspect of the present invention provides a data processing device based on a distributed database, comprising:

[0021] A first acquisition module is used for the main execution node to send the acquired operation request to the sub-execution node, and the operation request executes the operation on the target table;

[0022] An operation module, used for the sub-execution node to send the operation execution result to the main execution node, wherein the operation execution result is obtained by operating the target table according to the operation request;

[0023] A second acquisition module is used for, when the operation execution result indicates successful execution, the main execution node to acquire the operation version identifier corresponding to the operation execution result, and send the operation version identifier to the sub-execution node;

[0024] A generation module, used for the sub-execution node to generate a bitmap record file according to the operation version identifier and the operation record information related to the target table;

[0025] The flashback module is used to perform flashback query and flashback recovery operations on the data in the target table according to the bitmap record file and the operation version identifier.

[0026] The third aspect of the present invention provides an electronic device, comprising: one or more processors; a memory for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors execute the above-mentioned data processing method based on a distributed database.

[0027] According to the data processing method, device and electronic device based on a distributed database provided by the present invention, the operation version identifier corresponding to the operation execution result is obtained through the main execution node, and the operation version identifier is sent to the sub-execution node. The sub-execution node generates a bitmap record file according to the operation version identifier and the operation record information related to the target table; according to the bitmap record file and the operation version identifier, a flashback query and flashback recovery operation are performed on the data in the target table. Since the bitmap record file is used to set the validity of the data after each operation, the memory space of the target table is greatly saved. Without the need to add additional transaction logs, the distributed database is flashed back and flashback recovered in combination with the operation version identifier in the main execution node, which reduces the database and server performance overhead. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] The above and other objects, features and advantages of the present invention will become more apparent through the following description of the embodiments of the present invention with reference to the accompanying drawings, in which:

[0029] Figure 1 An application scenario of a data processing method based on a distributed database according to an embodiment of the present invention is shown;

[0030] Figure 2 A flow chart of a data processing method based on a distributed database according to an embodiment of the present invention is shown;

[0031] Figure 3 An example schematic diagram of a cluster server according to an embodiment of the present invention is shown;

[0032] Figure 4 It shows a structural block diagram of a data processing device based on a distributed database according to an embodiment of the present invention;

[0033] Figure 5 A block diagram of an electronic device suitable for implementing a data processing method based on a distributed database according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0034] Below, embodiments of the present invention will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present invention. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of embodiments of the present invention. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of concepts of the present invention.

[0035] The terms used herein are only for describing specific embodiments and are not intended to limit the present invention. The terms "comprise", "include", etc. used herein indicate the existence of features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.

[0036] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0037] When using expressions such as "at least one of A, B, and C, etc.", they should generally be interpreted according to the meaning of the expression commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0038] In the process of implementing the present invention, it is found that distributed analytical databases focus on transactional characteristics and capabilities. Therefore, if the flashback function is to be implemented in a distributed analytical database, a complete transaction mechanism (including transaction logs, multi-version control (MVCC) technology, etc.) must be implemented. Implementing MVCC not only requires implementing a complete transaction mechanism but also requires recording multiple versions of transaction logs and performing complex garbage collection later. The cost of implementing the flashback function is very high, and the technical threshold will increase accordingly.

[0039] In view of this, an embodiment of the present invention provides a data processing method, device and electronic device based on a distributed database. The method includes: the main execution node sends the acquired operation request to the sub-execution node, and the operation request performs an operation on the target table; the sub-execution node sends the operation execution result to the main execution node, wherein the operation execution result is obtained by operating the target table according to the operation request; when the operation execution result indicates successful execution, the main execution node obtains the operation version identifier corresponding to the operation execution result, and sends the operation version identifier to the sub-execution node; the sub-execution node generates a bitmap record file according to the operation version identifier and the operation record information related to the target table; according to the bitmap record file and the operation version identifier, a flashback query and flashback recovery operation are performed on the data in the target table.

[0040] In the technical solution of the present invention, the user information (including but not limited to user personal information, user image information, user device information, such as location information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved are all information and data authorized by the user or fully authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of the relevant data comply with relevant laws, regulations and standards, take necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0041] Figure 1 The application scenario of the data processing method based on a distributed database according to an embodiment of the present invention is shown.

[0042] like Figure 1 As shown, the business system architecture 100 according to this embodiment may include a first terminal device 101, a second terminal device 102, a third terminal device 103, a network 104, and a cluster server 105. The network 104 is used to provide a medium for a communication link between the first terminal device 101, the second terminal device 102, the third terminal device 103, and the cluster server 105. The network 104 may include various connection types, such as wired, wireless communication links, or optical fiber cables, etc.

[0043] The user may use at least one of the first terminal device 101, the second terminal device 102, and the third terminal device 103 to interact with the cluster server 105 through the network 104 to receive or send messages, etc. Various communication client applications may be installed on the first terminal device 101, the second terminal device 102, and the third terminal device 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc. (only for example).

[0044] The first terminal device 101, the second terminal device 102, and the third terminal device 103 may be various electronic devices having display screens and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, and the like.

[0045] The cluster server 105 may be a server that provides various services, such as a background management server (only as an example) that provides support for websites browsed by users using the first terminal device 101, the second terminal device 102, and the third terminal device 103. The background management server may analyze and process the received data such as user requests, and feed back the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal device.

[0046] It should be noted that the data processing method based on the distributed database provided in the embodiment of the present invention can generally be executed by the cluster server 105. Accordingly, the data processing device based on the distributed database provided in the embodiment of the present invention can generally be set in the cluster server 105. The data processing method based on the distributed database provided in the embodiment of the present invention can also be executed by a server or server cluster that is different from the cluster server 105 and can communicate with the first terminal device 101, the second terminal device 102, the third terminal device 103 and / or the cluster server 105. Accordingly, the data processing device based on the distributed database provided in the embodiment of the present invention can also be set in a server or server cluster that is different from the cluster server 105 and can communicate with the first terminal device 101, the second terminal device 102, the third terminal device 103 and / or the cluster server 105.

[0047] Alternatively, the data processing method based on the distributed database provided in the embodiment of the present invention may also be executed by the first terminal device 101, the second terminal device 102 or the third terminal device 103, or may also be executed by other terminal devices different from the first terminal device 101, the second terminal device 102 or the third terminal device 103. Accordingly, the data processing apparatus based on the distributed database provided in the embodiment of the present invention may also be arranged in the first terminal device 101, the second terminal device 102 or the third terminal device 103, or may be arranged in other terminal devices different from the first terminal device 101, the second terminal device 102 or the third terminal device 103.

[0048] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0049] It should be noted that the sequence numbers of the operations in the following method are only used as representations of the operations for the purpose of description, and should not be regarded as representing the execution order of the operations. Unless explicitly stated, the method does not need to be executed completely in the order shown.

[0050] Figure 2 A flow chart of a data processing method based on a distributed database according to an embodiment of the present invention is shown.

[0051] like Figure 2 As shown, the method 200 includes operations S210 to S250.

[0052] In operation S210 , the main execution node sends the acquired operation request to the sub-execution node.

[0053] Optionally, the target table represents a table corresponding to the operation request, the operation request represents a structured query statement related to the target table information and the operation information, and the operation request performs an operation on the target table, for example, performing data deletion, insertion, update and other operations on the target table.

[0054] Optionally, the cluster server includes a main execution node and a sub-execution node. The main execution node represents a node of the main server that responds to the client receiving the operation request, and the sub-execution node represents a node of the sub-server that executes the operation request based on the operation allocation information of the main execution node.

[0055] Optionally, the table data of the distributed database is distributed and stored on different sub-execution nodes, so that each sub-execution node can independently process its own table data. Each sub-execution node stores a target table, and the data in the target table on each sub-execution node can be the same or different. The main execution node generates operation allocation information according to the operation request, and the sub-execution node executes the operation request on the target table according to the operation allocation information.

[0056] For example, when the data in the target table on each sub-execution node is the same, the operation request is sent to each sub-execution node; when the data in the target table on each sub-execution node is different, the main execution node generates operation allocation information based on the operation request and sends the operation request to the sub-execution node specified in the operation allocation information.

[0057] In operation S220 , the sub-execution node sends the operation execution result to the main execution node.

[0058] Optionally, the sub-execution node operates the target table according to the operation request to obtain an operation execution result. The operation execution result represents the result of execution success or execution failure.

[0059] For example, the target table corresponding to the operation request is t1, and the sub-execution node 001 operates on the target table t1_001 on the sub-execution node 001 according to the operation request to obtain the operation execution result.

[0060] In operation S230, when the operation execution result indicates successful execution, the main execution node obtains an operation version identifier corresponding to the operation execution result, and sends the operation version identifier to the sub-execution node.

[0061] Optionally, the operation version identifier represents a unique identifier for the modification operation on the target table.

[0062] For example, taking the above-mentioned target table t1 as an example, the main execution node sends the obtained operation request to the designated sub-execution node 001. After the sub-execution node 001 updates the target table t1_001, it obtains the operation execution result indicating the successful execution. The sub-execution node 001 sends the operation execution result to the main execution node. The main execution node calls the operation version identification manager to obtain the operation version identification 12345 for the update operation on the target table t1.

[0063] For example, taking the above target table t1 as an example, the main execution node sends the acquired operation request to each sub-execution node (such as sub-execution node 001 and sub-execution node 002) respectively. After sub-execution node 001 performs an update operation on target table t1_001, it obtains an operation execution result indicating successful execution, and sub-execution node 001 sends the operation execution result to the main execution node; after sub-execution node 002 performs an update operation on target table t1_002, it obtains an operation execution result indicating successful execution, and sub-execution node 002 sends the operation execution result to the main execution node. The main execution node calls the operation version identifier manager to obtain the operation version identifier 12345 for the update operation on target table t1.

[0064] Optionally, after the main execution node obtains the operation version identifier corresponding to the operation execution result, the operation version identifier is sent to each sub-execution node respectively.

[0065] In operation S240, the child execution node generates a bitmap record file according to the operation version identifier and the operation record information related to the target table.

[0066] For example, the operation record information related to the target table may be information on deleting the data “cd” in the target table.

[0067] Optionally, the information in the bitmap record file is used to mark the validity of the data. The structure of the bitmap record file is that one bit corresponds to a row of data. If it is set to 1, the data is valid, and if it is set to 0, it is considered invalid or deleted.

[0068] For example, after a delete operation is performed on the data "cd" in the target table, the data "cd" in the bitmap record file is marked as 0; for example, after an insert operation is performed on the data "a" in the target table, the data "a" in the bitmap record file is marked as 1.

[0069] For example, taking the above-mentioned main execution node sending the obtained operation request to sub-execution node 001 and sub-execution node 002 respectively, after the main execution node sends the operation version identifier 12345 to sub-execution node 001 and sub-execution node 002 respectively, sub-execution node 001 obtains the bitmap record file of sub-execution node 001 according to the operation version identifier 12345 and the operation record information related to the target table t1_001; sub-execution node 002 obtains the bitmap record file of sub-execution node 002 according to the operation version identifier 12345 and the operation record information related to the target table t1_002.

[0070] In operation S250, a flashback query and a flashback recovery operation are performed on the data in the target table according to the bitmap record file and the operation version identifier.

[0071] Optionally, a flashback query and a flashback recovery operation are performed on the data in the target table according to the bitmap record files saved by each sub-execution node and the operation version identifier saved by the main execution node.

[0072] For example, query or restore data in the target table at a certain historical moment.

[0073] Optionally, by using a bitmap record file to set the validity of the data after each operation, the memory space of the target table is greatly saved. Without adding additional transaction logs, the distributed database can be flashed back and restored in combination with the operation version identifier in the main execution node, thereby reducing the database and server performance overhead.

[0074] Optionally, the cluster server includes multiple sub-execution nodes; the main execution node sends the acquired operation request to the sub-execution node, including:

[0075] In the case where the structure of the target table represents a distributed structure, the main execution node determines at least one benchmark sub-execution node and at least one reference sub-execution node from multiple sub-execution nodes; and sends an operation request to the at least one benchmark sub-execution node.

[0076] Optionally, for a target table of a distributed structure, each sub-execution node stores part of the data of the target table t1, and the data in the target table on each sub-execution node is different.

[0077] For example, for the target table t2 of the distributed structure, the cluster server includes the main execution node, the sub-execution node 003, and the sub-execution node 004. The data of the target table t2_003 on the sub-execution node 003 is different from that of the target table t2_004 on the sub-execution node 004. The main execution node generates operation allocation information according to the operation request and sends the operation request to the sub-execution node specified in the operation allocation information.

[0078] For example, if the operation allocation information specifies allocation to sub-execution node 003 , sub-execution node 003 is determined as the base sub-execution node, and sub-execution node 004 is determined as the reference sub-execution node. The main execution node sends the operation request to the base sub-execution node 003 .

[0079] Optionally, when the main execution node determines multiple benchmark sub-execution nodes, the main execution node sends the operation request to the multiple benchmark sub-execution nodes respectively.

[0080] Optionally, the child execution node generates a bitmap record file according to the operation version identifier and the operation record information related to the target table, including:

[0081] The benchmark sub-execution node generates benchmark bitmap record information based on the operation record information of the target table stored on the benchmark sub-execution node, wherein the benchmark bitmap record information represents the data change status after the operation is performed on the target table based on the operation request; generates a benchmark file name based on the benchmark sub-execution node name and the operation version identifier; generates a benchmark bitmap record file corresponding to the operation version identifier based on the benchmark bitmap record information and the benchmark file name.

[0082] Optionally, an operation request is executed on the target table to generate operation record information, thereby changing the bitmap record information in the bitmap record file.

[0083] Table 1 schematically shows the relationship between operation types and bitmap record files in an embodiment of the present invention.

[0084]

[0085] Optionally, for the operation type of creating a target table (create), it involves related modifications to the table attributes in the metadata file and has nothing to do with the data. Therefore, there is no need to generate a bitmap record file; for the operation type of forcibly deleting the target table (drop), it is necessary to delete the target table and data, and delete the bitmap record file at the same time; for the operation type of deleting all the data in the target table (truncate), it is related to the data, and a new bitmap record file is generated to mark the data validity; for the operation type of appending data after the data row (insert), it is related to the data, and a new bitmap record file is generated to mark the data validity; for the operation type of deleting part of the target table data (delete), it is related to the data, and a new bitmap record file is generated to mark the data validity; for the operation type of updating part of the target table data (update), it is related to the data, and a new bitmap record file is generated to mark the data validity.

[0086] Optionally, the reference sub-execution node performs a corresponding change operation on the target table according to the operation request to obtain operation record information. For example, after the reference sub-execution node 003 inserts data "b" into the target table t1_003, according to the operation record information of inserting data "b", the reference bitmap record information is obtained: data "b" is marked as 1.

[0087] For example, according to the benchmark sub-execution node name 003 and the operation version identifier 12345, a benchmark file name 12345_003 is generated.

[0088] For example, according to the reference file name 12345_003 and the reference bitmap record information: the data "b" is marked as 1, the reference bitmap record file corresponding to the reference sub-execution node 003 and the operation version identifier 12345 is obtained.

[0089] Table 2 schematically shows a reference bitmap record file table of different versions on the reference sub-execution node name 003 according to an embodiment of the present invention.

[0090]

[0091] For example, for the initialization operation request at the first request time, the benchmark sub-execution node 003 is determined, the main execution node sends the initialization operation request (such as inserting data 'abc', 'hello', 'world' in the target table t1) to the sub-execution node 003, the sub-execution node 003 sends the operation execution result to the main execution node, the main execution node applies for the operation version identifier 1111 and sends it to the sub-execution node 003, the sub-execution node 003 generates an initialized benchmark bitmap record file, the benchmark file name is 1111_003, and the benchmark bitmap records information: 'abc' mark 1, 'hello' mark 1, 'world' mark 1.

[0092] For example, for the operation request at the second request time, the benchmark sub-execution node 003 is determined, the main execution node sends the operation request (such as deleting data 'abc' in the target table t1) to the sub-execution node 003, the sub-execution node 003 sends the operation execution result to the main execution node, the main execution node applies for the operation version identifier 1112 and sends it to the sub-execution node 003, the sub-execution node 003 generates a benchmark bitmap record file, the benchmark file name is 1112_003, the benchmark bitmap records information: 'abc' mark 0, 'hello' mark 1, 'world' mark 1.

[0093] For example, for the operation request at the third request time, the benchmark sub-execution node 003 is determined, the main execution node sends the operation request (such as inserting data 'nice' and 'day' in the target table t1) to the sub-execution node 003, the sub-execution node 003 sends the operation execution result to the main execution node, the main execution node applies for the operation version identifier 1113 and sends it to the sub-execution node 003, the sub-execution node 003 generates a benchmark bitmap record file, the benchmark file name is 1113_003, and the benchmark bitmap records information: 'abc' mark 0, 'hello' mark 1, 'world' mark 1, 'nice' mark 1, 'day' mark 1.

[0094] For example, for the operation request at the fourth request time, the benchmark sub-execution node 003 is determined, the main execution node sends the operation request (such as updating the data 'world' to 'sun' and 'day' to 'weather' in the target table t1) to the sub-execution node 003, the sub-execution node 003 sends the operation execution result to the main execution node, the main execution node applies for the operation version identifier 1114 and sends it to the sub-execution node 003, the sub-execution node 003 generates a benchmark bitmap record file, the benchmark file name is 1114_003, the benchmark bitmap records information: 'abc' mark 10, 'hello' mark 1, 'world' mark 0, 'nice' mark 1, 'day' mark 0, 'sun' mark 1, 'weather' mark 1.

[0095] Optionally, if for the target table of the replication structure, all the data of the target table t1 is stored on each sub-execution node, and the data in the target table on each sub-execution node is the same. After obtaining the operation request, the main execution node sends the operation request to each sub-execution node respectively, and each sub-execution node executes the same operation request and sends the operation execution result to the main execution node. The main execution node applies for an operation version identifier for the operation request and sends it to each sub-execution node. Each sub-execution node generates its own bitmap record file based on the operation version identifier and the operation record information of the target table stored on the sub-execution node.

[0096] Optionally, a data block can have 65536 rows of data. Using a bitmap record file to represent the information of the data block only requires 65536 / 8 bytes. If the bits are set the same, no additional space is occupied, which saves the memory space occupied by the data and improves the flexibility of flashback query and flashback recovery of distributed databases.

[0097] Optionally, the sub-execution node generates a bitmap record file according to the operation version identifier and the operation record information related to the target table, and further includes:

[0098] The reference sub-execution node generates a reference file name according to the reference sub-execution node name and the operation version identifier; copies the bitmap record information in the previous version bitmap record file corresponding to the target table stored on the reference sub-execution node to obtain the reference bitmap record information; and generates a reference bitmap record file corresponding to the operation version identifier according to the reference bitmap record information and the reference file name.

[0099] For example, for the operation request at the fifth request time, the benchmark sub-execution node 004 is determined, the main execution node sends the operation request (such as inserting data 'strong wind' in the target table t1) to the benchmark sub-execution node 004, the benchmark sub-execution node 004 sends the operation execution result to the main execution node, the main execution node applies for the operation version identifier 1115 and sends it to the benchmark sub-execution node 004, the sub-execution node 004 generates a benchmark bitmap record file, the benchmark file name is 1115_004, and the benchmark bitmap record information: 'strong wind' mark 1.

[0100] For example, for the operation request at the sixth request time, the benchmark sub-execution node 003 and the reference sub-execution node 004 are determined, the main execution node sends the operation request (such as inserting data 'cloudy' in the target table t1) to the benchmark sub-execution node 003, the benchmark sub-execution node 003 sends the operation execution result to the main execution node, the main execution node applies for the operation version identifier 1116 and sends it to the benchmark sub-execution node 003, the sub-execution node 003 generates a benchmark bitmap record file, the benchmark file name is 1116_003, and the benchmark bitmap records information: 'abc' mark 0, 'hello' mark 1, 'world' mark 0, 'nice' mark 1, 'day' mark 0, 'sun' mark 1, 'weather' mark 1, 'weather' mark 1.

[0101] The reference sub-execution node 004 does not execute the operation request, and generates a reference bitmap record file. The reference file name is 1116_004, and the reference bitmap record information is the same as the bitmap record information of the previous version 1115_004: 'strong wind' mark 1.

[0102] Optionally, for each operation request, a baseline sub-execution node and a reference sub-execution node are determined. Even if the reference sub-execution node does not execute the operation request, a reference bitmap record file needs to be generated. This ensures that the bitmap record files in the baseline sub-execution node and the reference sub-execution node are maintained at the same generation time to facilitate subsequent flashback query and flashback recovery operations.

[0103] Optionally, the data processing method further includes: the main execution node adding the table name identifier, the operation version identifier and the operation request time of the target table to the flashback management table.

[0104] Optionally, each operation of the version identifier by the main execution node needs to be recorded in the flashback management table, and the flashback management table is stored on the main execution node.

[0105] Optionally, the main execution node processes the operation request and executes the table opening logic, reads the relevant attributes of the target table into the memory, and further determines whether the target table contains flashback attributes. If it contains flashback attributes, after the main execution node applies for the operation version identifier, the main execution node calls the system table manager to add the table name identifier of the target table, the library name identifier of the database where the target table is located, the operation version identifier, the operation request time and related operations to the flashback management table; if it does not contain flashback attributes, the write logic of the ordinary table does not need to be added to the flashback management table.

[0106] For example, taking the first initialization operation request for the target table t1 as an example, the target table t1 includes a flashback attribute. The sub-execution node 003 sends the operation execution result to the main execution node. The main execution node applies for the operation version identifier 1111 and calls the system table manager to add the table name identifier t1, the library name identifier db_1 of the database where the target table is located, the operation version identifier 1111, the operation request time 4.00 and the operation type "insert" to the flashback management table.

[0107] Figure 3 An exemplary schematic diagram of a cluster server according to an embodiment of the present invention is shown.

[0108] like Figure 3As shown, the cluster server includes a main execution node and multiple sub-execution nodes (such as sub-execution nodes 001, ..., sub-execution node 00n). The main execution node sends the obtained operation request to the sub-execution node, and the sub-execution node sends the operation execution result to the main execution node. When the operation execution result indicates successful execution, the main execution node calls the operation version identification manager to obtain the operation version identification corresponding to the operation execution result, calls the system table manager to add the table name identification of the target table, the library name identification of the database where the target table is located, the operation version identification, the operation request time and related operations to the flashback management table, and sends the operation version identification to the sub-execution node; the sub-execution node calls the file manager to generate a bitmap record file according to the operation version identification and the operation record information related to the target table, the data file stores the data after the target table data change operation, and the metadata file stores the attribute information of the target table, such as table structure information, column attribute information, flashback attribute information, etc.

[0109] Optionally, the main execution node obtains a flashback query operation or a flashback recovery request, and performs a flashback query or a flashback recovery operation on the data in the target table according to the bitmap record file in the child execution node and the operation version identifier stored in the flashback management table.

[0110] Optionally, based on the bitmap record file and the operation version identifier, perform flashback query and flashback recovery operations on the data in the target table, including:

[0111] The main execution node obtains a flashback query request for a target table, wherein the flashback query request includes target table attributes and query conditions; when the target table attributes include flashback attributes, the main execution node sends a first historical operation version identifier to the sub-execution node, wherein the first historical operation version identifier is obtained by matching the query conditions in the flashback management table, and the first historical operation version identifier represents an identifier for operating the target table at a historical target moment; the sub-execution node sends a bitmap record file corresponding to the first historical operation version identifier to the main execution node; the main execution node sends a data query result corresponding to the bitmap record file to the client.

[0112] Optionally, the flashback query request represents a structured query statement related to the target table information and query information. The main execution node processes the flashback query request and executes the table opening logic to obtain the target table attributes and query conditions. The target table attributes may include table structure attributes and flashback attributes. The query conditions may include query information such as the table name, library name, and historical target time of the target table.

[0113] Optionally, the table structure attribute can be replication or distribution. The flashback attribute is used to identify whether the target table is stored in the flashback management table.

[0114] Optionally, determine whether the target table attributes of the target table include flashback attributes. If so, match them in the flashback management table based on the query conditions. If not, return an error result to the client and indicate that the target table is not a flashback table and the flashback query function cannot be used.

[0115] Optionally, the main execution node calls the system table manager to match the flashback management table according to the query conditions to obtain the first historical operation version identifier of the target table. The main execution node sends the first historical operation version identifier in the flashback management table to the sub-execution node, which may include: sending the first historical operation version identifier to each sub-execution node when the table structure attribute of the target table is distributed; sending the first historical operation version identifier to any sub-execution node when the table structure attribute of the target table is replicated.

[0116] Optionally, the sub-execution node assembles a bitmap record file name according to the first historical operation version identifier and transmits it to the file manager. The file manager goes to a designated directory and loads the bitmap record file into the memory according to the bitmap record file name.

[0117] Optionally, a data query result corresponding to the bitmap record file is obtained according to the validity of the data recorded in the bitmap record file, and the bitmap record file and the data query result corresponding to the bitmap record file are sent to the main execution node. The main execution node sends the data query result corresponding to the bitmap record file to the client.

[0118] For example, taking the operation request at the second request time as an example, the table structure attribute of the target table t1 is distributed. For the flashback query request with the historical target time being the second request time, the main execution node matches the flashback management table according to the query conditions, and obtains the first historical operation version identifier 1112 of the target table t1. The main execution node sends the first historical operation version identifier 1112 in the flashback management table to each sub-execution node (base sub-execution node 003, reference sub-execution node 004). The base sub-execution node 003 assembles the bitmap record file name 1112_003 according to the first historical operation version identifier 1112. The file manager goes to the specified directory and loads the bitmap record file into the memory according to the bitmap record file name 1112_003. According to the validity of the data recorded in the bitmap record file, the data query results corresponding to the bitmap record file are obtained as: 'hello', 'world'. The bitmap record file and the data query results: 'hello', 'world' are sent to the main execution node

[0119] For example, the reference sub-execution node 004 assembles the bitmap record file name 1112_004 according to the first historical operation version identifier 1112, and the file manager goes to the designated directory to load the bitmap record file into the memory according to the bitmap record file name 1112_004. Since the reference sub-execution node 004 did not execute the operation request at the first request time and the second request time, the bitmap record file is an empty table, and the data query result corresponding to the bitmap record file is also empty. The bitmap record file and the empty data query result are sent to the main execution node.

[0120] For example, the main execution node performs integration and finally sends the data query results: 'hello', 'world' to the client.

[0121] Optionally, matching is performed in the flashback management table based on the query condition to obtain the first historical operation version identifier, including:

[0122] In the case where the query conditions include the target table name information, the target database name information, and the historical target time of the target table, a match is performed in the flashback management table based on the target table name information, the target database name information, and the historical target time information to obtain a first historical operation version identifier corresponding to the historical target time of the target table; in the case where the query conditions include the target table name information, the target database name information, and the preset operation version identifier of the target table, a match is performed in the flashback management table based on the target table name information, the target database name information, and the preset operation version identifier to obtain a matching result; in the case where the matching result indicates a successful match, the preset operation version identifier is determined as the first historical operation version identifier.

[0123] Optionally, the flashback management table includes table name information corresponding to each of the multiple user tables, database name information, operation time, and operation version identification information corresponding to each operation time, etc. The main execution node calls the system table manager to match the flashback management table according to the query condition, thereby obtaining the first historical operation version identification corresponding to the historical operation time of the target table.

[0124] For example, the query conditions include the target table name information t1, the target database name information db_1, and the historical target time 4.00 of the target table. In the flashback management table, a match is performed based on the target table name information t1, the target database name information db_1, and the historical target time 4.00 to obtain the first historical operation version identifier 1111 corresponding to the historical target time 4.00 of the target table.

[0125] For example, the query condition includes the target table name information t1, the target database name information db_1, and the preset operation version identifier 1111 of the target table. In the flashback management table, a matching result is obtained according to the target table name information t1, the target database name information db_1, and the preset operation version identifier 1111. The matching result can be a successful match or a failed match. When the matching result indicates a successful match, it means that the target table name information t1, the target database name information db_1, and the preset operation version identifier 1111 are stored in the flashback management table, and therefore, the preset operation version identifier 1111 is determined as the first historical operation version identifier 1111.

[0126] For example, when the matching result indicates a matching failure, an error result is returned and the flashback query program is exited.

[0127] Optionally, performing flashback query and flashback recovery operations on the data in the target table according to the bitmap record file and the operation version identifier, further comprising:

[0128] The main execution node obtains a flashback recovery request for a target table, wherein the flashback recovery request includes target table attributes, target table name information, target database name information, and a deletion operation request time; in the case where the target table attributes include flashback attributes, the main execution node sends a second historical operation version identifier to the sub-execution node, wherein the second historical operation version identifier is obtained by matching the target table name information, the target database name information, and the deletion operation request time in a flashback management table, and the second historical operation version identifier represents an identifier of the latest operation before the deletion operation is performed on the target table; the sub-execution node sends a bitmap record file corresponding to the second historical operation version identifier to the main execution node; the main execution node recovers data on the target table according to the bitmap record file.

[0129] Optionally, the flashback recovery request represents a structured query statement related to the target table information and the recovery information. The flashback recovery request is used to recover the information before the deletion operation is performed on the target table. The deletion operation includes table structure deletion and table data deletion.

[0130] Optionally, table data deletion operations (such as truncate) do not affect the use of the target table. Recovery can be performed based on the flashback management table and flashback recovery request.

[0131] Optionally, the main execution node processes the flashback recovery request and executes the table opening logic, determines the recovery operation identifier, calls the system table manager to match the flashback management table according to the target table name information, target database name information, and deletion operation request time, and obtains the second historical operation version identifier of the target table.

[0132] Optionally, the second historical operation version identifier represents the identifier of the latest operation before the target table is deleted. For example, taking the operation request at the second request time as an example, the latest operation before the target table is deleted is the operation at the first request time. According to the target table name information t1, the target database name information db_1, the deletion operation request time is the second request time, and the flashback recovery identifier is matched in the flashback management table, therefore, the second historical operation version identifier is determined to be 1111.

[0133] Optionally, the main execution node sending the second historical operation version identifier in the flashback management table to the child execution node may include: when the table structure attribute of the target table is distributed, sending the second historical operation version identifier to each child execution node; when the table structure attribute of the target table is replicated, sending the second historical operation version identifier to any child execution node.

[0134] Optionally, the sub-execution node sends a bitmap record file corresponding to the second historical operation version identifier to the main execution node; the main execution node restores data of the target table according to the bitmap record file.

[0135] Optionally, a table structure deletion operation (such as drop) affects the use of the target table. Therefore, when performing flashback recovery on the target table of the table structure deletion operation, it is determined that the target table attributes include flashback attributes and the table structure deletion operation information of the target table is recorded in the flashback management table. In this case, it is necessary to directly restore the table structure information based on the metadata file of the child execution node.

[0136] Based on the above-mentioned data processing method based on a distributed database, the present invention also provides a data processing device based on a distributed database. Figure 4 The device is described in detail.

[0137] Figure 4 A structural block diagram of a data processing device based on a distributed database according to an embodiment of the present invention is shown.

[0138] like Figure 4 As shown, the data processing device 400 based on a distributed database of this embodiment includes a first acquisition module 410 , an operation module 420 , a second acquisition module 430 , a generation module 440 and a flashback module 450 .

[0139] The first acquisition module 410 is used for the main execution node to send the acquired operation request to the sub-execution node, and the operation request performs an operation on the target table. In one embodiment, the first acquisition module 410 can be used to perform the operation S210 described above, which will not be repeated here.

[0140] The operation module 420 is used for the sub-execution node to send the operation execution result to the main execution node, wherein the operation execution result is obtained by operating the target table according to the operation request. In one embodiment, the operation module 420 can be used to perform the operation S220 described above, which will not be repeated here.

[0141] The second acquisition module 430 is used for the main execution node to acquire the operation version identifier corresponding to the operation execution result when the operation execution result indicates successful execution, and send the operation version identifier to the sub-execution node. In one embodiment, the second acquisition module 430 can be used to perform the operation S230 described above, which will not be repeated here.

[0142] The generation module 440 is used for the child execution node to generate a bitmap record file according to the operation version identifier and the operation record information related to the target table. In one embodiment, the generation module 440 can be used to perform the operation S240 described above, which will not be repeated here.

[0143] The flashback module 450 is used to perform flashback query and flashback recovery operations on the data in the target table according to the bitmap record file and the operation version identifier. In one embodiment, the flashback module 450 can be used to perform the operation S250 described above, which will not be described in detail here.

[0144] Optionally, the first acquisition module 410 includes a determination submodule and a sending submodule.

[0145] The determination submodule is used for, when the structure of the target table represents a distributed structure, the main execution node determines at least one benchmark sub-execution node and at least one reference sub-execution node from a plurality of sub-execution nodes.

[0146] The sending submodule is used to send the operation request to at least one benchmark sub-execution node.

[0147] Optionally, the generation module 440 includes a first generation submodule, a second generation submodule and a third generation submodule.

[0148] The first generating submodule is used for the base sub-execution node to generate base bitmap record information according to the operation record information of the target table stored on the base sub-execution node, wherein the base bitmap record information represents the data change state after the operation is performed on the target table based on the operation request.

[0149] The second generating submodule is used to generate a benchmark file name according to the benchmark sub-execution node name and the operation version identifier.

[0150] The third generating submodule is used to generate a reference bitmap record file corresponding to the operation version identifier according to the reference bitmap record information and the reference file name.

[0151] Optionally, the generation module 440 further includes a fourth generation submodule, a fifth generation submodule and a sixth generation submodule.

[0152] The fourth generating submodule is used for the reference sub-execution node to generate a reference file name according to the reference sub-execution node name and the operation version identifier.

[0153] The fifth generation submodule is used to copy the bitmap record information in the previous version bitmap record file corresponding to the target table stored on the reference sub-execution node to obtain the reference bitmap record information.

[0154] The sixth generation submodule is used to generate a reference bitmap record file corresponding to the operation version identifier according to the reference bitmap record information and the reference file name.

[0155] Optionally, the data processing device 400 based on the distributed database further includes an update module.

[0156] The update module is used by the main execution node to add the table name identifier, operation version identifier and operation request time of the target table to the flashback management table.

[0157] Optionally, the flashback module 450 includes a first flashback sub-module, a second flashback sub-module, a third flashback sub-module and a fourth flashback sub-module.

[0158] The first flashback submodule is used for the main execution node to obtain a flashback query request for a target table, wherein the flashback query request includes target table attributes and query conditions.

[0159] The second flashback sub-module is used for, when the target table attributes include flashback attributes, the main execution node sending the first historical operation version identifier to the sub-execution node, wherein the first historical operation version identifier is obtained by matching the flashback management table based on the query conditions, and the first historical operation version identifier represents the identifier of the operation on the target table at the historical target moment.

[0160] The third flashback sub-module is used for the sub-execution node to send the bitmap record file corresponding to the first historical operation version identifier to the main execution node.

[0161] The fourth flashback submodule is used for the main execution node to send the data query result corresponding to the bitmap record file to the client.

[0162] Optionally, the second flashback submodule includes a first flashback unit, a second flashback unit and a third flashback unit.

[0163] The first flashback unit is used to match the target table name information, target database name information and historical target time information in the flashback management table when the query conditions include the target table name information, target database name information and historical target time of the target table, so as to obtain the first historical operation version identifier corresponding to the historical target time of the target table.

[0164] The second flashback unit is used to match the target table name information, target database name information and preset operation version identifier in the flashback management table to obtain a matching result when the query conditions include the target table name information, target database name information and preset operation version identifier of the target table.

[0165] The third flashback unit is used to determine the preset operation version identifier as the first historical operation version identifier when the matching result indicates that the match is successful.

[0166] Optionally, the flashback module 450 further includes a fifth flashback sub-module, a sixth flashback sub-module, a seventh flashback sub-module and an eighth flashback sub-module.

[0167] The fifth flashback submodule is used for the main execution node to obtain a flashback recovery request for the target table, wherein the flashback recovery request includes target table attributes, target table name information, target database name information, and deletion operation request time.

[0168] The sixth flashback sub-module is used for, when the target table attributes include flashback attributes, the main execution node sending the second historical operation version identifier to the sub-execution node, wherein the second historical operation version identifier is obtained by matching the flashback management table based on the target table name information, the target database name information, and the deletion operation request time, and the second historical operation version identifier represents the identifier of the latest operation before the deletion operation is performed on the target table.

[0169] The seventh flashback submodule is used for the sub-execution node to send the bitmap record file corresponding to the second historical operation version identifier to the main execution node.

[0170] The eighth flashback submodule is used for the main execution node to restore data of the target table according to the bitmap record file.

[0171] Optionally, any multiple modules among the first acquisition module 410, the operation module 420, the second acquisition module 430, the generation module 440 and the flashback module 450 can be combined into one module for implementation, or any one of the modules can be split into multiple modules. Alternatively, at least part of the functions of one or more of these modules can be combined with at least part of the functions of other modules and implemented in one module. Optionally, at least one of the first acquisition module 410, the operation module 420, the second acquisition module 430, the generation module 440 and the flashback module 450 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application specific integrated circuit (ASIC), or can be implemented by hardware or firmware such as any other reasonable way of integrating or packaging the circuit, or implemented in any one of the three implementation methods of software, hardware and firmware or in a suitable combination of any of them. Alternatively, at least one of the first acquisition module 410 , the operation module 420 , the second acquisition module 430 , the generation module 440 , and the flashback module 450 may be at least partially implemented as a computer program module, which may perform corresponding functions when executed.

[0172] Figure 5 A block diagram of an electronic device suitable for implementing a data processing method based on a distributed database according to an embodiment of the present invention is shown.

[0173] Figure 5 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0174] like Figure 5 As shown, the computer electronic device 500 according to an embodiment of the present invention includes a processor 501, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 502 or the program loaded from the storage part 508 to the random access memory (RAM) 503. The processor 501 may include, for example, a general-purpose microprocessor (such as a CPU), an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (for example, an application-specific integrated circuit (ASIC)), etc. The processor 501 may also include an on-board memory for caching purposes. The processor 501 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present invention.

[0175] In RAM 503, various programs and data required for the operation of electronic device 500 are stored. Processor 501, ROM 502 and RAM 503 are connected to each other via bus 504. Processor 501 performs various operations of the method flow according to the embodiment of the present invention by executing the program in ROM 502 and / or RAM 503. It should be noted that the program can also be stored in one or more memories other than ROM 502 and RAM 503. Processor 501 can also perform various operations of the method flow according to the embodiment of the present invention by executing the program stored in one or more memories.

[0176] Optionally, the electronic device 500 may further include an input / output (I / O) interface 505, which is also connected to the bus 504. The electronic device 500 may further include one or more of the following components connected to the input / output (I / O) interface 505: an input portion 506 including a keyboard, a mouse, etc.; an output portion 507 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage portion 508 including a hard disk, etc.; and a communication portion 509 including a network interface card such as a LAN card, a modem, etc. The communication portion 509 performs communication processing via a network such as the Internet. A drive 510 is also connected to the input / output (I / O) interface 505 as needed. A removable medium 511, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 510 as needed, so that a computer program read therefrom is installed into the storage portion 508 as needed.

[0177] Optionally, the method flow according to the embodiment of the present invention can be implemented as a computer software program. For example, an embodiment of the present invention includes a computer program product, which includes a computer program carried on a computer-readable storage medium, and the computer program contains a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 509, and / or installed from the removable medium 511. When the computer program is executed by the processor 501, the above-mentioned functions defined in the system of the embodiment of the present invention are executed. Optionally, the system, equipment, device, module, unit, etc. described above can be implemented by a computer program module.

[0178] The present invention also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiment; or may exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the data processing method according to the embodiment of the present invention is implemented.

[0179] Optionally, the computer-readable storage medium may be a non-volatile computer-readable storage medium. For example, it may include, but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, apparatus, or device.

[0180] For example, optionally, the computer-readable storage medium may include the ROM 502 and / or the RAM 503 described above and / or one or more memories other than the ROM 502 and the RAM 503 .

[0181] An embodiment of the present invention also includes a computer program product, which includes a computer program, and the computer program contains program code for executing the method provided by the embodiment of the present invention. When the computer program product runs on an electronic device, the program code is used to enable the electronic device to implement the data processing method provided by the embodiment of the present invention.

[0182] When the computer program is executed by the processor 501, the above functions defined in the system / device of the embodiment of the present invention are performed. Optionally, the system, device, module, unit, etc. described above can be implemented by a computer program module.

[0183] In one embodiment, the computer program may rely on tangible storage media such as optical storage devices, magnetic storage devices, etc. In another embodiment, the computer program may also be transmitted and distributed in the form of signals on a network medium, and downloaded and installed through the communication part 509, and / or installed from the removable medium 511. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0184] Optionally, the program code for executing the computer program provided by the embodiment of the present invention can be written in any combination of one or more programming languages, specifically, these computing programs can be implemented using high-level process and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, such as Java, C++, python, "C" language or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on the remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (e.g., using an Internet service provider to connect through the Internet).

[0185] Optionally, the flowcharts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present invention. In this regard, each box in the flowchart or block diagram may represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of the code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box may also occur in an order different from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart, can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions. It can be understood by those skilled in the art that the features recorded in the various embodiments of the present invention can be combined and / or combined in a variety of ways, even if such a combination or combination is not clearly recorded in the present invention. In particular, without departing from the spirit and teachings of the present invention, the features described in the various embodiments of the present invention may be combined and / or combined in a variety of ways. All of these combinations and / or combinations fall within the scope of the present invention.

[0186] The embodiments of the present invention are described above. However, these embodiments are only for the purpose of illustration, and are not intended to limit the scope of the present invention. Although each embodiment is described above, it does not mean that the measures in each embodiment cannot be used in combination advantageously. Without departing from the scope of the present invention, those skilled in the art may make various substitutions and modifications, which should all fall within the scope of the present invention.

Claims

1. A data processing method based on a distributed database, characterized in that: The method is applied to a cluster server, the cluster server includes a main execution node and a sub-execution node, and the method includes: The main execution node sends the acquired operation request to the sub-execution node, and the operation request performs an operation on the target table; The sub-execution node sends the operation execution result to the main execution node, wherein the operation execution result is obtained by operating the target table according to the operation request; In the case where the operation execution result indicates successful execution, the main execution node obtains an operation version identifier corresponding to the operation execution result, and sends the operation version identifier to the sub-execution node; The child execution node generates a bitmap record file according to the operation version identifier and the operation record information related to the target table; The main execution node adds the table name identifier of the target table, the operation version identifier and the operation request time to the flashback management table; According to the bitmap record file and the operation version identifier, performing a flashback query and a flashback recovery operation on the data in the target table includes: The main execution node obtains a flashback query request for the target table, wherein the flashback query request includes target table attributes and query conditions; In the case where the target table attribute includes a flashback attribute, the main execution node sends a first historical operation version identifier to the child execution node, wherein the first historical operation version identifier is obtained by matching the flashback management table based on the query condition, and the first historical operation version identifier represents an identifier for operating the target table at a historical target time; The sub-execution node sends the bitmap record file corresponding to the first historical operation version identifier to the main execution node; The main execution node sends the data query result corresponding to the bitmap record file to the client.

2. The method according to claim 1, characterized in that The cluster server includes a plurality of sub-execution nodes; The main execution node sends the acquired operation request to the sub-execution node, including: In the case where the structure of the target table represents a distributed structure, the main execution node determines at least one benchmark sub-execution node and at least one reference sub-execution node from the plurality of sub-execution nodes; The operation request is sent to at least one of the benchmark sub-execution nodes.

3. The method according to claim 2, characterized in that The child execution node generates a bitmap record file according to the operation version identifier and the operation record information related to the target table, including: The reference sub-execution node generates reference bitmap record information according to the operation record information of the target table stored on the reference sub-execution node, wherein the reference bitmap record information represents the data change state after the operation is performed on the target table based on the operation request; Generate a benchmark file name according to the benchmark sub-execution node name and the operation version identifier; A reference bitmap record file corresponding to the operation version identifier is generated according to the reference bitmap record information and the reference file name.

4. The method according to claim 2, characterized in that: The sub-execution node generates a bitmap record file according to the operation version identifier and the operation record information related to the target table, and further includes: The reference sub-execution node generates a reference file name according to the reference sub-execution node name and the operation version identifier; Copying the bitmap record information in the previous version bitmap record file corresponding to the target table stored on the reference sub-execution node to obtain reference bitmap record information; A reference bitmap record file corresponding to the operation version identifier is generated according to the reference bitmap record information and the reference file name.

5. The method according to claim 1, characterized in that Matching the flashback management table based on the query condition to obtain the first historical operation version identifier includes: In a case where the query condition includes target table name information, target database name information, and the historical target time of the target table, matching is performed in the flashback management table according to the target table name information, the target database name information, and the historical target time information to obtain the first historical operation version identifier corresponding to the historical target time of the target table; In a case where the query condition includes the target table name information, the target database name information, and a preset operation version identifier of the target table, matching is performed in the flashback management table according to the target table name information, the target database name information, and the preset operation version identifier to obtain a matching result; When the matching result indicates a successful match, the preset operation version identifier is determined as the first historical operation version identifier.

6. The method according to claim 1, characterized in that According to the bitmap record file and the operation version identifier, a flashback query and a flashback recovery operation are performed on the data in the target table, further comprising: The main execution node obtains a flashback recovery request for the target table, wherein the flashback recovery request includes target table attributes, target table name information, target database name information, and deletion operation request time; In the case where the target table attribute includes a flashback attribute, the main execution node sends a second historical operation version identifier to the child execution node, wherein the second historical operation version identifier is obtained by matching the flashback management table based on the target table name information, the target database name information, and the deletion operation request time, and the second historical operation version identifier represents an identifier of the latest operation before the deletion operation is performed on the target table; The sub-execution node sends the bitmap record file corresponding to the second historical operation version identifier to the main execution node; The main execution node restores data of the target table according to the bitmap record file.

7. A data processing device based on a distributed database, characterized in that: The device comprises: A first acquisition module, configured for the main execution node to send the acquired operation request to the sub-execution node, wherein the operation request performs an operation on a target table; An operation module, used for the sub-execution node to send an operation execution result to the main execution node, wherein the operation execution result is obtained by operating the target table according to the operation request; A second acquisition module, configured to, when the operation execution result indicates successful execution, cause the main execution node to acquire an operation version identifier corresponding to the operation execution result, and send the operation version identifier to the sub-execution node; A generating module, used for the sub-execution node to generate a bitmap record file according to the operation version identifier and the operation record information related to the target table; An update module, configured for the main execution node to add the table name identifier of the target table, the operation version identifier and the operation request time to a flashback management table; a flashback module, used to perform flashback query and flashback recovery operations on the data in the target table according to the bitmap record file and the operation version identifier, wherein the flashback module includes a first flashback submodule, a second flashback submodule, a third flashback submodule and a fourth flashback submodule, A first flashback submodule, configured for the main execution node to obtain a flashback query request for the target table, wherein the flashback query request includes target table attributes and query conditions; A second flashback submodule is used for, when the target table attribute includes a flashback attribute, the main execution node sending a first historical operation version identifier to the sub-execution node, wherein the first historical operation version identifier is obtained by matching the flashback management table based on the query condition, and the first historical operation version identifier represents an identifier for operating the target table at a historical target time; A third flashback submodule, configured for the sub-execution node to send the bitmap record file corresponding to the first historical operation version identifier to the main execution node; The fourth flashback submodule is used for the main execution node to send the data query result corresponding to the bitmap record file to the client.

8. An electronic device, characterized in that: include: one or more processors; a memory for storing one or more instructions, Wherein, when the one or more instructions are executed by the one or more processors, the one or more processors are enabled to implement the method according to any one of claims 1 to 6.

Images