Data processing method, device and electronic device based on distributed database

Abstract

The embodiments of the present application provide a data processing method, device, electronic device and related products based on a distributed database, which relate to the field of distributed databases. The method is applied to any node in a distributed cluster, taking the current node as an example: after receiving a request carrying a first operation, the current state of the first operation and the hybrid timestamp of entering the current state of the first operation are determined; the target data row to be processed is determined according to the first operation; the processing method of the target data row is determined according to the hybrid timestamp of entering the current state and / or the committed hybrid timestamp of the target data row. Since each hybrid timestamp is determined by a hybrid clock including a physical clock and a logical clock, the hybrid clock or hybrid timestamp can effectively distinguish each transaction and each state in a concurrent multi-transaction scenario. Therefore, based on the hybrid timestamp with a clear distinguishing function, transaction consistency and transaction isolation can be guaranteed during the transaction processing process.

Description

Technical Field

[0001] The present application relates to the field of distributed databases. Specifically, the present application relates to a data processing method, device, electronic device, computer-readable storage medium, and computer program product based on a distributed database. Background Art

[0002] In the field of distributed data, how to coordinate transactions initiated by multiple terminals to ensure transaction consistency and isolation is a technical problem that urgently needs to be solved.

[0003] Currently, there are two ways to deal with this:

[0004] The first method is to use a global timestamp generation service in a distributed cluster to generate a timestamp for each transaction, and determine the execution order of transactions based on the timestamp, thereby achieving transaction consistency and isolation.

[0005] The second method is to use accurate GPS timing clocks (Global Positioning System timing service, Global Positioning System) or atomic clocks in distributed clusters to provide distributed clock services with small clock offsets, and obtain accurate time for each transaction through preset interfaces (such as truetime API). The execution order of transactions is determined based on the time sequence, thereby achieving transaction consistency and isolation.

[0006] However, these two methods may either bring additional time delays or require expensive and non-common hardware configurations, which are not ideal technical means. Summary of the Invention

[0007] The purpose of the embodiments of the present application is to provide a data processing method, device, and electronic device based on a distributed database to solve one of the above technical problems. To achieve this purpose, the embodiments of the present application provide the following solutions:

[0008] In one aspect, a data processing method based on a distributed database is provided, the method comprising:

[0009] Determine the current state of the first operation and the hybrid timestamp for entering the current state; wherein the hybrid timestamp is determined by a hybrid clock including a physical clock and a logical clock; determine the target data row to be processed according to the first operation; wherein each data row includes a commit hybrid timestamp recording the commit time and the recorded data, and the target data row belongs to at least one node; determine the processing method for the target data row according to the hybrid timestamp for entering the current state and / or the commit hybrid timestamp of the target data row.

[0010] Optionally, the current node is configured with a local physical clock for generating a physical timestamp and a hybrid clock for generating a first hybrid timestamp; determining the hybrid timestamp for entering a current state includes:

[0011] Obtain the current first hybrid timestamp and physical timestamp, and compare the values ​​of the physical timestamp and the physical clock part of the first hybrid timestamp; if the comparison result is equal, update the value of the logical clock part of the first hybrid timestamp in an auto-increment manner; if the comparison result is greater than, update the value of the physical clock part of the first hybrid timestamp to the physical timestamp, and update the value of the logical clock part of the first hybrid timestamp by assigning a zero value; determine that the hybrid timestamp entering the current state is the updated first hybrid timestamp.

[0012] Optionally, the first operation is a target commit operation; determining a processing method for the target data row according to the hybrid timestamp of entering the current state and / or the commit hybrid timestamp of the target data row includes:

[0013] If the target data row belongs to one node and the current state is the committed state, the processing method includes: performing an update operation on the target data row, and determining the second mixed timestamp of the committed state of the target commit operation as the committed mixed timestamp of the data row corresponding to the update operation; if the target data row belongs to at least two nodes and the current state is the pre-committed state, the processing method includes: determining the processing method for the target data row according to the third mixed timestamp of the pre-committed state of the target commit operation.

[0014] Optionally, determining a processing mode for the target data row according to a third hybrid timestamp of entering a pre-commit state of the target commit operation includes:

[0015] Obtain the third mixed timestamps of the pre-commit state of the target commit operation on at least two nodes, and filter out the largest commit timestamp; use the largest commit timestamp as the fourth mixed timestamp of the commit state of the target commit operation, and determine the processing method including: performing an update operation on the target data row, and determining the fourth mixed timestamp as the commit mixed timestamp of the data row corresponding to the update operation.

[0016] Optionally, if any request carrying the fifth mixed timestamp is received from a node connected to the user terminal, the method further includes:

[0017] If the fifth mixed timestamp is greater than the current first mixed timestamp, the current first mixed timestamp is updated to the fifth mixed timestamp.

[0018] Optionally, the hybrid timestamp further includes a corresponding state flag; the first operation is a target read operation, and the current state is a start state; determining a processing method for the target data row based on the hybrid timestamp of entering the current state and / or the commit hybrid timestamp of the target data row includes:

[0019] If the status flag corresponding to the commit hybrid timestamp of the target data row is an address flag, obtain the second operation of operating the target data row, and determine the processing method for the target data row based on the second operation and / or the sixth hybrid timestamp of entering the start state of the target read operation; if the status flag corresponding to the commit hybrid timestamp of the target data row is a time flag, determine the processing method for the target data row based on the sixth hybrid timestamp.

[0020] Optionally, determining a processing manner for the target data row according to the second operation and / or the sixth hybrid timestamp of entering the start state of the target read operation includes:

[0021] If the target read operation and the second operation are operations in the same transaction, the processing method includes: setting the data recorded in the target data row to be visible to the target read operation.

[0022] If the target read operation and the second operation are not operations in the same transaction, and if the second operation is a write operation or if the second operation is a commit operation in the start state, the processing method includes: setting the data recorded in the target data row to be invisible to the target read operation; if the second operation is a commit operation in the pre-commit state, the processing method includes: responding to the read signal, determining a processing method for the target data row based on the sixth hybrid timestamp. If the target read operation and the second operation are not operations in the same transaction, and if the second operation is a commit operation in the commit state, the processing method includes: determining a processing method for the target data row based on the sixth hybrid timestamp.

[0023] Optionally, determining a processing method for the target data row according to the sixth mixed timestamp includes:

[0024] Determine the difference between the sixth hybrid timestamp and the commit hybrid timestamp of the target data row; if the absolute value of the difference is not greater than a preset threshold, the processing method includes: restarting the target read operation to redetermine the processing method for the target data row; if the absolute value of the difference is greater than the preset threshold, and the difference is greater than zero, the processing method includes: setting the data recorded in the target data row to be visible to the target read operation; if the absolute value of the difference is greater than the preset threshold, and the difference is less than zero, the processing method includes: setting the data recorded in the target data row to be invisible to the target read operation.

[0025] Optionally, if the processing manner includes setting the data recorded in the target data row to be invisible to the target read operation, the method further includes:

[0026] Obtain a historical data row associated with the target data row; and set whether data recorded in the associated historical data row is visible to a target read operation according to the sixth hybrid timestamp and a commit hybrid timestamp of the associated historical data row.

[0027] Optionally, the method further includes:

[0028] Active operations on each node are counted, and the third operation with the smallest mixed timestamp for entering the start state of the corresponding operation is filtered out from the active operations; the mixed timestamp for entering the start state of the third operation is determined to be the seventh mixed timestamp; and historical data rows that meet the deletion criteria are deleted based on the seventh mixed timestamp.

[0029] On the other hand, an embodiment of the present application further provides a data processing device based on a distributed database, which is applied to a current node, where the current node is any node in a distributed cluster; the device includes:

[0030] The first determining module is configured to determine a current state of the first operation and a hybrid timestamp for entering the current state; wherein the hybrid timestamp is determined by a hybrid clock including a physical clock and a logical clock.

[0031] The second determination module is used to determine the target data row to be processed according to the first operation; wherein each data row includes a submission mixed timestamp recording the submission time and the recorded data, and the target data row belongs to at least one node.

[0032] The third determining module is configured to determine a processing method for the target data row according to the hybrid timestamp of entering the current state and / or the hybrid timestamp of committing the target data row.

[0033] On the other hand, an embodiment of the present application further provides an electronic device, comprising:

[0034] A memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of a data processing method based on a distributed database provided in an embodiment of the present application.

[0035] An embodiment of the present application also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of a data processing method based on a distributed database provided in an embodiment of the present application are implemented.

[0036] An embodiment of the present application also provides a computer program product, including a computer program, which, when executed by a processor, implements the steps of a data processing method based on a distributed database provided in an embodiment of the present application.

[0037] The beneficial effects of the technical solution provided by the embodiments of the present application are:

[0038] The embodiment of the present application provides a data processing method based on a distributed database, which is applied to any node in a distributed cluster. Taking the current node as an example, after receiving a request carrying a first operation, the current state of the first operation is determined, and a hybrid timestamp for entering the current state of the first operation is determined. Since the hybrid timestamp includes both a physical clock part and a logical clock part, even in the case of concurrent multiple operations, a different hybrid timestamp can still be assigned to the current state of each operation, and then each operation can be effectively distinguished by the assigned hybrid timestamp. Based on the hybrid timestamp with a clear distinguishing function, the read operation, write operation, commit operation and data row in the transaction processing process can be effectively distinguished, providing a feasibility basis for ensuring transaction consistency and isolation. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for describing the embodiments of the present application.

[0040] Figure 1 A flowchart of a data processing method based on a distributed database provided in an embodiment of the present application;

[0041] Figure 2a A schematic diagram of the structure of a hybrid timestamp provided in an embodiment of the present application;

[0042] Figure 2b A schematic diagram of the structure of a data row provided in an embodiment of the present application;

[0043] Figure 2c A flowchart of a two-stage submission process provided in an embodiment of the present application;

[0044] Figure 3a A schematic diagram of the structure of a data processing device based on a distributed database provided in an embodiment of the present application;

[0045] Figure 3b A schematic diagram of the structure of a data processing device based on a distributed database provided in an embodiment of the present application;

[0046] Figure 4 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0047] The following describes the embodiments of the present application in conjunction with the accompanying drawings. It should be understood that the embodiments described below in conjunction with the accompanying drawings are exemplary descriptions for explaining the technical solutions of the embodiments of the present application and do not constitute a limitation on the technical solutions of the embodiments of the present application.

[0048] Those skilled in the art will understand that, unless otherwise stated, the singular forms "a", "an", "said", and "the" used herein may also include plural forms. It should be further understood that the terms "including" and "comprising" used in the embodiments of the present application mean that the corresponding features can be implemented as the presented features, information, data, steps, operations, elements, and / or components, but do not exclude implementation as other features, information, data, steps, operations, elements, components, and / or combinations thereof supported by the present technical field. It should be understood that when we say that an element is "connected" or "coupled" to another element, the element can be directly connected or coupled to the other element, or it can refer to the element and the other element establishing a connection relationship through an intermediate element. In addition, the "connection" or "coupling" used here can include wireless connection or wireless coupling. The term "and / or" used here indicates at least one of the items defined by the term, for example, "A and / or B" can be implemented as "A", or as "B", or as "A and B".

[0049] In order to make the objectives, technical solutions and advantages of this application clearer, the implementation methods of this application will be further described in detail below with reference to the accompanying drawings.

[0050] In the field of distributed data, transaction management has always been a hot topic. Version control mechanisms are often used to coordinate transactions. Furthermore, when the amount of data to be processed reaches a certain level, multiple physical servers are used for distributed data storage (each physical server node corresponds to a database). One way to store data is in rows.

[0051] The operations on data rows on the physical server may specifically include: adding data rows, modifying data rows, deleting data rows, and querying data rows. The operations of adding / modifying / deleting data rows are understood as write operations on data rows, and querying data rows are understood as read operations on data rows.

[0052] Transactional consistency: After one or more transactions, previously consistent data and the database remain consistent. For example, after performing a write operation on multiple data rows in a distributed node, if the logic of the rows is consistent, transactional consistency is maintained. If the logic of the rows is inconsistent, transactional consistency is not maintained.

[0053] Transaction isolation: When multiple users access a database, such as operating on the same table, the transaction opened for each user cannot be interfered with by other transactions. Multiple concurrent transactions must be isolated from each other. For example, if transaction A, which includes a read operation, and transaction B, which includes a write operation, are running concurrently on the current node, and a data row being read by transaction A is being processed by transaction B's write operation, if the data row is unreadable by transaction A, then isolation exists between transactions A and B. If the data row is readable by transaction A, then isolation does not exist between transactions A and B.

[0054] Therefore, when performing transaction management, how to maintain transaction consistency and isolation becomes a technical problem that needs to be solved urgently.

[0055] Currently, there are two solutions for achieving transaction consistency and isolation: the first approach and the second approach. As mentioned in the background, existing solutions still have at least the following problems:

[0056] (1) For the first method, using a global timestamp will result in one transaction causing two network interactions to obtain timestamps from the global timestamp generation service. If the transaction is only performed on the local node, i.e., it is not a cross-node distributed transaction, it is still necessary to obtain timestamps from the global timestamp generation service through network interactions, which will bring additional delays.

[0057] (2) Regarding the second approach, physical clocks using customized hardware, such as GPS and atomic clocks (expensive and non-common hardware configurations), are not suitable for distributed databases built with general-purpose hardware.

[0058] Therefore, how to achieve transaction consistency and isolation based on existing conditions (such as a distributed database built by physical servers equipped with ordinary hardware) without sacrificing operation latency is still a technical problem that needs to be improved. In order to solve at least one problem in the existing solution, the embodiment of the present application provides a data processing method, device and electronic device based on a distributed database. The solution is based on a hybrid logical clock and a distributed transaction two-phase commit method. Through this solution, it is possible to coordinate the processing process of each operation in each transaction on the basis of existing hardware, thereby achieving the purpose of maintaining transaction consistency and isolation.

[0059] The data processing method based on a distributed database provided in the embodiment of the present application can be executed by any electronic device in a device cluster, such as can be executed by a physical server in a distributed cluster, and there are at least two physical servers in the distributed cluster. Optionally, the physical server can provide various cloud services, such as cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), and basic cloud computing services such as big data and artificial intelligence platforms. In the embodiment of the present application, the physical servers in the distributed cluster can be understood as nodes of the distributed cluster, that is, each server is a node. Optionally, the user terminal that interacts with the distributed cluster can be any electronic device, such as a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart wearable device, and a car terminal.

[0060] It should be noted that the data rows on each physical server can be data rows in various tables. For example, in banking services, this includes a customer account table, a customer information table, and an information table for each account. Alternatively, in product services, this includes an inventory table for region A, a shipment table for region A, a return table for region A, an inventory table for region B, a shipment table for region B, and a return table for region B. This application does not limit this.

[0061] Among them, in the version mechanism, the physical server also sets the isolation level of the transaction. For example, the RR (repeated-read) isolation level: based on the RR isolation level, the time to enter the start state of each read / write operation is the time to enter the start state of the corresponding operation. The RC (read-committed) isolation level: based on the RR isolation level, the time to enter the start state of each read / write operation is the time to enter the start state of the corresponding transaction. In the prior art, there are other transaction isolation level settings, and the specific time to enter the start state of each operation, or the specific time to enter other states, can be set according to the corresponding isolation level. For the sake of simplicity, it will not be repeated here.

[0062] In order to better understand the relationship between isolation levels and transaction operations, the embodiment of the present application also provides an example of transaction t1 for detailed explanation using RR isolation level and RC isolation level.

[0063] begin; / / Start transaction;

[0064] select * from accout; / / first read operation;

[0065] update account set balance = balance + 10 where account_id = 1; / / first write operation;

[0066] select * from accout; / / second read operation;

[0067] commit; / / Submit operation.

[0068] Transaction t1 includes five SQL statements, each of which is sent sequentially from the user terminal to the corresponding physical server. Next, we determine the operations that a physical server can perform based on the transaction isolation level settings and the SQL statements received by the physical server.

[0069] When the physical server receives the "begin;" statement: The type of transaction t1 cannot be determined.

[0070] When the physical server receives the first read operation, it determines the start time of the first read operation as start_time1 based on the RR level and the start time of the first read operation as start_time2 based on the RC level. It can be determined that the type of transaction t1 is a read transaction.

[0071] When the physical server receives the first write operation: the time to enter the start state of the first write operation is determined to be start_time1 based on the RR level, and the time to enter the start state of the first write operation is determined to be start_time3 based on the RC level; it can be determined that the type of transaction t1 is a read-write transaction.

[0072] When the physical server receives the second read operation: based on the RR level, the time to enter the start state of the second read operation is determined to be start_time1, and based on the RC level, the time to enter the start state of the second read operation is determined to be start_time4; it can be determined that the type of transaction t1 is still a read-write transaction.

[0073] When the physical server receives the "commit;" statement, it commits the data for the first write operation. Since the first write operation involves only one data row, the commit time (start_time5) can be directly determined. If the first write operation involves multiple data rows on different nodes, the start_time5 must be negotiated among the nodes.

[0074] In addition, there may also be transactions t2 and t3.

[0075] Transaction t2 is a read transaction, specifically:

[0076] begin; select * from accout; / / read operation

[0077] commit;

[0078] Transaction t3 is a write transaction, specifically:

[0079] begin;

[0080] update account set balance = balance + 10 where account_id = 1; / / write operation

[0081] commit;

[0082] The following describes several exemplary embodiments to illustrate the technical solutions of the embodiments of the present application and the technical effects produced by the technical solutions of the present application. It should be noted that the following embodiments can refer to, draw on, or combine with each other, and the same terms, similar features, and similar implementation steps in different embodiments will not be repeated.

[0083] The present application provides a data processing method based on a distributed database, which is applied to a current node, which is any node in a distributed cluster. Figure 1 As shown, the method includes steps S110 to S130.

[0084] S110, determining a current state of the first operation and a hybrid timestamp of entering the current state; wherein the hybrid timestamp is determined by a hybrid clock including a physical clock and a logical clock;

[0085] Optionally, any node in the distributed cluster can establish a connection with the user terminal to process the first operation. The target data row can be a data row on any node in the distributed cluster.

[0086] The hybrid timestamp is determined by a hybrid clock including a physical clock and a logical clock, and includes the numerical value of the physical clock part and the numerical value of the logical clock part respectively. Optionally, the unit of the numerical value of the physical clock part in the hybrid timestamp can be milliseconds; the numerical value of the logical clock part in the hybrid timestamp is an ordinary numerical value, which can be a numerical value in the range of non-negative integers. Optionally, each hybrid timestamp also corresponds to a status flag, and when stored, the hybrid timestamp is stored together with the corresponding status flag. If the data row is in a static state (e.g., stored in a hard disk and in a non-operating state), the status flag is a time flag, and the hybrid timestamp is essentially time data; if the data row is in a non-static state (e.g., stored in a dynamic memory and the data row is in an operating state), the status flag is an address flag, and the hybrid timestamp is essentially an address pointer.

[0087] like Figure 2a A specific storage example of a hybrid timestamp is shown. The stored hybrid timestamp includes three parts, namely: a status flag with a length of 2 bits, which is used to store the status flag of the hybrid timestamp; a physical clock part with a length of 46 bits, which is used to store the value of the physical clock part of the hybrid clock; and a logical clock part with a length of 16 bits, which is used to store the value of the logical clock part of the hybrid clock.

[0088] Optionally, the current node is configured with a local physical clock for generating a physical timestamp and a hybrid clock for generating a first hybrid timestamp, wherein each other node in the distributed cluster also has the same clock configuration.

[0089] Optionally, when initializing the current node, an empty hybrid timestamp is created (both the physical clock portion and the logical clock portion are null). The current physical timestamp is obtained and assigned to the physical clock portion of the empty hybrid timestamp, while the logical clock portion of the empty hybrid timestamp is assigned a value of zero. The empty hybrid timestamp, after the assignment process, is used as the first hybrid timestamp. This first hybrid timestamp may also be updated in subsequent processes.

[0090] Optionally, the local physical clock can be a clock supported by a hardware device configured on the current node or a configured clock service, such as a system clock supported by a quartz crystal resonator. Among them, a quartz crystal oscillator is a quartz crystal resonator made of quartz material. When installed on an electronic device, it can provide a stable frequency for the electronic device, and the unit time of the system clock is derived from this frequency. Quartz crystal oscillators have the characteristics of anti-interference and good performance, and are widely used in various electronic devices, such as physical servers as nodes in distributed clusters. Compared with other clocks supported by hardware devices (such as atomic clocks) or configured clock services (such as GPS timing services), the system clock supported by a quartz crystal oscillator appears to be more economical because quartz crystal oscillators are more affordable. It should be noted that the local physical clock can also be derived from other economical clock services.

[0091] Optionally, when determining the current state of the first operation, a hybrid timestamp for entering the current state may be determined according to the following steps Sa1 to Sa3.

[0092] Sa1, obtain the current first hybrid timestamp and physical timestamp, and compare the values ​​of the physical timestamp and the physical clock part of the first hybrid timestamp.

[0093] Specifically, when determining the state of the first operation, a first hybrid timestamp and a physical timestamp of the current node at the current moment are obtained, and the values ​​of the physical timestamp and the physical clock part of the first hybrid timestamp are compared.

[0094] Sa2: Update the first mixed timestamp according to different comparison results.

[0095] If the comparison result is equal, the value of the logical clock part of the first hybrid timestamp is updated in an auto-incremental manner; and the hybrid timestamp entering the current state is determined to be the updated first hybrid timestamp.

[0096] Specifically, based on the value of the existing logical clock part of the first hybrid timestamp, a preset value is added. Optionally, the preset value can be any value, such as value 1.

[0097] For example, the current node receives 10 concurrent operations and needs to determine the hybrid timestamps for entering the current state of each of the 10 operations. The 10 operations include: Operation 1, Operation 2, ..., Operation 10, and the value of the physical clock portion of the first hybrid timestamp is A, and the value of the logical clock portion is B. When determining the hybrid timestamp for the current state of each operation, the value of the logical clock portion is determined by incrementing by 1. Specifically, the logical value corresponding to the current state of Operation 1 is B+1, the logical value corresponding to the current state of Operation 2 is B+2, ..., and the logical value corresponding to the current state of Operation 10 is B+10. In other words, the hybrid timestamp for entering the current state of Operation 1 is determined by A and (B+1), the hybrid timestamp for entering the current state of Operation 2 is determined by A and (B+2), and the hybrid timestamp for entering the current state of Operation 10 is determined by A and (B+10). Therefore, even in a state of concurrent multiple operations, the hybrid timestamps for entering the current state of each operation are different.

[0098] If the comparison result is greater than, the value of the physical clock part of the first hybrid timestamp is updated to the physical timestamp, and the value of the logical clock part of the first hybrid timestamp is updated by assigning a zero value; the hybrid timestamp entering the current state is determined to be the updated first hybrid timestamp.

[0099] Specifically, the existing physical clock part of the first hybrid timestamp is replaced with the physical timestamp, and a zero value is assigned to the existing logical clock part of the first hybrid timestamp to obtain an updated first hybrid timestamp.

[0100] It is understood that at the current moment, the updated first hybrid timestamp is the latest first hybrid timestamp. For example, the method for obtaining the latest first hybrid timestamp can be set to obtain it through a preset interface, such as currentHLC(). Specifically, when the current node calls currentHLC(), it can obtain the latest first hybrid timestamp at the current moment.

[0101] In the prior art, if within a time period (e.g., within a unit time of updating the physical clock), the current node receives multiple requests carrying specific operations, that is, receives concurrent operations, if a timestamp is assigned to the current state of each operation according to the physical timestamp, a consistent timestamp will be assigned, resulting in little difference in the current state of each operation. The solution provided in the embodiment of the present application can assign different mixed timestamps to the current state of each operation by adjusting the value of the logical clock part, thereby effectively distinguishing the current state of each operation. Distinguishing the current state of each operation provides a feasibility basis for ensuring transaction consistency and isolation when performing read operations, write operations, or commit operations.

[0102] S120, determining a target data row to be processed according to the first operation; wherein each data row includes a commit hybrid timestamp recording a commit time and recorded data, and the target data row belongs to at least one node.

[0103] Specifically, the target data row can be understood as the data row that needs to be processed by the first operation. The data rows stored in each node are divided into two categories: historical data rows and active data rows. The first operation carries a unique data identifier that characterizes the characteristics of the data row. A relationship is established between the historical data row and the active data row through the data identifier that characterizes the characteristics of the data row. The historical data row records the historical data corresponding to the same data identifier, while the active data row records the latest data corresponding to the same data identifier. The data identifier that characterizes the characteristics of the data row carried by the first operation is mainly used to obtain the active data row corresponding to the data identifier, unless otherwise specified.

[0104] The commit hybrid timestamp of the record commit time is the commit timestamp when the data row is committed.

[0105] like Figure 2b In the example of three data rows shown, each data row consists of three parts: a mixed timestamp for that row (corresponding to the submitted mixed timestamps described above, such as mixed timestamp 1, mixed timestamp 2, and mixed timestamp 3), which is 8 bytes long; an identifier pointing to the previous historical data row for that data row, which is 4 bytes long (such as "201" in "row 200," indicating that the previous historical data row for "row 200" is "row 201"); and the data recorded in that row, whose length and content can be customized. The historical data row for the data row at the position identified by "row 200" is at the position identified by "row 201," and the historical data row for the data row at the position identified by "row 201" is at the position identified by "row 202." All three data rows correspond to the same data identifier.

[0106] S130 : Determine a processing method for the target data row according to the hybrid timestamp of entering the current state and / or the hybrid timestamp of committing the target data row.

[0107] The embodiment of the present application provides a data processing method based on a distributed database, which is applied to any node in a distributed cluster. Taking the current node as an example, after receiving a request carrying a first operation, the current state of the first operation is determined, and a hybrid timestamp for entering the current state of the first operation is determined. Since the hybrid timestamp includes both a physical clock part and a logical clock part, even in the case of concurrent multiple operations, a different hybrid timestamp can still be assigned to the current state of each operation, and then each operation can be effectively distinguished by the assigned hybrid timestamp. Based on the hybrid timestamp with a clear distinguishing function, the read operation, write operation, commit operation and data row in the transaction processing process can be effectively distinguished, providing a feasibility basis for ensuring transaction consistency and isolation.

[0108] In the above embodiment, the type of request carrying the first operation is exemplified, but different processing methods are provided for different operations and current states of transactions. Next, the first operation will be described in detail based on whether it is a read operation or a commit operation.

[0109] In an optional embodiment, the first operation is a target commit operation. The current state of the target commit operation can be the start state of the target commit operation, the commit state, or the pre-commit state. Furthermore, the number of nodes to which the target data row belongs also affects transaction processing. Therefore, step S130 can include the following steps Sb1 and S2.

[0110] Sb1, if the target data row belongs to a node and the current state is the committed state, the processing method includes: performing an update operation on the target data row, and determining the second mixed timestamp of the committed state of entering the target commit operation as the committed mixed timestamp of the data row corresponding to the update operation.

[0111] Specifically, after determining that the target data row belongs to a node, the current node begins to perform a commit operation: first, the current node determines the second hybrid timestamp of entering the commit state of the target commit operation; second, the current node notifies the node to which the target data row belongs to commit the updated content for the target data row, and determines the second hybrid timestamp of entering the commit state as the commit timestamp corresponding to the updated content.

[0112] Optionally, there are many ways to determine the node to which the target data row belongs, including: determining the node to which the target data row belongs based on the data identifier of the target data row carried in the target commit operation, the table identifier of the table to which the active data row or historical data row corresponding to the data identifier belongs, and the node to which the table carrying the table identifier belongs. It should be noted that there are many ways to determine the node to which the target data row belongs, and for the sake of simplicity, they are not listed here one by one.

[0113] Sb2: If the target data row belongs to at least two nodes and the current state is a pre-commit state, the processing method includes: determining the processing method for the target data row according to the third hybrid timestamp of entering the pre-commit state of the target commit operation.

[0114] Optionally, the at least two nodes may include the current node, or may not include the current node.

[0115] Specifically, Sb2 may include: obtaining third mixed timestamps of entering a pre-commit state of the target commit operation on at least two nodes, and screening out the largest commit timestamp therefrom;

[0116] The maximum commit timestamp is used as the fourth hybrid timestamp for entering the commit state of the target commit operation, and the processing method includes: performing an update operation on the target data row, and determining the fourth hybrid timestamp as the commit hybrid timestamp of the data row corresponding to the update operation.

[0117] Specifically, the current node sends a pre-submission statistics request to other nodes that are not the current node among the at least two nodes; after receiving the pre-submission statistics request, any other node determines a third hybrid timestamp for entering the pre-submission state of the target submission operation and feeds the hybrid timestamp back to the current node as feedback data; after receiving the feedback data sent by the other nodes, the current node determines the third hybrid timestamp for entering the pre-submission state of the target submission operation on each of the at least two nodes.

[0118] Next, the current node selects the largest commit timestamp from the at least two third hybrid timestamps of the at least two nodes that entered the pre-commit state of the target commit operation, and uses the largest commit timestamp as the fourth hybrid timestamp for entering the commit state of the target commit operation. The current node also sends a second commit request carrying the fourth hybrid timestamp to the other node among the at least two nodes, excluding the current node. After receiving the second commit request, the other node commits the updated content and determines the fourth hybrid timestamp as the commit timestamp corresponding to the updated content.

[0119] It should be noted that when any node performs a commit operation, it must pay attention to the following two aspects: On the one hand, the node needs to set the active data row corresponding to the data identifier carried by the target commit operation as the historical data row, that is, set the target data row as the historical data row. On the other hand, the node needs to create a new active data row (that is, the data row corresponding to the updated content) for the updated content based on the data identifier carried by the target commit operation, and use the commit timestamp corresponding to the updated content as the commit mixed timestamp of the new active data row. Among them, if the write operation is an operation to delete a data row, then there is no need to create an active data row.

[0120] Optionally, before sending the pre-submission statistics request, an update request is sent to another of the at least two nodes, the update request carrying the eighth hybrid timestamp of entering the start state of the target submission operation. After receiving the update request, the other node determines the hybrid timestamp carried in the update request as the eighth hybrid timestamp of entering the start state of the target submission operation. In other words, this process unifies the eighth hybrid timestamp of entering the start state of the target submission operation for the at least two nodes.

[0121] In an optional embodiment, if any request carrying the fifth mixed timestamp is received from a node connected to the user terminal, the method further includes:

[0122] If the fifth mixed timestamp is greater than the current first mixed timestamp, the current first mixed timestamp is updated to the fifth mixed timestamp.

[0123] For example, when the current node receives the second commit request carrying the fourth hybrid timestamp, it needs to compare the fifth hybrid timestamp with the current first hybrid timestamp. If the comparison result is not greater than, no processing is required on the current first hybrid timestamp. If the comparison result is less than, processing is required on the current first hybrid timestamp.

[0124] When performing transaction processing on a node, the first hybrid timestamps of some nodes may lag behind those of other nodes. In this case, the first hybrid timestamps of these nodes need to be updated to unify the first hybrid timestamps of each node.

[0125] In order to more clearly understand how the current node performs a commit operation based on a write operation, the embodiment of the present application also provides a specific implementation example. The background of this example is: Based on the user terminal, Xiao Ming transfers 100 yuan to Xiao Hong. Then the user terminal needs to cooperate with the current node to perform the following operations: update Xiao Ming's account to "decrease 100 yuan" and update Xiao Hong's account to "increase 100 yuan". The user terminal initiates a transaction t4, the specific content of transaction t4 is as follows:

[0126] begin;

[0127] updateaccountbalance=balance-100wherename='Xiaoming';

[0128] update account balance = balance + 100 where name = 'Xiaohong';

[0129] commit;

[0130] The fields 'Xiaoming' and 'Xiaohong' in this key statement can be understood as data identifiers that characterize the data row in the target submission operation. If it is determined that Xiaoming and Xiaohong's accounts are both on Node 1 (corresponding to the scenario in the above embodiment where the target data row belongs to a single node), before the transfer operation occurs, the details of Xiaoming and Xiaohong's accounts on Node 1 are shown in Table 1 below.

[0131] Table 1

[0132]

[0133] In Table 1, the row identifiers correspond to the actual row positions of the data rows in the table, e.g., "row 100" is the 100th row of the corresponding table, and "row 10" is the 10th row of the corresponding table. The commit hybrid timestamps shown in the table are only exemplary values; larger values ​​indicate larger hybrid timestamps. For example, the timestamp corresponding to 110005 is larger than the timestamp corresponding to 110000. Data rows with row identifiers "row 10" and "row 9" are historical data rows, while data rows with row identifiers "row 100" and "row 110" are active data rows.

[0134] When the current node receives the above key statement, it performs the corresponding operation to update the account data of Xiaoming and Xiaohong. After the update, two new data rows are added to Node 1, namely "Row 210" and "Row 211", as shown in Table 2 below.

[0135] Table 2

[0136]

[0137] If it is determined that Xiaoming and Xiaohong's accounts are on nodes 1 and 2 (which corresponds to the situation in the above embodiment where the target data row belongs to at least two nodes), it is necessary to coordinate how to submit the update between nodes 1 and 2. The update process can be referred to Figure 2c The process shown includes steps S1001 to S1003. In this example, node 0 is used as the coordinator and is used to identify any node in the distributed cluster, and node 1 and node 2 are used as participants of the target submission operation.

[0138] S1001: The coordinator and participants of the target submission operation unify the time for entering the start state of the target submission operation.

[0139] Specifically, the coordinator obtains the time ts1 of entering the start state of the target submission operation and sends an update request carrying ts1 to the participants; after receiving ts1, each participant updates the time of entering the start state of the target submission operation to ts1.

[0140] Wherein, after the update, it is determined that the participant sends a feedback of the successful update to the coordinator.

[0141] S1002: The coordinator initiates pre-commit statistics.

[0142] Specifically, the coordinator sends a pre-commit statistics request to each participant to obtain the time ts2 of entering the pre-commit state for the target commit operation. The participant determines ts2 and reports ts2 back to the coordinator. After counting multiple ts2 (two in this example), the largest ts2 is determined from the multiple ts2 and used as ts3 for entering the commit state for the target commit operation.

[0143] S1003, the participant performs submission operation according to ts3.

[0144] After determining ts3, the coordinator sends a second commit request with ts3 to each participant, urging them to commit their updates according to ts3. Specifically, the participant lists the current active data row as a historical data row, creates a new active data row for the updated content, and uses ts3 as the commit hybrid timestamp of the new active data row.

[0145] It should be noted that ts1 may correspond to the eighth mixed timestamp for entering the start state of the target submission operation in the above embodiment, ts2 may correspond to the third mixed timestamp in the above embodiment, and ts3 may correspond to the fourth mixed timestamp in the above embodiment.

[0146] S1001 to S1003 are the submission process for a target submission operation involving at least two participants, which can also be called a two-phase submission process. The submission process for a target submission operation involving one participant can also be called a one-phase submission process. In the one-phase submission process, the submission operation is directly performed by a single participant based on the second hybrid timestamp determined by the coordinator to enter the submission state of the target submission operation.

[0147] Next, the first operation, which is a read operation, will be described in detail.

[0148] In an optional embodiment, the first operation is a target read operation. Each hybrid timestamp also includes a corresponding status flag. For example, the commit hybrid timestamp of the target data row includes a corresponding status flag. The current state upon entering the target read operation is the starting state of the target read operation. In this embodiment, S130 may specifically include either step Sb3 or step Sb4.

[0149] In step Sb3, if the status flag corresponding to the commit hybrid timestamp of the target data row is an address flag, a second operation for operating the target data row is obtained, and a processing method for the target data row is determined based on the second operation and / or the sixth hybrid timestamp of the start state of the target read operation. The process of determining the processing method includes the following steps Sb3-1 and Sb3-2.

[0150] Sb3-1, when the target read operation and the second operation are operations in the same transaction, the processing method includes: setting the data recorded in the target data row to be visible to the target read operation.

[0151] Specifically, if the target read operation and the second operation are operations in the same transaction, any operation in the transaction corresponding to the second operation operating the target data row can access the target data row. Taking the above transaction ts1 as an example, the second operation can correspond to the first write operation, and the first operation can correspond to the second read operation.

[0152] The second operation may be a write operation or a commit operation.

[0153] Sb3-2: When the target read operation and the second operation are not operations in the same transaction, a judgment may be made based on the type and / or status of the second operation.

[0154] Optionally, if the second operation is a write operation or if the second operation is a commit operation in a start state, the processing method includes: setting the data recorded in the target data row to be invisible to the target read operation.

[0155] Optionally, if the second operation is a commit operation in a pre-commit state, the processing method includes: determining, in response to the read signal, a processing method for the target data row based on the sixth hybrid timestamp. Specifically, if the commit operation is determined to be in the pre-commit state, registering a read signal for the target read operation, and sending a read signal once the target data row changes from a locked state to an unlocked state; after receiving the read signal, the current node may continue to determine a processing method for the target data row based on the sixth hybrid timestamp.

[0156] Optionally, if the second operation is a commit operation in a committed state, the processing method includes: determining a processing method for the target data row according to the sixth hybrid timestamp.

[0157] Optionally, during the transaction initialization phase, the node assigns a unique number to each transaction. A transaction can include multiple operations, such as read operations, write operations, and commit operations. The transaction number can be used to determine whether any two operations belong to the same transaction.

[0158] Sb4: If the status flag corresponding to the committed hybrid timestamp of the target data row is a time flag, determine a processing method for the target data row according to the sixth hybrid timestamp.

[0159] Specifically, the difference between the sixth hybrid timestamp and the target data row's commit hybrid timestamp is determined, and then the processing method is determined based on the relationship between the difference and a preset threshold. The judgment result is as follows:

[0160] Optionally, if the absolute value of the difference is not greater than a preset threshold, the processing method includes: restarting the target read operation to redetermine a processing method for the target data row;

[0161] Optionally, if the absolute value of the difference is greater than a preset threshold, and the difference is greater than zero, the processing method includes: setting the data recorded in the target data row to be visible to the target read operation;

[0162] Optionally, if the absolute value of the difference is greater than a preset threshold and the difference is less than zero, the processing method includes: setting the data recorded in the target data row to be invisible to the target read operation.

[0163] Optionally, the preset threshold may be understood as an error value set based on experience, and thus, the error value may be a time in milliseconds, such as 1 ms.

[0164] In the above embodiment, a situation occurs where the target data row is not visible to the target read operation. How to respond to the target read operation is also an important issue in the transaction management process.

[0165] In an optional embodiment, if the processing method includes setting the data recorded in the target data row to be invisible to the first operation, the method further includes:

[0166] Obtain a historical data row associated with a target data row; and set whether data recorded in the associated historical data row is visible to a target read operation according to the sixth hybrid timestamp and a commit hybrid timestamp of the associated historical data row.

[0167] Specifically, the target data row can be associated with multiple historical data rows, each of which contains a row identifier that records the historical data row corresponding to the previous data version. The previous historical data row associated with the target data row is determined based on the row identifier recorded in the target data row, and the visibility of the data in that historical data row is determined. If the data in that historical data row is not visible, the data rows associated with that historical data row are retrieved and the visibility is determined again, until the last historical data row is reached.

[0168] In other words, if the target data row is determined to be unreadable, the system can check the data in chronological order, from the latest to the oldest data version. If a historical data row is found to be visible, the data for that historical data row is returned. This means that even if the active data row is determined to be unavailable, the visibility of the historical data row can still be determined, solving the problem of how to respond to the target read operation.

[0169] Due to the continuous writing and committing operations, a lot of redundant data is accumulated in the node, which is a waste of the node's storage resources.

[0170] In an optional embodiment, the method further includes:

[0171] Count the active operations on each node, and filter out the third operation with the smallest mixed timestamp of entering the start state from the active operations; determine the mixed timestamp of entering the start state of the third operation as the seventh mixed timestamp; delete the historical data rows that meet the deletion conditions according to the seventh mixed timestamp.

[0172] The historical data rows that meet the deletion condition for the third operation are deleted according to the seventh mixed timestamp. Specifically, the following operations may be performed for each group of data rows with the same data identifier on each node:

[0173] Filter out data rows from the group of data rows whose corresponding commit mixed timestamps are less than the seventh mixed timestamp; determine whether the data of the filtered data rows is visible to the third operation based on the seventh mixed timestamp and the commit mixed timestamp of each filtered data row; if not, delete the filtered data rows.

[0174] The embodiment of the present application also provides a data processing device based on a distributed database, which is applied to a current node, which is any node in a distributed cluster. Figure 3a As shown, the device 300 includes:

[0175] The first determining module 310 is configured to determine a current state of the first operation and a hybrid timestamp for entering the current state; wherein the hybrid timestamp is determined by a hybrid clock including a physical clock and a logical clock.

[0176] The second determination module 320 is configured to determine a target data row to be processed according to the first operation; wherein each data row includes a commit hybrid timestamp recording a commit time and recorded data, and the target data row belongs to at least one node.

[0177] The third determining module 330 is configured to determine a processing method for the target data row according to the hybrid timestamp of entering the current state and / or the hybrid timestamp of committing the target data row.

[0178] In an optional embodiment, the current node is configured with a local physical clock for generating a physical timestamp and a hybrid clock for generating a first hybrid timestamp; the first determination module 310, in determining the hybrid timestamp for entering the current state, is specifically configured to:

[0179] Obtain the current first hybrid timestamp and physical timestamp, and compare the values ​​of the physical timestamp and the physical clock part of the first hybrid timestamp; if the comparison result is equal, update the value of the logical clock part of the first hybrid timestamp in an auto-increment manner; if the comparison result is greater than, update the value of the physical clock part of the first hybrid timestamp to the physical timestamp, and update the value of the logical clock part of the first hybrid timestamp by assigning a zero value; determine that the hybrid timestamp entering the current state is the updated first hybrid timestamp.

[0180] In an optional embodiment, the first operation is a target commit operation; the third determination module 330, in determining the processing method for the target data row according to the hybrid timestamp of entering the current state and / or the hybrid commit timestamp of the target data row, is specifically configured to:

[0181] If the target data row belongs to a node and is currently in a committed state, the processing method includes: performing an update operation on the target data row, and determining the second hybrid timestamp of entering the committed state of the target commit operation as the committed hybrid timestamp of the data row corresponding to the update operation;

[0182] If the target data row belongs to at least two nodes and is currently in a pre-commit state, the processing method includes: determining a processing method for the target data row according to a third hybrid timestamp of entering the pre-commit state of the target commit operation.

[0183] Optionally, the third determination module 330, in determining the processing mode for the target data row according to the third hybrid timestamp of entering the pre-commit state of the target commit operation, is specifically configured to:

[0184] Obtain the third mixed timestamps of the pre-commit state of the target commit operation on at least two nodes, and filter out the largest commit timestamp; use the largest commit timestamp as the fourth mixed timestamp of the commit state of the target commit operation, and determine the processing method including: performing an update operation on the target data row, and determining the fourth mixed timestamp as the commit mixed timestamp of the data row corresponding to the update operation.

[0185] Alternatively, as Figure 3b The apparatus 300 further includes an updating module 340. Upon receiving any request carrying the fifth hybrid timestamp sent by a node connected to the user terminal, the updating module 340 is further configured to:

[0186] If the fifth mixed timestamp is greater than the current first mixed timestamp, the current first mixed timestamp is updated to the fifth mixed timestamp.

[0187] In an optional embodiment, the hybrid timestamp further includes a corresponding state flag; the first operation is a target read operation, and the current state is a start state; the third determination module 330, in determining a processing method for the target data row based on the hybrid timestamp for entering the current state and / or the commit hybrid timestamp of the target data row, is specifically configured to:

[0188] If the status flag corresponding to the commit hybrid timestamp of the target data row is an address flag, obtain the second operation of operating the target data row, and determine the processing method for the target data row based on the second operation and / or the sixth hybrid timestamp of entering the start state of the target read operation; if the status flag corresponding to the commit hybrid timestamp of the target data row is a time flag, determine the processing method for the target data row based on the sixth hybrid timestamp.

[0189] Optionally, the third determining module 330, in determining the processing manner for the target data row according to the second operation and / or the sixth hybrid timestamp of entering the start state of the target read operation, is specifically configured to:

[0190] When the target read operation and the second operation are operations in the same transaction, the processing manner includes: setting the data recorded in the target data row to be visible to the target read operation.

[0191] When the target read operation and the second operation are not in the same transaction:

[0192] If the second operation is a write operation or if the second operation is a commit operation in a start state, the processing method includes: setting the data recorded in the target data row to be invisible to the target read operation; and if the second operation is a commit operation in a pre-commit state, the processing method includes: in response to a read signal, determining a processing method for the target data row according to the sixth hybrid timestamp; and if the second operation is a commit operation in a committed state, the processing method includes: determining a processing method for the target data row according to the sixth hybrid timestamp.

[0193] Optionally, the third determining module 330, in determining a processing method for the target data row according to the sixth hybrid timestamp, is specifically configured to:

[0194] Determine the difference between the sixth hybrid timestamp and the commit hybrid timestamp of the target data row; if the absolute value of the difference is not greater than a preset threshold, the processing method includes: restarting the target read operation to redetermine the processing method for the target data row; if the absolute value of the difference is greater than the preset threshold, and the difference is greater than zero, the processing method includes: setting the data recorded in the target data row to be visible to the target read operation; if the absolute value of the difference is greater than the preset threshold, and the difference is less than zero, the processing method includes: setting the data recorded in the target data row to be invisible to the target read operation.

[0195] Optionally, if the processing method includes setting the data recorded in the target data row to be invisible to the target read operation, the third determining module 330 may further be configured to:

[0196] Obtain a historical data row associated with a target data row; and set whether data recorded in the associated historical data row is visible to a target read operation according to the sixth hybrid timestamp and a commit hybrid timestamp of the associated historical data row.

[0197] Alternatively, as Figure 3b The device further includes a deletion module 350, which can be used to:

[0198] Active operations on each node are counted, and the third operation with the smallest mixed timestamp for entering the start state of the corresponding operation is filtered out from the active operations; the mixed timestamp for entering the start state of the third operation is determined to be the seventh mixed timestamp; and historical data rows that meet the deletion criteria are deleted based on the seventh mixed timestamp.

[0199] An embodiment of the present application provides an electronic device, including a memory, a processor, and a computer program stored in the memory. The processor executes the above-mentioned computer program to implement the steps of a data processing method based on a distributed database. Compared with related technologies, it can achieve: under existing hardware conditions, ensuring transaction consistency and isolation during the processing process.

[0200] In an alternative embodiment, an electronic device is provided, such as Figure 4 As shown, Figure 4 The electronic device 4000 shown includes: a processor 4001 and a memory 4003. The processor 4001 and the memory 4003 are connected, for example, via a bus 4002. Optionally, the electronic device 4000 may further include a transceiver 4004, which may be used for data exchange between the electronic device and other electronic devices, such as data transmission and / or data reception. It should be noted that in actual applications, the number of transceivers 4004 is not limited to one, and the structure of the electronic device 4000 does not constitute a limitation on the embodiments of the present application.

[0201] Processor 4001 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It may implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. Processor 4001 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, and the like.

[0202] Bus 4002 may include a path for transmitting information between the above components. Bus 4002 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. Bus 4002 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 4 Only one thick line is used in the diagram, but this does not mean that there is only one bus or one type of bus.

[0203] The memory 4003 can be a ROM (Read Only Memory) or other types of static storage devices that can store static information and instructions, a RAM (Random Access Memory) or other types of dynamic storage devices that can store information and instructions, or an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory) or other optical disk storage, optical disk storage (including compact discs, laser discs, optical discs, digital versatile discs, Blu-ray discs, etc.), magnetic disk storage media, other magnetic storage devices, or any other medium that can be used to carry or store computer programs and can be read by a computer, without limitation here.

[0204] The memory 4003 is used to store the computer program for executing the embodiment of the present application, and the execution is controlled by the processor 4001. The processor 4001 is used to execute the computer program stored in the memory 4003 to implement the steps shown in the above method embodiment.

[0205] The electronic device includes but is not limited to any physical server in a distributed cluster.

[0206] An embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps and corresponding contents of the aforementioned method embodiment can be implemented.

[0207] The embodiments of the present application also provide a computer program product, including a computer program, which can implement the steps and corresponding contents of the aforementioned method embodiments when executed by a processor. The terms "first", "second", "third", "fourth", "1", "2", etc. (if any) in the specification and claims of this application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than the diagram or text description.

[0208] It should be understood that, although each operation step is indicated by arrows in the flowchart of the embodiment of the present application, the order of implementation of these steps is not limited to the order indicated by the arrows. Unless otherwise clearly stated herein, in some implementation scenarios of the embodiment of the present application, the implementation steps in each flowchart can be performed in other orders according to demand. In addition, some or all of the steps in each flowchart can include multiple sub-steps or multiple stages based on actual implementation scenarios. Some or all of these sub-steps or stages can be executed at the same time, and each sub-step or stage in these sub-steps or stages can also be executed at different times respectively. Under different scenarios at the execution time, the execution order of these sub-steps or stages can be flexibly configured according to demand, and the embodiment of the present application does not limit this.

[0209] The above description is only an optional implementation method for some implementation scenarios of this application. It should be pointed out that for ordinary technicians in this technical field, without departing from the technical concept of the solution of this application, the use of other similar implementation methods based on the technical ideas of this application also falls within the protection scope of the embodiments of this application.

Claims

1. A data processing method based on a distributed database, characterized in that: Applied to a current node, where the current node is any node in a distributed cluster; the method includes: Determining a current state of the first operation and a hybrid timestamp for entering the current state; wherein the hybrid timestamp is determined by a hybrid clock including a physical clock and a logical clock; Determining target data rows to be processed according to the first operation; wherein each data row includes a commit hybrid timestamp recording a commit time and recorded data, and the target data row belongs to at least one node; Determining a processing method for the target data row according to the hybrid timestamp of entering the current state and / or the hybrid timestamp of committing the target data row; The current node is configured with a local physical clock for generating a physical timestamp and a hybrid clock for generating a first hybrid timestamp; and determining the hybrid timestamp for entering the current state includes: Obtaining a current first hybrid timestamp and a physical timestamp, and comparing the values ​​of the physical timestamp and the physical clock portion of the first hybrid timestamp; If the comparison result is equal, updating the value of the logical clock part of the first hybrid timestamp in an auto-incremental manner; If the comparison result is greater than, updating the value of the physical clock part of the first hybrid timestamp to the physical timestamp, and updating the value of the logical clock part of the first hybrid timestamp by assigning a zero value; Determine the hybrid timestamp of entering the current state as an updated first hybrid timestamp; Each of the mixed timestamps corresponds to a status flag, and the mixed timestamp and the corresponding status flag are stored together.

2. The method according to claim 1, characterized in that The first operation is a target submission operation; The determining of a processing method for the target data row according to the hybrid timestamp of entering the current state and / or the hybrid timestamp of committing the target data row includes: If the target data row belongs to a node and the current state is a committed state, the processing method includes: performing an update operation on the target data row, and determining the second hybrid timestamp of entering the committed state of the target committed operation as the committed hybrid timestamp of the data row corresponding to the update operation; If the target data row belongs to at least two nodes and the current state is a pre-commit state, the processing method includes: determining a processing method for the target data row according to a third hybrid timestamp of entering the pre-commit state of the target commit operation.

3. The method according to claim 2, characterized in that The determining of a processing method for the target data row according to the third hybrid timestamp of entering the pre-commit state of the target commit operation includes: Obtaining third mixed timestamps of entering a pre-commit state of the target commit operation on the at least two nodes, and filtering out the largest commit timestamp therefrom; The maximum commit timestamp is used as the fourth hybrid timestamp for entering the commit state of the target commit operation, and the processing method is determined to include: performing an update operation on the target data row, and determining the fourth hybrid timestamp as the commit hybrid timestamp of the data row corresponding to the update operation.

4. The method according to claim 1, wherein If any request carrying the fifth mixed timestamp is received from a node connected to the user terminal, the method further includes: If the fifth mixed timestamp is greater than the current first mixed timestamp, the current first mixed timestamp is updated to the fifth mixed timestamp.

5. The method according to claim 1, characterized in that The hybrid timestamp also includes a corresponding state flag; the first operation is a target read operation, and the current state is a start state; determining a processing method for the target data row based on the hybrid timestamp of entering the current state and / or the commit hybrid timestamp of the target data row includes: If the state flag corresponding to the commit hybrid timestamp of the target data row is an address flag, obtaining a second operation for operating the target data row, and determining a processing method for the target data row based on the second operation and / or a sixth hybrid timestamp for entering a start state of the target read operation; If the status flag corresponding to the committed hybrid timestamp of the target data row is a time flag, a processing method for the target data row is determined according to the sixth hybrid timestamp.

6. The method according to claim 5, characterized in that The determining of a processing manner for the target data row according to the second operation and / or the sixth hybrid timestamp of the start state of the target read operation includes: When the target read operation and the second operation are operations in the same transaction, the processing method includes: setting the data recorded in the target data row to be visible to the target read operation; When the target read operation and the second operation are not in the same transaction: If the second operation is a write operation or if the second operation is a commit operation in a start state, the processing method includes: setting the data recorded in the target data row to be invisible to the target read operation; and if the second operation is a commit operation in a pre-commit state, the processing method includes: in response to a read signal, determining a processing method for the target data row according to the sixth hybrid timestamp; and if the second operation is a commit operation in a committed state, the processing method includes: determining a processing method for the target data row according to the sixth hybrid timestamp.

7. The method according to claim 5 or 6, characterized in that: The determining, according to the sixth mixed timestamp, a processing method for the target data row includes: Determine a difference between the sixth hybrid timestamp and the commit hybrid timestamp of the target data row; If the absolute value of the difference is not greater than a preset threshold, the processing method includes: restarting the target read operation to redetermine a processing method for the target data row; If the absolute value of the difference is greater than a preset threshold, and the difference is greater than zero, the processing method includes: setting the data recorded in the target data row to be visible to the target read operation; If the absolute value of the difference is greater than a preset threshold and the difference is less than zero, the processing method includes: setting the data recorded in the target data row to be invisible to the target read operation.

8. The method according to claim 7, characterized in that If the processing manner includes setting the data recorded in the target data row to be invisible to the target read operation, the method further includes: Obtaining historical data rows associated with the target data row; Whether the data recorded in the associated historical data row is visible to the target read operation is set according to the sixth mixed timestamp and the commit mixed timestamp of the associated historical data row.

9. The method according to claim 1, characterized in that The method further comprises: Count the active operations on each node, and select the third operation with the smallest mixed timestamp when entering the start state of the corresponding operation from the active operations; Determine the mixed timestamp for entering the start state of the third operation as the seventh mixed timestamp; Delete historical data rows that meet the deletion criteria based on the seventh mixed timestamp.

10. A data processing device based on a distributed database, characterized in that: Applied to a current node, where the current node is any node in a distributed cluster; the device comprises: A first determining module, configured to determine a current state of the first operation and a hybrid timestamp for entering the current state; wherein the hybrid timestamp is determined by a hybrid clock including a physical clock and a logical clock; a second determining module, configured to determine a target data row to be processed according to the first operation; wherein each data row includes a commit hybrid timestamp recording a commit time and recorded data, and the target data row belongs to at least one node; a third determining module, configured to determine a processing method for the target data row according to the hybrid timestamp of entering the current state and / or the hybrid timestamp of committing the target data row; The current node is configured with a local physical clock for generating a physical timestamp and a hybrid clock for generating a first hybrid timestamp; The first determining module is specifically configured to: Obtaining a current first hybrid timestamp and a physical timestamp, and comparing the values ​​of the physical timestamp and the physical clock portion of the first hybrid timestamp; If the comparison result is equal, updating the value of the logical clock part of the first hybrid timestamp in an auto-incremental manner; If the comparison result is greater than, updating the value of the physical clock part of the first hybrid timestamp to the physical timestamp, and updating the value of the logical clock part of the first hybrid timestamp by assigning a zero value; Determine the hybrid timestamp of entering the current state as an updated first hybrid timestamp; Each of the mixed timestamps corresponds to a status flag, and the mixed timestamp and the corresponding status flag are stored together.

11. An electronic device comprising a memory, a processor, and a computer program stored in the memory, wherein: The processor executes the computer program to implement the steps of the method according to any one of claims 1 to 9.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 9 are implemented.

13. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 9 are implemented.

Images