Reference management concurrency control

Abstract

A distributed system is provided that includes storage nodes individually arranged in a distributed arrangement, a memory storing instructions, and at least one processor in communication with the memory. The at least one processor, when executing the instructions, is configured to perform the following steps: modeling dependencies between transactions in the distributed system using a transaction priority graph partially constructed when executing the transactions, where the transactions are associated with keys stored in the storage nodes; the transaction is submitted in the distributed system in response to a loop check in a transaction priority graph of a transaction associated with the key in the transaction.

Description

Technical Field

[0001] The present disclosure relates to a system for processing application data, and more particularly to a method and apparatus associated with an in-memory transactional key-value storage system. Background Art

[0002] A storage system stores records, where users of the storage system can create, update, retrieve, and delete records by identifying them using keys. A key is a unique entity that can identify a specific record in the system. A record in the system is a collection of fields, and a key is a piece of information that allows a database search to find a record, including all associated fields. Keys can be user-defined. Examples of keys include, but are not limited to, email addresses. In non-distributed system designs, all records are stored on the same machine. In contrast, distributed systems divide (partition) the set of possible keys and assign each subrange of keys to a separate machine, often called a partitioned node or shard. A transaction reads and / or writes against a set of keys in a single atomic step so that these changes occur simultaneously. A transaction is a collection of operations initiated by one or more users.

[0003] Traditional databases retain the most recent version of their records. To meet isolation and consistency requirements, the database locks records when they are accessed, typically through a two-phase commit protocol (2PC). Under concurrent execution, this approach results in a high rate of aborted transactions, a measure of poor performance, because all keys shared by running transactions become blocking points.

[0004] Multi-Version Concurrency Control (MVCC) is the standard approach used by most state-of-the-art databases. MVCC requires retaining all writes within a time window (called the retention window). This retention eliminates conflicts between read and write transactions, significantly improving system throughput.

[0005] MVCC also has its drawbacks. When using MVCC, the system retains multiple versions of records for days or weeks, which incurs a significant space overhead. Delta encoding, a technique for storing or transmitting data as differences between sequential data rather than complete files, does help ensure that the cost is not linear, but comes at the expense of a runtime penalty for reconstruction. When using MVCC, garbage collection is required continuously. All current databases provide a background mechanism that continuously scans all data and deletes versions older than a configured retention window. Transactions that run longer than this retention window cannot benefit from MVCC and are automatically aborted or revert to a lock, which restricts access to the data. Summary of the Invention

[0006] It is an object of various embodiments to provide an efficient architecture and method for processing data for an application in a distributed manner. The architecture may include a distributed system, which is implemented as an in-memory transactional key-value store system, which may support high concurrency and efficient memory usage while providing strict serializability and reasonable latency for transactions. In the architecture, a transaction priority map may be used to identify and clear transaction versions that are no longer in use. The distributed system may include using a partially constructed transaction priority map, which may be constructed when a transaction is executed and may be maintained across multiple nodes. The constructed transaction priority map may be updated and combined when a transaction attempts to commit. The commit process may include: combining the partial priority maps and performing cycle checks in the transaction priority map to achieve the consistency goal of the distributed storage of the distributed system.

[0007] According to a first aspect of the present disclosure, a distributed system is provided, comprising: storage nodes separately arranged in a distributed arrangement; a memory for storing instructions; and at least one processor communicating with the memory, the at least one processor being configured to perform the following steps when executing the instructions: modeling dependencies between the transactions in the distributed system using a transaction priority graph partially constructed when executing transactions, wherein the transactions are associated with keys stored in the storage nodes; and committing the transactions in the distributed system in response to a loop check in the transaction priority graph of transactions associated with the keys in the transactions.

[0008] In a first implementation of the distributed system according to the first aspect itself, the at least one processor is used to: dynamically determine the data to be deleted from the distributed system in combination with the given transaction priority graph by determining that there is no path in the given transaction priority graph from an uncommitted transaction in the transaction priority graph to a committed transaction in the transaction priority graph, wherein the given transaction priority graph models dependencies based on associated keys and transaction commit times.

[0009] In a second implementation of the distributed system according to the first aspect itself or any of the above implementations of the first aspect, the storage nodes include data records and unique keys of the data records partitioned between the storage nodes, and each storage node includes a subset of the data records.

[0010] In a third implementation of the distributed system according to the first aspect itself or any of the above implementations of the first aspect, the at least one processor is used to: track whether the transaction in the distributed system is in progress, committed or aborted; maintain and update the transaction priority map of the transaction, and in response to detecting that other transaction priority maps affect keys shared with the transaction priority map, combine the transaction priority map with the other transaction priority maps.

[0011] In a fourth implementation of the distributed system according to the first aspect itself or any of the above implementations of the first aspect, the at least one processor is used to: in response to determining that the transaction is purgeable in the distributed system, delete the transaction and related information from the storage node.

[0012] In a fifth implementation of the distributed system according to the first aspect itself or any of the above implementations of the first aspect, the distributed system includes a client node, wherein the client node is used to issue read and write requests to the storage node, and the client node is arranged with an interface for an end user, and the end user is located outside the distributed system.

[0013] In a sixth implementation of the distributed system according to the first aspect itself or any of the above implementations of the first aspect, the at least one processor is used to: find a partial transaction priority graph including adjacent transactions of the transaction; add transitive dependency edges to the transaction priority graph of the transaction to generate a combined transaction priority graph of the transaction; check the commit time between the committed transactions of the partial transaction priority graph, and add edges based on the check of the commit time; perform a loop check in the combined transaction priority graph of the transaction; and determine whether to commit or abort based on the loop check.

[0014] In the seventh implementation of the distributed system according to the first aspect itself or any of the above implementations of the first aspect, each storage node includes: data records; transaction records, wherein each transaction record has a transaction identifier; transaction priority map records, wherein each transaction priority map record has a transaction priority map identifier.

[0015] In an eighth implementation of the distributed system according to the first aspect itself or any of the above implementations of the first aspect, the at least one processor is used to perform operations as multiple transaction coordinators and multiple directed acyclic graph (DAG) coordinators when executing the instructions, so that each active transaction in the distributed system has a transaction coordinator, the transaction coordinator tracks whether the active transaction is in progress, committed or aborted, and each DAG coordinator tracks a transaction priority map and updates and combines the transaction priority maps between other DAG coordinators.

[0016] In a ninth implementation of the distributed system according to the first aspect itself or any of the above implementations of the first aspect, when a given transaction starts, the transaction coordinator of the given transaction is assigned as the DAG coordinator of the given transaction.

[0017] In a tenth implementation of the distributed system according to the first aspect itself or any of the above-mentioned implementations of the first aspect, for a given transaction requested by a client node of the distributed system, in response to a commit request for the given transaction from the client node, the transaction coordinator of the given transaction is used to: determine the current status of the given transaction by checking the status of the given transaction in the transaction record of the given transaction; communicate with the DAG coordinator of the given transaction to evaluate whether the commit of the given transaction forms a loop in the given transaction priority graph of the given transaction; and implement the commit of the given transaction if no loop is formed in the given transaction priority graph.

[0018] According to a second aspect of the present disclosure, a method for operating a distributed data storage system is provided. The method comprises: modeling dependencies between transactions in a distributed system using a transaction priority graph partially constructed when executing transactions, wherein the distributed system has storage nodes separately arranged in a distributed arrangement, and when a transaction affects a common key stored in the storage node, the transaction is marked as a dependent transaction in the transaction priority graph; and in response to a loop check in the transaction priority graph of the transaction, committing the transaction in the distributed system.

[0019] In a first implementation of the method for operating a distributed data storage system according to the second aspect itself, by determining that there is no path in a given transaction priority graph from an uncommitted transaction in the transaction priority graph to a committed transaction in the transaction priority graph, the data to be deleted from the distributed system is dynamically determined in combination with the given transaction priority graph, wherein the given transaction priority models dependencies based on associated keys and transaction commit times.

[0020] In a second implementation of the method for operating a distributed data storage system according to the second aspect itself or any of the above implementations of the second aspect, data records and unique keys of the data records partitioned between the storage nodes are stored in the storage nodes, wherein each storage node includes a subset of the data records.

[0021] In a third implementation of the method for operating a distributed data storage system according to the second aspect itself or any of the above-mentioned implementations of the second aspect, track whether the transaction in the distributed system is in progress, committed or aborted; maintain and update the transaction priority map of the transaction, and in response to detecting that other transaction priority maps affect keys shared with the transaction priority map, combine the transaction priority map with the other transaction priority maps.

[0022] In a fourth implementation of the method for operating a distributed data storage system according to the second aspect itself or any of the above implementations of the second aspect, the method includes: in response to determining that the transaction is purgeable in the distributed system, deleting the transaction and related information from the storage node.

[0023] In a fifth implementation of the method for operating a distributed data storage system according to the second aspect itself or any of the above implementations of the second aspect, the method includes: issuing a read or write request from a client node of the distributed system to the storage node, wherein the client node is arranged with an interface for an end user, and the end user is located outside the distributed system.

[0024] In a sixth implementation of the method for operating a distributed data storage system according to the second aspect itself or any of the above-mentioned implementations of the second aspect, the method includes: finding a partial transaction priority graph including adjacent transactions of the transaction; adding transitive dependency edges to the transaction priority graph of the transaction to generate a combined transaction priority graph of the transaction; checking the commit time between the committed transactions of the partial transaction priority graph, and adding edges based on the check of the commit time; performing a loop check in the combined transaction priority graph of the transaction; and determining whether to commit or abort based on the loop check.

[0025] In a seventh implementation of the method for operating a distributed data storage system according to the second aspect itself or any of the above implementations of the second aspect, the method includes: maintaining data records, transaction records, and transaction priority map records in each storage node, wherein each transaction record has a transaction identifier, and each transaction priority map record has a transaction priority map identifier.

[0026] In an eighth implementation of the method for operating a distributed data storage system according to the second aspect itself or any of the above-mentioned implementations of the second aspect, the method includes: running multiple transaction coordinators by executing stored instructions by one or more first processors, so that each active transaction in the distributed system has a transaction coordinator, and the transaction coordinator tracks whether the active transaction is in progress, committed or aborted; running multiple directed acyclic graph (DAG) coordinators by executing stored instructions by one or more second processors, so that each DAG coordinator tracks a transaction priority graph and updates and combines the transaction priority graphs between other DAG coordinators.

[0027] In a ninth implementation of the method for operating a distributed data storage system according to the second aspect itself or any of the above implementations of the second aspect, the method includes: at the start of a given transaction, allocating the transaction coordinator of the given transaction as the DAG coordinator of the given transaction.

[0028] In a tenth implementation of the method for operating a distributed data storage system according to the second aspect itself or any of the above-mentioned implementations of the second aspect, the method includes: for a given transaction requested by a client node of the distributed system, in response to a commit request for the given transaction from the client node, the transaction coordinator of the given transaction executes: determining the current status of the given transaction by checking the status of the given transaction in the transaction record of the given transaction; communicating with the DAG coordinator of the given transaction to evaluate whether the commit of the given transaction forms a loop in the given transaction priority graph of the given transaction; and implementing the commit of the given transaction if no loop is formed in the given transaction priority graph.

[0029] According to a third aspect of the present disclosure, a non-transitory computer-readable storage medium storing instructions for processing data is provided, wherein the instructions, when executed by at least one processor, cause the at least one processor to perform operations, the operations comprising: modeling dependencies between transactions in a distributed system using a transaction priority graph partially constructed when executing transactions, wherein the distributed system has storage nodes separately arranged in a distributed manner, and when the transaction affects a common key stored in the storage node, the transaction is marked as a dependent transaction in the transaction priority graph; and committing the transaction in the distributed system in response to a loop check in the transaction priority graph of the transaction.

[0030] In a first implementation of the non-transitory computer-readable medium according to the third aspect itself, data to be deleted from the distributed system is dynamically determined in combination with a given transaction priority graph by determining that no path exists in the given transaction priority graph from an uncommitted transaction in the transaction priority graph to a committed transaction in the transaction priority graph, wherein the given transaction priority graph models dependencies based on associated keys and transaction commit times.

[0031] In a second implementation of the non-transitory computer-readable medium according to the third aspect itself or any of the above implementations of the third aspect, the operation includes: storing data records and unique keys of the data records partitioned between the storage nodes in the storage nodes, wherein each storage node includes a subset of the data records.

[0032] In a third implementation of the non-transitory computer-readable medium according to the third aspect itself or any of the above implementations of the third aspect, the operation includes: tracking whether the transaction in the distributed system is in progress, committed or aborted; maintaining and updating the transaction priority map of the transaction, and combining the transaction priority map with the other transaction priority map in response to detecting that the other transaction priority map affects a key shared with the transaction priority map.

[0033] In a fourth implementation of the non-transitory computer-readable medium according to the third aspect itself or any of the above implementations of the third aspect, the operation includes: in response to determining that the transaction is purgeable in the distributed system, deleting the transaction and related information from the storage node.

[0034] In a fifth implementation of the non-transitory computer-readable medium according to the third aspect itself or any of the above implementations of the third aspect, the operation includes: issuing a read or write request from a client node of the distributed system to the storage node, wherein the client node is arranged with an interface for an end user, and the end user is located outside the distributed system.

[0035] In a sixth implementation of the non-transitory computer-readable medium according to the third aspect itself or any of the above implementations of the third aspect, the operation includes: finding a partial transaction priority graph including adjacent transactions of the transaction; adding transitive dependency edges to the transaction priority graph of the transaction to generate a combined transaction priority graph of the transaction; checking the commit time between the committed transactions of the partial transaction priority graph, and adding edges based on the check of the commit time; performing a loop check in the combined transaction priority graph of the transaction; and determining whether to commit or abort based on the loop check.

[0036] In a seventh implementation of the non-transitory computer-readable medium according to the third aspect itself or any of the above implementations of the third aspect, the operation includes: maintaining data records, transaction records and transaction priority map records in each storage node, wherein each transaction record has a transaction identifier and each transaction priority map record has a transaction priority map identifier.

[0037] In an eighth implementation of the non-transitory computer-readable medium according to the third aspect itself or any of the above implementations of the third aspect, the operation includes: running multiple transaction coordinators by executing stored instructions by one or more first processors, so that each active transaction in the distributed system has a transaction coordinator, and the transaction coordinator tracks whether the active transaction is in progress, committed or aborted; running multiple directed acyclic graph (DAG) coordinators by executing stored instructions by one or more second processors, so that each DAG coordinator tracks a transaction priority graph and updates and combines the transaction priority graphs between other DAG coordinators.

[0038] In a ninth implementation of the non-transitory computer-readable medium according to the third aspect itself or any of the above implementations of the third aspect, the operation includes: at the start of a given transaction, allocating the transaction coordinator of the given transaction as the DAG coordinator of the given transaction.

[0039] In a tenth implementation of the non-transitory computer-readable medium according to the third aspect itself or any of the above implementations of the third aspect, the operation includes: for a given transaction requested by a client node of the distributed system, in response to a commit request for the given transaction from the client node, the transaction coordinator of the given transaction executes: determining the current state of the given transaction by checking the state of the given transaction in the transaction record of the given transaction; communicating with the DAG coordinator of the given transaction to evaluate whether the commit of the given transaction forms a loop in the given transaction priority graph of the given transaction; and implementing the commit of the given transaction if no loop is formed in the given transaction priority graph.

[0040] Any of the above examples may be combined with any other one or more of the above examples to create new embodiments according to the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] The accompanying drawings illustrate generally, by way of example and not limitation, various embodiments described herein.

[0042] Figure 1 is a representation of two transactions in a directed graph according to an exemplary embodiment.

[0043] Figure 2 According to an exemplary embodiment, a transaction and Figure 1 Representation of the transaction.

[0044] Figure 3 is a representation of a transaction priority graph with transactions that cycle according to an exemplary embodiment.

[0045] Figure 4 is a representation of a directed acyclic graph according to an exemplary embodiment.

[0046] Figures 5 to 7 A topological arrangement in a topological sort according to an exemplary embodiment is shown.

[0047] Figure 8 An arrangement between a given transaction and other transactions is shown according to an exemplary embodiment, wherein the given transaction attempts to commit.

[0048] Figure 9 is a representation of a distributed system of services and nodes that may be configured for reference management concurrency control according to an exemplary embodiment.

[0049] Figure 10 is a representation of the interaction of various components used to reference manage concurrency control in a data center according to an exemplary embodiment.

[0050] Figure 11 According to an exemplary embodiment, Figure 10 A mechanism for annotating transaction operations in a data center and enabling other transactions to discover a given directed acyclic graph for concurrent access.

[0051] Figure 12A and Figure 12B is an example flow chart of starting a transaction and performing operations in reference managed concurrency control according to an exemplary embodiment.

[0052] Figure 13A and Figure 13B is a flow chart of an exemplary submission request in reference management concurrency control according to an exemplary embodiment.

[0053] Figure 14A and Figure 14B is a flow chart of an exemplary cleanup communication in reference management concurrency control according to an exemplary embodiment.

[0054] Figure 15 is a flow chart of features of an exemplary method of operating a distributed data storage system according to an exemplary embodiment.

[0055] Figure 16 A block diagram of a computing system that implements algorithms and executes methods configured to process data for applications is shown in accordance with an exemplary embodiment. DETAILED DESCRIPTION

[0056] In the following description, reference is made to the accompanying drawings, which form a part of this specification, and in which are shown by way of illustration specific embodiments that may be practiced. These embodiments are described in sufficient detail to enable those skilled in the art to practice them, and it is understood that other embodiments may be used and that structural, logical, mechanical, and electrical changes may be made. Therefore, the following description of exemplary embodiments is not intended to be limiting.

[0057] The functions or algorithms described herein can be implemented in software in an embodiment. The software may include computer-executable instructions stored in a computer-readable medium or a computer-readable storage device (e.g., one or more non-transient memories or other types of hardware-based local or networked storage devices). In addition, these functions correspond to modules, which can be software, hardware, firmware, or any combination thereof. Multiple functions can be performed in one or more modules as needed, and the described embodiments are merely examples. The software can be executed on a digital signal processor, an application-specific integrated circuit (ASIC), a microprocessor, or other types of processors running on a computer system such as a personal computer (PC), a server, or other computer system, thereby turning such a computer system into a specifically programmed machine.

[0058] Computer-readable non-transitory media includes all types of computer-readable media, including magnetic storage media, optical storage media, and / or solid-state storage media, and specifically excludes signals. It should be understood that the software can be installed in and sold with a device that operates in association with the reference management concurrency control for data processing described herein. Alternatively, the software can be obtained and loaded into such a device, including through optical disk media or from any form of network or distribution system, including, for example, from a server owned by the software creator or from a server not owned by the software creator but used by the software creator. For example, the software can be stored on a server for distribution via the Internet.

[0059] In various embodiments, a system called reference managed concurrency control (RMCC) can be implemented as an in-memory transactional key-value store that supports high concurrency and efficient memory usage while guaranteeing strict serializability and reasonable latency. Serializability means that transactions can be serialized. In this approach, given a set of transactions, the transactions are said to be serializable if a certain ordering can be established in which the transactions execute sequentially. This total ordering on the set is a serial schedule. More than one total ordering can exist; for example, [T1->T2->T3] and [T3->T1->T2] are both serial schedules. These may not be the order in which the transactions would have executed in real time, but serializability is independent of temporal order. Given a set of transactions, if a serial schedule can be constructed, then each of the transactions is serializable with respect to the set. In a running system where all previously committed transactions are on a serial schedule, if a new transaction can be found within the existing ordering such that the entire set is still on a serial schedule, then the new transaction is serializable with respect to the system. Therefore, a transaction is serializable if it has been executed in some serial schedule. The system does not need to physically create such a schedule, but it provides a basis for analyzing the fact that transactions are isolated from each other and appear isolated to any observer of the system. If you try to add a new transaction to a given set of committed / historical transactions, if there is some possible ordering (aka serial order) called a schedule that makes all these transactions appear to have executed sequentially, then the new transaction is said to be serializable.

[0060] In a system that provides serial scheduling, transactions appear to execute as if they were non-interleaved, even if they do overlap. That is, no transaction can be seen to start before a running transaction has finished. Serial scheduling can include a total ordering of transactions. For any two non-concurrent transactions T1 and T2, if transaction T1 finishes before transaction T2 begins (finish(T1) < start(T2)), then T1 proceeds before T2, and transactions T1 and T2 are linearizable. When transactions are both serializable and linearizable, the system has strict serializability. In other words, the system convinces all observers that a total ordering of transactions exists that is consistent with the real-time ordering of transactions. Linearizability identifies a valid ordering but does not specifically mention any information about concurrently running transactions. Strict serializability, or both serializability and linearizability, ensures that any observer believes all historical transactions to have occurred sequentially, and that if the second transaction actually occurred after the first, any observer would see the result of the second transaction supersede the result of the first.

[0061] Compared to the traditional MVCC design that requires periodic garbage collection, the RMCC design can be used to efficiently delete unnecessary record versions on demand. This design includes the RMCC transaction protocol design and implementation considerations. RMCC can solve the problems associated with MVCC while retaining and improving the advantages. Multiple record versions can still be used in RMCC, but this retention in RMCC is performed on demand and only to satisfy ongoing transactions. Once the transaction is committed, the redundant versions can be discarded. Commit is the application of changes made to the subject records of the transaction when the transaction is considered complete. A transaction that performs one or more operations on the records of a distributed system can be started by receiving a begin statement from a user device. When the operations of the transaction are at the completion point, the user device can commit the transaction or abort the transaction, which means that the user device either issues a commit command to the system to apply all changes performed on the records, or reverts the changes made while executing the operations of the transaction.

[0062] In terms of memory usage, it is estimated that the memory overhead of supporting concurrent transactions can be reduced by nearly 95% when using RMCC compared to traditional MVCC. In terms of background overhead, when using RMCC, there is no need to collect background garbage, and instead graph-based transaction dependency tracking is used, which is cleaned up immediately at commit time. In terms of reducing latency, RMCC can reduce reliance on timestamp usage and support concurrent timestamp allocation, thereby reducing network latency for each transaction by up to 33%. In terms of reducing abort rates, due to the use of graph cycle detection, in some workloads, RMCC can reduce the conflict rate by up to 50% compared to traditional MVCC.

[0063] In various embodiments, RMCC can be implemented to address inefficient memory usage by clearing versions of transactions as soon as they are no longer needed by the transaction, while guaranteeing serializability and external causality. To achieve these goals, RMCC can use a transaction precedence graph. Using a transaction precedence graph enables RMCC to determine whether a commit would form a dependency cycle that would violate strict serializability. The same graph can be used to clean up multiple record versions, where older record versions in a committed transaction are cleared when no path exists in the transaction precedence graph from any uncommitted transaction to a committed transaction. The cleanup process can be triggered by a commit request or an abort request for a transaction, which means that multiple record versions are retained long enough to satisfy any outstanding transactions and no longer.

[0064] Figure 1 is an example of a representation 100 of two transactions T1 and T2 in a directed graph. A directed graph of transactions G(V,E) may include a set V of all transactions and associated edges E. Figure 1In the example of , an edge extends from transaction T1 to T2 if T1 occurs before T2 in the schedule, in which case T2 is said to be dependent on or contingent upon T1, where T1 and T2 are based on the same key. Figure 2 It includes transaction T3 and Figure 1 200 is an example of a representation of transactions T1 and T2. Edges can be added between T1, T2, and T3 by detecting conflicts (based on keys) or linearizability (based on time order).

[0065] Figure 3 FIG3 is an example of a transaction priority graph representation 300 with cycles. Transactions are represented by vertices 0, 1, 2, 3, and 4. The transaction priority graph is serializable if and only if there are no cycles in the transaction priority graph. If the transaction priority graph has cycles, there is no total ordering and serial scheduling of transactions. In representation 300, there are cycles between transaction vertices 2 and 4 and between transaction vertices 0, 1, 2, and 3.

[0066] Figure 4 is an example of a representation 400 of a directed acyclic graph (DAG). A directed graph without cycles is a DAG. Since the transaction priority graph is serializable, the RMCC that processes the serializable transaction priority graph operates on the DAG. A DAG has a topological sort if a total ordering of vertices is provided such that there are no backward edges. In representation 400, the edges between vertices 0, 1, 2, 3, 4, 5, 6, and 7 of representation 400 have no backward edges. A topological sort is a serial schedule.

[0067] Figures 5 to 7 An example of topological arrangement in topological sorting is shown. Topological arrangement is performed to arrange vertices starting from vertices with no incoming edges. Figure 5 A starting arrangement 500 of vertices A, B, C, D, E, F, and G is shown, in which no cycles exist. Vertex A has edges to vertices B and C. Vertex B has edges to vertices C and D. Vertex C has an edge to vertex E. Vertex D has edges to vertices F and E. Vertex E has no edges from vertex E to any vertices in the arrangement, and vertex G has edges to E and F, but no incoming edges from any vertex in the arrangement to vertex G. Figure 6 Shown Figure 5 An arrangement 600 of the vertices of arrangement 500 is provided, wherein the vertices are arranged in a linear manner, preserving the direction of the edges of arrangement 500 . Figure 7 Shown Figure 6Arrangement 700 of the vertices of arrangement 600 is performed, wherein the vertices are arranged in a linear manner, and the process always picks vertices without incoming edges. As a result of arrangement 700, there are no backward edges between vertices A, B, C, D, E, F, and G of arrangement 700, and there are no incoming edges to the vertices arranged at the beginning of arrangement 700. DAG, topological sorting, and topological permutation can be used in the operation of RMCC.

[0068] The RMCC described in this paper can be implemented to use a transaction priority graph to identify and clear transactions. RMCC's functionality can include using the transaction priority graph to achieve serializability in distributed systems while also providing external causality. Due to the overhead of maintaining and using the transaction priority graph for cycle checking, RMCC's network overhead may not be minimized, but for transactions without concurrent transactions, RMCC's performance can achieve microsecond-level latency.

[0069] RMCC can provide strict serializability. In RMCC, if the execution time of two transactions overlaps and the two transactions access common resources during execution, then the two transactions are concurrent. RMCC can provide the strictest transaction consistency guarantee for client devices, which is called external consistency or linearizability. Under external consistency, the behavior order of a distributed system using RMCC is consistent with real time. If one transaction T1 commits before another transaction T2 that is non-concurrent with T1 commits, the system guarantees that the client device will not see a state that includes the effects of the second transaction T2 but does not include the effects of the first T1. Intuitively, RMCC is semantically indistinguishable from a stand-alone database.

[0070] The architecture of RMCC can be constructed through several guiding design elements. RMCC can be constructed to achieve memory efficiency by identifying transactions to be cleared from a transaction precedence graph when a transaction commits or aborts. RMCC can be constructed to achieve serializability using a transaction precedence graph along with cycle checking on distributed storage. RMCC can be constructed to achieve linearizability and serializability without using start timestamps. RMCC can be constructed to use end timestamps that can be constrained to ensure linearizability rather than for selecting record versions. RMCC can be constructed to use a partially constructed transaction precedence graph maintained across multiple nodes, and to update and combine the transaction precedence graph when a transaction attempts to commit.

[0071] Figure 8An arrangement 800 is shown between a transaction T and other transactions C1, C2 and C3, where transaction T attempts to commit. In RMCC, the commit process can include cycle detection, which is based on an invariant condition that there are no cycles between committed transactions. In arrangement 800, there are three committed transactions C1, C2 and C3, where the three committed transactions do not include cycles between themselves. The committed transactions C1, C2, C3 have been committed and are fixed in the system. Undoing the decision that C1, C2, C3 have already committed would violate consistency. Since T has not yet committed, the committed transactions C1, C2, C3 are still in the DAG that includes T maintained by the system. Consider that transaction T attempts to commit. The cycle can be determined by performing cycle detection based on the last transaction that attempted to commit, which can be used to speed up the cycle search. A variety of different methods can be implemented to detect cycles. For the transaction priority graph of arrangement 800, only check whether there is a cycle consisting of transaction T and the committed transaction C. As Figure 8 As shown, adding an edge between the committed transaction C3 and transaction T forms a cycle. Figure 8 It shows that transaction C1 depends on transaction T, where transaction T has not yet committed. It should be noted that DAG is not used to enforce the order in which transactions are committed. Specifically, DAG represents the dependency relationship between concurrent transactions and is used to determine whether any particular transaction is allowed to commit. After detecting a loop, RMCC can not allow transaction T to commit because transaction T will form a loop. Not allowing transaction T to commit is the same as forcing transaction T to abort. The reason for aborting transaction T at commit time is that precisely because transactions C1, C2, and C3 have already committed, if transaction T is not aborted, then serializability will be violated. In DAG terms, committing transaction T will solidify a loop in the system, which is equivalent to non-serializable scheduling.

[0072] Figure 9This is a representation of a distributed system 900 of services and nodes that can be constructed for RMCC. The system can be implemented as a set of services that work together to achieve the system design goals. A service or service cluster can be implemented as a set of nodes for a specific component that are configured to work together. A node can be constructed as a running instance of the software for a given component. A node can include one or more processors to execute the software instance. A node can have a unique network address, which provides a mechanism for other nodes or other parts of the software to send messages to the given node. Other nodes can include, but are not limited to, client devices or nodes from other services. A single machine (e.g., a host) can run multiple nodes, as determined by the system operator. Components of distributed system 900 may include a timestamp oracle (TSO) service 916, a control plane oracle (CPO) service 926, storage services 906, and persistence services 936. As used herein, an oracle is a permission or mechanism used to make decisions for the entity to which it refers. The TSO service 916 can provide transaction end timestamps to ensure linearizability. The CPO service 926 can provide a central controller for a cluster of nodes. The storage service 906 can provide distributed storage of transaction data. The persistence service 936 can provide the ability to store data so that it can be made available again or to other users or services after the process that used or generated the data is no longer running. The functionality of these components can be implemented in software. The nodes of a service can all run instances of the same component software. For example, a TSO node can run an instance of TSO software, where all TSO nodes are configured as part of the same TSO service. When a software module communicates with a TSO service, the software communicates with a specific node that is part of a specific TSO service. Service-specific mechanisms can be implemented to decide which specific node to communicate with.

[0073] In the distributed system 900, the storage service 906 may include storage node 905-1, storage node 905-2... storage node 905-N. The TSO service 916 may include TSO node 915-1, TSO node 915-2... TSO node 915-N. The CPO service 926 may include CPO node 925-1, CPO node 925-2... CPO node 925-N. The persistence service 936 may include persistence node 935-1, persistence node 935-2... persistence node 935-N. Although each service of the distributed system 900 has the same number of nodes, similar distributed systems may include services with different numbers of nodes. Each node of the service may operate independently of each other and may be implemented by one or more processors executing storage instructions of independent nodes. Alternatively, the nodes may share one or more processors, and the one or more processors are used to support the functions of the nodes of the service. Figure 9 This reflects that during the execution of one or more applications, nodes of the active services of one or more applications can interact with each other.

[0074] Figure 10 1 and 1020-3. Components in data center 1000 include, but are not limited to, client nodes 1002-1, 1002-2, and 1002-3, storage nodes 1005-1, 1005-2, and 1005-3, CPO 1026, TSO 1016, transaction coordinators 1010-1 and 1010-2, and DAG coordinators 1020-1, 1020-2, and 1020-3. Although data center 1000 shows a plurality of these components, a data center (e.g., data center 1000) may include more than Figure 10 Greater or fewer quantities of each component may be shown.

[0075] TSO 1016 can be responsible for issuing real-time-based timestamps with error bounds for transactions. In RMCC, TSO 1016 can be configured to only obtain commit timestamps, not to select record versions. Multiple TSO instances can be used simultaneously. In such multiple TSO instances, the TSO instances can be implemented to agree on a maximum error bound for the timestamps provided to the TSO instances. The ordering of transactions is typically determined based on data dependencies. In some cases, there are transactions that do not access any common data but have actual dependencies, such as issuing a sequence after issuing it from a device separate from the device that issued the previous sequence. This is an external causal dependency because the dependency is external to the system. These transactions can be captured using the timestamps issued by the TSO. Since there are no data dependencies, but there are causal relationships, the associated DAG records the relationship of these transactions. This is recorded by inserting edges sorted according to the timestamps of the previous and subsequent transactions.

[0076] The CPO 1026 can be configured as a central controller for the node cluster of the data center 1000. The CPO 1026 can be responsible for managing cluster partitioning, scaling activities, or other management activities for the component cluster. The CPO 1026 can also serve as a versioned discovery system so that nodes and clients can discover the location of cluster components.

[0077] The client nodes 1002-1, 1002-2, and 1002-3 can be configured as uncoordinated client nodes. The client nodes 1002-1, 1002-2, and 1002-3 can be configured as the only components of the RMCC arrangement of the data center 1000 that are visible to the end user. Each of the client nodes 1002-1, 1002-2, and 1002-3 can communicate with a transaction coordinator node or a storage node to start a transaction, perform operations (read, write), and commit or abort the transaction. For these operations, the client nodes 1002-1, 1002-2, and 1002-3 can individually issue requests for the transaction that is the subject of the request to the storage node and the transaction coordinator. Figure 10 As shown, there may be multiple client nodes in the distributed system of data center 1000. Client nodes 1002-1, 1002-2, and 1002-3 are independent, so that each node does not need to coordinate with every other node.

[0078] Storage nodes 1005-1, 1005-2, and 1005-3 can be configured as key partitioned storage nodes. Each of the storage nodes 1005-1, 1005-2, and 1005-3 can include a pre-assigned subset of keys, where the keys are assigned by the CPO 1026, which is responsible for managing the keys and storage nodes of the data center 1000. The data center 1000 can be implemented as a storage system whose responsibility is to provide a unified view of a data set that cannot physically be accommodated on a single machine. This implementation can be achieved by partitioning (splitting) the data into smaller chunks (called partitions), where a partition is a subset of the entire data. In one embodiment of the data center 1000, it can be ensured that the size of each partition is limited and can be accommodated on a single machine. In the storage service of data center 1000 including storage nodes 1005-1, 1005-2 and 1005-3, each storage node in storage nodes 1005-1, 1005-2 and 1005-3 can be allocated exactly one partition in a one-to-one relationship. This allocation can be implemented as a mapping, which can be stored in a structure called a partition map. When any record-level operation is performed based on a given key (e.g., reading a record whose key is equal to a specific email address), it can be determined in which partition the given key is located. Through the partition map, it can be accurately determined which storage node should own the record that is the subject of the record-level operation. In one embodiment, the entire storage node cluster can be used to execute the protocol process on the partition map.

[0079] Each of the storage nodes 1005-1, 1005-2, and 1005-3 can receive read or write requests from client nodes 1002-1, 1002-2, and 1002-3, and can perform the requested operations by communicating with the transaction coordinator as appropriate and respond to the requesting client. Each of the storage nodes 1005-1, 1005-2, and 1005-3 can handle read, write, and clear version requests for keys and maintain a local transaction priority map. Figure 10 As shown, multiple storage nodes may exist in a distributed system in data center 1000. The key domain may be partitioned, and each storage node may be assigned keys from a different portion of the partition. The partition map may be implemented as (but not limited to) a hash value of the key, so that the storage node for a specific key can be identified solely by the key.

[0080] Transaction coordinators 1010-1 and 1010-2 can be implemented as transaction coordinator nodes. The transaction coordinator can be responsible for tracking transactions and committing or aborting the transaction. Each active transaction in the RMCC of the data center 1000 has a transaction coordinator that tracks whether the transaction is in progress, committed, or aborted. When a client (e.g., one of the client nodes 1002-1, 1002-2, and 1002-3) issues a commit request, the transaction coordinator communicates with the DAG coordinator of the transaction to check whether the transaction can be successfully committed. Figure 10 As shown, multiple transaction coordinator nodes may exist in the distributed system of data center 1000. Transactions may be mapped to transaction coordinators using a transaction identification (ID), which may be a universally unique identifier (UUID). In various embodiments, storage nodes (e.g., storage nodes 1005-1, 1005-2, and 1005-3) may also serve as transaction coordinators.

[0081] DAG coordinators 1020-1, 1020-2, and 1020-3 can be implemented as DAG coordinator nodes. Each transaction in data center 1000 has a DAG coordinator. The DAG coordinator can be responsible for tracking various parts of the partial transaction priority graph. The DAG coordinator can also be used to maintain, update, and combine partial transaction priority graphs between DAG coordinators. At the beginning of a transaction, the transaction coordinator for that transaction can be assigned as the DAG coordinator. As the transaction priority graph is updated and combined with other transactions, the DAG coordinator can become a different transaction coordinator, for example by merging with other DAG coordinators. The DAG coordinator can be responsible for checking whether a transaction can be committed by performing a loop check. The DAG coordinator can identify transactions that can be purged by finding committed transactions that do not include ongoing transactions that depend on it, can issue a purge request to the transaction coordinator, and can signal storage nodes when record versions can be purged. When the DAG coordinator determines that a transaction can be purged, the transaction can also be deleted from the transaction coordinator.

[0082] For the example of data center 1000, when a transaction begins, an asynchronous request is made to TSO 1016 to obtain a new timestamp for the transaction. A random storage node can be selected from the storage nodes of data center 1000 (e.g., storage nodes 1005-1, 1005-2, and 1005-3) to serve as the transaction coordinator and the DAG coordinator for the transaction. A message can be sent to the selected storage node so that it can begin tracking the transaction. All operations performed in the transaction can be annotated with the transaction coordinator location, which enables any other transaction to discover the DAG for any concurrent access.

[0083] Figure 11 Shown Figure 10 A mechanism for annotating transaction operations in a data center 1000 and enabling other transactions to discover the DAG for any concurrent access. Client nodes 1002-1, 1002-2, and 1002-3 may include client applications 1103-1, 1103-2, and 1103-3, and client libraries 1104-1, 1104-2, and 1104-3, respectively. The client library may be implemented as software that provides a programmatic interface that enables user applications (e.g., client applications 1103-1, 1103-2, and 1103-3) to execute transactions. Each of client libraries 1104-1, 1104-2, and 1104-3 may be responsible for discovering service nodes and assisting with some transaction coordination activities. Each of client libraries 1104-1, 1104-2, and 1104-3 may interact with one or more storage nodes 1005-1, 1005-2, and 1005-3. In a non-limiting example, client library 1104-1 can interact with storage nodes 1005-1 and 1005-2, client library 1104-2 can interact with storage nodes 1005-2 and 1005-3, and client library 1104-3 can interact with storage nodes 1005-1 and 1005-3. When a transaction begins, the client library can select a coordinator for the transaction and the associated DAG. First, the client library can select a storage partition as the coordinator for both the transaction and its DAG. The DAG coordinator may change later when the transaction crosses paths with other transactions in the RMCC system of data center 1000 (where such path interactions may cause DAG merges).

[0084] Storage node 1005-1 may include data record 1106-1, transaction record 1107-1, and DAG record 1108-1. Storage node 1005-2 may include data record 1106-2, transaction record 1107-2, and DAG record 1108-2. Storage node 1005-3 may include data record 1106-3, transaction record 1107-3, and DAG record 1108-3. Each of transaction records 1107-1, 1107-2, and 1107-3 may track the DAG to which a given transaction currently belongs. For example, transaction record 1107-3, representing transaction records 1107-1 and 1107-2, may include transaction file 1109, which includes multiple records, such as transaction record 1 and transaction record 2, where transaction record 1 and transaction record 2 are associated with transaction identifiers TxnId1 and TxnId2, respectively. In this example, transaction record 2 may include an identifier, TxnId2, that belongs to a DAG identified using a DAG identification (DAGId) set as DAGId2 in transaction record 2. DAG file 1111 may be searched for DAGId2 associated with DAG record 2, such that a transaction with TxnId2 tracks the DAG identified by DAGId2.

[0085] In transactional execution, all operations performed during a transaction can be recorded by the corresponding storage node as either a read intent or a write intent. Each intent can be annotated with the transaction coordinator that issued the transaction. If no other transactions are performing any concurrent access, the transaction can be trivially committed. Once the commit decision is recorded, the system can clean up the registered intent. In the concurrent execution model, the goal is to ensure strict serializability between the involved transactions. As mentioned earlier, this means maintaining the invariant condition that commits do not form cycles in the precedence graph.

[0086] In RMCC, intentions can be used to detect concurrent access. Consider the scenario of two concurrent transactions, T1 and T2, each of which places some intentions in the storage cluster, but without conflict. To accommodate T1's current intention placement, which overlaps with T2's currently held intention, a decision must be made on how to order these two transactions. This ordering can also be applied to all other intentions. Considering both T1 and T2 as trivial graphs of size 1, precedence can be registered by merging these two graphs into a single graph, where edges represent a predetermined order. As execution continues, T1 or T2 may encounter another set of concurrent transactions, such as transactions T3, T4, and T5. This situation can be detected by placing intentions. Therefore, the concurrent graphs {T1, T2} and {T3, T4, T5} can be merged and a single DAG owner designated. If any transaction in this merged set attempts to commit, the graph's single DAG owner can be consulted. The single DAG owner can now perform cycle checking for the entire set of overlapping transactions {T1, T2, T3, T4, T5}.

[0087] The primary tool for ensuring strict serializability in RMCC is the transaction precedence graph. The transaction precedence graph is a DAG that is built in a distributed fashion as running transactions access data. The edges in the DAG represent transaction dependencies, with the transactions themselves represented as nodes. Edges can be divided into two types. One type is the ordering between concurrent transactions, and the other type is the ordering between non-current transactions that use linearizability. This other type allows dependencies between non-concurrent transactions to be captured. If transaction t1 ends before t2, and the two are non-concurrent, then if an edge exists, then the edge is from t1 to t2. At any given time, multiple independent DAGs can be built in parallel in an RMCC cluster, where each DAG represents a set of transactions that have some dependencies on each other when performing read or write operations.

[0088] All transactions can initially begin as individual nodes, independent of their own DAG coordinator. As transactions execute operations on storage nodes, new dependencies can be added between them. Clients and transaction coordinators can track these dependencies locally. Only when a transaction attempts to commit are the dependency edges accumulated from executing operations on different storage nodes introduced into a single transaction priority graph. At this stage, the various partially constructed transaction priority graphs, which may be stored on different DAG coordinators, are merged together based on these new dependency edges. Next, a cycle check can be performed to see if the transaction can commit. During the cycle check, linearizability checks can also be performed and new dependency edges can be added. If the cycle check fails, the transaction can be aborted, and all edges attached to the transaction can be deleted.

[0089] Edges in the transaction priority graph can be divided into edges identified during operations on storage nodes and edges identified at commit time. After commit or abort, committed transactions that are unreachable from non-committed transactions are purged from the transaction priority graph. No longer needed data, namely committed key-value records from these purged transactions, are purged from storage nodes where updated committed records are available.

[0090] In various embodiments, to ensure linearizability after transaction clearing, a clearing time can be introduced. The clearing time (t) of a committed transaction t is the maximum of the commit time of any cleared transaction that can reach t and the commit time of t. u Maximum clearing time (mct(t u )) can be t u The maximum commit time of any cleared transaction with an incoming edge. Clear time (t) and mct (t u ) can be used to ensure linearizability after transactions are cleared.

[0091] Constructing and maintaining different parts of a transaction precedence graph in a distributed manner can include multiple segments. Multiple segments can include adding and maintaining dependencies when operating on storage nodes. Transaction precedence graphs can be merged during commits, and new edges can be added and cycle checks performed during commits. Committed transactions can be purged while maintaining a record of completed transactions.

[0092] When processing operations on storage nodes, new dependencies between transactions can be identified and introduced. During the operation, local knowledge can be leveraged to ensure that cycles are not formed and that linearizability can be maintained. If the operation cannot be performed without introducing a new cycle or violating linearizability, the operation can be returned as failed. Processing operations on storage nodes can be viewed as having two objectives: one is the performance of the operation, and the other is to ensure that new dependencies are added through local feasibility checks.

[0093] Each storage node in the RMCC can have a storage node manager (SNM) component that can track information that facilitates processing operations in a straightforward manner. This information can include a list of all transactions that have accessed any key on the storage node, and a set of transaction records for each transaction t and the key k accessed by t, sorted by operation ID, which can be reads and writes performed by t on k. This information can include a local transaction priority graph among all transactions that have accessed keys on the storage node based on local dependency knowledge. For each key k, this information can include a DAG sort of transactions that have accessed k. kFor each key, this information can include the value and purge time of the last cleared transaction in the transaction order. Additionally, the storage node manager can be updated with the transaction's new dependencies and records after each successful operation.

[0094] Depending on whether transaction t is accessing a key for the first time, the operation can be handled differently. If it is the first time, and t's first operation is a read, a committed transaction can be selected to read the value from it, thus respecting all transaction dependencies and linearizability. Similarly, if the first operation is a write, the transaction that wrote can be identified and the new write can be placed after it. To this end, for each key on the storage node, an ordering of all transactions that have accessed the key can be maintained, and among the cleared transactions, the value and commit time of the last cleared committed transaction that wrote to the key in the ordering can be stored. These actions can be maintained by the storage node's SNM. The task of identifying the transaction to read from for the first read or the transaction to write after for the first write becomes the task of correctly inserting transaction t into the ordering. Inserting t into the ordering introduces new dependencies, which can be tracked and used for performance checks.

[0095] If transaction t accesses a key for the second or more times, the handling of the transaction becomes easier. If the operation is the first write to the key by the transaction, additional dependencies need to be added to the transaction. For all other operations, a local version of the transaction can be looked up and the value found can be returned. To ensure linearizability, consider three transactions t, t1c, and t2c, where t1c and t2c have been committed and t has not. Suppose there is an edge from t to t1c and an edge from t1c to t2c. Transitivity cannot be used to infer the existence of an edge from t to t2c, which may violate linearizability because t1c may be concurrent with both t and t2c but t2c is committed before t starts. Therefore, when a new transaction is inserted into an existing order, there should be edges from the new transaction to all committed transactions that precede the new transaction in the order.

[0096] As described above, for each key on a storage node, an order of all transactions that have accessed that key can be maintained. Among the purged transactions, the value of the last purged committed transaction that wrote to that key in that order and its purging time can be purged. All transactions with writes, both committed and uncommitted, can be total-ordered. All transactions with only reads can be ranked after the transaction that reads the value of that key. Let t be the first time a transaction (Txn) attempts to operate on a key. After each operation, the SNM can be updated with the transaction's new dependencies and records. Furthermore, the dependencies of the purged records on the transaction with purging time ct can be captured by updating the transaction's mct to max(mct,ct).

[0097] When the first operation of transaction t on a key is a read, the first committed transaction t in the order with a write can be identified c , from t to the transaction t c There is no edge. c Insert t after and before the next transaction with a write. To do this, we can check whether all current dependencies are followed, that is, check whether all outgoing edges of t lead to t c The subsequent transactions, and whether all incoming edges of t come from t c The previous transaction or the transaction concurrent with t. Therefore, the new dependency is (i) from t c All previous transactions to t, (ii) edges from t to t c All subsequent transactions (only from t c For edges of new type (ii), messages are sent to these transactions to ensure that they are not committed. If the transaction has been committed, the process can be repeated to find new t c . Read the value from t c .

[0098] When the first operation of transaction t on a key is a write, identify the first committed transaction t in the order that has a write c , from t to the transaction t c There is no edge. If the next committed transaction with a write from t c If the operation fails, return the result. Otherwise, let t' be t c After that, there is only the first uncommitted transaction with writes from t to t c There is an edge. If there is no such uncommitted transaction, t' is considered to be the next committed transaction. Therefore, t is inserted before t' in the order. To do this, determine whether all current dependencies are followed, that is, check to determine whether all outgoing edges of t go to transactions after t', and whether all incoming edges of t come from transactions before t'. Therefore, the new dependencies are (i) edges from all transactions before t' to t, and (ii) edges from t to all transactions after t'. For edges of the new type (ii), messages are sent to these transactions to ensure that these transactions are not committed. If the transaction has been committed, the process can be repeated to find a new t c The value can be written in the record and the SNM can be updated.

[0099] When processing operations other than the first operation on a key by a transaction, the operations can be divided according to whether the operation is the first write operation on the key by the transaction. If so, it can be determined to check t c (i.e., the transaction after t in the sort)c A transaction (uncommitted or committed) that reads but also has writes. If so, return the case where the operation failed. If not, you can add c All uncommitted reads from t are recorded along the edges to t. Then, writes can be recorded.

[0100] If the operation is a read, the local recorded version of the transaction key is found and, based on the operation ID, the transaction key is retrieved from the past writes to the key or from the t c If the operation is a write, but not the first write, the write is inserted according to the operation ID, while ensuring that no read is done from a past write that is placed after the write.

[0101] For all of these operations, it is important to track which previous transaction read a key to prevent a situation where two transactions read and then write to it, causing both transactions to read from the same transaction. To do this, the process can include tracking those transactions that only read from committed transactions and ensuring that only one transaction reads and writes to the same key from a committed transaction. Figure 12A and Figure 12B Flowchart 1200 illustrates an embodiment similar to the example of starting a transaction and performing operations in an RMCC discussed above.

[0102] In RMCC, when a transaction t issues a commit request, the parts of the transaction priority graph involving the transaction can be merged. In this process, dependencies accumulated when operating on storage nodes locally known to t can be added and used. After merging the relevant parts of the transaction priority graph, it can be checked whether it is safe to commit t. This check can be performed by checking whether there are cycles in the transaction priority graph to ensure serializability. During this check, new edges are added to the transaction priority graph to ensure linear consistency. New edges are created so that if the timestamp of the first transaction (Txn1.timestamp) is less than the timestamp of the second transaction (Txn2.timestamp), a directed edge is inserted to indicate that Txn1 is before Txn2 in the priority graph. The DAG coordinator can be the component in RMCC responsible for handling the process of committing and managing the transaction priority graph.

[0103] The DAG coordinator can maintain a transaction priority graph for a subset of dependent transactions. The DAG coordinator can be responsible for checking whether transaction t can be committed. Upon receiving a commit request from transaction t, t's transaction coordinator can issue a commit request to its DAG coordinator. Upon receiving the request, t's DAG coordinator can obtain a partial transaction priority graph from the DAG coordinator of the dependent transaction. t's DAG coordinator can then build a larger graph that combines all partial graphs and can check whether there is a cycle consisting of committed transactions and t. The DAG coordinator can also be responsible for identifying and clearing transactions. In this clearing process, the DAG coordinator can issue a cleanup request to the transaction coordinator of the transaction identified by the DAG coordinator as no longer useful.

[0104] To account for cycle checking by the DAG coordinator, let t be the transaction attempting to commit. t's DAG coordinator obtains the transaction priority graph from the DAG coordinators of all of t's dependent transactions. After obtaining the transaction priority graph, t's DAG coordinator checks for cycles and adds additional edges, as appropriate, based on timestamp ordering, to ensure external causality.

[0105] For cycle checking, the invariant condition that there are no cycles between committed transactions in any partial DAG is maintained. The reasoning behind this invariant condition is that if a cycle includes an ongoing transaction, the cycle will be detected by the last committed transaction in the cycle. This invariant condition can be maintained to speed up the cycle finding process and thus the commit time, instead of performing a depth first search (DFS) on the entire graph every time. DFS is a recursive algorithm that is used to search all vertices of a graph or tree data structure. In this cycle finding process, when a transaction t attempts to commit, only cycles consisting of committed transactions and t can be checked. In addition, cycle checking can also include ensuring that no cycles are formed that include transactions that have been committed and cleared. The clearing time of committed transactions can be used to ensure that no cycles are formed. When transaction t attempts to commit, the following process can be implemented. All incoming and outgoing edges of t that are reachable through known neighboring transactions of t (i.e., dependencies known to t) can be added to t. Then, for each inner neighbor v of t, in and outer neighbor v out , you can check from v out to v in Is there a path that uses only committed transactions, because such a path would result in a cycle. This process can be simplified based on the following proposition: if there is an existing path from one committed transaction to another that uses committed transactions, then there is an edge between them. If from v out to v in If there is an edge, then transaction t can be aborted because it would cause a cycle. into v out There is an edge, then since from v out to v in There is no path that uses a committed transaction, so t has no problem (no loop). The last case occurs when v in and v out There is no edge between them (i.e. from v in to v out The path does not exist, from v out to v in In this case, the cleanup time and commit time can be checked to ensure linearizability if v in The submission time does not exceed v out , then you can get it from v in to v out Add an edge, otherwise abort t. If for all v in and v out Yes, no cycle is formed, so submit t.

[0106] The DAG coordinator can store information to facilitate commit or abort operations. This information can include a portion of the transaction priority graph. The DAG coordinator can track all committed and uncommitted nodes. For each committed transaction, the DAG coordinator can store the commit time and cleanup time of the transaction. The DAG coordinator can store the transaction coordinator for each transaction in the graph. Figure 13A and Figure 13B A flow chart 1300 is shown of an embodiment of an exemplary commit request in RMCC similar to that discussed above.

[0107] Cleanup can be triggered during each commit and each abort on the DAG coordinator. At the end of a commit request, the DAG coordinator stores a portion of the transaction priority graph. During cleanup, the DAG coordinator identifies and aborts certain uncommitted transactions and cleans up certain committed transactions. Aborted transactions include any uncommitted transaction t that has a committed outer neighbor t. u , the outer neighbor t has the same u Non-adjacent committed outer neighbors t'. That is, from t u There are edges to t and from t to t', but there is no edge from t u There is no edge to t'. Trying to transitively add an edge from t to t' would violate linearizability, causing t u After identifying the uncommitted transactions to be aborted, the cleaned-up transactions include some committed transactions for which cleanup operations may be issued, wherein the cleaned-up committed transactions do not have uncommitted neighbors.

[0108] Before issuing a cleanup request to the transaction coordinator, the cleanup times of all committed transactions can be updated in conjunction with the set of transactions identified for cleanup. Recall that cleanup time (t) is the maximum of commit time (t) and commit time (t) for all committed transactions that can reach cleanup. This update of cleanup times can be performed using a process similar to that used to find a topological sort. All edges can initially be set to non-traversable. Then, in each round, nodes can be iteratively found where all incoming edges from committed transactions have been traversed, and the cleanup times of the found nodes can be shared with their out-neighbors. Neighboring transactions can update their cleanup times accordingly.

[0109] When purging transaction versions, the purge time can be stored in the storage node along with the purged record. That is, during version cleanup, the value of the last purged transaction and its purge time can be stored for each key. This helps ensure that cycles are not formed by purging transactions during cycle checking.

[0110] Before a transaction is completely purged from the transaction coordinator, all uncommitted neighboring transactions of the purged transaction may be notified of the purge time of the transaction. All transaction coordinators may store the status of the transaction being purged and its purge time when the version is being cleaned up.

[0111] If a transaction attempts to commit and cannot find a dependent transaction with an incoming edge on its transaction coordinator, the transaction may be aborted. However, this should not happen because all uncommitted out-neighbors are notified of the purge time when the transaction is purged. Until then, the transaction may remain in the purging state on the transaction coordinator. Figure 14A and Figure 14B A flow chart 1400 is shown illustrating an embodiment of an exemplary cleanup communication in RMCC similar to that discussed above.

[0112] Figure 15 1 is a flow chart of features of an embodiment of an exemplary method 1500 for operating a distributed data storage system. At operation 1510, dependencies between transactions in a distributed system are modeled. The distributed system has storage nodes that are individually arranged in a distributed arrangement. Dependencies between transactions are modeled using a transaction priority graph that is partially constructed when executing transactions. When a transaction affects a common key stored in a storage node, the transaction is marked as a dependent transaction in the transaction priority graph. At operation 1520, in response to a cycle check in the transaction priority graph of the transaction, the transaction is committed in the distributed system.

[0113] Variants of method 1500 or methods similar to method 1500 may include multiple different embodiments, which may be combined depending on the application of such methods and / or the architecture of the device or system implementing such methods. A variant of such a method may include dynamically determining data to be deleted from a distributed system in conjunction with a given transaction priority graph, wherein the given transaction priority graph may model dependencies based on associated keys and transaction commit times. Dynamically determining the data may include determining that no path exists in the given transaction priority graph from an uncommitted transaction in the transaction priority graph to a committed transaction in the transaction priority graph. A variant may include storing data records and unique keys of the data records partitioned across the storage nodes in storage nodes, wherein each storage node includes a subset of the data records. A variant may include maintaining data records, transaction records, and transaction priority map records in each storage node. Each transaction record may have a transaction identifier, and each transaction priority map record may have a transaction priority map identifier. A variant may include issuing a read or write request from a client node of the distributed system to a storage node. The client node may be configured with an interface facing an end user, wherein the end user is external to the distributed system.

[0114] A variation of method 1500 or a method similar to method 1500 may include tracking whether a transaction in a distributed system is in progress, committed, or aborted, and maintaining and updating a transaction priority map for the transaction. In response to detecting that the other transaction priority map affects a key shared with the transaction priority map, the transaction priority map may be combined with the other transaction priority map. A variation may include deleting the transaction and related information from a storage node in response to determining that the transaction is purgeable in the distributed system.

[0115] A variation of method 1500 or a method similar to method 1500 may include finding a partial transaction priority graph that includes neighboring transactions of a transaction, and adding transitive dependency edges to the transaction priority graph of the transaction to generate a combined transaction priority graph for the transaction. Commit times between committed transactions in the partial transaction priority graph may be examined, and edges may be added based on the commit times. A cycle check may be performed in the combined transaction priority graph of the transaction, and a determination of whether to commit or abort may be made based on the cycle check.

[0116] A variation of method 1500 or a method similar to method 1500 may include: executing stored instructions by one or more first processors to execute multiple transaction coordinators, such that each active transaction in the distributed system has a transaction coordinator that tracks whether the active transaction is in progress, committed, or aborted; and executing stored instructions by one or more second processors to execute multiple DAG coordinators, such that each DAG coordinator tracks a transaction priority graph and updates and combines transaction priority graphs among other DAG coordinators. At the start of a given transaction, the transaction coordinator for the given transaction may be assigned as the DAG coordinator for the given transaction. A variation may include: for a given transaction requested by a client node of the distributed system, in response to a commit request for the given transaction from the client node, the transaction coordinator for the given transaction determines the current state of the given transaction by examining the state of the given transaction in a transaction record of the given transaction. The transaction coordinator for the given transaction may communicate with the DAG coordinator for the given transaction to evaluate whether committing the given transaction forms a cycle in the given transaction priority graph. If no cycle is formed in the given transaction priority graph, the transaction coordinator may implement the commit of the given transaction.

[0117] In various embodiments, a non-transitory machine-readable storage device (e.g., a computer-readable non-transitory medium) may include instructions stored thereon that, when executed by a machine, cause the machine to perform operations, wherein the operations include one or more features similar to or identical to features of the methods and techniques described in conjunction with method 1500 and its variants and / or features of other methods described herein. The physical structures of such instructions may be operated on by at least one processor. For example, executing the physical structures may cause the machine to perform operations including: modeling dependencies between transactions in a distributed system having storage nodes that are separately arranged in a distributed arrangement; using a transaction priority graph that is partially constructed when executing transactions, wherein when a transaction affects a common key stored in a storage node, the transaction is marked as a dependent transaction in the transaction priority graph; and committing a transaction in the distributed system in response to a loop check in the transaction priority graph of the transaction.

[0118] The operations may include dynamically determining data to be deleted from a distributed system in conjunction with a given transaction priority graph, wherein the given transaction priority graph may model dependencies based on associated keys and transaction commit times. Dynamically determining the data may include determining that no path exists in the given transaction priority graph from an uncommitted transaction in the transaction priority graph to a committed transaction in the transaction priority graph. The operations may include storing data records and unique keys of the data records partitioned across the storage nodes in storage nodes, wherein each storage node includes a subset of the data records. The operations may include maintaining data records, transaction records, and transaction priority graph records in each storage node, wherein each transaction record has a transaction identifier and each transaction priority graph record has a transaction priority graph identifier. The operations may include, in response to determining that the transaction is purgeable in the distributed system, deleting the transaction and related information from the storage node. The operations may include issuing a read or write request from a client node of the distributed system to the storage node, wherein the client node may be configured with an interface facing an end user, the end user being located external to the distributed system.

[0119] The operations may include tracking whether transactions in the distributed system are in progress, committed, or aborted, and maintaining and updating a transaction priority map for the transactions. The operations may include combining the transaction priority map with other transaction priority maps in response to detecting that the other transaction priority maps affect a key in common with the transaction priority map.

[0120] The operations may include: finding a partial transaction priority graph that includes adjacent transactions of the transaction; adding transitive dependency edges to the transaction priority graph of the transaction to generate a combined transaction priority graph of the transaction; checking commit times between committed transactions of the partial transaction priority graph and adding edges based on the check of commit times; performing a cycle check in the combined transaction priority graph of the transaction; and determining whether to commit or abort based on the cycle check.

[0121] The operations may include: running multiple transaction coordinators such that each active transaction in the distributed system has a transaction coordinator that tracks whether the active transaction is in progress, committed, or aborted; running multiple directed acyclic graph (DAG) coordinators such that each DAG coordinator tracks a transaction priority graph and updates and combines transaction priority graphs between other DAG coordinators. The operations may include: at the start of a given transaction, assigning the transaction coordinator of the given transaction as the DAG coordinator of the given transaction. The operations may include: for a given transaction requested by a client node of the distributed system, in response to a commit request for the given transaction from the client node, the transaction coordinator of the given transaction performs the following steps: determining the current state of the given transaction by checking the state of the given transaction in the transaction record of the given transaction; communicating with the DAG coordinator of the given transaction to evaluate whether the commit of the given transaction forms a cycle in the given transaction priority graph; and if no cycle is formed in the given transaction priority graph, implementing the commit of the given transaction.

[0122] In various embodiments, a distributed system may include storage nodes separately arranged in a distributed arrangement, a memory storing instructions, and at least one processor in communication with the memory. The at least one processor may be configured to perform multiple steps when executing the instructions. Dependencies between transactions in the distributed system may be modeled using a transaction priority graph partially constructed when executing the transactions. The transactions may be associated with keys stored in the storage nodes. In response to a cycle check in the transaction priority graph for the transaction, one of the transactions in the distributed system may be committed, wherein the transaction is associated with the key.

[0123] Variants of this distributed system or similar distributed systems may include multiple different embodiments, which may or may not be combined, depending on the application of such distributed system and / or the architecture of the distributed system implementing the methods described herein. In such a distributed system, at least one processor may be configured to dynamically determine data to be deleted from the distributed system in conjunction with a given transaction priority graph, wherein the given transaction priority graph may model dependencies based on associated keys and transaction commit times. The data to be deleted may be determined by determining that no path exists in the given transaction priority graph from an uncommitted transaction in the transaction priority graph to a committed transaction in the transaction priority graph. In such a distributed system, storage nodes may include data records and unique keys of the data records partitioned across the storage nodes, wherein each storage node includes a subset of the data records. Each storage node may include data records, transaction records, and transaction priority map records, wherein each transaction record may have a transaction identifier, and each transaction priority map record may have a transaction priority map identifier. The at least one processor may be configured to delete the transaction and related information from the storage node in response to determining that the transaction is purgeable in the distributed system. In a variant of such a distributed system, the distributed system may include a client node, wherein the client node is used to issue read and write requests to the storage node, and the client node is arranged with an interface facing an end user, and the end user is located outside the distributed system.

[0124] A variant of this distributed system or a similar distributed system may include at least one processor configured to track whether transactions in the distributed system are in progress, committed, or aborted, and to maintain and update a transaction priority map for the transactions. The at least one processor may be configured to combine the transaction priority map with the other transaction priority map in response to detecting that the other transaction priority map affects a key in common with the transaction priority map.

[0125] A variant of this distributed system or a similar distributed system may include: at least one processor configured to find a partial transaction priority graph that includes neighboring transactions of a transaction, and add transitive dependency edges to the transaction priority graph of the transaction to generate a combined transaction priority graph of the transaction. The at least one processor may be configured to: check commit times between committed transactions in the partial transaction priority graph and add edges based on the check of commit times; perform a cycle check in the combined transaction priority graph of the transaction. The at least one processor may be configured to determine whether to commit or abort based on the cycle check.

[0126] A variant of this distributed system or a similar distributed system may include: at least one processor, when executing instructions, configured to operate as multiple transaction coordinators and multiple directed acyclic graph (DAG) coordinators, such that each active transaction in the distributed system has a transaction coordinator that tracks whether the active transaction is in progress, committed, or aborted, and each DAG coordinator tracks a transaction priority graph and updates and combines transaction priority graphs between other DAG coordinators. In a variant of this distributed system, at the start of a given transaction, the transaction coordinator of the given transaction may be assigned as the DAG coordinator of the given transaction. In a variant of this distributed system, for a given transaction requested by a client node of the distributed system, in response to a commit request for the given transaction from the client node, the transaction coordinator of the given transaction may perform multiple functions. The transaction coordinator may determine the current state of the given transaction by examining the state of the given transaction in the transaction record of the given transaction. The transaction coordinator of the given transaction may communicate with the DAG coordinator of the given transaction to evaluate whether the commit of the given transaction forms a cycle in the given transaction priority graph. If no cycles are formed in the given transaction priority graph, the transaction coordinator of the given transaction may implement the commit of the given transaction.

[0127] Figure 16 The block diagram of the components of the computing system 1600 is shown. The computing system 1600 can implement algorithms and execute methods configured to process data for applications and in combination with RMCC for data processing. Not all components need to be used in various embodiments. The computing system 1600 may include a processor 1601, a memory 1612, a removable storage 1623, a non-removable storage 1622, and a cache 1628. The processor 1601 may be implemented as multiple processors. In different embodiments, the computing system 1600 may be configured in different forms. The computing system 1600 may be combined with Figure 9 Distributed systems 900 and Figure 10 1000. Although various data storage elements are shown as part of the computing system 1600, storage may also or alternatively include cloud-based storage accessible via a network, such as the Internet or remote server-based storage.

[0128] The memory 1612 may include a volatile memory 1614 and / or a non-volatile memory 1617. The computing system 1600 may include various computer-readable media, such as the volatile memory 1614, the non-volatile memory 1617, the removable storage 1623, and / or the non-removable storage 1622, or a computing environment having access to various computer-readable media, such as the volatile memory 1614, the non-volatile memory 1617, the removable storage 1623, and / or the non-removable storage 1622. Computer storage may include data storage servers, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD ROM), digital versatile disk (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium capable of storing computer-readable instructions.

[0129] Computing system 1600 may include an input interface 1627, an output interface 1624, and a communication interface 1631, or have access to a computing environment that includes input interface 1627, output interface 1624, and communication interface 1631. Output interface 1624 may include a display device such as a touch screen that can also function as an input device. Input interface 1627 may include one or more of the following: a touch screen, a touchpad, a mouse, a keyboard, a camera, one or more device-specific buttons, one or more sensors integrated within computing system 1600 or coupled to computing system 1600 via a wired or wireless data connection, and other input devices. Communication interface 1631 may communicate with external devices and networks. Computing system 1600 may operate in a networked environment via a communication connection to connect to one or more remote computers, such as one or more remote computing nodes. Remote computers may include PCs, servers, routers, network PCs, peer devices, or other common data flow network switches. Communication connections may include local area networks (LANs), wide area networks (WANs), cellular, Wi-Fi, Bluetooth, or other networks. In various embodiments, the components of computing system 1600 may be connected via a system bus 1621 .

[0130] Computer-readable instructions stored on a computer-readable medium, such as program 1613, can be executed by processor 1601 of computing system 1600. A hard disk, a CD-ROM, and RAM are some examples of items that include non-transitory computer-readable media (e.g., storage devices). The terms "computer-readable medium" and "storage device" do not include carrier waves, as carrier waves are considered volatile. Storage can also include network storage, such as a storage area network (SAN). Program 1613 of computing system 1600 can be used to cause processor 1601 to perform one or more methods or algorithms described herein.

[0131] The components of the illustrative devices, systems, and methods employed in accordance with the illustrated embodiments may be implemented at least in part in digital electronic circuitry, analog electronic circuitry, or computer hardware, firmware, software, or a combination thereof. For example, these components may be implemented as a computer program product (e.g., a computer program, program code, or computer instructions) tangibly embodied in a machine-readable storage device for execution by a data processing apparatus, such as a programmable processor, a computer, or multiple computers, or for controlling the operation of a data processing apparatus.

[0132] The various illustrative logical blocks, modules, and circuits described in conjunction with the embodiments disclosed herein may be implemented or performed with one or more general purpose processors, digital signal processors (DSPs), ASICs, field-programmable gate arrays (FPGAs) or other programmable logic devices, discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. A general purpose processor may be a microprocessor, but in the alternative, the processor may be any conventional processor, controller, microcontroller, or state machine. A processor may also be implemented as a combination of computing devices, for example, a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors combined with a DSP core, or any other such configuration.

[0133] Processors suitable for executing computer programs include general-purpose and special-purpose microprocessors, as well as any processor of any type of digital computer. Typically, a processor receives instructions and data from read-only memory and / or random access memory. Elements of a computer include a processor for executing instructions and one or more memory devices for storing instructions and data. Typically, a computer also includes one or more mass storage devices (e.g., magnetic, magneto-optical, or optical disks) for storing data, or is operatively coupled to one or more mass storage devices for storing data to receive data from and / or transmit data to the mass storage devices. Devices suitable for embodying computer program instructions and data include all forms of memory, for example, semiconductor memory devices such as electrically programmable read-only memory or ROM (EPROM), EEPROM, flash memory devices, and / or data storage disks (e.g., magnetic disks, internal hard disks, or removable magnetic, magneto-optical, CD-ROM, and DVD-ROM disks). The processor and memory may be supplemented by or incorporated into special-purpose logic circuitry.

[0134] The RMCC framework described in this paper addresses the inefficient memory usage of traditional distributed storage systems by immediately purging transaction versions when they are not part of active processing, while maintaining serializability and external causality. Using a transaction precedence graph, RMCC determines whether a commit would create a dependency cycle and thus violate serializability. The same transaction precedence graph can be used to purge multiple record versions, such that older record versions in a given committed transaction are purged from storage when no path exists from any uncommitted transaction to the given committed transaction in the transaction precedence graph. The purge process can be triggered by commit or abort requests generated by a transaction, ensuring that multiple record versions are retained long enough to satisfy any outstanding transactions without extending their duration. RMCC supports serializable isolation levels, external consistency (linearizability), increased concurrency, and global transactions (i.e., transactions spanning multiple geographic regions), while achieving efficient memory usage by purging versions as needed.

[0135] Current state-of-the-art systems cannot achieve both serializability and linearizability without incurring significant performance issues. RMCC provides a mechanism to partition transaction accounting, making this process distributed and scalable. Transaction relations are maintained for all ongoing transactions. This maintenance is performed by retaining fragments of the entire image (partial DAGs) in different nodes of the system. These DAGs represent the order in which transactions were recorded in the system to maintain serializability and linearizability properties. When a transaction attempts to commit, it is evaluated against a combined DAG consisting of all current partial DAGs. Any transaction is allowed to commit only if it does not form a cycle with committed transactions in the combined DAG. If a transaction commits, it remains in the committed state in the DAG so that the remaining ongoing transactions that are part of the DAG can evaluate whether a cycle will be formed when attempting to commit.

[0136] Although specific embodiments have been illustrated and described herein, it will be understood by those skilled in the art that any arrangement calculated to achieve the same purpose may be substituted for the specific embodiments shown. Various embodiments utilize permutations and / or combinations of the embodiments described herein. It should be understood that the foregoing description is intended to be illustrative, not restrictive, and that the wording or terminology used herein is for descriptive purposes. Combinations of the foregoing embodiments and other embodiments will be apparent to those skilled in the art upon studying the foregoing description.

Claims

1. A distributed system comprising: Storage nodes arranged separately in a distributed arrangement; a memory for storing instructions; At least one processor in communication with the memory, wherein the at least one processor, when executing the instructions, is configured to perform the following steps: modeling dependencies between the transactions in the distributed system using a transaction priority graph partially constructed when executing transactions, wherein the transactions are associated with keys stored in the storage nodes; In response to checking for a cycle in a transaction priority graph of transactions associated with the key in the transaction, committing the transaction in the distributed system.

2. The distributed system according to claim 1, wherein: The at least one processor is used to dynamically determine data to be deleted from the distributed system in combination with a given transaction priority graph by determining that no path exists in the given transaction priority graph from an uncommitted transaction in the transaction priority graph to a committed transaction in the transaction priority graph, wherein the given transaction priority graph models dependencies based on associated keys and transaction commit times.

3. A distributed system according to any one of the preceding claims, wherein: The storage nodes include data records and unique keys of the data records partitioned among the storage nodes, wherein each storage node includes a subset of the data records.

4. A distributed system according to any one of the preceding claims, wherein: The at least one processor is configured to: Tracking whether the transaction in the distributed system is in progress, committed, or aborted; The transaction priority map for the transaction is maintained and updated, and in response to detecting that other transaction priority maps affect a key in common with the transaction priority map, the transaction priority map is combined with the other transaction priority maps.

5. A distributed system according to any one of the preceding claims, wherein: The at least one processor is configured to, in response to determining that the transaction is purgeable in the distributed system, delete the transaction and related information from the storage node.

6. A distributed system according to any one of the preceding claims, wherein: The distributed system comprises a client node, wherein the client node is used to issue a read and write request to the storage node, and the client node is arranged with an interface facing an end user, and the end user is located outside the distributed system.

7. A distributed system according to any one of the preceding claims, wherein: The at least one processor is configured to: Finding a partial transaction priority graph including adjacent transactions of the transaction; adding a transitive dependency edge to the transaction priority graph of the transaction to generate a combined transaction priority graph of the transaction; checking commit times between committed transactions of the portion of the transaction priority graph, and adding edges based on the checking of the commit times; performing a cycle check in the combined transaction priority graph of the transaction; Determine whether to commit or abort based on the loop check.

8. A distributed system according to any one of the preceding claims, wherein: Each storage node includes: Data recording; Transaction records, where each transaction record has a transaction identifier; Transaction priority map records, where each transaction priority map record has a transaction priority map identifier.

9. A distributed system according to any one of the preceding claims, wherein: The at least one processor is configured to operate as a plurality of transaction coordinators and a plurality of directed acyclic graph (DAG) coordinators when executing the instructions, such that each active transaction in the distributed system has a transaction coordinator, the transaction coordinator tracks whether the active transaction is in progress, committed, or aborted, and each DAG coordinator tracks a transaction priority graph and updates and combines transaction priority graphs between other DAG coordinators.

10. The distributed system according to claim 9, wherein: When a given transaction starts, the transaction coordinator of the given transaction is assigned as the DAG coordinator of the given transaction.

11. The distributed system according to claim 9, wherein: For a given transaction requested by a client node of the distributed system, in response to a commit request for the given transaction from the client node, the transaction coordinator of the given transaction is configured to: determining a current state of the given transaction by examining a state of the given transaction in a transaction record of the given transaction; communicating with the DAG coordinator of the given transaction to evaluate whether a commit of the given transaction forms a cycle in a given transaction priority graph of the given transaction; If no cycle is formed in the given transaction priority graph, committing the given transaction is performed.

12. A method for operating a distributed data storage system, the method comprising: Modeling dependency relationships between transactions in a distributed system using a transaction priority graph partially constructed when executing transactions, wherein the distributed system has storage nodes separately arranged in a distributed arrangement, and when a transaction affects a common key stored in the storage node, the transaction is marked as a dependent transaction in the transaction priority graph; In response to checking for a cycle in the transaction priority graph of the transaction, committing the transaction in the distributed system.

13. The method according to claim 12, wherein: The method includes dynamically determining data to be deleted from the distributed system in combination with a given transaction priority graph by determining that no path exists in the given transaction priority graph from an uncommitted transaction in the transaction priority graph to a committed transaction in the transaction priority graph, wherein the given transaction priority graph models dependencies based on associated keys and transaction commit times.

14. The method according to claim 12 or 13, wherein: The method includes storing data records and unique keys of the data records partitioned among the storage nodes in the storage nodes, wherein each storage node includes a subset of the data records.

15. The method according to any one of claims 12 to 14, wherein: The method comprises: Tracking whether the transaction in the distributed system is in progress, committed, or aborted; The transaction priority map for the transaction is maintained and updated, and in response to detecting that other transaction priority maps affect a key in common with the transaction priority map, the transaction priority map is combined with the other transaction priority maps.

16. The method according to any one of claims 12 to 15, wherein: The method includes, in response to determining that the transaction is purgeable in the distributed system, deleting the transaction and related information from the storage node.

17. The method according to any one of claims 12 to 16, wherein: The method comprises: A read or write request is issued from a client node of the distributed system to the storage node, wherein the client node is arranged with an interface facing an end user, and the end user is located outside the distributed system.

18. The method according to any one of claims 12 to 17, wherein: The method comprises: Finding a partial transaction priority graph including adjacent transactions of the transaction; adding a transitive dependency edge to the transaction priority graph of the transaction to generate a combined transaction priority graph of the transaction; checking commit times between committed transactions of the portion of the transaction priority graph, and adding edges based on the checking of the commit times; performing a cycle check in the combined transaction priority graph of the transaction; Determine whether to commit or abort based on the loop check.

19. The method according to any one of claims 12 to 18, wherein: The method includes maintaining data records, transaction records, and transaction priority map records in each storage node, wherein each transaction record has a transaction identifier, and each transaction priority map record has a transaction priority map identifier.

20. The method according to any one of claims 12 to 19, wherein: The method comprises: running a plurality of transaction coordinators by executing stored instructions by one or more first processors such that each active transaction in the distributed system has a transaction coordinator that tracks whether the active transaction is in progress, committed, or aborted; Multiple directed acyclic graph (DAG) coordinators are run by executing the stored instructions by one or more second processors, so that each DAG coordinator tracks the transaction priority graph and updates and combines the transaction priority graphs between other DAG coordinators.

21. The method according to claim 20, wherein The method includes: when a given transaction starts, allocating the transaction coordinator of the given transaction as the DAG coordinator of the given transaction.

22. The method according to claim 20, wherein The method includes: for a given transaction requested by a client node of the distributed system, in response to a commit request for the given transaction from the client node, the transaction coordinator of the given transaction executes: determining a current state of the given transaction by examining a state of the given transaction in a transaction record of the given transaction; communicating with the DAG coordinator of the given transaction to evaluate whether a commit of the given transaction forms a cycle in a given transaction priority graph of the given transaction; If no cycle is formed in the given transaction priority graph, committing the given transaction is performed.

23. A non-transitory computer-readable storage medium storing instructions, wherein: The instructions, when executed by one or more processors, cause the one or more processors to perform operations including the method of any one of claims 12 to 22.

Images