System and method for global consistency in a distributed shared data database

Abstract

An apparatus, system, and method are provided for global consistency in a distributed shared data database. According to one aspect, a method for ensuring strong consistency in data retrieval can be provided. The method includes: a receiving node of the database receiving a query for the value of a data item in the database. The method may further include: the receiving node requesting a latest timestamp corresponding to the latest transaction from each of a plurality of nodes. The method may further include: the receiving node receiving the latest timestamp from each of the plurality of nodes. The method may further include: the receiving node managing a cache based on each received latest timestamp. The method may further include: the receiving node reading the value of the data item.

Description

[0001] This application claims priority to U.S. Patent Application No. 18 / 406,915, filed January 8, 2024, which is incorporated herein by reference in its entirety. Technical Field

[0002] This invention relates to the field of distributed databases, and more particularly to systems and methods for achieving global consistency in distributed shared data databases. Background Technology

[0003] Data consistency is a critical challenge in distributed databases. It involves the complex task of ensuring data remains accurate and up-to-date across all nodes and locations within the distributed database. However, the decentralized nature of distributed databases can delay this synchronization, leading to discrepancies throughout the system. Maintaining data consistency is crucial because it impacts decision-making and data integrity. Pursuing strong consistency—where every part of the system reads the same data simultaneously—can degrade system performance. Furthermore, managing network latency, hardware issues, and concurrent data access further complicates the process.

[0004] Therefore, it is necessary to improve the global consistency of distributed shared data databases.

[0005] This background information is provided to disclose information that the applicant believes may be relevant to the present invention. It is not intended to acknowledge, nor should it be construed as, prior art in relation to the present invention. Summary of the Invention

[0006] Apparatus, systems, and methods for achieving global consistency in distributed shared data databases can be provided. According to one aspect, a method for ensuring strong consistency in data retrieval can be provided. The method may include: a receiving node receiving a query for reading the value of a specific data item among a plurality of data items. The receiving node may be one of a plurality of nodes in a distributed database storing the plurality of data items. The method may further include: the receiving node requesting from each of the plurality of nodes (excluding itself) the latest timestamp of a corresponding latest transaction executed by each of the corresponding nodes (excluding itself). The method may further include: the receiving node receiving the corresponding latest timestamp of the corresponding latest transaction from each of the plurality of nodes (excluding itself). The method may further include: the receiving node managing a cache based on the corresponding latest timestamp of the corresponding latest transaction at each corresponding node among the plurality of nodes (excluding itself), according to the transactions executed by each of the plurality of nodes (excluding itself). The cache of the receiving node may include a set of values ​​corresponding to a set of data items among the plurality of data items. The method may further include: the receiving node reading the value of the specific data item.

[0007] Managing the cache of the receiving node may include one of the following operations: invalidating one or more values ​​in the set of values ​​in the cache, or updating one or more values ​​in the set of values ​​in the cache. The distributed database may be one of the following types: a shared data database, a shared-nothing database, a multi-write database, and a single-write multi-read database. The query may be a read-after-write (RAW) query. The latest timestamp may be a clock count value corresponding to the clock of each of the plurality of nodes other than the receiving node.

[0008] The clock corresponding to each of the plurality of nodes other than the receiving node can be either a vector clock or a vector-scalar clock, and the corresponding latest timestamp is a vector timestamp. The vector timestamp can include an array of values, where each value can correspond to a different node among the plurality of nodes in the distributed database. Wherein, the corresponding latest timestamp is a vector timestamp, and the receiving node manages the cache of the plurality of data items by: for each vector timestamp received from each of the plurality of nodes other than the receiving node, the receiving node processes the cache of the plurality of data items based on the maximum value in the value array corresponding to each of the plurality of nodes other than the receiving node.

[0009] The receiving node reading the value of the specific data item may include: the receiving node obtaining the value of the specific data item from one of the following: the cache of the plurality of data items, or the data storage of the distributed database. The receiving node reading the value of the specific data item may also include: the receiving node sending a reply, the reply including the obtained value of the specific data item.

[0010] The corresponding latest timestamp can be based on one or more of the following: a global clock or a local clock. The corresponding latest transaction can be the most recently committed transaction.

[0011] The method may further include: the receiving node determining that there is no ongoing request for the latest timestamp sent by the receiving node to one or more of the plurality of nodes. The method may further include: the receiving node adding the query to a current timestamp request queue used for collecting the latest timestamp. The current timestamp request queue may be a queue of one or more queries currently being served for collecting the latest timestamp. The method may further include: after the receiving node receives the corresponding latest timestamp of the corresponding latest transaction from each of the plurality of nodes other than the receiving node, the receiving node removes the query from the current timestamp request queue. The method may further include: the receiving node determining whether one or more queries exist in a next timestamp request queue. The next timestamp request queue may be a queue of one or more queries waiting to be transferred to the current timestamp request queue.

[0012] According to another aspect, a method for managing queries can be provided. The method includes: a receiving node among multiple nodes of a distributed database storing multiple data items receiving a first query for reading the values ​​of data items among the multiple data items. The method may further include: the receiving node determining whether one or more requests sent by the receiving node to one or more nodes among the multiple nodes are still in progress, each of the one or more requests requesting a latest timestamp corresponding to the latest transaction at a corresponding node among the one or more nodes. The method may further include: the receiving node receiving a set of responses from a group of nodes among the multiple nodes, each response corresponding to one node in the group, and based on the receiving node's request for the latest timestamp corresponding to the latest transaction executed with the corresponding node. Each response also indicates the latest timestamp. The method may further include: the receiving node managing a cache based on the received set of responses, the cache indicating a set of values ​​for a group of data items among the multiple data items.

[0013] Determining whether one or more requests sent by the receiving node to one or more of the plurality of nodes are still in progress may include: the receiving node determining that one or more requests for the latest timestamp sent by the receiving node to the one or more of the plurality of nodes are still in progress. The one or more requests may correspond to one or more queries in the current timestamp request queue. The current timestamp request queue may be a queue of one or more queries currently serving to collect the latest timestamp, and the one or more queries in the current timestamp request queue were received by the receiving node before the first query. The received set of responses may correspond to the one or more queries in the current timestamp request queue. The method may further include: the receiving node adding the first query to the next timestamp request queue. The next timestamp request queue may be a queue waiting to be transferred to the current timestamp request queue.

[0014] The set of responses from one group of the plurality of nodes may correspond to one or more ongoing requests sent by the receiving node to one or more of the plurality of nodes. The method may further include: the receiving node removing the one or more queries from the current timestamp request queue. The method may further include: the receiving node executing the one or more queries removed from the current timestamp request queue. The method may further include: the receiving node determining that one or more queries including the first query exist in the next timestamp request queue. The method may further include: the receiving node moving the one or more queries in the next timestamp request queue from the next timestamp request queue to the current timestamp request queue.

[0015] Determining whether one or more requests sent by the receiving node to one or more of the plurality of nodes are still in progress may include: the receiving node determining that there are no ongoing requests for the latest timestamp sent by the receiving node to the one or more of the plurality of nodes. The method may further include: the receiving node adding the first query to a current timestamp request queue. The current timestamp request queue may be a queue of one or more queries currently being served to collect the latest timestamp. The method may further include: the receiving node requesting the latest timestamp of the corresponding latest transaction executed by each of the plurality of nodes, excluding the receiving node itself.

[0016] Receiving the set of replies from the set of nodes among the plurality of nodes may include: the receiving node receiving the latest timestamp of the corresponding latest transaction from each node among the plurality of nodes other than itself. Managing the receiving node's cache based on the received set of replies may include: the receiving node managing its cache based on the replies received from each node among the plurality of nodes other than itself.

[0017] The method may further include: the receiving node removing the first query from the current timestamp request queue. The method may further include: the receiving node executing the first query. The method may further include: the receiving node determining whether one or more queries exist in the next timestamp request queue. The next timestamp request queue may be a queue waiting for one or more queries to be transferred to the current timestamp request queue.

[0018] According to another aspect, an apparatus may be provided. The apparatus includes one or more modules or electronic components for performing the methods and systems described herein.

[0019] According to one aspect, an apparatus may be provided, comprising: one or more processors and a memory for storing instructions, the instructions, when executed by the one or more processors, causing the apparatus to perform the following operations: a receiving node receives a query for reading the value of a specific data item from a plurality of data items, the receiving node being one of a plurality of nodes in a distributed database storing the plurality of data items. The instructions, when executed by the one or more processors, further cause the apparatus to perform the following operations: the receiving node requests from each of the plurality of nodes, excluding itself, the latest timestamp corresponding to the most recently executed transaction of each respective node. The instructions, when executed by the one or more processors, further cause the apparatus to perform the following operations: the receiving node receives from each of the plurality of nodes, excluding itself, the corresponding latest timestamp of the corresponding latest transaction. When executed by the one or more processors, the instruction further causes the device to perform the following operations: the receiving node manages its cache based on the latest timestamp corresponding to the latest transaction at each corresponding node among the plurality of nodes other than the receiving node, according to the transactions executed by each node among the plurality of nodes other than the receiving node, the cache of the receiving node comprising a set of values ​​corresponding to a set of data items among the plurality of data items. When executed by the one or more processors, the instruction further causes the device to perform the following operations: the receiving node reads the value of the specific data item.

[0020] According to one aspect, an apparatus may be provided, wherein the apparatus includes a non-transitory computer-readable medium storing instructions that, when executed by a processor, cause the apparatus to perform the following operations: a receiving node receives a query for reading the value of a specific data item from a plurality of data items, the receiving node being one of a plurality of nodes in a distributed database storing the plurality of data items; the receiving node requests from each of the plurality of nodes, excluding itself, the latest timestamp corresponding to the most recently executed transaction of each corresponding node; the receiving node receives from each of the plurality of nodes, excluding itself, the latest timestamp of the corresponding latest transaction of the latest transaction; the receiving node manages a cache of the receiving node based on the transactions executed by each of the plurality of nodes, excluding itself, and based on the latest timestamp corresponding to the latest transaction at each corresponding node of the plurality of nodes, excluding itself, the cache of the receiving node including a set of values ​​corresponding to a set of data items from the plurality of data items; and the receiving node reads the value of the specific data item.

[0021] In some embodiments, when the instructions are executed by the processor, the device also causes the receiving node to perform the following operations: the receiving node manages the receiving node's cache by: invalidating one or more values ​​in the set of values ​​in the cache, or updating one or more values ​​in the set of values ​​in the cache.

[0022] According to another aspect, an apparatus can be provided, wherein the apparatus includes a non-transitory computer-readable medium storing instructions that, when executed by a processor, cause the apparatus to perform the following operations: a receiving node among a plurality of nodes of a distributed database storing a plurality of data items receives a first query for reading the value of a data item among the plurality of data items; the receiving node determines whether one or more requests sent by the receiving node to one or more nodes among the plurality of nodes are still in progress, each of the one or more requests requesting a latest timestamp corresponding to the latest transaction at a corresponding node among the one or more nodes; the receiving node receives a set of responses from a set of nodes among the plurality of nodes, each response corresponding to one node in the set of nodes, and based on the receiving node's request for the latest timestamp, the latest timestamp corresponding to the latest transaction executed by the corresponding node, each response further indicating the latest timestamp; the receiving node manages a cache based on the received set of responses, the cache indicating a set of values ​​for a set of data items among the plurality of data items.

[0023] In some embodiments, determining whether one or more requests sent by the receiving node to one or more of the plurality of nodes are still in progress includes: the receiving node determining that there are no ongoing requests for the latest timestamp sent by the receiving node to the one or more of the plurality of nodes. Furthermore, when executed by the processor, the instruction may also cause the apparatus to perform the following operations: the receiving node adds the first query to a current timestamp request queue, the current timestamp request queue including one or more queries currently being served for collecting the latest timestamp; the receiving node requests the latest timestamp corresponding to the latest transaction executed by each of the plurality of nodes other than the receiving node, wherein: the receiving node receiving the set of responses from the set of nodes in the plurality of nodes includes: the receiving node receiving the latest timestamp of the corresponding latest transaction from each of the plurality of nodes other than the receiving node; the receiving node managing its cache based on the received set of responses includes: the receiving node managing its cache based on the responses received from each of the plurality of nodes other than the receiving node.

[0024] According to another aspect, a computer-readable medium may be provided, wherein the computer-readable medium stores program code executable by a device, the program code being used to perform one or more of the methods and systems described herein.

[0025] According to one aspect, a chip may be provided, wherein the chip includes a processor and a data interface, the processor reading instructions stored in a memory via the data interface to execute one or more of the methods and systems described herein. The aspect may also include the memory.

[0026] Other aspects of the invention provide apparatus and systems for implementing the methods according to the first aspect disclosed herein. For example, wireless stations and access points may be configured with machine-readable storage including instructions that, when executed by a processor of these devices, cause the devices to perform one or more of the methods and systems described herein.

[0027] Embodiments have been described above in conjunction with various aspects of the present invention, and these embodiments can be implemented based on these aspects. Those skilled in the art will understand that embodiments can be implemented in conjunction with the aspects described therein, but may also be implemented together with other embodiments of that aspect. It will be apparent to those skilled in the art that embodiments are mutually exclusive or incompatible with each other. Some embodiments may be described in conjunction with one aspect, but may also be applicable to other aspects, which will be apparent to those skilled in the art. Attached Figure Description

[0028] Further features and advantages of the invention will become apparent from the following detailed description, taken in conjunction with the accompanying drawings, in which: Figure 1 A schematic diagram of a distributed database is shown; Figure 2 An example of a data consistency problem is shown; Figure 3 This illustrates one aspect of a method for maintaining data consistency in a distributed database; Figure 4 A flowchart is shown for one aspect of a method for managing query requests; Figure 5A This illustrates one aspect of a method for maintaining data consistency in a distributed database; Figure 5B This illustrates one aspect of a method for managing queries at nodes in a distributed database; Figure 6 The invention provides apparatuses for performing any or all of the methods and features described herein, whether explicitly or implicitly.

[0029] It should be noted that the same features are identified by the same reference numerals throughout the accompanying drawings. Detailed Implementation

[0030] Apparatus, systems, and methods for global consistency in distributed shared data databases can be provided. According to one aspect, a method based on flowchart 400 can be provided for managing queries received at nodes in a database. The method may include: receiving a first query at a node for reading data at a specific time. The node may determine whether one or more timestamp requests are in progress. One or more timestamp requests may refer to one or more requests for the latest timestamp (or clock count value). If one or more timestamp requests are in progress, the first query is placed in a queue for the next requested timestamp. If no timestamp requests are in progress, the node may move the first query to the current timestamp request queue. The node may then issue a timestamp request, which may include sending a request for the latest clock count value to each node in the database. The node may then wait for the request to complete and obtain the latest clock count value. The node may then update or invalidate its cache based on the received clock count value and execute the query. After receiving the latest clock count value, the node may check whether any queries are in the next timestamp request queue. If there are no queries in the next timestamp request queue, the node may wait until another user query arrives. If a query exists in the next timestamp request queue, the node can move the query from the next timestamp request queue to the current timestamp request queue. The node can then issue another timestamp request.

[0031] According to another aspect, a method 500 for ensuring strong consistency in data retrieval can be provided. The method includes: a receiving node among a plurality of nodes in a database receiving a query for the value of a data item stored in the database. The method may further include: the receiving node requesting from each of the plurality of nodes a latest timestamp corresponding to the latest transaction associated with each of the corresponding nodes. The method may further include: the receiving node receiving the latest timestamp from each of the plurality of nodes. The method may further include: the receiving node managing a cache based on each received latest timestamp. The method may further include: the receiving node reading the value of the data item.

[0032] A distributed database can refer to any software that runs simultaneously on different hardware nodes connected via a network, allowing client applications to write, read, and modify previously written data. A shared data database can refer to a database where all database nodes have the same access rights to all database data. In contrast to a shared data database, in a shared-nothing approach (or shared-nothing database), each node has a dedicated, exclusive subset of data. A master-slave database model refers to a database where access to each piece of data is not exclusive, but there are dedicated nodes guaranteed to have the latest version of the data. A node can refer to a machine capable of performing one or more of the following: writing, reading, and routing. A node can also refer to one or more proxy nodes that hide other nodes and provide routing.

[0033] Figure 1 A schematic diagram of a distributed database 100 is shown. Database 100 may include data storage 101 and one or more database nodes (or nodes) 102 and 104. On the client side 110, clients 106 and 108 may send requests to nodes 102 and 104, including but not limited to read, write, modify, management, and transaction management requests.

[0034] In distributed databases, data consistency is a critical issue that needs to be addressed. This problem stems from the fact that multiple nodes may have copies of the same data items. When one node modifies a data item, other nodes may end up with an outdated copy of that data item, unless a specific protocol is followed. Outdated data items can lead to inconsistencies between the same data items stored in different locations within the database. For example, when money is transferred between two accounts, a node might end up serving the old value for the source account and the new value for the destination account, resulting in a larger total value for both accounts.

[0035] A key requirement for consistency in distributed systems is global reads after write (GRAW). It's naturally expected that users know any reads that occur after a write should return the value written. Data consistency wouldn't be an issue if there were only one copy of the data (typically on the data store), but this is rarely the case. For performance reasons, nodes frequently cache data locally because local access is much faster than querying a remote data store.

[0036] Figure 2 An example of a data consistency problem is shown. Figure 2In this context, item 1 can have an initial value of 'A'. Client 106 can request (201) node 102 to change item 1 (data item 1) to the value 'B' by executing a change request 201 on node 102. Therefore, both the data storage 101 and the cache of node 102 can change (202) the value of item 1 to the new value 'B'. However, node 104 may still have the old value of item 1 in its cache because it has not yet received notification of the change. In this case, if client 106 or another client (e.g., client 108) requests a read operation on item 1 from node 104, node 104 will return the value 'A', which is the old value of item 1. This inconsistency in the value of item 1 can be called a GRAW problem because many applications fail to function properly when the old value is returned.

[0037] Some existing solutions attempt to address the GRAW problem by using dedicated machines called caching facilities or CFs to cache data items. Each change must be propagated to the CF node before being confirmed, and all items are read from the CF node to be accessible to other nodes. Therefore, the CF node is always up-to-date. However, this approach has some limitations. For example, the CF method can increase network load due to the additional communication required to update the CF. Therefore, more powerful (and more expensive) network hardware may be needed. Furthermore, the CF method may suffer from performance penalties due to the need to delay transaction confirmation to propagate changes to the CF node. Additionally, scalability may be limited by the performance of the CF node, which can be the single bottleneck of the system.

[0038] Another existing approach attempts to address the GRAW problem by delaying change (write transaction) confirmation until every other node in the system has acknowledged the change. In these approaches, each read and write operation is tagged with a system change number (SCN) so that each node can correctly order each read relative to the writes already performed. However, this approach also has its limitations. For example, it can lead to increased update latency due to sending requests to all nodes. Furthermore, SCN conflicts between nodes may require retries. Another limitation is that on large clusters, SCN-related messages can account for up to 15% of total network traffic. Therefore, more powerful (and more expensive) network hardware may be required.

[0039] Another existing approach is specifically designed for databases with computer-storage separation, a single write node, and multiple read-only nodes. In the group refresh required for transaction commit, this approach relies on: sending a message with an invalid item (page) ID to the read-only replica via the storage node; receiving a response; and then confirming the transaction. This approach may have limitations, including increased update latency, increased traffic, and the need for higher-performance and more expensive network hardware.

[0040] Another existing approach to ensuring post-write read consistency of replicated data relies on associating write requests with unique version numbers and using these version numbers to retrieve data. The server associates a write request with a unique version number and returns the version number to the client, which can then use that version number in subsequent read requests to retrieve data. The drawback of this approach is that it requires modifications to the client software to support version number exchange. Therefore, the client may need to append a version number to every communication with the database and to communication between clients.

[0041] From one perspective, systems, methods, and apparatuses can be provided for resolving the GRAW consistency problem, which can guarantee global consistency of distributed databases. The systems, methods, and apparatuses described herein can resolve the GRAW consistency problem with minimal network performance overhead, while remaining transparent to system users (applications).

[0042] Figure 3 A method 320 for maintaining data consistency in a distributed database is illustrated. The distributed database 300 (or database as used herein) can be understood to have several attributes. One such attribute is the database state, which can be defined as information about the database 300 that is visible to users. This information about the database 300 may include information about data items stored in the database 300 or the values ​​of the data items themselves. At least in the context of a single database node, all changes to the database state (e.g., writes) can be ordered according to the time each change occurred relative to other changes. The database 300 may include multiple nodes, including nodes 302 and 304. The distributed database may also include a data store 301 for storing multiple data items. The progress of each database node in changing the database state can be measured. The progress of each database node can be measured by assigning an incrementing number to each change in the database state. This number may be referred to as a clock count value. Other terms may also be used to refer to this number, such as a logical sequence number (LSN) or a system change number (SCN). The clock can be local (each node can have its own independent clock) or global (a global clock can exist, taking into account the progress of all nodes). Advantageously, in some embodiments of the invention, the clocks or counters at the database nodes do not need to be synchronized with each other or with the master clock.

[0043] Furthermore, all nodes can track their changes and mechanisms can be used to identify which items have been modified. Accordingly, each node in nodes 302 and 304 can track its changes. For example, logging could be a mechanism where all changes to database 300 are written to persistent storage as records describing those changes.

[0044] According to one aspect, when client 308 from client side 310 needs to execute a query with GRAW consistency, client 308 can mark the query as such before sending query 322 to database node 304. Query 322 can be a query for the value of a data item stored in the database. When such a query reaches node 304, the node can send message 323 to each of the multiple nodes of the distributed database 300 other than node 304 (e.g., node 302) to inquire about their latest progress. Message 323 can be a request for the latest progress count or the latest timestamp, which corresponds to the latest change made by each node to the database state. This request may not be specific to the data item whose value was queried via query 322. Sending node 304 can then wait for a response (e.g., response 324) from each of the multiple nodes of the distributed database 300 other than node 304. Once all responses are received, query execution on sending node 304 will wait for changes made by other nodes to be locally processed 325 at sending node 304. According to one aspect, local processing may include reading log (change) records generated by other nodes from data store 301, up to and including the latest received progress count value (e.g., LSN, or the latest timestamp corresponding to the latest change made by each node to the database state). Local processing may also include reading the identification (ID) of all modified data items and invalidating them in a local cache, and then rereading the data item values ​​from data store 301.

[0045] For example, a query node sending a request for the latest count value (or timestamp) to five other nodes will receive five different count values. Based on these five different count values, the query node can read log records generated by each of the five nodes that provided the latest count value to the query node, up to and including the latest count value received, from the data store. The query node can then connect to the data store and retrieve the data item value identified by the log records of the five relevant nodes.

[0046] Log records can be stored in and shared in data storage 301. Each log record can be marked with a progress count value (clock count value or timestamp). Each log record can correspond to a modified data item (i.e., a data item whose value has changed). Each log record can also indicate or include the ID of the modified data item. The ID of the modified data item can be read from the log record accordingly. Therefore, whenever a log record is written to shared storage, the log record can be timestamped with the progress count value of the node that performed the write. Therefore, a log record stream can be created for each node and sorted according to the corresponding progress count value (clock count value). Thus, for example, when node 304 receives the latest progress count value (or latest timestamp) of node 302, node 304 can retrieve and read the log record of node 302 from the shared storage (data storage 301). In one embodiment, node 304 can read a portion of node 302's log record that has not been read previously.

[0047] Once processing is complete, it can be guaranteed or ensured that node 304, which executed the query, has the latest data up to the point where the read query 322 arrives at node 304, because node 304 has updated or invalidated all locally cached data items up to the point where the relevant query arrives, and can execute the query to return (327) a result consistent with the latest modification of the data items.

[0048] Method 320 may include: Client 306 updating (321) an item (data item, specific item) using node 302. For example, client 306 may send a request to node 302 to update the data item value of the data item to the new value. Node 302 may update the value of the item locally in its cache and data store 301. Subsequently, client 308 may send a request 322 to node 304 to read the latest value of the item. Request 322 may be a read query request indicating read-after-write (RAW) or GRAW.

[0049] In some embodiments, client 308 may be the same as client 306, wherein client 306 may perform queries on multiple nodes. In some embodiments, client 308 may be a different client that communicates with client 306 in a third application-specific manner.

[0050] Node 304 may send a request 323 to each node of database 300 (e.g., node 302) (other than node 304) for a latest progress count value (latest clock count value or latest timestamp), the latest progress count value corresponding to the latest change made to the database state by each node of database 300 (other than node 304). Each node of the database (e.g., 302) other than node 304 may send a reply 324 to node 304, the reply including the requested latest progress count value.

[0051] Node 304 can then manage its cache based on progress count values ​​received from each node in the database other than node 304. In some embodiments, managing its cache may include updating its cache according to the latest progress count value received from each node in the database other than node 304. Updating the cache may refer to updating one or more data items in the cache according to the corresponding latest received progress count value to reflect changes in the database state made by each node in the database other than node 304. That is, updating the cache may include updating the data item value of the relevant data item according to the latest data item value belonging to the data item. In some embodiments, managing its cache may include invalidating its cache according to the latest progress count value received from each node in the database other than node 304. Invalidating the cache may refer to marking one or more data items in its cache as no longer valid or latest.

[0052] Then, node 304 can process (326) the query received at action 322. Processing the query may include retrieving the latest value of the data item identified in the query. Retrieving the latest value of the data item may include reading the value from a local cache at node 304 or from data storage 301. In some embodiments, if node 304 manages its cache by updating items in the cache such that the items in the cache reflect their latest state (or value) as in the data storage, then node 304 may read the latest value of the item from its local cache. In some embodiments, if node 304 manages its cache by invalidating items in the cache such that items in the local cache are marked as invalid or outdated (and therefore the latest value of the item will be read from the data storage), then node 304 may read the latest value of the data item from data storage 301.

[0053] After obtaining the latest value of the data item, node 304 can send a reply 327 to the client, the reply including the latest value of the data item.

[0054] The methods, systems, and apparatus described herein can be applied to any distributed database system that allows reading and updating the values ​​of data items stored in a database. For example, this could include a relational multi-writer shared data database.

[0055] Figure 4 A flowchart of a method for managing query requests is shown in one aspect. Flowchart 400 can support reducing network traffic by queuing clock count value requests and batching clock count value requests. In some embodiments, flowchart 400 can be executed at the receiving node. In one embodiment, flowchart 400 can be used in conjunction with... Figure 3 The method combines 320.

[0056] refer to Figure 3 Sending a request for the latest clock count value to each of the multiple nodes in database 300 whenever a new query arrives can be very costly, as the query rate can be very high (e.g., 1 million queries per second). Flowchart 400 can support sending a single request at a time, with another request sent immediately after the previous one receives a response, thereby reducing the number of network requests (e.g., parallel or overlapping network requests). A request can refer to the receiving node requesting the latest clock count value from each node in the database.

[0057] Flowchart 400 may include: At action 401, a user query arrives at a receiving node (e.g., node 304) among multiple nodes in the database. The user query may target data items (data item values) stored in the database. At action 402, the receiving node may determine if there are any ongoing timestamp requests (referring to one or more pending requests for the latest clock count value sent from node 304 to one or more of the multiple nodes). If there are ongoing timestamp requests, at action 403, the user query is placed into the next timestamp request queue. If there are no ongoing timestamp requests, at action 404, the user query is moved to the current timestamp request queue to complete the user query. The current timestamp request queue is a queue of one or more queries being processed for the latest clock count value. Flowchart 400 may also include: issuing a timestamp request at action 405. Issuing a timestamp request may include the receiving node sending a request for the latest clock count value to each node in the database at action 405.

[0058] Flowchart 400 may further include: waiting for a request to complete and obtaining a clock value at action 406. At action 406, the receiving node may wait for a reply based on the latest clock count value of the request sent at action 405. Once the request completion arrives, queries on the wait list waiting for that request completion (e.g., one or more queries waiting for the latest clock count value in the current timestamp request queue) will be released based on the received clock value. After receiving the reply of the latest clock count value, all user queries waiting in the current timestamp request queue may be removed from the current timestamp request queue. Flowchart 400 may further include: at action 407, one or more user queries removed from the current timestamp request queue may continue to retrieve the data requested in one or more user queries. Flowchart 400 may further include: at action 408, waiting for a page cache update / invalidation until the received clock value. At action 408, the receiving node may update its cache or invalidate its cache until the received clock value. Flowchart 400 may further include: at action 409, executing one or more queries based on the updated / invalidated cache.

[0059] In some embodiments, flowchart 400 may include: after acquiring the latest clock value, at action 410, the receiving node may determine whether any queries exist in the next timestamp request queue. If there are no queries in the next timestamp request queue, at action 412, the receiving node may wait until another user query arrives. If there are one or more queries in the next timestamp request queue, at action 411, the receiving node may move one or more queries from the next timestamp request queue to the current timestamp request queue. Flowchart 400 may then loop back to action 405 shown and described herein.

[0060] Flowchart 400 can reduce network traffic and load. In some cases, latency can even be shorter due to reduced network utilization.

[0061] According to one aspect, one or more methods, systems, and apparatuses described herein (including method 320 and flowchart 400) can be applied to transactional databases. A transactional database can refer to a database in which all changes made to a database by a specific user session are not visible until the user confirms these changes (i.e., commits the transaction). Therefore, all changes appear atomically in the database after they are committed individually.

[0062] In the case of a transactional database, request 323 in method 300 could be a request for a clock count value corresponding to the most recently committed transaction. Therefore, node 340 could send request 323 for a clock count value (or timestamp) corresponding to the most recently committed transaction to each of the multiple nodes of database 300, except for node 304.

[0063] In a transactional database, only changes to committed transactions are visible to other queries. Therefore, the node executing the read request (e.g., node 304) does not need to see the latest but (not yet) committed changes. In a transactional database, method 320 and flowchart 400 can reduce query latency because there is less data that may need to be updated or invalidated on the query execution node (e.g., node 304).

[0064] According to one aspect, one or more methods, systems, and apparatuses described herein (including method 320 and flowchart 400) may be compatible with different types of clocks, including, for example, Lamport (scalar) clocks, vector clocks, matrix clocks, vector-scalar (VS) clocks, and hybrid clocks.

[0065] In some embodiments, for vector clocks or vector-scalar (VS) clocks (for multi-writer node configurations), the returned clock count value can be a vector, where each vector component C ij This indicates the timestamp received from node I, based on the clock on node j. In this type of embodiment, queries requiring consistency may need to receive a timestamp from node j in response 324 until the clock value of node j equals... The changes made. Among them, the formula... The clock value indicates that it will be the maximum value of each component of the timestamps received from all nodes. Here, C ij This corresponds to the j-th component of the vector clock received from node i, where i and j are between 0 and N–1, and N is the number of nodes in the database.

[0066] For example, node 302 may have a logical clock, which is either a vector clock or a VS clock. Response 324 may be a vector (e.g., a vector timestamp) comprising multiple vector components (or value arrays), each vector component corresponding to and representing a clock count value for a different node in the database. While response 324 from node 302 may include indications of clock count values ​​for nodes other than node 302, node 304 may still need to receive the corresponding latest clock count value (latest timestamp) from each node other than node 302. Once node 304 has received the latest clock count value from each of the multiple nodes in the database, node 304 can manage its cache based on the received latest clock count value. For example, if node 302 returns response 324 including a vector timestamp, node 304 can manage its cache based on the vector component (e.g., the vector timestamp) in response 324 corresponding to node 302. Therefore, for each vector timestamp received from each of the multiple nodes (with a vector clock or VS clock), node 304 can manage its cache based on the maximum value of the vector component (or value array) corresponding to each of the multiple nodes.

[0067] Therefore, in the case of vector clocks and VS clocks, the response 324 from each of the multiple nodes (each node having a vector clock, VS clock, or a clock that returns a vector timestamp response) can include a corresponding clock count value (e.g., a timestamp or latest timestamp) as a vector timestamp. The vector timestamp can include an array of values ​​(or vector components), where each value (or vector component) can correspond to a different node among the multiple nodes of the distributed database. For each vector timestamp received from each of the multiple nodes (excluding query node 304), the query node or node 304 can process its local cache based on the maximum value of the array of values ​​in the vector timestamp, which corresponds to each of the multiple nodes except the receiving node.

[0068] According to one aspect, methods, systems, and apparatuses related to data consistency can be configured at different granularities. For example, data consistency can be maintained at one or more levels, such as the database level, node level, user level, connection level, transaction level, and individual statement level. In an embodiment, only specific statements can be executed with the required consistency, while queries that are not important for consistency or achieve consistency through other means may not require this overhead.

[0069] According to one aspect, one or more methods, systems, and apparatuses described herein can be used in conjunction with broadcast transaction commit methods (e.g., those used by Oracle read), depending on the current workload. One or more methods described herein can be used for workloads where the number of writes is not significantly less than the number of reads requiring global consistency.

[0070] As described herein, one or more embodiments 320, 400, and 500 can be provided for establishing GRAW consistency. One or more methods can provide consistent data to users when using distributed databases. According to one aspect, one or more methods can eliminate the need for centralized caching, which avoids expensive components, eliminates scalability bottlenecks, and reduces network load (unlike methods used in existing solutions).

[0071] One or more methods described in this paper can also avoid the need to broadcast all write completions, which can further reduce network load (transaction rates can reach millions per second). Avoiding the need to broadcast all write completions can also reduce individual write transaction latency (unlike methods used in existing solutions). One or more methods described in this paper can also avoid modifications to client software, making it easier to adopt and compatible with legacy software.

[0072] Figure 5A A method for maintaining data consistency in a distributed database is illustrated in one aspect. Method 500 may include: a receiving node (e.g., node 304) receiving (501) a query for reading the value of a specific data item among a plurality of data items. The receiving node may be one of a plurality of nodes in a distributed database storing the plurality of data items. Method 500 may further include: the receiving node requesting (502) the latest timestamp of the latest corresponding transaction most recently executed by each of the plurality of nodes other than the receiving node itself from each of the respective nodes. Method 500 may further include: the receiving node receiving (503) the latest timestamp of the latest corresponding transaction from each of the plurality of nodes other than the receiving node itself. Method 500 may further include: the receiving node managing (504) a cache of the receiving node based on the latest timestamp of the latest corresponding transaction at each corresponding node among the plurality of nodes other than the receiving node, according to the transactions executed by each of the plurality of nodes other than the receiving node. The cache of the receiving node may include a set of values ​​corresponding to a set of data items among the plurality of data items. The method may further include: the receiving node reading the value of the specific data item.

[0073] In some embodiments, the receiving node management (504) of the receiving node's cache may include one of the following operations: invalidating one or more values ​​in a set of values ​​in the cache, or updating one or more values ​​in a set of values ​​in the cache. The distributed database may be one of the following types: a shared data database, a shared-nothing database, a multi-write database, and a single-write multi-read database. The query may be a read-after-write (RAW) query. The latest timestamp may be a clock count value corresponding to the clock of each of the multiple nodes other than the receiving node.

[0074] The clock corresponding to each node among the multiple nodes, excluding the receiving node, can be either a vector clock or a vector-scalar clock, where the corresponding latest timestamp is a vector timestamp. A vector timestamp can include an array of values, where each value can correspond to a different node among the multiple nodes in the distributed database. Since the corresponding latest timestamp is a vector timestamp, the receiving node manages the cache of multiple data items by: for each vector timestamp received from each node among the multiple nodes, excluding the receiving node, processing the cache of multiple data items based on the maximum value of the value array, where the maximum value corresponds to each node among the multiple nodes, excluding the receiving node.

[0075] Reading the value of a specific data item by the receiving node (505) may include: the receiving node obtaining the value of the specific data item from one of the following: a cache of multiple data items, or data storage in a distributed database. Reading the value of a specific data item by the receiving node may also include: the receiving node sending a response including the obtained value of the specific data item.

[0076] The latest timestamp can be based on one or more of the following: a global clock or a local clock. The latest transaction can be the most recently committed transaction.

[0077] In some embodiments, method 500 may further include: the receiving node determining that there is no ongoing request for the latest timestamp sent by the receiving node to one or more of a plurality of nodes. Method 500 may further include: the receiving node adding a query to a current timestamp request queue used for serving the collection of the latest timestamp. The current timestamp request queue may be a queue of one or more queries currently being served for collecting the latest timestamp.

[0078] In some embodiments, method 500 may further include: after the receiving node receives the corresponding latest timestamp of the corresponding latest transaction from each of the plurality of nodes other than the receiving node, the receiving node removes the query from the current timestamp request queue. In some embodiments, method 500 may further include: the receiving node determining whether one or more queries exist in the next timestamp request queue. The next timestamp request queue may be a queue of one or more queries waiting to be transferred to the current timestamp request queue.

[0079] In some embodiments, method 500 may further include: the receiving node determining whether one or more queries exist in the next timestamp request queue. The next timestamp request queue may be a queue of one or more queries waiting to be transferred to the current timestamp request queue.

[0080] Figure 5B A method for managing queries at nodes in a distributed database is shown as one aspect. Method 510 may include: a receiving node among multiple nodes of a distributed database storing multiple data items receiving (511) a first query for reading the value of a data item among the multiple data items. Method 510 may further include: the receiving node determining (512) whether one or more requests sent by the receiving node to one or more nodes among the multiple nodes are still in progress, each of the one or more requests requesting a latest timestamp corresponding to the latest transaction at the corresponding node among the one or more nodes. Method 510 may further include: the receiving node receiving (513) a set of responses from a set of nodes among the multiple nodes, each response corresponding to one node in the set of nodes, and based on the receiving node's request for a latest timestamp corresponding to the latest transaction executed by the corresponding node. Each response also indicates the latest timestamp. Method 510 may further include: the receiving node managing (514) a cache of the receiving node based on the received set of responses, the cache of the receiving node indicating a set of values ​​for a set of data items among the multiple data items.

[0081] In some embodiments, determining (512) whether one or more requests sent by the receiving node to one or more of the plurality of nodes are still in progress may include: the receiving node determining that one or more requests for the latest timestamp sent by the receiving node to one or more of the plurality of nodes are still in progress. The one or more requests may correspond to one or more queries in the current timestamp request queue. The current timestamp request queue may be a queue of one or more queries currently being served to collect the latest timestamp, and the one or more queries in the current timestamp request queue were received by the receiving node prior to the first query. A set of responses received may correspond to one or more queries in the current timestamp request queue. In some embodiments, method 510 may further include: the receiving node adding the first query to the next timestamp request queue. The next timestamp request queue may be a queue waiting to be transferred to the current timestamp request queue.

[0082] A set of responses from a group of nodes among multiple nodes may correspond to one or more ongoing requests sent by the receiving node to one or more nodes among the multiple nodes. Method 510 may further include: the receiving node removing one or more queries from the current timestamp request queue. Method 510 may further include: the receiving node executing the one or more queries removed from the current timestamp request queue. Method 510 may further include: the receiving node determining that one or more queries, including the first query, exist in the next timestamp request queue. Method 510 may further include: the receiving node moving one or more queries from the next timestamp request queue to the current timestamp request queue.

[0083] In some embodiments, determining (512) whether one or more requests sent by the receiving node to one or more of the plurality of nodes are still in progress may include: the receiving node determining that there are no ongoing requests for the latest timestamp sent by the receiving node to one or more of the plurality of nodes. Method 510 may further include: the receiving node adding a first query to a current timestamp request queue. The current timestamp request queue may be a queue of one or more queries currently being served to collect the latest timestamp. Method 510 may further include: the receiving node requesting the latest timestamp corresponding to the latest transaction executed by each of the plurality of nodes other than the receiving node.

[0084] In some embodiments, receiving a set of replies from a group of nodes among a plurality of nodes (513) may include: receiving the latest timestamp of the corresponding latest transaction from each node among the plurality of nodes other than the receiving node. Managing (514) the receiving node's cache based on the received set of replies may include: managing the receiving node's cache based on the replies received from each node among the plurality of nodes other than the receiving node.

[0085] In some embodiments, method 510 may further include: the receiving node removing the first query from the current timestamp request queue. Method 510 may further include: the receiving node executing the first query. Method 510 may further include: the receiving node determining whether one or more queries exist in the next timestamp request queue, whereby the next timestamp request queue may be a queue of one or more queries waiting to be transferred to the current timestamp request queue.

[0086] Figure 6 A device 600 is illustrated, providing different aspects of the invention, which can perform any or all of the methods and features described explicitly or implicitly herein. For example, a computer equipped with network capabilities can be configured as device 600. In some aspects, device 600 can be a device connected to network infrastructure via a wireless interface, such as a mobile phone, smartphone, or other device that can be classified as user equipment (UE). In some aspects, device 600 can be a machine-type communications (MTC) device (also known as a machine-to-machine (m2m) device), or other such devices that, although not providing direct service to a user, can be classified as UE. In some aspects, device 600 can be used to implement one or more components, systems, or mechanisms provided by one or more aspects described herein.

[0087] As shown, device 600 may include a processor 610, such as a central processing unit (CPU) or a dedicated processor, such as a graphics processing unit (GPU) or other such processing unit; memory 620; non-transient mass storage 630; an input-output interface 640; a network interface 650; and a transceiver 660, all of which are communicatively coupled via a bidirectional bus 670. The transceiver 660 may include one or more antennas. Depending on certain aspects, any or all of the described elements may be used, or only a subset thereof. Furthermore, device 600 may include multiple instances of certain elements, such as multiple processors, memories, or transceivers. Additionally, elements of the hardware device may be directly coupled to other elements without requiring a bidirectional bus. In addition to or in place of processors and memory, other electronic or processing electronic elements, such as integrated circuits, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital circuits, analog circuits, chips, dies, multi-chip modules, substrates, etc., or combinations thereof, may be used to perform the desired logical operations.

[0088] Memory 620 may include any type of non-transitory memory, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), read-only memory (ROM), or any combination thereof. Mass storage element 630 may include any type of non-transitory storage device, such as a solid-state drive, hard disk drive, disk drive, optical disk drive, USB drive, or any computer program product for storing data and machine-executable program code. According to some aspects, memory 620 or mass storage 630 may record statements and instructions executable by processor 610 thereon for performing any of the methods described herein.

[0089] The processor 610 and memory 620 can be provided together as a chipset for installation in the wireless communication device 600 to enable WLAN functionality. The chipset can be used to receive data, including but not limited to PPDUs, as input from the network interface 650. The chipset can also be used to output data, including but not limited to PPDUs, to the network interface 650.

[0090] The aspects of this invention can be implemented using electronic hardware, software, or a combination thereof. In some aspects, the invention can be implemented by one or more computer processors executing program instructions stored in memory. In some aspects, the invention is implemented in part or in whole in hardware, for example using one or more field programmable gate arrays (FPGAs) or application-specific integrated circuits (ASICs) to rapidly perform processing operations.

[0091] It should be understood that although specific embodiments of the technology have been described herein for illustrative purposes, various modifications may be made without departing from the scope of the technology. Therefore, the specification and drawings are to be considered merely as an illustration of the invention as defined in the appended claims, and any and all modifications, variations, combinations, or equivalents falling within the scope of this specification are contemplated. In particular, computer program products or program elements for storing machine-readable signals, or program storage or storage devices such as magnetic or optical fibers, magnetic tape, or optical discs, are provided within the scope of this technology for controlling the operation of a computer according to the methods of this technology and / or constructing some or all of its components according to the systems of this technology.

[0092] The actions associated with the methods described herein can be implemented as coded instructions in a computer program product. In other words, a computer program product is a computer-readable medium on which software code is recorded, which performs the methods when the computer program product is loaded into memory and executed on the microprocessor of a wireless communication device.

[0093] Furthermore, each operation of the method can be performed on any computing device such as a personal computer, server, or PDA, and based on one or more program elements, modules, or objects, or a portion thereof, generated from any programming language such as C++ or Java. Additionally, each operation, or the file or object implementing each operation, can be executed by dedicated hardware or circuit modules designed for this purpose.

[0094] Based on the description of the above embodiments, the present invention can be implemented using only hardware, or using software and necessary general-purpose hardware platforms. Based on this understanding, the technical solution of the present invention can be embodied in the form of a software product. The software product can be stored in a non-volatile or non-transitory storage medium, such as a compact disk read-only memory (CD-ROM), USB, flash drive, or portable hard drive. The software product includes a number of instructions that enable a computer device (personal computer, server, or network device) to perform the methods provided in the embodiments of the present invention. For example, such performance may correspond to the simulation of the logical operations described herein. According to embodiments of the present invention, additionally or alternatively, the software product may include multiple instructions that enable a computer device to perform operations configuring or programming digital logic devices.

[0095] Although the invention has been described with reference to specific features and embodiments thereof, it will be apparent that various modifications and combinations thereof can be made without departing from the invention. Therefore, the specification and drawings are to be regarded merely as an illustration of the invention as defined in the appended claims and any and all modifications, variations, combinations, or equivalents falling within the scope of this specification are to be considered.

Claims

1. A method characterized by, include: The receiving node receives a query for reading the value of a specific data item from a plurality of data items, wherein the receiving node is one of a plurality of nodes in a distributed database storing the plurality of data items; The receiving node requests the latest timestamp of the corresponding latest transaction from each of the plurality of nodes other than the receiving node. The receiving node receives the latest timestamp of the latest transaction from each of the plurality of nodes other than itself; The receiving node manages its cache based on the latest timestamp of the latest transaction at each corresponding node other than the receiving node, according to the transactions executed by each of the plurality of nodes other than the receiving node. The cache of the receiving node includes a set of values ​​corresponding to a set of data items among the plurality of data items. The receiving node reads the value of the specific data item.

2. The method according to claim 1, characterized in that, The receiving node manages the receiving node's cache by one of the following operations: invalidating one or more values ​​in the set of values ​​in the cache, or updating one or more values ​​in the set of values ​​in the cache.

3. The method according to claim 1 or 2, characterized in that, The distributed database is one of the following types: shared data database, shared-nothing database, multi-write database, and single-write multi-read database.

4. The method according to any one of claims 1 to 3, characterized in that, The query is a read-after-write RAW query.

5. The method according to any one of claims 1 to 4, characterized in that, The latest timestamp is the clock count value corresponding to the clock of each of the plurality of nodes, excluding the receiving node.

6. The method according to claim 5, characterized in that, The clock corresponding to each of the plurality of nodes, excluding the receiving node, is either a vector clock or a vector-scalar clock, and the corresponding latest timestamp is a vector timestamp comprising an array of values, each value corresponding to a different node among the plurality of nodes in the distributed database. Furthermore, the receiving node manages the cache of the plurality of data items by: For each vector timestamp received from each of the plurality of nodes other than the receiving node, the receiving node processes the cache of the plurality of data items based on the maximum value in the value array corresponding to each of the plurality of nodes other than the receiving node.

7. The method according to any one of claims 1 to 6, characterized in that, The receiving node reads the value of the specific data item including: The receiving node obtains the value of the specific data item from one of the following: the cache of the plurality of data items, or the data storage of the distributed database; The receiving node sends a reply, which includes the value obtained for the specific data item.

8. The method according to any one of claims 1 to 7, characterized in that, The corresponding latest transaction is the most recently committed transaction.

9. The method according to any one of claims 1 to 8, characterized in that, Also includes: The receiving node determines that there is no ongoing request for the latest timestamp sent by the receiving node to one or more of the plurality of nodes; The receiving node adds the query to the current timestamp request queue, which is a queue of one or more queries currently being served to collect the latest timestamp.

10. The method according to claim 9, characterized in that, Also includes: After the receiving node receives the corresponding latest timestamp of the corresponding latest transaction from each of the plurality of nodes other than the receiving node, the receiving node removes the query from the current timestamp request queue; The receiving node determines whether there are one or more queries in the next timestamp request queue, which is a queue of one or more queries waiting to be transferred to the current timestamp request queue.

11. A method for managing queries, characterized in that, The method includes: A receiving node in a distributed database storing multiple data items receives a first query for reading the value of a data item among the multiple data items; The receiving node determines whether one or more requests sent by the receiving node to one or more of the plurality of nodes are still in progress, each of the one or more requests being used to request the latest timestamp, the latest timestamp corresponding to the latest transaction at the corresponding node among the one or more nodes; The receiving node receives a set of replies from a group of nodes among the plurality of nodes, each reply corresponding to one node in the group of nodes, and each reply also indicates the latest timestamp based on the receiving node's request for the latest timestamp corresponding to the latest transaction executed by the corresponding node; The receiving node manages its cache based on the received set of responses, and the receiving node's cache indicates a set of values ​​for a set of data items among the plurality of data items.

12. The method according to claim 11, characterized in that, The method of determining whether one or more requests sent by the receiving node to one or more of the plurality of nodes are still in progress includes: the receiving node determining that one or more requests for the latest timestamp sent by the receiving node to one or more of the plurality of nodes are still in progress, the one or more requests corresponding to one or more queries in a current timestamp request queue, the current timestamp request queue being a queue of one or more queries currently serving to collect the latest timestamp, the one or more queries in the current timestamp request queue being received by the receiving node before the first query, and the received set of responses corresponding to the one or more queries in the current timestamp request queue; the method further includes: The receiving node adds the first query to the next timestamp request queue, which is a queue of one or more queries waiting to be transferred to the current timestamp request queue.

13. The method according to claim 12, characterized in that, The set of responses from one group of the plurality of nodes corresponds to one or more ongoing requests sent by the receiving node to one or more of the plurality of nodes, and the method further includes: The receiving node removes one or more queries from the current timestamp request queue; The receiving node executes the one or more queries removed from the current timestamp request queue; The receiving node determines that there are one or more queries, including the first query, in the next timestamp request queue; The receiving node moves one or more queries from the next timestamp request queue to the current timestamp request queue.

14. The method according to any one of claims 11 to 13, characterized in that, The method of determining whether one or more requests sent by the receiving node to one or more of the plurality of nodes are still in progress includes: the receiving node determining that there are no ongoing requests sent by the receiving node to one or more of the plurality of nodes for the latest timestamp, the method further comprising: The receiving node adds the first query to the current timestamp request queue, which is a queue of one or more queries currently being served to collect the latest timestamp; The receiving node requests the latest timestamp of the latest transaction executed by each of the plurality of nodes, excluding itself, where: The receiving node receiving the set of replies from the set of nodes among the plurality of nodes includes: the receiving node receiving the latest timestamp of the corresponding latest transaction from each of the plurality of nodes other than the receiving node; The receiving node manages its cache based on the received set of replies, including: the receiving node manages its cache based on the replies received from each of the plurality of nodes other than the receiving node itself.

15. The method according to claim 14, characterized in that, Also includes: The receiving node removes the first query from the current timestamp request queue; The receiving node executes the first query; The receiving node determines whether there are one or more queries in the next timestamp request queue, which is a queue of one or more queries waiting to be transferred to the current timestamp request queue.

16. An apparatus, characterized in that, include: One or more processors; A memory storing instructions that, when executed by the one or more processors, cause the apparatus to perform the method according to any one of claims 1 to 10 or 11 to 15.

17. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes instructions that, when executed by a computing system, cause the computing system to perform the method according to any one of claims 1 to 10 or 11 to 15.

Images