Distributed key-value database garbage collection methods, database and server systems

Abstract

In a distributed key-value database garbage collection method, within the database and server system, each client uses its local clock to obtain a timestamp `ts` and includes `ts` in write transaction requests sent to the server. The server maintains a timestamp `min_write_ts` and periodically updates `min_write_ts` to ensure it continuously increments. If the `ts` of a write transaction request is less than `min_write_ts`, the server rejects the write transaction request. The server periodically reports `min_write_ts` and `latest_gc_ts` to the BMD server. A garbage collection management table records the garbage collection timestamps of each server. ID and corresponding parameters A and B; periodically calculate the minimum parameter in the garbage collection management table; based on latest_gc_ts, initiate a lock release request to each server; after the BMD server successfully releases the lock on each server, it records the timestamp latest_gc_ts corresponding to the successful lock release; the KV-server receives latest_gc_ts and safe_gc_ts, and performs a two-phase confirmation in the next report, reporting latest_gc_ts together with the latest min_write_ts; the server periodically performs garbage collection, cleaning up expired data that is less than safe_gc_ts.

Description

Technical Field

[0001] This application belongs to the field of database technology, and specifically relates to a garbage collection method for distributed key-value databases based on rejection time. Background Technology

[0002] Definitions of abbreviations and key terms:

[0003] 1PC One-Phase Commit.

[0004] 2PC Two-Phase Commit.

[0005] Garbage collection (GC) is usually an automatic memory management mechanism that identifies objects that are no longer used in a program and releases the memory resources they occupy, thereby preventing memory leaks or program crashes. In this application, it refers to the system resources occupied by discarded data in a kv-server system, and is not limited to memory resources.

[0006] The PD cluster is the "brain" of distributed databases such as TiDB, consisting of multiple PD nodes that work together via the Raft protocol. Each node stores complete cluster metadata and achieves high availability through an election mechanism, ensuring uninterrupted service even if any node fails.

[0007] TiKV is a typical distributed key-value storage system, consisting of two basic components: a PD cluster and a KV cluster. The PD cluster handles the metadata of the KV storage system, while the KV cluster handles data storage. The basic concept of TiKV is to divide the entire key space of the system into connected but non-overlapping regions, called shards, in ascending order. Data is replicated across multiple nodes using the Raft protocol, with each shard region providing fault tolerance. Individual nodes use RocksDB for data persistence. The PD is the logical central control point, managing the current shard region partitioning and the Raft member node mappings for each shard region.

[0008] like Figure 1 This is a schematic diagram of the sharding of a distributed key-value storage system.

[0009] The client layer nodes write data to the storage nodes. The nodes store shards, and the Raft protocol is used to ensure write requests between nodes and data consistency between replicas.

[0010] Data consistency between sharded regions and replicas is maintained through the Raft protocol. Any write request can only be written to the Leader, and a write success will only be returned to the client after a majority of replicas have been written (the default configuration is 3 replicas, meaning that all requests must be successfully written to at least two replicas).

[0011] like Figure 2 This is a diagram illustrating the shard size on a storage node.

[0012] TiKV strives to maintain a suitable data size in each shard region, with a default of 96MB, which facilitates scheduling decisions by Product Processing (PD). When the size of a shard region exceeds a certain limit (144 MiB by default), TiKV will split it into two or more shard regions. Similarly, when a shard region becomes too small due to a large number of deletion requests (20 MiB by default), TiKV will merge two smaller adjacent shard regions into one.

[0013] Users can write multiple key-value pairs to the distributed KV store at once without worrying about whether these key-value pairs are on the same data slice (region). TiKV guarantees the ACID constraints of these read and write requests through distributed transactions based on two-phase commit.

[0014] Transactional data layout, distributed transactions, and 1PC / 2PC transactions: unify a series of operations so that either all operations succeed or none succeed. If any operation fails during execution, all previous operations are rolled back to the state before the series of operations were executed.

[0015] like Figure 3 , is a column family for storing transactional data.

[0016] Transaction data layout: The underlying storage engine of each key-value storage node stores transaction data, and its format is as follows.

[0017] There are three column families: CF_LOG, CF_IDX, and CF_TXN.

[0018] CF_IDX records the latest status of each key: the latest version in effect (wts), whether it is locked, and a short value.

[0019] The CF_LOG column records status information (lock, commit, abort) and a relatively large user_value, which is composed of user_key plus ts timestamp.

[0020] The CF_TXN column records all other keys in the same transaction.

[0021] In a distributed system environment, transactions are completed through remote collaboration between different services over a network. This is called a distributed transaction.

[0022] 2PC, or Two-Phase Commit Protocol, divides the entire transaction process into two phases: the Prepare phase and the Commit phase. The "2" refers to the two phases, "P" refers to the Prepare phase, and "C" refers to the Commit phase.

[0023] Preparation Phase (Write-Ahead Phase): Preparation requests are sent concurrently to the servers of all sharded Regions involved in the transaction. The servers perform concurrency control checks, locking, and other related operations, but do not commit, reaching a state where "everything is ready except for the final step." At this point, the transaction has entered a logically committed state and cannot be rolled back.

[0024] A write-ahead request is generated for W(key, value), and the contents of three columns are written during the preparation phase:

[0025] CF_IDX update key->{lock_wts: wts, lock_short_value: value};

[0026] CF_LOG writes key#wts ->{value, state: LOCK};

[0027] CF_TXN writes key ->{other_keys}.

[0028] like Figure 4 The process of writing using 2PC. Figure 4 The execution flow of the two-phase commit (2PC) protocol in distributed transactions is demonstrated.

[0029] The diagram illustrates the transaction initiation and prewrite process, where "the client initiates a transaction, and regions A and B participate."

[0030] The client initiates a transaction (run_as_txn) and sends a write-ahead request (2pc prewritek1 / 2pc prewrite k2) to region A / B. After receiving the request, region A / B performs operations such as rocks_merge (merge indexes) and rocks_put (write log / transaction information) to complete the "write-ahead" phase (i.e., the first phase: voting - asking each transaction data source if it is ready).

[0031] Clock Advance and Commit: After the pre-write is completed, region A / B confirms the status through HLC.advance (logical clock advance) and enters the second stage.

[0032] Entering the second phase, the commit phase: region A / B executes operations such as rocks_put (write commit information), rocks_merge (merge logs), and rocks_del (delete transaction temporary records) to complete the transaction commit.

[0033] Asynchronous Commit: After a transaction is committed, the client initiates an asynchronous commit (2pc async_commit k1 / 2pc async_commit k2). Regions A and B then advance the clock again through HLC.advance to perform load, check, and write operations to ensure eventual consistency of the transaction.

[0034] Figure 4 In the technical details and logical connections, HLC (Hybrid Logical Clock) is used for logical clock synchronization in distributed systems to ensure the sequentiality of cross-region operations, such as HLC.now() to get the current logical clock and HLC.advance to advance the clock.

[0035] RocksDB operations, such as rocks_merge (merge indexes), rocks_put (write log / transaction information), and rocks_del (delete transaction records), are the underlying storage layer's persistence operations on transaction states, supporting the atomicity and durability of 2PC.

[0036] Two-phase commit (2PC): Through a two-phase design of "write-ahead (first phase) → commit (second phase)," it ensures the atomicity of distributed transactions (either all succeed or all rollback), and is an important protocol for solving cross-node transaction consistency in distributed systems.

[0037] Figure 4Two-phase commit (2PC) scenarios and applications: This type of process is common in distributed databases and microservice architectures, used to coordinate transaction operations of multiple nodes (such as distributed transactions in order services and inventory services) and ensure the "either do all or do none" characteristic of business logic.

[0038] Commit Phase: If the client receives a failure message or timeout from the server, it sends a rollback message to each participant; otherwise, it sends a commit message. The server executes the commit or rollback operation according to the client's instructions, releasing all lock resources used during the transaction processing. Note: Lock resources must be released in the final phase.

[0039] CF_IDX update key->{lock_wts: wts, lock_short_value: value};

[0040] CF_LOG writes key#wts ->{value, state: LOCK};

[0041] CF_TXN writes key ->{other_keys}.

[0042] Lock removal: If there is a failure in the middle of the prewrite and commit stages 1 and 2, some keys will be in an intermediate state, that is, there is a lock residue in the intermediate state. At this time, lock removal is required.

[0043] 1. Both k1 and k2 are in the prewrit completed state and have locks, so it is necessary to advance to complete phase 2.

[0044] 2. Some keys are locked in the prewrit completion state and need to be rolled back. For example: k1 is locked, k2 is unlocked. Remove the lock on k1 and roll back the data written to the LOG by k1: CF_LOG update: key#wts -> {state: ABORT}; CF_IDX update key -> remove the lock state; CF_TXN delete key#wts.

[0045] Garbage collection: Over time, the storage engine accumulates a significant amount of expired data in the CF_LOG column. Each key only needs to retain its latest version; older versions are deleted by the server. To locate expired data, a safe timestamp `max_safe_ts` needs to be determined. `max_safe_ts` ensures that no meaningful requests with timestamps less than `max_safe_ts` enter the system (if a message with timestamps less than `max_safe_ts` enters, the server-side transaction processing needs to detect it and either return an error or ignore it). Simultaneously, it's necessary to ensure that all transactions before `max_safe_ts` are in a determined state, with no lock remnants in intermediate states. To achieve this, a new service, `bmd`, is introduced. `bmd`'s task is to advance transactions in an uncertain state to a determined state, calculate a suitable `max_safe_ts`, and remove all locks before `max_safe_ts`.

[0046] Generation of TS (Timestamp) for Distributed Key-Value Transactions: When initiating a distributed key-value transaction request, a TS timestamp is included. There are two methods for generating TS: distributed clock and centralized clock. Distributed timestamps are used to identify the transaction and the version of the key-value pair written in the transaction. Each node maintains an independent local logical clock instance. When initiating a transaction, a timestamp is generated using the local clock instance and sent to the server along with the write transaction request.

[0047] Centralized clocks: Centralized clocks have a unified clock source maintaining the clock service. The clock source guarantees that timestamps are increasing, and clients need to send a request to the clock source to obtain the timestamp ts. The unified clock source ensures that message ts are monotonically increasing, so the server does not need to use message rejection to prevent transactions with smaller ts from entering.

[0048] like Figure 5 Garbage collection based on a unified clock source under MVCC.

[0049] In an MVCC key-value storage system, the PD server maintains a unified clock source. Before any client initiates a transaction, it must first obtain the time stamp (ts) from the PD. The PD server guarantees that the timestamp of the ts is continuously incremented, so there is no situation where a client's transaction timestamp runs backwards. Once the BM server determines a certain safe timestamp, no transaction with a timestamp smaller than the safe timestamp will enter the system; otherwise, lock residue may occur, eventually leading to accidental deletion.

[0050] Step G1. client1 needs to obtain ts as the transaction timestamp from the unified clock source PD to start a transaction. The ts generated by the unified clock source is continuously incremented.

[0051] Step G2. Client1 and Client2 periodically report the transaction times (TS) of locally started transactions to the BMD server. The BMD server is responsible for the GC reclamation management service process. The BMD server collects the minimum value of all reported TS as safe_gc_ts.

[0052] Step G3. Scan and clean up any residual locks before safe_gc_ts to ensure that the key before safe_gc_ts is in a determined state, a committed state, or a rolled-back state.

[0053] Step G4. Perform GC reclamation and send a safe_gc_ts message to the server to clean up resources.

[0054] The advantage of a unified clock source is that all clients' time stamps (TS) come from the PD server, with the same timestamp base. From a global perspective, it can ensure that the TS is constantly increasing, preventing any client from obtaining past TS. It is also easy to calculate the minimum active TS globally, which can be used for appropriate GC to reclaim TS after unlocking. However, the client set is open, which can lead to incomplete collection of client TS. Moreover, a unified clock has a single point of performance bottleneck, which is not conducive to scalability. The periodic reporting by each client also incurs overhead, affecting system performance.

[0055] TiKV is a distributed key-value system implemented by the TiKV community. It maintains a unified clock source in the PD cluster. Before starting a transaction, all clients need to obtain the timestamp (ts) from PD. PD guarantees that the timestamp is continuously incremented. Therefore, it ensures that the timestamp generation is incremental and will not go backward.

[0056] Lock removal services (such as BMD) collect the timestamps (ts) of running transactions from all clients, and then select the minimum value of the timestamps as the safe timestamp, denoted as safe_gc_ts.

[0057] Attempt to resolve locks: The lock resolution service process sends a message to the server cluster to scan for residual locks before safe_gc_ts. If any exist, it attempts to resolve them and ensures that the key before safe_gc_ts is in a definite state: it changes transactions in an intermediate state to a committed state or rolls them back.

[0058] Once the lock is cleared, the safe_gc_ts value can be broadcast to all servers. The servers will periodically clean up data that has expired before safe_gc_ts.

[0059] Distributed key-value systems employ a centralized timestamp generation method, which is simple to implement and makes it relatively easy to determine the garbage collection timestamp (ts) for GC (Garbage Collection). However, the requirement to obtain the timestamp for each transaction can easily lead to a single point of failure and poor scalability. Furthermore, the client set is an open and dynamic collection; clients can be added or removed at any time, making it difficult to guarantee the collection of timestamps for all running transactions from all clients. Summary of the Invention

[0060] This invention proposes a distributed clock system where the client generates and reports transaction timestamps locally, while the server maintains rejection timestamps. This ensures the logical order of data writing, avoids the bottleneck of centralized clocks, and prevents the system from being limited in scale due to centralized clocks. By statistically analyzing the globally minimum rejection timestamp and using it as the system's garbage collection timestamp, the deletion of transactions with uncertain states in 2PC transactions during garbage collection is avoided.

[0061] A garbage collection method for a distributed key-value database is provided. The distributed key-value database includes a client, a server, and a BMD server. Each client uses its local logical clock to obtain a timestamp (ts) and includes it in write transaction requests sent to the server. The server maintains a timestamp (min_write_ts). The server periodically refreshes min_write_ts to ensure it continuously increments. If the timestamp of a write transaction request is less than min_write_ts, the server rejects the request. The server periodically reports min_write_ts and latest_gc_ts to the BMD server. The BMD server includes a garbage collection management table, which records the ID, parameter A, and parameter B reported by each server, where parameter A is min_write_ts and parameter B is latest_gc_ts. The BMD server periodically calculates the minimum parameter A in the aforementioned garbage collection management table; it calculates the minimum value in parameter A to obtain `latest_gc_ts`; it calculates the minimum value in parameter B to obtain `safe_gc_ts`; based on `latest_gc_ts`, the BMD server initiates lock release requests to each server; after successfully releasing the locks for each server, the BMD server records the timestamp `latest_gc_ts` corresponding to the successful lock release; the BMD server broadcasts the `latest_gc_ts` and `safe_gc_ts`; the KV server receives `latest_gc_ts` and `safe_gc_ts`, saves them, performs garbage collection, cleans up expired data smaller than `safe_gc_ts`, and reports the received `latest_gc_ts` along with the latest `min_write_ts` to the BMD server for a second round of calculation.

[0062] It is possible that if the ts of the client's write transaction request is less than min_write_ts, the server rejects the write transaction request; after receiving the rejection message, the client waits for a period of time, or speeds up its local logical clock; it then uses its local clock to obtain the ts again, and re-includes the ts in the transaction request before sending it to the server.

[0063] It is possible that if the ts of the client's write transaction request is less than min_write_ts, the server rejects the write transaction request; after receiving the rejection message, the client speeds up its local clock, re-acquires the ts, and resends the transaction request with the aforementioned ts to the server.

[0064] Alternatively, the server can maintain a local timestamp min_write_ts; the server can use the local logical clock as the timestamp min_write_ts; and the server can periodically refresh min_write_ts to ensure that min_write_ts continuously increments.

[0065] Alternatively, the server receives the broadcast's `latest_gc_ts` and `safe_gc_ts` and saves them locally.

[0066] This can be achieved by the server periodically reporting the `latest_gc_ts` and the latest `min_write_ts` to the BMD server; upon receiving the `latest_gc_ts` and the latest `min_write_ts`, the BMD server records parameters A and B in the garbage collection management table, where parameter A is `min_write_ts` and parameter B is `latest_gc_ts`; the BMD server periodically calculates the minimum value of parameters A and B to obtain new `latest_gc_ts` and `safe_gc_ts`, respectively.

[0067] It is possible that the BMD server, based on the latest_gc_ts calculated from parameter A, sends a lock release request to each server.

[0068] Alternatively, the server receives the lock release request, checks whether the transaction before the latest_gc_ts timestamp is in a committed state or a successfully rolled-back state. If so, it returns a successful lock release to the BMD server; if there is a lock residue, it attempts to release the lock and returns a success message upon successful lock release.

[0069] In the server example above, a write transaction corresponds to three column families: CF_LOG, CF_IDX, and CF_TXN. CF_IDX records the latest status of each key: the latest effective version number (wts), whether it is locked, and a short value. The CF_LOG column records state information and version information (wts). The state includes locked, committed, and aborted states. The CF_TXN column records other keys within the same transaction.

[0070] Alternatively, the server receives the lock release request, checks transactions prior to the latest_gc_ts timestamp, records transactions in the CF_LOG column that are in an intermediate state (abort), pushes them to transition to a locked state (lock) or a committed state (commit), and either changes the transaction in the intermediate state to a committed state or rolls it back.

[0071] This could be achieved by the server performing garbage collection based on the safe_gc_ts timestamp, cleaning up data that has expired before safe_gc_ts; and the BMD server obtaining safe_gc_ts by calculating the minimum value of latest_gc_ts reported by all servers.

[0072] This could mean that the aforementioned BMD server is either a standalone server or a service process running within a selected server.

[0073] A distributed key-value database, including the aforementioned garbage collection method for distributed key-value databases.

[0074] A distributed server system is provided for running the aforementioned distributed key-value database.

[0075] One of the technical benefits of the above solution is that it solves the single-point performance bottleneck problem under centralized clocks, improving scalability. It also addresses the issue of difficulty in collecting all client time signatures when the client set is open. The method based on rejecting timestamps ensures that transactions are processed sequentially.

[0076] One of the technical effects of the above-mentioned technical solution is to solve the single-point performance bottleneck and scalability problem under centralized clock.

[0077] One of the technical effects of the above solution is that timestamp reporting is shifted from the client side to the server side, and the number of servers is usually limited and does not change frequently. The benefits include: reducing the number of reports, having a negligible impact on system performance, and solving the problem of difficulty in collecting all timestamps in a distributed scenario where the client set is open and dynamically changing.

[0078] One of the technical effects of the above solution is that gc_ts undergoes a two-stage confirmation process. The first and second stages of confirmation ensure that all servers receive the GC reclamation timestamp and reach a consensus at this timestamp. If there is no second confirmation and GC reclamation is performed directly based on latest_gc_ts, since data will be migrated between different servers, if some servers do not receive latest_gc_ts and data migration has occurred, the servers that have received ts may mistakenly clean up and delete transaction data when they start cleanup, which may lead to inconsistent transaction states. Attached Figure Description

[0079] Figure 1 This is a schematic diagram of the sharding of a distributed key-value storage system;

[0080] Figure 2This is a diagram illustrating the shard size on a storage node;

[0081] Figure 3 This is a schematic diagram of column families for storing transactional data;

[0082] Figure 4 This is a flowchart illustrating the 2PC write process;

[0083] Figure 5 It is garbage collection based on a unified clock source under MVCC;

[0084] Figure 6 This is a schematic diagram of the write process in a distributed clock kv storage system;

[0085] Figure 7 This is a schematic diagram of the block diagram of the distributed key-value database system in Implementation Example 1;

[0086] Figure 8 This is a schematic diagram of the block diagram of the distributed key-value database system in Embodiment 2;

[0087] Figure 9 One of the schematic diagrams of the operation steps in Example 4;

[0088] Figure 10 The second schematic diagram of the operation steps of Example 4. Detailed Implementation

[0089] The contents of this application will be further described in detail below with reference to the accompanying drawings. It should be noted that the following description is of preferred embodiments of the present invention and does not constitute any limitation on the present invention. The description of the preferred embodiments of the present invention is merely an explanation of the general principles of the invention. The designations "first," "second," "A," and "B" used in this invention are for ease of explanation only and do not represent a temporal or spatial order. The combinations of letters and numbers "TA," "TB," and "H" used in this invention are for ease of explanation only, and their specific meanings are determined by the specific terms they represent.

[0090] This application primarily aims to determine a suitable and secure timestamp for garbage collection under a distributed clock system. The distributed key-value storage system based on a distributed clock has the following characteristics:

[0091] 1) Data distribution is dynamic, migrating between different nodes;

[0092] 2) Each node has an independent logical clock and obtains its local time (TS).

[0093] 3) Transactions are sorted according to logical timestamps.

[0094] Figure 6 A schematic diagram of the write process in a distributed clock kv storage system.

[0095] The above characteristics will bring some problems:

[0096] 1. The client set is open and changes frequently, making it difficult to collect all types.

[0097] 2. The logical timestamps of different clients are at different speeds, such as... Figure 6 The physical timestamp of the transaction at time t5 is later than the physical timestamp of the transaction at time t6, but the logical timestamp of the transaction request, t5, is less than t6. How can we ensure that after BMD calculates `safe_gc_ts`, the timestamp of subsequent client-written transactions is greater than `safe_gc_ts`? For example, if BMD calculates a GC collection timestamp of t6, and then a transaction at time t5 with a logical timestamp smaller than t6 enters the server, a problem will occur. (This is because if a transaction with a timestamp smaller than `safe_gc_ts` enters the server, the constraint that transactions must be in a definite state before the GC collection timestamp is not met, causing the GC to mistakenly delete these data.)

[0098] 3. Data is migrated between different servers.

[0099] To solve the above problem, the following method can be adopted:

[0100] 1) The data collection method was switched from client-side reporting to server-side reporting, which solved the problem of changes in client-side data sets;

[0101] 2) To ensure transactions are processed in logical timestamp order, a mechanism to reject timestamps needs to be added to the server. Transactions with timestamps (ts) less than min_write_ts are rejected. min_write_ts is a continuously incrementing timestamp maintained by the server. min_write_ts is reported to the garbage collection (GC) process (bmd), which calculates the minimum value to obtain safe_gc_ts. Therefore, even if the client's clock is slower, if a new write transaction's timestamp is less than safe_gc_ts, it will be rejected by the server, ensuring no new write transactions are written to the server.

[0102] 3) Establishing safe_gc_ts requires a two-stage confirmation process. In the first stage, the server does not immediately perform GC reclamation on the latest_gc_ts it receives. It needs to inform the BMD through the second round of reporting the latest_gc_ts to ensure that all servers receive the latest_gc_ts and reach a consensus at this timestamp. Only after consensus is reached can safe_gc_ts be generated and finally returned to the server. The server then performs GC reclamation on this ts.

[0103] Figure 7Example 1 is a schematic diagram of a distributed key-value database system. It includes a client, a server, and a BMD server. In the diagram, client_1 and client_2 are clients, server_1, server_2, and server_3 are servers in the key-value cluster, and the BMD server is a service or service process that manages garbage collection. The garbage collection management table is stored on the BMD server. The garbage collection management table includes the server's node ID and parameters A and B.

[0104] Figure 7 In the table, server_1 records the IP address or MAC address of server_1, and T1 is parameter A; that is, the specific value of min_write_ts, which is reported by server_1.

[0105] server_2 records the IP address or MAC address of server_2, and T2 is parameter A; that is, the specific value of min_write_ts, which is reported by server_2.

[0106] server_3 records the IP address or MAC address of server_3, and T3 is parameter A; that is, the specific value of min_write_ts, which is reported by server_3.

[0107] By using the IP address or MAC address, the location of the table can be found in the reported message records that contain the IP address or MAC address.

[0108] A garbage collection method for a distributed key-value database, wherein the distributed key-value database includes a client, a server, and a BMD server;

[0109] Each client uses its local clock to obtain the time signature (ts) and sends it to the server along with the ts in the write transaction request;

[0110] The server maintains the timestamp min_write_ts; the server periodically refreshes min_write_ts to ensure that min_write_ts continuously increases; if the timestamp of a write transaction request is less than min_write_ts, the server rejects the write transaction request.

[0111] The server periodically reports min_write_ts and latest_gc_ts to the bmd server.

[0112] The BMD server includes a garbage collection management table, which records the ID, parameter A, and parameter B of each server. Parameter A is min_write_ts, and parameter B is latest_gc_ts.

[0113] The BMD server periodically calculates the minimum value of parameter A in the above garbage collection management table; calculates the minimum value of parameter A to obtain latest_gc_ts; calculates the minimum value of parameter B to obtain safe_gc_ts.

[0114] On the BMD server side, based on the aforementioned latest_gc_ts, lock removal requests are sent to each server.

[0115] After the BMD server successfully unlocks each server, it records the timestamp latest_gc_ts corresponding to the successful unlock.

[0116] The BMD server broadcasts the aforementioned `latest_gc_ts` and `safe_gc_ts`.

[0117] The kv-server receives the latest_gc_ts and safe_gc_ts, performs garbage collection, and cleans up expired data that is less than safe_gc_ts.

[0118] It is possible that if the ts of the client's write transaction request is less than min_write_ts, the server rejects the write transaction request; after receiving the rejection message, the client waits for a period of time, or speeds up its local logical clock; it then uses its local clock to obtain the ts again, and re-includes the ts in the transaction request before sending it to the server.

[0119] It is possible that if the ts of the client's write transaction request is less than min_write_ts, the server rejects the write transaction request; after receiving the rejection message, the client speeds up its local clock, re-acquires the ts, and resends the transaction request with the aforementioned ts to the server.

[0120] It is possible that the server receives the broadcast of the above broadcast latest_gc_ts and records the above latest_gc_ts locally;

[0121] Figure 8Example 2 is a schematic diagram of a distributed key-value database system. The garbage collection management table includes the server's node ID and parameters A and B. It can be that the server receives the broadcast of `latest_gc_ts` and records it locally; the server periodically reports `latest_gc_ts` and the latest `min_write_ts` to the BMD server; the BMD server receives `latest_gc_ts` and the latest `min_write_ts` and records parameters A and B in the garbage collection management table, where parameter A is `min_write_ts` and parameter B is `latest_gc_ts`.

[0122] Figure 8 In the table, server_1 records the IP address or MAC address of server_1, and T11 is parameter B; that is, the specific value of latest_gc_ts, which is reported by server_1.

[0123] server_2 records the IP address or MAC address of server_2, and T22 is parameter B; that is, the specific value of latest_gc_ts, which is reported by server_2.

[0124] server_3 records the IP address or MAC address of server_3, and T33 is parameter B; that is, the specific value of latest_gc_ts, which is reported by server_3.

[0125] It is possible that the BMD server, based on the aforementioned latest_gc_ts, sends lock removal requests to each server;

[0126] The server receives the above lock release request, checks whether the transactions before the above latest_gc_ts timestamp are in a committed state or a successfully rolled-back state. If so, it returns a lock release success message to the BMD server; otherwise, it attempts the lock release operation and returns a lock release success message upon success.

[0127] It is possible that in the above server, a write transaction corresponds to 3 column families: CF_LOG, CF_IDX, and CF_TXN;

[0128] CF_IDX records the latest status of each key: the latest version number (wts) in effect, whether it is locked, and a short value.

[0129] The CF_LOG column records state information and version information (wts). The state includes lock, commit, and abort states.

[0130] The CF_TXN column records other keys within the same transaction.

[0131] It is possible that the server receives the above lock release request, checks the transactions before the above latest_gc_ts timestamp, changes the intermediate state recorded in the CF_LOG column from abort to lock state or commit state, and changes the transaction in the intermediate state to commit state or rolls back the transaction.

[0132] Yes, the server mentioned above can use the clock as the timestamp min_write_ts.

[0133] A distributed key-value database, including the aforementioned garbage collection method for distributed key-value databases.

[0134] A distributed server system is provided for running the aforementioned distributed key-value database.

[0135] Figure 9 One of the schematic diagrams of the operation steps in Example 4.

[0136] In a distributed clock system, the client does not have a unified clock source. Each node uses its local clock to obtain the time signature (TS) and sends it to the server along with the TS in the write transaction request.

[0137] Step K1.1: Message Txn: ts1 sent by client1;

[0138] Step K1.2: Message Txn: ts2 sent by client2.

[0139] The server's processing includes: Step K2.1: Periodically update `min_write_ts`. `min_write_ts` is a timestamp maintained by the server. Periodic updates ensure this timestamp continuously increments, while simultaneously preventing write transactions with `ts` less than `min_write_ts` from entering. That is, if `ts1` is less than `min_write_ts`, message `Txn:ts1` cannot be written successfully; if `ts2` is less than `min_write_ts`, message `Txn:ts2` also cannot be written successfully.

[0140] Step K2.2: The server periodically reports min_write_ts to bmd. The server's operation ensures that the server processes write transactions according to the logical timestamp increment. Transactions with timestamps less than the reported timestamp can be successfully written to the server. The client can wait for a certain timestamp or modify its local clock to make the timestamp greater than min_write_ts.

[0141] Step K3: The bmd process, responsible for GC management, collects the min_write_ts reported by each server and calculates the minimum value to obtain latest_gc_ts. Based on latest_gc_ts, it attempts to unlock the lock, and broadcasts the successful unlock to all servers.

[0142] Figure 10 The second schematic diagram of the operation steps in Example 4.

[0143] Step K4: The server periodically reports the latest received lastest_GC_ts and the current latest min_write_ts to bmd.

[0144] Step K5.1: bmd takes the minimum value of latest_gc_ts as safe_gc_ts, and takes the minimum value of min_write_ts as the new round of latest_gc_ts.

[0145] Step K5.2: Perform lock removal operations on each server based on the new round of latest_gc_ts. After the lock removal operation is successful, broadcast the safe_gc_ts timestamp to the server.

[0146] Step K6. The server receives the broadcast safe_gc_ts timestamp, performs GC garbage collection, and cleans up data in the LOG column ts that is less than the safe_gc_ts timestamp.

[0147] Second round of reporting confirmation: If steps K4, K5, and K6 are not performed, and GC is directly performed based on latest_gc_ts, and some servers do not receive latest_gc_ts and data migration has occurred, the benchmark for GC will be different for different servers, and some may accidentally delete transaction data.

[0148] This approach ensures that all servers receiving the latest_gc_ts reach a consensus at this timestamp. Only after consensus is reached can the safe_gc_ts be sent, which is then returned to the server. The server then performs garbage collection on this ts.

[0149] While the present invention has been described and illustrated with reference to preferred embodiments and several alternatives, the invention is not limited to the specific descriptions herein. Other alternatives or equivalent components may also be used to practice the invention.

Claims

1. A method for garbage collection in a distributed key-value database, the method comprising: The distributed key-value database comprises a client, a server and a bmd server. ​ Each client uses a local logical clock to obtain a timestamp ts, and sends the ts to the server in a write transaction request. The server locally maintains a timestamp min_write_ts, and periodically refreshes the min_write_ts to ensure that the min_write_ts is continuously increasing. If the ts of the write transaction request is less than the min_write_ts, the server rejects the write transaction request. The server periodically reports the min_write_ts and latest_gc_ts to the bmd server. The bmd server comprises a garbage collection management table, which records the ID, parameter A and parameter B of each server. The parameter A is the min_write_ts reported by the server, and the parameter B is the latest_gc_ts reported by the server. The bmd server periodically calculates the minimum parameter A and the minimum parameter B in the garbage collection management table, obtains the latest_gc_ts by calculating the minimum value in the parameter A, and obtains the safe_gc_ts by calculating the minimum value in the parameter B. The bmd server initiates a lock release request to each server based on the latest_gc_ts calculated based on the parameter A. After successfully completing the lock release of each server, the bmd server records the timestamp latest_gc_ts corresponding to the successful lock release. The bmd server broadcasts the latest_gc_ts and the safe_gc_ts. The kv-server receives the latest_gc_ts and the safe_gc_ts, saves them, performs garbage collection, and cleans up data that is less than the safe_gc_ts. In the next reporting, the kv-server reports the latest_gc_ts received this time together with the latest min_write_ts to the bmd server for second-round calculation.

2. The garbage collection method of the distributed key-value database according to claim 1, wherein if the ts of the write transaction request of the client is less than the min_write_ts, the server rejects the write transaction request; after receiving the rejection message, the client waits for a period of time or adjusts the local logical clock, and then uses the local clock to obtain the ts and sends the ts to the server in a transaction request.

3. The garbage collection method of the distributed key-value database according to claim 1, wherein the server maintains a local timestamp min_write_ts, and uses the local logical clock as the timestamp min_write_ts. The server periodically refreshes the min_write_ts to ensure that the min_write_ts is continuously increasing. ​ ​ ​ 4. The distributed key-value database garbage collection method of claim 1, wherein, The server receives the broadcast latest_gc_ts and safe_gc_ts, and records the latest_gc_ts and safe_gc_ts locally; The server periodically reports the latest_gc_ts and the current latest min_write_ts to the bmd server; the bmd server receives the latest_gc_ts and the current latest min_write_ts, records parameters A and B in the garbage collection management table, the parameter A is min_write_ts, and the parameter B is latest_gc_ts; the bmd server regularly calculates the minimum value of the parameters A and B to obtain new latest_gc_ts and safe_gc_ts.

5. The distributed key-value database garbage collection method according to claim 1, characterized in that, The bmd server initiates a lock release request to each server based on the latest_gc_ts calculated based on the parameter A; The server receives the lock release request, checks whether the transaction before the latest_gc_ts timestamp is in a commit state or a rollback success state, and returns a lock release success to the bmd server if yes; if there is a lock residual state, the server attempts to release the lock and returns success after the lock is successfully released.

6. The distributed key-value database garbage collection method according to claim 5, characterized in that, In the server, a write transaction corresponds to three column families: CF_LOG, CF_IDX, and CF_TXN; CF_IDX records the current latest state of each key; CF_LOG column records state status information and version information wts, and the state status includes a lock state lock, a commit state commit, and an intermediate state abort; CF_TXN column records all other keys of the same transaction.

7. The distributed key-value database garbage collection method according to claim 6, characterized in that, The server receives the lock release request, checks the transaction before the latest_gc_ts timestamp, pushes the CF_LOG column record in the intermediate state abort to the lock state lock and the commit state commit, and changes the transaction in the intermediate state to the commit state or rolls back the transaction.

8. The distributed key-value database garbage collection method according to claim 1, characterized in that, The server performs garbage collection based on the safe_gc_ts timestamp, and cleans up data expired before the safe_gc_ts; the bmd server obtains the safe_gc_ts by calculating the minimum value of the latest_gc_ts reported by all servers; The bmd server is an independently arranged server or a service process running in a selected server.

9. A distributed key-value database, characterized by: The distributed key-value database garbage collection method according to any one of claims 1 to 8.

10. A distributed server system, characterized by: The distributed server system is configured to run the distributed key-value database of claim 9.

Images