A concurrent data synchronization method of a Key-Value cache database, a computer device and a storage medium

Abstract

The application discloses a concurrent data synchronization method of a Key-Value cache database, computer equipment and a storage medium, and through a plurality of replication connections, a concurrent data synchronization link between Key-Value cache databases is established, and a parallel recovery command replication stream is established by a Key-Value cache database slave instance, so that the master instance and the slave instance of the Key-Value cache database can realize the synchronization of data changes concurrently through the plurality of replication connections under the condition of ensuring the eventual consistency, thereby better coping with the scene of high-concurrent write requests.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of databases, and particularly relates to a concurrent data synchronization method of a Key-Value cache database, a computer device and a storage medium. BACKGROUND

[0002] In order to improve the availability of a database and avoid the problem that a single point of failure causes a long-time unavailability of a database service, most databases support the deployment of a "one master and multiple backups" architecture, that is, a master instance supports data reading and writing in normal cases, and synchronizes (also referred to as replicates) data changes to each backup instance, so that the data of the master and each backup instance can be consistent. Once the master fails, the backup is switched to the master, thereby greatly reducing the service unavailability time caused by a single point of failure.

[0003] Currently, mainstream Key-Value (abbreviated as KV) cache databases such as Redis and memcached only use a single connection and a single thread for master-slave data synchronization. The advantage of single-connection and single-thread synchronization is that it is easy to implement and has low resource occupation, and the disadvantage is that the synchronization speed is slow when the database faces high concurrency pressure, especially when the master uses multi-threaded processing of write requests. The synchronization speed of single connection and single thread cannot match the processing speed of multi-threaded write requests. At this time, if synchronous replication is adopted, the master-slave synchronization will become the performance bottleneck of the entire service; if asynchronous replication is adopted, the master side will always accumulate a large amount of data to be synchronized, and can only perform full synchronization frequently, and once the master fails, these data will be lost, resulting in a large number of read requests being processed by the backend database, and the performance of the entire system will drop sharply. SUMMARY

[0004] The application aims to solve the problem of low data synchronization performance of traditional KV cache databases when facing high concurrency pressure. The application discloses a concurrent data synchronization method of a Key-Value cache database, a computer device and a storage medium, which greatly improves the data synchronization performance of traditional KV cache databases while ensuring data eventual consistency.

[0005] Technical solution: A concurrent data synchronization method of a Key-Value cache database, for a master instance, including the steps of processing write commands from a client, specifically including:

[0006] Step 1: The master instance receives and parses write commands from the client, hashes the write commands according to the Key contained in the write commands, and divides the write commands into a plurality of replication streams;

[0007] Step 2: The master instance maintains the following data structure for each replication stream:

[0008] a replication cache for storing commands;

[0009] a replication progress;

[0010] a replication synchronization progress for recording a command state, the command state comprising: a distributed command state and an executed command state;

[0011] Step 3: for a write command with a Key number of 1, the host instance appends the write command to the replication cache of the corresponding replication stream according to the contained Key through hashing;

[0012] for a write command with a Key number of multiple, the following steps are sequentially executed:

[0013] updating the replication synchronization progress of all replication streams to the distributed command state;

[0014] selecting an arbitrary replication stream as a specified replication stream;

[0015] inserting an RWAIT command into the replication cache of other replication streams; the RWAIT command comprises the number of the replication stream and the replication synchronization progress of the replication stream;

[0016] inserting the current write command into the replication cache of the specified replication stream.

[0017] Further, for the host instance, the steps of processing a replication handshake request from a slave instance further comprise:

[0018] the host instance receives the replication handshake request from the slave instance;

[0019] the host instance parses the replication handshake request to obtain the replication progress of all replication streams known by the slave instance;

[0020] the host instance determines whether there is a replication progress of a replication stream that is not within the replication cache range of the host instance among the replication progress of all replication streams known by the slave instance, if there is, the host instance initiates full synchronization, sends a replication handshake request response to the slave instance through the unique id for identifying the slave instance carried by the replication handshake request, and provides the slave instance with the replication progress and the replication synchronization progress of all replication streams contained by the host instance; if there is not, the host instance sends a replication handshake request response to the slave instance through the unique id for identifying the slave instance carried by the replication handshake request, and provides the slave instance with the replication progress and the replication synchronization progress of all replication streams contained by the host instance;

[0021] the host instance identifies whether all replication connections of the slave instance are ready, one replication stream corresponds to one replication connection; only when ready, the host instance creates a replication thread, and sends all commands in the replication cache of all replication streams to the slave instance through the replication connections according to the replication progress of each replication stream.

[0022] Further, for the slave instance, the following steps are included:

[0023] S100: The slave instance initiates a replication handshake request to the master instance using a replication connection according to the pre-configured master instance address, and provides the replication progress of each replication stream known by the slave instance to the master instance, and provides a unique id for identifying the slave instance to the master instance;

[0024] S200: After receiving the replication handshake request response of the master instance, the slave instance checks whether full synchronization is needed. If full synchronization is needed, the slave instance completes full synchronization on demand, and establishes a remaining replication connection responsible for all replication streams with the master instance according to the number of replication streams provided by the master instance; if full synchronization is not needed, the slave instance establishes a remaining replication connection responsible for all replication streams with the master instance according to the number of replication streams provided by the master instance;

[0025] S300: The slave instance receives commands in the replication cache of all replication streams sent by the master instance;

[0026] S400: The slave instance parses the commands in the replication cache of all replication streams sent by the master instance, and processes them according to the command type.

[0027] Further, S400 specifically includes:

[0028] The slave instance parses the commands in the replication cache of all replication streams sent by the master instance;

[0029] When the command type is a write command with a Key quantity of 1, the Key contained in the write command with a Key quantity of 1 is hashed, so that all write commands containing the same Key are distributed to the same execution thread, the replication synchronization progress of the replication stream where the write command with a Key quantity of 1 is located is updated to the executed command state, and the processing is completed;

[0030] When the command type is an RWAIT command, wait for the replication synchronization progress of the replication stream where the RWAIT command is located, update the replication synchronization progress of the replication stream where the RWAIT command is located to the executed command state, and then wait for the replication synchronization progress of the specified replication stream in the RWAIT command to be updated to the executed command state, and the processing is completed;

[0031] When the command type is a write command with multiple keys, first, the replication synchronization progress of other replication streams is waited to reach the executed command state, then it is checked whether the replication connection is valid, if it is not valid, it is directly exited and the execution is abandoned; if it is still valid, the replication synchronization progress of the replication stream where the write command with multiple keys is updated to the distributed command state, and the write command with multiple keys is distributed to an execution thread for execution, and the replication synchronization progress of the replication stream where the write command with multiple keys is updated to the executed command state before the next distribution.

[0032] Further, when the command type is a RWAIT command, in the process of waiting for the replication synchronization progress of the replication stream specified in the RWAIT command to be updated to the executed command state, first, it is checked whether the replication connection is valid, and only when the replication connection is invalid, it is checked whether the replication synchronization progress of the replication stream where the RWAIT command is reaches the distributed command state, if not, the replication synchronization progress of the replication stream where the RWAIT command is rolled back to the state of having executed the previous command, if it reaches, it is not necessary to roll back.

[0033] Further, in the execution thread of the slave instance, only when the replication synchronization progress of the replication stream where the write command with multiple keys is reaches the completed command state, the slave instance notifies the parsing thread of the replication stream to parse the next command.

[0034] Further, the parsing thread and the execution thread of the slave instance are merged, and the command of each replication stream is directly parsed and executed in the current thread.

[0035] Further, the slave instance uses a thread for receiving and parsing operation for each replication connection.

[0036] The application discloses a computer device, comprising a memory, a processor and a computer program stored in the memory and capable of running on the processor, wherein the processor implements the steps of the concurrent data synchronization method of the Key-Value cache database disclosed above when executing the computer program.

[0037] The application discloses a storage medium, wherein the storage medium stores a concurrent data synchronization method program, and the concurrent data synchronization method program implements the steps of the concurrent data synchronization method of the Key-Value cache database disclosed above when executed by at least one processor.

[0038] Advantages: compared with the prior art, the application has the following advantages:

[0039] (1) the method of the application can greatly improve the data synchronization performance of the traditional KV cache database under the premise of guaranteeing the final data consistency between the master and slave instances;

[0040] (2) The method can avoid frequent full synchronization and large data loss caused by host downtime when the host instance is under high concurrent pressure due to insufficient synchronization performance;

[0041] (3) The method establishes concurrent data synchronization links between Key-Value cache databases through multiple replication connections and parallel recovery command replication streams from Key-Value cache database slave instances, so that the master and slave instances of the Key-Value cache database can ensure eventual consistency and concurrently implement data change synchronization through multiple replication connections, thereby better coping with high-concurrency write request scenarios. BRIEF DESCRIPTION OF DRAWINGS

[0042] Figure 1 is a flowchart of the master implementation in a parallel data synchronization implementation method for a Key-Value cache database;

[0043] Figure 2 is a flowchart of the master implementation in a parallel data synchronization implementation method for a Key-Value cache database;

[0044] Figure 3 is a flowchart of the slave implementation in a parallel data synchronization implementation method for a Key-Value cache database;

[0045] Figure 4 is a flowchart of the slave implementation in a parallel data synchronization implementation method for a Key-Value cache database;

[0046] Figure 5 is a flowchart of the slave implementation in a parallel data synchronization implementation method for a Key-Value cache database; DETAILED DESCRIPTION

[0047] Example 1:

[0048] This embodiment can be used to implement the master instance data synchronization part in the parallel data synchronization implementation method for a Key-Value cache database, specifically including Figure 1 the part of processing the write command sent by the client of the master instance shown in Figure 2 the part of processing the replication handshake request of the slave instance shown in.

[0049] Now, referring to Figure 1 , the part of processing the write command sent by the client of the master instance will be further described. It includes:

[0050] S100: Create replication flow: receive and parse the write command from the client, check the number of keys contained in the write command; hash the keys contained in the write command, divide the write command into several groups, each group is called a replication flow.

[0051] The following related data structures are maintained for each replication flow:

[0052] Replication cache, used to store data synchronization commands for asynchronous replication; replication cache can use ring buffer, self-created and recycled buffer block linked list, etc. to realize the reuse of memory space; the data synchronization commands here include the commands generated by the host instance after receiving the write command from the client and the RWAIT command for controlling the synchronization between replication flows, and the implementation can be to store the write command in the replication cache as it is, or to do some processing, such as batch merging operation;

[0053] Replication progress, which is actually an integer with a large enough value range (such as 64 bits);

[0054] Replication synchronization progress, which is actually an integer with a large enough value range (such as 64 bits); replication synchronization progress is mainly for data synchronization commands containing multiple keys, and each such command in the replication flow needs to record two states with replication synchronization progress: already distributed command state and already executed command state. In some embodiments, 0 can be used to represent the initial state of the replication synchronization progress, and then the host increases the replication synchronization progress of the corresponding replication flow by 1 for each such command it distributes, and increases it by 1 again after the command is executed.

[0055] Replication progress and replication synchronization progress facilitate incremental synchronization and maintain data eventual consistency.

[0056] S110: For a write command with a Key quantity of 1, the write command is appended to the replication cache of the corresponding replication flow according to the contained Key through hashing; for a write command with multiple Keys, first update the replication synchronization progress of all replication flows to the executed write command state, then select any replication flow as the specified replication flow, and insert RWAIT commands into the cache of other replication flows, and finally insert the current write command into the replication cache of the specified replication flow.

[0057] If the host instance has not initiated a data synchronization request from the slave instance since it was running, S100-S110 can not be performed, and the related data structures do not have to be maintained, and S100-S110 and the related data structures can be dynamically enabled when a slave instance initiates a data synchronization request.

[0058] Now combined Figure 2 Further description is made to the part of the replication handshake request of the host instance processing the slave instance. Including:

[0059] S200: When the master instance receives the replication handshake request from the slave instance, i.e. the master instance and the slave instance perform data synchronization, check the replication progress of all replication streams provided by the slave instance; determine whether the replication progress of a replication stream is not within the replication cache range of the master instance, if the replication progress of a replication stream is not within the replication cache range of the master instance, the master initiates full synchronization, and provides the slave instance with the replication progress and replication synchronization progress of all replication streams contained in the master instance itself; if not, no full synchronization is performed, and the master provides the slave with the replication synchronization progress of all replication streams itself;

[0060] S210: After the master instance identifies that all replication connections of a certain slave instance are ready through the id carried by the replication handshake request, a replication thread is created, and the commands in the replication cache are sent to the slave instance through each replication connection from the replication progress of each replication stream.

[0061] Embodiment 2:

[0062] The embodiment can be used to implement the slave instance data synchronization part in the parallel data synchronization implementation method of the Key-Value cache database, and specifically includes Figure 3 the part of initiating a replication request to the master instance using a replication connection, Figure 4 the part of parsing the replication command sent by the master instance, and Figure 5 the part of executing the command by the slave instance.

[0063] Now, the part of initiating a replication request to the master instance using a replication connection will be further described in combination with Figure 3

[0064] including:

[0065] S300: The slave instance initiates a replication handshake request to the master instance using a replication connection, and provides the master instance with the replication progress of each replication stream known by the slave instance, and provides the master with a unique id for identifying the slave instance according to the pre-configured address of the master instance;

[0066] S310: After receiving the replication handshake request response of the master instance, check whether full synchronization is needed, if full synchronization is needed, the slave instance completes full synchronization as needed, and establishes the remaining replication connections responsible for all replication streams with the master according to the number of replication streams provided in the response, and initiates a replication request. The number of replication streams used by the slave is informed by the master during the handshake process, and the number of replication streams used by the slave is consistent with the number of replication streams of the master. If full synchronization is not needed, establish the remaining replication connections responsible for all replication streams with the master according to the number of replication streams provided in the response, and initiate a replication request.

[0067] ​S320: The slave instance uses one thread for each replication connection to receive and parse data, thereby achieving parallel processing. After each replication connection completes the handshake, it starts receiving the commands in the replication buffer sent by the master.

[0068] Now, the slave instance executing commands will be further described. Figure 4 The parsing of the replication command sent by the master instance will be further described. It includes:

[0069] S400: The command in the replication buffer sent by the master instance is parsed, the command type is checked, and the command is processed according to the command type:

[0070] For the data synchronization command containing only a single Key, the replication synchronization progress of other replication streams is waited for, and then the contained Key is hashed, so that all commands containing the same Key are distributed to the same execution thread, the replication synchronization progress of the replication stream where the command is located is updated to the executed command state, and the processing is completed;

[0071] For the RWAIT command, the replication synchronization progress of the replication stream where the command is located is waited for, the replication synchronization progress of the replication stream where the command is located is increased to the executed command state, and then (through a thread synchronization mechanism such as a semaphore, a condition variable, etc.) the replication synchronization progress of the replication stream specified in the RWAIT command is waited for to be updated to the executed command state, i.e., the processing is completed, and the next command can be parsed. If it is found during the waiting process that the replication connection is invalid, it is checked whether the replication synchronization progress of the replication stream where the command is located reaches the distributed command state, if not, the replication synchronization progress of the replication stream where the command is located is rolled back to the state of executing the previous command; otherwise, no rollback is needed.

[0072] For the data synchronization command that may contain multiple Keys, the replication synchronization progress of other replication streams is first waited for to reach the executed command state, and then it is checked whether the replication connection is valid, if it is invalid, it is directly exited and the execution is abandoned; if it is still valid, the replication synchronization progress of the current replication stream is increased to the distributed command state, and the command is arbitrarily (such as randomly, using a round-robin strategy, etc.) distributed to an execution thread for execution, and the replication synchronization progress of the replication stream where the command is located is waited for to be increased to the executed command state before the next distribution.

[0073] For the command that does not operate data (such as the command required to implement the heartbeat), it is arbitrarily (such as randomly, using a round-robin strategy, etc.) distributed to an execution thread for execution.

[0074] Now, the slave instance executing commands will be further described. Figure 5 The parsing of the replication command sent by the master instance will be further described. It includes:

[0075] When a command is executed from the execution thread of a slave instance, if the command is from a replication stream and is a data synchronization command that may contain multiple keys, the replication synchronization progress of the replication stream in which the command is located is increased to the completed command state to notify the parsing thread of the replication stream to continue working.

[0076] In this embodiment, the parsing thread of a replication stream can be merged with the execution thread, and the commands of each replication stream are directly parsed and executed in the current thread, and the distribution step is omitted. However, the step of increasing the replication synchronization progress of the replication stream in which the command is located to the completed current command state cannot be omitted.

[0077] Embodiment 3:

[0078] The embodiment discloses a concurrent data synchronization method of a Key-Value cache database, so that the master instance and the slave instance of the Key-Value cache database can concurrently implement synchronization of data changes through multiple replication connections under the condition of ensuring eventual consistency, thereby better coping with a high-concurrency write request scenario. The specific operation steps include:

[0079] The step of establishing a concurrent data synchronization link between Key-Value cache databases by using multiple replication connections;

[0080] The step of parallelly recovering a command replication stream of a Key-Value cache database slave instance.

[0081] The above-mentioned steps will be further described.

[0082] The step of establishing a concurrent data synchronization link between Key-Value cache databases by using multiple replication connections specifically includes:

[0083] Step 1: The master instance of the Key-Value cache database divides the received write commands into several groups according to the contained keys by using a hash, and each group is called a replication stream. When the master instance and the slave instance perform data synchronization, a network connection (referred to as a replication connection in the above text) is established for each replication stream to perform replication.

[0084] Step 2: The master instance of the Key-Value cache database maintains the following data structure for each replication stream:

[0085] A linear list for storing data synchronization commands, referred to as a replication cache, to facilitate asynchronous replication;

[0086] Two integers with a large enough range (such as 64 bits), referred to as a replication progress and a replication synchronization progress, to facilitate incremental synchronization and maintain data eventual consistency;

[0087] The replication synchronization progress mainly aims at the data synchronization command containing multiple keys, and each data synchronization command in the replication stream needs to record two states, the distributed command state and the executed command state, by the replication synchronization progress. In some embodiments, the initial state of the replication synchronization progress can be represented by 0, and then the replication synchronization progress of the corresponding replication stream is increased by 1 each time the host distributes a data synchronization command of this type, and is increased by 1 again after the command is executed.

[0088] The scenario of the host instance processing the write request includes the following steps:

[0089] The working thread / process of the host instance parses the write request received from the client and executes it, and after the execution is completed, the following processing is performed according to the number of keys contained in the data update command; because the replication can be asynchronous, the request execution can be completed first, and then the replication operation can be started; according to the needs, it can also be implemented synchronously, and the response can be returned to the client only after the replication is completed.

[0090] When the received data synchronization command request contains only a single-key data update (including insertion, deletion, and modification) operation, the working thread / process stores the data synchronization command into a certain replication cache according to the key contained in the data synchronization operation through hashing, and waits for asynchronous replication.

[0091] When the received data synchronization command request belongs to a data update operation that can contain multiple keys, the working thread / process first increases the replication synchronization progress of all replication streams to the executed command state, then randomly (such as using the round-robin strategy, etc.) selects a replication stream as the specified replication stream, and adds a RWAIT command for synchronization to the replication cache of the remaining replication streams, the RWAIT command contains the number and replication synchronization progress of the currently selected replication stream; finally, the data synchronization command containing multiple keys is added to the replication cache of the currently selected replication stream.

[0092] After the above classification processing, for any key, only the command containing the key exists in a single replication stream; for the command containing multiple keys, the slave instance will implement a certain synchronization waiting operation by means of the inserted RWAIT command. In this way, the slave instance can process each replication stream in parallel on the basis of ensuring the eventual consistency of data.

[0093] The scenario of the host instance processing the replication handshake request of the slave instance includes the following steps:

[0094] The slave instance first uses a replication connection to initiate a replication handshake request to the host instance, and provides the replication progress of each replication stream known by the slave instance and a unique id for identifying the slave instance to the host instance.

[0095] When the master instance receives the replication handshake request from the slave instance, it checks the replication progress of each replication stream provided by the slave instance to confirm the starting position of the data sent to the slave or to perform full synchronization; if full synchronization is needed, the master instance initiates full synchronization first, and the data file sent in full synchronization contains the replication progress and replication synchronization progress of all replication streams; then, the master instance provides the replication synchronization progress of all replication streams of the master instance to the slave instance, so that the slave instance uses the correct number of replication connections to connect to the master instance. The data file sent in full synchronization is not necessarily the latest one, and a previously generated data file can be used as long as the existing replication progress of the master instance can be connected to the record in the data file. Therefore, after the data file is synchronized, the master instance needs to send the latest data. After the slave instance receives the replication handshake request response, it establishes the remaining replication connections with the master for all replication streams according to the number of replication streams provided in the replication handshake request response, and completes full synchronization as needed.

[0096] When the master instance discovers that all replication connections have been established by identifying the id carried in the replication handshake request, it creates a new replication thread, and starts from the replication progress of each replication stream to send the commands in the replication cache of all replication streams to the slave instance through each replication connection, that is, the replication command stream.

[0097] The scenario of the master instance processing write requests and the scenario of the master instance processing replication handshake requests of the slave instance are relatively independent and have no time or logical order.

[0098] The steps of the Key-Value cache database slave instance recovering the replication command stream in parallel, specifically include:

[0099] Step 1: The slave instance uses a thread for data reception and analysis for each replication connection, thereby achieving parallel processing.

[0100] After a command is parsed, it is processed according to its type:

[0101] For data synchronization commands containing only a single Key, the Key is hashed to make all commands containing the same Key distributed to the same execution thread for execution.

[0102] For the RWAIT command, first increase the replication synchronization progress of the replication stream where the RWAIT command is located to the executed command state, then (through a thread synchronization mechanism such as a semaphore, a condition variable, etc.) wait for the replication synchronization progress (referred to as the target replication synchronization progress) of the replication stream (referred to as the target replication stream) corresponding to the record number in the RWAIT command to be updated to the replication synchronization progress recorded in the RWAIT command, that is, complete processing, and the next command can be parsed. If the replication connection is found to be invalid during the waiting process, check whether the target replication synchronization progress of the target replication stream reaches the executed command state, if not, roll back the replication synchronization progress of the replication stream where the RWAIT command is located to the state of the executed previous command; otherwise, it is indicated that the command to be synchronized has been distributed, and there is no need to roll back.

[0103] For the data synchronization command that can contain multiple keys, first wait for the replication synchronization progress of other replication streams to reach the executed command state (to ensure that the relative execution order of the current data synchronization command and the data synchronization command in other replication streams is consistent with the host), then check whether the replication connection is valid, if not, directly exit and give up execution; if still valid, increase the synchronization progress of the current replication stream to the distributed command state, and distribute the data synchronization command to an execution thread in an arbitrary manner (such as randomly, using a round-robin strategy, etc.), and wait for the synchronization progress of the replication stream where the data synchronization command is located to increase to the executed command state before the next distribution.

[0104] For the command that does not operate data (such as the command required to implement a heartbeat), distribute it to an execution thread in an arbitrary manner (such as randomly, using a round-robin strategy, etc.).

[0105] Step 2: When a data synchronization command is executed in the execution thread of the slave instance, if the data synchronization command comes from a replication stream and is a data synchronization command that can contain multiple keys, the replication synchronization progress of the replication stream where the data synchronization command is located is increased to the executed command state, to inform the parsing thread of the replication stream to continue working.

[0106] The application can also support the scenario that multiple databases are contained in the same instance, and only the database number operated by the command is added to the calculation of the hash, to ensure that for any key in any database, only the command operating the key is distributed to a certain replication stream.

[0107] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer readable storage medium, and when executed, can include the processes of the above-mentioned embodiment methods. Any reference to memory, storage, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory or optical memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration but not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0108] Any combination of the technical features of the above embodiments can be made. In order to make the description simple, all possible combinations of the technical features in the above embodiments are not described, but as long as the combination of the technical features does not exist, it should be considered as the scope of the present application.

[0109] The above embodiments only express several implementation manners of the present application, and the description is more specific and detailed, but it should not be understood as a limitation on the scope of the patent. It should be pointed out that for those skilled in the art, without departing from the concept of the present application, some modifications and improvements can be made, which are all within the protection scope of the present application. Therefore, the protection scope of the patent of the present application should be subject to the appended claims.

Claims

1. A concurrent data synchronization method for a Key-Value cache database, characterized in that: For the host instance, the steps include processing a write command from a client, specifically including: Step 1: The host instance receives and parses the write command from the client, hashes according to the Key contained in the write command, and divides the write command into several replication streams; Step 2: The host instance maintains the following data structures for each replication stream: A replication cache for storing commands; A replication progress; A replication synchronization progress for recording the command state, including: a distributed command state and an executed command state; Step 3: For a write command with a Key quantity of 1, the host instance appends the write command to the corresponding replication stream of the replication cache according to the contained Key through hashing; For a write command with multiple Key quantities, the following steps are sequentially executed: Update the replication synchronization progress of all replication streams to the distributed command state; Select any one replication stream as a specified replication stream; Insert a RWAIT command into the replication cache of the other replication streams; the RWAIT command includes the number of the replication stream and the replication synchronization progress of the replication stream; Insert the current write command into the replication cache of the specified replication stream; For the host instance, the steps also include processing a replication handshake request from a slave instance, specifically including: The host instance receives a replication handshake request from a slave instance; The host instance parses the replication handshake request to obtain the replication progress of all replication streams known to the slave instance; The host instance determines whether there is a replication progress of a replication stream that is not within the replication cache range of the host instance among the replication progress of all replication streams known to the slave instance, if there is, the host instance initiates a full synchronization, sends a replication handshake request response to the slave instance through the unique id carried by the replication handshake request for identifying the slave instance, and provides the slave instance with all replication progress and replication synchronization progress of the replication streams contained by the host instance itself; if not, the host instance sends a replication handshake request response to the slave instance through the unique id carried by the replication handshake request for identifying the slave instance, and provides the slave instance with all replication progress and replication synchronization progress of the replication streams contained by the host instance itself; The host instance identifies whether all replication connections of the slave instance are ready, one replication stream corresponds to one replication connection; only when ready, the host instance creates a replication thread, starts from the replication progress of each replication stream, and sends all commands in the replication cache of the replication streams to the slave instance through each replication connection.

2. The concurrent data synchronization method of a Key-Value cache database according to claim 1, characterized in that: For the slave instance, the steps include: S100: The slave instance initiates a replication handshake request to the host instance using a replication connection according to the pre-configured host instance address, provides the host instance with the replication progress of each replication stream known by the slave instance, and provides the host instance with a unique id for identifying the slave instance; S200: After receiving the replication handshake request response of the master instance, the slave instance checks whether full synchronization is needed. If full synchronization is needed, the slave instance completes full synchronization on demand, and establishes remaining replication connections with the master instance for all replication streams according to the number of replication streams provided by the master instance; if full synchronization is not needed, the slave instance establishes remaining replication connections with the master instance for all replication streams according to the number of replication streams provided by the master instance; S300: The slave instance receives commands in the replication cache of all replication streams sent by the master instance; S400: The slave instance parses the commands in the replication cache of all replication streams sent by the master instance, and processes them according to the command type.

3. The concurrent data synchronization method of a Key-Value cache database according to claim 2, characterized in that: S400 specifically includes: The slave instance parses the commands in the replication cache of all replication streams sent by the master instance. When the command type is a write command with one Key, the Key contained in the write command with one Key is hashed, so that all write commands containing the same Key are distributed to the same execution thread, the replication synchronization progress of the replication stream where the write command with one Key is located is updated to the executed command state, and the processing is completed. When the command type is an RWAIT command, the replication synchronization progress of the replication stream where the RWAIT command is located is waited for, the replication synchronization progress of the replication stream where the RWAIT command is located is updated to the executed command state, and then the replication synchronization progress of the replication stream specified in the RWAIT command is waited for to be updated to the executed command state, and the processing is completed. When the command type is a write command with multiple Keys, the replication synchronization progress of other replication streams is first waited for to reach the executed command state, then it is checked whether the replication connection is valid, if it is not valid, it is directly exited and the execution is abandoned; if it is still valid, the replication synchronization progress of the replication stream where the write command with multiple Keys is located is updated to the distributed command state, and the write command with multiple Keys is arbitrarily distributed to an execution thread for execution, and the replication synchronization progress of the replication stream where the write command with multiple Keys is located is waited for to be updated to the executed command state before the next distribution.

4. The concurrent data synchronization method of a Key-Value cache database according to claim 3, characterized in that: When the command type is an RWAIT command, during the process of waiting for the replication synchronization progress of the replication stream specified in the RWAIT command to be updated to the executed command state, it is first checked whether the replication connection is valid, and only when the replication connection is invalid, it is checked whether the replication synchronization progress of the replication stream where the RWAIT command is located reaches the distributed command state, if not, the replication synchronization progress of the replication stream where the RWAIT command is located is rolled back to the state of executing the previous command, if yes, no rollback is needed.

5. The concurrent data synchronization method of a Key-Value cache database according to claim 3, characterized in that: In the execution thread of the slave instance, only when the replication synchronization progress of the replication stream where the write command with multiple Keys is located is in the completed command state, the slave instance notifies the parsing thread of the replication stream to parse the next command.

6. The concurrent data synchronization method of a Key-Value cache database according to claim 3, characterized in that: The parsing thread and the execution thread of the slave instance are merged, and the commands of each replication stream are directly parsed and executed in the current thread.

7. The concurrent data synchronization method of a Key-Value cache database according to claim 2, characterized in that: The slave instance uses a thread for receiving and parsing operations for each replication connection.

8. A computer device, comprising: The computer program is stored in the memory and can be run on the processor, and the processor executes the computer program to implement the steps of the concurrent data synchronization method of the Key-Value cache database according to any one of claims 1 to 7.

9. A storage medium, characterized by The storage medium stores the concurrent data synchronization method program, and the concurrent data synchronization method program is executed by at least one processor to implement the steps of the concurrent data synchronization method of the Key-Value cache database according to any one of claims 1 to 7.

Images