Data update method, data block, storage medium, electronic device, and computer program product

Abstract

Provided are a data update method, a data block, a storage medium, an electronic device, and a computer program product. The data update method comprises: reading a data block in a memory (S202); performing lock contention operation on an unlocked first slot in the data block by means of a compare-and-swap (CAS) operation instruction (S204); when lock contention is successful, locking the first slot by modifying a lock flag (S206); and writing, in an appending mode, a key-value pair to the first slot having been locked by a current thread (S208), and making the key-value pair valid by modifying a tail flag.

Description

Data updating method, data block, storage medium, electronic device and computer program product

[0001] Cross-reference to related applications

[0002] The present disclosure is based on Chinese Patent Application No. CN202410992529.X entitled “Data updating method, data block, storage medium, electronic device and computer program product” filed on July 23, 2024, and claims priority to the patent application, the disclosure of which is incorporated herein by reference in its entirety. TECHNICAL FIELD

[0003] The present disclosure relates to the field of information technology, and in particular, to a data updating method, a data block, a storage medium, an electronic device and a computer program product. BACKGROUND

[0004] The traditional computing storage separation architecture only splits computing resources and storage resources, and with the development of high-speed networks, new computing storage decoupling systems that decouple computing, memory and storage are gradually widely applied under distributed architecture. In the new computing storage decoupling system, a computing node manages and accesses a remote decoupled distributed memory pool through Remote Direct Memory Access (RDMA) technology. The memory pool generally uses a tree or skip list index structure to store data, and each leaf node at the bottom (leaf layer) can be regarded as a data block containing a plurality of groups of Key-Value (KV) form data.

[0005] To improve the efficiency of network interaction, the computing node usually reads multiple groups of KV from the remote memory pool to the local cache of the computing node at one time for operation. However, when the computing node is multi-threaded and concurrently writes to the same data block, or when a data block is written full by a thread and triggers a merge (with sorting and splitting) process, if other threads write (update) KV to the data block, data conflicts and data anomalies may occur, making it difficult to guarantee data consistency and security isolation.

[0006] To ensure the security isolation of inter-thread operations, a general solution is to add a block granularity level mutex to the data block by the thread currently accessing the data block. Although this lock mechanism guarantees the security isolation of concurrent access to the same data block by different threads, it also easily leads to a significant reduction in the write concurrency capability of the system, and thus reduces the write concurrency performance of the system.

[0007] In summary, there is no good solution to the above problems. SUMMARY

[0008] The data updating method, the data block, the storage medium, the electronic device and the computer program product provided in the embodiments of the present disclosure can at least solve the problem that data consistency and security isolation are difficult to guarantee due to concurrent operation between threads in the related art.

[0009] According to an embodiment of the present disclosure, a data updating method is provided, which includes: reading a data block in a memory, wherein the data block includes a plurality of slot positions for storing key-value pairs, and each slot position is attached with a lock flag bit and an end flag bit; performing a lock acquisition operation on a first slot position in the data block that is not locked by a compare-and-swap (CAS) operation instruction, wherein the CAS operation instruction has atomicity; in the case that the lock acquisition is successful, locking the first slot position by modifying the lock flag bit; and writing the key-value pair into the first slot position locked by the current thread by using an append-write manner, and making the key-value pair valid by modifying the end flag bit.

[0010] According to another embodiment of the present disclosure, a data block is provided, which includes a plurality of slot positions for storing key-value pairs, and each slot position is attached with a lock flag bit and an end flag bit, the lock flag bit is used to indicate a lock state of the slot position, and the lock state includes locked and not locked, and the end flag bit is used to indicate whether the key-value pair is valid.

[0011] According to still another embodiment of the present disclosure, a computer-readable storage medium is further provided, and the storage medium stores a computer program, wherein the computer program is run by a processor to perform the steps in any of the method embodiments.

[0012] According to still another embodiment of the present disclosure, an electronic device is further provided, which includes a memory and a processor, the memory stores a computer program, and the processor is configured to run the computer program to perform the steps in any of the method embodiments.

[0013] According to still another embodiment of the present disclosure, a computer program product is further provided, which includes a computer program, and the computer program is run by a processor to perform the steps in any of the method embodiments.

[0014] The data updating method, the data block, the storage medium, the electronic device and the computer program product provided in the embodiments of the present disclosure can at least solve the problem that data consistency and security isolation are difficult to guarantee due to concurrent operation between threads in the related art. BRIEF DESCRIPTION OF DRAWINGS

[0015] FIG. 1 is a hardware structure block diagram of a data updating method according to an embodiment of the present disclosure;

[0016] FIG. 2 is a flow chart of a data updating method according to an embodiment of the present disclosure;

[0017] FIG. 3 is a structural schematic diagram of a data block according to an embodiment of the present disclosure;

[0018] FIG. 4 is a structural schematic diagram of a data block according to an embodiment of the present disclosure;

[0019] FIG. 5 is a flow chart of multi-thread concurrent read and write according to an embodiment of the present disclosure;

[0020] FIG. 6 is a schematic diagram of multi-thread lock acquisition operation according to an embodiment of the present disclosure;

[0021] FIG. 7 is a schematic diagram of next slot lock acquisition operation after thread lock acquisition failure according to an embodiment of the present disclosure;

[0022] FIG. 8 is a schematic diagram of key-value pair writing after thread 1 lock acquisition success according to an embodiment of the present disclosure;

[0023] FIG. 9 is a schematic diagram of key-value pair writing after thread 2 lock acquisition success according to an embodiment of the present disclosure;

[0024] FIG. 10 is a schematic diagram of data block reading by thread 3 according to an embodiment of the present disclosure;

[0025] FIG. 11 is a flow chart of data block merging / splitting according to an embodiment of the present disclosure;

[0026] FIG. 12 is a schematic diagram of data block merging by thread 2 according to an embodiment of the present disclosure;

[0027] FIG. 13 is a schematic diagram of data block splitting according to an embodiment of the present disclosure;

[0028] FIG. 14 is a schematic diagram of data block merging and data read-write operation concurrency according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0029] Hereinafter, the embodiments of the present disclosure will be described in detail with reference to the accompanying drawings and in conjunction with embodiments.

[0030] It should be noted that the terms "first", "second", and the like in the specification and claims of the present disclosure and the above-described accompanying drawings are used to distinguish similar objects, and do not necessarily have to describe a specific order or sequence.

[0031] The method embodiments provided in the embodiments of the present disclosure can be executed in a mobile terminal, a computer terminal or similar computing device. Taking the case of running on a computer terminal, Fig. 1 is a hardware structure block diagram of the data updating method according to an embodiment of the present disclosure, as shown in Fig. 1, the hardware single board can include one or more (only one is shown in Fig. 1) processors 12 (the processor 12 can include but is not limited to a processing device such as a microprocessor MCU or programmable logic device) and a memory 14 for storing data, wherein the above-mentioned computer terminal can also include a transmission device 16 for communication function and an input and output device 18. Those skilled in the art can understand that the structure shown in Fig. 1 is only schematic, which does not limit the structure of the above-mentioned computer terminal. For example, the computer terminal can also include more or less components than those shown in Fig. 1, or have a different configuration from that shown in Fig. 1.

[0032] The memory 14 can be used to store computer programs, for example, software programs of application software and modules, such as the computer program corresponding to the data updating method in the embodiments of the present disclosure, and the processor 12 executes various functional applications and the data updating method by running the computer program stored in the memory 14, that is, implements the above-mentioned method. The memory 14 can include a high-speed random access memory, and can also include a non-volatile memory, such as one or more magnetic storage devices, flash memories, or other non-volatile solid-state memories. In some examples, the memory 14 can further include a memory remotely arranged with respect to the processor 12, and these remote memories can be connected to the computer terminal through a network. Examples of the above-mentioned network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network and a combination thereof.

[0033] The transmission device 16 is used to receive or send data via a network. Specific examples of the above-mentioned network can include a wireless network provided by a communication provider. In one example, the transmission device 16 includes a network adapter (Network Interface Controller, NIC for short), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 16 can be a radio frequency (Radio Frequency, RF for short) module, which is used to communicate with the Internet in a wireless manner.

[0034] In an embodiment of the present disclosure, a data updating method is provided, and Fig. 2 is a flow chart of the data updating method according to an embodiment of the present disclosure, as shown in Fig. 2, the flow includes the following steps:

[0035] In step S202, a data block in the memory is read, wherein the data block includes a plurality of slot positions for storing key-value pairs, and each slot position is attached with a lock flag bit and an end flag bit;

[0036] In step S204, a first slot in the data block that is not locked is subjected to a lock acquisition operation through a compare-and-swap (CAS) operation instruction, where the CAS operation instruction has atomicity.

[0037] In step S206, in the case of successful lock acquisition, the first slot is locked by modifying the lock flag.

[0038] In step S208, the key-value pair is written to the first slot locked by the current thread through an append-write manner, and the key-value pair is validated by modifying the end flag.

[0039] In this embodiment, the compare-and-swap (CAS) operation instruction is an atomic operation used to manage shared data in a multi-processor system. The CAS operation has atomicity, meaning that it will not be interrupted by other threads or processes during execution, which guarantees the consistency and synchronization of data in a multi-threaded environment.

[0040] In this embodiment, append-write (update) is a data update strategy to ensure data consistency and integrity, in which the update of data is not directly modified on the original data, but only new data records (i.e., new key-value pairs) are added to achieve the update.

[0041] The execution subject of the above steps can be a thread. A thread is the smallest unit of operation scheduling by an operating system and also the smallest unit of program execution flow. In an operation device, multiple threads can run concurrently, and in a high-concurrency operation scenario, multiple threads can simultaneously perform update operations on a data block.

[0042] In the embodiments of this disclosure, through the above steps S202 to S208, each thread can first acquire a slot lock through a CAS operation instruction, and then write a key-value pair to the locked slot, which ensures the safe isolation between concurrent multi-threads, avoids multiple threads from simultaneously operating a slot, guarantees data consistency, and thus solves the problem that data consistency and safe isolation are difficult to guarantee due to concurrent thread operations in related technologies. In addition, the granularity of data locks is reduced to the key-value level in the embodiments of this disclosure, which can guarantee the write concurrency capability of the system compared to the granularity of the data block level, and can avoid the reduction of the write concurrency capability of the system caused by frequent lock operations when different threads access a data block.

[0043] In some embodiments, step S204 can include the following steps:

[0044] In step S2042, the CAS operation instruction is sent to the memory. The CAS operation instruction is used to compare the lock flag of the first slot with an expected original value carried in the CAS operation instruction, and modify the lock flag of the first slot to a new value carried in the CAS operation instruction if the lock flag of the first slot is the same as the expected original value. The lock flag being the expected original value indicates that the slot is not locked, and the lock flag being the new value indicates that the slot is locked.

[0045] In step S2044, a return value of the CAS operation instruction is obtained. The return value is used to indicate a lock acquisition state of the first slot by the current thread, and the lock acquisition state includes lock acquisition success and lock acquisition failure.

[0046] In this embodiment, the CAS operation instruction generally consists of three parameters: a memory location (address), an expected original value, and a new value. The execution flow of the CAS operation is as follows: comparison: check whether the current value of the specified memory location matches the expected value; exchange: if the current value matches the expected value, update the value of the memory location to the new value; return: after the operation is completed, return a result indicating whether the operation is successful.

[0047] In this embodiment, a thread can first determine an empty slot to be locked according to a data block read in advance, and set the address in the CAS operation instruction to the lock flag corresponding to the slot. Since each thread reads a data block into the local cache of the thread first, and then sends a CAS operation instruction to acquire a lock, multiple threads may simultaneously attempt to acquire a lock of a slot. The atomic property of the CAS operation instruction can ensure that only one CAS operation instruction succeeds in acquiring a lock, and once a CAS operation instruction succeeds in acquiring a lock, the lock flag is modified to a new value, and other CAS operation instructions cannot match successfully.

[0048] In some embodiments, if multiple threads simultaneously perform a lock acquisition operation on a slot, only one thread can succeed in acquiring a lock and receive a return value indicating lock acquisition success, and the other threads all fail to acquire a lock and receive a return value indicating lock acquisition failure. For example, if the lock acquisition succeeds, the return value can be TRUE, and if the lock acquisition fails, the return value can be FALSE, but the present disclosure is not limited thereto.

[0049] In this embodiment, the lock flag has two locking states: locked and unlocked. A thread can determine whether a slot is locked by other threads according to the lock flag. If a slot is locked, it means that the slot is being written with data or has been written with data, and other threads cannot operate on the slot, thereby avoiding repeated writing of data in the same slot and ensuring data consistency.

[0050] In an example embodiment, the expected original value in the CAS operation instruction can be set as 0, i.e. the lock flag is 0, indicating that the slot is not locked; the new value in the CAS operation instruction can be set as 1, i.e. the lock flag is 1, indicating that the slot is locked. But the specific values of the two locking states of the lock flag are not limited in the present disclosure.

[0051] In some embodiments, the method further comprises: step S207, in the case of lock failure, continuing to perform a lock operation on a second slot in the data block which is not locked by using the CAS operation instruction.

[0052] In the present embodiment, the CAS operation instruction of each thread can search the slots of the data block from front to back for a slot which is not locked, and lock the first slot which is not locked. If the current slot fails to be locked, the next slot is searched. The present disclosure does not require the order of the written key-value pairs, and the subsequent sorting and merging can be performed.

[0053] In some embodiments, the step S202 of reading the data block in the memory can comprise the following steps:

[0054] Step S2022, reading the lock flag and the end flag of each slot in the data block;

[0055] Step S2024, in the case that the end flag is a preset valid value, reading the key-value pair stored in the corresponding slot; or,

[0056] Step S2026, in the case that the end flag is a preset invalid value, not reading the key-value pair stored in the corresponding slot.

[0057] In the present embodiment, the end flag comprises two states of valid and invalid. For example, the preset valid value can be 1 and the preset invalid value can be 0, but the specific values of the two states of the end flag are not limited in the present disclosure.

[0058] In some embodiments, each data block is attached with a start version number and an end version number, and each slot is further attached with a key-value pair version number.

[0059] In some embodiments, the method further comprises the following step: step S201, in the case that the data block satisfies the following conditions simultaneously, determining that the data block is in a stable state, and performing a read operation or a write operation on the data block:

[0060] Condition 1, the start version number of the data block is the same as the end version number;

[0061] Condition 2, the end flag bit of at least one slot in the data block is a preset invalid value;

[0062] Condition 3, the key-value pair version number of the slot that has been locked in the data block is greater than the start version number or the end version number by 1;

[0063] Condition 4, the key-value pair version number of the slot that has not been locked in the data block is the same as the start version number or the end version number.

[0064] In this embodiment, step S201 can occur before step S202 or after step S202, and the present disclosure does not limit this.

[0065] In this embodiment, the stable state means that the data block is not currently performing merging, splitting, or the like, and each key-value pair in the data block is stable and reliable. During the merging or splitting operation of the data block, if a read-write operation is performed, data anomalies may occur, and data consistency cannot be guaranteed. Therefore, in the present embodiment, the stability of the data block can be judged based on the start version number, the end version number, and the key-value pair version number, and the safe isolation of the read-write operation and the merging / splitting operation is realized.

[0066] In some embodiments, after reading the data block in the memory in step S202, the method further includes: step S203, re-reading the data block if the data block meets a preset condition.

[0067] In some embodiments, the preset condition includes at least one of the following:

[0068] Preset condition 1, the end flag bit of each slot in the data block is a preset valid value;

[0069] Preset condition 2, the key-value pair version number of the slot that has been locked in the data block is the same as the start version number;

[0070] Preset condition 3, the start version number of the data block is different from the end version number.

[0071] In this embodiment, if the data block meets any of the above preset conditions, it indicates that the read data block is in a non-stable state. If the data block meets the preset condition 1, it indicates that all slots in the data block are full, and since the slot is full, it will trigger the splitting / merging operation of the data block, so the state of the data block at this time is not stable. If the data block meets the preset condition 2 or the preset condition 3, it may be due to the update of the start version number of the data block, and at this time the data block is performing version update (such as merging or splitting) and write back, and the state of the data block is not stable.

[0072] In the embodiments of the present disclosure, the data block can only read data from the data block or write data into the data block when the data block is stable, and if the data is unstable, no operation can be performed, and the data block needs to be read from the memory again. In the embodiments, the stability of the data block can be judged based on the start version number, the end version number and the version number of the key-value pair, and then the safe isolation of the read-write operation and the merging / splitting operation is realized.

[0073] In some embodiments, the CAS operation instruction is also used to compare the version number of the key-value pair with the start version number, and in the case where the version number of the key-value pair is the same as the start version number, the version number of the key-value pair is incremented by one.

[0074] In an exemplary embodiment, the thread can determine the slot to be operated by the lock acquisition operation according to the data block read in advance, perform the lock acquisition operation through the CAS operation instruction, and at the same time modify the lock flag bit and the version number of the key-value pair corresponding to the slot.

[0075] In some embodiments, after step S208, the method further comprises: step S209, in the case where the first slot is the last slot in the data block in which the key-value pair is written, performing a merging operation on the data block.

[0076] In the embodiments, the thread that fills the data block automatically triggers the merging operation of the data block. Other threads can only perform read-write operations on the data block after the merging operation is completed.

[0077] In some embodiments, the merging operation on the data block in step S209 comprises the following steps:

[0078] Step S2092, in the current thread, deleting duplicates and sorting the key-value pairs stored in the plurality of slots in the data block;

[0079] Step S2094, updating the start version number, the end version number, the version number of the key-value pair attached to each slot, the lock flag bit and the end flag bit attached to the data block;

[0080] Step S2096, writing the updated start version number attached to the data block back to the memory;

[0081] Step S2098, writing the remaining part of the updated data block back to the memory.

[0082] In the embodiment, the merging operation is processed by the thread, and the data block after the merging operation is written back to the memory by the thread. The deduplication in step S2092 refers to deleting the same key-value pairs in the data block. The sorting is because the append write strategy is adopted, and the thread does not care about the data order when writing data, that is, the data order in each slot in the data block is not guaranteed before the data block is full. The multiple key-value pairs in the data block are sorted in descending order or ascending order during the merging operation. After the merging operation, some slots in the data block may be empty. According to the slot state and the preset stable state of the data block, the auxiliary information of each slot, such as the lock flag, the end flag, and the version number of the key-value pair, can be updated.

[0083] In the embodiment, the overall version number (the start version number and the end version number) of the data block is used to control the version of the data block. For example, the start version number and the end version number are both incremented each time the merging operation is performed on the data block. However, the specific value of the version number is not limited in the disclosure. Further, because the start version number and the end version number are written back to the memory separately, if other threads read the data block from the memory during the process, it can be determined that the data block is being merged (that is, the data block is unstable) according to the inconsistency of the start version number and the end version number, so as to avoid reading and writing operations on the data block, prevent abnormal reading and writing of data, ensure data consistency, and achieve safe isolation of the merging operation and the reading and writing operation.

[0084] In some embodiments, step S2094 can specifically include the following steps:

[0085] Step S2094-2, incrementing the start version number and the end version number by one.

[0086] Step S2094-4, updating the version number of the key-value pair, the lock flag, and the end flag according to whether the key-value pair is written in each slot.

[0087] In some embodiments, step S2094-4 can specifically include: in the case where the key-value pair is written in the slot, incrementing the version number of the corresponding key-value pair by one, modifying the lock flag to the locked state, and modifying the end flag to the preset valid value; or in the case where the key-value pair is not written in the slot, keeping the version number of the corresponding key-value pair unchanged, modifying the lock flag to the unlocked state, and modifying the end flag to the preset invalid value.

[0088] In the embodiment, step S2094-4 is performed according to the condition of the stable state of the data block. If other methods are used to limit the stable state of the data block and each slot, step S2094-4 can also be adjusted correspondingly, and the version number of each slot in the data block is updated to the stable state.

[0089] In the embodiment, the version number of the key-value pair of the locked slot in the data block is greater than the start version number or the end version number by 1, and the start version number / end version number has been incremented by 1. Therefore, the version number of the key-value pair of the non-empty slot also needs to be incremented by 1, and the version number of the key-value pair of the empty slot remains unchanged.

[0090] In an example embodiment, the lock flag and the end flag of the empty slot can be set to 0, and the lock flag and the end flag of the non-empty slot can be set to 1. If the start version number / end version number is 1, the version number of the key-value pair of the empty slot is also 1, and the version number of the key-value pair of the non-empty slot is 2. However, the specific values of the version number of the data block and the accompanying information of each slot are not limited in the disclosure.

[0091] In some embodiments, the merging operation on the data block in step S209 can further include the following steps:

[0092] In step S2093, the data block is split into a plurality of updated data blocks in the current thread, and the start version number of each updated data block is written back to the memory first, and the remaining part is written back to the memory later.

[0093] In the embodiment, the splitting of the data block can be triggered according to a preset rule. For example, after sorting and deduplication, if the slots of the data block are still full or most of the slots are full, the splitting of the data block can be triggered, and the written key-value pairs can be re-divided into two or more data blocks according to a preset rule. Through the splitting of the data block, it can be ensured that there is still an empty slot in each data block when the data block is in a stable state.

[0094] In the embodiment of the disclosure, the slot can be locked by using the CAS operation instruction first, and then the key-value pair can be written into the locked slot, which ensures the safe isolation between concurrent multi-threads, avoids the simultaneous operation of a slot by multiple threads, ensures data consistency, and further solves the problem that data consistency and safe isolation are difficult to ensure due to concurrent operation between threads in related technologies. The data block version number and the key-value pair version number can be used to determine whether the data block is in the merging / splitting process, and the safe isolation of the merging / splitting and the data read / write operation is further implemented.

[0095] In an embodiment of the disclosure, a data block is also provided.

[0096] FIG. 3 is a schematic diagram of a structure of a data block according to an embodiment of the present disclosure. As shown in FIG. 3, the data block includes a plurality of slot positions for storing key-value pairs. The present disclosure does not limit the number of slot positions included in each data block.

[0097] In the embodiment, each slot position is attached with a lock flag and an ending tag. The lock flag is used to indicate a lock state of the slot position, and the lock state includes locked and unlocked. The ending tag is used to indicate whether the key-value pair is valid.

[0098] In the embodiment, each key-value pair (KV) is composed of a key and a value, and each slot position can store one key-value pair.

[0099] In the embodiment, each thread can modify the lock flag of each slot position through a CAS operation of RDMA, to indicate that the corresponding slot position is successfully locked. Based on the atomicity of the CAS operation instruction, it can be ensured that only one thread can successfully lock when multiple threads concurrently write, so as to avoid multiple threads repeatedly writing data in one slot position and prevent data writing abnormity.

[0100] In the embodiment of the present disclosure, a data block is designed, and the granularity of data lock is reduced to the key-value level. Compared with the granularity of the data block level, the write concurrency capability of the system can be guaranteed, and the write concurrency capability of the system caused by frequent lock operations when different threads access the data block can be avoided. In the embodiment of the present disclosure, an ending tag is designed for each slot position, and the ending tag is used to realize safe isolation of read operation and write operation, so as to guarantee the safe isolation between concurrent multi-threads. The multiple threads can be prevented from simultaneously operating one slot position, the data consistency is guaranteed, and the problem that the data consistency and safe isolation are difficult to guarantee caused by concurrent operation between threads in the related art can be solved.

[0101] In some embodiments, when the ending tag is a preset valid value, the key-value pair stored in the corresponding slot position is allowed to be read; or when the ending tag is a preset invalid value, the key-value pair stored in the corresponding slot position is not allowed to be read.

[0102] In an exemplary embodiment, the preset valid value is 1, and the preset invalid value is 0. When the ending tag is 1, the thread can read the key-value pair in the slot position. By setting the ending tag of the slot position, the correctness of data reading can be guaranteed when multiple threads concurrently read and write data.

[0103] In some embodiments, the data block further comprises a start version number and an end version number; each of the slots further comprises a key-value pair version number.

[0104] FIG. 4 is a structural diagram of a data block in an embodiment of the present disclosure. As shown in FIG. 4, the data block comprises a plurality of slots for storing key-value pairs, a start version number (Start Ver) and an end version number (End Ver). Each slot comprises a lock flag, an end flag and a key-value pair version number (Ver).

[0105] In the embodiments of the present disclosure, the version number can be used to achieve safe isolation of read-write operations and merging / splitting operations of the data block, so as to avoid reading and writing data during the merging operation of the data block, and further ensure data consistency.

[0106] In some embodiments, the data block is in a stable state when the following conditions are met simultaneously:

[0107] Condition 1: the start version number of the data block is the same as the end version number;

[0108] Condition 2: the end flag of at least one of the slots in the data block is a preset invalid value;

[0109] Condition 3: the key-value pair version number of the locked slot in the data block is greater than the start version number or the end version number by 1;

[0110] Condition 4: the key-value pair version number of the unlocked slot in the data block is the same as the start version number or the end version number.

[0111] In some embodiments, the data block is in an unstable state when the data block meets a preset condition; wherein the preset condition comprises at least one of the following:

[0112] Preset Condition 1: the end flag of each of the slots in the data block is a preset valid value;

[0113] Preset Condition 2: the key-value pair version number of the locked slot in the data block is the same as the start version number;

[0114] Preset Condition 3: the start version number of the data block is different from the end version number.

[0115] In this embodiment, if the data block satisfies any of the above preset conditions, it indicates that the read data block is in an unstable state. If the data block satisfies the preset condition 1, it indicates that all the slot positions in the data block are full, and since the full slot position triggers the splitting / merging operation of the data block, the state of the data block at this time is unstable. If the data block satisfies the preset condition 2 or the preset condition 3, it is possible that the start version number of the data block is updated, and at this time, the data block is performing version update (such as merging or splitting) and write back, and the state of the data block is unstable.

[0116] Through the embodiments of the present disclosure, the safe isolation of the multi-thread read-write operation can be implemented, and the safe isolation of the data read-write and the data block merging / splitting can be ensured, the abnormal read-write of other threads in the data block merging process can be avoided, the data consistency is ensured, and thus the problem that the data consistency and the safe isolation are difficult to ensure due to the concurrent operation between threads in the related art can be solved.

[0117] The present disclosure mainly relates to the following multi-thread concurrent write (update) scenarios:

[0118] Scenario 1, thread 1 writes data, and other threads concurrently read data or write data.

[0119] Scenario 2, thread 1 writes data and triggers merging / splitting, and other threads concurrently read data or write data.

[0120] The concurrent operation of multiple threads will be described in detail based on the above scenarios.

[0121] FIG. 5 is a flowchart of the multi-thread concurrent read-write in an embodiment of the present disclosure, as shown in FIG. 5, the flowchart includes the following steps:

[0122] Step S1, thread 1 and thread 2 read the data block at the same time, and perform the lock acquisition operation on the first empty slot position in the data block through the CAS of RDMA. The lock acquisition operation includes: simultaneously modifying the lock flag bit and the key-value pair version number Ver, changing the lock flag bit to the locked state, and increasing the key-value pair version number Ver by one.

[0123] Step S2, thread 1 acquires the lock of the first empty slot position successfully, that is, thread 1 can write the KV into the slot position; thread 2 fails to acquire the lock, and continues to acquire the lock of the next empty slot position; wherein the characteristics of the CAS operation can ensure that only one thread acquires the lock successfully.

[0124] Step S3, thread 2 acquires the lock of the second empty slot position successfully (at this time, thread 1 may not have written the KV).

[0125] Step S4, thread 1 writes KV in the first empty slot where the lock is successfully acquired, and modifies the end flag of the slot from the default value (preset invalid value) to the preset valid value, to indicate that the KV of the slot is completed (written), and other threads cannot modify it.

[0126] Step S5, thread 2 writes KV in the second empty slot where the lock is successfully acquired, and also modifies the end flag.

[0127] Step S6, the multi-thread concurrent writing (updating) is completed. Among them, the first empty slot and the second empty slot of the data block in the memory are sequentially written with corresponding KV. In order to ensure concurrency, the order of KV before the data block is full is not guaranteed.

[0128] In some embodiments, the process can further include: step S7, thread 3 performs a read operation during the concurrent writing (updating) process of thread 1 and thread 2. Thread 3 can determine whether the KV of each slot is written by judging the end flag of each slot, to avoid dirty reading. This step can occur between any of the above steps.

[0129] In an exemplary embodiment, the read operation of thread 3 can occur between step S4 and step S5, and thread 3 reads the entire data block. Even though the lock flag and Ver of the second empty slot have been modified, the end flag of the second empty slot is still the preset invalid value, indicating that the KV of the second empty slot has not been written (by thread 2), and the KV of the second empty slot is not allowed to be acquired by the read operation of thread 3. For other slots, such as the end flag of the first empty slot (which is now a non-empty slot) in step S1, the end flag is the preset valid value, indicating that the KV of the slot has been written (by thread 1), and therefore, the KV of the first empty slot is allowed to be acquired by the read operation of thread 3.

[0130] In this embodiment, thread 1 and thread 2 concurrently perform data writing operations, and through the CAS operation instruction to acquire the slot, only one thread can successfully acquire the lock, and after successfully acquiring the lock, the KV is written into the data block, which can avoid multiple threads repeatedly writing data in a slot. Thread 3 concurrently performs data reading operations while thread 1 and thread 2 write data, and can ensure the correctness of the read data based on the end flag of the slot, thereby ensuring the data consistency of the data block. In this embodiment, the data lock is set based on the key-value pair (slot) granularity, and multiple threads can simultaneously perform read and write operations on a data block, thereby improving the concurrency of data updating.

[0131] The changes of the data block structure of the memory and the changes of each thread in the multi-thread concurrent read-write process described above will be described in detail below.

[0132] Figure 6 is a schematic diagram of a multi-threaded lock acquisition operation in an embodiment of the present disclosure. As shown in Figure 6, corresponding to steps S1 and S2, thread 1 and thread 2 first read the data block (structure 1) from the memory, and simultaneously perform a lock acquisition operation on the first empty slot (i.e., the third slot) in the data block from front to back. Only one thread can successfully acquire the lock.

[0133] The data block (structure 1) corresponds to step S1, and is the data block read from the memory by thread 1 and thread 2. The data block structure 1 is a stable state data block. The first slot and the second slot in the data block are filled with key-value pairs, and the third slot and the fourth slot are not filled with key-value pairs. For a stable state data block, the start version number (StartVer) is the same as the end version number (EndVer). The key-value pair version number (Ver) of the slot (Slot) that has been filled with a key-value pair is one greater than StartVer / EndVer. The ending flag is a preset valid value. The Ver of the Slot that has not been filled with a key-value pair is equal to StartVer / EndVer. The Ending flag is a default value.

[0134] The data block (structure 2) corresponds to step S2, and is the result of the lock acquisition operation on the third slot of the data block by thread 1 / thread 2. Thread 1 successfully acquires the lock, and the lock flag of the third slot is modified from "not locked" to "locked" by thread 1. The Ver of the third slot is incremented by one. Thread 2 fails to acquire the lock. The other data in the data block remains unchanged.

[0135] In this embodiment, the lock acquisition operation is implemented through a CAS operation instruction. Thread 1 that successfully acquires the lock can receive a return value indicating that the lock acquisition is successful. Thread 2 that fails to acquire the lock can only receive a return value indicating that the lock acquisition fails.

[0136] Figure 7 is a schematic diagram of a lock acquisition operation on the next slot after a thread fails to acquire the lock in an embodiment of the present disclosure. As shown in Figure 7, corresponding to step S3, after thread 2 fails to acquire the lock the first time, thread 2 can directly perform a lock acquisition operation on the next empty slot (i.e., the fourth slot) in the data block without re-reading the data block.

[0137] The data block (structure 3) corresponds to step S3, and is the result of the lock acquisition operation on the fourth slot of the data block by thread 2. Thread 2 successfully acquires the lock, and the lock flag of the fourth slot is modified from "not locked" to "locked" by thread 2. The Ver of the fourth slot is incremented by one.

[0138] Figure 8 is a schematic diagram of writing a key-value pair after thread 1 successfully acquires the lock in an embodiment of the present disclosure. As shown in Figure 8, after thread 1 successfully acquires the lock, thread 1 writes the KV into the third slot successfully locked by thread 1, obtaining the data block (structure 4).

[0139] Data block (structure 4), corresponding to step S4. In step S4, thread 1 writes key-value pair K3-V3 into the third slot, and modifies the end flag of the slot to a preset valid value (for example, 1), to indicate that the KV of the slot is completed updating (writing).

[0140] Figure 9 is a schematic diagram of writing key-value pairs by thread 2 after successfully acquiring the lock in an embodiment of the present disclosure. As shown in Figure 9, after successfully acquiring the lock, thread 2 writes KV into the fourth slot successfully locked by thread 2, to obtain a data block (structure 5).

[0141] Data block (structure 5), corresponding to step S5 and step S6. In step S5, thread 2 writes key-value pair K3-V3 into the fourth slot, and modifies the end flag of the slot to a preset valid value, to indicate that the KV of the slot is completed updating (writing). The data block (structure 5) is also the final data block structure obtained after thread 1 and thread 2 complete data updating.

[0142] Figure 10 is a schematic diagram of reading a data block by thread 3 in an embodiment of the present disclosure. As shown in Figure 10, corresponding to step S7, thread 3 reads the entire data block from the memory after step S4, at which time thread 1 has written KV, and thread 2 has only locked the fourth slot and has not written KV. The data block read from the memory by thread 3 is the same as the data block (structure 4) described above,

[0143] In the data block (structure 4), although the lock flag and Ver of the fourth slot have been modified, the end flag of the fourth slot is still a preset invalid value, indicating that the KV of the fourth slot is not completed, and the KV of the slot is not allowed to be acquired by the read operation of thread 3. For other slots, such as the third slot, the end flag is a preset valid value, indicating that the KV of the slot is completed, and therefore, the KV of the third slot can be acquired by the read operation of thread 3. Further, the read operation of thread 3 can occur at any time in the above process, which is not limited by the present disclosure.

[0144] In the embodiment of the present disclosure, if a thread fills the data block, a merging operation of the data block is automatically triggered, and a splitting operation can also be triggered in this process. Therefore, after the above steps S1 to S7, since thread 2 writes into the last free slot in the data block, thread 2 will trigger the merging operation of the data block after writing the key-value pair.

[0145] Figure 11 is a flowchart of data block merging / splitting in an embodiment of the present disclosure. As shown in Figure 11, the flowchart includes the following steps:

[0146] Step S11, thread 2 fills the data block and starts the merging operation;

[0147] Step S12, thread 2 deletes the redundancy and sorts the KV in the full data block. This process is accompanied by the modification of the lock flag, the end flag, the version number of the key-value pair, the start version number and the end version number. After deleting the redundancy and sorting, new empty slots may appear in the data block. Thread 2 will unlock the empty slots, change the end flag of the empty slots to a preset invalid value, and increase the start version number and the end version number by one, and increase the version number of the key-value pair in the slot that is not deleted by one. The changes in the data block in this process only occur within the thread.

[0148] If the data block is not split, the data block write-back can be divided into the following steps:

[0149] Step S13, write the start version number (StartVer) of the data block back to the memory;

[0150] Step S14, write the remaining part of the data block back to the memory.

[0151] In this embodiment, the remaining part of the data block includes the lock flag, the version number of the key-value pair, the key-value pair, the end flag of each slot, and the end version number of the data block.

[0152] If the data block is split into two, the data block write-back can also be divided into the following steps:

[0153] Step S13', write the start version number of the two data blocks back to the memory;

[0154] Step S14', write the remaining part of the data block to the memory in the order of the positions of the two data blocks.

[0155] In this embodiment, the data block can be split according to a preset splitting rule. For example, when the number of empty slots is less than a specified number or a specified proportion, the data in the original data block can be split into two or more data blocks. In the split data blocks, only one data block occupies the memory position of the original data block, and the other split data blocks occupy new memory spaces, and the split data blocks are empty before the thread writes back data.

[0156] In this embodiment, if a thread wants to write data during the above data block merging / splitting process, the following steps will be triggered:

[0157] Step S15, if thread 1 reads the data block (steps S11 and S12) when thread 2 has not written back the StartVer, it can be determined by the end flag that the data block is full and is being merged, that is, the data block is in an unstable state, and thread 1 cannot read and write the current read-back data block, and needs to be read again.

[0158] Step S16, if thread 1 reads the data block (step S13 or step S13') when thread 2 only writes back the StartVer, it can be determined that the data block is being merged and written back according to the inconsistency between the start version number and the end version number, that is, the data block is in an unstable state, and thread 1 cannot read and write the currently read data block, and needs to be re-read.

[0159] Step S17, only when the data block is in a stable state, thread 1 can read or write the data block.

[0160] FIG. 12 is a schematic diagram of merging the data block by thread 2 in an embodiment of the present disclosure. As shown in FIG. 12, based on the above-mentioned FIG. 9, after thread 2 writes the KV into the fourth slot to obtain the data block (structure 5), since thread 2 writes the last empty slot of the data block, thread 2 will trigger the merging operation of the data block.

[0161] In this embodiment, through step S12, thread 2 completes the deduplication and sorting of the key-value pairs in each slot, for example, the key-value pairs K3-V3 in the third and fourth slots are the same, and only one is retained. During the deduplication and sorting process, thread 2 will unlock the empty slot, change the end flag of the empty slot to a preset invalid value, and increase the start version number and the end version number by one, and increase the version number of the key-value pair in the slot which is not deleted by one. The data block changes in this process only occur within the thread.

[0162] In this embodiment, through steps S13 and S14, the data block processed by thread 2 can be written back to the memory in sequence. Through the process of writing back the start version number first and then writing back the remaining part of the data block, other threads can be made to explicitly know that the data block is in the merging and writing back process, thereby avoiding abnormal writing or reading of data by other threads during the merging of the data block.

[0163] In this embodiment, the data block (structure 6) only updates the start version number and is in an unstable state. The data block (structure 7) has completed the entire merging operation process and is in a stable state.

[0164] FIG. 13 is a schematic diagram of data block splitting in an embodiment of the present disclosure. As shown in FIG. 13, during the merging of the data block, data block splitting can be triggered. At this time, the data can be written into the two split data blocks respectively when the data block is written back to the memory. The present disclosure does not limit the number of data block splitting.

[0165] In this embodiment, the data block processed by thread 2 can be split into two data blocks by step S13' and step S14', one of which occupies the memory location of the original data block, and the other occupies the newly divided memory space (initially empty). By writing back the start version number of the two data blocks first, and then writing back the remaining part of the two data blocks, other threads can be made to explicitly understand that the data block is in the merging and writing back process, so as to avoid abnormal writing or reading of data by other threads when the data block is merging.

[0166] FIG. 14 is a schematic diagram of concurrent data block merging and data read / write operation in an embodiment of the present disclosure. As shown in FIG. 14, in the process of merging the data block by thread 2, thread 1 attempts to read the data block from the memory multiple times to perform read / write operation.

[0167] Through step S15, thread 1 obtains the data block from the memory (structure 5), at this time, thread 2 has not written back StartVer (steps S11 and S12), thread 1 can determine that the data block is being merged and written back by the end flag, that is, the data block is in an unstable state, and thread 1 cannot perform read / write operation on the currently read data block, and needs to be re-read.

[0168] Through step S16, thread 1 obtains the data block from the memory (structure 6), at this time, thread 2 has only written back StartVer (step S13 or step S13'), thread 1 can determine that the data block is being merged and written back according to the inconsistency between the start version number and the end version number of the data block, that is, the data block is in an unstable state, and thread 1 cannot perform read / write operation on the currently read data block, and needs to be re-read.

[0169] Through step S17, thread 1 obtains the data block from the memory (structure 7), at this time, the data block is in a stable state, and thread 1 can perform read operation or write operation on the data block.

[0170] Through the embodiments of the present disclosure, safe isolation of concurrent operation of data block merging / splitting and data read / write can be realized, and the problem that data consistency and safe isolation are difficult to guarantee due to concurrent operation between threads in the related art can be solved.

[0171] The embodiments of the present disclosure can be applied to a computing memory decoupling system under a distributed architecture, introduce a data update scheme based on key value granularity, can realize multi-thread concurrent writing (updating) of different KVs of the same data block through the CAS operation of RDMA, and at the same time, through the version number and flag, safe isolation of concurrent read / write operation and data block merging / splitting operation is realized, which guarantees the data safety between different thread operations while not reducing the concurrency degree of the system by using fine-grained lock.

[0172] The embodiments of the present disclosure further provide a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to perform the steps in any of the method embodiments.

[0173] In an example embodiment, the computer readable storage medium described above can include, but is not limited to, a U disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and various media that can store computer programs.

[0174] The embodiments of the present disclosure further provide an electronic device, which comprises a memory storing a computer program and a processor configured to execute the computer program to perform the steps in any of the method embodiments.

[0175] In an example embodiment, the electronic device described above can further comprise a transmission device connected to the processor and an input and output device connected to the processor.

[0176] The embodiments of the present disclosure further provide a computer program product, which comprises a computer program executed by a processor to perform the steps in any of the method embodiments.

[0177] The specific examples in the present embodiment can refer to the examples described in the above embodiments and example embodiments, which will not be repeated here.

[0178] Obviously, those skilled in the art should understand that the modules or steps of the present disclosure described above can be realized by general computing devices, which can be concentrated on a single computing device or distributed on a network composed of multiple computing devices, and they can be realized by program codes executable by computing devices, so that they can be stored in storage devices and executed by computing devices, and in some cases, the steps shown or described can be executed in different order, or they can be manufactured into individual integrated circuit modules, or multiple modules or steps can be manufactured into a single integrated circuit module. Therefore, the present disclosure is not limited to any specific combination of hardware and software.

[0179] The above only describes example embodiments of the present disclosure and is not intended to limit the present disclosure. For those skilled in the art, the present disclosure can have various modifications and changes. Any modification, equivalent replacement, improvement, etc. made within the principles of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A data updating method, the method comprising: reading a data block in a memory, wherein the data block comprises a plurality of slots for storing key-value pairs, and each slot is attached with a lock flag and an end flag; performing a lock acquisition operation on a first slot in the data block that is not locked by a current thread via a compare-and-swap (CAS) operation instruction, wherein the CAS operation instruction is atomic; in a case where the lock acquisition operation is successful, locking the first slot by modifying the lock flag; writing the key-value pair into the first slot that is locked by the current thread via an append-write manner, and making the key-value pair valid by modifying the end flag.

2. The method of claim 1, wherein, performing a lock acquisition operation on a first slot in the data block that is not locked by a current thread via a compare-and-swap (CAS) operation instruction, comprising: sending the CAS operation instruction to the memory, wherein the CAS operation instruction is used to compare the lock flag of the first slot with an expected original value carried in the CAS operation instruction, and modify the lock flag of the first slot to a new value carried in the CAS operation instruction in a case where the lock flag of the first slot is the same as the expected original value, wherein the lock flag being the expected original value indicates that the slot is not locked, and the lock flag being the new value indicates that the slot is locked; obtaining a return value of the CAS operation instruction, wherein the return value is used to indicate a lock acquisition state of the first slot by the current thread, and the lock acquisition state comprises a lock acquisition success and a lock acquisition failure.

3. The method of claim 1, wherein, The method further comprises: in a case where the lock acquisition operation is failed, continuing to perform a lock acquisition operation on a second slot in the data block that is not locked by the current thread via the CAS operation instruction.

4. The method of claim 1, wherein, reading a data block in a memory, comprising: reading the lock flag and the end flag of each slot in the data block; in a case where the end flag is a preset valid value, reading the key-value pair stored in the corresponding slot; or in a case where the end flag is a preset invalid value, not reading the key-value pair stored in the corresponding slot.

5. The method of claim 1, wherein, Each data block is attached with a start version number and an end version number, and each slot is further attached with a key-value pair version number.

6. The method of claim 5, wherein, The method further comprises: in a case where the data block meets the following conditions simultaneously, determining that the data block is in a stable state, and performing a read operation or a write operation on the data block: the start version number of the data block is the same as the end version number of the data block; the end flag of at least one slot in the data block is a preset invalid value; the key-value pair version number of the slot that has been locked in the data block is greater than the start version number or the end version number by 1; and the key-value pair version number of the slot that has not been locked in the data block is the same as the start version number or the end version number.

7. The method of claim 5, wherein, After reading a data block in a memory, the method further comprises: in a case where the data block meets a preset condition, re-reading the data block; wherein the preset condition comprises at least one of the following: The end flag bit of each slot in the data block is a preset effective value; The key-value pair version number of the slot in the data block that has been locked is the same as the start version number; The start version number of the data block is different from the end version number.

8. The method of claim 5, wherein, The CAS operation instruction is also used to compare the key-value pair version number with the start version number, and in the case where the key-value pair version number is the same as the start version number, the value of the key-value pair version number is incremented by one.

9. The method of claim 5, wherein, After the key-value pair is written into the first slot locked by the current thread by means of append write and the key-value pair is validated by modifying the end flag bit, the method further comprises: In the case where the first slot is the last slot in the data block to which the key-value pair is written, a merge operation is performed on the data block.

10. The method of claim 9, wherein performing a merge operation on the data block comprises: in the current thread, deduplicating, sorting the key-value pairs stored in the plurality of slots in the data block; updating the start version number, the end version number of the data block, the key-value pair version number of each slot, the lock flag bit and the end flag bit; writing the updated start version number of the data block back to the memory; writing the remaining part of the updated data block back to the memory.

11. The method of claim 10, wherein updating the start version number, the end version number of the data block, the key-value pair version number of each slot, the lock flag bit and the end flag bit comprises: incrementing the start version number and the end version number by one; updating the key-value pair version number, the lock flag bit and the end flag bit of each slot according to whether the key-value pair is written into the slot.

12. The method of claim 11, wherein updating the key-value pair version number, the lock flag bit and the end flag bit of each slot according to whether the key-value pair is written into the slot comprises: in the case where the key-value pair is written into the slot, incrementing the corresponding key-value pair version number by one, modifying the corresponding lock flag bit to a locked state, and modifying the corresponding end flag bit to a preset effective value; or in the case where the key-value pair is not written into the slot, keeping the corresponding key-value pair version number unchanged, modifying the corresponding lock flag bit to an unlocked state, and modifying the corresponding end flag bit to a preset invalid value.

13. The method of claim 10, wherein performing a merge operation on the data block further comprises: in the current thread, splitting the data block into a plurality of updated data blocks, wherein the start version number of each updated data block is written back to the memory first, and the remaining part is written back to the memory later.

14. A data block, wherein, The data block comprises a plurality of slots for storing key-value pairs, wherein each slot is attached with a lock flag and an end flag, the lock flag is used to indicate the lock state of the slot, the lock state comprises locked and unlocked, and the end flag is used to indicate whether the key-value pair is valid.

15. The data block of claim 14, wherein, in the case that the end flag is a preset valid value, the key-value pair stored in the corresponding slot is allowed to be read; or, in the case that the end flag is a preset invalid value, the key-value pair stored in the corresponding slot is not allowed to be read.

16. The data block of claim 14, wherein, the data block further comprises a start version number and an end version number; each slot is further attached with a key-value pair version number.

17. The data block of claim 16, wherein, in the case that the data block meets the following conditions simultaneously, the data block is in a stable state: the start version number of the data block is the same as the end version number; the key-value pair version number of the locked slot in the data block is greater than the start version number or the end version number by 1; the key-value pair version number of the unlocked slot in the data block is the same as the start version number or the end version number; the end flag of at least one slot in the data block is a preset invalid value.

18. The data block of claim 16, wherein, in the case that the data block meets a preset condition, the data block is in an unstable state; wherein the preset condition comprises at least one of the following: the end flag of each slot in the data block is a preset valid value; the key-value pair version number of the locked slot in the data block is the same as the start version number; the start version number of the data block is different from the end version number.

19. A computer readable storage medium, wherein, The storage medium stores a computer program, wherein the computer program is run by the processor to execute the method in any one of claims 1 to 13.

20. An electronic device comprising a memory and a processor, wherein, The memory stores a computer program, and the processor is configured to run the computer program to execute the method in any one of claims 1 to 13.

21. A computer program product comprising a computer program, wherein, The computer program is executed by the processor to implement the steps of the method in any one of claims 1 to 13.

Images