An RDMA management method, apparatus, device, and readable storage medium

Abstract

This specification provides an RDMA management method, apparatus, device, and readable storage medium. The method includes: reading the value of a lock counter based on a lock counter address; obtaining the address of a target lock entry in a lock list based on the value; writing its own identification information into the corresponding memory space based on the target lock entry address; and adjusting the value of the lock counter according to preset rules; monitoring a preset lock identifier area in the local memory space; and executing RDMA services in response to an event where lock holding information is written into the lock identifier area; and writing lock operation completion information into the corresponding memory space based on the target lock entry address in response to an event where the RDMA service is completed. Through the technical solution described in this specification, distributed lock management without server CPU participation is achieved through unilateral RDMA operations. The client directly reads and writes to the server memory to complete lock acquisition and release, significantly reducing communication latency and CPU overhead.

Description

Technical Field

[0001] This specification relates to the field of communication technology, and in particular to an RDMA management method, apparatus, device, and readable storage medium. Background Technology

[0002] In distributed storage, distributed database, and high-performance computing scenarios, multiple client nodes often need to concurrently access shared resources, such as shared files, data blocks, metadata items, or global configuration information. To ensure data consistency and operational correctness, an efficient distributed locking mechanism must be introduced to ensure that at most one client can exclusively access the target resource at any given time.

[0003] In a single-machine environment, primitives such as mutexes, semaphores, or read-write locks provided by the operating system kernel can achieve strong consistency control with nanosecond-level latency by relying on shared memory and a single-point scheduler. However, in a distributed environment, due to the unreliability of the network environment, the normalization of node failures, and the lack of a global clock, achieving an equivalent locking mechanism faces severe challenges. First, network communication in distributed systems has inherent uncertainties, including message loss, out-of-order delivery, duplicate transmissions, and significant round-trip time jitter. This makes it impossible for lock state synchronization to reach the deterministic level of local memory access, and the design must fully consider the security issues in network partition (split-brain) scenarios. Second, to avoid single points of failure, the lock service itself must have high availability, which usually requires replicating the lock state among multiple server nodes. This introduces complex distributed consensus problems, requiring a fine trade-off between security (such as mutual exclusion and deadlock prevention) and liveness (such as eventuality). Furthermore, handling abnormal behavior of lock holders is particularly complex. Unlike single-machine systems where process crashes can be automatically reclaimed by the operating system, in distributed scenarios, clients may become disconnected due to process freezes, network interruptions, or prolonged garbage collection pauses, resulting in locks not being released in time. The system needs to securely clean up "orphan locks" through lease mechanisms, heartbeat detection, and timeout control, while strictly preventing the "dual master" phenomenon caused by misjudgments due to momentary network latency, which could lead to multiple clients simultaneously believing they hold the lock. In addition, each acquisition and release of a distributed lock involves cross-network interaction, the performance overhead of which is far higher than local memory operations. The design must comprehensively weigh the impact of lock granularity (coarse-grained locks are simple to manage but have low concurrency, while fine-grained locks have high concurrency but complex coordination) and implementation protocol (such as master-slave replication, Raft / Paxos-like consensus algorithms, or timestamp-based sorting) on ​​the overall system throughput and latency.

[0004] Remote Direct Memory Access (RDMA) allows data in one host's memory to be written directly from the network interface card (NIC) to the memory of another host without intervention from the operating system kernel and CPU, significantly reducing communication latency and CPU utilization. RDMA provides two main types of communication primitives: one-sided and two-sided operations. One-sided operations, such as RDMA Write, RDMA Read, and atomic operations (including Fetch-and-Add and Compare-and-Swap), require only the initiating end to know the remote memory address and access key (rkey), with no involvement from the remote CPU, resulting in optimal performance. Two-sided operations, such as Send / Receive, require collaboration between the communicating parties, completing the interaction through message passing. While flexible, their performance is relatively lower. Constructing lock-free data structures or distributed locks based on the Compare-and-Swap primitives of RDMA atomic operations is a current research hotspot. Their atomicity is guaranteed by the NIC hardware, effectively reducing server-side CPU overhead.

[0005] Current RDMA-based distributed lock schemes employ an interaction mode similar to traditional message passing: the client sends a lock request to the server, the server CPU parses the request, updates the lock state, and returns the result. When multiple clients compete for the lock, the server must process the requests serially, checking the lock state sequentially and returning a response. The lock holder must also explicitly notify the server to update the state when unlocking. This approach not only introduces multiple network round trips but also requires the server CPU to be deeply involved in every lock operation, easily becoming a performance bottleneck in high-concurrency scenarios. Furthermore, due to the uncertainty of network transmission, the server's first-to-first-receive strategy cannot guarantee fairness in lock acquisition. Later-sent lock requests may be processed first due to shorter network paths, causing some clients to be unable to acquire the lock for extended periods due to network latency, leading to starvation. In addition, existing schemes often rely on dynamic maintenance of the lock queue by the server software, making it difficult to fully utilize the atomic operation capabilities of RDMA hardware to achieve efficient first-in-first-out queuing. Summary of the Invention

[0006] In view of this, this specification provides an RDMA management method, apparatus, device, and readable storage medium to improve the aforementioned problem of RDMA lock policy instability.

[0007] The specific technical solution is as follows: This specification provides an RDMA management method applied to a client in an RDMA network, which includes a server and several clients. The method includes: reading the value of a lock counter based on a lock counter address; obtaining the address of a target lock entry in the lock list based on the value; writing its own identification information into the corresponding memory space based on the target lock entry address; and adjusting the value of the lock counter according to a preset rule. The lock counter address points to a specific fixed area in the server's memory, and the lock counter address is pre-distributed by the server to the clients in the RDMA network. The lock list includes several lock entry addresses, each corresponding to a different memory space on the server. The method monitors a preset lock identifier area in the local memory space and executes RDMA services in response to an event where lock holding information is written to the lock identifier area. The lock holding information is the identification information recorded in the memory space corresponding to the lock entry address selected by the server based on the lock pointer, and written into the corresponding lock identifier area of ​​the client. In response to an event indicating the completion of the RDMA service, lock operation end information is written into the corresponding memory space based on the target lock entry address. The lock operation end information is used to indicate that the server's lock pointer points to the next lock entry address in the lock list.

[0008] As a technical solution, the lock operation end information is used to instruct the server's lock pointer to point to the address of the next lock entry in the lock list. If the lock pointer was pointing to the address of the last lock entry in the lock list before being adjusted by the server according to the lock operation end information, then the lock pointer is adjusted by the server to point to the address of the first lock entry in the lock list according to the lock operation end information.

[0009] As a technical solution, the number of lock entry addresses included in the lock list is equal to the number of clients included in the RDMA network. Each lock entry address is consecutive and corresponds to the same memory space size. The step of reading the lock counter value based on the lock counter address and obtaining the target lock entry address in the lock list based on this value includes: taking the remainder of the lock counter value based on the number of lock entry addresses; multiplying the remainder by the size of the memory space corresponding to a lock entry address to obtain an offset; and adding the offset to the base address of the lock list to obtain the target lock entry address. The step of adjusting the lock counter value according to a preset rule includes: incrementing the lock counter value by 1.

[0010] As a technical solution, the server stores lock identifier area communication information pre-sent by the client of the RDMA network. The lock identifier area communication information includes the associated client's identity information, IP address, lock identifier area memory address, and the correspondence between the three.

[0011] This specification also provides an RDMA management method applied to the server side of an RDMA network, which includes a server and several clients. The method includes: responding to an event in which lock operation completion information is written into the memory space corresponding to the lock entry address in the lock list currently pointed to by the lock pointer, adjusting the lock pointer to point to the next lock entry address in the lock list; the lock operation completion information is written by the client in response to an event indicating that the RDMA service has been completed in the memory space corresponding to the lock entry address; based on the identity information stored in the memory space corresponding to the newly pointed-to lock entry address, writing lock holding information into the lock identifier area in the memory space of the client corresponding to the identity information; responding to a request initiated by the client based on the lock holding information, executing the RDMA service; the identity information is obtained by the client reading the value of the lock counter according to the lock counter address, obtaining the target lock entry address in the lock list based on the value, and writing it into the corresponding memory space based on the target lock entry address; the lock counter address points to a specific fixed area in the server's memory, and the lock counter address is pre-issued by the server to the clients in the RDMA network; after the client writes its own identity information into the memory space corresponding to the lock entry address, the value of the lock counter is adjusted according to a preset rule.

[0012] This specification also provides an RDMA management device for use in clients of an RDMA network, which includes a server and several clients. The device includes: a first module, configured to read the value of a lock counter based on a lock counter address, obtain the address of a target lock entry in the lock list based on the value, write its own identification information into the corresponding memory space based on the target lock entry address, and adjust the value of the lock counter according to preset rules; the lock counter address points to a specific fixed area in the server's memory, and the lock counter address is pre-distributed by the server to the clients of the RDMA network; the lock list includes several lock entry addresses, each lock entry... The addresses correspond to different memory spaces on the server. The second module monitors the preset lock identifier area in the local memory space and executes RDMA services in response to the event that lock holding information is written to the lock identifier area. The lock holding information is the identification information recorded in the memory space corresponding to the lock entry address selected by the server according to the lock pointer, and is written to the lock identifier area of ​​the corresponding client. The third module responds to the event that the RDMA service is completed and writes lock operation end information to the corresponding memory space according to the target lock entry address. The lock operation end information is used to indicate that the server's lock pointer points to the next lock entry address in the lock list.

[0013] As a technical solution, the lock operation end information is used to instruct the server's lock pointer to point to the address of the next lock entry in the lock list. If the lock pointer was pointing to the address of the last lock entry in the lock list before being adjusted by the server according to the lock operation end information, then the lock pointer is adjusted by the server to point to the address of the first lock entry in the lock list according to the lock operation end information.

[0014] As a technical solution, the number of lock entry addresses included in the lock list is equal to the number of clients included in the RDMA network. Each lock entry address is consecutive and corresponds to the same memory space size. The step of reading the lock counter value based on the lock counter address and obtaining the target lock entry address in the lock list based on this value includes: taking the remainder of the lock counter value based on the number of lock entry addresses; multiplying the remainder by the size of the memory space corresponding to a lock entry address to obtain an offset; and adding the offset to the base address of the lock list to obtain the target lock entry address. The step of adjusting the lock counter value according to a preset rule includes: incrementing the lock counter value by 1.

[0015] As a technical solution, the server stores lock identifier area communication information pre-sent by the client of the RDMA network. The lock identifier area communication information includes the associated client's identity information, IP address, lock identifier area memory address, and the correspondence between the three.

[0016] This specification also provides an RDMA management device applied to the server side of an RDMA network, which includes a server and several clients. The device includes: a fourth module, used to adjust the lock pointer to point to the next lock entry address in the lock list in response to an event where lock operation completion information is written to the memory space corresponding to the lock entry address currently pointed to by the lock pointer; the lock operation completion information is written by the client to the memory space corresponding to the lock entry address in response to an event indicating the completion of RDMA service execution; and a fifth module, used to, based on the identity information stored in the memory space corresponding to the newly pointed-to lock entry address, write the identity information to the memory space of the client corresponding to the identity information. The lock identification area is written with lock holding information; the sixth module is used to execute RDMA services in response to a request initiated by the client based on the lock holding information; the identification information is obtained by the client reading the value of the lock counter according to the lock counter address, obtaining the address of the target lock entry in the lock list according to the value, and writing it into the corresponding memory space according to the target lock entry address; the lock counter address points to a specific fixed area in the server's memory, and the lock counter address is pre-sent by the server to the client in the RDMA network; after the client writes its own identification information into the memory space corresponding to the lock entry address, the value of the lock counter is adjusted according to a preset rule.

[0017] This specification also provides an electronic device including a processor and a readable storage medium storing machine-executable instructions that can be executed by the processor, which executes the machine-executable instructions to implement the aforementioned RDMA management method.

[0018] This specification also provides a readable storage medium storing machine-executable instructions that, when invoked and executed by a processor, cause the processor to implement the aforementioned RDMA management method.

[0019] The technical solutions provided in this specification offer at least the following beneficial effects: The distributed lock management system, which enables lock-free server CPU participation, is achieved through one-sided RDMA operations. Clients directly read and write server memory to complete lock acquisition and release, significantly reducing communication latency and CPU overhead. The FIFO queuing mechanism based on lock counters ensures fairness, and the asynchronous server notification mechanism improves throughput, making it suitable for high-concurrency distributed scenarios. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of this specification or the prior art, the drawings used in the description of the embodiments of this specification or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this specification. For those skilled in the art, other drawings can be obtained based on these drawings of the embodiments of this specification.

[0021] Figure 1 This is a flowchart of an RDMA management method according to one embodiment of this specification; Figure 2 This is a flowchart of an RDMA management method according to one embodiment of this specification; Figure 3 This is a structural diagram of an RDMA management device according to one embodiment of this specification; Figure 4 This is a structural diagram of an RDMA management device according to one embodiment of this specification; Figure 5 This is a hardware structure diagram of an electronic device according to one embodiment of this specification.

[0022] Reference numerals: Module 1 31, Module 2 32, Module 33, Module 41, Module 5 42, Module 6 43. Detailed Implementation

[0023] The terminology used in the embodiments described herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of this specification. The singular forms “a,” “described,” and “the” as used in this specification and claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to any and all possible combinations comprising one or more of the associated listed items.

[0024] It should be understood that although the terms first, second, third, etc., may be used to describe various information in embodiments of this specification, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, without departing from the scope of this specification, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" may also be interpreted as "when," "when," or "in response to a determination."

[0025] In view of this, this specification provides an RDMA management method, apparatus, device, and readable storage medium to at least improve one of the above-mentioned technical problems.

[0026] The specific technical solution is described below.

[0027] In one embodiment, this specification provides an RDMA management method applied to a client in an RDMA network, the RDMA network including a server and several clients. The method includes: reading the value of a lock counter according to a lock counter address; obtaining the address of a target lock entry in a lock list according to the value; writing its own identification information into the corresponding memory space according to the target lock entry address; and adjusting the value of the lock counter according to a preset rule. The lock counter address points to a specific fixed area in the server's memory, and the lock counter address is pre-distributed by the server to the clients in the RDMA network. The lock list includes several lock entry addresses, each lock entry address corresponding to a different memory space on the server. The method also includes monitoring a preset lock identifier area in the local memory space; and executing RDMA services in response to an event where lock holding information is written into the lock identifier area. The lock holding information is the identification information recorded in the memory space corresponding to the lock entry address selected by the server according to the lock pointer, and written into the lock identifier area of ​​the corresponding client. Finally, in response to an event indicating that the RDMA service has been completed, lock operation end information is written into the corresponding memory space according to the target lock entry address. The lock operation end information is used to indicate that the server's lock pointer points to the next lock entry address in the lock list.

[0028] As the central hub for lock resource management, the server pre-allocates and initializes a series of data structures in its local memory. The first is the lock counter, LastLock, defined as a 64-bit unsigned integer, initialized to 0. The memory address of this variable and its corresponding remote access key, rkey, are distributed by the server to all clients within the RDMA network domain via the control plane channel. The 64-bit width handles extremely high-concurrency scenarios; even with billions of locking operations per second, the counter would only wrap around in decades, thus avoiding complex sequence number wrapping logic. However, the counter width can be arbitrarily adjusted according to actual needs.

[0029] The server creates a lock list, specifically a lock ring buffer (LockRing) in this implementation. LockRing consists of a fixed number of lock entries (LockEntry), typically determined by the maximum number of concurrent clients supported by the system, for example, 32768. Each LockEntry occupies 64 bytes of memory. Its internal structure is finely divided into status bits and an identification field. The least significant bit, Lockstate, indicates whether the entry is valid. The following 15 bits store the client's identification information (CID), and the remaining bits are reserved for future expansion. The server also needs to announce the starting memory address and rkey of LockRing to all clients. Furthermore, the server internally maintains a current lock pointer (Currlock), which is also initialized to 0 and points to the first lock entry in LockRing, used to record the position of the client currently holding the lock in the ring buffer.

[0030] Each client is assigned a unique network-wide identification information CID, such as a 15-bit numeric identifier generated based on its MAC address or IP address. Each client needs to register a dedicated memory area in its local memory as a lock identifier area, namely ClientLock. This memory area only needs to be a few bytes in size, but it must be registered with the RDMA network card to obtain the corresponding lkey and rkey, and then report the memory address and rkey to the server. The server establishes a mapping table to maintain the association between the CID and the corresponding client lock identifier area address and rkey.

[0031] like Figure 1 This includes the following steps, the order of which can be changed depending on the needs of the actual application scenario: Step S11: Read the value of the lock counter according to the lock counter address, obtain the address of the target lock entry in the lock list according to the value, write its own identification information into the corresponding memory space according to the target lock entry address, and adjust the value of the lock counter according to the preset rules.

[0032] When a client, say client A with CID 1001, attempts to execute an RDMA service that requires mutually exclusive access to shared resources, it will initiate a lock acquisition process. Client A's application will first trigger the lock request logic. Client A will not send any messages carrying lock request semantics to the server; instead, it will directly initiate an atomic operation on the server's memory using the RDMA network card.

[0033] Client A's RDMA network card initiates a Fetch-and-Add (FAA) atomic operation instruction to the server based on the pre-known lock counter address and rkey, with operand 1. This instruction is encapsulated into a specific RDMA data packet by the client's RNIC and sent to the server. Upon receiving this data packet, the server's network card processes the request directly at the hardware level without interrupting the server's CPU. The RNIC locates the physical memory address where the lock counter resides, performs the actions of reading the current value, incrementing it by 1, and writing back the new value, then returns the old value to client A via an RDMA response packet. Because the RDMA hardware guarantees the atomicity of the FAA operation, even if hundreds or thousands of clients simultaneously initiate FAA operations on the same lock counter, the network card serializes these requests, ensuring that each client receives a unique, incrementing sequence number. For example, if the current value of the lock counter is 5000 when client A initiates the FAA operation, client A's RNIC will receive a return value of 5000, and simultaneously, the lock counter in the server's memory will be atomically updated to 5001. The return value 5000 represents client A's queue number in this lock contention. After obtaining this number, client A needs to determine the address of its target lock entry in the lock list according to preset rules. These preset rules typically use a modulo operation, where the remainder after dividing the obtained queue number by the length n of the lock list is used as the index.

[0034] If the length of LockRing is 32768, then index i equals 5000 mod 32768, resulting in 5000. Client A calculates the exact offset of the target lock entry in LockRing based on this, thus obtaining its actual memory address. Next, client A initiates a second RDMA operation, this time an RDMA Write operation. Client A's RNIC will directly write a data structure containing its own identification information CID (1001) and lock status bit (set to 1, indicating waiting for or holding the lock) into the memory address of the target lock entry just calculated. Since each client obtains a unique queue number through the FAA operation, although there is a theoretical possibility that multiple numbers could map to the same index after the modulo operation, due to the monotonically increasing lock counter and the server's sequential processing, in reality, each valid lock entry will only be declared by one client before being released. Even if an index conflict occurs (e.g., sequence numbers 5000 and 37768 both map to index 5000), due to the timing nature of the FAA, the later write operation will find that the status bit has already been set. However, in the optimized design of this scheme, this situation is naturally avoided due to the Currlock pointer advancement mechanism. In other words, each client only needs to be responsible for writing its own entry without worrying about whether the entry is already occupied, because the FAA has implicitly allocated the order. After the write is completed, client A does not immediately know whether it has acquired the lock. It can choose to execute other non-critical section tasks or enter a low-power waiting state.

[0035] Step S12: Monitor the preset lock identifier area in the local memory space, and execute RDMA service in response to the event that lock holding information is written to the lock identifier area.

[0036] On the server side, a background lock scheduling logic exists. This logic also runs with the hardware assistance of the server-side RNIC, or is executed by a low-overhead daemon thread. Its core task is to monitor the state of the lock list and advance the current lock pointer. The server continuously checks the lock entry pointed to by the Currlock pointer. Taking the initial state as an example, assuming Currlock points to index 0, the server checks the LockEntry at index 0. If the Lockstate of this entry is 0, it means that no client currently holds the lock and the queue is empty. The server then continues to check the next entry until it finds a valid entry with a Lockstate of 1, or finds that Currlock coincides with LastLock (the current value of the lock counter is decremented by 1), indicating that there are no waiting parties in the queue. When a valid waiting party is found, for example, Currlock points to index 5 and the CID field of this entry is 1001, the server will perform a notification operation. The server will query the previously established mapping table and find the corresponding client A's lock identifier area memory address and rkey based on CID 1001. Subsequently, the server-side RNIC initiates an RDMA WritewithImmediate operation. This operation writes specific lock holding information (such as a specific immediate value or flag) into the lock identifier area memory of client A. Unlike a normal RDMA Write, the Write withImmediate operation generates a Completion Queue Event (CQE) on the receiving end, i.e., client A's RNIC, after the data is written. This event is hardware-triggered, eliminating the need for client A's CPU to continuously poll memory for data changes. Upon detecting this CQE, client A's RDMA network card wakes up the waiting application thread via an interrupt or event notification mechanism. Upon receiving this hardware event, client A's application knows it has successfully acquired the distributed lock and can safely execute subsequent RDMA operations. This hardware event-based wake-up mechanism significantly reduces CPU overhead and response latency, making it more efficient than traditional packet-based polling or interrupt-based handling.

[0037] After receiving notification of the lock holding event, client A enters the critical section to execute RDMA operations. During this phase, client A is certain that it is the only authorized operator. For example, client A might need to write data directly to a specific file region on a remote storage node or modify an entry in a distributed metadata server. Due to RDMA's zero-copy nature, these operations can bypass the operating system kernel and directly transfer data between user-space memory and remote memory, resulting in extremely high efficiency. During RDMA execution, client A retains its lock holding; the Currlock pointer in the lock list still points to client A's entry. While other clients' entries may have been written to the LockRing and have wait flags set, they are behind the Currlock pointer and can only wait. The server does not actively intervene in client A's execution during this period; it simply silently maintains the Currlock pointer's position, waiting for the unlock signal.

[0038] Step S13: In response to the event that the RDMA service has been completed, write lock operation completion information into the corresponding memory space according to the target lock entry address.

[0039] After client A completes all RDMA business operations, it releases the lock to allow other waiting clients to execute. Specifically, it triggers the lock release logic. Similar to the locking process, the lock release process is also mainly completed through RDMA one-sided operations, without requiring the server CPU to make complex decisions.

[0040] Client A first initiates an RDMA Write with Immediate operation based on the previously recorded target lock entry address, writing lock operation completion information to the lock entry's memory space on the server. This information essentially resets the Lockstate bit in the LockEntry to 0 and may retain CID information for debugging or verification.

[0041] Meanwhile, this Write with Immediate operation generates a completion queue event on the server. The server listens for such events from clients, and once it detects this unlock event, it knows that the client currently holding the lock has completed its operation. The server's lock scheduling logic is then triggered, incrementing the value of the current lock pointer Currlock by 1. To ensure continuous operation within the finite lock list length, this increment operation also requires a modulo operation on the lock list length n, moving the Currlock pointer to the next position. For example, if Currlock previously pointed to index 5, after incrementing, it points to index 6. If the Lockstate at index 6 is 0, it means there is no waiting client at that position. The server will continue traversing backwards, skipping all invalid empty entries, until it finds the next entry with Lockstate 1, or finds that Currlock has caught up with LastLock, indicating that all waiting clients have finished processing. Once the next valid lock entry is found, the server repeats the previous lock granting process: it looks up the corresponding client lock identifier address based on the CID in the entry, initiates an RDMA Write with Immediate operation to write the lock holding information, and thus wakes up the next waiting client, such as client B. This process is repeated to implement a strict First-In-First-Out (FIFO) lock queue.

[0042] In one implementation, the lock operation end information is used to instruct the server's lock pointer to point to the address of the next lock entry in the lock list. If the lock pointer was pointing to the last lock entry address in the lock list before being adjusted by the server according to the lock operation end information, then the lock pointer is adjusted by the server to point to the first lock entry address in the lock list according to the lock operation end information.

[0043] In one implementation, the number of lock entry addresses included in the lock list is equal to the number of clients included in the RDMA network. Each lock entry address is consecutive and corresponds to the same memory space size. The step of reading the lock counter value based on the lock counter address and obtaining the target lock entry address in the lock list based on that value includes: taking the remainder of the lock counter value based on the number of lock entry addresses; multiplying the remainder by the size of the memory space corresponding to a lock entry address to obtain an offset; and adding the offset to the base address of the lock list to obtain the target lock entry address. The step of adjusting the lock counter value according to a preset rule includes: incrementing the lock counter value by 1.

[0044] In one implementation, the server stores lock identifier area communication information pre-sent by the client of the RDMA network. The lock identifier area communication information includes the associated client's identity information, IP address, lock identifier area memory address, and the correspondence between the three.

[0045] Specifically, such as Figure 2 This includes the following steps, the order of which can be changed depending on the needs of the actual application scenario: Step S21: In response to the event that lock operation completion information is written into the memory space corresponding to the lock entry address in the lock list currently pointed to by the lock pointer, adjust the lock pointer to point to the address of the next lock entry in the lock list.

[0046] Step S22: Based on the identity recognition information stored in the memory space corresponding to the newly pointed-to lock entry address, write the lock holding information into the lock identifier area in the memory space of the client corresponding to the identity recognition information.

[0047] Step S23: In response to the request initiated by the client based on the lock holding information, execute the RDMA service.

[0048] In one implementation, the system defines a unique numerical ID (hereinafter referred to as Client ID, or CID) for each client. A mapping relationship between CID and IP address can be established in advance. The IP address of the corresponding client can be determined based on the CID. Each client needs to be allocated one memory space for the server to notify the client to hold the lock via RDMA WRITE WITH IMM, hereinafter referred to as ClientLock. The CID, the corresponding IP address of the CID, the corresponding ClientLock address of the CID, and rkey need to be sent to the server in advance for storage.

[0049] The server pre-allocates a lock ring buffer (hereinafter referred to as LockRing) for the lock. The memory address of this buffer, along with the corresponding rkey required for RDMA access, needs to be sent to all clients beforehand. Simultaneously, the server also needs to allocate a global variable, a 64-bit LastLock, and its address and rkey also need to be announced to all clients beforehand. The ring buffer LockRing consists of n LockEntries, where n is the maximum number of clients supported by the system.

[0050] Two pointers, Currlock and LastLock, are used. The server uses Currlock to indicate the client currently holding the lock, and LastLock to point to the position that the client most recently attempted to acquire the lock is atomically trying to preempt. The LockEntries between Currlock and LastLock represent all clients waiting for lock processing. Each LockEntry is a 64-bit data entry, including a 1-bit Lockstate, a 15-bit CID, and reserved data space. Lockstate indicates whether the client is waiting for the lock (1 for waiting, 0 for not waiting). The CID's bit count can be adjusted based on the maximum number of clients supported by the system. This implementation uses 15 bits as an example, supporting 2^15 clients.

[0051] When a client attempts to acquire the lock, it first directly manipulates the server's LastLock memory using an RDMA Fetch-and-Add (FAA) atomic operation, incrementing LastLock by 1. (The initial value of LastLock can be set to 0). Because of the atomicity of RDMA FAA, different clients will obtain different LastLock values ​​when competing for the lock. Furthermore, the LastLock acquired by the first client will be prioritized over those acquired by subsequent clients, thus ensuring a FIFO (First-In, First-Out) principle.

[0052] After the client obtains LastLock through FAA operations, it takes the LastLock value modulo the circular buffer size n (let's assume the result is i). i is the position of the LockEntry entry to be set. The LastLock value is a 64-bit value, an astronomical number, which guarantees that it will not overflow.

[0053] The client initiates another RDMA Write operation, setting the LockEntry entry at position i. The entry content is its own CID, and Lockstate is set to 1. Since each client only sets its own CID, there will be no conflict with other CIDs, so a normal RDMA Write operation can be used. The memory address of the operation is: RDMA Write Address = LockRing Base address + (i * sizeof(LockEntry)). After acquiring the lock, the client waits for notification from the server.

[0054] The server iterates through the LockRing circular queue using the Currlock pointer (the initial value of Currlock can be set to 0). Based on the Lockentry entry indicated by currlock, it checks: if lockstate is 1, it retrieves the client's ClientLock memory address and rkey using the CID, and writes a lock holding event to the client's ClientLock memory via RDMA WRITE WITH IMM. The WRITE WITH IMM operation generates one event on the client side, notifying the client that it holds the lock. If lockstate is 0, it skips this step and proceeds to the next step. The client reads the lock holding event generated by WRITE WITH IMM and performs the lock operation. The server waits for the client to finish processing the lock.

[0055] After the client currently holding the lock finishes its operation, it writes the lockentry back to the server via RDMA WRITE WITH IMM to indicate the end of the lock operation. The lockstate value in the lockentry is set to 0. WRITE WITH IMM generates a lock release event. The server processes the lock release event, increments currlock by 1, and then performs a modulo operation on n before notifying the next client to execute the lock operation.

[0056] In one embodiment, this specification also provides an RDMA management method applied to a server in an RDMA network, the RDMA network including a server and several clients. The method includes: in response to an event in which lock operation completion information is written into the memory space corresponding to the lock entry address in the lock list currently pointed to by the lock pointer, adjusting the lock pointer to point to the next lock entry address in the lock list; the lock operation completion information is written by the client in response to an event indicating that the RDMA service has been completed in the memory space corresponding to the lock entry address; based on the identity information stored in the memory space corresponding to the newly pointed-to lock entry address, in the memory space of the client corresponding to the identity information... Lock holding information is written into the lock identifier area; in response to a request initiated by the client based on the lock holding information, RDMA services are executed; the identification information is obtained by the client reading the value of the lock counter according to the lock counter address, obtaining the address of the target lock entry in the lock list based on the value, and writing it into the corresponding memory space according to the target lock entry address; the lock counter address points to a specific fixed area in the server's memory, and the lock counter address is pre-sent by the server to the clients in the RDMA network; after the client writes its own identification information into the memory space corresponding to the lock entry address, the value of the lock counter is adjusted according to preset rules.

[0057] In one implementation, such as Figure 3 This specification also provides an RDMA management device for use in clients of an RDMA network, which includes a server and several clients. The device includes: a first module, configured to read the value of a lock counter based on a lock counter address, obtain the address of a target lock entry in the lock list based on the value, write its own identification information into the corresponding memory space based on the target lock entry address, and adjust the value of the lock counter according to preset rules; the lock counter address points to a specific fixed area in the server's memory, and the lock counter address is pre-distributed by the server to the clients of the RDMA network; the lock list includes several lock entry addresses, each lock entry... The addresses correspond to different memory spaces on the server. The second module monitors the preset lock identifier area in the local memory space and executes RDMA services in response to the event that lock holding information is written to the lock identifier area. The lock holding information is the identification information recorded in the memory space corresponding to the lock entry address selected by the server according to the lock pointer, and is written to the lock identifier area of ​​the corresponding client. The third module responds to the event that the RDMA service is completed and writes lock operation end information to the corresponding memory space according to the target lock entry address. The lock operation end information is used to indicate that the server's lock pointer points to the next lock entry address in the lock list.

[0058] In one implementation, the lock operation end information is used to instruct the server's lock pointer to point to the address of the next lock entry in the lock list. If the lock pointer was pointing to the last lock entry address in the lock list before being adjusted by the server according to the lock operation end information, then the lock pointer is adjusted by the server to point to the first lock entry address in the lock list according to the lock operation end information.

[0059] In one implementation, the number of lock entry addresses included in the lock list is equal to the number of clients included in the RDMA network. Each lock entry address is consecutive and corresponds to the same memory space size. The step of reading the lock counter value based on the lock counter address and obtaining the target lock entry address in the lock list based on that value includes: taking the remainder of the lock counter value based on the number of lock entry addresses; multiplying the remainder by the size of the memory space corresponding to a lock entry address to obtain an offset; and adding the offset to the base address of the lock list to obtain the target lock entry address. The step of adjusting the lock counter value according to a preset rule includes: incrementing the lock counter value by 1.

[0060] In one implementation, the server stores lock identifier area communication information pre-sent by the client of the RDMA network. The lock identifier area communication information includes the associated client's identity information, IP address, lock identifier area memory address, and the correspondence between the three.

[0061] In one implementation, such as Figure 4 This specification also provides an RDMA management device applied to the server side of an RDMA network, which includes a server and several clients. The device includes: a fourth module, used to adjust the lock pointer to point to the next lock entry address in the lock list in response to an event where lock operation completion information is written into the memory space corresponding to the lock entry address currently pointed to by the lock pointer; the lock operation completion information is written by the client in response to an event indicating the completion of RDMA service execution in the memory space corresponding to the lock entry address; and a fifth module, used to, based on the identity information stored in the memory space corresponding to the newly pointed-to lock entry address, add the lock operation completion information to the memory space of the client corresponding to the identity information. The lock identification area is written with lock holding information; the sixth module is used to execute RDMA services in response to a request initiated by the client based on the lock holding information; the identification information is obtained by the client reading the value of the lock counter according to the lock counter address, obtaining the address of the target lock entry in the lock list according to the value, and writing it into the corresponding memory space according to the target lock entry address; the lock counter address points to a specific fixed area in the server's memory, and the lock counter address is pre-sent by the server to the client in the RDMA network; after the client writes its own identification information into the memory space corresponding to the lock entry address, the value of the lock counter is adjusted according to a preset rule.

[0062] The implementation methods of the apparatus are the same as or similar to the corresponding implementation methods, and will not be described again here.

[0063] In one embodiment, this specification provides an electronic device including a processor and a readable storage medium storing machine-executable instructions executable by the processor. The processor executes the machine-executable instructions to implement the aforementioned RDMA management method. From a hardware perspective, a hardware architecture diagram can be found... Figure 5 As shown.

[0064] In one embodiment, this specification provides a readable storage medium storing machine-executable instructions that, when invoked and executed by a processor, cause the processor to implement the aforementioned RDMA management method.

[0065] Here, a readable storage medium can be any electronic, magnetic, optical, or other physical storage device that can contain or store information, such as executable instructions, data, etc. For example, a readable storage medium can be: RAM (Random Access Memory), volatile memory, non-volatile memory, flash memory, storage drives (such as hard disk drives), solid-state drives, any type of storage disk (such as optical discs, DVDs, etc.), or similar storage media, or combinations thereof.

[0066] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer, which can take the form of a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email sending and receiving device, game console, tablet computer, wearable device, or any combination of these devices.

[0067] For ease of description, the above devices are described in terms of function, divided into various units. Of course, in implementing this specification, the functions of each unit can be implemented in one or more software and / or hardware components.

[0068] Those skilled in the art will understand that embodiments of this specification can be provided as methods, systems, or computer program products. Therefore, this specification can take the form of a completely hardware implementation, a completely software implementation, or an implementation combining software and hardware aspects. Furthermore, embodiments of this specification can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0069] This specification is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments thereof. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0070] Furthermore, these computer program instructions can also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in the process. Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0071] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0072] Those skilled in the art will understand that embodiments of this specification can be provided as methods, systems, or computer program products. Therefore, this specification can take the form of a completely hardware implementation, a completely software implementation, or an implementation combining software and hardware aspects. Furthermore, this specification can take the form of a computer program product implemented on one or more computer-usable storage media (which may include, but are not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0073] The above description is merely an embodiment of this specification and is not intended to limit this specification. Various modifications and variations can be made to this specification by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this specification should be included within the scope of the claims of this specification.

Claims

1. An RDMA management method, characterized in that, A client-side application in an RDMA network, the RDMA network comprising a server and several clients, the method comprising: The lock counter value is read based on the lock counter address, the target lock entry address in the lock list is obtained based on the value, the identity information is written into the corresponding memory space based on the target lock entry address, and the lock counter value is adjusted according to preset rules; the lock counter address points to a specific fixed area in the server's memory, and the lock counter address is pre-sent by the server to the client in the RDMA network; the lock list includes several lock entry addresses, each lock entry address corresponding to a different memory space on the server; The system monitors a preset lock identifier area in the local memory space and executes RDMA services in response to an event where lock holding information is written to the lock identifier area. The lock holding information is the identification information recorded in the memory space corresponding to the lock entry address selected by the server according to the lock pointer, and is written to the lock identifier area of ​​the corresponding client. In response to the completion of the RDMA service, lock operation end information is written into the corresponding memory space according to the target lock entry address; the lock operation end information is used to indicate that the server's lock pointer points to the address of the next lock entry in the lock list.

2. The method according to claim 1, characterized in that, The lock operation end information is used to instruct the server's lock pointer to point to the address of the next lock entry in the lock list. If the lock pointer was pointing to the address of the last lock entry in the lock list before being adjusted by the server according to the lock operation end information, then the lock pointer is adjusted by the server to point to the address of the first lock entry in the lock list according to the lock operation end information.

3. The method according to claim 1, characterized in that, The lock list includes a number of lock entry addresses equal to the number of clients in the RDMA network. All lock entry addresses are consecutive and correspond to the same memory space size. The step of reading the lock counter value based on the lock counter address and obtaining the target lock entry address from the lock list based on that value includes: The value of the lock counter is modulo the number of lock entry addresses. The offset is obtained by multiplying the remainder by the size of the memory space corresponding to a lock entry address. The target lock entry address is obtained by adding the offset to the base address of the lock list. The step of adjusting the value of the lock counter according to preset rules includes: Increment the value of the lock counter by 1.

4. The method according to claim 1, characterized in that, The server stores lock identifier area communication information pre-sent by the client of the RDMA network. The lock identifier area communication information includes the associated client's identity information, IP address, lock identifier area memory address, and the correspondence between the three.

5. An RDMA management method, characterized in that, The method, applied to a server in an RDMA network, which includes a server and several clients, comprises: In response to an event where lock operation completion information is written to the memory space corresponding to the lock entry address in the lock list currently pointed to by the lock pointer, the lock pointer is adjusted to point to the next lock entry address in the lock list; the lock operation completion information is written by the client to the memory space corresponding to the lock entry address in response to the RDMA service completion event; Based on the identification information stored in the memory space corresponding to the newly pointed-to lock entry address, write the lock holding information into the lock identifier area in the memory space of the client corresponding to the identification information; In response to a request initiated by the client based on the lock holding information, execute the RDMA service; The identification information is obtained by the client reading the value of the lock counter based on the lock counter address, obtaining the address of the target lock entry in the lock list based on the value, and writing it into the corresponding memory space based on the target lock entry address; the lock counter address points to a specific fixed area in the server's memory, and the lock counter address is pre-sent by the server to the client in the RDMA network; after the client writes its own identification information into the memory space corresponding to the lock entry address, the value of the lock counter is adjusted according to preset rules.

6. An RDMA management device, characterized in that, A client application for an RDMA network, the RDMA network including a server and several clients, the device comprising: The first module is used to read the value of the lock counter according to the lock counter address, obtain the address of the target lock entry in the lock list according to the value, write its own identification information into the corresponding memory space according to the target lock entry address, and adjust the value of the lock counter according to preset rules; the lock counter address points to a specific fixed area of ​​the server memory, and the lock counter address is pre-sent by the server to the client of the RDMA network; the lock list includes several lock entry addresses, and each lock entry address corresponds to a different memory space on the server; The second module is used to monitor the preset lock identifier area in the local memory space, and to execute RDMA services in response to the event that lock holding information is written to the lock identifier area; the lock holding information is the identity recognition information recorded in the memory space corresponding to the lock entry address selected by the server according to the lock pointer, and written to the lock identifier area of ​​the corresponding client. The third module is used to respond to the event that the RDMA service has been completed by writing lock operation end information into the corresponding memory space according to the target lock entry address; the lock operation end information is used to instruct the server's lock pointer to point to the address of the next lock entry in the lock list.

7. The apparatus according to claim 6, characterized in that, The lock operation end information is used to instruct the server's lock pointer to point to the address of the next lock entry in the lock list. If the lock pointer was pointing to the address of the last lock entry in the lock list before being adjusted by the server according to the lock operation end information, then the lock pointer is adjusted by the server to point to the address of the first lock entry in the lock list according to the lock operation end information.

8. The apparatus according to claim 6, characterized in that, The lock list includes a number of lock entry addresses equal to the number of clients in the RDMA network. All lock entry addresses are consecutive and correspond to the same memory space size. The step of reading the lock counter value based on the lock counter address and obtaining the target lock entry address from the lock list based on that value includes: The value of the lock counter is modulo the number of lock entry addresses. The offset is obtained by multiplying the remainder by the size of the memory space corresponding to a lock entry address. The target lock entry address is obtained by adding the offset to the base address of the lock list. The step of adjusting the value of the lock counter according to preset rules includes: Increment the value of the lock counter by 1.

9. The apparatus according to claim 6, characterized in that, The server stores lock identifier area communication information pre-sent by the client of the RDMA network. The lock identifier area communication information includes the associated client's identity information, IP address, lock identifier area memory address, and the correspondence between the three.

10. An RDMA management device, characterized in that, A server-side application for an RDMA network, wherein the RDMA network includes a server and several clients, the device includes: The fourth module is used to adjust the lock pointer to point to the address of the next lock entry in the lock list in response to an event in which lock operation completion information is written into the memory space corresponding to the lock entry address in the lock list currently pointed to by the lock pointer. The lock operation completion information is written by the client into the memory space corresponding to the lock entry address in response to the event that the RDMA service has been completed. The fifth module is used to write lock holding information into the lock identifier area in the memory space of the client corresponding to the newly pointed-to lock entry address, based on the identity recognition information stored in the memory space corresponding to the new lock entry address. The sixth module is used to execute RDMA services in response to requests initiated by the client based on lock holding information; The identification information is obtained by the client reading the value of the lock counter based on the lock counter address, obtaining the address of the target lock entry in the lock list based on the value, and writing it into the corresponding memory space based on the target lock entry address; the lock counter address points to a specific fixed area in the server's memory, and the lock counter address is pre-sent by the server to the client in the RDMA network; after the client writes its own identification information into the memory space corresponding to the lock entry address, the value of the lock counter is adjusted according to preset rules.

11. An electronic device, characterized in that, include: A processor and a readable storage medium storing machine-executable instructions that can be executed by the processor to implement the method of any one of claims 1-5.

12. A readable storage medium, characterized in that, The readable storage medium stores machine-executable instructions that, when invoked and executed by a processor, cause the processor to implement the method described in any one of claims 1-5.

Images