A multi-producer multi-consumer bounded lock-free queue and enqueue / dequeue method

Abstract

The application discloses a multi-producer and multi-consumer bounded lock-free queue and a queue-in and queue-out method, relates to the technical field of data structures, and aims to solve the problem that the sequential shrinkage operation in the catchup stage of the prior art reduces concurrency and additional problems caused by replacing the CAS instruction with a high-efficiency instruction; the application comprises a production area and a consumption area, and head and tail pointers are introduced for the two areas respectively, wherein the head and tail pointers of the production area are respectively referred to as phead and ptail, the head and tail pointers of the consumption area are respectively referred to as chead and ctail, and version number areas are included in the phead, ptail, chead and ctail; the application uses a FAA instruction with better performance to replace the CAS instruction in the request stage to update the head pointer, and simultaneously solves the abnormal pointer problem that may occur through the designed rollback stage, so that the execution efficiency of the queue is improved, and the effect of reducing the overhead is achieved.

Description

Technical Field

[0001] This invention relates to the field of data structure technology, specifically to a bounded lock-free queue with multiple producers and multiple consumers and a method for enqueueing and dequeuing. Background Technology

[0002] A lock-free queue with multiple producers and consumers is a high-performance data structure that allows for the safe transfer of data between multiple threads without the need for synchronization mechanisms such as mutexes or semaphores. Lock-free queues are categorized into bounded and unbounded types based on their use cases. Bounded queues are typically implemented using circular arrays, while unbounded queues are usually implemented using linked lists. Compared to unbounded queues, bounded queues generally offer higher execution efficiency and are a better choice for scenarios where dynamic expansion of the queue capacity is not required.

[0003] A basic multi-producer, multi-consumer bounded lock-free queue typically uses a circular array to store data and maintains two pointers: a head pointer and a tail pointer. The head pointer points to the first element of the queue, while the tail pointer points to the next writable position. When a producer performs an enqueue operation, it first checks if the successor of the tail pointer is the head pointer. If they are, the queue is full; otherwise, it stores the new data at the position pointed to by the tail pointer and increments the tail pointer by one using a CAS operation (modulo the array size). When a consumer performs a dequeue operation, it first checks if the head and tail pointers are equal. If they are equal, the queue is empty; otherwise, it removes the element pointed to by the head pointer and increments the head pointer by one using a CAS operation (modulo the array size).

[0004] The aforementioned queue, as a basic implementation of a bounded, lock-free queue, has some performance aspects that require improvement. In multi-producer, multi-consumer scenarios, this implementation suffers from severe enqueue contention (hereinafter referred to as enq-enq contention) and dequeue contention (hereinafter referred to as deq-deq contention). When one producer or consumer is performing an enqueue or dequeue operation, other producers and consumers must wait for the operation to complete before they can continue their operations.

[0005] To address the aforementioned issues, the DPDK solution replaces the circular queue implementation with two pointers (head and tail) with two separate consumer and producer regions. This design significantly reduces contention during enq-enq and deq-deq operations. Each region, including the producer and consumer regions, is controlled by head and tail pointers. Taking the producer region as an example, multiple producers compete for slots at the head pointer, then are allocated slots sequentially, fill in their content, and finally submit their work to shrink the tail pointer. This allows producers to complete slot allocation without waiting for other producers to finish.

[0006] The DPDK solution is as follows: Figure 1As shown, the producer's enqueue operation is divided into two phases: request and catchup. In the request phase, multiple threads use CAS atomic instructions to allocate slots sequentially and return the old value of phead (i.e., the head pointer of the production area). In the catchup phase, the tail pointer is shrunk sequentially according to the order of request. Figure 1 Thread A in the process allocates earlier. It can check ptail (i.e., the tail pointer of the production area) and realize that it was allocated earliest by phead = ptail. Then it uses the CAS instruction to move ptail. After the update is completed, thread B can also realize that phead = ptail and further move ptail to complete the update.

[0007] The DPDK approach reduces enq-enq and deq-deq contention, but its drawbacks are that the overhead of CAS instructions is relatively large, and the sequential shrinking action in the catchup phase also reduces concurrency.

[0008] Building upon the above, researchers have proposed the Block-based Bounded Queue (BBQ), which inherits the DPDK approach and is divided into a producer region and a consumer region. Its main driving force is improving the three types of contention between enq-enq, deq-deq, and enq-deq instructions, starting from the FAA / MAX instructions. The FAA instruction is an atomic fetch_and_add instruction, which has much lower overhead than the CAS instruction. However, the FAA instruction lacks conditional checks, making its execution somewhat blind. The atomic MAX instruction, currently implemented on ARM systems, atomically determines the maximum value of two values ​​and assigns the maximum value to the target address. Because the MAX instruction includes conditional checks, its performance is stronger than FAA.

[0009] To accommodate FAA / MAX instructions, the BBQ solution introduces a block structure, restricting the semantics of FAA instructions to blocks. When the allocated slots exceed the block size, slot allocation fails. BBQ uses the MAX instruction to adjust the block pointer, which reduces enq-deq contention. However, unfortunately, the MAX instruction is limited to the ARM platform, making the BBQ solution difficult to implement on x86 systems. Summary of the Invention

[0010] To address the problems existing in the prior art, this invention proposes to provide a bounded lock-free queue with multiple producers and multiple consumers and an enqueue / dequeue method, which aims to solve the problem that the sequential shrinking action in the catchup phase of the prior art reduces concurrency and the additional problems caused by replacing CAS instructions with high-efficiency instructions.

[0011] A bounded, lock-free queue with multiple producers and multiple consumers includes a production region and a consumption region, and introduces head and tail pointers for each region. The head and tail pointers of the production region are called phead and ptail, respectively, and the head and tail pointers of the consumption region are called chead and ctail, respectively. phead, ptail, chead, and ctail include a version number region.

[0012] Preferably, the pointer areas in phead, ptail, chead, and ctail are connected to the version number area. The pointer area represents the index of the circular queue array pointed to by the current pointer, and its length determines the queue length. The version number area is initially zero and increases with the number of times the pointer is modified.

[0013] Preferably, the following invariant is used to determine whether the queue is empty or full:

[0014] The ptail pointer will always lag behind the phead pointer;

[0015] The ctail pointer will always lag behind the chead pointer;

[0016] The ctail pointer will always lag behind the ptail pointer;

[0017] The version number of ptail is always less than or equal to the version number of phead;

[0018] The version number of ctail is always less than or equal to the version number of chead.

[0019] A method for enqueuing a bounded, lock-free queue with multiple producers and consumers includes stages for updating phead, ptail, chead, and ctail, such as request, rollback, and catchup. The instruction for updating the pointer area and version number area is the FAA instruction.

[0020] Preferably, before the request phase ends, it is necessary to determine whether there is an exception in the pointer, i.e., whether phead has exceeded ctail. If there is an exception, a rollback operation is required to roll back the pointer that has exceeded the boundary. The rollback operation is shown in the rollback phase. After the rollback is successful, the catchup phase is entered.

[0021] Preferably, the order in which the out-of-bounds pointer is rolled back is Last-In-First-Out (LIFO).

[0022] Preferably, in the catchup phase of the enqueue operation in the production area, the version number of ptail needs to be incremented first. If the version number of ptail is the same as the version number of the current phead, the ptail pointer is modified; otherwise, the current ptail pointer is compared with the old phead pointer obtained by the thread in the request phase through the FAA instruction. If they are the same, the ptail pointer is incremented.

[0023] Preferably, the steps include:

[0024] S1: Based on the number of elements transmitted, use FAA instructions to move phead, increment the version number, and obtain the old value of phead;

[0025] S2: Check if the moved phead is out of bounds. If not, proceed to S3. If it is, check if the old value of phead held by this thread corresponds to the current value of phead. If the old value of phead held by this thread does not correspond to the current value of phead, repeat S2. If the old value of phead held by this thread corresponds to the current value of phead, proceed to S4.

[0026] S3: Fill the data elements into the allocated slots;

[0027] S4: Use the FAA directive to increment the ptail version number;

[0028] S5: Compare the version numbers of phead and ptail; if they match, proceed to S6; otherwise, the operation ends.

[0029] S6: Update ptail using the CAS command.

[0030] A dequeue method for a bounded, lock-free queue with multiple producers and multiple consumers includes the following steps:

[0031] S1: Based on the number of elements transmitted, use the FAA instruction chead at one end, increment the version number, and obtain the old value of chead;

[0032] S2: Check if the moved chead is out of bounds. If not, proceed to S3. If it is, check if the old chead value held by this thread corresponds to the current chead value. If the old chead value held by this thread does not correspond to the current chead value, repeat S2. If the old chead value held by this thread corresponds to the current chead value, proceed to S4.

[0033] S3: Fill the data elements into the allocated slots;

[0034] S4: Use the FAA directive to increment the ctail version number;

[0035] S5: Compare the version numbers of chead and ctail; if they match, proceed to S6; otherwise, the operation ends.

[0036] S6: Update ctail using the CAS command.

[0037] The beneficial effects of this invention include:

[0038] This invention, based on the DPDK solution, mainly introduces two improvements: First, it replaces the CAS instruction in the request phase with the more efficient FAA instruction for updating the head pointer, and solves potential pointer exceptions through a designed rollback phase, thereby improving queue execution efficiency and reducing overhead. Second, it introduces a mechanism for out-of-order modification of the tail pointer in the catchup phase, improving the concurrency of enqueue and dequeue operations. These two improvements enable the queue to achieve higher execution efficiency in multi-producer, multi-consumer scenarios, representing an 8% improvement over the DPDK solution.

[0039] Compared to traditional and DPDK solutions, the design of this invention solves the problem of reduced concurrency caused by sequential contraction actions in the catchup phase, thus improving concurrency. At the same time, the FAA instruction used has lower overhead than the CAS instruction. Compared to the BBQ solution using the MAX instruction, it removes platform limitations and introduces a rollback mechanism to solve the problem that the FAA instruction may cause by blindly executing without conditional judgment. Attached Figure Description

[0040] Figure 1 This is a queue implementation scheme for DPDK in the existing technology.

[0041] Figure 2 This refers to the phead rollback operation involved in Example 1.

[0042] Figure 3 This refers to the catchup process involved in Example 1.

[0043] Figure 4 This is a flowchart of the catchup process involved in Example 1.

[0044] Figure 5 This represents the normal state of the circular queue involved in Example 1.

[0045] Figure 6 This refers to the abnormal state of the circular queue involved in Example 1.

[0046] Figure 7 This is an example of the pointer structure involved in Example 1.

[0047] Figure 8(a) is the queuing flowchart involved in Example 1.

[0048] Figure 8 (b) is the queuing flowchart involved in Example 2. Detailed Implementation

[0049] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely represents selected embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0050] Example 1

[0051] The following is in conjunction with the appendix Figure 1-8 Specific embodiments of the present invention will be described in detail;

[0052] This invention references the basic design of the DPDK scheme, such as... Figure 1 The system introduces a production region and a consumption region for the queue, and introduces head and tail pointers for each region. The head and tail pointers of the production region are called phead and ptail, and the head and tail pointers of the consumption region are called chead and ctail. Then, the operation of moving pointers using CAS instructions in the DPDK scheme is replaced with the more efficient FAA instructions. An optimistic strategy is first used to update the pointers, and then the system checks for pointer out-of-bounds exceptions that may occur during the update. If an exception is found, a rollback is attempted.

[0053] The pointer structures phead, ptai l, chead, and ctai l used in this embodiment can be derived from... Figure 7 The structure shown indicates that the pointer itself is a 64-bit numeric type, consisting of a pointer area and a version number area. The pointer (or queue index area) represents the index of the circular queue array that the current pointer points to, and its length determines the queue length. Taking a 12-bit pointer area as an example, the queue length (i.e., the number of data elements it can hold) will be 4096 (i.e., 2 to the power of 12). The version number area is initially zero and increments with the number of pointer modifications. By concatenating the pointer area and the version number area, the effect of moving the pointer and incrementing the version number can be achieved simultaneously using only one FAA instruction.

[0054] The DPDK uses two phases, request and catchup, to update pointers in the production and consumption regions. This invention, due to the introduction of a rollback operation, adds a rollback phase to ensure the correctness of pointer update operations. The following section uses the production region as an example to describe the three phases of the enqueue operation in a multi-producer, multi-consumer bounded lock-free queue. Figure 8 As shown in (a).

[0055] For the request phase, the actions in this invention are relatively simple. First, the version number of phead is incremented using the FAA instruction, and the pointer is moved simultaneously. Then, the position of the pointer is checked to determine if it is out of bounds. In most cases, no out-of-bounds exception will occur, and the producer can fill the allocated slots with data and then directly enter the catchup phase. In a few cases, if an enq-deq conflict causes an exception, a rollback operation is required to roll back the out-of-bounds phead pointer. The rollback action is described in the rollback phase. After a successful rollback, the catchup phase begins.

[0056] During the rollback phase, because this invention directly uses FAA instructions to move the pointer during the request phase, it may lead to a pointer out-of-bounds exception, i.e., the production region head pointer phead crosses the consumer region tail pointer ctail. This exception is not allowed in the DPDK and BBQ schemes, but this invention overcomes this constraint to simplify the design. The resulting problem is that, to ensure the correctness of the pointer position, phead needs to be rolled back. Figure 2 As shown in the diagram, the lower part represents the initial normal state of the queue, where phead is still some distance from ctail. phead_max represents the maximum value phead can reach in the current scenario, corresponding to the position ctail-1. Suppose two producer threads, a and b, simultaneously request empty slots, but after allocation, it's found that both threads have exceeded ctail. In this case, it's necessary to roll back the movement of phead by these two threads. Since exceptions are introduced during the request phase, it's necessary to check for pointer exceptions before the request phase ends. If an exception exists, such as... Figure 2 As shown in the middle section, the rollback needs to be performed sequentially. Since the pointer is moved in the request phase in the order of thread A first, then thread B, the rollback must be performed in the order of thread B first, then thread A. The reason why the rollback cannot be performed out of order is that in a multi-threaded scenario, ctail is also moving, making it difficult to guarantee correctness.

[0057] For the catchup phase, to reduce enq-enq and deq-deq contention, this invention introduces version numbers in phead, ptail, chead, and ctail. For example... Figure 3 As shown, in the catchup phase, the ptail version number needs to be incremented first. If the ptail version number is the same as the current phead version number, the ptail pointer is modified. Otherwise, the current ptail pointer is compared with the old phead pointer obtained by the thread during the request phase via the FAA instruction; if they are the same, the ptail pointer is incremented. The entire catchup process is as follows: Figure 4 As shown, it is important to note that the CAS instruction is required in this process. After introducing the version number, multiple producers can submit out of order in the catchup phase. The thread that submits last is responsible for modifying the ptial pointer, while ensuring that the thread that allocates first can move the ptial pointer when submitting. This improves the overall parallelism.

[0058] Finally, because the design of this invention breaks two potential rules that a circular queue must follow—namely, that *tai* cannot surpass *head*, and that *head* cannot chase *tai*—the invalidation of these rules creates a new problem in determining whether a circular queue is full or empty. This invention uses the following invariant to derive the state of the circular queue:

[0059] The ptail pointer always lags behind the phead pointer.

[0060] The ctail pointer always lags behind the chead pointer.

[0061] The ctail pointer always lags behind the ptail pointer.

[0062] The version number of ptai l is always less than or equal to the version number of phead.

[0063] The version number of ctai l is always less than or equal to the version number of chead.

[0064] Analyzing the above rules, we can use `ptail` and `ctail` to replace the original rules to determine if the queue is empty or full, although this may introduce some exceptions. For example... Figure 5 As shown, under normal conditions, the head region and the stage region are far apart; however, as Figure 6 As shown, in the abnormal state, the data has almost filled the entire buffer, and the producer area is almost catching up with the consumer area. Since the circular queue determines whether the queue is full based on ptal and ctal, in... Figure 6In the illustrated state, although the queue is not full, if a new producer thread enqueues elements exceeding the number of remaining slots, a pointer out-of-bounds exception will occur. These exceptions can be resolved by the rollback mechanism described earlier, ensuring the correctness of the queue.

[0065] Example 2

[0066] like Figure 8 As shown in (b), this embodiment introduces the three stages of the dequeue operation of a multi-producer, multi-consumer bounded lock-free queue, specifically including the following steps:

[0067] S1: Based on the number of elements transmitted, use the FAA instruction chead at one end, increment the version number, and obtain the old value of chead;

[0068] S2: Check if the moved chead is out of bounds. If not, proceed to S3. If it is, check if the old chead value held by this thread corresponds to the current chead value. If the old chead value held by this thread does not correspond to the current chead value, repeat S2. If the old chead value held by this thread corresponds to the current chead value, proceed to S4.

[0069] S3: Fill the data elements into the allocated slots;

[0070] S4: Use the FAA directive to increment the ctai l version number;

[0071] S5: Compare the version numbers of chead and ctail; if they match, proceed to S6; if they do not match, the operation ends.

[0072] S6: Update ctai l using the CAS command.

[0073] The embodiments described above merely illustrate specific implementation methods of this application, and while the descriptions are detailed and specific, they should not be construed as limiting the scope of protection of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the technical solution of this application, and these modifications and improvements all fall within the scope of protection of this application.

Claims

1. A bounded, lock-free queue with multiple producers and multiple consumers, characterized in that, It includes a production area and a consumption area, and introduces head and tail pointers for each area. The head and tail pointers of the production area are called phead and ptail, respectively, and the head and tail pointers of the consumption area are called chead and ctail, respectively. phead, ptail, chead and ctail include version number areas. The pointer areas in phead, ptail, chead, and ctail are connected to the version number area. The pointer area represents the index of the circular queue array pointed to by the current pointer, and its length determines the length of the queue. The version number area is initially zero and increases with the number of times the pointer is modified. Use the following invariant to determine if the queue is empty or full: The ptail pointer will always lag behind the phead pointer; The ctail pointer will always lag behind the chead pointer; The ctail pointer will always lag behind the ptail pointer; The version number of ptail is always less than or equal to the version number of phead; The version number of ctail is always less than or equal to the version number of chead.

2. A method for enqueuing a multi-producer, multi-consumer bounded lock-free queue, used in the multi-producer, multi-consumer bounded lock-free queue as described in claim 1, characterized in that, The stages for updating phead, ptail, chead, and ctail include request, rollback, and catchup, using single CAS or FAA instructions to atomically modify the values ​​in the pointer area and version number area; Before the request phase ends, it is necessary to determine whether there is an exception in the pointer, i.e., whether phead has exceeded ctail. If there is an exception, a rollback operation is required to roll back the out-of-bounds pointer. The rollback operation is shown in the rollback phase. After the rollback is successful, the catchup phase is entered. The out-of-bounds pointer is rolled back in the order of Last-In-First-Out (LIFO). In the catchup phase of the enqueue operation in the production area, the version number of ptail needs to be incremented first. If the version number of ptail is the same as the version number of the current phead, the ptail pointer is modified; otherwise, the current ptail pointer is compared with the old phead pointer obtained by the thread in the request phase through the FAA instruction. If they are the same, the ptail pointer is incremented.

3. The enqueue method for a bounded, lock-free queue with multiple producers and multiple consumers according to claim 2, characterized in that, The steps include the following: S1: Based on the number of elements transmitted, use FAA instructions to move phead, increment the version number, and obtain the old value of phead; S2: Check if the moved phead is out of bounds. If not, proceed to S3. If it is, check if the old value of phead held by this thread corresponds to the current value of phead. If the old value of phead held by this thread does not correspond to the current value of phead, repeat S2. If the old value of phead held by this thread corresponds to the current value of phead, proceed to S4. S3: Fill the data elements into the allocated slots; S4: Use the FAA directive to increment the ptail version number; S5: Compare the version numbers of phead and ptail; if they match, proceed to S6; otherwise, the operation ends. S6: Update ptail using the CAS command.

4. A dequeue method for a multi-producer, multi-consumer bounded lock-free queue, used in the multi-producer, multi-consumer bounded lock-free queue as described in claim 1, characterized in that... The steps include the following: S1: Based on the number of elements transmitted, use the FAA instruction to move chead, increment the version number, and obtain the old value of chead; S2: Check if the moved chead is out of bounds. If not, proceed to S3. If it is, check if the old chead value held by this thread corresponds to the current chead value. If the old chead value held by this thread does not correspond to the current chead value, repeat S2. If the old chead value held by this thread corresponds to the current chead value, proceed to S4. S3: Fill the data elements into the allocated slots; S4: Use the FAA directive to increment the ctail version number; S5: Compare the version numbers of chead and ctail; if they match, proceed to S6; otherwise, the operation ends. S6: Update ctail using the CAS command.

Images