Lock-free queue enqueue and dequeue method and device, equipment and storage medium

Abstract

The invention provides a lock-free queue enqueue and dequeue method and device, equipment and a storage medium, and belongs to the technical field of data processing. The lock-free queue comprises a plurality of queue elements, at most at least one producer thread and at least one consumer thread exist in each queue element for operation at the same time, and the method comprises the steps that when the producer threads store data to be enqueued in the lock-free queue, if the producer threads correspond to the queue elements in the lock-free queue, the producer threads store the data to be enqueued in the lock-free queue; if the producer thread corresponds to the queue element in the lock-free queue, the producer thread is controlled to execute enqueue operation in the corresponding queue element, and if the producer thread does not correspond to the queue element in the lock-free queue or fails to enqueue in the corresponding queue element, a new queue element is allocated to the producer thread, and enqueue operation is carried out. By adopting the scheme disclosed by the invention, the throughput of the queue can be improved.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of data processing, and in particular, to a method and device for enqueuing and dequeuing a lock-free queue, an apparatus, and a storage medium. BACKGROUND

[0002] A queue is a basic infrastructure in system software such as an operating system, which serves as a core data structure in a variety of key scenarios, including but not limited to scheduling scenarios, log scenarios, and message communication scenarios. A lock-free queue with multiple producers and multiple consumers (mpmc) is widely used because it is a high-performance data structure. Producers and consumers are different threads, and multiple producers mean that multiple threads simultaneously store data in the queue, and multiple consumers mean that multiple threads simultaneously read data from the queue.

[0003] In a lock-free queue with multiple producers and multiple consumers, multiple different threads compete for metadata of the queue and access shared queue data, which respectively causes serious data competition and cache line invalidation, and as the number of threads accessing the queue simultaneously increases, the throughput of the queue also decreases. SUMMARY

[0004] The present disclosure provides a method and device for enqueuing and dequeuing a lock-free queue, which can reduce data competition and improve the throughput of the queue.

[0005] In a first aspect, the present disclosure provides a method for enqueuing a lock-free queue, the lock-free queue including a plurality of queue elements, each queue element having at most one producer thread and at least one consumer thread operating thereon at the same time, the method comprising: when a first producer thread stores to-be-enqueued data in the lock-free queue, if the first producer thread corresponds to a first queue element in the lock-free queue, controlling the first producer thread to perform an enqueuing operation in the first queue element; and if the first producer thread does not correspond to a queue element in the lock-free queue or the enqueuing in the first queue element fails, allocating a second queue element for the first producer thread in the lock-free queue, and controlling the first producer thread to perform an enqueuing operation in the second queue element.

[0006] In the scheme shown in the present disclosure, the lock-free queue is divided into a plurality of queue elements, and a producer thread operates on each queue element, which can avoid competition between all producer threads in the lock-free queue. Moreover, the queue element performs an enqueuing operation in the queue element in which the last enqueuing operation was performed each time enqueuing is performed, which ensures data locality within a period of time, reduces cache consistency overhead, and ultimately improves throughput.

[0007] In an optional manner, each queue element has at most one producer thread and one consumer thread operating at the same time, the control that the first producer thread performs the enqueue operation in the first queue element includes: determining the state of the first queue element; if the state of the first queue element is not closed, storing the data to be enqueued into the storage space of the first queue element; if the state of the first queue element is closed, determining that the first producer thread fails to enqueue in the first queue element.

[0008] In the scheme shown in the disclosure, each queue element corresponds to a state, and based on the state of the queue element, it can be determined whether the data to be enqueued corresponding to the queue element is enqueued in the queue element, thereby improving the enqueue efficiency.

[0009] In an optional manner, the determination of the state of the first queue element includes: reading the producer thread count value corresponding to the first queue element, the producer thread count value being used to indicate the state of the first queue element and the position index of the storage space of the first queue element; based on the producer thread count value, determining the state of the first queue element; if the state of the first queue element is not closed, storing the data to be enqueued into the storage space of the first queue element, including: if the state of the first queue element is not closed, storing the data to be enqueued into the position indicated by the position index.

[0010] In the scheme shown in the disclosure, the queue element corresponds to a producer thread count value, and based on the producer thread count value, the state of the queue element and the position index in the queue element are accurately determined.

[0011] In an optional manner, after the data to be enqueued is stored into the storage space of the first queue element, it further includes: if the storage space of the first queue element is fully occupied, determining whether the first queue element is consumed completely and whether the current consumer thread leaves the first queue element; if the first queue element is consumed completely and the consumer thread leaves the first queue element, setting the first producer thread as not corresponding to a queue element, and recycling the first queue element.

[0012] In the scheme shown in the disclosure, after the data to be enqueued is stored into the queue element, it is determined whether the queue element is full, and in the case of full, if the data in the queue element has been read completely, the queue element can be recycled to be used by other producer threads as early as possible, thereby improving the utilization rate of the queue element.

[0013] In an alternative manner, after determining that the state of the first queue element is closed, further comprising: judging whether the first queue element is consumed completely and whether the consumer thread leaves the first queue element; and if the first queue element is consumed completely and the consumer thread leaves the first queue element, recycling the first queue element.

[0014] In the scheme shown in the present disclosure, after a queue element is closed, it is indicated that no data can be stored in the queue element. If a consumer thread has read all the data in the queue element and leaves the queue element, it is indicated that all the data in the queue element has been read, and the queue element can be recycled to be used by other producer threads as early as possible, thereby improving the utilization rate of the queue element.

[0015] In an alternative manner, each queue element has at most one producer thread and one consumer thread operating at the same time. In the lock-free queue, allocating the second queue element for the first producer thread comprises: allocating the i-th queue element for the first producer thread in the lock-free queue, performing an enqueue-close operation on the (i-D1)-th queue element, D1 being the maximum number of queue elements enqueued simultaneously in the lock-free queue; and initializing the i-th queue element as the second queue element.

[0016] In the scheme shown in the present disclosure, the maximum number of queue elements enqueued simultaneously in the lock-free queue is fixed. In order to ensure the maximum number D1 of queue elements enqueued simultaneously, when the i-th queue element is allocated, the (i-D1)-th queue element is ensured to be closed, so as to ensure the degree of disorder of the producer threads.

[0017] In an alternative manner, the enqueue-close operation performed on the (i-D1)-th queue element comprises: reading the producer thread count value of the (i-D1)-th queue element, and setting the state of the (i-D1)-th queue element to closed in the producer thread count value through an atomic operation.

[0018] In the scheme shown in the present disclosure, when a queue element is closed, the closing is performed through an atomic operation, which does not affect the producer thread currently producing in the queue element.

[0019] In an alternative manner, D1=K / (2*NE), K being a global disorder degree value of the lock-free queue, and NE being the storage space capacity of each queue element.

[0020] In an alternative manner, each queue element is a lock-free queue, a lock-free stack or a lock-free tree.

[0021] In a second aspect, the disclosure provides an out-queue method of a lock-free queue, the lock-free queue comprising a plurality of queue elements, each queue element being operated by at most one producer thread and at least one consumer thread at the same time, the method comprising: when a first consumer thread reads data from the lock-free queue, if the first consumer thread corresponds to a third queue element in the lock-free queue, controlling the first consumer thread to perform an out-queue operation in the third queue element; if the first consumer thread does not correspond to a queue element in the lock-free queue or fails to out-queue in the third queue element, allocating a fourth queue element for the first consumer thread in the lock-free queue, and controlling the first consumer thread to perform an out-queue operation in the fourth queue element.

[0022] In the scheme shown in the disclosure, in the lock-free queue, when a consumer thread reads data from the lock-free queue, if there is a queue element corresponding to the consumer thread, the data is read from the queue element first, and if the data fails to be read from the queue element or there is no queue element corresponding to the consumer thread, a new queue element is applied to read data. In this way, not only is the competition between all consumer threads in the lock-free queue avoided, but also each time the out-queue is performed, the out-queue operation is performed in the queue element of the last out-queue, ensuring the data locality in a period of time.

[0023] In an optional manner, each queue element is operated by at most one producer thread and one consumer thread at the same time, and the controlling the first consumer thread to perform an out-queue operation in the third queue element comprises: judging whether there is unread data in the third queue element; if there is unread data in the third queue element, reading the unread data; and if there is no unread data in the third queue element, judging whether the first consumer thread fails to out-queue in the third queue element based on the current state of the third queue element.

[0024] In the scheme shown in the disclosure, when a consumer thread performs an out-queue operation in a queue element, if there is unread data in the queue element, the data is read, and if there is no unread data in the queue element, whether the consumer thread fails to out-queue in the queue element is quickly determined based on the current state of the queue element.

[0025] In an optional manner, the judging whether the first consumer thread fails to out-queue in the third queue element based on the current state of the third queue element comprises: if the state of the third queue element is closed, it is determined that the first consumer thread fails to out-queue in the third queue element; and if the state of the third queue element is not closed, a close operation is performed on the third queue element, if the close operation is successful, it is determined that the first consumer thread fails to out-queue in the third queue element, and if the close operation fails, the process of judging whether there is unread data in the third queue element is returned.

[0026] In the scheme shown in the present disclosure, the queue element corresponds to a state, and whether the queue element is dequeued unsuccessfully is determined based on whether the state of the queue element is closed. When the state is closed, the dequeuing is unsuccessful. When the state is not closed, a closing operation is performed to determine whether the dequeuing is unsuccessful. In this way, whether the consumer thread is dequeued unsuccessfully in the queue element can be accurately determined based on the state.

[0027] In an optional manner, after the closing operation is successfully performed, the third queue element is recycled if the current producer thread leaves the third queue element.

[0028] In the scheme shown in the present disclosure, the queue element is recycled only when there is no producer thread in the queue element, so that the producer thread in the queue element is not affected.

[0029] In an optional manner, after the unread data is read, it is determined whether the current producer thread leaves the third queue element if the third queue element has no unread data. If the current producer thread leaves the third queue element, the first consumer thread is set as not corresponding to a queue element, and the third queue element is recycled.

[0030] In the scheme shown in the present disclosure, after the consumer thread reads the unread data in a certain queue element, if the queue element has no unread data and the producer thread has left the queue element, it means that there will be no data in the queue element. The queue element can be recycled to release the queue element.

[0031] In an optional manner, each queue element has at most one producer thread and one consumer thread operating at the same time. Before the fourth queue element is allocated to the first consumer thread, it is determined that the number of times of searching for a queue element in a first direction by the current dequeuing is less than a first threshold value, or it is determined that there is an unconsumed queue element between the third queue element and the queue element where the head of the producer thread is located.

[0032] In the scheme shown in the present disclosure, before a new queue element is allocated to the consumer thread, if the consumer thread has applied for a new queue element multiple times, it means that a pair of producer thread and consumer thread repeatedly apply for a new queue element, and the consumer thread closes the queue element of the producer thread each time. In this case, the consumer thread cannot be allocated a new queue element. Otherwise, the consumer thread can be allocated a queue element. Or, there is still data in the lock-free queue that has not been read, which means that there will be no blockage, and a new queue element can be allocated. In this way, the allocation of a new queue element will not affect other consumer threads and producer threads.

[0033] In an alternative manner, each queue element is operated by at most one producer thread and one consumer thread at the same time, and the fourth queue element allocated for the first consumer thread comprises: in a case that a current queue element to be allocated is an i-th queue element, if an i-D2-th queue element is completed to be recycled, an operation of reserving the i-th queue element is performed, D2 is a maximum number of queue elements simultaneously dequeued from the lock-free queue; if the operation of reserving the i-th queue element succeeds, the i-th queue element is determined as the fourth queue element.

[0034] In the scheme shown in the disclosure, the maximum number of queue elements simultaneously dequeued from the lock-free queue is fixed, and in order to ensure the maximum number D2 of queue elements simultaneously dequeued, when the i-th queue element is allocated, it is ensured that the i-D2-th queue element is completed to be recycled, so as to ensure the out-of-order degree of the consumer thread.

[0035] In an alternative manner, D2=K / (2*NE), K is a global out-of-order degree value of the lock-free queue, and NE is a storage space capacity of each queue element.

[0036] In a third aspect, the disclosure provides a multi-producer and multi-consumer lock-free queue, the lock-free queue comprising a plurality of queue elements, and each queue element is allowed to be operated by at most one producer thread and one consumer thread at the same time.

[0037] In an alternative manner, the number of queue elements simultaneously performing the enqueue operation in the lock-free queue is a first value, the first value is equal to K / (2*NE), NE is equal to a storage space capacity of each queue element, and K is a global out-of-order degree value of the lock-free queue.

[0038] In an alternative manner, the number of queue elements simultaneously performing the dequeue operation in the lock-free queue is a second value, the second value is equal to K / (2*NE), NE is equal to a storage space capacity of each queue element, and K is a global out-of-order degree value of the lock-free queue.

[0039] In an alternative manner, each queue element is a lock-free queue, a lock-free stack, or a lock-free tree.

[0040] In a fourth aspect, the disclosure provides an enqueue device of a lock-free queue, the lock-free queue comprising a plurality of queue elements, and each queue element is operated by at least one producer thread and at least one consumer thread at the same time, and the device comprises:

[0041] The enqueuing module is configured to, when the first producer thread deposits the data to be enqueued into the lock-free queue, if the first producer thread corresponds to a first queue element in the lock-free queue, control the first producer thread to perform an enqueuing operation in the first queue element.

[0042] The application module is configured to, if the first producer thread does not correspond to a queue element in the lock-free queue or fails to enqueue in the first queue element, allocate a second queue element for the first producer thread in the lock-free queue.

[0043] The enqueuing module is further configured to control the first producer thread to perform an enqueuing operation in the second queue element.

[0044] In an optional manner, each queue element is operated by at most one producer thread and one consumer thread at the same time, and the enqueuing module is configured to:

[0045] determine the state of the first queue element;

[0046] if the state of the first queue element is not closed, store the data to be enqueued into the storage space of the first queue element;

[0047] if the state of the first queue element is closed, determine that the first producer thread fails to enqueue in the first queue element.

[0048] In an optional manner, the enqueuing module is configured to:

[0049] read a producer thread count value corresponding to the first queue element, the producer thread count value being used to indicate the state of the first queue element and a position index of the storage space of the first queue element;

[0050] determine the state of the first queue element based on the producer thread count value;

[0051] The enqueuing module is configured to:

[0052] if the state of the first queue element is not closed, store the data to be enqueued into the position indicated by the position index.

[0053] In an optional manner, the enqueuing module is further configured to:

[0054] after storing the data to be enqueued into the storage space of the first queue element, if the storage space of the first queue element is fully occupied, determine whether the first queue element is consumed completely and whether a current consumer thread leaves the first queue element.

[0055] If the first queue element is consumed completely and the consumer thread leaves the first queue element, the first producer thread is set as not corresponding to a queue element and the first queue element is recycled.

[0056] In an optional mode, the enqueuing module is further used for:

[0057] After determining that the state of the first queue element is closed, it is judged whether the first queue element is consumed completely and whether the current consumer thread leaves the first queue element;

[0058] If the first queue element is consumed completely and the consumer thread leaves the first queue element, the first queue element is recycled.

[0059] In an optional mode, each queue element has at most one producer thread and one consumer thread operating at the same time, and the applying module is used for:

[0060] In the lock-free queue, the first producer thread is allocated with an i-th queue element, and an enqueuing closed operation is performed on an i-D1-th queue element, D1 being the maximum number of queue elements simultaneously enqueued in the lock-free queue;

[0061] The i-th queue element is initialized as the second queue element.

[0062] In an optional mode, the applying module is used for:

[0063] A producer thread count value of the i-D1-th queue element is read, and through an atomic operation, a state of the i-D1-th queue element is set as closed in the producer thread count value.

[0064] In an optional mode, D1=K / (2*NE), K being a global out-of-order degree value of the lock-free queue and NE being a storage space capacity of each queue element.

[0065] In an optional mode, each queue element is a lock-free queue, a lock-free stack or a lock-free tree.

[0066] In a fifth aspect, the disclosure provides a de-queuing device of a lock-free queue, the lock-free queue including a plurality of queue elements, each queue element having at most at least one producer thread and at least one consumer thread operating at the same time, and the device including:

[0067] A de-queuing module is used for, when a first consumer thread reads data from the lock-free queue, if the first consumer thread corresponds to a third queue element in the lock-free queue, controlling the first consumer thread to perform a de-queuing operation in the third queue element.

[0068] allocate a fourth queue element for the first consumer thread in the lock-free queue if the first consumer thread does not correspond to a queue element in the lock-free queue or fails to dequeue in the third queue element,

[0069] The dequeue module is further configured to control the first consumer thread to perform a dequeue operation in the fourth queue element.

[0070] In an optional manner, each queue element has at most one producer thread and one consumer thread operating at the same time, and the dequeue module is configured to:

[0071] determine whether there is unread data in the third queue element;

[0072] read the unread data if there is unread data in the third queue element;

[0073] if there is no unread data in the third queue element, determine whether the first consumer thread fails to dequeue in the third queue element based on a current state of the third queue element.

[0074] In an optional manner, the dequeue module is configured to:

[0075] if the state of the third queue element is closed, determine that the first consumer thread fails to dequeue in the third queue element;

[0076] if the state of the third queue element is not closed, perform a close operation on the third queue element, if the close operation is successful, determine that the first consumer thread fails to dequeue in the third queue element, and if the close operation fails, return to perform the process of determining whether there is unread data in the third queue element.

[0077] In an optional manner, the dequeue module is further configured to:

[0078] if the current producer thread leaves the third queue element after the close operation is successful, recycle the third queue element.

[0079] In an optional manner, the dequeue module is further configured to:

[0080] if there is no unread data in the third queue element after the unread data is read, determine whether the current producer thread leaves the third queue element;

[0081] if the current producer thread leaves the third queue element, set the first consumer thread to not correspond to a queue element, and recycle the third queue element.

[0082] In an optional mode, at most one producer thread and one consumer thread operate on each queue element at the same time, and the allocation module is further configured to:

[0083] Before allocating the fourth queue element for the first consumer thread, it is determined that the number of times of searching for queue elements in a first direction by the current dequeue is less than a first threshold value, the first direction being a direction in which the third queue element is toward a queue element at the head of the producer thread; or,

[0084] It is determined that there is an unconsumed queue element between the third queue element and a queue element at the head of the producer thread.

[0085] In an optional mode, at most one producer thread and one consumer thread operate on each queue element at the same time, and the allocation module is configured to:

[0086] In a case where the current queue element to be allocated is an i-th queue element, if an i-D2-th queue element is completed to be recycled, an operation of reserving the i-th queue element is performed, D2 being a maximum number of queue elements that are simultaneously dequeued in the lock-free queue;

[0087] If the operation of reserving the i-th queue element is successful, the i-th queue element is determined as the fourth queue element.

[0088] In an optional mode, D2=K / (2*NE), K being a global out-of-order degree value of the lock-free queue, and NE being a storage space capacity of each queue element.

[0089] In a sixth aspect, the present disclosure provides a computing device, comprising a processor and a memory;

[0090] The processor is configured to execute instructions stored in the memory, so that the computing device performs the method according to the first aspect or any optional mode of the first aspect.

[0091] In a seventh aspect, the present disclosure provides a computing device, comprising a processor and a memory;

[0092] The processor is configured to execute instructions stored in the memory, so that the computing device performs the method according to the second aspect or any optional mode of the second aspect.

[0093] In an eighth aspect, the present disclosure provides a computer program product comprising instructions which, when executed by a computing device, cause the computing device to perform the method according to the first aspect or any optional mode of the first aspect.

[0094] In a ninth aspect, this disclosure provides a computer program product comprising instructions that, when executed by a computing device, cause the computing device to perform the method as described in the second aspect or any alternative method of the second aspect.

[0095] In a tenth aspect, this disclosure provides a computer-readable storage medium including computer program instructions that, when executed by a computing device, perform the method as described in the first aspect or any alternative method of the first aspect.

[0096] In one aspect, this disclosure provides a computer-readable storage medium including computer program instructions that, when executed by a computing device, perform the method as described in the second aspect or any of the alternative methods of the second aspect.

[0097] In a twelfth aspect, this disclosure provides a lockless queue system comprising the means as described in the fourth aspect or any alternative of the fourth aspect and the means as described in the fifth aspect or any alternative of the fifth aspect. Attached Figure Description

[0098] Figure 1 This is a schematic diagram of the structure of a computing device provided in an exemplary embodiment of this disclosure;

[0099] Figure 2 This is a schematic diagram of the structure of a lockless queue provided in an exemplary embodiment of this disclosure;

[0100] Figure 3 This is a schematic diagram of a method for enqueuing in a lock-free queue, provided by an exemplary embodiment of this disclosure;

[0101] Figure 4 This is a schematic diagram of the enqueueing process in a lock-free queue provided by an exemplary embodiment of the present disclosure;

[0102] Figure 5 This is a schematic diagram of the process of enqueuing elements in a queue, provided by an exemplary embodiment of this disclosure;

[0103] Figure 6 This is a schematic diagram illustrating the process of allocating queue elements to a producer thread in a lock-free queue, provided by an exemplary embodiment of this disclosure.

[0104] Figure 7 This is a schematic diagram of the process of closing a queue element in a lockless queue, provided by an exemplary embodiment of this disclosure;

[0105] Figure 8 This is a schematic diagram of a method for dequeuing from a lock-free queue, provided by an exemplary embodiment of this disclosure;

[0106] Figure 9 Figure 1 is a flowchart of a dequeue operation in a lock-free queue according to an example embodiment of the present disclosure;

[0107] Figure 10 Figure 2 is a flowchart of a dequeue operation in a queue element according to an example embodiment of the present disclosure;

[0108] Figure 11 Figure 3 is a schematic diagram of a consumer thread head and a producer thread head of a lock-free queue according to an example embodiment of the present disclosure;

[0109] Figure 12 Figure 4 is a schematic diagram of allocating a queue element to a consumer thread in a lock-free queue according to an example embodiment of the present disclosure;

[0110] Figure 13 Figure 5 is a schematic diagram of an enqueue device of a lock-free queue according to an example embodiment of the present disclosure;

[0111] Figure 14 Figure 6 is a schematic diagram of a dequeue device of a lock-free queue according to an example embodiment of the present disclosure. DETAILED DESCRIPTION

[0112] In order to make the purpose, technical solutions and advantages of the present disclosure clearer, the embodiments of the present disclosure will be described in further detail below with reference to the accompanying drawings.

[0113] Some terms and concepts related to the embodiments of the present disclosure will be explained below.

[0114] 1. A queue is a basic infrastructure in system software such as an operating system, which is used as a core data structure in a variety of key scenarios, including but not limited to scheduling scenarios, log scenarios, and message communication scenarios. A plurality of basic operations can be performed in a queue, the most basic of which are enqueue and dequeue operations. The enqueue operation refers to storing a new data in the queue, and the dequeue operation refers to taking a data out of the queue.

[0115] 2. A lock-free queue refers to a queue that does not use a mutex to ensure the correctness of multi-threaded operations. That is, when multiple threads operate on the queue at the same time, the lock-free queue can still ensure correctness, and the multiple threads include multiple producer threads or multiple consumer threads. In a lock-free queue, the same data cannot be dequeued and obtained by multiple threads at the same time, the dequeue operations of multiple threads cannot obtain incorrect or incomplete data, and the enqueue operations of multiple threads cannot cover and lose data.

[0116] 3. Atomic operation, through which the simultaneous existence of multiple threads executing on the core at the same time does not cause the same phenomenon as being interrupted when executing in an interleaved manner, and once started, the operation can run until the end. Atomic operations include compare and swap (CAS) operations and fetch and add (FFA) operations, etc., and the atomic CAS operation atomically replaces the variable with the provided new value only when the value of the variable is the same as the old value provided by the thread, which can guarantee atomicity, i.e., no other thread can modify the variable after the comparison is completed but before the modification to the new value. The atomic FFA operation atomically reads the old value and adds a preset value, such as 1, to the old value, so that the atomicity between the obtained old value and the changed new value can be guaranteed.

[0117] 4. Cache coherence overhead, which refers to the overhead of ensuring consistency of accessed data when multiple cores simultaneously modify or access shared data. For example, current processors have multiple cores, and each core has a cache (L1 cache) that can cache the cache line in which the data accessed by the core is located. When a core needs to modify data, the core needs to invalidate the data cache line in the cache of other cores through a cache coherence protocol and wait for the operation to end. When a core needs to read data, if the cache line where the data is located is not local to the core or has been marked as invalid by another core, the core needs to obtain the cache line through the cache coherence protocol.

[0118] 5. Data locality, which refers to whether adjacent accessed data is within the cache line local to the core and the cache line has not been set as invalid by another core. When data locality is good, the overhead of accessing data is low.

[0119] 6. Multi-producer and multi-consumer lock-free queue, which refers to the simultaneous existence of multiple threads in the queue to store data in the queue and multiple threads to take data out of the queue. For example, consider a request queue that processes user requests, and multiple different users will put requests into the global request queue, while multiple different task threads will take requests from the request queue and process them. Lock-free means that the queue does not need to use an additional mutex to ensure mutual exclusion, but instead relies on hardware-provided atomic operations to ensure correctness. Compared to a queue that uses a mutex, the biggest advantage of a lock-free queue is that it avoids blocking, and even when multiple producer threads and multiple consumer threads exist at the same time, it can still guarantee progress.

[0120] 7. The enqueue and dequeue order of a queue refers to the order in which data is stored in the queue and the order in which data is read from the queue. Currently, many queues use a first-in-first-out (FIFO) order for dequeueing. For example, if the enqueue order is 1, 2, 3, 4, 5, then the dequeue order is also 1, 2, 3, 4, 5. However, in multi-producer, multi-consumer scenarios, strictly guaranteeing the FIFO order would serialize enqueueing and dequeueing, thus compromising performance scalability. Therefore, the relaxed K-FIFO order was proposed. K-FIFO means that the oldest element in the current queue will be dequeued within a maximum of K steps (including K steps). For example, in the example above, if K = 3, the dequeue order could be 2, 3, 1, 5, 4. The oldest data, 1, will definitely be dequeued within the third step (including the third step), and subsequent data will follow the same pattern. K-FIFO provides room for improving the performance scalability of lock-free queues in multi-producer, multi-consumer scenarios.

[0121] The practical significance of K-FIFO: Although K-FIFO does not strictly guarantee the exact order of enqueue and dequeue, the order guarantee it provides is sufficient for most application needs. For example, in a request queue, not strictly processing requests in the order they arrive will not affect the correctness of the program, but it will impact the processing latency. Specifically, a larger K value means that requests may be processed later, thus increasing the tail latency. Conversely, a smaller K value means that the order is closer to FIFO, and the tail latency can be reduced.

[0122] The execution subject of the embodiments of this disclosure is described below.

[0123] This disclosure provides a method for enqueuing a lock-free queue, wherein the execution entity of the method is the enqueue device. Optionally, the device is a hardware device, such as a computing device like a server or terminal. Optionally, the device is a software device, such as a software program running on the computing device.

[0124] This disclosure provides a method for dequeuing a lock-free queue, wherein the method is executed by a dequeueing device. Optionally, the device is a hardware device, such as a computing device like a server or terminal. Optionally, the device is a software device, such as a software program running on the computing device.

[0125] Among them, such as Figure 1 The diagram shows a schematic of a computing device 100, which may optionally be implemented using a general bus architecture. The computing device 100 includes at least one processor 101, a communication bus 102, a memory 103, and at least one network interface 104.

[0126] The processor 101 is, for example, a general central processing unit (CPU), a network processer (NP), a graphics processing unit (GPU), a neural-network processing units (NPU), a data processing unit (DPU), a microprocessor, or one or more integrated circuits for implementing the present disclosure. For example, the processor 101 includes an application-specific integrated circuit (ASIC), a programmable logic device (PLD) or a combination thereof. The PLD is, for example, a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL) or any combination thereof.

[0127] The communication bus 102 is used to transmit information between the above-mentioned components. The communication bus 102 can be divided into an address bus, a data bus, a control bus, etc. For the convenience of representation, Figure 1 In the figure, only one thick line is used to represent the communication bus 102, but it does not mean that there is only one bus or only one type of bus.

[0128] The memory 103 is, for example, a read-only memory (ROM) or other type of static storage device that can store static information and instructions that are not to be changed by the computer; alternatively, the memory 103 is, for example, a random access memory (RAM), or other type of dynamic storage device that can store information and instructions for execution by the computer; alternatively, the memory 103 is, for example, an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM), or other optical disk storage, including compressed optical disk, laser disk, optical tape, and the like; a magnetic disk storage or other magnetic storage devices or media; or any other medium that can be used to carry or store desired program code means in the form of instructions or data structures and that can be accessed by the computer, but is not limited to. The memory 103 is, for example, independent and connected to the processor 101 through the communication bus 102. The memory 103 can also be integrated with the processor 101.

[0129] The network interface 104 uses any transceiver-like mechanism for communicating with other devices or computer networks. The network interface 104 includes a wired network interface, and can also include a wireless network interface. The wired network interface can be, for example, an Ethernet interface. The Ethernet interface can be an optical interface, an electrical interface, or a combination thereof. The wireless network interface can be, for example, a wireless local area networks (WLAN) interface, a cellular network interface, or a combination thereof.

[0130] In a specific implementation, as an example, the processor 101 can include one or more CPUs.

[0131] In a specific implementation, as an example, the computing device 100 can include multiple processors. Each of these processors can be a single-CPU or a multi-CPU. The processor herein can refer to one or more devices, circuits, and / or processing cores for processing data (such as computer program instructions).

[0132] In a particular implementation, as an example, the computing device 100 can also include an output device and an input device. The output device is in communication with the processor 101 and can display information in a variety of ways. For example, the output device can be a liquid crystal display (LCD), a light emitting diode (LED) display device, a cathode ray tube (CRT) display device, a projector, or the like. The input device is in communication with the processor 101 and receives input from a user in a variety of ways. For example, the input device can be a mouse, a keyboard, a touch screen device, a sensor device, or the like.

[0133] In some embodiments, the memory 103 is configured to store program code 1031 for implementing the enqueueing in the present disclosure, and the processor 101 executes the program code 1031 stored in the memory 103. That is, the computing device 100 can implement the enqueueing method provided by the method embodiments through the processor 101 and the program code 1031 in the memory 103.

[0134] In some embodiments, the memory 103 is configured to store program code 1031 for implementing the dequeueing in the present disclosure, and the processor 101 executes the program code 1031 stored in the memory 103. That is, the computing device 100 can implement the dequeueing method provided by the method embodiments through the processor 101 and the program code 1031 in the memory 103.

[0135] The application scenarios of the embodiments of the present disclosure are described below.

[0136] In the embodiments of the present disclosure, the lock-free queue can be applied to the operating system kernel, system software, and user software, and the system software includes databases and parallel frameworks, etc. In the embodiments of the present disclosure, the lock-free queue is distributed in the form of source code or a high-performance concurrency library. Specifically, the software can directly implement a similar queue in the source code and call the corresponding interface, or provide a linkable library containing the implementation manner of the present disclosure, and the linkable library exposes the queue-related interfaces to the outside, and the software can call these interfaces in the form of dynamic linking or static linking.

[0137] The lock-free queue can also be fixed in hardware, and a similar high-performance queue can be implemented at the hardware level. For example, the hierarchical competition-avoiding algorithm is fixed in an addressing module, and the software uses the addressing module to find a suitable position to place, and the hardware can exist in a smart network card or a hardware performance monitoring unit (PMU) event record, etc.

[0138] The lock-free queue can be a request queue or a task queue without strict order requirement, and the time delay of task processing is controlled by adjusting K.

[0139] The lock-free queue can also be a message queue without strict order requirement, and the time when the message is seen at last is controlled by adjusting K.

[0140] The lock-free queue can also be a log or a trace. In these scenarios, the messages are reordered, and thus there is no need to maintain the original order when dequeuing.

[0141] The above are several examples of the lock-free queue, and any queue without strict order requirement can be implemented by using the lock-free queue in the embodiments of the present disclosure.

[0142] Before describing the enqueue process and the dequeue process, the lock-free queue in the embodiments of the present disclosure is first described.

[0143] The lock-free queue includes a plurality of queue elements, and each queue element is operated by at most one producer thread and one consumer thread at the same time. Thus, each queue element can be operated by at most one producer thread at the same time, can be operated by at most one consumer thread at the same time, or can be operated by at most one producer thread and one consumer thread at the same time. Alternatively, each queue element can be operated by at most a plurality of producer threads and a plurality of consumer threads at the same time, or can be operated by at most a plurality of producer threads and one consumer thread at the same time, or can be operated by at most one producer thread and a plurality of consumer threads at the same time. Thus, the lock-free queue is a hierarchical queue, and the outer layer is a multi-producer multi-consumer queue, such as a block-based bounded queue (BBQ) and a data plane development kit (DPDK) queue, and the inner layer is a single-producer single-consumer queue or a multi-producer multi-consumer queue. For example, Figure 2 An architectural diagram of the lock-free queue is provided, and at most one producer thread and one consumer thread are allowed to operate on a queue element at the same time, Figure 2The diagram shows six queue elements, numbered 1 through 6. Consumer thread 0 operates on queue element 1, consumer thread 1 operates on queue element 2, and consumer thread 2 and producer thread 0 operate on queue element 3. In queue element 3, consumer thread 2 is reading data from the position indicated by the second position index, and producer thread 0 is writing data from the position indicated by the second-to-last position index (white squares indicate positions where no data has been written, striped squares indicate positions where data has been read, and black squares indicate positions where data is stored but not yet read). Producer thread 1 operates on queue element 4, and producer thread 2 operates on queue element 5. Queue element 6 is an unoccupied queue element.

[0144] Furthermore, within a given period, in a lock-free queue, the same consumer thread reads (i.e. consumes) data from the same queue element, and the same producer thread produces (i.e. stores) data from the same queue element. This avoids contention between all producer threads and all consumer threads in the lock-free queue on the performance-critical path, while also ensuring data locality within a certain timeframe.

[0145] Especially when a queue element is operated on by at most one producer thread and one consumer thread, the queue element, on the performance-critical path, avoids contention between multiple producer threads and multiple consumer threads, as is common in lock-free queues, and allows the use of the lower-overhead single-producer single-consumer (SPSC) queue. Furthermore, it ensures data locality over a period of time. Ideally, either the consumer thread or the producer thread will exclusively occupy a queue element, and its cached lines can be retained locally, thus avoiding cache consistency overhead and enhancing data locality.

[0146] In addition, Figure 2 The lock-free queue shown also indicates that the out-of-order value for both the consumer thread and the producer thread is K, meaning that the global out-of-order value of the lock-free queue is K.

[0147] In the lock-free queue, the outer multi-producer multi-consumer queue can use any multi-producer multi-consumer queue implementation, which is not limited by the embodiments of the present disclosure. The outer multi-producer multi-consumer queue needs to provide producer-side operations including: an application (i.e., allocation) operation, a storage (put) operation, and a complete production operation, etc. The application operation refers to applying for new space to store content, the storage operation refers to storing data in the applied position, and the complete production operation refers to updating the mark, which represents the completion of production. The outer multi-producer multi-consumer queue needs to provide consumer-side functions including: a reservation operation, a read (get) operation, and a complete consumption operation. The reservation operation refers to reading the address of a data that has completed production from the queue, the read operation refers to reading data from the reserved position, and the complete consumption operation refers to updating the mark, which represents the completion of consumption.

[0148] In the lock-free queue, the inner queue element can be any data structure, for example, a lock-free queue, a lock-free stack, and a lock-free tree, and the data structure supports the eviction operation. Among them, the queue element can be a single-producer single-consumer queue or a multi-producer multi-consumer queue (used only as a single-producer single-consumer queue) or a lock-free stack or a lock-free tree in the case that at most one producer thread and one consumer thread operate on the queue element at the same time.

[0149] In the lock-free queue, the stored data can be real data or a storage address corresponding to the real data, etc. For each queue element, the storage space can be represented by a position index, one position index can store one data, and the storage space capacity is the number of position indexes. Regarding the outer multi-producer multi-consumer queue, the metadata includes but is not limited to the position index of the storage space of the queue element and the indication value of whether the queue element is closed, etc.

[0150] Figure 3 The method flow of enqueuing is provided, see steps 301 to 302.

[0151] Step 301, when the first producer thread stores the data to be enqueued in the lock-free queue, if the first producer thread has a first queue element in the lock-free queue, the first producer thread is controlled to perform an enqueuing operation in the first queue element.

[0152] In this embodiment, multiple producer threads exist on the processor core of the computing device. The first producer thread is any thread that has data to be enqueued. When the first producer thread stores data to be enqueued in the lock-free queue, it determines whether there is a corresponding queue element. If there is a corresponding queue element (which can be called the first queue element), it controls the first producer thread to perform an enqueue operation on the first queue element. For example, the first producer thread reads a cached local variable. When the first producer thread has a corresponding first queue element, the local variable corresponds to a pointer to the first queue element; when the first producer thread does not have a corresponding queue element, the local variable does not correspond to a pointer to a queue element.

[0153] Step 302: If the first producer thread does not have a corresponding queue element in the lock-free queue or fails to enqueue in the first queue element, then allocate a second queue element to the first producer thread in the lock-free queue and control the first producer thread to perform an enqueue operation in the second queue element.

[0154] In this embodiment, when the first producer does not have a corresponding queue element in the lock-free queue, or when enqueuing in the first queue element fails, a new queue element is attempted to be allocated to the first producer in the lock-free queue. After successful allocation, the allocated queue element is called the second queue element, and the first producer thread performs the enqueue operation in the second queue element.

[0155] use Figure 3 The process shown avoids competition among all producer threads in the lock-free queue on the performance-critical path. On the other hand, each time an element is enqueued, the enqueue operation is performed on the previously enqueued element, ensuring data locality for a period of time, reducing cache consistency overhead, and ultimately improving throughput.

[0156] The following example assumes that at most one producer thread and one consumer thread exist in the queue at any given time. Figure 3 The process will be explained.

[0157] Figure 4 The process of enqueuing the first producer thread in the lock-free queue is provided. This process is executed by the first producer thread, see steps 401 to 409.

[0158] Step 401: Enqueueing begins. Check if the first producer thread has a corresponding queue element. If it does, proceed to step 402; otherwise, proceed to step 407.

[0159] In the embodiment, the first producer thread reads the local variable of the cache, which is a pointer of the corresponding queue element when the first producer thread corresponds to the queue element, and which is not a pointer of the queue element when the first producer thread does not correspond to the queue element.

[0160] In step 402, the first producer thread calls the enqueue operation of the queue element, and executes the enqueue operation in the corresponding queue element. If the enqueue operation in the corresponding queue element is successful, the step 403 is executed, otherwise the step 407 is executed.

[0161] In the embodiment, the flow of the enqueue operation in the first queue element is described with reference to steps 501 to 507 in Figure 5

[0162] In step 403, after the enqueue operation is successful, it is judged whether the storage space of the corresponding queue element is fully occupied. If yes, the step 404 is executed, otherwise the step 406 is executed.

[0163] In the embodiment, after the first producer thread successfully enqueues in the first queue element, in order to determine whether the first queue element can continue to store data next time, the first producer thread acquires the current position index in the first queue element. If the current position index is equal to the position index of the storage space of the first queue element, it is determined that the storage space of the first queue element is fully occupied, otherwise it is determined that the storage space is not fully occupied, or the first queue element corresponds to the current storage amount, and if the current storage amount is equal to the storage space size of the first queue element, it is determined that the storage space of the first queue element is fully occupied, otherwise it is determined that the storage space is not fully occupied. When the first queue element is fully occupied, it means that the first queue element cannot continue to store data, and the first producer thread needs to re-enable a new queue element. When the first queue element is not fully occupied, it means that the first queue element can continue to store data, and the enqueue operation is executed.

[0164] In step 404, it is judged whether the corresponding queue element is consumed completely and whether the consumer thread is away. If the corresponding queue element is consumed completely and there is no consumer thread currently, the step 405 is executed, otherwise the step 406 is executed.

[0165] ​In the embodiment, after the first queue element is fully occupied, it is determined whether the data in the first queue element is read by all the consumer threads. If the data is read by all the consumer threads, it is determined whether the current consumer thread leaves the first queue element. For example, the first producer thread acquires the consumer thread count value (the consumer thread count value is the metadata of the first queue element), the consumer thread count value is used to indicate the position index of the next data read by the consumer thread, it is determined whether the read position indicated by the consumer thread count value exceeds the storage space size of the first queue element. If the read position exceeds the storage space size of the first queue element, it is determined that the data in the first queue element is read by all the consumer threads. Otherwise, it is determined that the data in the first queue element is not read by all the consumer threads. If the data is read by all the consumer threads and the current consumer thread leaves the first queue element, the first queue element can be recycled. If the data is read by all the consumer threads but the consumer thread is in the first queue element, the first queue element is not recycled, and it is determined that the enqueuing is completed. If the data is not read by all the consumer threads, it is determined that the enqueuing is completed, and the first queue element is not processed. For example, each queue element further corresponds to an additional flag bit. For the first queue element, the additional flag bit indicates whether the current first queue element has the consumer thread. It is determined whether the consumer thread exists by atomically operating the additional flag bit. The initial value of the additional flag bit is a first value. The first value is reduced by 1 each time the thread operating leaves. When the consumer thread leaves, the first value is reduced by 1 to obtain a second value. When the first producer thread leaves, the second value is further reduced by 1 to obtain a third value. The first producer thread determines whether the third value is a preset value. If the third value is the preset value, it is determined that the current first queue element does not have the consumer thread. Otherwise, it is determined that the current first queue element has the consumer thread, that is, the consumer thread has not left. The first value can be set according to actual needs. For example, if the first value is 2, the preset value is 0.

[0166] It should be noted that the atomic operation is taken as an example of the atomic FAA operation. Other atomic operations can also be used, and the embodiments of the present disclosure are not limited thereto. For example, the atomic CAS operation can also be used.

[0167] In step 405, the first queue element is recycled, and the first producer thread is set to not correspond to the queue element.

[0168] In the embodiment, the first producer thread sets the local variable corresponding thereto to not correspond to the queue element. In this way, when the first producer thread performs the enqueuing operation again, the queue element needs to be allocated again. The first producer thread performs the process of recycling the first queue element, which is also referred to as the process of consuming the first queue element, and changes the flag bit indicating that the first queue element has ended to a first value. The first value indicates that the first queue element has been completely consumed. When the first queue element is not completely consumed, the flag bit indicating that the first queue element has ended is a second value.

[0169] The process of consuming the first queue element is described by taking the ring queue as an example of the outer multi-producer multi-consumer queue. In a typical multi-producer multi-consumer queue, when the first queue element is consumed, the consumer count value of the multi-producer multi-consumer queue is increased by 1 through an atomic operation. The consumer count value is used to record the number of queue elements that have been consumed. When the producer thread is back to the position, it is determined whether the consumer count value is greater than the producer count value-Tube_SIZE. The producer count value is the producer count value of the multi-producer multi-consumer queue, which is used to record the number of queue elements that have been allocated. Tube_SIZE is the maximum number of queue elements included in the outer multi-producer multi-consumer queue. If the consumer count value is greater than the producer count value-Tube_SIZE, it indicates that the consumer count value has consumed the data produced in the last round at the position, and the producer thread can select the first queue element to produce.

[0170] The process is similar to the ring queue when other structures of queues are used.

[0171] In step 406, the enqueuing is completed, and the enqueuing success is returned.

[0172] In step 407, the queue element is allocated to the first producer thread in the lock-free queue. It is determined whether the allocation is successful. If the allocation is successful, the process proceeds to step 408; otherwise, the process proceeds to step 409.

[0173] In the embodiment, the process of allocating the queue element by the first producer thread is described by referring to the flow shown in FIG. 4B. Figure 6

[0174] In step 408, the allocated queue element is saved to the first producer thread locally, and the process returns to step 402.

[0175] In the embodiment, after the second queue element is successfully allocated, the first producer thread sets the local variable to correspond to the second queue element, and returns to the flow of step 402.

[0176] In step 409, there is no free queue element, and the enqueuing failure is returned.

[0177] In the embodiment, when the first producer thread fails to allocate the queue element, it indicates that there is no free queue element in the lock-free queue, and the enqueuing fails. The first producer thread can perform the enqueuing operation in the lock-free queue again after a preset time length, that is, the process returns to the flow of step 402. The size of the preset time length can be set according to actual needs. Figure 4

[0178] Figure 5 ​​A process of enqueuing of a first producer thread in a first queue element is provided, the process is executed by the first producer thread, see steps 501 to 507.

[0179] In step 501, the enqueuing of the queue element starts, and the state of the first queue element is determined.

[0180] In this embodiment, the first producer thread reads the producer thread count value corresponding to the first queue element (the producer thread count value is the metadata of the first queue element), which is used to indicate the state of the first queue element and the position index of the storage space of the first queue element. The indication bit of the state of the first queue element can be set at any position of the producer thread count value. For example, the highest bit of the producer thread count value is used to identify whether the first queue element is closed, and the remaining bits indicate the position index of the storage space of the first queue element. If it is closed, the first producer thread cannot continue to write data into the first queue element, but needs to return an enqueuing failure, and in the flow of Figure 4 allocating a new queue element, if it is not closed, the first producer thread can write the data to be enqueued into the position indicated by the position index in the producer thread count value.

[0181] Optionally, since the consumer thread also has the processing of closing the queue element, in order to achieve consistency, the first producer thread reads the producer thread count value by atomic FAA operation and increases the producer thread count value by 1. Here, the atomic operation is taken as an example of atomic FAA operation, and other atomic operations can also be used, which are not limited in the embodiments of the present disclosure, for example, atomic CAS operation can also be used.

[0182] In step 502, it is judged whether the state of the first queue element is closed. If it is closed, step 504 is executed, otherwise step 502 is executed.

[0183] In this embodiment, the first producer thread judges whether the state of the first queue element is closed based on the content indicated by the highest bit of the producer thread count value. For example, if the highest bit indicates 1, it is determined that the state of the first queue element is closed, and if the highest bit indicates 0, it is determined that the state of the first queue element is not closed.

[0184] In step 503, the data to be enqueued is stored in the storage space of the first queue element.

[0185] In this embodiment, if the first queue element is not closed, the data to be enqueued is written into the position indicated by the position index in the producer thread count value.

[0186] In step 504, an enqueuing success is returned, and the enqueuing operation ends.

[0187] Step 505, judging whether the first queue element is consumed completely and whether the consumer thread leaves, if the first queue element is consumed completely and the current consumer thread leaves the first queue element, turning to execute step 506, otherwise turning to execute step 507.

[0188] In the embodiment, the detailed process of step 505 can refer to the description of step 404, which is not repeated here.

[0189] Step 506, recycling the first queue element.

[0190] In the embodiment, the first producer thread executes the process of recycling the first queue element, also called the process of consuming the first queue element, and changes the flag bit of the first queue element having ended to the first value, which indicates that the first queue element has completed consumption.

[0191] Step 507, returning the enqueue failure.

[0192] In the embodiment, the first producer thread determines the enqueue failure when determining that the first queue element is consumed completely or the current consumer thread does not leave the first queue element, and needs to apply for a new queue element.

[0193] Figure 6 The process of allocating a new queue element for the first producer thread is provided, which is executed by the first producer thread, and can refer to steps 601 to 604.

[0194] Step 601, calling the application operation of the outer multi-producer multi-consumer queue, judging whether the application operation is successful, if the application operation is successful, turning to execute step 602, otherwise turning to execute step 604.

[0195] In the embodiment, the first producer thread calls the application operation of the outer multi-producer multi-consumer queue, which means applying for a new space as a queue element. The specific application operation process can be any way, which is not limited by the embodiment of the disclosure.

[0196] Step 602, allocating the i th queue element for the first producer thread in the lock-free queue, and executing the enqueue close operation on the i th-D1 queue element, D1 being the maximum number of queue elements being enqueued simultaneously in the lock-free queue.

[0197] In the present embodiment, in the lock-free queue, in order to guarantee a certain order of enqueuing and dequeuing, the number of queue elements that can be simultaneously enqueued or dequeued at the same time is limited, that is, the maximum number of queue elements simultaneously enqueued or the maximum number of queue elements simultaneously dequeued is limited. The maximum number is related to the storage space capacity NE of each queue element and the global out-of-order degree value K of the lock-free queue. For example, D1 = K / (2*NE), the global out-of-order degree value K is set according to actual needs, or D1*NE+D2*NE = K, D2 is the maximum number of queue elements simultaneously dequeued in the lock-free queue, the out-of-order degree value of the producer thread is K1, equal to D1*NE, and the out-of-order degree value of the consumer thread is K2, equal to D2*NE. When D1 and D2 are the same, it means that the out-of-order degree values of the producer thread and the consumer thread are the same, otherwise it means that the out-of-order degree values of the producer thread and the consumer thread are different. After the first producer thread occupies all the storage space of the first queue element, a new queue element needs to be applied, that is, after the first producer thread consumes the local first queue element, a new queue element needs to be applied. When applying a new queue element, it is determined whether there are already D1 queue elements being operated in parallel. If there are already D1 queue elements, the earliest queue element among the D1 queue elements is closed and is required to move forward. In this way, when the new queue element is the i-th queue element, the i-D1 queue element needs to be closed. The first producer thread needs to be closed when there is no producer thread in the i-D1 queue element, so the closing operation is realized through an atomic operation. The flow of the closing operation is shown in the flowchart of FIG. 8. Figure 7

[0198] It should be noted that if the outer multi-producer multi-consumer queue is a bounded queue or a ring queue, the i-D1 queue element may need to be wrapped, that is, (Tube_SIZE+i-D1) % Tube_SIZE, where Tube_SIZE is the maximum number of queue elements included in the outer multi-producer multi-consumer queue.

[0199] In this way, the number of simultaneously enqueued queue elements is additionally checked to avoid the out-of-order degree of the producer exceeding the requirement.

[0200] In step 603, the i-th queue element is initialized as the second queue element, and the allocation is successful.

[0201] ​In the embodiment, after the closing of the i-D1th queue element is completed, one queue element is initialized and placed in the position of the i th queue element, and the process is implemented through the put operation of the outer multi-producer multi-consumer queue. Then the first producer thread calls the produce operation of the outer multi-producer multi-consumer queue to complete the production of the second queue element and returns the second queue element, indicating that the allocation of the second queue element is completed.

[0202] Here, after the allocation is successful, the producer count value mentioned above is added by 1.

[0203] Step 604, return the allocation failure.

[0204] In the flow of the embodiment, by controlling the number of queue elements simultaneously performing the enqueue operation, the out-of-order degree of the producer thread can be avoided to exceed the requirement. Figure 6

[0205] The process of the first producer thread closing the queue element is provided, which is executed by the first producer thread, and see steps 701 to 704. Figure 7 Step 701, start closing, read the current producer thread count value of the i-D1th queue element to the first local variable.

[0206] In the embodiment, each queue element maintains a producer thread count value, which is used to indicate the state of the first queue element and the position index of the storage space of the first queue element. It is assumed that the current producer thread count value is represented by prod, and the first local variable is represented by iprod.

[0207] Step 702, set the value indicating the state in the producer thread count value to a value indicating closing, and store it to the second local variable.

[0208] In the embodiment, the first producer thread sets the value indicating the state in the producer thread count value to a third value, which indicates that the queue element has been closed, for example, the third value is 1. The changed local variable is stored to the second local variable, which is represented by nprod.

[0209] Step 703, through an atomic operation, determine whether the local variable and the producer thread count value are guaranteed to be atomic, if the first local variable and the producer thread count value are still guaranteed to be atomic, go to step 704, otherwise return to step 701 to retry.

[0210]

[0211] ​In this embodiment, the atomic CAS operation is used as an example for explanation. Through the atomic CAS operation, it is atomically determined whether prod is still iprod. If it is true, the process proceeds to step 704. If it is false, it means that the producer thread has modified prod for the i-D1th queue element, and the i-D1th queue element cannot be closed at this time.

[0212] Step 704: Replace the producer thread count with the second local variable to confirm that the (i-D1)th queue element has been successfully closed.

[0213] In this embodiment, the first producer thread replaces `prod` with `nprod`, so that after reading the producer thread count value again, the indicator state value is closed accordingly. Thus, after closing the (i-D1)th queue element, subsequent producer threads cannot store new data into the (i-D1)th queue element.

[0214] The above describes the process of enqueuing the producer thread; the following describes the process of dequeuing the consumer thread. Figure 8 The process for dequeuing consumer threads is provided, see steps 801 to 802.

[0215] Step 801: When the first consumer thread reads data from the lock-free queue, if the first consumer thread has a corresponding third queue element in the lock-free queue, then the first consumer thread is controlled to perform a dequeue operation in the third queue element.

[0216] Step 802: If the first consumer thread does not have a corresponding queue element in the lock-free queue or fails to dequeue from the third queue element, then allocate a fourth queue element to the first consumer thread in the lock-free queue and control the first consumer thread to perform a dequeue operation in the fourth queue element.

[0217] exist Figure 8 In the process shown, when the first consumer thread reads data from the lock-free queue, it prioritizes reading data from the corresponding queue element if one exists. Only if reading data from the corresponding queue element fails, or if no queue element exists, does it request a new queue element to read data. In this way, on the performance-critical path, the queue element avoids competition among all consumer threads in the lock-free queue. Furthermore, each time dequeuing occurs, the dequeue operation is first performed on the previously dequeued queue element, ensuring data locality for a period of time.

[0218] The following example assumes that at most one producer thread and one consumer thread exist in the queue at any given time. Figure 8 The process will be explained.

[0219] Figure 9 A process of de-queuing of a first consumer thread in a lock-free queue is provided, which is executed by the first consumer thread, see step 901 to step 913.

[0220] Step 901, de-queuing starts, detecting whether the first consumer thread corresponds to a queue element, if corresponding to a queue element, then go to step 902, otherwise go to step 910.

[0221] In this embodiment, the first consumer thread is currently performing de-queuing operation, the first consumer thread reads the local variable of the cache, which is the pointer corresponding to the queue element when the first consumer thread corresponds to a queue element, and which is the pointer not corresponding to the queue element when the first consumer thread does not correspond to a queue element.

[0222] Step 902, the first consumer thread calls the de-queuing operation of the queue element, and performs the de-queuing operation in the corresponding queue element. It is judged whether the de-queuing is successful, if the de-queuing is successful, go to step 903, otherwise go to step 907.

[0223] In this embodiment, the first consumer thread de-queues in the corresponding queue element, see step 1101 to step 1106 in the Figure 11 .

[0224] Step 903, judging whether the corresponding queue element of the first consumer thread is all read, if all read, go to step 904, otherwise go to step 906.

[0225] In this embodiment, it is assumed that the corresponding queue element of the first consumer thread in the lock-free queue is called the third queue element, after the first consumer thread de-queues successfully in the third queue element, in order to determine whether the next time can continue to read data in the third queue element, the first consumer thread obtains the consumer thread count value, which indicates the position index of the next data to be read, and obtains the producer thread count value of the third queue element, judges whether the position index is less than the position index in the producer thread count value, if not less than, it is determined that the data in the third queue element is all read, otherwise it is determined that the data in the third queue element is not all read. When the data in the third queue element is not all read, it means that the third queue element can continue to read data in the third queue element, and the current de-queuing operation is executed, when the data in the third queue element is all read, it means that the data in the third queue element cannot be read, and the next time the data needs to be read needs to enable a new queue element.

[0226] Step 904, judging whether the producer thread in the corresponding queue element of the first consumer thread leaves, if leaves, go to step 905, otherwise go to step 906.

[0227] In the embodiment, the first consumer thread judges whether the producer thread still exists in the third queue element, which can also be considered as judging whether the producer thread leaves the third queue element. In the case that the producer thread does not exist, it is indicated that the third queue element is produced and no data is stored in the third queue element, and the third queue element can be recycled. In the case that the producer thread exists, it is indicated that the third queue element is not produced completely, and the producer thread can store data in the third queue element again, and the de-queue is completed. The process of judging whether the producer thread exists in the third queue element is as follows: each queue element also corresponds to an additional flag bit. For the third queue element, the additional flag bit indicates whether the producer thread currently exists in the third queue element. Whether the producer thread exists is judged by atomically operating the additional flag bit. The initial value of the additional flag bit is a first value. The first value is reduced by 1 each time the thread operating leaves. When the producer thread leaves, the first value is reduced by 1 to obtain a second value. When the first consumer thread leaves, the second value is reduced by 1 to obtain a third value. The first consumer thread judges whether the third value is a preset value. If the third value is the preset value, it is determined that the producer thread currently does not exist in the third queue element. Otherwise, it is determined that the producer thread currently exists in the third queue element. The first value can be set according to actual needs. For example, if the first value is 2, the preset value is 0.

[0228] It should be noted that the atomic operation is taken as an example of the atomic FAA operation, and other atomic operations can also be used. The embodiment of the present disclosure is not limited, for example, the atomic CAS operation can also be used.

[0229] In step 905, the queue element corresponding to the first consumer thread is recycled, and it is set that the first consumer thread does not correspond to a queue element.

[0230] In the embodiment, the first consumer thread recycles the third queue element. After recycling is completed, the flag bit indicating that the third queue element has ended is changed to a first value, and the first value indicates that the third queue element has completed consumption. The local variable is set to not correspond to a queue element by the first consumer thread. Next time when data is read, the queue element needs to be determined again.

[0231] In step 906, the de-queue is completed, and the de-queue success is returned.

[0232] In step 907, it is judged whether the first consumer thread corresponding queue element is successfully closed due to the first consumer thread passing the producer thread. If yes, go to step 908. Otherwise, go to step 910.

[0233] In the embodiment, after the first consumer thread fails to dequeue the third queue element, it is determined whether the first consumer thread successfully closes the third queue element and the closing reason is that the position index of the first consumer thread exceeds the position index of the data written by the producer thread. If yes, it is indicated that the producer thread can be driven out of the third queue element. Subsequently, it is determined whether the producer thread leaves the third queue element. If not, it is indicated that the producer thread is not driven out. Then, a new queue element can be applied to be consumed. If yes, it is indicated that the producer thread is driven out. Then, the third queue element is recycled. The process of determining whether the producer thread leaves the third queue element is described in step 904, which will not be repeated here.

[0234] Here, the first consumer thread successfully closes the third queue element, but the producer thread has not left the third queue element. The producer thread reenters the third queue element, obtains the indication bit indicating the closing in the producer thread count value, and finds that the first consumer thread has left the third queue element. The producer thread performs the recycling operation on the third queue element.

[0235] In step 908, it is determined whether the producer thread corresponding to the queue element of the first consumer thread leaves. If yes, the step 909 is performed. Otherwise, the step 910 is performed.

[0236] In the embodiment, the process in step 908 is described in step 904, which will not be repeated here.

[0237] In step 909, the queue element corresponding to the first consumer thread is recycled.

[0238] In step 910, it is determined whether the queue element can be moved forward. If yes, the step 911 is performed. Otherwise, the step 913 is performed.

[0239] In the embodiment, in order to avoid that a pair of producer thread and consumer thread repeatedly apply for a new queue element and the consumer thread closes the queue element of the producer thread each time (for example, the producer thread newly applies for a queue element, the producer thread does not write data in time, the consumer thread is scheduled to the queue element to read data, and the data cannot be read, so the queue element is closed), it is determined whether the queue element can be moved forward based on the following manner:

[0240] Method 1: The first consumer thread determines the number of queue elements to be moved forward in this dequeue operation. If this number exceeds a first threshold, or if it is less than the first threshold, then it is determined that queue elements can be moved forward; otherwise, it is determined that queue elements cannot be moved forward. Here, "moving forward" refers to the direction of searching for queue elements in the first direction, which is the direction from the third queue element towards the queue element at the head of the producer thread. The queue element at the head of the producer thread refers to the last queue element in the lock-free queue to be allocated to the producer thread.

[0241] Method 2: The first consumer thread iterates between the third and fifth queue elements to check for any unconsumed queue elements. Unconsumed queue elements refer to those that haven't been read yet. The fifth queue element is the queue element at the head of the producer thread. If any exist, it means other consumer threads won't block (make progress) and can move queue elements; otherwise, it's certain that queue elements cannot be moved forward. Alternatively, the first consumer thread can iterate between the queue element at the head of the consumer thread and the fifth queue element, because consumer threads exist between the third and fifth queue elements, preventing movement. For example, see... Figure 10 The diagram shows six queue elements, numbered 1 through 6. Consumer thread 0 operates on queue element 1, consumer thread 1 operates on queue element 2, consumer thread 2 and producer thread 0 operate on queue element 3, producer thread 1 operates on queue element 4, and producer thread 2 operates on queue element 5. The producer thread head is producer thread 2, and the consumer thread head is consumer thread 2. This indicates that there is an unallocated queue element 4 between the consumer thread head and the producer thread head, so a queue element can be allocated to the first consumer thread. The queue element where the consumer thread head is located refers to the last queue element in the lock-free queue to be allocated to a consumer thread.

[0242] Step 911: Request a new queue element to consume in the lock-free queue. If the request is successful, proceed to step 912; otherwise, proceed to step 913.

[0243] In this embodiment, when a queue element can be moved forward, the first consumer thread calls the outer interface for obtaining a new queue element to request and consume the new queue element. If the request is successful, it indicates that there is data to be read, and the process proceeds to step 912. If the request fails, it indicates that there is no queue element to be read, and the process proceeds to step 913. The process of requesting a queue element in the lock-free queue in step 911 is described below. Figure 12 Steps 1201 to 1205.

[0244] Step 912, save the new queue element to the cache corresponding to the first consumer thread, and go to step 902.

[0245] In this embodiment, the first consumer thread updates the local variable to the new queue element (referred to as the fourth queue element), and then performs the dequeue operation in the new queue element, that is, returns to step 902.

[0246] Step 913, return the dequeue failure.

[0247] In this embodiment, if the new queue element is not successfully applied or the queue element cannot be moved forward, it means that there is no queue element to be read, and this time the dequeue in the lock-free queue fails.

[0248] Figure 11 The first consumer thread performs the dequeue operation in the corresponding queue element, and the process is performed by the first consumer thread, as shown in steps 1101 to 1106.

[0249] Step 1101, start the dequeue in the third queue element, and determine whether there is unread data in the third queue element. If there is unread data, go to step 1102, and if there is no unread data, go to step 1104.

[0250] In this embodiment, the first consumer thread starts the dequeue in the third queue element, the first consumer thread obtains the producer thread count value and the consumer thread count value of the third queue element, and removes the first indication bit in the producer thread count value, which is used to indicate the state of the third queue element, and stores it as a variable. The consumer thread count value is used to record the position index of the data to be read. The first consumer thread determines whether the position index is less than the variable. If it is less, it is determined that there is unread data, otherwise it is determined that there is no unread data. This step is to determine whether the first consumer thread has reached before the producer thread. For example, in Figure 10 , for queue element 3, the consumer thread count value is the second position index, and the producer thread count value is the second last position index. The second position index is less than the second last position index, indicating that there is unread data.

[0251] Step 1102, read data.

[0252] In this embodiment, the first consumer thread reads data at the position index in the consumer thread count value. After reading successfully, the position index is incremented by 1, indicating the position index of the data to be read next time.

[0253] It should be noted that since only the first consumer thread exists in the third queue element, no other consumer thread exists, so no atomic operation is needed when adding 1 to the position index to save processing resources.

[0254] Step 1103: Dequeue success.

[0255] Step 1104: Determine whether the third queue element is closed. If yes, go to step 1106; otherwise, go to step 1105.

[0256] In this embodiment, the first consumer thread acquires the first indication bit in the producer thread count value of the third queue element. If the first indication bit indicates closing, it is determined that the third queue element is closed by the producer thread; otherwise, it is determined that the third queue element is not closed by the producer thread.

[0257] Step 1105: Try to close the third queue element by an atomic operation. If the closing setting is successful, go to step 1106; otherwise, go to step 1101.

[0258] In this embodiment, when the third queue element is not closed by the producer thread, the producer thread can also be in the third queue element and can also perform a closing operation on the third queue element. In order to ensure consistency, the first consumer thread tries to close the third queue element by an atomic operation, that is, sets the first indication bit to indicate closing. If successful, it means that the third queue element is closed, and the dequeue operation fails. If failed, it means that the producer thread is still in the third queue element and can still store data in the third queue element. Go to step 1101 to perform the dequeue operation again.

[0259] The closing process of step 1105 can refer to the description of closing the i-D1 queue element in Figure 7 , and the process is similar, which will not be described here.

[0260] Step 1106: Return dequeue failure.

[0261] It should be noted that in the process shown in Figure 11 , after the data reading is completed in step 1102, the position index can also not be added by 1, but after the position index in the consumer thread count value is acquired in step 1101, the position index is first added by 1 and then compared with the variable in the producer thread count value.

[0262] Figure 12 The process in which the first consumer thread acquires a new queue element in the lock-free queue is provided, which can be seen from steps 1201 to 1205.

[0263] Step 1201, call the outer multi-producer multi-consumer queue to get the index i corresponding to the new queue element, and determine whether the i-D2 queue element is consumed. If consumed, go to step 1202, otherwise go to step 1205.

[0264] In this embodiment, the outer multi-producer multi-consumer queue is called to get the index i corresponding to the new queue element, that is, the new queue element is the i-th queue element, which has not actually occupied i. Determine whether the i-D2 queue element indicating the completion of consumption is the first value. If it is the first value, it is determined that the i-D2 queue element is consumed, and step 1202 is executed, otherwise it is not consumed, and it is considered that the allocation of the new queue element fails, because only D2 queue elements can be dequeued at the same time. Here D2 = K / (2*NE), or D1*NE+D2*NE = K is satisfied.

[0265] It should be noted that if the outer multi-producer multi-consumer queue uses a limited queue or is called a ring queue, the i-D2 queue element may need to be wrapped, that is, (Tube_SIZE+i-D2) % Tube_SIZE, where Tube_SIZE is the maximum number of queue elements included in the outer multi-producer multi-consumer queue.

[0266] In this way, the number of simultaneously dequeued queue elements is additionally checked to avoid the degree of consumer disorder exceeding the requirement.

[0267] Step 1202, call the outer multi-producer multi-consumer queue to reserve the i-th queue element. If the reservation is successful, go to step 1203, otherwise go to step 1204.

[0268] In this embodiment, the reservation interface of the outer multi-producer multi-consumer queue is called to reserve the i-th queue element. For example, when the first consumer thread calls the reservation interface, input i, and determine whether the queue element at the head of the current consumer thread is i-1. If it is i-1, take i, and the reservation is successful, otherwise, i cannot be taken, and the reservation fails.

[0269] Step 1203, successfully get the new queue element, and return the new queue element.

[0270] Step 1204, determine whether the reservation fails due to the i-th queue element being consumed by other consumer threads. If so, go to step 1201, otherwise go to step 1205.

[0271] In the embodiment, after the reservation fails, it is judged whether the queue element where the current consumer thread head is located is i-1. If it is not i-1, it is determined that the acquisition fails due to that the i th queue element has been consumed by other consumer threads, and the reservation of the i th queue element can be re-applied. The remaining failure cases are considered as allocation failures, such as that the producer has not produced to cause the reservation failure.

[0272] In step 1205, the allocation failure is returned.

[0273] In the embodiment of the present disclosure, through simulation analysis, compared with other schemes, the average throughput rate of the de-queue and en-queue operations of the lock-free queue in the embodiment of the present disclosure is higher than that of other schemes in a period of time, which indicates that the performance is lower than that of other schemes.

[0274] The device provided by the embodiment of the present disclosure is introduced below.

[0275] Figure 13 is a structural diagram of the en-queue device of the lock-free queue provided by the embodiment of the present disclosure. The device can be realized as part or all of the device by software, hardware or a combination of both. The lock-free queue includes a plurality of queue elements, and each queue element has at least one producer thread and at least one consumer thread operating at the same time. The device provided by the embodiment of the present disclosure can realize the method provided by the embodiment of the present disclosure. Figure 3 to Figure 7 The flow, the device includes an en-queue module 1310 and an application module 1320, wherein:

[0276] The en-queue module 1310 is configured to, when a first producer thread deposits data to be en-queued into the lock-free queue, if the first producer thread corresponds to a first queue element in the lock-free queue, control the first producer thread to perform an en-queue operation in the first queue element. Specifically, the en-queue module 1310 can be configured to realize the en-queue function of step 301 and perform the implicit steps included in step 301.

[0277] The application module 1320 is configured to, if the first producer thread does not correspond to a queue element in the lock-free queue or the en-queue in the first queue element fails, allocate a second queue element for the first producer thread in the lock-free queue. Specifically, the application module 1320 can be configured to realize the allocation function of step 302 and perform the implicit steps included in step 302.

[0278] The en-queue module 1310 is further configured to control the first producer thread to perform an en-queue operation in the second queue element. Specifically, the en-queue module 1310 can be configured to realize the en-queue function of step 302 and perform the implicit steps included in step 302.

[0279] In an alternative implementation, each queue element is operated by at most one producer thread and one consumer thread at the same time, and the enqueuing module 1310 is configured to:

[0280] determine the state of the first queue element;

[0281] if the state of the first queue element is not closed, store the data to be enqueued into the storage space of the first queue element;

[0282] if the state of the first queue element is closed, determine that the first producer thread fails to enqueue in the first queue element.

[0283] In an alternative implementation, the enqueuing module 1310 is configured to:

[0284] read a producer thread count value corresponding to the first queue element, the producer thread count value being used to indicate the state of the first queue element and a position index of the storage space of the first queue element;

[0285] determine the state of the first queue element based on the producer thread count value;

[0286] The enqueuing module 1310 is configured to:

[0287] if the state of the first queue element is not closed, store the data to be enqueued into the position indicated by the position index.

[0288] In an alternative implementation, the enqueuing module 1310 is further configured to:

[0289] after storing the data to be enqueued into the storage space of the first queue element, if the storage space of the first queue element is fully occupied, determine whether the first queue element is consumed completely and whether the current consumer thread leaves the first queue element;

[0290] if the first queue element is consumed completely and the consumer thread leaves the first queue element, set the first producer thread as not corresponding to any queue element, and recycle the first queue element.

[0291] In an alternative implementation, the enqueuing module 1310 is further configured to:

[0292] after determining that the state of the first queue element is closed, determine whether the first queue element is consumed completely and whether the current consumer thread leaves the first queue element;

[0293] if the first queue element is consumed completely and the consumer thread leaves the first queue element, recycle the first queue element.

[0294] In an optional mode, each queue element has at most one producer thread and one consumer thread operating at the same time, and the application module 1320 is configured to:

[0295] In the lock-free queue, the first producer thread is assigned the i th queue element, and an enqueue-close operation is performed on the i-D1 th queue element, where D1 is the maximum number of queue elements that are simultaneously enqueued in the lock-free queue;

[0296] The i th queue element is initialized as the second queue element.

[0297] In an optional mode, the application module 1320 is configured to:

[0298] The producer thread count value of the i-D1 th queue element is read, and the state of the i-D1 th queue element is set to closed in the producer thread count value through an atomic operation.

[0299] In an optional mode, D1 = K / (2*NE), where K is a global out-of-order degree value of the lock-free queue, and NE is the storage space capacity of each queue element.

[0300] In an optional mode, each queue element is a lock-free queue, a lock-free stack, or a lock-free tree.

[0301] Figure 13 The device for enqueuing the lock-free queue shown in the figure performs the detailed process of the enqueue operation, which is described in the foregoing embodiments and will not be repeated here. Figure 13 The device for enqueuing the lock-free queue shown in the figure is attached to the computing device 100 in Figure 1 or the smart NIC in the foregoing or the like.

[0302] Figure 14 is a structural diagram of a device for dequeuing a lock-free queue provided by an embodiment of the present disclosure. The device can be implemented as part or all of the device by software, hardware, or a combination of both. The lock-free queue includes a plurality of queue elements, each of which has at most at least one producer thread and at least one consumer thread operating at the same time, and the device provided by an embodiment of the present disclosure can implement the flow described in Figure 8 、 Figure 9 、 Figure 11 and Figure 12 The device includes a dequeue module 1410 and an assignment module 1420, where:

[0303] The dequeuing module 1410 is configured to, when a first consumer thread reads data from the lock-free queue, if the first consumer thread corresponds to a third queue element in the lock-free queue, control the first consumer thread to perform a dequeuing operation in the third queue element, and can be specifically used for implementing the dequeuing function of step 801 and performing the implicit steps included in step 801.

[0304] The allocating module 1420 is configured to, if the first consumer thread does not correspond to a queue element in the lock-free queue or fails in the dequeuing operation in the third queue element, allocate a fourth queue element for the first consumer thread in the lock-free queue, and can be specifically used for implementing the allocating function of step 802 and performing the implicit steps included in step 802.

[0305] The dequeuing module 1410 is further configured to control the first consumer thread to perform a dequeuing operation in the fourth queue element, and can be specifically used for implementing the dequeuing function of step 802 and performing the implicit steps included in step 802.

[0306] In an optional manner, each queue element has at most one producer thread and one consumer thread operating at the same time, and the dequeuing module 1410 is configured to:

[0307] determine whether there is unread data in the third queue element;

[0308] if there is unread data in the third queue element, read the unread data;

[0309] if there is no unread data in the third queue element, determine whether the first consumer thread fails in the dequeuing operation in the third queue element based on the current state of the third queue element.

[0310] In an optional manner, the dequeuing module 1410 is configured to:

[0311] if the state of the third queue element is closed, determine that the first consumer thread fails in the dequeuing operation in the third queue element;

[0312] if the state of the third queue element is not closed, perform a closing operation on the third queue element, if the closing operation is successful, determine that the first consumer thread fails in the dequeuing operation in the third queue element, and if the closing operation fails, return to performing the process of determining whether there is unread data in the third queue element.

[0313] In an optional manner, the dequeuing module 1410 is further configured to:

[0314] After the closing operation is successfully executed, if the current producer thread leaves the third queue element, the third queue element is recycled.

[0315] In an optional mode, the dequeuing module 1410 is further configured to:

[0316] After the unread data is read, if the third queue element has no unread data, it is determined whether the current producer thread leaves the third queue element.

[0317] If the current producer thread leaves the third queue element, the first consumer thread is set as not corresponding to a queue element, and the third queue element is recycled.

[0318] In an optional mode, each queue element has at most one producer thread and one consumer thread operating at the same time, and the allocation module 1420 is further configured to:

[0319] Before the fourth queue element is allocated to the first consumer thread, it is determined that the number of times of searching for a queue element in a first direction by the current dequeuing is less than a first threshold value, the first direction being a direction in which the third queue element is towards a queue element at a head of the producer thread; or,

[0320] It is determined that there is an unconsumed queue element between the third queue element and a queue element at the head of the producer thread.

[0321] In an optional mode, each queue element has at most one producer thread and one consumer thread operating at the same time, and the allocation module 1420 is configured to:

[0322] If the i-D2 queue element is recycled, the operation of reserving the i queue element is executed, D2 being a maximum number of queue elements being dequeued at the same time in the lock-free queue.

[0323] If the operation of reserving the i queue element is successfully executed, the i queue element is determined as the fourth queue element.

[0324] In an optional mode, D2=K / (2*NE), K being a global out-of-order degree value of the lock-free queue, and NE being a storage space capacity of each queue element.

[0325] Figure 14 The detailed process of the lock-free queue dequeuing device in performing the dequeuing operation is described in the foregoing embodiments, and thus is not repeated here. Figure 14 The lock-free queue dequeuing device is applicable to Figure 1The computing device 100 in FIG. 1 or the smart network card in the foregoing or the like.

[0326] In some embodiments, a computer program product is provided, which includes computer instructions stored in a computer readable storage medium. A processor of a computing device reads the computer instructions from the computer readable storage medium, and the processor executes the computer instructions, so that the computing device performs the flow shown in FIG. Figure 3 to Figure 7 In some embodiments, a computer program product is provided, which includes computer instructions stored in a computer readable storage medium. A processor of a computing device reads the computer instructions from the computer readable storage medium, and the processor executes the computer instructions, so that the computing device performs the flow shown in FIG.

[0327] In some embodiments, a computer program product is provided, which includes computer instructions stored in a computer readable storage medium. A processor of a computing device reads the computer instructions from the computer readable storage medium, and the processor executes the computer instructions, so that the computing device performs the flow shown in FIG. Figure 8 Figure 9 Figure 11 Figure 12 In some embodiments, a computer program product is provided, which includes computer instructions stored in a computer readable storage medium. A processor of a computing device reads the computer instructions from the computer readable storage medium, and the processor executes the computer instructions, so that the computing device performs the flow shown in FIG.

[0328] Those skilled in the art can appreciate that, in combination with the method steps and units described in the embodiments disclosed in the present disclosure, the methods can be implemented in electronic hardware, computer software or a combination of both. In order to clearly illustrate the interchangeability of hardware and software, the steps and components of the embodiments have been described in the above description. Whether the functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present disclosure.

[0329] In several embodiments provided in the present disclosure, it should be understood that the disclosed system architecture, device and method can be implemented in other ways. For example, the above-described device embodiments are only schematic. For example, the division of the modules is only a logical function division. In actual implementation, another division mode can be used, for example, a plurality of modules or components can be combined or integrated into another system, or some features can be omitted or not executed. In addition, the coupling or direct coupling or communication connection between the displayed or discussed ones can be indirect coupling or communication connection through some interfaces, devices or modules, and can also be electrical, mechanical or other forms of connection.

[0330] The modules described as separate components can or can not be physically separated, and the components shown as modules can or can not be physical modules, i.e. can be located in one place, or can be distributed to a plurality of network modules. Part or all of the modules can be selected according to actual needs to achieve the purpose of the embodiments of the present disclosure. ​​​

[0331] In addition, each module in various embodiments of the present disclosure can be integrated in one processing module, or each module can exist physically separately, or two or more modules can be integrated in one module. The integrated module can be realized in the form of hardware or in the form of a software module.

[0332] When the integrated module is realized in the form of a software function module and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present disclosure essentially or the part that contributes to the prior art, or all or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes a number of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the methods in various embodiments of the present disclosure. The foregoing storage medium includes: a U disk, a mobile hard disk, a ROM, a RAM, a magnetic disk or an optical disk, and various media that can store program codes.

[0333] In the present disclosure, the terms "first" and "second" and the like are used to distinguish between the same or similar items with substantially the same function and action, and it should be understood that there is no logical or time sequence between "first" and "second", and the quantity and execution order are not limited. It should also be understood that although the following description uses the terms "first" and "second" and the like to describe various elements, these elements should not be limited by the terms. These terms are only used to distinguish one element from another. For example, without departing from the scope of various examples, a first queue element can be referred to as a second queue element, and similarly, a second queue element can be referred to as a first queue element. The first queue element and the second queue element can both be queue elements, and in some cases, can be separate and different queue elements.

[0334] In the present disclosure, the term "at least one" means one or more, and the term "multiple" in the present disclosure means two or more.

[0335] The above description is only exemplary embodiments of the present disclosure, but the protection scope of the present disclosure is not limited thereto. Any skilled person in the art can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present disclosure, and these modifications or replacements should be covered in the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be subject to the protection scope of the claims.

Claims

1. A method for enqueuing in a lock-free queue, characterized in that, The lock-free queue includes a plurality of queue elements, each queue element being operated by at most one producer thread and at most one consumer thread at the same time, and the method comprises: When a first producer thread deposits data to be queued into the lock-free queue, if the first producer thread corresponds to a first queue element in the lock-free queue, the first producer thread is controlled to perform a queue-in operation in the first queue element; If the first producer thread does not correspond to a queue element in the lock-free queue or the queue-in operation in the first queue element fails, a second queue element is allocated for the first producer thread in the lock-free queue, and the first producer thread is controlled to perform a queue-in operation in the second queue element.

2. The method of claim 1, wherein, Each queue element is operated by at most one producer thread and at most one consumer thread at the same time, and the control of the first producer thread to perform a queue-in operation in the first queue element comprises: determining a state of the first queue element; if the state of the first queue element is not closed, storing the data to be queued into a storage space of the first queue element; if the state of the first queue element is closed, determining that the first producer thread fails to queue in the first queue element.

3. The method of claim 2, wherein, The determination of the state of the first queue element comprises: reading a producer thread count value corresponding to the first queue element, the producer thread count value being used to indicate the state of the first queue element and a position index of the storage space of the first queue element; based on the producer thread count value, determining the state of the first queue element; if the state of the first queue element is not closed, storing the data to be queued into the storage space of the first queue element, comprises: if the state of the first queue element is not closed, storing the data to be queued into a position indicated by the position index.

4. The method according to claim 2 or 3, characterized in that, After the data to be queued is stored into the storage space of the first queue element, further comprising: if the storage space of the first queue element is fully occupied, determining whether the first queue element is consumed completely and whether a current consumer thread leaves the first queue element; if the first queue element is consumed completely and the consumer thread leaves the first queue element, setting the first producer thread as not corresponding to a queue element and recycling the first queue element.

5. The method according to any one of claims 2 to 4, characterized in that, After determining that the state of the first queue element is closed, further comprising: determining whether the first queue element is consumed completely and whether a current consumer thread leaves the first queue element; if the first queue element is consumed completely and the consumer thread leaves the first queue element, recycling the first queue element.

6. The method according to any one of claims 1 to 5, characterized in that, Each queue element is operated by at most one producer thread and at most one consumer thread at the same time, and the allocation of the second queue element for the first producer thread in the lock-free queue comprises: In the lock-free queue, an i-th queue element is allocated to the first producer thread, and a close-in operation is performed on an i-D1-th queue element, where D1 is a maximum number of queue elements simultaneously enqueued in the lock-free queue; The i-th queue element is initialized as the second queue element.

7. The method of claim 6, wherein, The close-in operation performed on the i-D1-th queue element includes: A producer thread count value of the i-D1-th queue element is read, and a state of the i-D1-th queue element is set to close in the producer thread count value through an atomic operation.

8. The method according to claim 6 or 7, characterized in that, D1 = K / (2*NE), where K is a global out-of-order degree value of the lock-free queue, and NE is a storage space capacity of each queue element.

9. The method according to any one of claims 1 to 8, characterized in that, Each queue element is a lock-free queue, a lock-free stack, or a lock-free tree.

10. A method for dequeuing a lock-free queue, the method comprising: The lock-free queue includes a plurality of queue elements, each of which is operated by at most one producer thread and at least one consumer thread at the same time, and the method includes: When a first consumer thread reads data from the lock-free queue, if the first consumer thread corresponds to a third queue element in the lock-free queue, the first consumer thread is controlled to perform a dequeue operation in the third queue element; If the first consumer thread does not correspond to a queue element in the lock-free queue or fails to dequeue in the third queue element, a fourth queue element is allocated to the first consumer thread in the lock-free queue, and the first consumer thread is controlled to perform a dequeue operation in the fourth queue element.

11. The method of claim 10, wherein, Each queue element is operated by at most one producer thread and one consumer thread at the same time, and the control of the first consumer thread to perform a dequeue operation in the third queue element includes: determining whether there is unread data in the third queue element; if there is unread data in the third queue element, reading the unread data; if there is no unread data in the third queue element, determining whether the first consumer thread fails to dequeue in the third queue element based on a current state of the third queue element.

12. The method of claim 11, wherein, The determination of whether the first consumer thread fails to dequeue in the third queue element based on the current state of the third queue element includes: if the state of the third queue element is close, it is determined that the dequeue in the third queue element fails; if the state of the third queue element is not close, a close operation is performed on the third queue element, if the close operation is successful, it is determined that the dequeue in the third queue element fails, and if the close operation fails, the process of determining whether there is unread data in the third queue element is returned.

13. The method of claim 12, wherein, After the close operation is successful, it further includes: if the current producer thread leaves the third queue element, the third queue element is recycled.

14. The method according to any one of claims 11 to 13, characterized in that, After the unread data is read, it further includes: if there is no unread data in the third queue element, it is determined whether the current producer thread leaves the third queue element; If the current producer thread leaves the third queue element, the first consumer thread is set to not correspond to a queue element, and the third queue element is recycled.

15. The method according to any one of claims 11 to 14, characterized in that, Each queue element is operated on by at most one producer thread and one consumer thread at the same time, and before the fourth queue element is allocated to the first consumer thread, further comprising: determining that the number of queue elements searched in the first direction by the dequeueing this time is less than a first threshold value, the first direction being a direction in which the third queue element is toward a queue element in which a head of the producer thread is located; or determining that there is an unconsumed queue element between the third queue element and the queue element in which the head of the producer thread is located.

16. The method according to any one of claims 11 to 15, characterized in that, Each queue element is operated on by at most one producer thread and one consumer thread at the same time, and the fourth queue element is allocated to the first consumer thread, comprising: if the i-th queue element is the current queue element to be allocated, and if the i-D2-th queue element is completed recycling, performing an operation of reserving the i-th queue element, D2 being a maximum number of queue elements that are simultaneously dequeued in the lock-free queue; if the operation of reserving the i-th queue element succeeds, determining the i-th queue element as the fourth queue element.

17. The method of claim 16, wherein, D2=K / (2*NE), K being a global out-of-order degree value of the lock-free queue, and NE being a storage space capacity of each queue element.

18. A multi-producer multi-consumer lock-free queue, characterized by The lock-free queue comprises a plurality of queue elements, each queue element being operated on by at most one producer thread and one consumer thread at the same time.

19. The lock-free queue of claim 18, wherein, A number of queue elements that simultaneously perform enqueueing operations in the lock-free queue is a first number value, the first number value being equal to K / (2*NE), NE being equal to a storage space capacity of each queue element, and K being a global out-of-order degree value of the lock-free queue.

20. The lock-free queue according to claim 18 or 19, c h a r a c t e r i z e d b y A number of queue elements that simultaneously perform dequeueing operations in the lock-free queue is a second number value, the second number value being equal to K / (2*NE), NE being equal to a storage space capacity of each queue element, and K being a global out-of-order degree value of the lock-free queue.

21. The lock-free queue according to any of claims 18 to 20, characterized in that, Each queue element is a lock-free queue, a lock-free stack, or a lock-free tree.

22. An apparatus for enqueuing in a lock-free queue, the apparatus comprising: The lock-free queue comprises a plurality of queue elements, each queue element being operated on by at most one producer thread and one consumer thread at the same time, and the apparatus comprises: an enqueuing module configured to, when a first producer thread stores data to be enqueued in the lock-free queue, if the first producer thread corresponds to a first queue element in the lock-free queue, control the first producer thread to perform an enqueueing operation in the first queue element; an applying module configured to, if the first producer thread does not correspond to a queue element in the lock-free queue or the enqueueing in the first queue element fails, allocate a second queue element to the first producer thread in the lock-free queue; the enqueuing module is further configured to control the first producer thread to perform an enqueueing operation in the second queue element.

23. An apparatus for dequeuing a lock-free queue, the apparatus comprising: The lock-free queue includes a plurality of queue elements, each queue element being operated on by at most one producer thread and at most one consumer thread at the same time, and the apparatus comprises: an enqueuing module configured to, when a first producer thread writes data into the lock-free queue, control the first producer thread to perform an enqueuing operation in a third queue element corresponding to the first producer thread in the lock-free queue; an allocating module configured to, if the first producer thread does not correspond to a queue element in the lock-free queue or the enqueuing operation in the third queue element fails, allocate a fourth queue element for the first producer thread in the lock-free queue; the enqueuing module is further configured to control the first producer thread to perform an enqueuing operation in the fourth queue element.

24. A computing device, comprising: comprising a processor and a memory; the processor is configured to execute instructions stored in the memory to cause the computing device to perform the method of any one of claims 1 to 9.

25. A computing device, comprising: comprising a processor and a memory; the processor is configured to execute instructions stored in the memory to cause the computing device to perform the method of any one of claims 10 to 17.

26. A computer program product comprising instructions, wherein: when the instructions are run by a computing device, cause the computing device to perform the method of any one of claims 1 to 9.

27. A computer program product comprising instructions, wherein: when the instructions are run by a computing device, cause the computing device to perform the method of any one of claims 10 to 17.

28. A computer-readable storage medium, characterized in that, comprising computer program instructions which, when executed by a computing device, cause the computing device to perform the method of any one of claims 1 to 9.

29. A computer-readable storage medium, characterized in that, comprising computer program instructions which, when executed by a computing device, cause the computing device to perform the method of any one of claims 10 to 17.

Images