Lock-free round-robin queue capacity expansion method and lock-free round-robin queue capable of dynamically expanding capacity
By dynamically expanding index memory and data memory when the lock-free loop queue is full, the lag and memory waste problems of traditional lock-free loop queues during peak data processing are solved, and efficient multi-reader concurrent access and memory management are achieved.
Patent Information
- Application Number
- CN202411521024.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-29
- Publication Date
- 2025-02-25
AI Technical Summary
Traditional lock-free loop queues are prone to system lag or memory waste due to fixed size when processing peak data, and most existing technologies do not support dynamic expansion, resulting in poor performance.
It provides a lock-free circular queue expansion method, which dynamically expands index memory and data memory when the queue is full, supports concurrent access for multiple readers, and reduces capacity when the load is reduced to save memory.
It realizes lock-free one-write and multiple read support, improves performance, supports dynamic expansion and shrinkage, effectively utilizes memory resources, and ensures the security and high efficiency of reading data.
Smart Images
Figure CN119512780A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of data processing, and in particular to a method for expanding a lock-free circular queue and a lock-free circular queue capable of dynamic expansion. Background Art
[0002] In high-performance financial trading systems, multi-threaded models are usually used for various IO operations and business processing. Lock-free circular queue (hereinafter referred to as RB) is a data structure used for inter-thread communication. Through specific technologies and algorithms, data can be efficiently transferred between threads without using locks. The traditional RB has a fixed length, and access rights are controlled by moving the read and write index forward, and it is used cyclically.
[0003] The initialization size of traditional RB needs to be carefully evaluated according to the specific operating environment and data load. If it is set too small, the system will be stuck because it cannot cache enough data when peak data arrives; if it is set too large, it will cause a huge waste of memory. In different scenarios, peak traffic varies and it is difficult to accurately evaluate.
[0004] Most of the mainstream RBs on the market do not support dynamic expansion, such as spsc_queue of boost c++ library, rte_ring of dpdk, MPMCQueue of libcds, etc. These circular queues rely on fixed-size buffers or pre-allocated memory to access data through indexes. If you want to support dynamic expansion, you need to develop it again. The way to support expansion is generally to lock the data as a whole, which has poor performance. Summary of the invention
[0005] In view of this, the present invention provides a method for expanding a lock-free circular queue and a lock-free circular queue that can be expanded dynamically. The initialization size of the queue only needs to be evaluated according to normal traffic. After the peak data arrives and causes the queue to be full, the queue can be dynamically expanded; after the peak, when the pressure is low, the queue can be reduced to the initial value to save memory.
[0006] To this end, the present invention provides the following technical solutions:
[0007] The present invention discloses a method for expanding a lock-free circular queue, comprising:
[0008] Applying for a first index memory with a fixed capacity and a first data memory for storing data; the first index memory uses a lock-free circular queue to store a pointer to the corresponding data storage; the lock-free circular queue has a read index and a write index, and the movement of the read and write indexes of the lock-free circular queue is controlled by a data writing thread;
[0009] When the lock-free circular queue has no writable position, the data writing thread expands the index memory and the data memory;
[0010] The expansion of the index memory includes: applying for a second index memory; the capacity of the second index memory is twice the capacity of the first index memory; relocating and copying all data in the first index memory to the second index memory, and ensuring that the data items copied from the first index memory can be correctly accessed in the second index memory according to the read-write index;
[0011] The expansion of the data memory includes: applying for a second data memory and associating it with an index memory in the second index memory that does not store data items.
[0012] Furthermore, it also includes:
[0013] Determine whether all data reading threads no longer access the first index memory;
[0014] If yes, delete the first index memory.
[0015] Further, determining whether all data reading threads no longer access the first index memory includes:
[0016] Check periodically or during the next capacity expansion to make sure that all data reading threads no longer access the first index memory.
[0017] Furthermore, the lock-free circular queue has no writable position, including:
[0018] The data writing thread calculates the writable position through the read and write indexes and the current capacity of the lock-free circular queue.
[0019] Furthermore, the number of capacity expansions does not exceed a preset maximum number of capacity expansions.
[0020] Furthermore, it also includes: when the difference between the read and write positions of the lock-free circular queue is reduced and is smaller than an initial value, the write data thread shrinks the index memory and the data memory.
[0021] Furthermore, it supports multiple data reading threads to read the full data without affecting each other, and each data reading thread maintains its own reading index;
[0022] The read index of each data reading thread is less than or equal to the write index of the lock-free circular queue, and greater than or equal to the read index of the lock-free circular queue.
[0023] Furthermore, it supports dynamic addition and removal of multiple data reading threads.
[0024] Furthermore, the capacity of the lock-free circular queue is a power of 2.
[0025] The present invention also provides a lock-free circular queue that can be dynamically expanded, and the lock-free circular queue is dynamically expanded using the expansion method of the lock-free circular queue.
[0026] Advantages and positive effects of the present invention: The present invention realizes lock-free one-write-multiple-read support, supports multiple readers to read the full amount of data, and multiple readers are unaware of each other. The present invention, through the design idea of separating the index memory from the data memory, on the one hand increases the cache hit rate to improve performance, and on the other hand supports data memory reuse during capacity expansion, thereby speeding up capacity expansion. The writer of the present invention will not block the reader who is reading the data during the capacity expansion process, thereby ensuring the safety and high efficiency of reading data. According to tests, under the same other conditions, the capacity expansion performance of the present invention is more than twice the capacity expansion speed of the stl vector of the same capacity. BRIEF DESCRIPTION OF THE DRAWINGS
[0027] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.
[0028] Figure 1 A schematic diagram of a circular queue read index and write index in an embodiment of the present invention;
[0029] Figure 2 A schematic diagram of a read index and a write index of a circular queue maintained by each thread in an embodiment of the present invention;
[0030] Figure 3 A schematic diagram of a data memory and an index memory in an embodiment of the present invention;
[0031] Figure 4 A schematic diagram of a full state of a circular queue in an embodiment of the present invention;
[0032] Figure 5 A schematic diagram of a memory layout of data copying after the circular queue is expanded in an embodiment of the present invention;
[0033] Figure 6 It is another memory layout schematic diagram of data copy after the circular queue is expanded in an embodiment of the present invention;
[0034] Figure 7 A schematic diagram of a data memory newly applied for in an embodiment of the present invention;
[0035] Figure 8 This is a schematic diagram of a data memory newly applied in an embodiment of the present invention. DETAILED DESCRIPTION
[0036] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.
[0037] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.
[0038] A method for expanding a lock-free circular queue in an embodiment of the present invention specifically includes the following steps:
[0039] S1. Apply for a first index memory with a fixed capacity and a first data memory for storing data; the first index memory uses a lock-free circular queue to store a pointer to the corresponding data storage;
[0040] Use read index and write index to quickly locate and control the data reading and writing of the queue. Read index and write index are natural numbers and can only increase monotonically. By constraining the capacity of the queue to a power of 2, the computer can quickly calculate the actual read and write offset in the queue based on the read and write index, and use the queue in a loop.
[0041] like Figure 1 The circular queue with a capacity of 8 is shown, and the actual positions to be read and written correspond to the scenario in which the read index r_idx of the queue is 9 and the write index w_idx is 15.
[0042] It supports multiple data reading threads (hereinafter referred to as readers) to read the full data without affecting each other. Each reader maintains its own reading index. It supports dynamic addition and removal of readers. The movement of queue reading and writing indexes is controlled by the data writing thread (hereinafter referred to as writer).
[0043] like Figure 2As shown, the writer maintains the write and read indexes (w_idx / r_idx) of the queue; readers maintain their own read indexes: reader 1 maintains its own read index r1_idx, and reader 2 maintains its own read index r2_idx. The read index of each reader is less than or equal to the write index w_idx of the queue, and greater than or equal to the read index r_idx of the queue.
[0044] After writing data, the writer moves the write index; after reading data, the reader moves its own read index. Use memory order-related technologies to ensure read and write order and visibility, avoid using locks, and improve performance.
[0045] In an embodiment of the present invention, a circular queue is designed to be separated from data storage. The actual data is no longer stored in the circular queue, and only a pointer to the corresponding data storage is retained. At this time, the circular queue is called an index memory, and the actual data storage is called a data memory. The data memory actually pointed to by the index memory is located through the read and write index of the queue to achieve true reading and writing of the queue. Through this design, the continuity of the queue is retained. At the same time, the data memory is a large block memory for actually storing information, and there is no requirement for the continuity of the memory between the data memory blocks.
[0046] like Figure 3 As shown, the index memory stores the data memory pointer, which points to the data memory. Generally speaking, the memory space occupied by a data block in the index memory is the pointer length, which is usually 8 bytes. The memory space occupied by a data block in the data memory is generally much larger than the memory space occupied by a data block in the index memory, for example, 1024 bytes.
[0047] S2. When there is no writable position in the lock-free circular queue, the writer expands the index memory and the data memory;
[0048] When the queue is "full" and there is no writable position, the writer will perform the expansion operation. The read index and write index of the queue are controlled by the writer. When the writer calculates the write data based on the read and write index of the queue and the current capacity, if there is no writable position, the queue is in the "full" state. The writer calculates and updates the read index of the queue based on the smallest read index of all readers. If there is still no writable space after the update, the queue is truly "full" and needs to be expanded.
[0049] For example, Figure 4 As shown in the figure, if the current queue r_idx = 2, queue w_idx = 10, the queue is full, and the two readers of the queue r1_idx = 2, r2_idx = 2. The writer will fail to move the queue r_idx forward. At this time, the circular queue is truly full and will attempt to expand.
[0050] S21, the expansion of the index memory includes: applying for a second index memory; the capacity of the second index memory is twice the capacity of the first index memory; relocating and copying all data in the first index memory to the second index memory, and ensuring that the data items copied from the first index memory can be correctly accessed in the second index memory according to the read-write index;
[0051] When the writer expands capacity, first apply for twice the index memory as the new index memory. After applying for the new index memory, all data in the old index memory needs to be relocated and copied to the new index memory, and it needs to be ensured that the data items copied from the old index memory can be correctly accessed in the new index memory according to the read-write index.
[0052] For example, Figure 5 , 6 As shown in the figure, when the index memory is expanded to twice its original size, in order to ensure that the original index data can still be correctly addressed through w_idx / r_idx in the new index memory, the index memory before the expansion needs to be copied to the new index memory in a certain order. Since the length of the index memory after expansion is twice that before expansion, there are only two cases for the memory layout of data copy, which are explained separately. Figure 5 , Figure 6 In the example, the circular queue capacity before expansion is n=8, and the circular queue capacity after expansion is 2n=16:
[0053] Case ①: If w_idx = 10, r_idx = 2, the capacity is expanded, and the old index memory is copied to the new index memory (dark yellow is copied from the old index memory):
[0054] Case ②: If w_idx=18, r_idx=10, the capacity is expanded and the old index memory is copied to the new index memory (dark yellow is copied from the old index memory).
[0055] S22, the expansion of the data memory includes: applying for a second data memory and associating it with the index memory of the second index memory that does not store data items
[0056] After executing the previous step, the new index memory has been filled with half of the valid data, reusing the existing data memory. The remaining half of the data memory needs to be newly allocated and associated with the other half of the index memory.
[0057] The schematic diagram of the newly applied data memory is as follows Figure 7 , Figure 8 As shown:
[0058] Case ①: If w_idx = 10 and r_idx = 2, the capacity is expanded and the new data memory is as follows: Figure 7As shown: the blue one is the newly applied data memory. The index memory in the newly applied index memory that is not copied from the old index memory needs to be associated with the newly applied data memory.
[0059] Case ②: If w_idx = 18, r_idx = 10, the capacity is expanded, and the new data memory is as follows Figure 8 As shown: the blue one is the newly applied data memory. The index memory in the newly applied index memory that is not copied from the old index memory needs to be associated with the newly applied data memory.
[0060] The entire expansion minimizes the shared data between readers and writers to the size of two long integers, namely the new index memory address and the circular queue capacity. With a high-performance lightweight lock for one-write-multiple-read scenarios, it achieves a "seamless" switch to access the new queue after expansion.
[0061] The maximum expansion times of the circular queue can be set to prevent the queue from expanding indefinitely and ensure the stable operation of the system.
[0062] In another embodiment, after the expansion, the following step is further performed: delaying the release of the recycled circular queue index memory, because the index memory before the expansion may still be accessed by readers, and therefore the release needs to be delayed. The old index memory can be deleted only after all readers no longer access the old index memory. The timing for attempting to release the index memory before the expansion can be a scheduled check or a check at the next expansion.
[0063] In another embodiment, when the queue read-write position gap is reduced and is smaller than the initial value, the writer will perform memory shrinkage to reduce the capacity of the circular queue to the initial value and release memory.
[0064] In another embodiment, a lock-free circular queue that can be dynamically expanded can be obtained by using the expansion method of the lock-free circular queue obtained in the above embodiment to perform dynamic expansion.
[0065] The capacity doubles after each expansion, and the queue capacity remains a power of 2. The computer can quickly calculate the actual read and write positions of the read and write indexes in the new queue and use the queue in a circular manner.
[0066] The present invention can greatly simplify the configuration and application complexity of RB. RB can adapt to complex data high and low pressure change scenarios and make full and effective use of memory resources. In the design of financial trading systems and multi-threaded communication systems, it is usually necessary to efficiently realize multi-threaded data sharing. For systems using lock-free circular queues, the method designed by the present invention can be used.
[0067] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.
Claims
1. A method for expanding a lock-free circular queue, characterized in that: include: Applying for a first index memory with a fixed capacity and a first data memory for storing data; The first index memory uses a lock-free circular queue to store pointers to corresponding data storage; The lock-free circular queue has a read index and a write index, and the movement of the read and write indexes of the lock-free circular queue is controlled by a write data thread; When the lock-free circular queue has no writable position, the data writing thread expands the index memory and the data memory; The expansion of the index memory includes: applying for a second index memory; the capacity of the second index memory is twice the capacity of the first index memory; relocating and copying all data in the first index memory to the second index memory, and ensuring that the data items copied from the first index memory can be correctly accessed in the second index memory according to the read-write index; The expansion of the data memory includes: applying for a second data memory and associating it with an index memory in the second index memory that does not store data items.
2. The method for expanding a lock-free circular queue according to claim 1, characterized in that: Also includes: Determine whether all data reading threads no longer access the first index memory; If yes, delete the first index memory.
3. The method for expanding a lock-free circular queue according to claim 2, characterized in that: Determine whether all data reading threads no longer access the first index memory, including: Check periodically or during the next capacity expansion to make sure that all data reading threads no longer access the first index memory.
4. The method for expanding a lock-free circular queue according to claim 1, characterized in that: The lock-free circular queue has no writable position, including: The data writing thread calculates the writable position through the read and write indexes and the current capacity of the lock-free circular queue.
5. The method for expanding a lock-free circular queue according to claim 1, characterized in that: The number of expansions does not exceed the preset maximum number of expansions.
6. The method for expanding a lock-free circular queue according to claim 1, characterized in that: Also includes: When the gap between the read and write positions of the lock-free circular queue is reduced and is smaller than the initial value, the data writing thread shrinks the index memory and the data memory.
7. The method for expanding a lock-free circular queue according to claim 1, characterized in that: Supports multiple data reading threads to read the full data without affecting each other, and each data reading thread maintains its own reading index; The read index of each data reading thread is less than or equal to the write index of the lock-free circular queue, and greater than or equal to the read index of the lock-free circular queue.
8. The method for expanding a lock-free circular queue according to claim 1, characterized in that: Supports dynamic addition and removal of multiple data reading threads.
9. The method for expanding a lock-free circular queue according to claim 1, characterized in that: The capacity of the lock-free circular queue is a power of 2.
10. A lock-free circular queue with dynamic expansion, characterized in that: The lock-free circular queue is dynamically expanded using the expansion method of the lock-free circular queue as described in any one of claims 1 to 9.