Improved method of one-write multi-read high concurrency lock-free ring buffer
By employing a lock-free circular cache with shared memory to enable inter-process communication in the stock exchange market data generation system, the system reliability problem caused by concurrent read and write operations of multiple processes is solved, achieving security and stability in high-concurrency processing and reducing market data latency.
Patent Information
- Application Number
- CN202210872713.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-07-21
- Publication Date
- 2026-02-17
- Estimated Expiration
- 2042-07-21
AI Technical Summary
In stock exchange market data generation systems, when multiple processes are concurrently reading and writing, abnormal failures can affect normally running threads, leading to reduced system reliability. Existing technologies such as message queues and sockets introduce complexity or increase latency.
A shared memory is used as the message bus for inter-process communication. A high-concurrency lock-free circular cache with one-write and multiple-read capabilities is designed. Data sharing and interaction are achieved through the shared memory region. The cache structure is stored in the shared memory and supports high-concurrency processing of multiple processes.
It improves the security, reliability, and stability of the market data generation system, reduces market data latency, and enhances the system's usability and scalability.
Smart Images
Figure CN115344402B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of software caching technology, specifically an improved method for high-concurrency lock-free circular caching with one-write-many-read capability. Background Technology
[0002] The securities market is a crucial component of the capital market. Major stock exchanges, as the central hubs of the securities market, provide an open, fair, and secure trading platform. The real-time, efficient, and accurate nature of market data ensures the high quality of the trading platform and constantly influences investors' strategy choices and behaviors. Therefore, major stock exchanges are continuously developing market data generation systems with low latency, high throughput, strong scalability, and high availability.
[0003] In many software systems, including market data generation systems, memory read / write operations are typically faster than data computation and processing. To accommodate this speed difference, caching mechanisms are often employed to provide a buffer for data computation and processing. While using multithreading to process data concurrently can improve performance, a drawback is that if one concurrent thread experiences a failure and the entire process needs to be restarted to restore service, it will affect other normally functioning threads. This is especially problematic if these concurrent threads correspond to different business processes; a failure in one process can impact other normally functioning processes, leading to a decrease in system reliability. This is something that market data generation systems must avoid in their architectural design.
[0004] Methods for implementing inter-process communication (IPC) typically include message queues and sockets. By using IPC to transmit data, different business processes can be isolated and run independently. However, introducing message queue components reduces system availability and increases system complexity; using sockets indirectly increases the length of market data transmission links, increasing the latency of market data transmission. Summary of the Invention
[0005] The purpose of this invention is to solve the "one write, many reads" scenario of concurrent read and write in multiple processes. Based on the invention of high-concurrency lock-free circular cache for one write and many reads, this invention is modified and optimized for multi-process scenarios, and can support the needs of high-concurrency processing of multiple processes.
[0006] To achieve the above objectives, an improved method for high-concurrency lock-free circular cache with one-write-multiple-read operation is designed. This method uses shared memory to realize inter-process communication, treating the shared memory as a message bus. Different processes can access the same memory area to achieve data sharing and interaction. The cache structure is stored in the shared memory, and in multi-read scenarios, the data in the cache can be mapped to different processes for processing.
[0007] Preferred method: The method is as follows:
[0008] S1. Fixed-length shared memory object. The data content in the object is stored page by page. When adding or looking up a record, the page index is found first, and then the position of the corresponding node is obtained through the offset. When a page fault occurs, the size of the shared memory increases dynamically according to the length of the page until the shared memory capacity is exhausted.
[0009] S2. The structure of a ring cache based on shared memory involves pre-allocating and holding a block of shared memory with 2... n A fixed-length shared memory object consisting of (2 to the power of n, where n is a positive integer) + 1 nodes, numbered from 0 to 2... n Nodes with a value of -1 are called storage nodes. Each storage node consists of a private information field and a data field. The private information field includes the index value of the currently stored data and the actual length of the stored data, and is numbered 2. n The nodes are called antenna nodes, these 2 n +1 nodes form a circular cache;
[0010] S3. Tentacle Node: The tentacles store the private information of the latest written storage node. The information in the tentacles is updated using the CAS method.
[0011] S4. Interface design of a shared memory-based circular cache: The shared memory-based circular cache can store any data shorter than the length of a shared memory object node. It provides two interfaces for data operations: read and write. In the read interface, the index value of the node to be read is compared with the latest data index value in the tentacles node. The number of the storage node to be read is then located using a mask. Before and after reading data, the index value stored in the storage node to be read is verified to confirm the read data or return a read failure. In the write interface, the index of the storage node to be written is obtained, located using the circular cache mask, and the data is written to the storage node. Then, the index value to be written and the private information in the tentacles node are updated.
[0012] S5. Concurrency Implementation Strategy Based on Shared Memory Ring Cache: The shared memory-based ring cache prioritizes write operations. When there is data in the node to be written, the old data will be overwritten.
[0013] Preferably, step S2 further includes: applying for and maintaining an array pointing to the location of shared memory object storage nodes to reduce the time spent reading and writing storage node data, and recording the mask of the circular cache to facilitate locating the storage node number by index.
[0014] Preferably, step S4 further includes: providing two initialization methods: default initialization and loading existing data. In the default initialization interface, all storage nodes are cleared, and the private information of the 2n-1th node is stored in the antenna node. The loading mode can import information from all storage nodes and antenna nodes in a circular cache of the same length and size.
[0015] Compared with existing technologies, the advantages of this invention are as follows: This invention utilizes shared memory to achieve inter-process communication, treating shared memory as a message bus. Different processes can access the same memory area, thereby achieving data sharing and interaction. By storing the cache structure in shared memory, in multi-read scenarios, the data in the cache can be mapped to different processes for processing. Through the application of a high-concurrency, lock-free circular cache in shared memory, the processing speed of market data can be accelerated, market data latency reduced, and the usability and scalability of the market data generation system improved, all while ensuring the security, reliability, and stability of the market data generation system. Attached Figure Description
[0016] Figure 1 This is a flowchart illustrating the circular cache of the present invention during single-process write and multi-process read operations.
[0017] Figure 2 This invention provides a process for handling circular cache writes in a multi-threaded environment. Detailed Implementation
[0018] An improved method for high-concurrency, lock-free circular cache with one-write-many-read operation includes the following parts:
[0019] A. Fixed-length shared memory objects
[0020] Fixed-length shared memory objects are shared memory containers where the length of each node is fixed at creation time. Data within the object is stored page by page, with each page requiring pre-allocation of space. When adding / searching for a record, the page index is first located, and then the position of the corresponding node is obtained using the offset. In the event of a page fault, the size dynamically increases by the page length until the shared memory capacity is exhausted.
[0021] B. The structure of a shared memory-based ring cache
[0022] Pre-allocate and hold a block of shared memory with 2... n A fixed-length shared memory object consisting of (2 to the power of n, where n is a positive integer) + 1 nodes, numbered from 0 to 2... n Nodes with a value of -1 are called storage nodes. Each storage node consists of a private information field and a data field. The private information field includes the index value of the currently stored data and the actual length of the stored data, and is numbered 2. n The nodes are called antenna nodes, these 2 n+1 nodes form a circular cache, with the structure as follows: Figure 1 The circular cache structure is shown in the figure.
[0023] In addition, an array pointing to the locations of shared memory object storage nodes is allocated and maintained to reduce the time spent reading and writing storage node data. A mask for the circular cache is recorded to facilitate locating storage node numbers via indexes.
[0024] C. Antennae
[0025] The tentacles store the private information of the most recently written storage node, and the information in the tentacles is updated using the CAS method. When calling the read interface, the index value of the most recently stored data is obtained from the tentacles. This avoids the need to set up critical sections in the storage nodes to prevent conflicts when concurrently calling the read and write interfaces, as is done in a circular cache, making the circular cache lock-free.
[0026] D. Interface design of a ring cache based on shared memory
[0027] A shared memory-based circular cache can store any data shorter than the length of a shared memory object node, and provides both read and write interfaces for data operations.
[0028] In the read interface, the index value of the node to be read is compared with the latest data index value in the antenna node, and the number of the storage node to be read is located by means of a mask. Before and after reading the data, the index value stored in the storage node to be read is checked to confirm the data to be read or to return the read failure.
[0029] In the write interface, the index of the storage node to be written is obtained, the data is located and written to the storage node through the ring cache mask, and then the index value to be written and the private information in the antenna node are updated.
[0030] In addition, two initialization methods are provided: default initialization and loading existing data. In the default initialization interface, all storage nodes are cleared, and the second node is stored in the antenna node. n -1 node's private information; the loading mode can import information from all storage nodes and antenna nodes in a circular cache of the same length and size.
[0031] E. Concurrency Implementation Strategy Based on Shared Memory Ring Cache
[0032] A shared-memory-based circular cache prioritizes write operations; if data already exists in the node to be written to, the old data will be overwritten. During system operation... n Each storage node remains readable and writable. Because the write interface updates private information first and then updates the stored data, the verification of the private data of the storage node before and after reading data ensures the concurrency and accuracy of multiple read processes and their several child threads reading data.
[0033] In the market data transmission chain, a large amount of market data is transmitted from system to system. For example, the data pushed from the market data source to the market data generation system is compressed and encoded. The market data generation system needs to decompress and decode the received market data before it can be distributed to subsequent systems.
[0034] Considering that the process of decompressing and decoding market data is more time-consuming than receiving data from the network, and that the flow of decompressed and decoded market data varies across different business units, a caching mechanism is particularly suitable between market data reception and processing. On the other hand, considering that the market data generation system needs to meet high availability and stability requirements, it is more suitable to separate data container management, data reception, and data processing into different processes than to encapsulate all market data operation functions in a single process. Since the processes are isolated from each other, when one process crashes, only the current process is affected, and the operation of other processes is not directly affected. The shared memory-based one-write-many-read high-concurrency lock-free ring cache proposed in this invention is particularly suitable for the described scenario.
[0035] In the market data generation system, the shared memory management process maintains the shared memory environment, initializing and maintaining various containers within it. Several sub-threads of the data receiving process retrieve data packets from different market data sources over the network. After removing the network packet portion, they call the write interface to write the packets to the corresponding storage nodes pointed to by the antenna nodes in the shared memory's circular buffer. Once writing is complete, they can continue receiving data packets without waiting for the market data to be decompressed and decoded. Several sub-threads of the data processing process, based on different business needs, read the corresponding data from the shared memory's circular buffer through the read interface, decompress and decode it, and then provide it for use by other parts of the system before continuing to read the next piece of data to be processed. When the data receiving process needs to retrieve data packets from different market data sources, its sub-threads can store the data in different circular buffers within the shared memory, forming multiple "one-write-many-read" scenarios.
[0036] The above description is merely a specific embodiment of the invention, but the scope of protection of the invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the technical scope disclosed in the invention, based on the technical solutions and novel concepts of the invention, should be covered within the scope of protection of the invention.
Claims
1. An improved method for one-write-multiple-read high-concurrency lock-free circular cache, characterized in that... The method uses shared memory to achieve inter-process communication, treating shared memory as a message bus. Different processes can access the same memory area to achieve data sharing and interaction. The cache structure is stored in shared memory, and in multi-read scenarios, the data in the cache can be mapped to different processes for processing. The method is as follows: S1. Fixed-length shared memory object. The data content in the object is stored page by page. When adding or looking up a record, the page index is found first, and then the position of the corresponding node is obtained through the offset. When a page fault occurs, the size of the shared memory increases dynamically according to the length of the page until the shared memory capacity is exhausted. S2. The structure of a ring cache based on shared memory: A block of shared memory with 2... n A fixed-length shared memory object consisting of (2 to the power of n, where n is a positive integer) + 1 nodes, where the nodes are numbered 0 to 2. n Nodes with a value of -1 are called storage nodes. Each storage node consists of a private information field and a data field. The private information field includes the index value of the currently stored data and the actual length of the stored data, and is numbered 2. n The nodes are called antenna nodes, these 2 n +1 nodes form a circular cache; S3. Tentacle Node: The tentacles store the private information of the most recently written storage node. The information in the tentacles is updated using the CAS method. S4. Interface design of a shared memory-based circular cache: The shared memory-based circular cache can store any data shorter than the length of a shared memory object node. It provides two interfaces for data operations: read and write. In the read interface, the index value of the node to be read is compared with the latest data index value in the tentacles node. The node number is then located using a mask. Before and after reading data, the index value stored in the node to be read is verified to confirm the read data or return a read failure. In the write interface, the index of the node to be written is obtained, located using the circular cache mask, and the data is written to the node. Then, the index value to be written and the private information in the tentacles node are updated. S5. Concurrency Implementation Strategy Based on Shared Memory Ring Cache: The shared memory-based ring cache prioritizes write operations. When there is data in the node to be written, the old data will be overwritten.
2. The improved method for one-write-multiple-read high-concurrency lock-free circular cache as described in claim 1, characterized in that... Step S2 further includes: applying for and maintaining an array pointing to the location of shared memory object storage nodes to reduce the time spent reading and writing storage node data, and recording the mask of the circular cache to facilitate locating the storage node number by index.
3. An improved method for a one-write-many-read high-concurrency lock-free circular cache as described in claim 1, characterized in that... Step S4 further includes: providing two initialization methods: default initialization and loading existing data. In the default initialization interface, all storage nodes are cleared, and the antenna node stores the second... n -1 node's private information; the loading mode can import information from all storage nodes and antenna nodes in a circular cache of the same length and size.