A write-one-multiple-read high-concurrency lock-free circular cache and its implementation method

Abstract

The present invention relates to the technical field of software system cache, and in particular to a one-write-multiple-read high-concurrency lock-free circular cache and an implementation method thereof, wherein the circular cache is composed of m storage nodes, where m is a positive integer, and each storage node is composed of a read-write tag field, an actual length field of data, an index value field of a node, and a data field. The number of nodes is regarded as the length of the circular cache, and the length needs to satisfy the nth power of 2, where n is a positive integer. The circular cache will number the position of each node according to its length, and the numbering is a natural number that increases in sequence from 0 until the length of the circular cache is reduced by 1. The method has the advantages that, compared with single-threaded cacheless data processing and cache implemented by a lock mechanism under multi-threading, the method of using a high-concurrency lock-free circular cache greatly improves the processing performance of the system, and provides a solution for systems with high requirements for low latency and high concurrency.

Description

Technical Field

[0001] The present invention relates to the technical field of software system cache technology, and in particular to a write-one-multiple-read high-concurrency lock-free circular cache and an implementation method thereof. Background Art

[0002] In software systems, data access speed is often faster than data processing speed. In order to adapt to the speed difference between the two, cache is a commonly used mechanism to solve the problem of insufficient data processing performance. At the same time, in order to improve data processing performance, multi-threaded concurrent data processing is also used. The use of cache in multi-threaded environments is divided into read and write. The cache needs to be read when data is accessed, and data that will not be processed immediately needs to be written to the cache. When performing read and write operations on the same cache in a multi-threaded environment, concurrent read and write problems will be encountered. Concurrent read and write operations will cause data confusion and lead to data anomalies.

[0003] There are two scenarios for concurrent reading and writing of multiple threads: one write and multiple reads and multiple writes and one read. One write and one read can be considered as a special scenario of one write and multiple reads or multiple writes and one read. In order to solve the problem of concurrent reading and writing under multi-threading, the usual solution is to introduce a lock mechanism. The introduction of locks solves the problem of concurrent reading and writing, but it will cause performance degradation. The larger the granularity of the lock, the more concurrent operations on the cache, and the more the system performance is degraded.

[0004] Therefore, a high-performance solution is needed for multi-threaded concurrent reading and writing scenarios. Summary of the invention

[0005] The purpose of the present invention is to solve the "one write, multiple reads" scenario of multi-threaded concurrent reading and writing, and proposes a one-write, multiple-read, high-concurrency lock-free circular cache and its implementation method to solve the performance problem of concurrent reading and writing under multi-threading. It is a true lock-free operation and supports one-write, multiple-read, lock-free, high-concurrency performance operation, which can adapt to the needs of multi-threaded concurrent processing.

[0006] In order to achieve the above purpose, a one-write-many-read high-concurrency lock-free circular cache is designed, characterized in that the circular cache consists of m storage nodes, m is a positive integer, and each storage node consists of a read-write tag field, an actual length field of the data, an index value field of the node and a data field. The number of nodes is regarded as the length of the circular cache, and the length needs to satisfy the nth power of 2, n is a positive integer. The circular cache will number the position of each node according to its length, and the number is a natural number that increases sequentially from 0 until the length of the circular cache is reduced by 1.

[0007] Preferably, the circular cache can store data of any data type. The circular cache provides two operation interfaces for reading and writing data. The read interface will map any index value to the actual numbered position of the circular cache data storage node, read the data at the numbered position, and return the data domain and the index value stored at the node number; the write interface is used to write data into the cache. The write interface starts writing from node number 0 in sequence, and returns to the position of number 0 when the maximum number is reached, overwriting the original data, and repeating the cycle to meet the requirements of circular storage. When writing data, the node index value needs to be updated, and the node index value is equal to the cumulative amount of data written in the cache minus 1.

[0008] The implementation method of the above-mentioned one-write-multiple-read high-concurrency lock-free circular cache is characterized in that the circular cache performs concurrency control according to the read-write tag value of each storage node, and each node maintains a writing tag and a read tag. Nodes with a read tag can write and read; the write interface can only write data when it determines that the read-write tag of the current storage node is readable. When writing data, the tag will first be set to writing, and then the tag will be set to read after writing. If the write interface determines that the tag is not readable, it will wait until the tag is readable before writing data. The modification of the tag value by the write interface needs to satisfy atomic operations; the read interface can only read data when it determines that the read-write tag of the current storage node is greater than or equal to the read tag, and the writing tag directly exits the read operation; when reading data from a storage node, the read interface first adds 1 to the read tag of the storage node, and subtracts 1 from the tag after reading the data. Modifying the tag value must satisfy atomic operations. When all read operations are completed, the storage node is marked as readable, so that the write interface can write data at this position.

[0009] Compared with the prior art, the present invention has the following advantages:

[0010] 1. Compared with single-threaded data processing without cache and cache implemented by lock mechanism in multi-threaded mode, the method of using high-concurrency lock-free circular cache greatly improves the processing performance of the system. For systems with high requirements for low latency and high concurrency, high-concurrency lock-free circular cache provides a solution;

[0011] 2. Good interface design provides convenience and flexibility for users, and reasonable scenario design also provides users with autonomy, allowing them to choose to read data from any location in the cache based on actual needs. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 It is a schematic diagram of the structure of the ring buffer of the present invention;

[0013] Figure 2 It is a flowchart of the interface writing of the present invention;

[0014] Figure 3 It is a flow chart of the reading interface of the present invention;

[0015] Figure 4 It is a performance comparison between the lock-free cache and the locked cache of the present invention; DETAILED DESCRIPTION

[0016] The present invention is further described below in conjunction with the accompanying drawings, and the structure and principle of the present invention are very clear to people in the field. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0017] 1. As Figure 1 As shown, the ring buffer is composed as follows:

[0018] The ring cache is composed of m (m is a positive integer) storage nodes. Each storage node consists of a read / write tag field, the actual length field of the data, the index value field of the node, and the data field. The number of nodes is considered the length of the ring cache, and the length needs to satisfy 2^n (2 to the power of n, n is a positive integer). The ring cache will number each node according to its length. The number is a natural number that increases from 0 until the length of the ring cache is reduced by 1. The schematic diagram of the ring cache is as follows: Figure 1 shown.

[0019] 2. Ring cache interface design:

[0020] The ring cache can store data of any data type. It provides two operation interfaces, read and write data. The read interface will map any index value to the actual number position of the ring cache data storage node, read the data at the number position, and return the data domain and the index value stored at the node number; the write interface is used to write data into the cache. The write interface starts writing from node number 0 in sequence, and returns to the position of number 0 when the maximum number is reached, overwriting the original data, and repeating the cycle to meet the requirements of ring storage. When writing data, the index value of the node needs to be updated. The index value of the node is equal to the cumulative number of data written in the cache minus 1. For example, in the first round, the index value of the node at position 0 is 0, and the index value of position 1 is 1. In the second round, the index value of the node at position 0 is the length of the ring cache, and the index value of position 1 is the length of the ring cache - 1.

[0021] 3. Circular cache concurrent implementation strategy:

[0022] The circular cache performs concurrency control based on the read / write tag value of each storage node. Each node maintains a "writing" (value 0) tag and a "can read" (value 1) tag. Nodes marked with a "can read" tag can be written or read. The write interface determines that the read / write tag of the current storage node is "can read" before writing data. When writing data, the tag is first set to "writing" and then set to "can read" after writing. If the write interface determines that the tag is not "can read", it will wait until the tag is "can read" before writing data. The modification of the tag value by the write interface needs to satisfy atomic operations. The flowchart of the write interface is as follows: Figure 2 As shown. The read interface can read data only when it determines that the read-write mark of the current storage node is greater than or equal to the "can read" mark. The "writing" mark directly exits the read operation; when the read interface reads data from a storage node, it first adds 1 to the "can read" mark of the storage node, and subtracts 1 from the mark after reading the data. Modifying the mark value must satisfy atomic operations. When all read operations are completed, the storage node is marked as "can read", so that the write interface can write data at this location. The flowchart of the read interface is shown as follows Figure 3 shown.

[0023] A preferred implementation method: In the securities market, market data is very important data. Securities transactions can be completed based on market data, and quantitative strategies can also be verified, thereby better serving securities transactions. Therefore, the low latency feature of market data is a very important indicator in the market system. In order to speed up the processing of market data, the market system usually uses multi-threading and caching technology to process market data.

[0024] The market data pushed by the market source system is compressed and encoded. The market system needs to decompress and decode the received market data before it can be used by other systems. Considering that the process of data decompression and decoding is a relatively time-consuming process, a single-threaded approach cannot be used, that is, decompression and decoding are performed immediately after receiving the market source data. If this is done, it will increase the market delay and other problems. Therefore, the market system adopts multi-threading and caching technology. At this time, high-concurrency lock-free circular cache under multi-threading provides a solution.

[0025] The market system will have a receiving thread dedicated to receiving market source data, multiple decompression and decoding threads responsible for decompressing and decoding market source codes, and an instance of a high-concurrency lock-free circular cache. The data exchange between the receiving thread and the decompression and decoding thread is through a high-concurrency lock-free circular cache (referred to as cache). After receiving the market source data, the receiving thread calls the cache write interface to write the market source data to the cache. After the data is written, the receiving thread continues to receive the market source data, and the cycle continues until the market source data is received; the decompression and decoding thread reads the data in the cache through the cache read interface, decompresses and decodes the data after reading, and provides it to other modules after the data processing is completed, and then continues to read the next data through the read interface, and loops in until all the data in the cache are processed.

Claims

1. A method for implementing a high-concurrency lock-free circular cache with one write and multiple reads, characterized in that: The circular cache is composed of m storage nodes, where m is a positive integer. Each storage node is composed of a read-write tag field, an actual length field of the data, an index value field of the node, and a data field. The number of storage nodes is regarded as the length of the circular cache, and the length needs to satisfy the nth power of 2, where n is a positive integer. The circular cache will number each storage node according to its length, and the number is a natural number that increases in sequence from 0 until the length of the circular cache is reduced by 1; The circular cache can store data of any data type. The circular cache provides two operation interfaces, read and write data. The read interface will map any index value to the actual number position of the storage node of the circular cache, read the data at the number position, and return the data domain and the index value stored at the number position of the storage node. The write interface is used to write data into the cache. The write interface starts writing from the storage node number 0 in sequence. When the maximum number is reached, it returns to the position of number 0, overwriting the original data, and repeats the process in a loop to satisfy the circular storage. When writing data, the index value of the storage node needs to be updated. The index value of the storage node is equal to the cumulative amount of data written in the circular cache minus 1. In the first loop, the index value of the storage node at position 0 is 0, the index value of the storage node at position 1 is 1, and so on. In the second loop, the index value of the storage node at position 0 is the length of the circular cache, the index value of the storage node at position 1 is the length of the circular cache + 1, and so on. The concurrency implementation strategy of the circular cache is: the circular cache performs concurrency control according to the read-write tag value of each storage node, and each storage node maintains a writing tag and a read-enabled tag, wherein the value of the writing tag is 0, and the value of the read-enabled tag is 1. The storage node with the read-enabled tag can write and read; The write interface can write data only when it determines that the read-write mark of the current storage node is a readable mark. When writing data, the read-write mark will be set to the writing mark first, and then the read-write mark will be set to the readable mark after writing. If the write interface determines that the read-write mark is a non-readable mark, it will wait until the read-write mark is a readable mark before writing data. The write interface's modification of the read-write mark value needs to satisfy atomic operations; the read interface can read data only when it determines that the read-write mark value of the current storage node is greater than or equal to the read mark value. If it is a writing mark, the read operation is directly exited; when the read interface reads data from a storage node, it first adds 1 to the value of the readable mark of the storage node, and subtracts 1 from the value of the readable mark of the storage node after reading the data. Modifying the value of the readable mark must satisfy atomic operations. When all read operations are completed, the read-write mark of the storage node is set to the readable mark, so that the write interface can write data at the location of the storage node.

Images