Data processing method and device

Abstract

The invention discloses a data processing method and device. The data processing method comprises the steps of obtaining information of a first buffer area; acquiring a first cache data block corresponding to the first buffer area according to the information of the first buffer area and a mapping relationship between the buffer area and the cache data block; setting a writing state of the first cache data block to forbid a reading operation for the first cache data block; obtaining a version number of the first cache data block; judging whether the obtained version number of the first cache data block is the same as the current version number of the first cache data block or not; If yes, writing data to be written into the first cache data block, and clearing a writing state of the firstcache data block to allow a reading operation for the first cache data block. By adopting the data processing method provided by the invention, the problem of concurrence of cache data reading and writing of a single producer and multiple consumers is solved.

Description

technical field [0001] The present application relates to the field of data processing, in particular to a method and device for writing and reading cached data. Background technique [0002] In a distributed system, multi-threads cooperate to complete tasks. A typical scenario is that one thread (producer thread) produces data, and multiple threads (consumer threads) consume data. A cache is usually used to store these data, and the interaction between multiple threads is performed on this cache. [0003] In the above scenario, one producer thread writes some data into a buffer in memory, multiple consumer threads read some data from the buffer, and repeat the process individually. In the process of reading and writing the buffer, it is necessary to ensure that the consumer will not fetch inconsistent data from the buffer. [0004] A classic solution is to use a mutex. There is a read-write lock (RwLock) in the Cache for synchronization. When the producer writes data, i...

AI Technical Summary

Problems solved by technology

However, due to the mutual exclusion of read-write locks, thread switching overhead is caused, which affects the degree of concurrency. In systems with relatively high performance requirements, it is easy to become a bottleneck.

Read-write locks also have the problem of reader priority and writer priority, which will also affect the priority order of readers and writers

Images