Multi-level intelligent cache system and cache data processing method and program product

Abstract

The present invention relates to the technical field of distributed key-value storage systems, and specifically to a multi-level intelligent cache system and a method and program product for processing cache data. The present invention sets a first-level cache in DRAM and a second-level cache in SCM persistent memory, and the first-level cache and the second-level storage are used to store data of different popularity. At the same time, the cache efficiency factor model is used to calculate the priority of the cache data entry, and when replacing low-priority entries, the decision of data movement between multi-level caches is made by reinforcement learning. The present invention can effectively reduce the access pressure of the DAOS system to the hard disk, save a part of the hard disk bandwidth and memory resources, and improve the overall IO performance of the DAOS system.

Description

Technical Field

[0001] The present invention belongs to the technical field of distributed key-value storage systems, and specifically relates to a multi-level intelligent cache system and a cache data processing method and program product. Background Art

[0002] As one of the representative works of distributed key-value storage, DAOS (The Distributed Asynchronous Object Storage) fully utilizes the performance of next-generation storage technologies by supporting next-generation storage technologies such as Storage Class Memory (SCM) and NVMexpress (NVMe), and using SPDK and PMDK performance development kits to complete end-to-end IO operations in user space to meet the requirements of latency and high throughput in high-performance computing, large-scale data centers, etc. At the same time, DAOS provides libdaos interfaces to support a variety of upper-layer application types, such as POSIX file systems, SQL databases, HDF5, etc.

[0003] DAOS uses a multi-level heterogeneous storage system to store batch data on NVMe SSDs and maintain internal metadata indexes in SCM persistent memory to maximize the original NVMe bandwidth for data reading and writing. The DAOS server maintains multiple IO processes to handle data reading and writing, and independently maintains a DMA buffer in memory for each IO process to avoid memory resource competition for concurrent IO. Its main IO process is as follows: For read operations, the DAOS service first traverses the persistent index in the SCM and creates a Scatter-GatherList (SGL) of the requested data, which contains the physical address of the requested data and the DMA Buffer location information allocated for the data, then calls the SPDK blob I / O interface to start local DMA transmission, reads the data to the corresponding DMA buffer location according to the address information in the SGL, and finally transfers the data from the buffer to the client application through RDMA. The update operation is the opposite.

[0004] DAOS's high performance over other distributed storage systems is due to its separation of frequently accessed metadata from large-volume data, and the use of technologies such as RDMA and DMA to avoid the performance overhead caused by the frequent conversion of traditional IO operations between kernel space and user space, giving full play to the performance of next-generation storage technologies such as SCM and NVMe. However, DAOS does not consider the obvious spatiotemporal locality of upper-level IO loads. For the reading and writing of massive data, the DAOS server needs to frequently access the disk and complete the migration of data between the hard disk and the DMA buffer. Especially under tilted loads, a large number of IO accesses may be limited to a small part of the data. At the same time, in the context of high multi-threaded concurrency, since the data requested by threads are independent of each other, there may be multiple copies of the same data belonging to different threads in the DMA buffer at the same time, resulting in a waste of hard disk bandwidth and memory resources. Therefore, how to give full play to the existing architectural characteristics of DAOS, optimize its use of the spatiotemporal locality of upper-level IO loads, and more efficiently manage data sharing and reuse between multiple threads will be the key challenge to improve system performance.

[0005] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present application, and therefore may include information that does not constitute the prior art known to ordinary technicians in the field. Summary of the invention

[0006] In order to provide a basic understanding of some aspects of the disclosed embodiments, a brief summary is given below. The summary is not an extensive review, nor is it intended to identify key / critical elements or delineate the scope of protection of these embodiments, but rather serves as a prelude to the detailed description that follows.

[0007] The technical problem to be solved by the present invention is to provide a multi-level intelligent cache system and a cache data processing method and program product, by introducing a cache efficiency factor that is more in line with the DAOS data storage characteristics, and combining it with a reinforcement learning algorithm to improve the accuracy and efficiency of data migration between multi-level caches, thereby reducing the access pressure of the DAOS system to the hard disk, saving part of the hard disk bandwidth and memory resources, and improving the overall IO performance of the DAOS system.

[0008] In order to solve the technical problem, the technical solution adopted by the present invention is: a multi-level intelligent cache system, including a first-level cache set in DRAM and a second-level cache set in SCM persistent memory, the first-level cache is used to store a first category of data, the second-level cache is used to store a second category of data, and the third category of data is stored on a hard disk, wherein the access frequency of the first category of data is greater than the access frequency of the second category of data, and the access frequency of the second category of data is greater than the access frequency of the third category of data.

[0009] Furthermore, the system is based on a distributed storage system DAOS, and the hard disk is an NVME SSD.

[0010] The present invention also discloses a method for processing cache data, which is implemented based on the above-mentioned multi-level intelligent cache system and includes two parts: cache replacement and data migration. The cache replacement implements adaptive cache replacement based on a cache efficiency factor. The cache efficiency factor takes into account the memory space occupied by different cache data entries, the time saved by storing in different levels of cache, and the cache data hit rate. It comprehensively evaluates the priority of each cache data entry, sorts the cache internally according to the priority, and when the cache area is full, replaces entries with lower priorities first. Data migration uses a reinforcement learning method to learn the characteristics of different IO loads, generates an adaptive scheduling strategy, and implements intelligent data transmission between caches at all levels and between caches and SSDs.

[0011] Furthermore, the calculation formula of the cache efficiency factor is:

[0012] ,

[0013] Among them, memory size is a parameter that indicates the cache space occupied by cache data entries; savedtime is the time saved when hitting the cache, which is obtained by multiplying the standard speed difference between accessing the hard disk and accessing the corresponding cache medium by the data stripe size; hostness indicates the popularity of the cache data entry, which is the numerical ratio of the entry being accessed in a fixed number of accesses; last two access time is the difference between the most recent access time and the second most recent access time.

[0014] Furthermore, reinforcement learning methods are used to learn the characteristics of different IO loads, including:

[0015] Design space status, which includes the remaining capacity of the first-level cache, the remaining capacity of the second-level cache, the cache hit rate of the previous cycle, and the action of the previous cycle;

[0016] Design action space, which includes not replacing cache, replacing L1 cache to L2 cache, replacing L1 cache to SSD, and replacing L2 cache to SSD;

[0017] Design a reward function to guide the relationship between the spatial state and the action space. The reward function is:

[0018] ,

[0019] in Indicates the average response time of data, , are two time thresholds, < The reward function is designed as a piecewise function, which gives different positive or negative feedback according to the average response time, thereby affecting the corresponding Q-value to increase or decrease the update, thereby optimizing the model decision effect.

[0020] Furthermore, when designing the state space, the cache capacity and hit rate are divided into several levels according to the numerical values.

[0021] Furthermore, the data reading process based on this method is:

[0022] S01, searching for data to be read in the order of the first-level cache, the second-level cache, and the hard disk, and directly reading the data after finding it;

[0023] S02, using a reinforcement learning method to determine the data migration direction between the multi-level caches, and transmitting data according to the determined data migration direction;

[0024] S03. Recalculate the cache entry priority according to the cache efficiency factor, and update the priority ranking of the cache entries in the cache.

[0025] Furthermore, the process of determining the data migration decision between multi-level caches based on reinforcement learning is as follows:

[0026] S01, obtain a reinforcement learning entity;

[0027] S02, selecting an action according to the current state of the input;

[0028] S03, execute the selected action and obtain the next state;

[0029] S04, calculating the reward function after the action is executed;

[0030] S05. Update the corresponding Q-value in the Q-table.

[0031] The present invention further discloses a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the cache data processing method of the present invention is implemented.

[0032] Beneficial effects of the present invention: The heterogeneous storage characteristics of the DAOS distributed system of the present invention design a secondary cache, achieve more detailed and accurate distinction between hot and cold data, and further improve the overall IO performance of the system. The characteristics of DAOS access data are deeply analyzed from the perspective of data and storage mode, and a new adaptive cache replacement algorithm is designed based on this. By introducing a reinforcement learning method, the data transmission between caches at all levels and between caches and SSDs is optimized according to the characteristics of IO load, achieving more accurate and intelligent data migration.

[0033] The above general description and the following description are exemplary and explanatory only and are not intended to limit the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] One or more embodiments are exemplarily described by corresponding drawings, which do not limit the embodiments. Elements with the same reference numerals in the drawings are shown as similar elements, and the drawings do not constitute a scale limitation, and wherein:

[0035] Figure 1 This is the architecture diagram of DAOS multi-level cache. DETAILED DESCRIPTION

[0036] In order to be able to understand the features and technical contents of the embodiments of the present disclosure in more detail, the implementation of the embodiments of the present disclosure is described in detail below in conjunction with the accompanying drawings. The attached drawings are for reference only and are not used to limit the embodiments of the present disclosure. In the following technical description, for the convenience of explanation, a full understanding of the disclosed embodiments is provided through multiple details. However, one or more embodiments can still be implemented without these details. In other cases, to simplify the drawings, well-known structures and devices can be simplified for display.

[0037] The terms "first", "second", etc. in the specification and claims of the embodiments of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the terms used in this way can be interchanged where appropriate, so as to describe the embodiments of the embodiments of the present disclosure described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions.

[0038] Unless otherwise stated, the term "plurality" means two or more.

[0039] In the embodiment of the present disclosure, the character " / " indicates that the preceding and following objects are in an "or" relationship. For example, A / B indicates: A or B.

[0040] The term "and / or" is a description of the association relationship between objects, indicating that three relationships can exist. For example, A and / or B means: A or B, or, A and B.

[0041] The term "correspondence" may refer to an association relationship or a binding relationship. The correspondence between A and B means that there is an association relationship or a binding relationship between A and B.

[0042] Example 1

[0043] like Figure 1As shown in the figure, the DAOS distributed storage system is a multi-level heterogeneous storage system composed of DRAM, SCM persistent memory and NVMe SSD. The storage medium is selected according to data granularity, semantics and other conditions to give full play to the performance advantages of the new generation of storage technology. In the original DAOS design, latency-sensitive IO such as metadata and byte-granularity data are stored in the faster SCM, and checkpoints and batch data are stored in NVMe SSD. However, the temporal and spatial locality of data is not fully considered in the data access logic of DAOS, and there is a lack of distinction and separation between hot and cold data in the heterogeneous storage system, which cannot further give play to the speed advantage of the upper storage device, resulting in DAOS's insufficient reading performance in massive data.

[0044] Based on this, this embodiment discloses a multi-level intelligent cache system based on DAOS, which utilizes the performance gap between DRAM, SCM persistent memory and NVMe SSD under DAOS, including a first-level cache set in DRAM and a second-level cache set in SCM persistent memory, wherein the first-level cache is used to store the first type of data, the second-level cache is used to store the second type of data, and the third type of data is stored on the hard disk, wherein the access frequency of the first type of data is greater than the access frequency of the second type of data, and the access frequency of the second type of data is greater than the access frequency of the third type of data. This system divides cold, hot and warm data more carefully and accurately, and when working, optimizes access to data stored in the first-level cache without going through a complete IO station, further improving the overall IO performance of the system.

[0045] Example 2

[0046] This embodiment discloses a method for processing cache data based on the system described in Embodiment 1, which includes two parts: cache replacement and data migration. Cache replacement implements adaptive cache replacement based on cache efficiency factor, and data migration uses reinforcement learning method to learn different IO load characteristics and generate adaptive scheduling strategy. The cache efficiency factor model is used to calculate the priority of cache data entries, and reinforcement learning is used to make decisions on data movement between multi-level caches when replacing low-priority entries. This project utilizes the characteristics of DAOS heterogeneous storage system and maintains multi-level cache to reduce the overhead of DAOS IO process constantly accessing disk in high concurrency scenarios.

[0047] Cache replacement strategy based on cache efficiency factor: In DAOS, data is stored in a cluster of servers using multiple copies or erasure code redundancy, and considering factors such as system fault tolerance and load balancing, data is allowed to be dynamically migrated between servers. Therefore, the data request initiated by the client needs to be reorganized by a specific algorithm and then distributed to a specific group of servers. In general, the reorganized IO request contains multiple data segments of inconsistent sizes and discontinuous storage locations. We call these segments "extents", and each extent is also a cache data entry in our multi-level cache. The use of traditional cache replacement algorithms has the advantages of simple implementation and wide application scenarios. However, the use of traditional replacement algorithms to maintain DAOS multi-level caches easily leads to ignoring the characteristics of different extents occupying different cache spaces and saving different time, thereby reducing the cache's contribution to the overall IO performance. Therefore, this project introduces the concept of cache efficiency factor, which fully considers the memory space occupied by different cache data entries, the time saved by storing in different levels of cache, the cache data hit rate and other factors, and comprehensively evaluates the priority of each cache data entry. The cache is sorted according to priority. When the cache area is full, the entries with lower priority are replaced first. The calculation method of the cache efficiency factor model is shown in (1):

[0048] (1),

[0049] Among them, memory size is a parameter that indicates the occupancy of cache space by cache data entries; savedtime is the time saved when hitting the cache, which is obtained by multiplying the standard speed difference between accessing the hard disk and accessing the corresponding cache medium by the data stripe size; hostness indicates the popularity of the cache data entry, which is the numerical proportion of the entry being accessed in a fixed number of accesses; last two access time is the difference between the most recent access time and the second most recent access time. This metric is used to alleviate the adverse effects of a large number of burst accesses on cache replacement.

[0050] In this embodiment, the multi-level cache decision based on reinforcement learning is specifically: maintaining data in a multi-level cache according to the degree of coldness and heat can greatly reduce the IO pressure of DAOS. However, the movement of data entries between caches at all levels needs to be intelligently scheduled, otherwise, frequent and invalid data movement will seriously block system tasks and reduce IO performance. In order to solve this problem, this project uses reinforcement learning methods to learn different IO load characteristics, generate adaptive scheduling strategies, and minimize the maintenance cost of multi-level caches. The key to reinforcement learning is to design state space, action space and reward function. First of all, the design of the state space (state) should be able to fully reflect the usage of multi-level caches and past cache hit rates. Therefore, the expression of the state space includes four aspects: the remaining capacity of the first-level cache, the remaining capacity of the second-level cache, the cache hit rate of the last cycle, and the action of the last cycle. In implementation, we need to divide indicators such as cache capacity and hit rate into several levels according to the numerical size, so as to effectively control the scale of the state space. Secondly, in reinforcement learning, an action refers to the operation or decision performed by an agent in a specific environment. In this project, it is reflected in the replacement of cache data items under multi-level cache, including four categories: no cache replacement, replacement of the first-level cache to the second-level cache, replacement of the first-level cache to the SSD, and replacement of the second-level cache to the SSD. Finally, a reward function as shown in formula (2) is designed to guide the relationship between the agent's learning state and action, and the average response time of the data is used to evaluate the current model decision effect.

[0051] (2),

[0052] in Indicates the average response time of data, , are two time thresholds, < The reward function is designed as a piecewise function, which gives different positive or negative feedback according to the average response time, thereby affecting the corresponding Q-value to increase or decrease the update, thereby optimizing the model decision effect. , Dynamically adjust according to the distribution of IO response time to adapt to different application scenarios.

[0053] The data reading process based on this method is:

[0054] S01, searching for data to be read in the order of the first-level cache, the second-level cache, and the hard disk, and directly reading the data after finding it;

[0055] S02, using a reinforcement learning method to determine the data migration direction between the multi-level caches, and transmitting data according to the determined data migration direction;

[0056] S03. Recalculate the cache entry priority according to the cache efficiency factor, and update the priority ranking of the cache entries in the cache.

[0057] The program code for implementing data reading under multi-level cache is:

[0058]

[0059] The process of determining data migration decisions between multi-level caches based on reinforcement learning is as follows:

[0060] S01, obtain a reinforcement learning entity;

[0061] S02, selecting an action according to the current state of the input;

[0062] S03, execute the selected action and obtain the next state;

[0063] S04, calculating the reward function after the action is executed;

[0064] S05. Update the corresponding Q-value in the Q-table.

[0065] The implementation code for data migration decision between multi-level caches based on reinforcement learning is:

[0066]

[0067] Example 3

[0068] This embodiment discloses a computer program product, including a computer program. When the computer program is executed by a processor, the cache data processing method described in Embodiment 1 is implemented.

[0069] The technical solution of the embodiment of the present disclosure can be embodied in the form of a software product, which is stored in a storage medium and includes one or more instructions for enabling a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in the embodiment of the present disclosure. The aforementioned storage medium may be a non-transient storage medium, including: a USB flash drive, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a disk or an optical disk, and other media that can store program codes, or a transient storage medium.

[0070] The above description and the accompanying drawings fully illustrate the embodiments of the present disclosure so that those skilled in the art can practice them. Other embodiments may include structural, logical, electrical, process and other changes. The embodiments represent only possible changes. Unless explicitly required, separate components and functions are optional, and the order of operations may vary. Parts and features of some embodiments may be included in or replace parts and features of other embodiments. Moreover, the words used in this application are only used to describe the embodiments and are not used to limit the claims. As used in the description of the embodiments and the claims, unless the context clearly indicates, the singular forms of "a", "an" and "the" are intended to include plural forms as well. Similarly, the term "and / or" as used in this application refers to any and all possible combinations of one or more associated listings. In addition, when used in this application, the term "comprise" and its variants "comprises" and / or comprising refer to the presence of stated features, wholes, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or groups thereof. In the absence of further restrictions, the elements defined by the sentence "including one..." do not exclude the presence of other identical elements in the process, method or device including the elements. In this article, each embodiment may focus on the differences from other embodiments, and the same and similar parts between the embodiments may refer to each other. For the methods, products, etc. disclosed in the embodiments, if they correspond to the method part disclosed in the embodiments, then the relevant parts can refer to the description of the method part.

[0071] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software may depend on the specific application and design constraints of the technical solution. The technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the embodiments of the present disclosure. The technicians may clearly understand that for the convenience and simplicity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here.

[0072] In the embodiments disclosed herein, the disclosed methods and products (including but not limited to devices, equipment, etc.) can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units can be only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between each other shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms. The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to implement this embodiment. In addition, each functional unit in the embodiment of the present disclosure may be integrated in a processing unit, or each unit may exist physically separately, or two or more units may be integrated in one unit.

[0073] The flowchart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to the embodiment of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of the code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. In some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, which can depend on the functions involved. In the description corresponding to the flowchart and the block diagram in the accompanying drawings, the operations or steps corresponding to different boxes can also occur in a different order from the order disclosed in the description, and sometimes there is no specific order between different operations or steps. For example, two consecutive operations or steps can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, which can depend on the functions involved. Each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented by a dedicated hardware-based system that performs the specified functions or actions, or may be implemented by a combination of dedicated hardware and computer instructions.

Claims

1. A method for processing cache data, the method is implemented based on a multi-level intelligent cache system, the multi-level intelligent cache system includes a first-level cache set in DRAM and a second-level cache set in SCM persistent memory, the first-level cache is used to store first-class data, the second-level cache is used to store second-class data, and the third-class data is stored on a hard disk, wherein the access frequency of the first-class data is greater than the access frequency of the second-class data, and the access frequency of the second-class data is greater than the access frequency of the third-class data, and the characteristics are: It includes two parts: cache replacement and data migration. Cache replacement implements adaptive cache replacement based on cache efficiency factor. The cache efficiency factor considers the memory space occupied by different cache data entries, the time saved by storing in different levels of cache, and the cache data hit rate. It comprehensively evaluates the priority of each cache data entry. When the cache area is full, the entries with low priority are replaced first. Data migration uses reinforcement learning algorithm to learn the characteristics of different IO loads, generate adaptive scheduling strategies, and realize intelligent data transmission between caches at all levels and between caches and SSDs. The calculation formula of cache efficiency factor is: ， Among them, memory size is a parameter that indicates the cache space occupied by cache data entries; saved time is the time saved when hitting the cache, which is obtained by multiplying the standard speed difference between accessing the hard disk and accessing the corresponding cache medium by the data stripe size; hotness indicates the hotness of the cache data entry, which is the numerical ratio of the entry being accessed in a fixed number of accesses; last two access time is the difference between the most recent access time and the second most recent access time.

2. The method for processing cache data according to claim 1, characterized in that: Using reinforcement learning algorithms to learn different IO load characteristics include: Design space status, which includes the remaining capacity of the first-level cache, the remaining capacity of the second-level cache, the cache hit rate of the previous cycle, and the action of the previous cycle; Design action space, which includes not replacing cache, replacing L1 cache to L2 cache, replacing L1 cache to SSD, and replacing L2 cache to SSD; Design a reward function to guide the relationship between the spatial state and the action space. The reward function is: ， in Indicates the average response time of data, , are two time thresholds, < The reward function is designed as a piecewise function, which gives different positive or negative feedback according to the average response time, thereby affecting the corresponding Q-value to increase or decrease the update, thereby optimizing the model decision effect.

3. The method for processing cache data according to claim 2, characterized in that: When designing the state space, the cache capacity and hit rate are divided into several levels according to the numerical values.

4. The method for processing cache data according to claim 2, characterized in that: The data reading process based on this method is: S01, searching for data to be read in the order of the first-level cache, the second-level cache, and the hard disk, and directly reading the data after finding it; S02, using a reinforcement learning method to determine the data migration direction between the multi-level caches, and transmitting data according to the determined data migration direction; S03. Recalculate the cache entry priority according to the cache efficiency factor, and update the priority ranking of the cache entries in the cache.

5. The method for processing cache data according to claim 2, characterized in that: The process of determining data migration decisions between multi-level caches based on reinforcement learning is as follows: S01, obtain a reinforcement learning entity; S02, selecting an action according to the current state of the input; S03, execute the selected action and obtain the next state; S04, calculating the reward function after the action is executed; S05. Update the corresponding Q-Value in the Q-Table.

6. The method for processing cache data according to claim 1, characterized in that: The multi-level intelligent cache system is based on the distributed storage system DAOS, and the hard disk is NVME SSD.

7. A computer program product comprising a computer program, characterized in that: When the computer program is executed by a processor, the method for processing cache data as described in any one of claims 1 to 6 is implemented.

Images