Service-aware multi-queue node memory management system and method in cloud memory pool

Abstract

A cloud memory pool service-aware multi-queue node memory management system and method, comprising: a memory service offline analysis module, a service-aware memory node management module and a high-bandwidth parallel data exchange subsystem, wherein: the memory service offline analysis module performs data pre-analysis on the cloud memory pool-based memory service, and assigns a data offload tag to the memory service; the service-aware memory node management module constructs a hierarchical storage, adds a data offload tag to the data entering the cloud memory pool according to the data offload tag of the service, and differentially manages the data according to the data offload tag to control the data offload; and the high-bandwidth parallel data exchange subsystem is used for data offload I / O of the data in the memory node. The application constructs a multi-level memory storage structure supporting high parallel high-bandwidth data exchange, expands the memory with cloud storage to reduce the memory resource use, analyzes the data characteristics of the cloud memory service and realizes the differential management of the service data in the cloud memory pool, improves the overall data access efficiency, and further reduces the memory resource occupation and improves the memory resource efficiency in the data center.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of distributed data processing, and particularly relates to a service-aware multi-queue node memory management system and method in a cloud memory pool. BACKGROUND

[0002] With the rapid growth of application data, the traditional data center has the problem of unbalanced load of memory resources. In order to promote the balanced use of memory resources, in recent years, a cloud memory pool is proposed, which integrates the memory resources in the data center to build a cloud memory pool. The cloud memory pool architecture in cloud computing builds a unified cloud memory pool and provides various memory services for applications. However, the existing cloud memory pool is still not efficient. On the one hand, the existing cloud memory pool is only composed of memory resources in the data center, and the cost is high. On the other hand, the existing cloud memory pool system does not perceive the access characteristics of application data, and the use efficiency is low. SUMMARY

[0003] The present application proposes a service-aware multi-queue node memory management system and method in a cloud memory pool to solve the problems of high hardware cost caused by only supporting a single storage medium in the cloud memory pool and low management efficiency caused by not supporting application data access feature awareness. The system and method construct a multi-level memory storage structure supporting high parallel high-bandwidth data exchange, expand memory with cloud storage to reduce memory resource use, analyze data characteristics of cloud memory services and realize differentiated management of service data in the cloud memory pool, improve overall data access efficiency, and thus reduce memory resource occupation and improve data center memory resource efficiency.

[0004] The present application is implemented by the following technical solutions:

[0005] The present application relates to a service-aware multi-queue node memory management system in a cloud memory pool, comprising: a memory service offline analysis module, a service-aware memory node management module, and a high-bandwidth parallel data exchange subsystem, wherein: the memory service offline analysis module performs data pre-analysis on the memory service based on the cloud memory pool, and assigns a data offloading tag to the memory service; the service-aware memory node management module builds a layered storage, adds a data offloading tag to the data entering the cloud memory pool according to the data offloading tag of the service, and differentially manages the data according to the data offloading tag to control the offloading of the data; and the high-bandwidth parallel data exchange subsystem is used for data offloading I / O of data in the memory node.

[0006] The cloud memory pool refers to interconnecting the memory resources of the data center through a network to form a unified memory aggregate. The memory service is built based on the cloud memory pool, and the application stores data in the cloud memory pool through the memory service for data caching and data sharing.

[0007] The memory service refers to a server-unaware memory service provided in cloud computing, and application is unaware of memory management in a service node at a bottom layer, and a memory service is called to realize data caching and data sharing of the memory.

[0008] The memory service offline analysis module comprises a service pre-execution unit, a data access statistics unit and a memory service marking unit, wherein the service pre-execution unit calls to execute a service to be processed, the data access statistics unit counts a data reading frequency of the service called in the service execution unit, and the memory service marking unit assigns a data offloading tag to the service according to the data reading frequency in the service.

[0009] The memory node management module comprises a data object segmentation unit, a multi-queue data storage unit, an access counter and a data offloading strategy algorithm, wherein the data object segmentation unit divides data transmitted into the memory node management module through the memory service into data blocks of a fixed size; the multi-queue data storage unit is composed of three least recently used queues, and the data blocks enter respective queues for storage according to data offloading tags assigned by the memory service; the access counter counts data access times in the multi-queue data storage unit; and the data offloading strategy algorithm controls data to enter a parallel data exchange subsystem for data offloading according to information of the access counter and a current system state.

[0010] The parallel data exchange subsystem comprises a local storage I / O unit, a local storage, a cloud storage I / O unit and a cloud storage unit, wherein the local storage I / O unit and the cloud storage I / O unit cache data blocks from the memory node management module, the local storage I / O unit controls data offloading and acquisition between the memory node and the local storage, and the cloud storage I / O unit controls data offloading and acquisition between the memory node and the cloud storage unit.

[0011] The application relates to a node memory management method in a cloud memory pool based on the above system, wherein pre-analysis and data offloading tag assignment are performed for a memory service in cloud computing in an offline stage, and the data offloading tag is attached to an application through the memory service when the application is executed online; the memory node management module adopts a hierarchical storage mode, data is respectively assigned to different least recently used queues according to the data offloading tag, and when the memory resource reaches an upper limit, parallel data exchange is performed through a parallel data exchange subsystem to offload data to cloud storage, and when the data is accessed again, the data is pre-fetched in parallel through the data exchange subsystem.

[0012] Technical effects

[0013] This invention pre-analyzes data access characteristics of memory services in cloud computing by constructing an offline analysis module for memory services, builds differentiated node memory multi-queue data storage units, and proposes a data offloading strategy algorithm that combines offline pre-analysis information and online data access information to control data offloading from node memory in the cloud memory pool. A two-level parallel high-bandwidth parallel data exchange subsystem is constructed, using a collaborative approach of queue-level parallel data exchange and data block object-level parallel data exchange for data offloading. Compared with existing technologies, this invention reduces memory usage in the cloud memory pool, thereby reducing the construction cost of the cloud memory pool. It also improves the data access efficiency in the cloud memory pool, thus improving the operational efficiency of cloud memory services based on the cloud memory pool. Attached Figure Description

[0014] Figure 1 This is a schematic diagram of the system structure of the present invention;

[0015] Figure 2 This is a schematic diagram of the service-aware memory node management module in this invention;

[0016] Figure 3 This is a flowchart of the data unloading strategy for the memory node management module of the present invention;

[0017] Figure 4 This is a schematic diagram of the high-bandwidth parallel data exchange subsystem in this invention;

[0018] Figure 5 This is a flowchart of an implementation example;

[0019] Figure 6 This is a schematic diagram of a scenario for an example embodiment;

[0020] Figure 7 The following is a graph showing the execution latency results for an example. Detailed Implementation

[0021] like Figure 1 As shown in this embodiment, a service-aware multi-queue node memory management system in a cloud memory pool is disclosed. The system includes: a memory service offline analysis module, a service-aware memory node management module, and a high-bandwidth parallel data exchange subsystem. Specifically: the memory service offline analysis module performs data pre-analysis on memory services based on the cloud memory pool and assigns data offloading tags to the memory services; the service-aware memory node management module constructs hierarchical storage, adds data offloading tags to data entering the cloud memory pool based on the service's data offloading tags, and performs differentiated management of the data based on these tags to control data offloading; the high-bandwidth parallel data exchange subsystem is used for data offloading I / O in the memory nodes.

[0022] The embodiment relates to a node memory management method in a cloud memory pool based on the above system, pre-analysis and data offload tag allocation are carried out for a memory service in cloud computing in an offline stage, and the data offload tag is attached to the application by the memory service during online execution; a memory node management module adopts a hierarchical storage mode, data is allocated to different least recently used queues according to the data offload tag, and when the memory resource reaches an upper limit, parallel data exchange is carried out through a parallel data exchange subsystem, the data is offloaded to cloud storage, and when the data is accessed again, the data is pre-fetched in parallel through the data exchange subsystem.

[0023] The pre-analysis refers to the following: before the memory service based on the cloud memory pool is put online, the access frequency of service data is counted, the service is classified according to the access frequency of data passing through the service in a life cycle, and a corresponding data offload tag is assigned, and when the data is stored in the cloud memory pool by the service during online use, the data offload tag of the service is attached, and the data offload tag is used for management of the memory node in the cloud memory pool.

[0024] The classification refers to the following: the access frequency of data passing through the memory service based on the cloud memory pool in a life cycle is counted, data with an access frequency greater than or equal to 100 times is marked as inedible data (IED), indicating that the data cannot be offloaded when being managed, for example, cache data in a computing process; data with an access frequency greater than 1 and less than 100 times is marked as low-evict data (LED), indicating that the data can be appropriately offloaded when being managed, for example, shared data in data sharing; and data with an access frequency of only 1 time is marked as high-evict data (MED), indicating that the data can be offloaded first when being managed, for example, random sorting data in big data computing.

[0025] Because part of the data in the memory service has unpredictability, the selection of the data offload tag can be exposed to the application by the part of the data service, and the selection of the data offload tag is performed by the application.

[0026] As shown in Figure 2 The hierarchical storage refers to the following: the memory node divides the application data passing through the service and stores the data in a data block mode, and differentiates the application data according to the data offload tag, that is, three least recently used queues are maintained, which are an IED queue, an LED queue and an MED queue. After the data passes through the memory service and enters the cloud memory pool, the data has respective data offload tags, and enters the corresponding queue according to the respective data offload tags, the inedible data enters the IED queue, the low-evict data enters the LED queue, and the high-evict data enters the MED queue.

[0027] As shown in Figure 3As shown, the data unloading refers to: preferentially unloading data in the LED queue and the MED queue when unloading data, and further unloading data in the IED queue when the memory still exceeds the limit after unloading data in the two queues, and specifically comprising: obtaining data unloading tags of data and calculating unloading amounts of the LED queue and the MED queue, then unloading data in the LED queue and the MED queue in turn, further calculating the unloading amount of the IED queue and unloading data in the IED queue when the data unloading is not completed.

[0028] The unloading amounts of the LED queue and the MED queue are calculated by the following method: the LED queue and the MED queue need to unload n size data in total, the access frequency of the LED queue in the current stage is x, and the access frequency of the MED queue is y, then this time unloading will unload LED data of n*y / (x+y) size and MED data of n*x / (x+y) size.

[0029] As shown in the Figure 4 The parallel data exchange refers to: adopting data queue parallel and data exchange channel parallel when performing data exchange, wherein: the data queue parallel refers to simultaneously unloading data in the IED queue through local storage, and unloading data in the LED queue and the MED queue through cloud storage; the data exchange channel parallel refers to simultaneously unloading data in the IED queue through local storage I / O exchange, and unloading data in the LED queue and the MED queue through cloud storage I / O exchange.

[0030] The local storage I / O refers to: using local high-speed storage to perform data exchange and unload data to the local storage of the memory node.

[0031] The cloud storage I / O refers to: using cloud storage to perform data exchange and unload data to the cloud storage.

[0032] In the data exchange channel parallel, data entering the cloud memory pool is divided into data blocks of equal size, and the local storage I / O and the cloud storage I / O perform corresponding size parallel data exchange according to the number of data exchange backends currently configured when unloading. For example, when m cloud storage backends are configured for the cloud storage I / O, the cloud storage I / O will unload m data blocks simultaneously when unloading data, and m data blocks will also be prefetched when retrieving data.

[0033] As shown in the Figure 5 The memory node realizes data block mode storage by creating and reading data blocks, wherein:

[0034] The process of creating a data block includes: i) calling the data block creation interface to create the data block; ii) if creation fails, the system is currently unloading data, wait for the data unloading to complete, and then return to step i); otherwise, proceed to step iii); iii) if the data block is created successfully, the creation result is returned asynchronously, and it is simultaneously determined whether the current memory capacity has triggered data unloading. If data unloading has not been triggered, return directly; otherwise, proceed to step iv); iv) perform data unloading, and after data unloading is complete, change the status to unloading complete.

[0035] The process of reading a data block includes: i) calling the data block reading interface to read the data block; ii) if the data block is in memory, return the data block directly; otherwise, proceed to step iii); iii) if the data block is not in memory but in storage, reload the data; iv) if the data is reloaded into memory, call the data block creation interface to recreate the data block, and return to step i) reading the data block.

[0036] Through specific practical experiments, this embodiment uses three identically configured servers to construct a cloud memory pool. Each server is equipped with two Intel(R) Gold 6161 CPUs with 22 physical cores each, 692GB of memory, four NVMe SSDs as local storage media, and four cloud disks as cloud storage media. The three servers are connected via a local area network to form the cloud memory pool. A database caching service is built based on this cloud memory pool, which caches the data tables required for database calculations in the cloud memory pool based on this invention. During database calculations, data is directly read from the cloud memory pool. A data sharing service is also implemented, where different tasks call the data sharing service to store data in the cloud memory pool and then read the data from the cloud memory pool to complete data sharing. Finally, a vector caching service is implemented, where intermediate results during the calculation process are cached in the cloud memory pool, and subsequent retrieval of intermediate results directly reads the cached vectors from the cloud memory pool.

[0037] like Figure 6 As shown, this is the application scenario of this embodiment: it is executed in a distributed manner in the above hardware environment. The embodiment uses the cloud memory pool by calling the service provided by the cloud memory pool. The memory nodes in the cloud memory pool use the present invention to manage the data stored in the memory nodes.

[0038] like Figure 7The performance comparison chart of the application calling the memory service based on the application is shown. In the above experimental environment, the TPC-H application is run based on the database cache service, the 128G data storage and retrieval application is run based on the data sharing service, and the video generation large model training application is run based on the vector cache service. Compared with the prior art, the cloud memory pool based on the application can reduce the execution time of the TPC-H application by 213.2s, improve the performance by 68.6%, reduce the execution time of the 128G data single access by 49.1s, improve the performance by 34.4%, and reduce the execution time of the video generation large model application by 140.0s, improve the performance by 28.6%.

[0039] In summary, the application solves the problem of large memory resource overhead in the prior art cloud memory pool system, increases the storage of the hierarchical memory, and reduces the memory resource usage in the data center. It fills the gap in the management of memory nodes in the cloud memory pool system, uses a service-aware memory management mechanism, improves the data hit rate, and significantly improves the overall data access efficiency. Using high-bandwidth data exchange further reduces the performance overhead of hierarchical memory. The application enables the cloud memory pool to provide services with less memory resources, saving the resource overhead of the data center. The same size of memory resources can carry more tasks and also improve the throughput of the tasks. Compared with the prior art, the application can provide differentiated management according to the characteristics of the memory service, optimize the data offloading strategy, and at the same time use multi-level parallel to improve the data exchange bandwidth, thereby improving the memory efficiency.

[0040] The above specific embodiments can be adjusted in different ways by those skilled in the art without departing from the principles and purposes of the application. The protection scope of the application is subject to the claims and is not limited by the above specific embodiments. Each implementation scheme within the scope is subject to the constraints of the application.

Claims

1. A service-aware multi-queue node memory management system in a cloud memory pool, characterized by, The application relates to a cloud memory pool system, which comprises a memory service offline analysis module, a service-aware memory node management module and a high-bandwidth parallel data exchange subsystem, wherein the memory service offline analysis module performs data pre-analysis on a cloud memory pool-based memory service, and data unloading tags are given to the memory service; the service-aware memory node management module constructs a hierarchical storage, adds data unloading tags to data entering the cloud memory pool according to the data unloading tags of the service, and differentially manages and controls the unloading of the data according to the data unloading tags; and the high-bandwidth parallel data exchange subsystem is used for data unloading I / O of data in the memory node. The cloud memory pool refers to that memory resources of a data center are interconnected through a network to form a unified memory aggregate, a memory service is constructed based on the cloud memory pool, and data is stored in the cloud memory pool by the memory service for data caching and data sharing. The memory service refers to a server-unaware memory service in cloud computing, and the application does not perceive the memory management in the service node, and the memory service is called to realize data caching and data sharing of the memory. The memory node management module comprises a data object segmentation unit, a multi-queue data storage unit, an access counter and a data unloading strategy algorithm, wherein the data object segmentation unit divides data transmitted into the memory node management module through the memory service into data blocks with a fixed size; the multi-queue data storage unit is composed of three least recently used queues, and the data blocks enter respective queues for storage according to the data unloading tags given by the memory service; the access counter counts the data access times in the multi-queue data storage unit; and the data unloading strategy algorithm controls the data to enter the parallel data exchange subsystem for data unloading according to the information of the access counter and the current system state. The memory service offline analysis module comprises a service pre-execution unit, a data access statistical unit and a memory service marking unit, wherein the service pre-execution unit calls and executes a service to be processed, the data access statistical unit counts the data read frequency of the service in the service execution unit, and the memory service marking unit allocates data unloading tags to the service according to the data read frequency in the service.

2. The cloud in-memory-pool service-aware multi-queue node memory management system according to claim 1, characterized in that, The parallel data exchange subsystem comprises a local storage I / O unit, a local storage, a cloud storage I / O unit and a cloud storage unit, wherein the local storage I / O unit and the cloud storage I / O unit cache data blocks from the memory node management module, the local storage I / O unit controls the unloading and acquisition of the data between the memory node and the local storage, and the cloud storage I / O unit controls the unloading and acquisition of the data between the memory node and the cloud storage unit.

3. The cloud in-memory-pool service-aware multi-queue node memory management system of claim 1, wherein, The offline stage preanalyzes the memory service in cloud computing and allocates data unloading tags, which are attached to the application through the memory service during online execution.

4. A method for service-aware multi-queue node memory management in a cloud memory pool based on the system of any of claims 1-3, characterized in that ​ The memory node management module adopts a hierarchical storage manner, and according to the data offloading label, data is respectively distributed to different least recently used queues, and when the memory resource reaches the upper limit, parallel data exchange is performed through a parallel data exchange subsystem to offload data to the cloud storage, and when the data is accessed again, the data is pre-fetched in parallel through the data exchange subsystem; The hierarchical storage refers to that the memory node cuts the application data after the service and stores the data in the form of data blocks, and differentiates the management of the application data according to the data offloading label, that is, three least recently accessed queues are maintained, which are an IED queue, an LED queue and an MED queue, after the data enters the cloud memory pool through the memory service, the data has the respective data offloading label, and enters the corresponding queue according to the respective data offloading label, the non-evictable data enters the IED queue, the low-evictable data enters the LED queue, and the high-evictable data enters the MED queue.

5. The multi-queue node memory management method of claim 4, wherein, The pre-analysis refers to that before the memory service based on the cloud memory pool is put into operation, the access frequency of the service data is counted, the service is classified according to the access frequency of the data passing through the service in the life cycle, and the corresponding data offloading label is assigned, and when the data of the service is stored in the cloud memory pool, the data offloading label of the service is used for the management of the memory node in the cloud memory pool; The classification refers to that the access frequency of the data calling the memory service based on the cloud memory pool in the life cycle is counted, and the data with an access frequency greater than or equal to 100 times is marked as non-evictable data (IED), which indicates that the data cannot be unloaded when the data is managed.

6. The multi-queue node memory management method of claim 4, wherein, The data offloading refers to that when the data is unloaded, the data in the LED queue and the MED queue is preferentially unloaded, and when the data in the two queues is unloaded and the memory still exceeds the limit, the data in the IED queue is further unloaded, and specifically, the data offloading label of the data is obtained, the unloading amount of the LED queue and the MED queue is calculated, the data in the LED queue and the MED queue is unloaded in turn, it is judged whether the data offloading is completed, and when the data offloading is not completed, the unloading amount of the IED queue is further calculated and the data in the IED queue is unloaded. The unloading amount of the LED queue and the MED queue is calculated in the following manner: the LED queue and the MED queue need to unload n size of data in total, the access frequency of the LED queue in the current stage is x, and the access frequency of the MED queue is y, then this time of unloading will unload n*y / (x+y) size of LED data and n*x / (x+y) size of MED data.

7. The multi-queue node memory management method of claim 4, wherein, The parallel data exchange refers to that when the data is exchanged, the data queue parallel and the data exchange path parallel are adopted to realize, wherein the data queue parallel refers to that the data in the IED queue is unloaded through the local storage, and the data in the LED queue and the MED queue is unloaded through the cloud storage; the data exchange path parallel refers to that the data in the IED queue under the I / O exchange unloading is unloaded through the local storage, and the data in the LED queue and the MED queue under the I / O exchange unloading is unloaded through the cloud storage. The local storage I / O refers to data exchange by using local high-speed storage and data unloading to local storage of the memory node. The cloud storage I / O refers to data exchange by using cloud storage and data unloading to the cloud storage.

8. The multi-queue node memory management method of claim 4, wherein, The memory node stores data in the form of data blocks by creating and reading the data blocks. The process of creating the data block comprises: i) calling a data block creation interface to create the data block; ii) when the creation fails, judging that the current system is performing data unloading, waiting for the data unloading to be completed, and then returning to step i); otherwise, entering step iii); iii) when the data block creation succeeds, the creation result is returned asynchronously, and whether the current memory capacity triggers data unloading is judged; if the data unloading is not triggered, the data block is directly returned; otherwise, entering step iv); iv) performing data unloading, and modifying the state to unloading completion after the data unloading is completed. The process of reading the data block comprises: i) calling a data block reading interface to read the data block; ii) when the data block is in the memory, the data block is directly returned; otherwise, entering step iii); iii) the data block is not currently in the memory, and the data is reloaded; iv) the data is reloaded into the memory, the data block creation interface is called to re-create the data block, and step i) of reading the data block is returned.

Images