Memory cold and hot scheduling method and device, electronic equipment and storage medium

Abstract

The invention provides a memory cold and hot scheduling method and device, electronic equipment and a storage medium, and relates to the technical field of computers. The method comprises the steps that state information of the hybrid memory system is obtained, and the state information comprises system page access information and system performance information; the state information is input into each preset rewarded intelligent agent, an action is output, the action comprises a page upgrading threshold value / a page degrading threshold value, and the intelligent agents are deployed in each memory node of the hybrid memory system; and according to the action, migrating the page meeting the threshold range to a node corresponding to the hybrid memory system. According to the method and the device, the performance information and the access information of the system are comprehensively considered, and based on reinforcement learning, respective agents are set for each node, so that the problem that the performance improvement of the system is not targeted due to the fact that page scheduling cannot face different nodes is solved.

Description

Background Art

[0002] With the rapid development of information technology, the demand for memory capacity in server systems is increasing day by day. Various new memory devices such as non-volatile memory and extended memory have emerged, greatly expanding the memory capacity of servers. However, compared with local memory, these remote memories have significant differences in read and write performance, often much lower than the speed of local memory. In application scenarios that require high-frequency data access, the response speed and throughput of the system may be severely affected.

[0003] In related technologies, by implementing memory page hot and cold scheduling in a hybrid memory system composed of local memory and remote extended memory, hot pages that are frequently accessed and cold pages that are infrequently accessed are identified and distinguished. Then, the hot pages are migrated to local memory with higher read and write performance, while the cold pages are placed in remote memory with lower performance. However, this way of local scheduling based on the access behavior of individual pages is prone to memory page migration oscillation when the page access changes violently. For example, when hot pages are frequently switched, it will cause frequent upgrades of remote pages and frequent downgrades of local pages, resulting in a large amount of resources consumed for page upgrade / downgrade, causing a decrease in system performance, and it is also impossible to perform flexible dynamic adjustment for various different nodes, resulting in it being difficult to improve the system performance in a targeted manner.

[0004] It should be noted that the information disclosed in the above background art section is only used to enhance the understanding of the background of the present disclosure, and therefore may include information that does not constitute the prior art known to those of ordinary skill in the art. Summary of the Invention

[0005] The present disclosure provides a method, apparatus, electronic device and storage medium for memory hot and cold scheduling, which at least overcome to a certain extent the problem that the system performance improvement is not targeted due to the page scheduling in related technologies not being applicable to different nodes.

[0006] Other features and advantages of the present disclosure will become apparent through the following detailed description, or will be learned in part through the practice of the present disclosure.

[0007] According to one aspect of the present disclosure, a method for memory hot and cold scheduling is provided, including: obtaining status information of a hybrid memory system, where the status information includes system page access information and system performance information; inputting the status information into agents with preset rewards respectively, and outputting actions, where the actions include page upgrade thresholds / page downgrade thresholds, and the agents are deployed on each memory node of the hybrid memory system; according to the actions, migrating pages that meet the threshold range to corresponding nodes of the hybrid memory system.

[0008] In some embodiments, the preset reward includes: determining a reward based on a single target; or, determining a reward based on multiple targets.

[0009] In some embodiments, determining the reward based on multiple targets includes: determining a reward based on the memory allocation ratio, the memory access ratio, and the migration information.

[0010] In some embodiments, determining the reward based on the memory allocation ratio, the memory access ratio, and the migration information includes: when the memory allocation ratio is higher than a first value, giving a first positive reward; when the memory allocation ratio is lower than the first value, giving a first negative reward; when the memory access ratio is higher than a second value, giving a second positive reward; when the memory access ratio is lower than the second value, giving a second negative reward; when a memory migration occurs, giving a third negative reward; determining the reward based on the first positive reward / first negative reward, the second positive reward / second negative reward, and the third negative reward.

[0011] In some embodiments, deploying the agent on each memory node of the system includes: deploying a page upgrade agent and / or a page downgrade agent for each memory node to control the hot and cold migration of pages of the memory node, wherein, deploying a page downgrade agent for the local memory node, deploying a page upgrade agent for the remote memory node, and deploying a page downgrade agent and a page upgrade agent for the intermediate state node.

[0012] In some embodiments, the method further includes: obtaining the state information after the action is executed, and calculating the reward generated by the action; in response to the end of the action, saving the quadruple data, and training the corresponding agent according to the quadruple data, wherein the quadruple data includes the state, the action, the next state, and the reward.

[0013] According to another aspect of the present disclosure, there is also provided a memory hot and cold scheduling device, including: a state information acquisition module, configured to acquire the state information of the hybrid memory system, wherein the state information includes system page access information and system performance information; an agent module, configured to input the state information into an agent with each preset reward, and output an action, wherein the action includes a page upgrade threshold / a page downgrade threshold, and the agent is deployed on each memory node of the hybrid memory system; a page migration module, configured to migrate pages within a threshold range to corresponding nodes of the hybrid memory system according to the action.

[0014] According to another aspect of the present disclosure, there is also provided an electronic device, including: a processor; and a memory, configured to store executable instructions of the processor; wherein, the processor is configured to execute the memory hot and cold scheduling method according to any one of the above by executing the executable instructions.

[0015] According to another aspect of the present disclosure, there is also provided a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the memory hot and cold scheduling method described in any one of the above is implemented.

[0016] According to another aspect of the present disclosure, there is also provided a computer program product, including a computer program, and when the computer program is executed by a processor, the memory hot and cold scheduling method described in any one of the above is implemented.

[0017] In the memory hot and cold scheduling method provided in the embodiments of the present disclosure, the state information of the hybrid memory system is obtained, where the state information includes system page access information and system performance information; the state information is input into agents with respective preset rewards, and actions are output, where the actions include page upgrade thresholds / page downgrade thresholds, and the agents are deployed on each memory node of the hybrid memory system; according to the actions, the pages that meet the threshold range are migrated to the corresponding nodes of the hybrid memory system. By comprehensively considering the system performance information and access information and based on reinforcement learning, the present disclosure sets respective agents for each node, and can make scheduling decisions that are more in line with the current system node and resource conditions, solving the problem that page scheduling cannot be targeted at different nodes, resulting in no targeted improvement in system performance.

[0018] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The drawings herein are incorporated into the specification and constitute a part of this specification, showing embodiments consistent with the present disclosure, and are used together with the specification to explain the principles of the present disclosure. Obviously, the drawings in the following description are only some embodiments of the present disclosure, and those of ordinary skill in the art can obtain other drawings based on these drawings without creative efforts.

[0020] Figure 1 A schematic diagram showing the structure of a memory hot and cold scheduling system in an embodiment of the present disclosure; Figure 2 A flowchart showing a memory hot and cold scheduling method in an embodiment of the present disclosure; Figure 3 A flowchart showing a specific example of a memory hot and cold scheduling method in an embodiment of the present disclosure; Figure 4 A flowchart showing another specific example of a memory hot and cold scheduling method in an embodiment of the present disclosure; Figure 5 A flowchart showing yet another specific example of a memory hot and cold scheduling method in an embodiment of the present disclosure; Figure 6Flowchart showing another specific example of a memory hot and cold scheduling method in an embodiment of the present disclosure; Figure 7 Schematic diagram of a memory hot and cold scheduling device in an embodiment of the present disclosure; Figure 8 Block diagram of a computer device in an embodiment of the present disclosure. Detailed implementation manners

[0021] Example embodiments will now be described more fully with reference to the accompanying drawings. However, the example embodiments can be implemented in various forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be more complete and comprehensive, and will fully convey the concept of the example embodiments to those skilled in the art. The features, structures, or characteristics described may be combined in any suitable manner in one or more embodiments.

[0022] In addition, the drawings are only schematic illustrations of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and thus their repeated description will be omitted. Some of the block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0023] For ease of understanding, before introducing the embodiments of the present disclosure, several terms involved in the embodiments of the present disclosure are first explained as follows: Central processing unit: CPU, Central Processing Unit; Input / output: IO, Input / Output; Non-uniform memory access: NUMA, Non-Uniform Memory Access; Cycles per instruction: CPI, Cycles Per Instruction; Non-volatile memory: NVM, Non Volatile Memory; High bandwidth memory: HBM, High Bandwidth Memory; Compute express link: CXL, Compute Express Link; Remote direct memory access: RDMA, Remote Direct Memory Access; Hybrid Memory System: New memory devices such as NVM, HBM, and remote memory provided through interconnection technologies such as CXL / RDMA can be accessed by processes in the form of remote nodes. The performance such as access latency and bandwidth of remote nodes is lower than that of local nodes. The hybrid memory system consists of the local node's memory and the remote node's memory; Memory Hot and Cold Scheduling: Due to the different access characteristics of memory pages between local and remote nodes, hot pages that need to be frequently accessed are placed on local nodes to improve process performance, and cold pages that are not frequently accessed are placed on remote nodes to save the memory space of local nodes; Reinforcement Learning Algorithm: An intelligent scheduling algorithm where the agent tries different actions in the environment and learns the optimal policy based on the rewards obtained; Deep Q-Network: DQN, Deep Q-Network; Deep Deterministic Policy Gradient: DDPG, Deep Deterministic Policy Gradient; Proximal Policy Optimization: PPO, Proximal Policy Optimization; Perf: Performance Events, a comprehensive performance analysis tool built into the Linux kernel; Psi: Pressure Stall Information, pressure stall information, a resource pressure monitoring mechanism provided by the Linux kernel; Vmstat: Virtual Memory Statistics, a command-line tool in the Linux system for monitoring the usage of system resources; Numastat: A performance analysis tool in the Linux system for monitoring the memory allocation and usage of each node under the NUMA architecture.

[0024] The following will describe in detail the specific implementation manners of the embodiments of the present disclosure with reference to the accompanying drawings.

[0025] Figure 1 The exemplary application system architecture diagram to which the memory hot and cold scheduling method in the embodiments of the present disclosure can be applied is shown. As Figure 1 shown, the system architecture may include a local memory 101, a remote memory 102, a system performance monitor 103, a page access scanner 104, a local page demotion agent 105, a remote page promotion agent 106, and a page scheduler 107.

[0026] Among them, the page access scanner 104 samples and scans the access bits of the page table entries of the pages on each node (local memory 101 and remote memory 102) to estimate the access conditions of the pages.

[0027] The system performance monitor 103 infers the degradation and upgrade thresholds by inputting the status such as the page access conditions and system performance metrics of the system into the neural network of the agent. Through the status and reward feedback of the executed actions, the neural network / value table of the agent is updated to make the output reward iterate in the increasing direction.

[0028] The page scheduler 107 receives the degradation thresholds / upgrade thresholds transmitted by the local page degradation agent 105 and the remote page upgrade agent 106 to set the page upgrade threshold and degradation threshold, and migrates the pages that meet the upgrade / degradation thresholds.

[0029] The local page degradation agent and the remote page upgrade agent are agents of reinforcement learning. For example, the neural network algorithm structure (such as DQN, PPO, DDPG algorithms) and the value table (Q-Learning algorithm).

[0030] Those skilled in the art can know that Figure 1 the numbers of the local memory, remote memory, system performance monitor, page access scanner, local page degradation agent, remote page upgrade agent, and page scheduler in

[0031] Figure 2 shows a flowchart of a memory hot and cold scheduling method in an embodiment of the present disclosure. As Figure 2 shown, the memory hot and cold scheduling method provided in the embodiment of the present disclosure includes the following steps: S202, obtain the status information of the hybrid memory system, where the status information includes system page access information and system performance information.

[0032] It should be noted that the above-mentioned hybrid memory system can be a system including local storage and remote storage. Among them, the above-mentioned local storage can be dynamic random access memory, solid-state drive, flash memory, etc. The above-mentioned remote storage can be non-volatile memory, network-attached storage, storage area network, etc. The above-mentioned status information can be information reflecting the overall situation of the system memory. The above-mentioned system page access information can be information about the page access situation in the system. For example, the page access situation in the system is monitored and evaluated through a page access scanner, and the access frequency and popularity of pages are estimated by sampling and scanning the access bits of page table entries of pages on each node. The smaller the scanning interval time and the fewer the sampled pages, the higher the accuracy of the obtained memory access information. The above-mentioned system performance information can be system performance index information. For example, various performance indicators of the system are comprehensively obtained and evaluated through a system performance monitor. For example, the number of cycles per instruction, cache misses and other micro-architecture level performance indicators are obtained through the Perf tool; the system resource pressure is monitored through the Psi tool, and the blocking indicators of CPU, input / output and memory are obtained to understand the pressure situation of the system on different resources; the memory usage trend is monitored through the Vmstat tool, and the information of page indicators such as memory utilization rate and memory allocation rate is obtained; the memory usage situation of each NUMA node is obtained through the Numastat tool, including statistical information of operations such as allocation, release and migration. It should be noted that NUMA consists of multiple nodes, and each node contains a set of processors (or cores) and local memory. Through Numastat, the memory access mode of NUMA can be optimized, and the latency and overhead caused by cross-node memory access can be reduced.

[0033] S204. Input the status information into each intelligent agent with a preset reward, and output an action, where the action includes a page upgrade threshold / page downgrade threshold, and the intelligent agents are deployed on each memory node of the hybrid memory system.

[0034] It should be noted that the above-mentioned reward can be the feedback obtained by the intelligent agent after taking an action in the environment, which is related to the optimization goal of the system and is used to guide the intelligent agent to learn and maximize the long-term cumulative reward. For example, the immediate feedback obtained by the intelligent agent from the environment after taking a certain action under the obtained status information. This feedback can be positive (reward) or negative (punishment), and is used to guide the learning process of the intelligent agent, so that the intelligent agent gradually tends to adopt a behavior strategy that can obtain a higher cumulative reward. The above-mentioned page upgrade threshold can be the upgrade threshold of the remote memory node. For example, when the page upgrade threshold is met, the page is upgraded to a hot page and adjusted from the remote memory node to the local memory node. The above-mentioned page downgrade threshold can be the page downgrade threshold of the local node. For example, when the page downgrade threshold is met, the page is downgraded to a cold page and the page is adjusted from the local memory node to the remote memory node.

[0035] In an example of the present disclosure, the above-mentioned preset reward includes: determining the reward according to a single target; or, determining the reward according to multiple targets.

[0036] For example, taking the optimization target as the reward value. For example, when it is necessary to improve the local memory utilization rate, the local memory utilization rate is taken as the reward; when it is necessary to reduce the number of page ins and outs, the reduction amount of the number of page ins and outs is taken as the reward.

[0037] In an example of the present disclosure, determining the reward according to multiple targets includes: determining the reward according to the memory allocation ratio, the memory access ratio, and the migration information.

[0038] For example, by integrating various metrics to obtain a comprehensive reward to improve the overall system performance. For instance, a weighted sum is performed on the memory access ratio (the ratio of local page accesses in the overall accesses) and the memory allocation ratio (the ratio of local node memory allocation in the overall memory allocation), and the number of pages migrated each time is introduced as a negative reward (frequent migration will cause system performance loss) to serve as the comprehensive reward.

[0039] S206. According to the action, migrate the pages that meet the threshold range to the corresponding node of the hybrid memory system.

[0040] It should be noted that the above-mentioned action can be the action of the agent. For example, the actions that the page scheduler can take are the action space. The access threshold of the page can be selected as the action. For example, the upgrade action can be set to perform an upgrade operation when the page is accessed more than N times (N is a positive integer) within the sampling time. For downgrading, it can be set that if the page has not been accessed for consecutive M hours (M is a positive number), it is considered a cold page and is downgraded. The above-mentioned migration can be the cold and hot migration of the page.

[0041] The above-mentioned page upgrade threshold can be the upgrade threshold of the remote memory node. For example, when the page upgrade threshold is met, the page is upgraded to a hot page and adjusted from the remote memory node to the local memory node. The above-mentioned page downgrade threshold can be the page downgrade threshold of the local node. For example, when the page downgrade threshold is met, the page is downgraded to a cold page and the page is adjusted from the local memory node to the remote memory node.

[0042] By comprehensively considering the performance information and access information of the system and based on reinforcement learning, the present disclosure sets respective agents for each node, enabling more targeted scheduling decisions that are more in line with the current system node and resource conditions, and solving the problem that page scheduling is not node-oriented, resulting in no targeted improvement in system performance.

[0043] In an embodiment of the present disclosure, as Figure 3As shown in the figure, in the memory cold and hot scheduling method provided in the embodiments of the present disclosure, determining the reward according to the memory allocation ratio, memory access ratio, and migration information includes the following steps. By giving feedback to the agent at each step and avoiding reward conflicts, the agent can be guided to learn faster: S302: When the memory allocation ratio is higher than the first value, give the first positive reward; S304: When the memory allocation ratio is lower than the first value, give the first negative reward; S306: When the memory access ratio is higher than the second value, give the second positive reward; S308: When the memory access ratio is lower than the second value, give the second negative reward; S310: When a memory migration occurs, give the third negative reward; S312: Determine the reward according to the first positive reward / first negative reward, second positive reward / second negative reward, and third negative reward.

[0044] It should be noted that whenever a memory migration occurs, a negative reward (migration penalty) is given. For example, the migration penalty can be adjusted according to the amount of data migrated, the frequency of migration, and the impact of migration on system performance.

[0045] In an embodiment of the present disclosure, as Figure 4 shown in the figure, in the memory cold and hot scheduling method provided in the embodiments of the present disclosure, the agent is deployed on each memory node of the system through the following steps. Through the collaborative work of each agent, the system can dynamically adjust the pages and improve the overall system performance: S402: Deploy a page upgrade agent and / or a page downgrade agent for each memory node to control the cold and hot migration of pages in the memory node. Among them, deploy a page downgrade agent for the local memory node, deploy a page upgrade agent for the remote memory node, and deploy a page downgrade agent and a page upgrade agent for the intermediate state node.

[0046] In an embodiment of the present disclosure, as Figure 5 shown in the figure, in the memory cold and hot scheduling method provided in the embodiments of the present disclosure, the agent is updated through the following steps. The complete process of the agent's interaction with the environment is recorded by quadruple data, enabling the agent to continuously improve its strategy through learning and optimization: S502: Obtain the state information after the action is executed and calculate the reward generated by the action; S504: After the action ends, save the quadruple data and train the corresponding agent according to the quadruple data. Among them, the quadruple data includes the state, action, next state, and reward.

[0047] The present disclosure provides a stable foundation for the training of an agent through the saved quadruple data. Even if the environment changes or fluctuations occur during the training process, the agent can rely on the saved quadruple data to maintain the stability and continuity of learning.

[0048] In one embodiment of the present disclosure, as Figure 6 shown, the flowchart of the hybrid memory scheduling process combined with the agent update process in the memory hot and cold scheduling method provided in the embodiment of the present disclosure.

[0049] S601: Obtain system status information (system performance metrics, page access conditions, node status), save the system status information in tuple information, and input it into the agent; S6021: Each time the status information is obtained, calculate the corresponding reward value according to the designed reward mechanism, and save the reward value in the tuple information; S6022: Obtain the tuple information saved in the tuple information, where the tuple information includes (status, action, next status, reward); S6023: Train the agent neural network according to the tuple information and update the parameters; S6024: The agent infers the page demotion threshold of the local node and the upgrade threshold of the remote node based on the system status information, and saves the new thresholds in the tuple information; S603: The page scheduler obtains the new page upgrade / demotion threshold and adjusts it to the new threshold; S604: Return to S601 to obtain the system status information under the new threshold.

[0050] Among them, the hybrid memory scheduling process includes: S601, S6024, S603, and S604.

[0051] The agent update process includes: S6021, S6022, and S6023.

[0052] Based on reinforcement learning, the present disclosure infers the hot and cold thresholds of page scheduling by the reinforcement learning agent through comprehensive information such as system memory page access, performance metrics, and node status. And through reward feedback, the agent is updated in real time. By comprehensively considering the page access status and system performance metrics, the system status is fully reflected, and the scheduling reaches the global optimum; the reward is flexibly designed to achieve different tuning goals and adapt to different types of nodes; the agent algorithm is updated in real time according to the status and reward feedback and automatically adapts to node changes.

[0053] Based on the same inventive concept, an embodiment of the present disclosure also provides a memory cold and hot scheduling device as described in the following embodiments. Since the principle of problem-solving in this device embodiment is similar to that of the above method embodiment, the implementation of this device embodiment can refer to the implementation of the above method embodiment, and repeated parts will not be elaborated.

[0054] Figure 7 The following shows a schematic diagram of a memory cold and hot scheduling device in an embodiment of the present disclosure. As Figure 7 shown, the device includes: a status information acquisition module 71, an agent module 72, and a page migration module 73.

[0055] Among them, the status information acquisition module 71 is used to acquire the status information of the hybrid memory system, where the status information includes system page access information and system performance information; The agent module 72 is used to input the status information into agents with various preset rewards and output actions, where the actions include page upgrade thresholds / page downgrade thresholds, and the agents are deployed on each memory node of the hybrid memory system; The page migration module 73 is used to migrate pages within the threshold range to the corresponding nodes of the hybrid memory system according to the actions.

[0056] In an example of the present disclosure, the preset rewards in the above agent module include: determining rewards according to a single target; or, determining rewards according to multiple targets.

[0057] In an example of the present disclosure, the above agent module is further used to determine rewards according to the memory allocation ratio, memory access ratio, and migration information.

[0058] In an example of the present disclosure, the above agent module is further used to give a first positive reward when the memory allocation ratio is higher than a first value, give a first negative reward when the memory allocation ratio is lower than the first value, give a second positive reward when the memory access ratio is higher than a second value, give a second negative reward when the memory access ratio is lower than the second value, give a third negative reward when memory migration occurs, and determine rewards according to the first positive reward / first negative reward, second positive reward / second negative reward, and third negative reward.

[0059] In an example of the present disclosure, the above agent module is further used to deploy a page upgrade agent and / or a page downgrade agent for each memory node to control the cold and hot migration of pages in the memory node. Among them, a page downgrade agent is deployed for the local memory node, a page upgrade agent is deployed for the remote memory node, and a page downgrade agent and a page upgrade agent are deployed for the intermediate state node.

[0060] In an example of the present disclosure, the above-mentioned agent module is further configured to obtain the state information after the execution of the action and calculate the reward generated by the action; after each action, save the quadruple data, and train the corresponding agent according to the quadruple data, where the quadruple data includes the state, action, next state, and reward.

[0061] It should be noted here that the above-mentioned state information acquisition module 71, agent module 72, and page migration module 73 correspond to S202-S206 in the method embodiment. The examples and application scenarios implemented by the above modules and the corresponding steps are the same, but are not limited to the content disclosed in the above method embodiment. It should be noted that the above modules, as part of the device, can be executed in a computer system such as a set of computer-executable instructions.

[0062] Those skilled in the art can understand that various aspects of the present disclosure can be implemented as a system, method, or program product. Therefore, various aspects of the present disclosure can be specifically implemented in the following forms, namely: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or an implementation combining hardware and software aspects, which can be collectively referred to as "circuit", "module", or "system" here.

[0063] The following refers to Figure 8 to describe the electronic device 800 according to this embodiment of the present disclosure. Figure 8 The electronic device 800 shown is only an example and should not impose any limitation on the functions and usage scope of the embodiments of the present disclosure.

[0064] As Figure 8 shown, the electronic device 800 is presented in the form of a general-purpose computing device. The components of the electronic device 800 may include, but are not limited to: the above-mentioned at least one processing unit 810, the above-mentioned at least one storage unit 820, and a bus 830 connecting different system components (including the storage unit 820 and the processing unit 810).

[0065] Among them, the storage unit stores program code, and the program code can be executed by the processing unit 810, so that the processing unit 810 executes the steps according to various exemplary embodiments of the present disclosure described in the above "Exemplary Method" section of this specification.

[0066] For example, the processing unit 810 may execute the following steps of the above method embodiment: obtain the state information of the hybrid memory system, where the state information includes system page access information and system performance information; input the state information into agents with various preset rewards, and output actions, where the actions include page upgrade thresholds / page downgrade thresholds, and the agents are deployed on each memory node of the hybrid memory system; according to the actions, migrate the pages that meet the threshold range to the corresponding nodes of the hybrid memory system.

[0067] For example, the processing unit 810 executes the preset rewards in the above method embodiments, including: determining rewards according to a single target; or, determining rewards according to multiple targets.

[0068] For example, the processing unit 810 may execute the following steps of the above method embodiments: determining rewards according to the memory allocation ratio, memory access ratio, and migration information.

[0069] For example, the processing unit 810 may execute the following steps of the above method embodiments: when the memory allocation ratio is higher than a first value, giving a first positive reward; when the memory allocation ratio is lower than the first value, giving a first negative reward; when the memory access ratio is higher than a second value, giving a second positive reward; when the memory access ratio is lower than the second value, giving a second negative reward; when memory migration occurs, giving a third negative reward; determining rewards according to the first positive reward / first negative reward, second positive reward / second negative reward, and third negative reward.

[0070] For example, the processing unit 810 may execute the following steps of the above method embodiments: deploying a page upgrade agent and / or a page downgrade agent for each memory node to control the page cold and hot migration of the memory node, wherein a page downgrade agent is deployed for the local memory node, a page upgrade agent is deployed for the remote memory node, and a page downgrade agent and a page upgrade agent are deployed for the intermediate state node.

[0071] For example, the processing unit 810 may execute the following steps of the above method embodiments: obtaining the state information after the action execution and calculating the reward generated by the action; after each action, saving the quadruple data and training the corresponding agent according to the quadruple data, wherein the quadruple data includes state, action, next state, and reward.

[0072] The storage unit 820 may include a readable medium in the form of a volatile storage unit, such as a random access storage unit (RAM) 8201 and / or a cache 8202, and may further include a read-only storage unit (ROM) 8203.

[0073] The storage unit 820 may further include a program / utility 8204 having a set (at least one) of program modules 8205. Such program modules 8205 include, but are not limited to: an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include the implementation of a network environment.

[0074] The bus 830 can represent one or more of several types of bus structures, including a memory unit bus or a memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of the various bus structures.

[0075] The electronic device 800 can also communicate with one or more external devices 840 (such as a keyboard, a pointing device, a Bluetooth device, etc.), can also communicate with one or more devices that enable a user to interact with the electronic device 800, and / or can communicate with any device that enables the electronic device 800 to communicate with one or more other computing devices (such as a router, a modem, etc.). Such communication can be carried out through the input / output (I / O) interface 850. Moreover, the electronic device 800 can also communicate with one or more networks (such as a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) through the network adapter 860. As shown in the figure, the network adapter 860 communicates with other modules of the electronic device 800 through the bus 830. It should be understood that, although not shown in the figure, other hardware and / or software modules can be used in conjunction with the electronic device 800, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, etc.

[0076] Through the description of the above embodiments, those skilled in the art can easily understand that the example embodiments described herein can be implemented through software, or can be implemented in a manner of software combined with necessary hardware. Therefore, the technical solutions according to the embodiments of the present disclosure can be embodied in the form of a software product, and the software product can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to the embodiments of the present disclosure.

[0077] In particular, according to the embodiments of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer program product, and the computer program product includes: a computer program, and when the computer program is executed by a processor, it implements the above-mentioned memory cold and hot scheduling method.

[0078] In an exemplary embodiment of the present disclosure, there is also provided a computer-readable storage medium, which may be a readable signal medium or a readable storage medium. A program product capable of implementing the above methods of the present disclosure is stored thereon. In some possible implementation manners, various aspects of the present disclosure may also be implemented in the form of a program product, which includes program code. When the program product runs on a terminal device, the program code is used to cause the terminal device to execute the steps according to various exemplary embodiments of the present disclosure described in the "Exemplary Method" section of this specification.

[0079] More specific examples of the computer-readable storage medium in the present disclosure may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0080] In the present disclosure, the computer-readable storage medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries the readable program code. Such a propagated data signal may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The readable signal medium may also be any readable medium other than the readable storage medium, and this readable medium may send, propagate, or transmit a program used by or in combination with an instruction execution system, apparatus, or device.

[0081] Optionally, the program code contained on the computer-readable storage medium may be transmitted by any appropriate medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination of the above.

[0082] In specific implementation, the program code for executing the operations of the present disclosure may be written in any combination of one or more programming languages. The programming languages include object-oriented programming languages such as Java, C++, etc., and also include conventional procedural programming languages such as the "C" language or similar programming languages. The program code may be executed entirely on the user's computing device, partially on the user's device, executed as an independent software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user's computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (for example, by using an Internet service provider to connect through the Internet).

[0083] It should be noted that although several modules or units of the device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of the two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0084] In addition, although the steps of the method in the present disclosure are described in a specific order in the drawings, this does not require or imply that these steps must be executed in that specific order, or that all the steps shown must be executed to achieve the desired result. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step for execution, and / or one step may be decomposed into multiple steps for execution, etc.

[0085] Through the description of the above embodiments, those skilled in the art can easily understand that the exemplary embodiments described herein can be implemented by software, or by a combination of software and necessary hardware. Therefore, the technical solutions according to the embodiments of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, USB flash drive, mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, server, mobile terminal, or network device, etc.) to execute the method according to the embodiments of the present disclosure.

[0086] After considering the specification and practicing the invention disclosed herein, those skilled in the art will readily conceive of other embodiments of the present disclosure. The present disclosure is intended to cover any variations, uses, or adaptations of the present disclosure, which follow the general principles of the present disclosure and include common general knowledge or conventional technical means in the technical field not disclosed in the present disclosure. The specification and examples are only regarded as exemplary, and the true scope and spirit of the present disclosure are pointed out by the appended claims.

Claims

1. A memory cold and hot scheduling method, characterized in that including: Obtain the status information of the hybrid memory system, where the status information includes system page access information and system performance information; Input the status information into agents with various preset rewards, and output actions, where the actions include page upgrade thresholds / page downgrade thresholds, and the agents are deployed on each memory node of the hybrid memory system; According to the actions, migrate the pages that meet the threshold range to the corresponding nodes of the hybrid memory system.

2. The memory hot and cold scheduling method according to claim 1, characterized in that The preset rewards include: Determine the reward according to a single target; or Determine the reward according to multiple targets.

3. The memory hot and cold scheduling method according to claim 2, wherein The determining the reward according to multiple targets includes: Determine the reward according to the memory allocation ratio, memory access ratio, and migration information.

4. The memory hot and cold scheduling method according to claim 3, wherein The determining the reward according to the memory allocation ratio, memory access ratio, and migration information includes: When the memory allocation ratio is higher than the first value, give the first positive reward; When the memory allocation ratio is lower than the first value, give the first negative reward; When the memory access ratio is higher than the second value, give the second positive reward; When the memory access ratio is lower than the second value, give the second negative reward; When memory migration occurs, give the third negative reward; Determine the reward according to the first positive reward / first negative reward, second positive reward / second negative reward, and third negative reward.

5. The memory hot and cold scheduling method according to claim 1, characterized in that The agents being deployed on each memory node of the system includes: Deploy a page upgrade agent and / or a page downgrade agent for each memory node to control the hot and cold migration of pages in the memory node. Among them, deploy a page downgrade agent for the local memory node, deploy a page upgrade agent for the remote memory node, and deploy a page downgrade agent and a page upgrade agent for the intermediate state node.

6. The memory hot and cold scheduling method according to any one of claims 1 to 5, characterized in that The method further includes: Obtain the status information after the execution of the action, and calculate the reward generated by the action; In response to the end of the action, save the quadruple data, and train the corresponding agent according to the quadruple data, where the quadruple data includes state, action, next state, and reward.

7. A memory hot and cold scheduling device, characterized in that including: A status information acquisition module, configured to obtain the status information of the hybrid memory system, where the status information includes system page access information and system performance information; An agent module, configured to input the status information into agents with various preset rewards and output actions, where the actions include page upgrade thresholds / page downgrade thresholds, and the agents are deployed on each memory node of the hybrid memory system; A page migration module, configured to migrate the pages that meet the threshold range to the corresponding nodes of the hybrid memory system according to the actions.

8. An electronic device, characterized in that, including: A processor; and A memory, configured to store the executable instructions of the processor; wherein, the processor is configured to execute the memory hot and cold scheduling method according to any one of claims 1 to 6 by executing the executable instructions.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the memory hot and cold scheduling method according to any one of claims 1 to 6.

10. A computer program product, comprising: A computer program or instruction, characterized in that when the computer program or instruction is executed by the processor, it implements the memory hot and cold scheduling method according to any one of claims 1 to 6.

Images