An AI load-oriented server memory pool optimization management system and method

Abstract

This invention relates to the field of server memory pool optimization technology, and more particularly to a server memory pool optimization management system and method for AI workloads. It is applied to a cluster memory pool constructed by at least two servers through a virtualization platform, and includes the following steps: real-time monitoring of one or more virtual machines running in the cluster memory pool, collecting virtual machine performance metrics, and identifying the workload running on each virtual machine as a predefined AI workload type based on the performance metrics using a load classifier; for virtual machines identified as AI workloads, predicting their memory demand sequence within a future set time window based on their historical memory usage time-series data and the identified load type using a time-series prediction model; and performing dynamic resource scheduling according to the AI ​​load type and the predicted memory demand sequence. This invention achieves intelligent scheduling and efficient utilization of memory resources, realizes end-to-end automated management, and enables resource scheduling response times to reach the second level.

Description

Technical Field

[0001] This invention relates to a server memory pool optimization management system and method for AI workloads, belonging to the field of server memory pool optimization technology. Background Technology

[0002] Traditional server memory management uses a static allocation model, where the memory resources of each physical server are isolated from each other, forming "resource islands". This is problematic when running AI workloads (especially in scenarios that combine training and inference tasks).

[0003] This model exposes three core problems: First, low memory utilization, uneven load on each server, and idle memory cannot be used by urgently needed tasks; second, it cannot adapt to the dynamic nature of AI load, as the memory requirements of training tasks increase dramatically over time, while inference tasks require stable low latency guarantees, which is difficult to achieve with traditional fixed allocation methods; and third, slow operation and maintenance response, with expansion or migration requiring manual intervention and causing long-term service interruptions.

[0004] In small-scale clusters (such as 2-3 servers), resource constraints are particularly prominent, severely restricting the efficiency of AI research and development and deployment. Therefore, there is an urgent need to improve a server memory pool optimization management system and method for AI workloads to solve the above-mentioned problems. Summary of the Invention

[0005] The purpose of this invention is to provide a server memory pool optimization management system and method for AI workloads, in order to solve the problem that resource constraints are particularly prominent in small-scale clusters (such as 2-3 servers), which seriously restricts the efficiency of AI research and development and deployment.

[0006] To achieve the above objectives, the present invention provides the following technical solution:

[0007] A server memory pool optimization management system and method for AI workloads, applied to a cluster memory pool constructed from at least two servers via a virtualization platform, includes the following steps:

[0008] The system monitors one or more virtual machines running in the cluster memory pool in real time, collects the running metrics of the virtual machines, and identifies the workload running on each virtual machine as a predefined AI workload type based on the running metrics through a load classifier.

[0009] For virtual machines identified as AI workloads, based on their historical memory usage time series data and the identified workload type, a time series prediction model is used to predict their memory demand sequence within a set future time window.

[0010] Based on the AI ​​load type and the predicted memory requirement sequence, dynamic resource scheduling is performed, the scheduling including at least:

[0011] Configure differentiated memory allocation and reclamation strategies for the virtual machines based on their load types;

[0012] When it is predicted that the peak memory demand of a virtual machine will exceed the real-time available memory capacity of its physical server, based on the checkpoint awareness mechanism, it is migrated to another physical server with more abundant memory resources in the cluster memory pool during the checkpoint cycle of the AI ​​task.

[0013] Furthermore, the operational metrics include basic resource metrics, graphics processor-related metrics, and process and behavior characteristic metrics.

[0014] Furthermore, the basic resource metrics include the virtual machine's CPU utilization, memory usage and its instantaneous growth rate, memory access frequency, and paging rate;

[0015] The graphics processor-related metrics include graphics processor memory utilization, graphics processor computing core utilization, and data copy bandwidth between host system memory and graphics processor memory.

[0016] The process and behavior characteristics indicators include the process name and command line characteristics running within the virtual machine, specific system call patterns related to model training or inference, and input / output activity characteristics of periodic memory snapshots or model checkpoint files pointing to shared storage.

[0017] Furthermore, the process of identifying the virtual machine's workload as a predefined AI workload type using a load classifier is as follows:

[0018] Extract an n-dimensional feature vector F from the aforementioned performance metrics, where... The features include at least: the exponentially weighted moving average growth rate of memory usage, and the ratio of graphics processor memory usage to system memory usage. And binary feature vectors reflecting the existence of a specific AI framework runtime process. ;

[0019] The feature vector F is input into a multi-level classifier, which classifies each predefined AI workload type. Calculate an attribution probability score ,in k is the total number of load types;

[0020] pass Determine the final load type of the virtual machine, the predefined set of AI load types. It includes at least: model training load, model inference load, and model fine-tuning load.

[0021] Furthermore, in the intelligent scheduling and execution steps, configuring differentiated memory allocation and reclamation strategies specifically involves:

[0022] For virtual machines identified as model training loads, a "base working set locking + elastic buffer" strategy is adopted, allocating a large contiguous block of memory locked in physical memory as the base working set. And additionally configure a capacity of A recyclable elastic buffer;

[0023] in , This is a preset proportional coefficient. Save the frequency of checkpoints for this virtual machine;

[0024] For virtual machines identified as model inference workloads, a "high-guarantee reservation + scalable buffer" strategy is adopted to allocate a fixed amount of memory with high priority that is prohibited from being reclaimed by the memory balloon driver. At the same time, it is associated with a capacity of Short-term buffer memory pool, , The scaling factor is... Let t be the length of the inference request queue at time t.

[0025] Furthermore, the time series prediction model is a time series prediction model based on a long short-term memory network, and the execution flow of the time series prediction model is as follows:

[0026] At each prediction time step t, construct the input vector. , ,in This is a time-series segment of historical memory usage. Encoding periodic task events, This is the one-hot encoded vector of the AI ​​payload type c. ;

[0027] Input sequence The Long Short-Term Memory (LSTM) network is input, and the network is computed within time step t as follows:

[0028]

[0029] in, denoted by Sigmoid, and tanh denotes the hyperbolic tangent activation function; These are the weight matrices for the forget gate, input gate, cell state, and output gate, respectively. These are the bias vectors for the corresponding gates; This represents a vector concatenation operation; This represents element-wise multiplication;

[0030] After each prediction period, calculate the absolute prediction error at step T+1. Maintain an error sliding window Calculate the average error of this window: ,when At that time, the parameters of the Long Short-Term Memory network were adjusted using new observational data. Perform online gradient descent fine-tuning, where This is a preset dynamic threshold.

[0031] Furthermore, the virtual machine migration based on the checkpoint-aware mechanism specifically includes the following steps:

[0032] By monitoring the virtual machine's specific file writing patterns to a preset shared storage path, or by receiving standardized event notifications from inside the virtual machine, the start and end times of checkpoint save operations in model training tasks can be accurately identified.

[0033] When a virtual machine identified as a model training load needs to be migrated, the migration scheduler waits for or triggers a checkpoint save operation to complete, and then initiates the live migration of the virtual machine within the window after the checkpoint is persisted and before the start of the next training iteration.

[0034] Furthermore, the method also includes a closed-loop feedback and arbitration step:

[0035] Define the overall system performance index :

[0036] ;

[0037] in, This represents the average utilization of the cluster memory pool during the evaluation period. This indicates the average completion time for key AI tasks. This indicates the average duration of task downtime due to maintenance migration. The preset positive weighting coefficients, and satisfy the following conditions: ;

[0038] Continuously monitor the performance indicators When a decrease in the relative baseline value exceeds the tolerance limit is detected. When a critical task's performance indicator deteriorates beyond its threshold, it is considered to meet the requirement. or or When this occurs, an arbitration mechanism is automatically triggered. This arbitration mechanism includes: suspending the allocation of new elastic memory to low-priority virtual machines, performing proactive memory reclamation on specified virtual machines, or forcibly migrating some non-critical workloads to release resources; wherein, As the baseline performance value, , , This is a preset threshold.

[0039] Furthermore, it includes a monitoring and data acquisition module, an AI load analysis module, a memory demand prediction module, an intelligent scheduling engine, a resource optimizer, and a feedback arbitration module.

[0040] Furthermore, the monitoring and acquisition module is used to continuously collect performance indicator data of each virtual machine and physical host within the cluster;

[0041] The AI ​​load analysis module is connected to the monitoring and acquisition module and is used to identify the AI ​​workload type on the virtual machine based on the acquired indicator data.

[0042] The memory demand prediction module is connected to the AI ​​load analysis module and has a built-in time-series prediction model for predicting future memory demand sequences based on historical data and load types.

[0043] The intelligent scheduling engine is connected to the AI ​​load analysis module and the memory demand prediction module, respectively, and is used to generate and execute memory policy configuration and virtual machine migration instructions based on the load type and prediction results.

[0044] The resource optimizer works in conjunction with the intelligent scheduling engine to execute hardware-level collaborative optimization strategies.

[0045] The feedback arbitration module is used to monitor system performance and execute closed-loop feedback and arbitration logic.

[0046] The present invention has at least the following beneficial effects:

[0047] This invention significantly improves resource utilization by constructing an intelligent memory pool management system, and optimizes both AI task performance and reliability. Based on LSTM-based accurate prediction and checkpoint-aware migration technology, it shortens task interruption time caused by resource adjustments and ensures P99 latency for inference services. The degree of operation and maintenance automation is fundamentally improved, realizing full-link automation from load identification and prediction to scheduling, enabling resource management response to go from "manual hours" to "system seconds". Even in small-scale scenarios with 2-3 servers, it can efficiently coordinate mixed AI workloads such as training and inference. Attached Figure Description

[0048] Figure 1 This is a flowchart of a server memory pool optimization management method for AI workloads according to the present invention. Detailed Implementation

[0049] The following will describe in detail the implementation of this application with reference to the accompanying drawings and embodiments, so that the implementation process of how this application uses technical means to solve technical problems and achieve technical effects can be fully understood and implemented accordingly.

[0050] like Figure 1 As shown in the figure, the server memory pool optimization and management system for AI load provided in this embodiment further includes a monitoring and acquisition module, an AI load analysis module, a memory demand prediction module, an intelligent scheduling engine, a resource optimizer, and a feedback arbitration module.

[0051] Among them, the monitoring and acquisition module is used to continuously collect performance index data of each virtual machine and physical host in the cluster.

[0052] The AI ​​workload analysis module, connected to the monitoring and acquisition module, is used to identify the type of AI workload on the virtual machine based on the collected indicator data.

[0053] The memory demand prediction module is connected to the AI ​​load analysis module and has a built-in time-series prediction model to predict future memory demand sequences based on historical data and load types.

[0054] The intelligent scheduling engine is connected to the AI ​​load analysis module and the memory demand prediction module respectively. It is used to generate and execute memory policy configuration and virtual machine migration instructions based on the load type and prediction results.

[0055] The resource optimizer works in conjunction with the intelligent scheduling engine to execute hardware-level collaborative optimization strategies.

[0056] The feedback arbitration module is used to monitor system performance and execute closed-loop feedback and arbitration logic.

[0057] The system fully implements a server memory pool optimization management method for AI workloads in a modular manner, providing full-link automated memory management capabilities from monitoring, analysis, prediction to scheduling, optimization, and arbitration. It uses advanced AI technologies (such as LSTM) for resource management itself, which can intelligently adapt to dynamically changing AI workloads and significantly improve the resource utilization efficiency, task performance, and operation and maintenance automation level of small-scale AI infrastructure.

[0058] like Figure 1 As shown, the principle of the server memory pool optimization management method for AI workloads provided in this embodiment is as follows: applied to a cluster memory pool built by at least two servers through a virtualization platform, including the following steps:

[0059] Real-time monitoring of one or more virtual machines running in the cluster memory pool, collection of virtual machine operation metrics, and identification of the workload running on each virtual machine as a predefined AI workload type based on the operation metrics through a load classifier;

[0060] For virtual machines identified as AI workloads, based on their historical memory usage time series data and the identified workload type, a time series prediction model is used to predict their memory demand sequence within a set future time window.

[0061] Based on the AI ​​workload type and the predicted memory requirement sequence, perform dynamic resource scheduling, which includes at least:

[0062] Configure differentiated memory allocation and reclamation strategies for virtual machines based on load type;

[0063] When it is predicted that the peak memory demand of a virtual machine will exceed the real-time available memory capacity of its physical server, based on the checkpoint awareness mechanism, the virtual machine is migrated to another physical server with more abundant memory resources in the cluster memory pool during the checkpoint cycle of the AI ​​task. By performing fine-grained identification and demand prediction of AI load, proactive and intelligent scheduling of memory resources is achieved, which solves the problem of coarse memory management in traditional virtualization environments and inability to adapt to the dynamic memory demand of AI tasks, and significantly improves the overall memory utilization of the cluster and the execution efficiency of AI tasks.

[0064] The present invention further implements a scheme in which the operating indicators include basic resource indicators, graphics processor related indicators, and process and behavior characteristic indicators; the basic resource indicators include the virtual machine's CPU utilization, memory usage and its instantaneous growth rate, memory access frequency, and page swapping rate;

[0065] Graphics processor-related metrics include graphics processor memory utilization, graphics processor compute core utilization, and data copy bandwidth between host system memory and graphics processor memory.

[0066] Process and behavioral characteristic indicators include process names and command line characteristics running within the virtual machine, specific system call patterns related to model training or inference, and input / output activity characteristics of periodic memory snapshots or model checkpoint files pointing to shared storage. By collecting multi-dimensional and fine-grained runtime indicators, especially combining GPU indicators and process behavioral characteristics, a rich and reliable data foundation is provided for subsequent accurate AI load classification and memory demand prediction, overcoming the limitations of inaccurate identification caused by relying solely on traditional resource indicators.

[0067] In a further embodiment of this invention, the process of identifying the workload of a virtual machine as a predefined AI workload type using a workload classifier is as follows:

[0068] Feature extraction: Extract an n-dimensional feature vector F from the performance metrics, where The features include at least: the exponentially weighted moving average growth rate of memory usage, and the ratio of graphics processor memory usage to system memory usage. And binary feature vectors reflecting the existence of a specific AI framework runtime process. ;

[0069] Load scoring: The feature vector F is input into a multi-level classifier, which scores each predefined AI load type. Calculate an attribution probability score ,in k is the total number of load types;

[0070] Type determination: Pass Determine the final load type of the virtual machine, using a predefined set of AI load types. It includes at least: model training load, model inference load, and model fine-tuning load. The load identification method uses structured features and probabilistic models to accurately and automatically distinguish different types of AI workloads, providing a key basis for the subsequent implementation of differentiated memory management strategies and avoiding resource mismatch and performance loss caused by the "one-size-fits-all" strategy.

[0071] In a further embodiment of this invention, the configuration of differentiated memory allocation and reclamation strategies in the intelligent scheduling and execution steps specifically includes:

[0072] For virtual machines identified as model training loads, a "base working set locking + elastic buffer" strategy is adopted, allocating a large contiguous block of memory locked in physical memory as the base working set. And additionally configure a capacity of A recyclable elastic buffer;

[0073] in , This is a preset proportional coefficient. Save the frequency of checkpoints for this virtual machine;

[0074] For virtual machines identified as model inference workloads, a "high-guarantee reservation + scalable buffer" strategy is adopted to allocate a fixed amount of memory with high priority that is prohibited from being reclaimed by the memory balloon driver. At the same time, it is associated with a capacity of Short-term buffer memory pool, , The scaling factor is... The length of the inference request queue at time t;

[0075] Customized allocation strategies were designed to address the differences in memory access patterns between training and inference workloads; the elastic buffering strategy for training workloads balances memory usage and checkpoint efficiency; and the scalable buffering strategy for inference workloads effectively handles request bursts, improving memory utilization while ensuring latency.

[0076] The present invention further implements a scheme in which the time series prediction model is a time series prediction model based on a long short-term memory network, and the execution flow of the time series prediction model is as follows:

[0077] Input encoding: At each prediction time step t, construct the input vector. , ,in This is a time-series segment of historical memory usage. Encoding periodic task events, Let c be the one-hot encoded vector of the AI ​​payload type. ;

[0078] Temporal feature learning and prediction: This involves learning and predicting the input sequence. The input is a Long Short-Term Memory (LSTM) network, which is computed within time step t as follows:

[0079]

[0080] in, denoted by Sigmoid, and tanh denotes the hyperbolic tangent activation function; These are the weight matrices for the forget gate, input gate, cell state, and output gate, respectively. These are the bias vectors for the corresponding gates; This represents a vector concatenation operation; This represents element-wise multiplication;

[0081] Model adaptive calibration: After each prediction period, calculate the absolute prediction error at step T+1. Maintain an error sliding window Calculate the average error of this window: ,when At that time, the parameters of the Long Short-Term Memory network were analyzed using new observational data. Perform online gradient descent fine-tuning, where The preset dynamic threshold;

[0082] By leveraging the powerful time-series modeling capabilities of LSTM networks and combining load type information, high-precision, multi-step prediction of AI task memory requirements was achieved. The adaptive calibration mechanism ensures the long-term prediction accuracy of the model in dynamic environments, providing a reliable basis for forward-looking resource scheduling decisions and effectively avoiding resource shortages or excesses.

[0083] The present invention provides a further embodiment of the present invention, wherein virtual machine migration based on a checkpoint-aware mechanism specifically includes the following steps:

[0084] By monitoring the virtual machine's specific file writing patterns to a preset shared storage path, or by receiving standardized event notifications from inside the virtual machine, the start and end times of checkpoint save operations in model training tasks can be accurately identified.

[0085] When a virtual machine identified as a model training load needs to be migrated, the migration scheduler waits for or triggers a checkpoint save operation to complete, and then initiates the real-time migration of the virtual machine within the window after the checkpoint is persisted and before the start of the next training iteration.

[0086] By sensing the checkpoint cycle of the training task, the migration operation is aligned with the natural breakpoint of the task, which greatly reduces the number of dirty memory pages that need to be transferred during the migration process, thereby significantly shortening the migration time and reducing the performance interference and progress loss caused by migration to long-running AI training tasks.

[0087] A further embodiment of the present invention is described, wherein the method further includes a closed-loop feedback and arbitration step:

[0088] Define the overall system performance index :

[0089] ;

[0090] in, This represents the average utilization of the cluster memory pool during the evaluation period. This indicates the average completion time for key AI tasks. This indicates the average duration of task downtime due to maintenance migration. The preset positive weighting coefficients, and satisfy the following conditions: ;

[0091] Continuous monitoring of performance indicators When a decrease in the relative baseline value exceeds the tolerance limit is detected. When a critical task's performance indicator deteriorates beyond its threshold, it is considered to meet the requirement. or or When this occurs, an arbitration mechanism is automatically triggered. This mechanism includes: suspending the allocation of new elastic memory to low-priority virtual machines, performing proactive memory reclamation on specified virtual machines, or forcibly migrating some non-critical workloads to release resources; among these, As the baseline performance value, , , The preset threshold;

[0092] Through quantitative comprehensive performance indicators and automated arbitration mechanisms, continuous evaluation and autonomous optimization of system operation status are achieved. When system performance deviates from expectations or resource conflicts occur, corrective measures can be automatically taken to ensure the service quality of high-priority AI tasks and improve the robustness and self-healing capability of the entire memory pool management system.

[0093] Example 1

[0094] In a specific application scenario, taking a VMware cluster deployed on 3 physical servers (each configured with 512GB of memory and A100 GPU) as an example, the system successfully identified the virtual machine carrying the large language model training task through real-time monitoring, classified it as "model training load", and accurately predicted (prediction error <8%) based on the LSTM model that its memory requirement would increase from 128GB to 300GB after 4 hours.

[0095] Based on this, the intelligent scheduling engine triggers checkpoint-aware migration 2 hours in advance: during the low dirty page rate window after the model automatically saves the checkpoint (which takes 3 minutes), the virtual machine is migrated from the source server (with only 50GB of available memory remaining) to the target server. The migration process takes only 3.5 minutes and reduces business interruption time by 76.7%. At the same time, 64GB of fixed memory and 25GB of elastic buffer memory are guaranteed for the online inference service virtual machines in the same cluster to cope with sudden traffic.

[0096] After optimization, the overall memory utilization of the cluster increased from 60% to 85%, and the iteration time of key AI training tasks was reduced by 15%, effectively achieving efficient, automated, and differentiated memory resource scheduling for mixed AI workloads under limited hardware resources.

[0097] Example 2

[0098] In a specific application scenario, based on two physical servers: in a Proxmox virtualization cluster consisting of two servers each configured with 256GB of memory, the system identifies the image recognition model training task running on the first server (memory usage increases from 64GB at 5GB / hour) and the real-time inference service on the second server (requiring a stable 32GB of memory) through monitoring.

[0099] When the prediction module predicts that the training task will require 128GB of memory after 4 hours (exceeding the remaining capacity of the first server) based on the LSTM algorithm, the intelligent scheduling engine initiates checkpoint-aware migration after the phase checkpoint is saved (which takes 90 seconds). Taking advantage of the window period after the checkpoint when the dirty page rate drops below 3%, the training virtual machine is migrated to the second server. The migration process takes only 2 minutes and 15 seconds (70% shorter than traditional migration). At the same time, dynamic memory reclamation technology ensures 32GB of fixed memory and 8GB of elastic buffer memory for the inference service, ensuring that its P99 latency is always below 20ms.

[0100] After optimization, the overall memory utilization of the two servers increased from 58% to 82%, and the number of interruptions caused by insufficient memory in training tasks was reduced to zero, achieving efficient collaborative management of mixed AI workloads of training and inference under extremely limited hardware resources.

[0101] This invention achieves three core technological effects by constructing an intelligent memory pool management system: a significant leap in resource utilization, with the overall cluster memory utilization increasing from 50%-60% in the traditional mode to 82%-85%; dual optimization of AI task performance and reliability, based on LSTM-based accurate prediction (error <8%) and checkpoint-aware migration technology, reducing task interruption time caused by resource adjustments by more than 70%, and ensuring that the inference service P99 latency is less than 20ms; and a fundamental improvement in the degree of operation and maintenance automation, realizing end-to-end automation from load identification, prediction to scheduling, enabling resource management response to go from "manual hours" to "system seconds," and even in small-scale scenarios with 2-3 servers, it can efficiently coordinate mixed AI workloads such as training and inference.

[0102] The foregoing description illustrates and describes several preferred embodiments of the present invention. However, as previously stated, it should be understood that the present invention is not limited to the forms disclosed herein and should not be construed as excluding other embodiments. It can be used in various other combinations, modifications, and environments, and can be altered within the scope of the inventive concept described herein through the foregoing teachings or techniques or knowledge in related fields. Any modifications and variations made by those skilled in the art that do not depart from the spirit and scope of the present invention should be within the protection scope of the appended claims.

Claims

1. A server memory pool optimization management system and method for AI workloads, applied to a cluster memory pool constructed by at least two servers through a virtualization platform, characterized in that, Includes the following steps: The system monitors one or more virtual machines running in the cluster memory pool in real time, collects the running metrics of the virtual machines, and identifies the workload running on each virtual machine as a predefined AI workload type based on the running metrics through a load classifier. For virtual machines identified as AI workloads, based on their historical memory usage time series data and the identified workload type, a time series prediction model is used to predict their memory demand sequence within a set future time window. Based on the AI ​​load type and the predicted memory requirement sequence, dynamic resource scheduling is performed, the scheduling including at least: Configure differentiated memory allocation and reclamation strategies for the virtual machines based on their load types; When it is predicted that the peak memory demand of a virtual machine will exceed the real-time available memory capacity of its physical server, based on the checkpoint awareness mechanism, it is migrated to another physical server with more abundant memory resources in the cluster memory pool during the checkpoint cycle of the AI ​​task.

2. The server memory pool optimization management system and method for AI workloads according to claim 1, characterized in that: The operational metrics include basic resource metrics, graphics processor-related metrics, and process and behavior characteristic metrics.

3. The server memory pool optimization management system and method for AI workloads according to claim 2, characterized in that: The basic resource metrics include the virtual machine's CPU utilization, memory usage and its instantaneous growth rate, memory access frequency, and page swapping rate. The graphics processor-related metrics include graphics processor memory utilization, graphics processor computing core utilization, and data copy bandwidth between host system memory and graphics processor memory. The process and behavior characteristics indicators include the process name and command line characteristics running within the virtual machine, specific system call patterns related to model training or inference, and input / output activity characteristics of periodic memory snapshots or model checkpoint files pointing to shared storage.

4. The server memory pool optimization management system and method for AI workloads according to claim 1, characterized in that: The process of identifying the virtual machine's workload as a predefined AI workload type using a load classifier is as follows: Extract a multi-dimensional feature vector from the aforementioned operational metrics. The feature vector shall include at least the features reflecting the memory growth trend, the ratio of graphics processor to system memory usage, and the existence of runtime processes for a specific AI framework. The feature vector is input into a multi-level classifier, which calculates an attribution probability score for each predefined AI load type. Based on the maximum value in the attribution probability score, the final load type of the virtual machine is determined to be at least one of the following predefined types: model training load, model inference load, and model fine-tuning load.

5. The server memory pool optimization management system and method for AI workloads according to claim 1, characterized in that: In the intelligent scheduling and execution steps, configuring differentiated memory allocation and reclamation strategies specifically involves: For virtual machines identified as model training load, a "base working set locking + elastic buffer" strategy is adopted. A large block of memory locked in physical memory is allocated as the base working set, and an additional reclaimable elastic buffer with a capacity inversely proportional to the virtual machine's checkpoint saving frequency is configured. For virtual machines identified as model inference load, a "high-guarantee reservation + scalable buffer" strategy is adopted to allocate fixed-capacity memory with high priority that is prohibited from being reclaimed by the memory balloon driver, and at the same time associate it with a short-term buffer memory pool whose capacity dynamically expands and contracts with the length of the inference request queue.

6. The server memory pool optimization management system and method for AI workloads according to claim 1, characterized in that: The time series prediction model is a time series prediction model based on a long short-term memory network. The execution flow of the time series prediction model is as follows: At each prediction time step, an input vector is constructed, which is composed of a time-series segment of historical memory usage, periodic task event encoding, and a one-hot encoded vector of the AI ​​workload type. The input sequence is fed into the Long Short-Term Memory network. The network learns temporal features and outputs the final hidden state through gating mechanisms such as forget gate, input gate, cell state and output gate, as well as state update calculation. The hidden state is then mapped to a sequence of predicted memory requirements at multiple future time points through a fully connected layer. After each prediction period, the absolute error between the predicted value and the actual observed value is calculated, and an error sliding window is maintained. When the average error of the window exceeds a preset dynamic threshold, the parameters of the long short-term memory network are fine-tuned online using new observation data.

7. The server memory pool optimization management system and method for AI workloads according to claim 1, characterized in that: The virtual machine migration based on the checkpoint-aware mechanism specifically includes the following steps: By monitoring the virtual machine's specific file writing patterns to a preset shared storage path, or by receiving standardized event notifications from inside the virtual machine, the start and end times of checkpoint save operations in model training tasks can be accurately identified. When a virtual machine identified as a model training load needs to be migrated, the migration scheduler waits for or triggers a checkpoint save operation to complete, and then initiates the live migration of the virtual machine within the window after the checkpoint is persisted and before the start of the next training iteration.

8. The server memory pool optimization management system and method for AI workloads according to claim 1, characterized in that: The method also includes a closed-loop feedback and arbitration step: Calculate a comprehensive system performance index, which is the weighted sum of the average cluster memory utilization, the reciprocal of the average completion time of key AI tasks, and the reciprocal of the average downtime caused by migration, with the sum of the weighting coefficients being 1. The performance metrics are continuously monitored. When the decline relative to the baseline exceeds the tolerance limit, or when the average completion time or average downtime of critical tasks exceeds its threshold, an arbitration mechanism is automatically triggered. The arbitration mechanism includes: suspending the allocation of elastic memory to low-priority virtual machines, performing proactive memory reclamation, or forcibly migrating non-critical workloads.

9. A server memory pool optimization management system and method for AI workloads as described in any one of claims 1-9, characterized in that: It includes a monitoring and data acquisition module, an AI load analysis module, a memory demand prediction module, an intelligent scheduling engine, a resource optimizer, and a feedback arbitration module.

10. A server memory pool optimization management system and method for AI workloads according to claim 9, characterized in that: The monitoring and acquisition module is used to continuously collect performance index data of each virtual machine and physical host in the cluster. The AI ​​load analysis module is connected to the monitoring and acquisition module and is used to identify the AI ​​workload type on the virtual machine based on the acquired indicator data. The memory demand prediction module is connected to the AI ​​load analysis module and has a built-in time-series prediction model for predicting future memory demand sequences based on historical data and load types. The intelligent scheduling engine is connected to the AI ​​load analysis module and the memory demand prediction module, respectively, and is used to generate and execute memory policy configuration and virtual machine migration instructions based on the load type and prediction results. The resource optimizer works in conjunction with the intelligent scheduling engine to execute hardware-level collaborative optimization strategies. The feedback arbitration module is used to monitor system performance and execute closed-loop feedback and arbitration logic.

Images