A method for inhibiting preemption of an NPU training task and related equipment

Abstract

The application belongs to the technical field of NPU computing resource optimization, and particularly relates to an NPU training task preemption suppression method and related equipment. The method first collects historical execution portraits and real-time running state features of to-be-scheduled training tasks, generates a prediction sequence of memory occupancy in a specified future time window through a memory occupancy time sequence prediction model of online inference, synchronously collects core performance indicators such as the time delay and throughput of online inference services in a shared computing power pool, generates dynamic SLA priority labels of training tasks in combination with preset static business weights, carries out pre-distribution simulation and preemption risk prediction based on the prediction results and the priority labels, outputs a memory conflict task set, performs bandwidth throttling, computing power quota contraction and other resource suppression operations on low-priority tasks, and performs memory page locking reservation on high-priority tasks. Finally, model incremental fine-tuning and evaluation parameter closed-loop optimization are completed based on actual running indicators, and the comprehensive utilization rate of computing power resources is improved.

Description

Technical Field

[0001] This invention belongs to the field of NPU computing power resource optimization technology, specifically involving an NPU training task preemption suppression method and related equipment. Background Technology

[0002] With the development of artificial intelligence technology, NPUs are widely used in scenarios such as cloud computing, autonomous driving, smart healthcare, and smart terminals. In practical applications, the same NPU cluster typically runs model training tasks and online inference tasks simultaneously. Training tasks are characterized by large memory consumption, significant fluctuations, and sudden spikes in performance, while online inference tasks have strict Service Level Agreement (SLA) requirements for response latency, throughput, and service continuity. When the memory demand of training tasks suddenly increases, it may consume the memory resources required by online inference tasks, leading to increased latency, decreased throughput, or even service interruption.

[0003] Existing NPU resource scheduling methods mainly include static priority allocation, preemption after real-time monitoring, fixed GPU memory reservation, and node isolation between training and inference tasks. Static priority allocation struggles to dynamically adjust resource allocation based on the real-time load and SLA status of the inference service; passive preemption based on real-time monitoring typically only takes action after GPU memory shortages occur, resulting in delayed responses, and directly terminating or pausing training tasks wastes computational resources; fixed GPU memory reservation is ill-suited to the dynamic changes in task GPU memory usage, easily leading to idle or insufficient GPU memory; while node isolation can reduce resource conflicts, it reduces NPU resource utilization and lacks the flexibility of resource sharing.

[0004] Therefore, existing technologies struggle to simultaneously identify memory conflict risks in advance while ensuring SLA guarantees for online inference services, continuous operation of training tasks, and NPU resource utilization. It is necessary to propose an NPU training task preemption suppression technology that can dynamically schedule tasks based on changes in memory usage and the status of inference services. Summary of the Invention

[0005] The technical problem to be solved by the present invention is to provide an NPU training task preemption suppression method and related equipment to address the shortcomings of the prior art, thereby solving the technical problems of the prior art which generally adopts static memory quota pre-allocation and task priority fixing.

[0006] The objective of this invention is achieved through the following technical solutions: In a first aspect, the present invention provides a method for suppressing preemption in NPU training tasks, comprising: The historical execution data and current status data of the NPU training task to be scheduled are obtained, and the memory usage prediction data of the NPU training task in the future time window is output through a pre-deployed real-time memory usage prediction model. Collect real-time performance metrics of the NPU training task that share computing resources and online inference services, and calculate the dynamic SLA priority of the NPU training task based on the real-time performance metrics and the static business importance preset by the task. Based on the predicted memory usage data and the dynamic SLA priority, resource allocation is simulated and preemption risk assessment is performed within the future time window, and the risk judgment result and the set of target tasks where memory preemption conflict occurs are output. Based on the risk assessment results, resource suppression is performed on lower priority tasks in the target task set, and memory resource locking is performed on higher priority tasks in the target task set. The actual execution metrics of the NPU training task and the actual achievement metrics of the online inference service are collected. The real-time memory usage prediction model is updated based on the actual execution metrics, and the parameters of the preemption risk assessment are optimized in a closed loop based on the actual achievement metrics.

[0007] As a further improvement of the present invention, the step of acquiring historical execution data and current status data of the NPU training task to be scheduled and running, and outputting the predicted memory usage data of the NPU training task within a future time window through a pre-deployed real-time memory usage prediction model, specifically includes: Historical execution trajectory data is collected from the NPU cluster, and a regular sequence is obtained by resampling with a preset time window. Feature vectors are then extracted, including the current memory value, the mean and standard deviation of memory within the historical time window, the memory growth rate, and the amplitude of the periodic component extracted by Fourier transform. The feature vector is input into the attention-based time series prediction model. The encoder maps the historical feature sequence into a context vector, and the decoder generates memory usage prediction data for future time steps.

[0008] As a further improvement of the present invention, the step of calculating the dynamic SLA priority of the NPU training task based on the real-time performance indicators and the static business importance preset by the task specifically includes: The average request latency and throughput of the online inference service are compared with the corresponding preset SLA thresholds to calculate the SLA risk index that combines the deviation of the current performance from the preset SLA thresholds. When the SLA risk index exceeds the preset warning threshold, the dynamic SLA impact factor is increased through an exponential function; When the SLA risk index is lower than the warning threshold, the dynamic SLA impact factor is reduced by a decay function. The dynamic SLA priority of the NPU training task is calculated based on the static business importance and the dynamic SLA impact factor.

[0009] As a further improvement of the present invention, the step of simulating resource allocation and performing preemption risk assessment within the future time window based on the predicted memory usage data and the dynamic SLA priority, and outputting the risk assessment result and the set of target tasks where memory preemption conflicts occur, specifically includes: Within the future time window, calculate the sum of the predicted memory usage data for tasks that are active according to the default scheduling policy at each discrete time point; When the sum of the predicted video memory usage data is greater than the total video memory capacity of the system, virtual allocation is performed according to the task startup order, and tasks that cannot meet the video memory usage due to insufficient resources are marked as potentially preempted tasks. Obtain the highest priority among all potentially preempted tasks in the dynamic SLA priority list. If the highest priority is greater than a preset higher priority threshold, it is determined that there is a risk of preemption, and the corresponding task is assigned to the target task set.

[0010] As a further improvement of the present invention, the suppression of execution resources for low-priority tasks in the target task set includes dynamically adjusting the computation frequency, specifically as follows: Calculate the dynamic inhibition factor based on the risk level in the risk assessment results and the priority difference between lower priority tasks and higher priority thresholds; Calculate the target computing power allocation ratio based on the dynamic suppression factor and the default computing power allocation ratio; The computation frequency of the low-priority task is adjusted to a frequency corresponding to the target computing power allocation ratio through the NPU driver interface.

[0011] As a further improvement of the present invention, the steps of suppressing the execution of low-priority tasks in the target task set and locking the execution of video memory resources for high-priority tasks in the target task set specifically include: Identify the set of idle time periods for the higher priority tasks, and use a composite time window with safety boundaries to restrict the computation process of the lower priority tasks to be executed within the set of idle time periods; The driver function is called to lock the physical memory region containing the peak memory requirements and safety buffer of the higher priority task, and the locked physical memory region is marked as non-swapable.

[0012] As a further improvement of the present invention, the closed-loop optimization of the parameters for the preemption risk assessment based on the actual achieved indicators specifically includes: Calculate the proportion of tasks in the target task set that are subject to resource suppression to obtain the resource suppression aggression index, and calculate the average performance loss caused by resource suppression; Based on the resource suppression aggression index and the average performance loss, the weight coefficients in the dynamic SLA priority calculation are adjusted using a feedback loop, and the higher priority threshold for preemption risk assessment is dynamically adjusted based on the false alarm rate.

[0013] Secondly, the present invention provides an NPU training task preemption suppression system, comprising: The prediction module is used to obtain historical execution data and current status data of NPU training tasks to be scheduled and running, and outputs the predicted memory usage data of the NPU training task in the future time window through a pre-deployed real-time memory usage prediction model. The priority calculation module collects real-time performance metrics of online inference services that share computing resources with the NPU training task, and calculates the dynamic SLA priority of the NPU training task based on the real-time performance metrics and the static business importance preset by the task. The risk assessment module simulates resource allocation within the future time window based on the predicted memory usage data and the dynamic SLA priority to assess the risk of preemption, and outputs the risk judgment result and the set of target tasks where memory preemption conflicts occur. The scheduling and execution module, based on the risk assessment result, performs resource suppression on low-priority tasks in the target task set and locks video memory resources on high-priority tasks in the target task set. The closed-loop optimization module collects the actual execution metrics of the NPU training task and the actual achievement metrics of the online inference service, updates the real-time memory usage prediction model based on the actual execution metrics, and performs closed-loop tuning of the parameters for the preemption risk assessment based on the actual achievement metrics.

[0014] Thirdly, the present invention provides a computer device, comprising: a processor and a computer-readable storage medium; A processor, adapted to execute computer programs; A computer-readable storage medium storing a computer program that, when executed by the processor, implements the NPU training task preemption suppression method as described above.

[0015] Fourthly, the present invention provides a computer-readable storage medium storing a computer program adapted to be loaded by a processor and executed as described above for the NPU training task preemption suppression method.

[0016] The beneficial effects of this invention are as follows: This invention provides an NPU training task preemption suppression method, which acquires historical execution data and current state data of the NPU training task to be scheduled, calls a pre-deployed real-time memory usage prediction model to output the memory usage prediction data of the NPU training task within a preset future time window, and completes trend prediction before the actual arrival of the task's memory usage peak, eliminating the technical defect of existing static memory allocation mechanisms that cannot identify subsequent memory usage peaks in advance; it collects real-time performance indicators of online inference services that share the computing power resource pool with the NPU training task, and calculates and generates the dynamic SLA priority of the NPU training task by combining the task's preset static business importance weight, which allows the task priority to dynamically adapt to the real-time load status of the online inference service, overcoming the technical deficiency of traditional fixed priority scheduling strategies that cannot respond to real-time performance fluctuations of the computing power pool; based on the aforementioned memory usage prediction data and dynamic SLA priority, it completes multi-task resource allocation simulation and performs memory preemption risk pre-assessment within the future time window, outputting the risk judgment result and the set of target tasks where memory preemption conflicts will occur. By identifying risks before memory overflow conflicts actually occur, this invention avoids the technical shortcomings of existing post-event scheduling mechanisms that cannot proactively address memory conflicts. Based on the risk assessment results, resource suppression operations such as computing power quota reduction and memory access bandwidth limiting are performed on tasks with lower priority in the target task set, while memory resource locking operations with memory page table pre-binding are performed on tasks with higher priority. This allows for differentiated resource pre-allocation before conflicts occur, preventing high-priority tasks from being illegally preempted by low-priority tasks. The actual execution metrics of the NPU training task and the actual SLA achievement metrics of the online inference service are collected. The actual execution metrics are used as labeled samples to incrementally update the real-time memory occupancy prediction model, and the actual achievement metrics are used as feedback factors to complete the closed-loop optimization of the preemption risk assessment parameters. This allows the prediction model and assessment rules to continuously iterate and adapt to the actual operating conditions of the cluster. Ultimately, this invention significantly reduces the occurrence rate of memory preemption conflicts in NPU heterogeneous computing power clusters, greatly improves the SLA compliance rate of high-priority training tasks and online inference services, and significantly enhances the overall utilization efficiency of computing resources. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a flowchart of the NPU training task preemption suppression method provided by the present invention; Figure 2 This is an internal structural diagram of the computer device provided by the present invention. Detailed Implementation

[0019] To make the objectives and technical solutions of this invention clearer and easier to understand, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. The specific embodiments described herein are for illustrative purposes only and are not intended to limit the invention.

[0020] The technical solution of the present invention will be clearly and completely described below with reference to the accompanying drawings and specific embodiments. The described embodiments are only some embodiments of the present invention, and not all embodiments.

[0021] Example 1 This embodiment provides a method for suppressing preemption in NPU training tasks, as follows: Figure 1 As shown, the following is a detailed implementation method.

[0022] S1. Obtain historical execution data and current status data of NPU training tasks to be scheduled and running, and output the memory usage prediction data of the NPU training task in the future time window through a pre-deployed real-time memory usage prediction model.

[0023] The NPU cluster monitoring system continuously collects historical execution trajectory data of completed training tasks to form raw time series. ,in For sampling timestamps, This represents the video memory usage value at the corresponding moment. Assign task status labels (e.g., data loading, forward propagation, backward propagation, gradient update, etc.). Clean and align the original sequence, using a fixed time window. Resampling is performed to obtain a regular sequence. Based on this, multidimensional feature vectors are extracted. Including the current memory value ,past Mean within a time window with standard deviation The growth rate of current video memory usage relative to the time of task startup And the amplitude of the periodic components extracted by Fourier transform. These together constitute the input features of the model.

[0024] Design and train a time series prediction model based on an attention mechanism. A Transformer model with an encoder-decoder architecture is employed, with multi-head self-attention as its core. The encoder will input data of length... Historical feature sequence This is mapped to a context vector. Attention weights are calculated based on the scaled dot product of the query, key, and value:

[0025] in , , , For learnable parameter matrix, Let be the dimension of the key vector. Multi-head attention is executed in parallel. The above calculations are then concatenated, allowing the model to simultaneously focus on dependencies across different time scales. This is particularly effective for capturing sudden spikes and periodic patterns in memory usage. The decoder then generates future sequences autoregressively based on the encoder output and the predicted future sequences. Predicted memory usage at each time step:

[0026] The model loss function uses smooth L1 loss to balance robustness to outlier peaks and prediction accuracy:

[0027] in:

[0028] The model was trained offline until convergence using historical data.

[0029] The trained model is packaged into a standalone prediction service, providing a REST API or gRPC interface. The task scheduling system calls this service in the following two scenarios: periodically (e.g., every...) for running tasks. (seconds) collect its most recent The actual memory usage characteristics of each time window are sent to the prediction service to obtain future... The prediction service generates a sequence of steps; for new tasks to be scheduled, it matches similar historical task trajectories based on their task type identifiers (such as model structure and batch size), and uses the starting segment of that trajectory as input for forward prediction. The prediction service internally employs model warm-up and request queuing techniques to ensure low-latency response.

[0030] Establish a continuous model update mechanism. Deploy a background data pipeline to periodically (e.g., daily) collect execution data from newly completed tasks and add it to the training dataset. Configure model performance monitoring; when the moving average of the prediction error exceeds a threshold, automatically trigger incremental training or full retraining to update the model parameters. This mechanism ensures that the prediction model can adapt to changes in data distribution brought about by NPU hardware updates, deep learning framework version iterations, and new training task modes, providing a long-term, reliable foundation of prediction data for the scheduling system.

[0031] S2. Collect real-time performance metrics of online inference services that share computing resources with the NPU training task, and calculate the dynamic SLA priority of the NPU training task based on the real-time performance metrics and the static business importance preset by the task.

[0032] The system needs to define and maintain a dynamic priority for each training task. This priority is based not only on the user-preset static business importance, but more importantly, on the task's impact on the inference service SLA. By monitoring the performance metrics of the online inference service associated with the training task, such as latency and throughput, the system automatically increases the protective priority of the training task when it predicts or detects a risk of SLA violation due to resource preemption by the training task. Conversely, when the inference service load is light, its priority can be appropriately reduced. This step is executed in parallel with the prediction step, ensuring that the priority reflects the system status in real time. The specific implementation method of this step is as follows: Each training task Dynamic priority At time step Importance of static business With dynamic SLA impact factor The calculation model is jointly determined as follows: ,in These are preset weighting coefficients used to balance static business requirements with dynamic system states. Dynamic factors. The core is to quantify tasks. Potential or actual impact on the SLA of the correlated inference service. Continuous system monitoring and tasks. Key performance metrics for inference services using shared computing resources (such as GPU clusters) include average request latency. and throughput To assess the impact, an SLA risk indicator is defined. It combines current performance with preset SLA thresholds (such as maximum allowable latency). Minimum guaranteed throughput The deviation. An effective calculation method is:

[0033] in This is a weighting parameter for latency and throughput. This formula normalizes and linearly combines the risks of exceeding latency limits and insufficient throughput, thus providing a comprehensive and quantitative reflection of the degree of SLA violation. When Exceeding a preset warning threshold When this indicates that the inference service SLA is at risk of being violated, the system uses a monotonically increasing function. To improve the dynamic impact factor, i.e. For example, using an exponential function ,in This is the sensitivity coefficient. This design ensures that once a risk is detected, the task's dynamic priority increases rapidly and non-linearly, thus increasing the probability of pausing or slowing down the training task in scheduling decisions. This frees up resources for inference services, acting as a protective boost. Conversely, when... Below the threshold And when the system load is light, This is then reduced according to a decay function (such as linear decay), allowing training tasks to run at a lower priority or preempt more resources. This step is performed over a fixed period. It runs in parallel with the resource demand forecasting step, continuously collecting the latest monitoring data and recalculating. And update This ensures that priorities can reflect the operational status of inference services and SLA guarantee requirements in real time and with high sensitivity. By non-linearly mapping SLA risk indicators to priorities, the system can respond quickly at critical moments and prioritize the service quality of online services. This dynamic mechanism significantly improves the intelligence level of resource utilization and the overall service stability of the system compared to a fixed priority strategy.

[0034] S3. Based on the predicted memory usage data and the dynamic SLA priority, simulate resource allocation within the future time window to assess the risk of preemption, and output the risk assessment result and the set of target tasks where memory preemption conflicts occur.

[0035] Preemption risk assessment and decision-making are performed based on prediction and priority. The scheduler triggers the decision-making process periodically or when a new task is submitted. The decision engine comprehensively analyzes information from two dimensions: first, the future memory usage prediction curves of all active tasks and new tasks provided in step one; and second, the current dynamic priority of each task provided in step two. By simulating resource allocation, it is determined whether executing according to the default scheduling strategy will cause high-priority tasks to suffer performance degradation due to insufficient resources, especially to prevent high-priority training tasks from being unexpectedly preempted, which would lead to a decrease in the SLA of their associated inference service. This step is the core judgment link, connecting monitoring and prediction with final execution. The specific implementation method of this step is as follows: The decision engine maintains a global time window. ,in For the current moment, Let this be the horizon length for prediction and decision-making. For the set... Each task Step one provides the discrete time points within the time window. Predicted video memory usage Step two provides its current dynamic priority. This value is updated in real time and A larger value indicates a higher priority. The core decision-making process involves executing a resource allocation simulation. The simulation is based on a default scheduling strategy, which typically allocates resources according to task submission order or simple priority. During the simulation, for each future time step... Calculate the sum of predicted memory usage for all tasks that should be running according to the default strategy:

[0036] in for The set of active tasks under the default time policy. Sum this up with the total system video memory capacity. Compare. If there exists any moment... Make If this happens, it's determined that a memory overflow is imminent. At this point, the decision engine needs to identify the potentially affected tasks. It does this by analyzing... Given a set of active tasks at any given time, identify the highest priority task that is predicted to be preempted. Specifically, define a risk function. However, only when This only applies to tasks that cannot obtain the required video memory due to insufficient resources under the default scheduling. A more precise determination is made when... At that time, virtual allocation is performed according to the task startup order of the default policy. If a task started later cannot be satisfied due to insufficient resources, it will be temporarily excluded. Then the task Marked as a potential preemptor. Calculate the highest priority among all potential preemptor tasks. .like Higher than the preset high priority threshold If this happens, a high-risk warning is triggered, indicating that executing the default strategy may lead to the unexpected preemption of high-priority tasks (especially training tasks related to critical inference services), potentially violating the Service Level Agreement (SLA). This simulation-based prediction, compared to traditional decisions based on the current instantaneous state, can detect resource conflicts in advance, providing a crucial time window and basis for proactive preventative scheduling (such as pausing or delaying low-priority tasks), effectively avoiding performance fluctuations and SLA degradation caused by reactive responses. The decision result output is a binary flag. And a list of related tasks, for use in subsequent preemption or scheduling adjustments.

[0037] S4. Based on the risk assessment result, perform resource suppression on low-priority tasks in the target task set and lock video memory resources on high-priority tasks in the target task set.

[0038] Adaptive task scheduling and resource isolation are implemented. Based on the risk decision results from step three, the scheduler takes specific actions. If a risk is identified, suppression measures are implemented for low-priority tasks that may cause resource conflicts. For example, their computation frequency is dynamically adjusted, part of their computation process is temporarily suspended, or their computation is restricted to specific non-conflicting time windows, rather than simply and abruptly terminating them completely. Simultaneously, necessary GPU memory resources are reserved or locked for high-priority tasks through the NPU's hardware or driver-level interface to ensure the stability of their training process. This step is the execution phase of the decision and directly affects the NPU runtime environment. The specific implementation method of this step is as follows: The scheduler receives the risk assessment result from step three, which includes a risk level. and conflict task set ,in For the task priority, This is the threshold for high-priority tasks. If... Then the executor module is started. For sets Each low-priority task The scheduler employs a dynamic suppression strategy. This strategy first calculates a dynamic suppression factor. The calculation method is as follows:

[0039] in This is a system-adjustable sensitivity coefficient. This factor directly affects the task's computation frequency and is controlled through the driver interface. The computing power allocation ratio is set from the default. Adjusted to This continuous adjustment based on risk level and priority difference avoids a "one-size-fits-all" interruption, achieves smooth resource degradation, helps maintain the basic service capabilities of low-priority tasks, and improves the overall resource utilization efficiency of the system.

[0040] For suspending computation processes, the scheduler defines a state checkpointing mechanism. During gaps in critical tensor operations (such as convolution or matrix multiplication), lightweight signals are injected to suspend the task. The context (including register state and stream processor state) is saved to a preset host memory area, and then the corresponding computing core is put into a sleep state via driver commands. The timing of suspension is determined by the risk level. and current system load The decision is made jointly, and the decision function is:

[0041] in For indicator functions, This is the suspension threshold. This non-preemptive suspension, performed during computational gaps, minimizes disruption to the task's internal computational graph.

[0042] For time window constraints, the scheduler allocates periodic, non-conflicting time slots to low-priority tasks. Let's assume a high-priority task... One calculation cycle is The scheduler analyzes the memory access patterns and identifies the set of idle time periods. Then, low-priority tasks will be... The computation process is limited to a composite time window:

[0043] Internal execution, of which This serves as a safety boundary to avoid overlap caused by switching overhead. It is implemented collaboratively between the kernel scheduler's cgroupcpuset and the NPU compute flow controller.

[0044] Meanwhile, the scheduler uses the physical address locking interface provided by the NPU driver to lock high-priority tasks. Reserve memory for video recording. Assume... The estimated peak memory requirement is The scheduler calls the driver function `npumem_lock(va, size, flags)` to lock the virtual address starting at `va` with a size of [missing information]. The video memory region is locked, in which For safety buffering. Locked page frames will be marked as non-swapable and prohibited from being assigned other tasks, thus ensuring... The training process can obtain the necessary storage resources at any time, ensuring its execution stability and performance predictability. All scheduling and isolation operations take effect directly through the runtime environment, completing the final control over NPU hardware resources.

[0045] S5. Collect the actual execution metrics of the NPU training task and the actual achievement metrics of the online inference service, update the real-time memory usage prediction model based on the actual execution metrics, and perform closed-loop optimization of the parameters of the preemption risk assessment based on the actual achievement metrics.

[0046] The system continuously monitors the actual task execution, the accuracy of memory usage prediction, and the final achievement of the inference SLA. Predicted values ​​are compared with actual values, and the discrepancy is used for periodic retraining or online adjustment of the prediction model in step one. Simultaneously, the effectiveness of scheduling decisions is analyzed, such as checking whether resource suppression is too aggressive or conservative, and the parameters of the priority calculation algorithm in step two or the risk assessment threshold in step three are dynamically adjusted based on this feedback. This process forms a closed loop from execution to evaluation to optimization, enabling the entire system to adaptively and continuously improve. The specific implementation method of this step is as follows: The system continuously collects monitoring data during runtime, forming a time-series sample set. For task execution, it records the actual completion time of each inference task. and the predicted completion time in step one Regarding video memory usage, record the actual peak video memory usage for the task. and predicted peak memory For inference SLAs, record whether the task is within the deadline. Prior to completion, define the actual SLA achievement markers. ,in This indicates that an agreement has been reached.

[0047] Based on the collected data, the bias index of the prediction model is calculated. For time prediction, the mean absolute percentage error is calculated:

[0048] For memory prediction, calculate the root mean square error:

[0049] when or Exceeding the preset threshold , This triggers the model update process. Updates can be achieved through periodic retraining or online learning, such as adding new samples to the training set to minimize the prediction error loss function.

[0050] To update model parameters and ,in and These are time-based and memory-based prediction models, respectively. As a task characteristic, To balance the weights, this model update mechanism based on real-time bias ensures that prediction accuracy can adaptively improve with changes in system load and task characteristics, providing reliable input for preceding steps.

[0051] Simultaneously, the system analyzes the effectiveness of scheduling decisions to optimize strategy parameters. A resource suppression aggression index is defined. ,in The number of tasks subject to resource suppression. Given the total number of tasks. Calculate the average performance loss due to suppression:

[0052] in This refers to throughput or latency metrics. Too high and A significant difference indicates that the suppression is too aggressive, while a less aggressive one may be too conservative. Based on the analysis results, the parameters of the priority calculation algorithm in step two are dynamically adjusted, for example, by adjusting the weighting coefficients for balancing resource requirements and SLA urgency. The priority calculation formula can be adjusted as follows:

[0053] Updated via feedback loop and The update rules are as follows:

[0054]

[0055] in This is the learning rate. This enables the scheduling policy parameters to be self-tuned.

[0056] For the risk assessment in step three, monitor the risk trigger frequency and false alarm rate. Define the risk false alarm rate as the proportion of tasks that were marked as high-risk even though resource overflow did not actually occur. Adjust the risk assessment threshold based on this indicator. The strategy is adjusted as follows:

[0057] in To adjust the step size, False alarm rate. Dynamically optimized through closed-loop feedback. This enables the system to achieve a better balance between risk prevention and resource utilization. All monitoring, analysis, and adjustment processes are executed automatically, forming a complete closed loop from execution data collection to model and strategy evaluation and parameter optimization, driving the system to continuously adapt and improve.

[0058] Example 2 This invention successfully provides an NPU training task preemption suppression system. The system specifically includes: The prediction module is used to obtain historical execution data and current status data of NPU training tasks to be scheduled and running, and outputs the predicted memory usage data of the NPU training task in the future time window through a pre-deployed real-time memory usage prediction model. The priority calculation module collects real-time performance metrics of online inference services that share computing resources with the NPU training task, and calculates the dynamic SLA priority of the NPU training task based on the real-time performance metrics and the static business importance preset by the task. The risk assessment module simulates resource allocation within the future time window based on the predicted memory usage data and the dynamic SLA priority to assess the risk of preemption, and outputs the risk judgment result and the set of target tasks where memory preemption conflicts occur. The scheduling and execution module, based on the risk assessment result, performs resource suppression on low-priority tasks in the target task set and locks video memory resources on high-priority tasks in the target task set. The closed-loop optimization module collects the actual execution metrics of the NPU training task and the actual achievement metrics of the online inference service, updates the real-time memory usage prediction model based on the actual execution metrics, and performs closed-loop tuning of the parameters for the preemption risk assessment based on the actual achievement metrics.

[0059] Specific limitations regarding the NPU training task preemption suppression system can be found in the limitations of the NPU training task preemption suppression method described above; the corresponding technical effects are equivalent and will not be repeated here. Each module in the aforementioned NPU training task preemption suppression system can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of the computer device in hardware form or independent of it, or they can be stored in the memory of the computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0060] Example 3 Figure 2An internal structural diagram of a computer device is shown in one embodiment. This computer device may specifically be a terminal or a server. Figure 2 As shown, the computer device includes a processor, memory, network interface, display, camera, and input device connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The network interface is used to communicate with external terminals via a network connection. When the computer program is executed by the processor, it implements a preemption suppression method for NPU training tasks. The display screen can be an LCD screen or an e-ink display. The input device can be a touch layer covering the display screen, buttons, a trackball, or a touchpad mounted on the computer device casing, or an external keyboard, touchpad, or mouse.

[0061] As will be understood by those skilled in the art, computer equipment Figure 2 The structure shown is merely a block diagram of a portion of the structure related to the present invention and does not constitute a limitation on the computer device to which the present invention is applied. A specific computing device may include more or fewer components than those shown in the figure, or combine certain components, or have the same component arrangement.

[0062] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the method described above.

[0063] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps of the above-described method.

[0064] The various embodiments in this specification are described in a progressive manner. For directly identical or similar parts of the embodiments, refer to each other. Each embodiment focuses on its differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments. It should be noted that the technical features of the above embodiments can be combined arbitrarily. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as the combination of these technical features does not contradict each other, it should be considered within the scope of this specification.

[0065] The above-described embodiments are merely preferred embodiments of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the invention. It should be noted that those skilled in the art can make various improvements and substitutions without departing from the principles of the present invention, and these improvements and substitutions should also be considered within the scope of protection of the present invention. Therefore, the scope of protection of this invention should be determined by the scope of the claims.

Claims

1. A method for suppressing preemption in NPU training tasks, characterized in that, include: The historical execution data and current status data of the NPU training task to be scheduled are obtained, and the memory usage prediction data of the NPU training task in the future time window is output through a pre-deployed real-time memory usage prediction model. Collect real-time performance metrics of the NPU training task that share computing resources and online inference services, and calculate the dynamic SLA priority of the NPU training task based on the real-time performance metrics and the static business importance preset by the task. Based on the predicted memory usage data and the dynamic SLA priority, resource allocation is simulated and preemption risk assessment is performed within the future time window, and the risk judgment result and the set of target tasks where memory preemption conflict occurs are output. Based on the risk assessment results, resource suppression is performed on lower priority tasks in the target task set, and memory resource locking is performed on higher priority tasks in the target task set. The actual execution metrics of the NPU training task and the actual achievement metrics of the online inference service are collected. The real-time memory usage prediction model is updated based on the actual execution metrics, and the parameters of the preemption risk assessment are optimized in a closed loop based on the actual achievement metrics.

2. The NPU training task preemption suppression method according to claim 1, characterized in that, The process of acquiring historical execution data and current status data of NPU training tasks to be scheduled and running, and outputting predicted memory usage data of the NPU training tasks within a future time window through a pre-deployed real-time memory usage prediction model, specifically includes: Historical execution trajectory data is collected from the NPU cluster, and a regular sequence is obtained by resampling with a preset time window. Feature vectors are then extracted, including the current memory value, the mean and standard deviation of memory within the historical time window, the memory growth rate, and the amplitude of the periodic component extracted by Fourier transform. The feature vector is input into the attention-based time series prediction model. The encoder maps the historical feature sequence into a context vector, and the decoder generates memory usage prediction data for future time steps.

3. The NPU training task preemption suppression method according to claim 2, characterized in that, The step of calculating the dynamic SLA priority of the NPU training task based on the real-time performance metrics and the pre-defined static business importance of the task specifically includes: The average request latency and throughput of the online inference service are compared with the corresponding preset SLA thresholds to calculate the SLA risk index that combines the deviation of the current performance from the preset SLA thresholds. When the SLA risk index exceeds the preset warning threshold, the dynamic SLA impact factor is increased through an exponential function; When the SLA risk index is lower than the warning threshold, the dynamic SLA impact factor is reduced by a decay function. The dynamic SLA priority of the NPU training task is calculated based on the static business importance and the dynamic SLA impact factor.

4. The NPU training task preemption suppression method according to claim 3, characterized in that, The step of simulating resource allocation and assessing preemption risk within the future time window based on the predicted memory usage data and the dynamic SLA priority, and outputting the risk assessment result and the set of target tasks where memory preemption conflicts occur, specifically includes: Within the future time window, calculate the sum of the predicted memory usage data for tasks that are active according to the default scheduling policy at each discrete time point; When the sum of the predicted video memory usage data is greater than the total video memory capacity of the system, virtual allocation is performed according to the task startup order, and tasks that cannot meet the video memory usage due to insufficient resources are marked as potentially preempted tasks. Obtain the highest priority among all potentially preempted tasks in the dynamic SLA priority list. If the highest priority is greater than a preset higher priority threshold, it is determined that there is a risk of preemption, and the corresponding task is assigned to the target task set.

5. The NPU training task preemption suppression method according to claim 1, characterized in that, The resource suppression for low-priority tasks in the target task set includes dynamically adjusting the computation frequency, specifically through the following steps: Calculate the dynamic inhibition factor based on the risk level in the risk assessment results and the priority difference between lower priority tasks and higher priority thresholds; Calculate the target computing power allocation ratio based on the dynamic suppression factor and the default computing power allocation ratio; The computation frequency of the low-priority task is adjusted to a frequency corresponding to the target computing power allocation ratio through the NPU driver interface.

6. The NPU training task preemption suppression method according to claim 1, characterized in that, The specific steps of performing resource suppression on low-priority tasks in the target task set and locking video memory resources on high-priority tasks in the target task set include: Identify the set of idle time periods for the higher priority tasks, and use a composite time window with safety boundaries to restrict the computation process of the lower priority tasks to be executed within the set of idle time periods; The driver function is called to lock the physical memory region containing the peak memory requirements and safety buffer of the higher priority task, and the locked physical memory region is marked as non-swapable.

7. The NPU training task preemption suppression method according to claim 1, characterized in that, The closed-loop optimization of the parameters for the risk assessment based on the actual achieved indicators specifically includes: Calculate the proportion of tasks in the target task set that are subject to resource suppression to obtain the resource suppression aggression index, and calculate the average performance loss caused by resource suppression; Based on the resource suppression aggression index and the average performance loss, the weight coefficients in the dynamic SLA priority calculation are adjusted using a feedback loop, and the higher priority threshold for preemption risk assessment is dynamically adjusted based on the false alarm rate.

8. A preemption suppression system for NPU training tasks, characterized in that, include: The prediction module is used to obtain historical execution data and current status data of NPU training tasks to be scheduled and running, and outputs the predicted memory usage data of the NPU training task in the future time window through a pre-deployed real-time memory usage prediction model. The priority calculation module collects real-time performance metrics of online inference services that share computing resources with the NPU training task, and calculates the dynamic SLA priority of the NPU training task based on the real-time performance metrics and the static business importance preset by the task. The risk assessment module simulates resource allocation within the future time window based on the predicted memory usage data and the dynamic SLA priority to assess the risk of preemption, and outputs the risk judgment result and the set of target tasks where memory preemption conflicts occur. The scheduling and execution module, based on the risk assessment result, performs resource suppression on low-priority tasks in the target task set and locks video memory resources on high-priority tasks in the target task set. The closed-loop optimization module collects the actual execution metrics of the NPU training task and the actual achievement metrics of the online inference service, updates the real-time memory usage prediction model based on the actual execution metrics, and performs closed-loop tuning of the parameters for the preemption risk assessment based on the actual achievement metrics.

9. A computer device, characterized in that, include: Processor and computer-readable storage media; A processor, adapted to execute computer programs; A computer-readable storage medium storing a computer program that, when executed by the processor, implements the NPU training task preemption suppression method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program adapted to be loaded by a processor and executed as described in any one of claims 1 to 7.

Images