Fine-grained lightweight DPU performance isolation method

Abstract

The invention provides a fine-grained lightweight DPU performance isolation method. The method comprises the following steps: firstly, entering an offline stage, representing a partial systematic analysis task execution mode through offline analysis resources, and modeling a mapping relationship between task parameters and resource consumption; then entering an online stage, intercepting submission of an application task, evaluating a resource demand according to an offline model, decomposing a large task by applying a self-adaptive task splitting and result recombination part, executing a scheduling process guided by a working load, and realizing dynamic allocation through a global scheduler and an application agent; the global scheduler allocates time slices or subtasks based on workload characteristics; after execution, results are recombined and returned to the application. Therefore, under the condition that hardware or proprietary software is not modified, accurate resource management is achieved by directly constructing on an existing manufacturer SDK, the isolation challenge of the DPU is effectively solved through the technologies, and efficient resource utilization of cloud providers in PaaS and SaaS modes is promoted.

Description

Technical Field

[0001] This disclosure relates to the field of computers, and more specifically, to a fine-grained, lightweight method for DPU performance isolation. Background Technology

[0002] A Data Processing Unit (DPU) is an advanced hardware device evolved from Remote Direct Memory Access (RDMA) Network Interface Cards (RNICs) and Smart Network Interface Controllers (SmartNICs). It integrates various high-efficiency hardware modules, including Application-Specific Integrated Circuit (ASIC) Network Interface Controllers (NICs), System-on-Chips (SoC) processor cores and memory, bypass ASIC accelerators (such as modules for encryption and redundancy processing), and programmable data path accelerators (DPAs). These components enable the DPU to efficiently handle infrastructure tasks in the data center, such as network virtualization, storage services, and security protocols, thereby significantly reducing the Total Cost of Ownership (TCO). Taking the NVIDIA BlueField-3 DPU as an example, its hardware architecture includes a ConnectX-7 RNIC to support communication between the host and external networks; 8 to 16 ARM cores with 16 to 32 GB of DRAM SoC memory for running standard Linux distributions; a fixed-function ASIC accelerator capable of processing data at speeds up to 100 Gbps; and a 16-core, 256-thread RISC-V processor for multi-threaded data plane applications. Furthermore, a PCIe switch ensures efficient interaction between the SoC, the host, and the NIC. On the software side, the DPU provides task abstractions through a unified API such as NVIDIA DOCA. These tasks represent basic operations for building applications, such as generating redundant data in buffers or performing RDMA communication operations. DOCA employs an asynchronous task submission model combined with callback functions to implement a pipelined application architecture. This design allows the DPU to bypass the host CPU without sacrificing performance, achieving high-throughput, low-latency data processing and transmission, and has been widely deployed in hyperscale data centers to improve overall efficiency.

[0003] In cloud service environments, the deployment of DPUs provides robust support for Platform as a Service (PaaS) and Software as a Service (SaaS). These service models require cloud providers to efficiently manage resources to meet the performance demands of tenants for diverse applications. DPUs significantly improve resource utilization by offloading infrastructure workloads such as virtual network functions, storage indexes, and the TCP data plane. For example, in distributed storage systems, DPUs can accelerate indexing operations on discrete storage; in high-performance computing scenarios, they can provide low-latency support for distributed deep learning. Looking ahead, cloud providers can rent DPUs to tenants as PaaS or offer acceleration services as SaaS to support more high-performance applications. This multi-application coexistence model is expected to further improve data center resource utilization, provided that hardware resource allocation can be precisely controlled to avoid performance degradation caused by resource contention. The advantages of DPUs make them particularly suitable for cross-application collaboration in cloud environments. However, the lack of effective isolation mechanisms can hinder cloud providers from fulfilling their guarantees for Service Level Agreements (SLAs), such as key metrics like throughput and Job Completion Time (JCT).

[0004] Existing DPU performance isolation mechanisms have significant shortcomings and cannot meet the needs of multi-application coexistence. First, DPU vendors offer limited mechanisms, typically abstracting hardware resources only through high-level proprietary APIs (such as DOCA). These APIs encapsulate hardware functions as tasks, obscuring the relationship between task parameters and underlying resource consumption, making direct resource allocation impossible, such as using cgroups-like mechanisms to isolate ASIC accelerators. Second, accelerators employ coarse-grained First-Come-First-Served (FCFS) scheduling strategies, with task sizes ranging from bytes to hundreds of megabytes and execution times from microseconds to tens of milliseconds. This causes large tasks from bandwidth-sensitive applications to monopolize hardware, violating the SLAs of latency-sensitive applications and reducing overall throughput. For example, in coexistence scenarios, small tasks in JCT-sensitive applications may experience an average JCT increase of 308.32% due to head-of-line blocking by large tasks, with an SLA violation rate as high as 48.51%. Furthermore, existing related work has not adequately addressed these issues. For RNICs, frameworks such as Justitia, Harmonic, and Tassel primarily focus on RDMA transmission performance, such as throughput and latency, but neglect the ASIC accelerators unique to DPUs. For SmartNICs, software-based solutions like FairNIC schedule tasks at the API level, failing to provide fine-grained protection for JCT-sensitive tasks; hardware-based solutions like Panic and S-NIC rely on proprietary hardware features that are typically unavailable or not exposed in commercial DPUs, limiting their practical deployment. These shortcomings make it difficult for cloud providers to control resource allocation, leading to performance degradation and hindering the widespread adoption of DPUs in PaaS and SaaS. Summary of the Invention

[0005] The purpose of this disclosure is to provide a fine-grained, lightweight DPU performance isolation method, which aims to solve the shortcomings of existing DPU performance isolation mechanisms in multi-application coexistence scenarios.

[0006] In one general aspect, a fine-grained, lightweight method for DPU performance isolation is provided, comprising: First, we enter the offline phase, and systematically analyze the task execution mode through the resource representation part of the offline analysis, and model the mapping relationship between task parameters and resource consumption. Then, in the online phase, the application task submission is intercepted, resource requirements are evaluated based on the offline model, the application adaptively splits and reassembles the results to partially decompose large tasks, and executes a workload-guided scheduling process, which is dynamically allocated through a global scheduler and application agent; the global scheduler allocates time slices or subtasks based on workload characteristics; after execution, the results are reassembled and returned to the application.

[0007] The implementation method of the resource representation part through offline analysis is as follows: the accelerator resources are represented as time slices, and the task resource usage is mapped to the execution time by taking advantage of FCFS scheduling; in the offline stage, a variety of representative tasks are run, the execution time and input parameters are collected, and a regression model is established.

[0008] The adaptive task splitting and result reorganization implementation method is as follows: For a given accelerator task, first evaluate the correlation between task size and execution time. If the task exceeds a preset threshold, split it into several sub-tasks. Each sub-task is submitted to the hardware queue for execution independently. The splitting process takes into account the algorithm characteristics. Meanwhile, several optimization measures are adopted: First, a reusable object pool is used to pre-allocate task objects and buffers, reducing the latency of dynamic memory allocation; second, the pipelined overlap of software and hardware execution prepares the input of the next subtask immediately after the subtask is submitted, realizing parallel processing; third, a zero-copy scatter / aggregate list avoids unnecessary memory copy operations and only references data fragments through pointers.

[0009] The workload-driven scheduling process specifically includes a global scheduler and an agent for each application. The global scheduler schedules every [period]. MS periodically performs low-frequency decisions, uses an exponentially weighted moving average algorithm to predict application resource requirements, and implements a compensation mechanism based on historical SLA performance; Meanwhile, the global scheduler dynamically adjusts the granularity of task splitting: when competition is fierce, it increases the splitting to ensure fine-grained fairness; when the load is low, it reduces the splitting to maximize throughput.

[0010] The technical effects to be achieved by the embodiments of the present invention are as follows: This invention provides a fine-grained, lightweight DPU performance isolation method and system, namely ASTRAEA. This system leverages experimental evaluation and theoretical analysis of DPU hardware characteristics and task behavior to derive two key insights: although the mapping between tasks and resource consumption is opaque, custom resource representations can be defined through task behavior analysis, and the correlation between parameters and consumption can be established; simultaneously, based on the mathematical properties of task functions, large tasks can be decomposed into subtasks for independent execution, and the recombined results are equivalent to the original task processing. These insights enable ASTRAEA to be built directly on existing vendor SDKs without modifying hardware or proprietary software, achieving precise resource management. These technologies effectively address the isolation challenges of DPUs, promoting efficient resource utilization for cloud providers in PaaS and SaaS models. Attached Figure Description

[0011] The above and other objects and features of this disclosure will become clearer from the following description taken in conjunction with the accompanying drawings.

[0012] Figure 1 This is a schematic diagram illustrating the operational hardware architecture of a fine-grained, lightweight DPU performance isolation method according to an embodiment of the present disclosure. Figure 2 This is a schematic diagram illustrating the system architecture and workflow of ASTRAEA according to an embodiment of the present disclosure; Figure 3 This is a schematic flowchart illustrating an optimization mechanism for task splitting according to an embodiment of the present disclosure. Detailed Implementation

[0013] The following detailed embodiments are provided to aid the reader in gaining a comprehensive understanding of the methods, apparatus, and / or systems described herein. However, various changes, modifications, and equivalents of the methods, apparatus, and / or systems described herein will become apparent upon understanding this disclosure. For example, the order of operations described herein is merely illustrative and is not limited to those orders set forth herein, but may be changed as will become clear upon understanding this disclosure, except for operations that must occur in a specific order. Furthermore, for clarity and conciseness, descriptions of features known in the art may be omitted.

[0014] The features described herein may be implemented in different forms and should not be construed as limited to the examples described herein. Rather, the examples described herein are provided only to illustrate some of the many feasible ways of implementing the methods, apparatus, and / or systems described herein, which will become clear upon understanding the disclosure of this application.

[0015] As used herein, the term “and / or” includes any one of the associated listed items and any combination of any two or more.

[0016] Although terms such as “first,” “second,” and “third” may be used herein to describe various components, assemblies, regions, layers, or parts, these components, assemblies, regions, layers, or parts should not be limited by these terms. Rather, these terms are used only to distinguish one component, assembly, region, layer, or part from another. Thus, without departing from the teaching of the examples described herein, the first component, first assembly, first region, first layer, or first part referred to as the first component, first assembly, first region, first layer, or first part may also be referred to as the second component, second assembly, second region, second layer, or second part.

[0017] In the specification, when an element (such as a layer, region, or substrate) is described as being "on" another element, "connected to," or "bonded to" another element, the element may be directly "on" another element, directly "connected to," or "bonded to" the other element, or one or more other elements may be present in between. Conversely, when an element is described as being "directly on" another element, "directly connected to," or "directly bonded to" another element, no other elements may be present in between.

[0018] The terminology used herein is for the purpose of describing various examples only and is not intended to limit disclosure. Unless the context clearly indicates otherwise, the singular form is intended to include the plural form as well. The terms “comprising,” “including,” and “having” indicate the presence of the described features, quantities, operations, components, elements, and / or combinations thereof, but do not preclude the presence or addition of one or more other features, quantities, operations, components, elements, and / or combinations thereof.

[0019] Unless otherwise defined, all terms used herein (including technical and scientific terms) shall have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure pertains upon understanding this disclosure. Unless expressly defined herein, terms (such as those defined in a general dictionary) shall be interpreted as having a meaning consistent with their meaning in the context of the relevant field and in this disclosure, and shall not be interpreted in an idealized or overly formalistic manner.

[0020] Furthermore, in the description of the examples, detailed descriptions of well-known related structures or functions will be omitted when it is believed that such detailed descriptions would lead to a vague interpretation of this disclosure.

[0021] Figure 1 This is a schematic diagram of the hardware architecture illustrating a fine-grained, lightweight DPU performance isolation method according to an embodiment of the present disclosure.

[0022] To achieve the aforementioned objectives, the present invention employs the following technical framework: Figure 2 As shown.

[0023] This invention addresses the shortcomings of existing DPU performance isolation mechanisms by providing a fine-grained, lightweight DPU performance isolation method and system, namely ASTRAEA. Based on in-depth analysis of DPU hardware characteristics and task behavior, this system is directly built on existing vendor SDKs (such as NVIDIA DOCA) without requiring hardware or proprietary software modifications. Through offline profiling to establish a resource model, adaptive task splitting for fine-grained scheduling, and a workload-driven scheduling algorithm, it ensures precise control over the allocation of computing resources provided by ASIC accelerators in multi-application coexistence scenarios, maintaining fairness and utilization. Figure 2As shown, ASTRAEA's architecture includes an offline profiling phase and an online execution phase. In the offline phase, task execution patterns are systematically analyzed to establish a mapping relationship between task parameters and resource consumption. In the online phase, the system intercepts application task submissions, decomposes large tasks using application task splitting and reorganization techniques, and dynamically allocates tasks through a global scheduler and application broker. The specific workflow is as follows: First, the application submits tasks via the DOCA API; the ASTRAEA broker intercepts these tasks, evaluates resource requirements based on the offline model, and may split the tasks; subsequently, the global scheduler allocates time slices or subtasks based on workload characteristics (such as demand forecasting and SLA compensation); after execution, the results are reorganized and returned to the application. This system is implemented on an NVIDIA BlueField-3 DPU, supports PaaS and SaaS models in cloud environments, and can reduce the average JCT of latency-sensitive applications from 1955.23µs to 730.26µs in competitive scenarios while maintaining less than 2% performance overhead.

[0024] Resource representation through offline profiling: Although the relationship between DPU tasks and underlying hardware resource consumption becomes opaque due to the abstraction of high-level APIs, this invention utilizes systematic offline profiling techniques to define custom resource representations and establish a precise mapping between task parameters and resource consumption, thereby achieving fine-grained resource management. Specifically, for critical components such as ASIC accelerators, while hardware partitioning cannot be directly observed, task execution time exhibits a strong correlation with input / output data size, such as a near-linear relationship, and accelerators have a fixed processing capacity (e.g., bytes per second). Leveraging this characteristic, this invention represents accelerator resources as time slices and, through the advantages of FCFS scheduling, maps task resource usage to execution time. Specifically, by observing the strong correlation between task execution time and input / output scale (e.g., a near-linear or polynomial relationship) and considering the fixed processing capacity of the accelerator (bytes per second), this invention defines the resource consumption of each task as its predicted execution time, thereby achieving precise time-slice allocation of hardware resources. This mapping method innovatively transforms opaque hardware abstractions into quantifiable time-domain models, avoiding the limitations of traditional direct partitioning mechanisms. During offline profiling, the system runs various representative tasks (such as erasure coding / decoding and Advanced Encryption Standard Galois / Counter Mode (AES-GCM) encryption tasks) and collects execution time and input parameters (such as the number of data blocks). Number of redundant blocks Block size Or plain text size Additional authentication data size Data such as [data missing] are used, and then a nonlinear least squares method is employed to fit a custom function form to establish a regression model. The specific fitting process involves defining a hypothesis function model (such as a polynomial linear combination or product form) and optimizing the function parameters by minimizing the sum of squared errors between the observed data and the predicted values. This method not only considers the differences between pipelined and non-pipelined execution but also develops specific function forms for different accelerator task types (such as erasure coding and encryption) to improve model accuracy. For example, for erasure coding tasks in non-pipelined scenarios, this invention fits a function [data missing]. The first term captures the linear effect of the number of data blocks, the second term handles the contribution of redundant blocks, and the third term quantizes the adjustment factor for block size. Similarly, other scenarios (such as pipelined encoding, decoding, and encryption) employ similar product or additive combinations to ensure the model's robustness and prediction accuracy under complex workloads. This fitting method introduces multivariate interaction terms (such as...) and This technique (which eliminates implicit coupling) surpasses traditional linear models, providing an innovative foundation for subsequent fine-grained scheduling. For example, for erasure coding accelerators, profiling shows that pipelined execution of small tasks (e.g., eight 1MB tasks) is faster than a single large task (e.g., one 8MB task), revealing the potential for splitting. Furthermore, the profiling confirms no cross-interference between accelerators and quantifies pipelined overhead. This technique provides a foundation for subsequent scheduling, ensuring that resource allocation accurately matches application requirements and avoiding SLA violations caused by coarse-grained FCFS.

[0025] Adaptive task splitting and result reorganization: To overcome the head-of-line blocking problem caused by the coarse-grained FCFS scheduling of DPU accelerators, this invention introduces adaptive task decomposition and result reassembly techniques. By analyzing the hardware execution logic and mathematical properties of tasks, large tasks are decomposed into multiple independent subtasks, enabling fine-grained resource allocation and scheduling. This technique is based on insights into task decomposability; for example, in erasure coding tasks, matrix splitting transforms large blocks of data processing into the parallel or sequential execution of multiple smaller blocks of data, ultimately reassembling the results to ensure output consistency.

[0026] Specifically, for a given accelerator task, this invention evaluates the task size based on the execution time prediction model established through the aforementioned offline profiling. This model considers task parameters (such as the number of data blocks)... Number of redundant blocks Block size Mapped to predicted execution time This serves as a quantitative representation of task resource consumption.

[0027] This invention employs a dynamic granularity control mechanism to determine the task splitting threshold. Let the current splitting granularity be... (Unit: bytes), its value range is a predefined ordered discrete set. ,in , This represents the total number of particle size levels. In one embodiment of the invention, byte, i.e. The design of this ordered set allows for expansion to more granular levels as needed. Granularity of splitting. The ASTRAEA scheduler dynamically sets the parameters based on the system runtime state and passes them to the ASTRAEA agents of each application via shared memory.

[0028] For the data block size of the original task The ASTRAEA agent calculates the number of subtasks using the following formula. : When there are multiple competing applications in the system (i.e., the number of applications is greater than 1) and In this invention, the original task is broken down into... Subtasks are created; otherwise, the tasks are submitted as is to avoid unnecessary splitting overhead.

[0029] The decomposition process for erasure coding accelerator tasks is based on the mathematical separability of erasure coding algorithms. For tasks containing... Source data matrix of data blocks This invention divides it according to the column direction (i.e., data dimension) into Submatrix, the nth submatrix Submatrix Defined as: in Representation matrix The Listed to number Submatrices composed of columns. The size of each submatrix is... It can perform encoding or decoding operations independently.

[0030] For the encoding task, let the encoding matrix be... The encoded output of the original task is: Due to the modular nature of matrix multiplication, the output of each subtask... It can be calculated independently, and the final result is reassembled by horizontal splicing: This partitioning method guarantees the mathematical equivalence of the output to the original, unpartitioned task. For the decoding task, the same partitioning principle applies to the operations on the recovery matrix and available data blocks.

[0031] Predicted execution time for each subtask Calculations were performed using the aforementioned offline profiling model: This forecast is used for subsequent time quota management and scheduling decisions.

[0032] At the same time, such as Figure 3 As shown, to minimize the overhead introduced by task splitting, this invention designs a complete task processing pipeline, including four stages: task splitting, buffer management, queue scheduling, and result reassembly. The specific operational flow of the accelerator task from splitting to completion is described in detail below: Phase 1: Task Interception and Splitting Decisions The ASTRAEA agent intercepts application calls to the accelerator API using dynamic link library preloading (LD_PRELOAD) technology. When an application submits a raw task, the agent first reads the current split granularity set by the ASTRAEA scheduler from shared memory. The number of subtasks is calculated according to the aforementioned formula. At the same time, the agent calculates the expected completion time for this task: in This is the current timestamp. Service level objectives (such as latency caps) configured for the application. If the task is successful, proceed to the task splitting process; otherwise, submit the original task directly to the next stage.

[0033] Phase Two: Subtask Creation and Buffer Allocation For tasks that need to be split, the agent handles each subtask. ( Perform the following operations:

[0034] Source buffer construction: Sub-source buffers are created using a scatter / gather list technique, with pointer references used for each data block. ( The starting address of the sub-buffer is calculated as follows: Data length is Multiple sub-buffers are linked together using a linked list structure to form a complete subtask input, eliminating the need for data copying throughout the process.

[0035] Target Buffer Acquisition: Acquire a free target buffer from a pre-allocated pool of reusable buffers. The buffer pool is created during system initialization and contains a fixed number (e.g., 32) of pre-allocated buffer objects, each large enough to hold output data at the maximum granularity. This is achieved through circular indexing. It enables fast allocation and avoids the overhead of dynamic memory allocation at runtime.

[0036] Subtask encapsulation: The source buffer, target buffer, encoding / decoding matrix, and user callback information are encapsulated into a subtask object, while the predicted execution time of the subtask is recorded. Subtask Index And a flag indicating whether it is the last subtask.

[0037] Phase 3: Virtual Queue Management and Pipeline Submission The encapsulated subtasks are placed into a virtual queue maintained by the ASTRAEA agent. This queue uses a lock-free circular buffer design, supporting concurrent access by a single producer (application thread) and a single consumer (submission thread). Let the queue capacity be... The head pointer is The tail pointer is The enqueue operation is as follows: The agent prepares a buffer for the next subtask immediately after the subtask is enqueued, enabling pipelined overlap between software processing and hardware execution.

[0038] The agent maintains a dedicated submission thread that continuously polls a virtual queue. When a subtask awaiting submission is detected in the queue (i.e.,...), the agent submits it. When submitting, the thread first queries the current time quota in shared memory. If one of the following conditions is met, the subtask will be submitted to the hardware queue: Condition 1 (Sufficient quota): This means that there is still a remaining time quota within the current scheduling period; Condition 2 (Tail Delay Protection): This means that the task is about to time out, triggering the active tail delay protection mechanism to avoid SLA violation.

[0039] After each successful submission, update the time quota and cumulative usage: in This represents the cumulative resource usage of the application during this scheduling period.

[0040] Phase Four: Result Callback and Data Reorganization Once the hardware accelerator completes its subtask execution, the registered callback function is triggered. The callback function performs the following operations: Data copying: If it is a subtask resulting from splitting, the output data of the subtask is copied from the temporary target buffer to the corresponding location in the original task's target buffer. For encoding tasks, the output includes... The redundant blocks are copied to the target address as follows: Resource release: Release the source buffer reference occupied by the subtask (through the reference counting mechanism), reset the data length of the temporary target buffer to 0, so that it can be reused by subsequent subtasks.

[0041] Completion of judgment and result return: If the current subtask is the last subtask of the original task ( If ), then perform the following operations: Check actual completion time Did it exceed the expected time? ,like Then increment the SLA violation counter for shared memory: ; Call the original callback function registered by the application and return the complete reassembly result.

[0042] Through the above process, this invention ensures that the execution efficiency after splitting is close to that of the original task. For example, in an 8MB data task, splitting it into eight 1MB subtasks only increases the total execution time by a small percentage. The result reassembly module merges the output data and metadata after all subtasks complete their callbacks and returns them to the application layer, appearing as if it were the execution result of a single task. This technology effectively mitigates SLA violations in JCT-sensitive applications in coexisting scenarios, for example, restoring the average JCT from 996µs under contention to 730.26µs, while simultaneously improving overall system utilization.

[0043] Workload-driven scheduling: To achieve a balance between fairness and resource utilization, this invention designs a workload-driven scheduling technique. This technique dynamically adjusts resource allocation through a two-tier architecture, predicting demand based on application workload characteristics and compensating for SLA violations. Specifically, the scheduling system includes a global ASTRAEA scheduler and an ASTRAEA agent for each application.

[0044] The global ASTRAEA scheduler runs as a daemon process, every [time period]. (In one embodiment of the invention, the value is 1ms, i.e., 1000µs) Resource allocation decisions are executed periodically. The scheduler communicates with the ASTRAEA agent of each application through shared memory to read the resource usage of each application in the previous period. and number of SLA violations And write the time quota for the new cycle. and split granularity .

[0045] The scheduler uses the Exponential Weighted Moving Average (EWMA) algorithm to predict the resource requirements of each application. Let the application... In the The actual resource usage for each scheduling cycle is: (Unit: microseconds), allocation amount is (Unit: microseconds), then the demand forecast for the next period. Calculate using the following formula: in This is the smoothing coefficient, and its value range is... Smaller Values ​​make forecasts smoother and can filter out short-term fluctuations; larger values... This value makes the forecast more sensitive to recent changes. In one embodiment of the invention, The formula was designed considering two factors: firstly, historical usage data. Secondly, it reflects the actual load characteristics of the application, and secondly, it reflects the historical allocation. Maintain the stability of allocation and avoid resource fluctuations caused by drastic changes.

[0046] Based on demand forecasting, the scheduler further incorporates historical SLA violations to implement compensatory resource allocation. Let the application... In the The number of SLA violations within each period is The sum of the predicted demand for all applications is: The sum of the number of SLA violations is: in This represents the number of currently active applications. The total allocatable time for each scheduling cycle is... A portion of this is reserved as SLA compensation quota: in To reserve a certain percentage, in one embodiment of the invention, this value is set to 0.7. The remaining portion is the regular allocation quota. application In the Time quota for each cycle Calculated according to the following rules: Scenario 1: When (No SLA violation) At this point, resources are allocated entirely according to the predicted demand ratio, without the need for compensation.

[0047] Scenario 2: When (When SLA violation exists) At this time, the regular quota Allocation is based on demand, while reserved compensation quotas are allocated proportionally. Distributing resources proportionally to the number of SLA violations to affected applications ensures fair long-term compensation. This mechanism guarantees that applications disrupted in the previous cycle will receive more resources in subsequent cycles, thereby restoring their service quality.

[0048] To prevent resource starvation in extreme cases (e.g., when the application starts up and usage is zero), a lower limit protection is set if the calculated allocation is too small: At the end of each scheduling cycle, the scheduler resets the SLA violation counters and usage accumulators for each application to zero, in preparation for the statistics of the next cycle.

[0049] In addition to time quota allocation, the ASTRAEA scheduler dynamically adjusts the granularity of task splitting for each application to strike a trade-off between fairness and utilization. Let's say the application... The current splitting granularity is In ordered granularity sets The index in is (Right now ).application Resource utilization rate is defined as the ratio of actual usage to allocated amount: The scheduler adjusts the granularity of each application in each scheduling cycle according to the following rules: Rule 1: Increased granularity driven by utilization When the application's resource utilization is low ( In one embodiment of the present invention This indicates that the application's allocation exceeds actual demand, and the task size is small or the load is light. In this case, the additional overhead caused by fine-grained splitting (such as buffer management and callback processing) becomes a larger proportion of the task's execution time, leading to decreased efficiency. Therefore, the scheduler elevates the application's splitting granularity to the next larger level within the ordered set: That is, in ordered sets Select the successor element of the current granularity; if it is already the largest granularity, leave it unchanged.

[0050] Rule 2: Reduce the granularity of SLA violation drivers When competing applications (let's call them applications) , When the number of SLA violations exceeds the threshold ( In one embodiment of the present invention This indicates that the current scheduling granularity is too large, causing head-of-line blocking, and delay-sensitive applications are being disrupted as they wait for large tasks to complete. In this case, the scheduler will apply... The splitting granularity is reduced to the next smaller level in the ordered set: That is, in ordered sets Select the predecessor element of the current granularity; if it is already the smallest granularity, leave it unchanged.

[0051] The core principle of this dynamic adjustment mechanism is: when the system load is low and resource contention is not intense, a larger granularity is used to reduce splitting overhead and maximize throughput; when the system load is high and there is significant resource contention, a smaller granularity is used to achieve fine-grained time slice interleaving, ensuring that the SLA for latency-sensitive applications is met. By combining granularity adjustment with time quota allocation, this invention achieves fine-grained management of DPU accelerator resources.

[0052] Each application's ASTRAEA agent intercepts task submissions via LD_PRELOAD, manages subtasks using a virtual queue, and pushes them to a hardware queue via a dedicated submission thread. This supports proactive tail latency protection to prevent low-probability high-latency events. The core principles of this technology include periodic low-frequency decision-making, workload-aware allocation (such as demand forecasting), and long-term fair compensation, ensuring the recovery of performance for latency-sensitive applications in contention scenarios. For example, it increases the accelerator usage time acquired by CDN applications in each scheduling period from 164.96µs in a starved state to a stable 354.99µs.

[0053] Application examples: The ASTRAEA system of this invention primarily runs on a Data Processor (DPU), specifically implemented for commercial DPU platforms such as NVIDIA BlueField-3. The core components of the system are written in C / C++, with a total code size of approximately 1800 lines, ensuring lightweight deployment without requiring modification of the DPU hardware or vendor-specific software (such as the DOCA SDK). The system is transparently integrated into existing application environments: the ASTRAEA agent (AB) is a dynamic library preloaded into the application process using the LD_PRELOAD environment variable to intercept DOCA API calls; the global ASTRAEA scheduler (AS) runs as an independent daemon process, utilizing shared memory to achieve lock-free communication between AS and AB, minimizing overhead and ensuring stability. The deployment process includes two phases: offline profiling and online execution. First, in the offline profiling phase, the user runs a profiling script on the DPU. This script systematically submits various task loads, collects execution data, and generates resource model files, which are subsequently loaded into the scheduling system. Secondly, during the online execution phase, the ASTRAEA source code is compiled and installed: the binary files are built using the standard Meson toolchain, the agent library is placed in the application path, and the scheduler daemon is started. By specifying SLA parameters (such as weight allocation or JCT thresholds) in the configuration file, the system can enter the running state, supporting multiple applications coexisting. The entire deployment requires no DPU kernel restart, takes only a few minutes to complete, and seamlessly supports PaaS or SaaS models in cloud environments. For example, on NVIDIA BlueField-3, the scheduler runs every 1ms cycle, consuming less than 3.54% of the CPU and 2.6MB of memory of one ARM core, while the static overhead of the agent is 32MB per application. This implementation is open-sourced on GitHub for easy expansion and verification.

[0054] While some embodiments of this disclosure have been shown and described, those skilled in the art will understand that modifications may be made to these embodiments without departing from the principles and spirit of this disclosure, which are defined by the claims and their equivalents.

Claims

1. A fine-grained, lightweight method for isolating DPU performance, characterized in that, include: First, we enter the offline phase and systematically analyze the task execution mode through the resource representation part of the offline analysis, and establish a time prediction model to model the mapping relationship between task parameters and resource consumption. Then, in the online phase, the application task submission is intercepted, resource requirements are evaluated based on the offline model, the application adaptively splits and reassembles the results to partially decompose large tasks, and executes a workload-guided scheduling process, which is dynamically allocated through a global scheduler and application agent; the global scheduler allocates time slices or subtasks based on workload characteristics; after execution, the results are reassembled and returned to the application.

2. The fine-grained, lightweight DPU performance isolation method as described in claim 1, characterized in that, The implementation method of the resource representation part through offline analysis is as follows: the accelerator resources are represented as time slices, and the task resource usage is mapped to the execution time by taking advantage of FCFS scheduling; in the offline stage, a variety of representative tasks are run, the execution time and input parameters are collected, and a regression model is established.

3. The fine-grained, lightweight DPU performance isolation method as described in claim 1, characterized in that, The adaptive task splitting and result reorganization implementation method is as follows: For a given accelerator task, first evaluate the task size and its correlation with execution time. If the task exceeds a preset threshold, split it into several sub-tasks. Each sub-task is submitted to the hardware queue for execution independently. The splitting process takes into account the algorithm characteristics. Meanwhile, several optimization measures are adopted: First, a reusable object pool is used to pre-allocate task objects and buffers, reducing the latency of dynamic memory allocation; second, the pipelined overlap of software and hardware execution prepares the input of the next subtask immediately after the subtask is submitted, realizing parallel processing; third, a zero-copy scatter / aggregate list avoids unnecessary memory copy operations and only references data fragments through pointers.

4. The fine-grained, lightweight DPU performance isolation method as described in claim 1, characterized in that, The workload-driven scheduling process specifically includes a global scheduler and an agent for each application. The global scheduler schedules every [period]. MS periodically performs low-frequency decisions, uses an exponentially weighted moving average algorithm to predict application resource requirements, and implements a compensation mechanism based on historical SLA performance; Meanwhile, the global scheduler dynamically adjusts the granularity of task splitting: when competition is fierce, it increases the splitting to ensure fine-grained fairness; when the load is low, it reduces the splitting to maximize throughput.

5. The fine-grained, lightweight DPU performance isolation method as described in claim 3, characterized in that, The method for evaluating the task size and its correlation with execution time, and for splitting the task into several sub-tasks if it exceeds a preset threshold, is as follows: For a given accelerator task, the task size is evaluated based on the time prediction model, and the model includes task parameters such as the number of data blocks. Number of redundant blocks Block size Mapped to predicted execution time This serves as a quantitative representation of task resource consumption; a dynamic granularity control mechanism is employed to determine the task splitting threshold, assuming the current splitting granularity is... Its value range is a predefined ordered discrete set. ,in , The total number of granularity levels, split into granularities. The ASTRAEA scheduler dynamically sets the parameters based on the system runtime state and passes them to the application agents of each application via shared memory. For the data block size of the original task The application agent calculates the number of subtasks using the following formula. : When there are multiple competing applications in the system and At that time, the original task was broken down into Subtasks; otherwise, submit the task as is to avoid unnecessary splitting overhead. The decomposition process for erasure coding accelerator tasks is based on the mathematical separability of erasure coding algorithms, for tasks containing... Source data matrix of data blocks Divide it into columns Submatrix, the nth submatrix Submatrix Defined as: in Representation matrix The Listed to number Submatrices composed of columns, each submatrix having a size of It can perform encoding or decoding operations independently; For the encoding task, let the encoding matrix be... The encoded output of the original task is: Output of each subtask Calculated independently, the final result is reassembled horizontally: ; Predicted execution time for each subtask ,calculate: This forecast is used for subsequent time quota management and scheduling decisions.

6. The fine-grained, lightweight DPU performance isolation method as described in claim 5, characterized in that, In order to minimize the overhead introduced by the process of splitting into several subtasks, a complete task processing pipeline is designed, including four stages: task splitting, buffer management, queue scheduling, and result reassembly. Phase 1: Task Interception and Splitting Decisions The proxy intercepts the application's calls to the accelerator API using dynamic link library preloading technology. When the application submits the original task, the proxy first reads the current splitting granularity set by the scheduler from shared memory. The number of subtasks is calculated according to the aforementioned formula. Meanwhile, the agent calculates the expected completion time for this task: in This is the current timestamp. If the service level target configured for the application is Then proceed to the task splitting process; Otherwise, submit the original task directly to the next stage; Phase Two: Subtask Creation and Buffer Allocation For tasks that need to be split, the agent handles each subtask. Perform the following operations: Source buffer construction: Sub-source buffers are created using a scatter / cluster list technique, with pointer references. For each data block... ( The starting address of the sub-buffer is calculated as follows: The data length is Multiple sub-buffers are linked together through a linked list structure to form a complete subtask input, and the entire process does not require data copying; Target Buffer Acquisition: A free target buffer is acquired from a pre-allocated pool of reusable buffers. This pool is created during system initialization and contains a fixed number of pre-allocated buffer objects, each sized to hold the maximum granularity of output data. This is determined by a circular index. Enables fast allocation and avoids the overhead of dynamic memory allocation at runtime; Subtask encapsulation: The source buffer, target buffer, encoding / decoding matrix, and user callback information are encapsulated into a subtask object, while the predicted execution time of the subtask is recorded. Subtask Index And a flag indicating whether it is the last subtask; Phase 3: Virtual Queue Management and Pipeline Submission The encapsulated subtasks are placed into a virtual queue maintained by the agent. This queue employs a lock-free circular buffer design, with a capacity of [value missing]. The head pointer is The tail pointer is The enqueue operation is as follows: The agent prepares a buffer for the next subtask immediately after the subtask is enqueued; The proxy maintains a dedicated commit thread that continuously polls the virtual queue. When it detects a subtask to be committed in the queue, the commit thread first queries the current time quota in shared memory. If one of the following conditions is met, the subtask will be submitted to the hardware queue: Condition 1: Sufficient quota. This means that there is still a remaining time quota within the current scheduling period; Condition 2, Tail Delay Protection: This means that the task is about to time out, triggering the active tail delay protection mechanism to avoid SLA violation; After each successful submission, update the time quota and cumulative usage: in This represents the cumulative resource usage of the application during this scheduling period. Phase Four: Result Callback and Data Reorganization Once the hardware accelerator completes the subtask execution, it triggers the registered callback function, which performs the following operations: Data copying: If it is a subtask resulting from splitting, the output data of the subtask is copied from the temporary target buffer to the corresponding position in the target buffer of the original task. For encoding tasks, the output includes... The redundant blocks are copied to the target address as follows: Resource release: Release the source buffer reference occupied by the subtask, reset the data length of the temporary target buffer to 0, so that it can be reused by subsequent subtasks; Complete the judgment and return the result: If the current subtask is the last subtask of the original task, then perform the following operations: Check actual completion time Did it exceed the expected time? ,like Then increment the SLA violation counter for shared memory: ; Call the original callback function registered by the application and return the complete reassembly result.

7. The fine-grained, lightweight DPU performance isolation method as described in claim 4, characterized in that, The global scheduler every The process by which MS periodically performs low-frequency decisions, uses an exponentially weighted moving average algorithm to predict application resource requirements, and implements a compensation mechanism based on historical SLA performance is as follows: every [period]... = The scheduler periodically performs resource allocation decisions. It communicates with each application's agent via shared memory, reading the application's resource usage from the previous cycle. and number of SLA violations And write the time quota for the new cycle. and split granularity ; The scheduler uses an exponentially weighted moving average algorithm to predict the resource requirements of each application. Let the application... In the The actual resource usage for each scheduling cycle is: The allocation amount is Then the demand forecast for the next period Calculate using the following formula: in This is the smoothing coefficient, and its value range is... ; Based on demand forecasting, the scheduler further incorporates historical SLA violations to implement compensatory resource allocation, setting up applications... In the The number of SLA violations within each period is The sum of the predicted demand for all applications is: The sum of the number of SLA violations is: in This represents the number of currently active applications. The total allocatable time for each scheduling cycle is... A portion of this is reserved as SLA compensation quota: in To reserve a certain percentage, the remaining portion is the regular allocation quota: application In the Time quota for each cycle Calculated according to the following rules: Scenario 1: When When there is no SLA violation At this point, resources are allocated entirely according to the predicted demand ratio, and no compensation is required; Scenario 2: When When there is an SLA violation At this time, the regular quota Allocation is based on demand, while reserved compensation quotas are allocated proportionally. Distribute compensation to affected applications proportionally based on the number of SLA violations to achieve long-term fair compensation. To prevent resource starvation in extreme cases, a lower limit is set to protect resources if the calculated allocation is too small: At the end of each scheduling cycle, the scheduler clears the SLA violation counters and usage accumulators for each application to prepare for the statistics of the next cycle. In addition to time quota allocation, the scheduler also dynamically adjusts the task splitting granularity for each application, setting the application... The current splitting granularity is In ordered granularity sets The index in is ,Right now ,application Resource utilization rate is defined as the ratio of actual usage to allocated amount: The scheduler adjusts the granularity of each application in each scheduling cycle according to the following rules: Rule 1: Increased granularity driven by utilization When the application's resource utilization is low, The scheduler then elevates the granularity of the application's split to the next larger level within the ordered set: That is, in ordered sets Select the successor element of the current granularity; if it is already the largest granularity, leave it unchanged. Rule 2: Reduce the granularity of SLA violation drivers When the number of SLA violations by competing applications exceeds a threshold, The scheduler will apply The splitting granularity is reduced to the next smaller level in the ordered set: That is, in ordered sets Select the predecessor element of the current granularity; if it is already the smallest granularity, leave it unchanged.

Images