A unified time-division resource isolation method and system for heterogeneous accelerators

Abstract

The application discloses a kind of unified time division resource isolation methods and systems for heterogeneous accelerator, belong to electronic information technical field, including S1, in task abstraction and scheduling object modeling stage;S2, in time slice parameter calculation and scheduling strategy generation stage;S3, in scheduling control entity construction stage;S4, in the access admission and runtime constraint stage of execution stage;S5, in state feedback and dynamic adjustment stage.The application is not simply in a certain specific level patch type modification, but is through hierarchical abstraction, and the closed loop control path proposed.The implementation of the application can provide a set of logical consistent, flexible and performance loss controlled unified time division resource management paradigm for heterogeneous accelerator.

Description

Technical Field

[0001] This invention belongs to the field of electronic information technology, and specifically relates to a unified time-division resource isolation method and system for heterogeneous accelerators. Background Technology

[0002] With the explosive growth of artificial intelligence, high-performance computing (HPC), and large-scale data processing applications, heterogeneous accelerators (such as GPUs, NPUs, and domestically developed architecture accelerators) have evolved from auxiliary computing units into the core computing power engines of modern computing systems. In typical scenarios such as cloud computing, multi-tenant servers, and scientific research shared clusters, the concurrent sharing of a single physical acceleration device by multiple processes has become an inevitable choice to improve hardware utilization, reduce energy efficiency ratios, and optimize overall throughput.

[0003] In existing heterogeneous computing systems, the resource management logic of accelerators is mainly carried out by the driver and runtime environment provided by the vendor. Its core task submission mechanism usually follows the path below: the user program constructs the computing task through the runtime application programming interface (API), encapsulates it into a sequence of instructions and writes it into the command buffer, and then notifies the accelerator hardware to execute it through the doorbell register or a specific system call (ioctl).

[0004] From the perspective of underlying architecture, current mainstream heterogeneous accelerators exhibit two distinctly different technical approaches in task submission and scheduling models: The first is the user-space direct mapping model. Mature architectures, exemplified by NVIDIA, allow drivers to directly map command buffers to the user-space address space. User programs can bypass kernel space, continuously writing instructions and updating the producer pointer in user space. This "decentralized" submission method significantly reduces system call overhead and can support extremely high task throughput. The second is the kernel-space centralized management model. Some domestic accelerator architectures, for security, instruction auditing, or hardware control considerations, require all task submissions to enter kernel space via the ioctl system call, with the driver kernel module uniformly handling instruction construction and hardware triggering.

[0005] In multi-tasking concurrent environments, both models face the challenge of achieving fair resource allocation and isolation in the absence of underlying hardware preemption capabilities. Time-slicing, a recognized mature resource reuse method in the operating system field, theoretically provides clear scheduling semantics and strong isolation. However, due to the significant generational differences in physical characteristics such as driver interfaces, instruction boundary control, and controllable granularity among heterogeneous accelerators, traditional time-division multiplexing mechanisms are difficult to directly migrate. How to construct a unified time-division resource scheduling model across devices and architectures while maintaining application transparency has become a critical issue urgently needing to be addressed in the field of heterogeneous computing system software.

[0006] Deficiencies and shortcomings of existing technologies While existing technologies have made some progress in supporting accelerator concurrency, they still have significant limitations in achieving cross-platform consistency isolation and efficient scheduling, specifically in the following dimensions: First, existing mechanisms are highly dependent on specific hardware architectures and lack a unified scheduling abstraction across platforms. Accelerators from different vendors differ fundamentally in command buffer organization, task granularity, and synchronization mechanisms. Existing scheduling schemes are often deeply coupled to the underlying drivers of a specific vendor, resulting in scattered scheduling points with inconsistent semantics. This makes it difficult for system administrators to implement a unified resource quota policy in hybrid clusters containing multiple heterogeneous accelerators, creating de facto "vendor lock-in" and "scheduling silos."

[0007] Secondly, existing technologies face the risk of control failure under both "open" and "closed" models. In the user-mode direct mapping model, the kernel struggles to detect the timing of user-mode instruction writes in a timely manner, lacking effective "braking" mechanisms. This results in randomness in task execution order, making critical tasks susceptible to interference from background tasks. In the kernel-mode centralized management model, while the kernel possesses theoretical control, frequent task submissions lead to an exponential increase in system call counts. If the scheduling mechanism simply relies on this path, context switching overhead will be further amplified, resulting in severe performance jitter and making it difficult to support high-precision microsecond-level time slicing.

[0008] Secondly, existing control paths are singular and lack unified constraints on execution boundaries. Current mainstream solutions are typically limited to a single level (such as simple API interception or simple kernel ioctl interception). Simple API interception (the existing technology described in Solution 3) is easily bypassed and struggles to handle non-standard library calls; while simple kernel interception, when dealing with highly abstract instruction flows, often lacks semantic context and thus fails to achieve fine-grained control. The existing technology stack lacks a framework capable of fusing and modeling control paths at different levels (user mode and kernel mode), resulting in poor robustness in dealing with complex driver evolution.

[0009] Finally, the scheduling strategy is deeply coupled with the physical implementation, lacking the flexibility for dynamic adjustment. Existing solutions typically hard-code the scheduling logic into specific interception plugins, making it impossible to dynamically adjust weights based on the accelerator's real-time load, task characteristics (latency-sensitive vs. throughput-intensive), and hardware topology. This static scheduling model struggles to meet the dual requirements of modern data centers for deterministic latency and fairness guarantees.

[0010] In summary, existing heterogeneous accelerator management technologies have significant shortcomings in multi-task concurrency isolation, cross-architecture versatility, and the balance between scheduling accuracy and performance loss. This not only restricts the large-scale deployment of domestically produced heterogeneous chips in complex system environments but also hinders the evolution of computing resources towards standardization, refinement, and measurability. Summary of the Invention

[0011] Existing heterogeneous accelerator management technologies have significant shortcomings in terms of multi-task concurrency isolation, cross-architecture versatility, and the balance between scheduling accuracy and performance loss. This invention proposes a unified time-division resource scheduling model and method for heterogeneous accelerators, demonstrating its significant necessity. This invention does not simply involve patching modifications at a specific level, but rather proposes a closed-loop control path through layered abstraction. The implementation of this invention can provide heterogeneous accelerators with a logically consistent, flexible, and performance-controlled unified time-division resource management paradigm.

[0012] To achieve the above objectives, the present invention adopts the following technical solution: A unified time-division resource isolation method for heterogeneous accelerators includes: S1. In the task abstraction and scheduling object modeling stage: heterogeneous computing tasks from different processes are standardized and abstracted, and tasks of different sources and types are uniformly converted into scheduling objects SO containing process identifiers, device identifiers, queue positions, weight parameters, and execution states. i This results in a standardized set of scheduling objects Ω that eliminates differences in the underlying hardware. S2. In the time slice parameter calculation and scheduling strategy generation stage: Based on the set of scheduling objects Ω, define the system scheduling period T_cycle, and according to each scheduling object SO i Weight parameter W i Calculate the time slice length Δt i Generate a scheduling policy sequence SP containing the execution order and duration, thereby obtaining an ordered execution permission allocation scheme within a scheduling cycle; S3. In the scheduling control entity construction phase: Based on the scheduling policy sequence SP, a cross-process scheduling control base entity is constructed for real-time issuance of admission identifiers; S4. During the execution phase, in the access admission and runtime constraint phase: Based on the scheduling control base entity, physical or logical admission control and runtime constraints are implemented on task submission behavior to ensure that only admitted processes are allowed within the specified time slice parameter Δt. i Internal access accelerator; S5. During the status feedback and dynamic adjustment phase: Real-time monitoring of the actual execution time and resource utilization during the execution phase. i The feedback results are then used to dynamically adjust the time slice parameter Δt for the next cycle. i and weight parameter W i This forms a closed-loop adaptive scheduling, thereby achieving deterministic time-division resource isolation and fair sharing of heterogeneous accelerators under multi-task concurrency.

[0013] A further improvement of this invention is that, in the task abstraction and scheduling object modeling stage, S1 includes: capturing heterogeneous computing call requests submitted by user processes; parsing the process identifier and target device identifier of the task; and constructing a scheduling object SO for each task, containing a process identifier, device identifier, queue position, weight parameters, and execution status. i ; and will schedule all SO objects i Insert it into the standardized scheduling object set Ω.

[0014] A further improvement of this invention is that, in the time slice parameter calculation and scheduling strategy generation stage, S2 includes: defining a system global scheduling period T_cycle; within the scheduling period T_cycle, according to each scheduling object SO... i Weight parameter W i The time slice length Δt for each scheduled object is calculated using a weighted proportional model. i , where Δt i = max(T_min, (W i / ΣW j )·T_cycle); and generate an ordered scheduling policy sequence SP = [(SO1, Δt1),(SO2, Δt2), …, (SO k, Δt k )).

[0015] A further improvement of this invention is that, in the scheduling control entity construction phase, S3 includes: constructing a shared memory data structure as a carrier for issuing scheduling instructions; the shared memory data structure includes an admission process identifier, a cycle start timestamp, and status slot information for each process; the independent controller process reads the system monotonic clock to determine the current cycle start point; and according to the scheduling policy sequence SP, in each time slice Δt... i The starting point updates the admission process identifier through atomic operations; and triggers the wake-up mechanism of the controlled process through a named semaphore.

[0016] A further improvement of this invention is that, in step S4, during the access permission and runtime constraint phase of the execution phase, at least one of the following three implementation schemes is adopted: The first implementation scheme is as follows: For the direct mapping model accelerator, the memory area corresponding to the accelerator instruction submission buffer is set to read-only. When a non-admitted process attempts to write, a page protection error signal is triggered. In the signal handling function, it is checked whether the current process is an admitted process. If they match, the write permission is temporarily restored and the single-step execution mode is enabled to immediately fall back the permission after completing a single write. Otherwise, the process is suspended and waits for the next admitted flag. The second implementation scheme is as follows: For the centralized management model accelerator, the driver layer intercepts system calls and injects microsecond-level delays into high-frequency task submission instructions. When the call counter reaches the preset trigger factor, the kernel sleep interface is called to force the current process to delay submission. The third implementation scheme: For user-space applications, symbol redirection is performed in the application programming interface call path. First, an admission check is performed to obtain a suggested rate-limiting sleep time. If this time is greater than zero, a microsecond-level sleep operation is executed. Then, notifications are inserted before and after the original task to record the start and end points, and the timing results are pushed to an asynchronous timing thread pool for resource utilization. i Accumulate.

[0017] A further improvement of this invention is that, in the state feedback and dynamic adjustment phase, step S5 includes: real-time monitoring of each scheduling object SO. i The actual physical delay L from task submission trigger to completion i and resource utilization rate r i When the actual usage deviates from the expected time slice Δt i At that time, the weighting parameter W is adjusted according to the proportional coefficient. i If a process makes no requests within its allocated time slice, the time slice is reclaimed, and the corrected weight parameter W is applied. i and time slice parameter Δt i Feedback is fed back to the time slice parameter calculation and scheduling strategy generation stage to achieve closed-loop dynamic evolution.

[0018] A further improvement of this invention is that it also includes unified processing of graphics rendering load and computation load: by intercepting and analyzing the general instruction flow of the underlying driver, graphics rendering tasks and computation tasks are included in the same scheduling object set Ω and scheduling policy sequence SP, based on a unified weight parameter W. i Dynamically adjust the proportion of accelerator core usage for various tasks.

[0019] A unified time-division resource isolation system for heterogeneous accelerators includes: Task Abstraction and Scheduling Object Modeling Unit: This unit standardizes and abstracts heterogeneous computing tasks from different processes, transforming tasks of different origins and types into scheduling objects (SOs) that include process identifiers, device identifiers, queue positions, weight parameters, and execution states. i This results in a standardized set of scheduling objects Ω that eliminates differences in the underlying hardware. Time slice parameter calculation and scheduling strategy generation unit: Based on the set of scheduling objects Ω, it defines the system scheduling period T_cycle and generates scheduling strategies according to each scheduling object SO. i Weight parameter W i Calculate the time slice length Δt i Generate a scheduling policy sequence SP containing the execution order and duration, thereby obtaining an ordered execution permission allocation scheme within a scheduling cycle; Scheduling control entity construction unit: Based on the scheduling policy sequence SP, a cross-process scheduling control base entity is constructed for real-time issuance of admission identifiers; The access admission and runtime constraint unit during the execution phase: Based on the scheduling control base entity, it implements physical or logical admission control and runtime constraints on task submission behavior, thereby ensuring that only admitted processes are allowed within the specified time slice parameter Δt. i Internal access accelerator; Status feedback and dynamic adjustment unit: Real-time monitoring of actual execution time and resource utilization during the execution phase. i The feedback results are then used to dynamically adjust the time slice parameter Δt for the next cycle. i and weight parameter W i This forms a closed-loop adaptive scheduling, thereby achieving deterministic time-division resource isolation and fair sharing of heterogeneous accelerators under multi-task concurrency.

[0020] A further improvement of this invention lies in that the task abstraction and scheduling object modeling unit specifically includes: capturing heterogeneous computing call requests submitted by user processes; parsing the process identifier and target device identifier of the task; and constructing a scheduling object SO for each task, containing a process identifier, device identifier, queue position, weight parameters, and execution status. i ; and will schedule all SO objects iInsert it into the standardized scheduling object set Ω.

[0021] A computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of a unified time-division resource isolation method for heterogeneous accelerators.

[0022] Compared with the prior art, the present invention has at least the following beneficial technical effects: This invention addresses the common problems in existing heterogeneous accelerator resource management technologies, such as limited isolation methods, insufficient support for closed drivers, fragmented scheduling semantics, and inability to effectively adapt to mixed loads. It proposes a unified time-division resource isolation method and system for heterogeneous accelerators, which has the following significant advantages: I. By employing a "three-layer parallel, optional adaptation" isolation path design, this invention fundamentally solves the compatibility problem caused by driver differences across heterogeneous platforms. Existing accelerator resource isolation solutions often adhere to only a single level (such as user-space API interception only), leading to failure on platforms with closed driver code or opaque hardware interfaces. This invention, by designing three parallel, optional isolation paths—kernel layer, driver interaction layer, and application library layer—during the S4 execution phase, can flexibly match the optimal physical control method based on the driver openness and instruction submission characteristics of different accelerator hardware. For direct mapping model platforms such as NVIDIA, a page protection path can be enabled; for centralized management model platforms such as Cambricon, an ioctl interception path can be enabled. This "multi-path mapping, unified access" design significantly improves the system's versatility, enabling this invention to cover the vast majority of known vendors and domestically developed heterogeneous accelerator architectures.

[0023] Second, a kernel-mode access control mechanism based on page protection errors is introduced, achieving mandatory physical-level resource isolation. Addressing the issues of existing user-mode interception schemes (such as Scheme 3) being easily bypassed and having weak control, this invention employs a technique that manipulates the page table permissions of the Memory Management Unit (MMU) in Scheme 1. Because the instruction commit area (UserD) is set to read-only by mprotect, any write operation by a process outside the permitted time slice will trigger a hardware-level page protection error (SIGSEGV). Therefore, this invention can forcibly block illegal requests at the hardware access entry point. This mechanism does not rely on the voluntary cooperation of user programs, establishing an absolute access boundary from the operating system level, significantly enhancing the reliability, determinism, and security of isolation.

[0024] Third, by employing ioctl hooking technology in the driver interaction layer, the scheduling overhead of high-frequency task requests in a closed driver environment is effectively solved. In closed architecture platforms where the driver source code is unavailable, this invention directly hijacks the critical path of instruction issuance by implementing ioctl interception in the driver interaction layer (as in implementation scheme two). Because high-frequency task codes such as SBTS_INVOKE_TASK_NR are identified and injection delayed in the kernel-mode hook function, this invention can achieve precise fine-tuning of the task submission rhythm without modifying the driver source code or relying on complex API semantics. This design not only avoids the context overhead caused by frequent switching between user mode and kernel mode, but also enables the system to handle tens of thousands of task requests per second with minimal performance cost, significantly improving scheduling accuracy in high-performance computing scenarios.

[0025] Fourth, by constructing a unified resource scheduling semantic layer, consistency in scheduling semantics across levels and paths is achieved. In existing technologies, different isolation methods often lead to fragmentation of control logic. This invention addresses this by introducing a unified five-tuple scheduling object SO in stage S1. i The model maps requests intercepted from different levels (such as kernel-mode page faults, driver-mode ioctl calls, and user-mode API calls) into standardized scheduling entities. This layered abstraction and fusion design allows the upper-level scheduler to calculate the time slice Δt in phase S2. i When using policy sequences (SP), there is no need to be aware of the underlying physical capture method. Therefore, this invention can manage multiple accelerators with a single set of logic, greatly reducing the development and maintenance complexity of the scheduling system and achieving deep decoupling between scheduling logic and execution paths.

[0026] Fifth, it achieves unified time-division sharing of computational and graphics rendering workloads, improving resource utilization in complex scenarios. Existing management solutions mostly focus on computational workloads such as CUDA / MLU, making it difficult to handle graphics rendering tasks such as OpenGL / Vulkan. This invention achieves unified ordering and execution of computational and rendering tasks on the same accelerator by performing equivalent identification of graphics API requests at the driver interaction layer and incorporating them into the weighted proportional allocation model of the S2 stage. This feature ensures that in scenarios such as cloud workstations and hybrid computing nodes, rendering tasks will not arbitrarily preempt computing resources, and critical computational tasks will not experience uncontrollable delays due to the presence of rendering tasks, thereby achieving optimal overall accelerator resource allocation.

[0027] VI. A closed-loop dynamic adjustment mechanism based on execution duration feedback is introduced, enabling adaptive evolution of the scheduling strategy. Traditional time-division multiplexing mechanisms typically employ static time slice allocation. This invention, by introducing an execution duration feedback loop based on a notifier and clock timing in stage S5, can calculate the actual occupancy rate r of each scheduled object in real time. i The execution delay L_i is considered. When the actual consumption deviates from the expected model, the system can automatically provide feedback and correct the weight parameter W in stage S2. i This closed-loop logic of "monitoring-analysis-feedback-correction" enables the invention to have strong self-healing and anti-jitter capabilities when facing production environments with drastic fluctuations in task load, ensuring business continuity.

[0028] VII. Balancing the enforceability of isolation with the engineering flexibility of deployment, this invention significantly reduces the marginal cost of system implementation. The three-layer linkage configuration of this invention allows system administrators to flexibly switch according to security requirements: lightweight API awareness can be enabled in R&D environments pursuing ultimate performance; kernel-level page protection can be enabled in production environments requiring stringent isolation. Because of this selectable configuration method, this invention can balance performance overhead and isolation strength, avoiding excessive performance loss in pursuit of isolation, and significantly reducing the engineering implementation cost of heterogeneous computing platforms during resource integration. Attached Figure Description

[0029] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0030] Figure 1 Overall architecture diagram of a three-layer linkage unified time-division resource isolation system for heterogeneous accelerators; Figure 2 A schematic diagram of the closed-loop control process and five-stage evolution of heterogeneous resource scheduling; Figure 3 Flowchart for task access control based on memory paging permission inducement and single-step debugging (Implementation Scheme 1); Figure 4 Logic diagram for kernel-level ioctl hook interception and task injection delay control (Scheme 2); Figure 5 This is a resource monitoring architecture diagram based on user-space API hijacking and asynchronous timer thread pool (Implementation Scheme 3). Figure 6 A timing diagram for the weighted time slice allocation of multiple processes within a unified scheduling period T_cycle; Figure 7 This is a block diagram of a unified time-division resource isolation system for heterogeneous accelerators. Detailed Implementation

[0031] In the following description, only certain exemplary embodiments are briefly described. As those skilled in the art will recognize, the described embodiments can be modified in various ways without departing from the spirit or scope of the invention. Therefore, the drawings and description are considered to be exemplary in nature and not restrictive.

[0032] In the description of this invention, it should be understood that, when used in this specification and the appended claims, the terms "comprising" and "including" indicate the presence of the described features, integrals, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0033] It should also be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the invention. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0034] It should also be further understood that the term "and / or" as used in this specification and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0035] The accompanying drawings illustrate various structural schematic diagrams according to embodiments disclosed in this invention. These drawings are not to scale, and some details have been enlarged for clarity, and some details may have been omitted. The shapes of the various regions and layers shown in the drawings, as well as their relative sizes and positional relationships, are merely exemplary and may deviate from reality due to manufacturing tolerances or technical limitations. Furthermore, those skilled in the art can design regions / layers with different shapes, sizes, and relative positions as needed.

[0036] The embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0037] Terminology Explanation: I. Basic Terminology of Heterogeneous Computing and Resource Management 1. A heterogeneous accelerator refers to a hardware device specifically designed to perform specific high-parallelism computing tasks. Its architecture differs from a general-purpose central processing unit (CPU), featuring more computing cores and a specially customized instruction set. Examples include graphics processing units (GPUs), neural network processors (NPUs), and artificial intelligence accelerators (AIAccelerators). The heterogeneous accelerator described in this invention is the controlled entity of this scheduling system.

[0038] 2. Resource Isolation refers to a technology that, through software or hardware-software collaborative mechanisms, ensures that the resources allocated to each task (such as execution time and computing units) do not encroach on each other when multiple tasks concurrently use the same accelerator. This invention solves the problem of performance jitter and quality of service (QoS) degradation caused by resource contention through "time-division multiplexing".

[0039] 3. Time-Slicing: A resource reuse technique that divides continuous physical execution time into discrete time slots. Within each time slot, only one specified task is allowed to access hardware resources. By rapidly rotating these time slots, multiple tasks can run "simultaneously" at a macroscopic level, but are executed strictly sequentially at a microscopic level.

[0040] 4. Deterministic Execution refers to the predictable and controllable execution order, start time, and duration of tasks on the accelerator. In complex concurrent environments, this invention ensures that critical tasks achieve the expected computational response time through time-division isolation.

[0041] II. Task Abstraction and Model-Related Terminology 5. Scheduling Object (SO) i This invention refers to the standardized logical abstraction of accelerator task submission behavior. It unifies tasks from different sources (different processes) and of different types (inference or training) into a single object containing a PID (Process Identifier), DID (Device Identifier), and W... i (Weight) and S i The five-tuple model of (states). This is the basic unit for the system to perform policy calculations.

[0042] 6. The Scheduling Policy Sequence (SP) refers to the execution instruction table calculated by the scheduler within a preset scheduling cycle (T_cycle) based on the weights and priorities of each scheduled object. This sequence specifies which process will start execution at what time within a cycle, and for how long.

[0043] 7. Occupancy Ratio (r) i This refers to the ratio of the total computing resources (cores × runtime) actually consumed by a specific task within the observation window to the total theoretical capacity of the accelerator. It is a core mathematical metric for evaluating task load intensity and triggering rate limiting logic.

[0044] III. Kernel-level and Hardware Interaction Layer Terminology 8. Doorbell Mechanism: A hardware notification mechanism for heterogeneous accelerators. Once the software has prepared the task instructions to be executed, it writes a specific value to a reserved memory address (the doorbell address) in the hardware, triggering the accelerator hardware to fetch instructions from memory and execute them.

[0045] 9. Doorbell Hijacking & Page Protection: By manipulating the CPU's Memory Management Unit (MMU), memory pages originally belonging to the accelerator task submission channel are set to a "read-only" or "inaccessible" state. When an application attempts to perform a "ring the doorbell" operation (i.e., a write operation), a page protection exception is triggered in the CPU. This invention utilizes this mechanism to capture the timing of task launch.

[0046] 10. Input / Output Control (ioctl): The standard system call interface in operating systems such as Linux for communication between user-space programs and device drivers. Accelerator task submission, status queries, and resource requests are typically accomplished through specific ioctl codes (such as SBTS_INVOKE_TASK_NR).

[0047] 11. ioctl hooking / interception refers to replacing or modifying the function jump table (file_operations) of the driver in the operating system kernel, causing pointers that originally pointed to the driver's native processing functions to point to hook functions defined in this invention. Through this method, the system can delay or reorder tasks before they enter the hardware driver.

[0048] 12. Page Fault Signal (SIGSEGV / Segmentation Fault): A signal triggered by hardware and delivered to the process by the operating system when a program attempts to access a memory region it does not have permission to access (such as writing data to a read-only page). In Implementation Scheme 1, this invention utilizes this signal as the scheduler's "admission trigger point".

[0049] 13. Trap Flag & Single Step Debugging: A flag bit in x86 and other processor architectures. When this bit is set, the processor automatically triggers an exception (SIGTRAP) after executing an instruction. This invention utilizes this mechanism to immediately re-close permissions after allowing a write operation, achieving precise control over single task submissions.

[0050] IV. User-space and API Interaction Layer Terminology 14. Dynamic Link Library Interception (LD_PRELOAD Interception): A technique in Linux environments that preloads a custom shared library, causing the application to enter a custom hook function before calling functions in the standard accelerator runtime library (such as libcndrv.so). This technique achieves zero-intrusion monitoring of the application.

[0051] 15. Symbol Redirection: When a program dynamically loads symbols, symbol references that originally pointed to the vendor's official API (such as cnInvokeKernelEx) are redirected to the wrapper function of this invention. This is the technical basis for achieving user-mode time-division isolation.

[0052] 16. Notifier / Event: Synchronization primitives provided by accelerator vendors. By inserting a notifier into the task queue, the precise timestamps of the task's start and end times on the hardware side can be obtained. This invention utilizes it to calculate the actual overhead of the task.

[0053] 17. Asynchronous Measurement and Reporting refers to a separate background thread that monitors task completion and calculates resource consumption, rather than synchronously waiting in the main computation process. This approach avoids additional computational latency introduced by timing operations, ensuring the high throughput of the accelerator.

[0054] V. Terminology for Dispatch Control and Closed-Loop Feedback 18. Weight Decay & Reset: A dynamic adjustment strategy. Over time, the influence of historical resource usage records on current scheduling decisions gradually decreases (decay), or statistical data is periodically cleared (reset) to ensure that the system can quickly respond to instantaneous changes in task load.

[0055] 19. Injection Delay: A small amount of sleep time artificially inserted into the task submission path. By controlling the length of the delay, the amount of tasks submitted to the hardware by each process per unit of time can be precisely adjusted, thereby achieving time-division sharing on a macroscopic level.

[0056] 20. Closed-loop Feedback Adjustment refers to a scheduling system continuously monitoring the deviation between the actual latency of task execution and the expected time slice, and automatically adjusting the scheduling weight W for the next cycle. i This mechanism enables the scheduling system to adapt and cope with task flows of varying computational intensity.

[0057] Example 1 This invention provides a unified time-division resource isolation method for heterogeneous accelerators, comprising: S1. In the task abstraction and scheduling object modeling stage: heterogeneous computing tasks from different processes are standardized and abstracted, and tasks of different sources and types are uniformly converted into scheduling objects SO containing process identifiers, device identifiers, queue positions, weight parameters, and execution states. i This results in a standardized set of scheduling objects Ω that eliminates differences in the underlying hardware. S2. In the time slice parameter calculation and scheduling strategy generation stage: Based on the set of scheduling objects Ω, define the system scheduling period T_cycle, and according to each scheduling object SO i Weight parameter W i Calculate the time slice length Δt i Generate a scheduling policy sequence SP containing the execution order and duration, thereby obtaining an ordered execution permission allocation scheme within a scheduling cycle; S3. In the scheduling control entity construction phase: Based on the scheduling policy sequence SP, a cross-process scheduling control base entity is constructed for real-time issuance of admission identifiers; S4. During the execution phase, in the access admission and runtime constraint phase: Based on the scheduling control base entity, physical or logical admission control and runtime constraints are implemented on task submission behavior to ensure that only admitted processes are allowed within the specified time slice parameter Δt. i Internal access accelerator; S5. During the status feedback and dynamic adjustment phase: Real-time monitoring of the actual execution time and resource utilization during the execution phase. i The feedback results are then used to dynamically adjust the time slice parameter Δt for the next cycle. i and weight parameter W i This forms a closed-loop adaptive scheduling, thereby achieving deterministic time-division resource isolation and fair sharing of heterogeneous accelerators under multi-task concurrency.

[0058] In this embodiment, S1, the task abstraction and scheduling object modeling stage, includes: capturing heterogeneous computing call requests submitted by user processes; parsing the process identifier and target device identifier of the task; and constructing a scheduling object SO for each task, containing the process identifier, device identifier, queue position, weight parameters, and execution status. i ; and will schedule all SO objects i Insert it into the standardized scheduling object set Ω.

[0059] In this embodiment, S2, the time slice parameter calculation and scheduling strategy generation stage, includes: defining a system global scheduling period T_cycle; within the scheduling period T_cycle, according to each scheduling object SO i Weight parameter W i The time slice length Δt for each scheduled object is calculated using a weighted proportional model. i , where Δt i = max(T_min, (W i / ΣW j )·T_cycle); and generate an ordered scheduling policy sequence SP = [(SO1, Δt1), (SO2, Δt2), …, (SO k , Δt k )).

[0060] In this embodiment, S3, the scheduling control entity construction phase includes: constructing a shared memory data structure as a carrier for issuing scheduling instructions; the shared memory data structure includes an admission process identifier, a cycle start timestamp, and status slot information for each process; the independent controller process reads the system monotonic clock to determine the current cycle start point; and according to the scheduling policy sequence SP, in each time slice Δt... i The starting point updates the admission process identifier through atomic operations; and triggers the wake-up mechanism of the controlled process through a named semaphore.

[0061] In this embodiment, S4, during the access permission and runtime constraint phase of the execution phase, at least one of the following three implementation schemes is adopted: The first implementation scheme is as follows: For the direct mapping model accelerator, the memory area corresponding to the accelerator instruction submission buffer is set to read-only. When a non-admitted process attempts to write, a page protection error signal is triggered. In the signal handling function, it is checked whether the current process is an admitted process. If they match, the write permission is temporarily restored and the single-step execution mode is enabled to immediately fall back the permission after completing a single write. Otherwise, the process is suspended and waits for the next admitted flag. The second implementation scheme is as follows: For the centralized management model accelerator, the driver layer intercepts system calls and injects microsecond-level delays into high-frequency task submission instructions. When the call counter reaches the preset trigger factor, the kernel sleep interface is called to force the current process to delay submission. The third implementation scheme: For user-space applications, symbol redirection is performed in the application programming interface call path. First, an admission check is performed to obtain a suggested rate-limiting sleep time. If this time is greater than zero, a microsecond-level sleep operation is executed. Then, notifications are inserted before and after the original task to record the start and end points, and the timing results are pushed to an asynchronous timing thread pool for resource utilization. i Accumulate.

[0062] In this embodiment, S5, during the state feedback and dynamic adjustment phase, includes: real-time monitoring of each scheduling object SO. i The actual physical delay L from task submission trigger to completion i and resource utilization rate r i When the actual usage deviates from the expected time slice Δt i At that time, the weighting parameter W is adjusted according to the proportional coefficient. i If a process makes no requests within its allocated time slice, the time slice is reclaimed, and the corrected weight parameter W is applied. i and time slice parameter Δt i Feedback is fed back to the time slice parameter calculation and scheduling strategy generation stage to achieve closed-loop dynamic evolution.

[0063] In this embodiment, the unified processing of graphics rendering load and computing load is also included: by intercepting and analyzing the general instruction stream of the underlying driver, graphics rendering tasks and computing tasks are incorporated into the same scheduling object set Ω and scheduling policy sequence SP, based on a unified weight parameter W. i Dynamically adjust the proportion of accelerator core usage for various tasks.

[0064] Example 2 This invention provides a unified time-division resource isolation method for heterogeneous accelerators, comprising: I. Overall Technical Workflow This invention revolves around the core objective of "unified time-division scheduling of heterogeneous computing resources." Addressing issues such as uncontrollable resource contention, uncertain execution order, and severe performance fluctuations when multiple processes / tasks concurrently access heterogeneous acceleration devices, it proposes a layered, controllable, and scalable unified time-division technology approach. This achieves deterministic isolation of heterogeneous resources at a micro-time scale. The hierarchical configuration of the system and the logical connections between each layer are shown in Figure 1.

[0065] From a system structure perspective, the overall technical roadmap consists of five sequential stages: Phase S1: Task abstraction and scheduling object modeling phase; Phase S2: Time slice parameter calculation and scheduling strategy generation phase; Phase S3: Scheduling control entity construction phase; Phase S4: Access admission and runtime constraints during execution phase; Phase S5: State feedback and dynamic adjustment phase. (For example...) Figure 2 As shown, the five stages form a complete closed-loop control path: from task abstraction to dynamic adjustment, it realizes adaptive evolution from input to output and ensures deterministic time-division isolation.

[0066] The above stages logically constitute a closed-loop control path, and its overall process can be abstracted into the following sequence: Input: A set of heterogeneous computing tasks T = {t1, t2, …, t n} The set of acceleration devices is D = {d1, d2, …, d m} System operation constraint set C Processing: Scheduling modeling → Time slice allocation → Execution control → Status monitoring → Parameter tuning Output: An accelerated task execution sequence that satisfies fairness, determinism, and controllability. In this invention, all heterogeneous computing tasks are uniformly abstracted as "SchedulingObjects" and managed through a "Unified Time-Slicing Model." This model does not depend on specific hardware implementation details but achieves consistent scheduling semantics across platforms by controlling task submission and device access behaviors.

[0067] II. Phase S1: Task Abstraction and Scheduling Object Modeling (a) Definition of scheduling object In phase S1, the system first abstracts and models the heterogeneous computing tasks from different processes. A scheduling object SO is defined. i For a quintuple: SO i = PID i DID i Q i W i , S i

[0068] in: PID i Indicates the process identifier that initiated the task; DID i Indicates the target heterogeneous device identifier; Q i Indicates the position of the task in the logical scheduling queue; W i The weight parameters represent the task and are used for subsequent time slice calculations. S i This indicates the current execution status of the task.

[0069] The purpose of this abstraction is to transform the task representations that were originally tightly coupled with specific APIs, drivers, or hardware into logical entities that can be processed by the scheduler.

[0070] (II) Input and Output enter: Heterogeneous computing call requests submitted by user processes Process ID (PID) Output: The scheduler object SO has been initialized. i (III) Implementation Method After the system captures the task submission behavior in user mode or kernel mode, it immediately performs the following steps: Step S1.1: Parse the process and target device information of the task; Step S1.2: Assign a unique scheduling object identifier to the task; Step S1.3: Initialize the weight parameter W. i With initial state S i Step S1.4: Place SO i Insert it into the set of scheduling objects Ω.

[0071] like Figure 1 As shown, this invention utilizes a unified scheduling object SO. i The quintuple model abstracts heterogeneous tasks of different processes and types into standardized entities, completely eliminating the differences in underlying hardware and laying the foundation for subsequent cross-platform scheduling.

[0072] III. Stage S2: Time Slice Parameter Calculation and Scheduling Strategy Generation The goal of phase S2 is to generate an executable unified time-division scheduling strategy based on the set of scheduling objects Ω.

[0073] (I) Unified Time Division Model Define the system scheduling cycle as During this period, the acceleration device allows only one scheduled object to obtain execution permission. Each scheduled object SO i A time slice Δt is allocated i It satisfies: Δt i ≥ 0 Σ Δt i ≤ Tcycle In the first implementation, time slice allocation uses a weighted proportional allocation model: Δt i = (W i / Σ j W j ) · Tcycle Where Σ j W j This represents the sum of the weights of all active scheduling objects within the current scheduling period.

[0074] (ii) Scheduling strategy generation The scheduling policy SP is defined as an ordered sequence: SP = [(SO1, Δt1), (SO2, Δt2), …, (SO k , Δt k )] This sequence explicitly defines the order and duration in which each scheduled object gains device access permissions within a scheduling cycle. For example... Figure 6 As shown, the multi-process weighted time slice allocation timing within the unified scheduling period T_cycle can be seen through Δt. i = max(T_min, (W i / ΣW j The T_cycle model enables orderly and fair rotation of execution permissions.

[0075] (III) Input and Output enter: Set of scheduling objects Ω Scheduling cycle parameter Tcycle Output: Scheduling Policy SP IV. Phase S3: Construction of the Scheduling Control Entity (I) Definition and initialization of scheduling control entity The system constructs a cross-process scheduling control base entity to distribute scheduling policy sequences (SPs) as executable admission identifiers in real time. This entity uses a shared memory data structure as its core, which includes admission process identifiers, period start timestamps, and status slot information for each process.

[0076] (ii) Control logic entity operation mechanism The controller process, independent of the business processes, reads the system monotonic clock to determine the start point of the current cycle; according to the scheduling policy sequence SP, in each time slice Δt... i The starting point updates the admission process identifier through atomic operations; and triggers the wake-up mechanism of the controlled process through a named semaphore, thereby completing the transfer from policy to actual execution control.

[0077] (III) Input and Output Input: Scheduling policy sequence SP, system monotonic clock.

[0078] Output: Real-time updated admission flags and controlled process wake-up signals.

[0079] V. Phase S4: Access Control and Runtime Constraints during the Execution Phase (I) Definition of Access Admission and Runtime Constraint Mechanisms Based on the admission identifier issued by the scheduling control base entity, the system imposes physical or logical constraints on task submission behavior to ensure that only the currently admitted process can participate in the task submission within the specified time slice parameter Δt. i Internal access accelerator instruction submission channel.

[0080] (ii) Runtime constraint execution logic Submission requests from non-admissioned processes will be intercepted and blocked, or constrained through methods such as injection delays or memory permission rollback, until the next admission flag arrives, thus achieving strict time-division isolation at the micro-time scale. This phase supports flexible selection of three implementation schemes.

[0081] (III) Input and Output Input: Admission identifier and task submission request issued by the scheduling control base entity.

[0082] Output: Controlled task submission behavior (only admitted processes in Δt) i Submission is completed within the specified timeframe; non-permissioned processes are restricted.

[0083] VI. Stage S5: Status Feedback and Dynamic Adjustment (I) Definition of State Feedback and Dynamic Adjustment Mechanism The system monitors the actual operation during the execution phase in real time and uses the feedback results to dynamically adjust the time slice parameter Δt for the next cycle. i and weight parameter Wi This forms a complete closed-loop adaptive scheduling.

[0084] (II) Logic for Monitoring and Closed-Loop Correction of Operating Indicators Real-time collection of SO data from each scheduling object i The actual physical delay L from submission trigger to completion i and resource utilization rate r i When the actual usage deviates from the expected time slice Δt i At that time, the weighting parameter W is adjusted according to the proportional coefficient. i If the process makes no request within the allocated time slice, the time slice is reclaimed; and the corrected parameters are fed back to stage S2.

[0085] (III) Input and Output Input: Actual physical delay L during the execution phase i Resource utilization rate r i .

[0086] Output: Corrected weight parameters W i and time slice parameter Δt i (Feedback to S2 stage).

[0087] VII. Implementation Plan 1: Unified Time-Division Resource Isolation Based on Kernel-Level ioctl Hooks and Dynamic Page Protection Overall Technical Flow Description: This implementation scheme proposes a strongly isolated time-division scheduling method that combines kernel-level ioctl hooking with dynamic memory page permission control for accelerator architectures where user space can directly map command buffers. By hooking critical system call interfaces of device drivers in kernel space, task channel establishment and instruction submission behaviors are captured; simultaneously, hardware memory protection mechanisms are used to forcibly set the instruction submission channel path to a non-writable state during non-admission time slices, thereby achieving micro-level physical admission control and deterministic time-division isolation.

[0088] Phase S1: Task Abstraction and Scheduling Object Modeling (I) Task Capture and Feature Extraction The system hooks the accelerator driver's system call interface (mainly ioctl) in kernel mode to intercept all control requests related to resource allocation, channel creation, and instruction queue binding, and parses out the key context information of the task in the interception function.

[0089] (ii) Scheduling object SO iThe construction process is as follows: S1.1 Resource Location: Identify specific ioctl requests established or bound to hardware task submission channels (command queues / channels) to determine if a process is requesting a user-space execution channel that can be directly written to. S1.2 Identifier Association: Extract the process identifier that initiated the request and associate it with the corresponding user-space memory mapping region (instruction submission buffer). S1.3 Object Modeling: Construct the scheduling object SO. i = Process ID, Device ID, Queue Position, Weight Parameter, Execution Status The initial state is set to wait for scheduling. S1.4 Set maintenance: The generated scheduling object is inserted into a global scheduling object set maintained based on shared memory, for cross-process access.

[0090] Phase S2: Time Slice Parameter Calculation and Scheduling Strategy Generation (Part 1) A unified time-division model defines the system's global scheduling cycle T_cycle, within which only one scheduling object is allowed to obtain execution permission for the accelerator. Each scheduling object obtains its corresponding execution duration based on its weight parameters.

[0091] (II) Time Slice Calculation and Strategy Sequence Generation S2.1 Time Slice Length Calculation: A weighted proportional allocation model is adopted, and the calculation formula is: Δt i = max(T_min, (W i / Σ W j ) · T_cycle) where Σ W j T_min is the sum of the weights of the currently active scheduling objects, and T_min is the minimum time slice protection threshold.

[0092] S2.2 Scheduling Policy Generation: Based on the weights and priorities (if any) of each scheduled object and the calculated time slice length, generate an ordered scheduling policy sequence SP = [(SO1, Δt1), (SO2, Δt2), …, (SO k , Δt k This sequence explicitly defines the execution order and duration of each process within a scheduling cycle, and is written into the shared control structure for subsequent distribution.

[0093] Phase S3: Construction of Scheduling Control Entity (I) Cross-Process Control Base Creation Based on shared memory control structure, the core of which includes: current admission process identifier, scheduling cycle time base, and admission waiting synchronization mechanism for each process.

[0094] (ii) The independent controller process advances the cycle according to the system's monotonic clock. At the beginning of each time slice, it updates the admission process identifier through atomic operations and notifies the relevant controlled processes through inter-process synchronization primitives.

[0095] Phase S4: Access Admission and Runtime Constraints in the Execution Phase (I) Dual Enforcement Control Mechanism This scheme combines two complementary enforcement methods: 1. Kernel-level ioctl hooking: Intercept ioctl calls during the task channel creation phase, record and associate the memory area of ​​the user-mode instruction buffer, and prepare for subsequent page protection.

[0096] 2. Dynamic page protection: During the scheduling cycle, only the instruction buffer memory pages of the currently admitted process are allowed to have write permissions; the corresponding memory areas of non-admitted processes are set to read-only.

[0097] (II) Runtime Constraint Triggering and Removal Process S4.1 Permission Preset: After the channel is established or during scheduling cycle switching, the instruction submission buffer memory page of the non-admitted process is set to read-only via the kernel interface. S4.2 Exception Capture: When a non-admitted process attempts to write to the instruction buffer, a hardware memory protection exception is triggered, entering the kernel exception handling path. S4.3 Admission Verification: The current admitted process identifier in the shared control structure is read in the exception handling function for matching. S4.4 Temporary Authorization and Single-Time Allow: If a match is found with the current admitted process, the write permission of the memory page is temporarily restored, and the processor's single-step execution mechanism (trap mode) ensures that only the current write operation is completed, after which it is immediately reset to read-only. S4.5 Non-Admitted Processing: If a match is not found, the process is placed in a waiting state, or the write operation is directly rejected until it is scheduled as an admitted process.

[0098] like Figure 3 As shown, the task admission control process based on memory paging permission inducement and single-step debugging, through SIGSEGV exception capture and Trap Flag single-step mechanism, implements physical-level forced admission control in kernel mode to ensure that non-admitted processes cannot write to the instruction buffer.

[0099] Phase S5: Status Feedback and Dynamic Adjustment (I) Runtime Indicator Monitoring records the actual physical delay of each scheduled object from triggering a memory protection exception to completing a single instruction submission, serving as the primary feedback basis.

[0100] (ii) Closed-loop correction If the actual delay continues to deviate from the expected time slice, or if the process is inactive during the allocated time, the weights are adjusted proportionally or the time slice is reclaimed, and the correction parameters are fed back to the scheduling strategy generation stage.

[0101] VIII. Implementation Plan 2: Unified Time-Division Resource Isolation Based on Kernel-Level ioctl Hooks and Task Injection Flow Control (Applicable to heterogeneous accelerators with kernel-mode centralized management models, such as some domestic AI computing platforms) Overall Technical Process Description: This implementation plan addresses the centralized management of heterogeneous accelerators where task submissions must be initiated through kernel-mode ioctl system calls to enter the driver, with the kernel uniformly handling instruction construction and hardware triggering. It proposes a time-division resource isolation method that constructs ioctl hooks and injects delay flow control at the driver interaction layer (kernel mode).

[0102] The main difference from Implementation Plan 1 is: The accelerator in Scheme 1 allows user-mode direct mapping of the command buffer. ioctl is mainly used to initialize channel / resource allocation, and the actual high-frequency instruction submission bypasses the kernel; therefore, Scheme 1 relies on page protection to forcibly block user-mode writes.

[0103] The accelerator in Implementation Scheme 2 adopts a centralized management model. All computing tasks (including high-frequency small tasks) must be submitted through the ioctl system call to enter the kernel, and the final hardware trigger is completed by the driver. Therefore, this scheme directly injects delay or counting rate limiting into the ioctl interception path to achieve precise control over the task submission rhythm without relying on memory page permission switching.

[0104] Phase S1: Task Abstraction and Scheduling Object Modeling (I) Driver Layer Task Capture and Parsing System By loading kernel auxiliary modules, the system dynamically hooks the device file operation interface (mainly ioctl) of the accelerator driver in kernel mode, captures all task submissions, execution triggers and related control requests issued by user mode, and extracts process and task context at the interception point.

[0105] (ii) Scheduling object SO i The construction process is as follows: S1.1 Interception point identification: Focus on filtering ioctl requests that are highly related to the actual computation task issuance and kernel push instructions to hardware. S1.2 Identifier extraction: Obtain the process identifier of the request and associate it with the target device information. S1.3 Object modeling: Construct the scheduling object SO. i = Process ID, Device ID, Queue Position, Weight Parameter, Execution Status The initial state is generally set to wait for scheduling or suspended. S1.4 Set maintenance: The scheduling objects are recorded in the global scheduling configuration table maintained by the kernel (usually based on a hash table or linked list) to complete the logical abstraction.

[0106] Phase S2: Time Slice Parameter Calculation and Scheduling Strategy Generation (Part 1) Kernel-mode Time-Division Control Model defines a unified scheduling cycle (measured in system clock units). Since task submission is synchronous and blocking and involves ioctl each time, the "time slice" in this scheme is mainly transformed into the control of the average submission interval or forced waiting time for each process.

[0107] (II) Flow Control Parameter Calculation and Policy Distribution S2.1 Delay / Interval Calculation: Based on the weight ratio, calculate the average injection delay or submission interval that each scheduled object should endure within a unit period. S2.2 Trigger Granularity Configuration: Set a counting threshold, indicating how many ioctl task submissions are required before a forced delay is applied. S2.3 Parameter Distribution: Write the calculated delay baseline value and trigger counting threshold into the kernel-level process flow control configuration table as a dynamic basis for subsequent interception.

[0108] Phase S3: Construction of Scheduling Control Entity (I) Kernel-mode Configuration and Synchronization Mechanism A dynamic configuration table with process identifier as the key is maintained in the kernel space, recording the cumulative commit count, the most recent active timestamp, the currently allocated delay parameters, and the triggering factor.

[0109] (ii) The auxiliary monitoring entity periodically checks the activity status of processes and cleans up records that have been inactive for a long time through kernel timed tasks or delayed work queues; at the same time, it provides a limited user-space configuration interface to allow dynamic adjustment of weights or flow control intensity.

[0110] Phase S4: Access Admission and Runtime Constraints in the Execution Phase (Part 1) Kernel-mode ioctl flow control mechanism In the hooked ioctl processing path, a counting judgment and delayed injection are performed on each task submission request to achieve micro-control of the submission rhythm.

[0111] (II) Runtime Execution of Sequence Constraint Algorithm S4.1 Count Update: Each time a task-related ioctl request is captured, the cumulative counter of the corresponding process is atomically incremented. S4.2 Flow Control Determination: The flow control configuration of the process is read. If the current count reaches the trigger threshold and there is a non-zero latency requirement, a short-term sleep or delayed scheduling is performed in kernel mode to postpone the actual delivery of this request to the hardware. S4.3 Request Pass-through: After completing the possible delayed operation, the original driver processing function is called to send the instruction to the accelerator normally. S4.4 Statistical Recording: The number of times a delay is applied and the total delay duration are accumulated for subsequent feedback.

[0112] like Figure 4 As shown, the kernel-level ioctl hook interception and task injection delay control logic achieves precise flow control of the high-frequency task submission rhythm in a closed driver environment through a mechanism of counting triggers and microsecond-level sleep, without modifying the driver source code.

[0113] (The core of this process is the rhythm control based on kernel-mode ioctl interception + count triggering + active injection delay, which is in stark contrast to the page protection exception-driven approach of Scheme 1.) Phase S5: Status Feedback and Dynamic Adjustment (I) Activity and Load Monitoring Update the most recent active timestamp of the process at each interception point, and count the total amount of actual latency and effective submission frequency.

[0114] (II) Dynamic Parameter Closed-Loop Correction S5.1 Idle Process Reclamation: Processes that have not requested any tasks for an extended period of time have their weight reduced or removed from the active process list. S5.2 Load Feedback Adjustment: Based on the overall latency accumulation or the actual throughput performance of a single process, the trigger factor or latency baseline value is automatically adjusted to gradually align the scheduling with the actual load characteristics. S5.3 Parameter Write-back: The corrected flow control parameters are rewritten to the kernel configuration table, forming a closed loop with stage S2.

[0115] 9. Implementation Plan 3: Unified Time-Division Resource Isolation Based on User-Space API Interception and Dynamic Rate Limiting (Applicable to heterogeneous accelerators where the runtime library provides interceptable interfaces and supports task completion notifications) Overall Technical Process Description: This implementation plan proposes a lightweight time-division resource management method that is fully implemented in user space for heterogeneous accelerators that expose many interceptable functions in their runtime library and provide task start / end notification mechanisms.

[0116] The core principle is user-space API interception: by using dynamic link library preloading or symbol redirection techniques, key interfaces in the accelerator runtime library, such as task creation, queue management, and kernel function calls, are hijacked, and control logic is inserted at the very beginning of the task submission path.

[0117] The main difference from the first two implementation schemes is: Both Option 1 and Option 2 implement forced or semi-forced control (page protection exception or delayed injection) in kernel space through ioctl hooks, which are suitable for scenarios that require strong isolation or driver-level control.

[0118] This solution runs entirely in user space, without modifying the kernel or depending on the openness of the driver source code. It achieves soft rhythm control and dynamic rate limiting through active sleep, asynchronous real-time statistics, and occupancy backpressure. It is suitable for application environments that are sensitive to performance overhead and allow for a certain degree of cooperation.

[0119] Phase S1: Task Abstraction and Scheduling Object Modeling (I) User-mode API Interception and Redirection The system uses dynamic link library preloading mechanism or symbol substitution technology to intercept key functions (queue creation, task launch, execution trigger, etc.) of the accelerator runtime library, and captures task intent and context at the front end of the call chain.

[0120] (ii) Scheduling object SO iThe construction process is as follows: S1.1 Queue / Task Wrapping: Upon intercepting queue creation or task submission related calls, a logical wrapper object is constructed, recording the original handle and attaching monitoring attributes (such as resource consumption counters). S1.2 Identifier Association: The current process identifier is extracted, and the resource consumption statistics for that object are initialized. S1.3 Object Modeling: The scheduling object SO is constructed. i = Process ID, Device ID, Queue Position, Weight Parameter, Execution Status The weight parameter is usually preset through configuration files or environment, representing the expected maximum resource usage share. S1.4 Set maintenance: The scheduling object is recorded in the global task management structure (in-process list or hash table) maintained in user space.

[0121] Phase S2: Time Slice Parameter Calculation and Scheduling Strategy Generation (I) Control Model Based on Resource Utilization This scheme does not adopt a strict fixed time slice division, but is based on a "core-time" cumulative utilization model. Define the theoretical total capacity within a time window (number of accelerator cores × time length) to evaluate the real-time consumption ratio of each scheduling object.

[0122] (II) Rate Limiting Parameter Generation and Policy Update S2.1 Utilization Calculation: Based on the actual execution time obtained from asynchronous statistics, calculate the resource utilization rate (actual consumption / theoretical capacity) of each scheduled object in real time. S2.2 Over-limit Judgment and Rate Limiting Step Size: When the utilization rate exceeds the preset upper limit, calculate the required backpressure sleep duration (positively correlated with the over-limit ratio and window length). S2.3 Parameter Smoothing: Introduce an exponential decay factor and a periodic reset mechanism to avoid long-term accumulation of historical data that makes it difficult for new tasks to obtain resources.

[0123] Phase S3: Construction of the scheduling and control entity (I) The asynchronous timing and statistics entity starts an independent background thread (or thread pool / event loop) to be responsible for receiving task completion notifications, calculating the actual hardware execution time, and accumulating resource usage points (number of cores × milliseconds) to avoid blocking the main computing path.

[0124] (ii) The global rate limiting decision unit constructs a singleton resource throttling controller, which is responsible for maintaining the total usage statistics of the current window, providing a query interface for the suggested sleep duration of each task, and periodically performing decay or reset operations.

[0125] Phase S4: Access Admission and Runtime Constraints in the Execution Phase (Part 1) Soft Admission Check Before Task Submission In the core task emission function that is intercepted, the current resource status is queried first, rather than directly calling the original implementation.

[0126] (II) Runtime Execution Constraint Algorithm S4.1 Rate Limiting Decision: Call the global controller interface to obtain the currently suggested sleep duration. If it is greater than zero, execute a microsecond-level sleep, actively postponing the actual submission of this task. S4.2 Notifier Injection: Insert completion notification / event objects before and after the original task to accurately capture the start and end times on the hardware side. S4.3 Task Submission: Call the original runtime function to complete the task submission. S4.4 Asynchronous Reporting: Push the task context containing the notifier to the background timing thread. After the task is completed, calculate the current consumption and update the global usage statistics.

[0127] like Figure 5 As shown, the resource monitoring architecture based on user-space API hijacking and asynchronous timer thread pool achieves lightweight soft rate limiting and real execution time feedback through symbol redirection, notifier injection, and background occupancy accumulation, making it suitable for performance-sensitive scenarios.

[0128] (The core of this process is a soft control mechanism that combines user-space API interception, pre-submission sleep, asynchronous real-time feedback, and occupancy backpressure.) Phase S5: Status Feedback and Dynamic Adjustment (I) Continuous Monitoring of Resource Usage The background thread periodically performs exponential decay (simulating a sliding window) on the accumulated usage value to ensure that the statistics mainly reflect recent behavior; it supports periodic complete reset to avoid long-term drift.

[0129] (II) Closed-Loop Feedback and Parameter Adaptation S5.1 Dynamic Threshold Adjustment: The maximum allowable occupancy rate of each scheduled object can be modified in real time through external configuration interfaces or internal strategies. S5.2 Backpressure Intensity Adjustment: If the occupancy rate of a process remains consistently high, the baseline coefficient of its next sleep step is automatically increased, forming negative feedback. S5.3 Statistical Transmission: Optionally, real-time occupancy rate, rate limiting frequency, and other indicators can be output through files, shared memory, or network interfaces for observation and intervention by the upper-level management system. Figure 7 As shown, the unified time-division resource isolation system for heterogeneous accelerators provided by this invention consists of five core units: "task abstraction and scheduling object modeling unit," "time slice parameter calculation and scheduling strategy generation unit," "scheduling control entity construction unit," "access admission and runtime constraint unit during execution phase," and "state feedback and dynamic adjustment unit." Each unit operates through a unified scheduling object SO. i Working in conjunction with the cross-process scheduling control base entity, a complete closed-loop adaptive scheduling architecture is formed, thereby achieving deterministic time-division resource isolation and fair sharing of heterogeneous accelerators under multi-task concurrency.

[0130] Example 3 like Figure 7As shown, the present invention provides a unified time-division resource isolation system for heterogeneous accelerators, comprising: Task Abstraction and Scheduling Object Modeling Unit: This unit standardizes and abstracts heterogeneous computing tasks from different processes, transforming tasks of different origins and types into scheduling objects (SOs) that include process identifiers, device identifiers, queue positions, weight parameters, and execution states. i This results in a standardized set of scheduling objects Ω that eliminates differences in the underlying hardware. Time slice parameter calculation and scheduling strategy generation unit: Based on the set of scheduling objects Ω, it defines the system scheduling period T_cycle and generates scheduling strategies according to each scheduling object SO. i Weight parameter W i Calculate the time slice length Δt i Generate a scheduling policy sequence SP containing the execution order and duration, thereby obtaining an ordered execution permission allocation scheme within a scheduling cycle; Scheduling control entity construction unit: Based on the scheduling policy sequence SP, a cross-process scheduling control base entity is constructed for real-time issuance of admission identifiers; The access admission and runtime constraint unit during the execution phase: Based on the scheduling control base entity, it implements physical or logical admission control and runtime constraints on task submission behavior, thereby ensuring that only admitted processes are allowed within the specified time slice parameter Δt. i Internal access accelerator; Status feedback and dynamic adjustment unit: Real-time monitoring of actual execution time and resource utilization during the execution phase. i The feedback results are then used to dynamically adjust the time slice parameter Δt for the next cycle. i and weight parameter W i This forms a closed-loop adaptive scheduling, thereby achieving deterministic time-division resource isolation and fair sharing of heterogeneous accelerators under multi-task concurrency.

[0131] In this embodiment, the task abstraction and scheduling object modeling unit specifically includes: capturing heterogeneous computing call requests submitted by user processes; parsing the process identifier and target device identifier of the task; and constructing a scheduling object SO for each task, containing the process identifier, device identifier, queue position, weight parameters, and execution status. i ; and will schedule all SO objects i Insert it into the standardized scheduling object set Ω.

[0132] Example 4 The present invention provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of a unified time-division resource isolation method for heterogeneous accelerators.

[0133] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0134] This application is described with reference to flowchart illustrations and / or block diagrams of methods, systems, and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A system that specifies functions in one or more boxes.

[0135] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0136] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0137] The inventive points protected by this invention are as follows: This invention proposes a unified time-division resource isolation method and system for heterogeneous accelerators, the core innovations of which are summarized as follows: 1. Kernel-mode physical admission mechanism based on memory page permission control: For direct-mapped accelerators, by dynamically manipulating the "read-only / read-write" attribute of the memory page where the instruction queue is located, and in conjunction with the single-step debugging technique of the signal handling function (SignalHandler) and the processor trap flag (Trap Flag), the kernel layer forcibly captures and arbitrates the hardware task issuance behavior, thereby achieving forced time-division isolation that does not rely on hardware preemption.

[0138] 2. Task injection flow control technology based on ioctl hooks: For driver-centralized management accelerators, an ioctl system call interception hook is built at the driver kernel layer. By real-time parsing of task submission codes (such as SBTS_INVOKE_TASK_NR), microsecond-level delays are precisely injected into the internal execution path of the driver, enabling non-intrusive scheduling of high-frequency computing, graphics, and proprietary operator requests in a closed driver environment.

[0139] 3. Three-layer parallel optional heterogeneous isolation configuration: A collaborative architecture consisting of a kernel layer, a driver interaction layer, and an application library layer is constructed. The system dynamically selects a single optimal path or a combination of paths from the above three layers for task interception based on the driver openness and memory mapping characteristics of the target hardware, rather than simply superimposing multiple layers of logic, thus ensuring scheduling consistency and low overhead.

[0140] 4. Hardware-Independent Unified Resource Scheduling Semantic Layer: A method is proposed to abstract heterogeneous control events (page faults, ioctl calls, API symbolic calls) intercepted at different levels into a standardized scheduling object $SO_i$. This model shields the differences in underlying physical interception methods and provides a standardized time slice ($\Delta t_i$) allocation interface for upper-layer strategies, achieving deep decoupling between scheduling logic and execution control.

[0141] 5. Unified Time-Division Sharing Mechanism Across Load Types: By intercepting and analyzing the common instruction streams of the underlying drivers, computational loads and graphics rendering loads are incorporated into the same scheduling sequence. Based on a unified weight $W_i$, the utilization ratio of various tasks on the accelerator cores is dynamically adjusted, solving the technical pain point of fragmented resource management in heterogeneous devices under mixed load scenarios.

[0142] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. It will be apparent to those skilled in the art that the invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered illustrative and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the scope of the invention. No reference numerals in the claims should be construed as limiting the scope of the claims.

[0143] Furthermore, it should be understood that although this specification describes embodiments, not every embodiment contains only one independent technical solution. This narrative style is merely for clarity. Those skilled in the art should consider the specification as a whole, and the technical solutions in each embodiment can be appropriately combined to form other embodiments that can be understood by those skilled in the art. The above content is only for illustrating the technical concept of the present invention and should not be construed as limiting the scope of protection of the present invention. Any modifications made based on the technical concept proposed in this invention shall fall within the scope of protection of the claims of this invention.

Claims

1. A unified time-division resource isolation method for heterogeneous accelerators, characterized in that, include: S1. In the task abstraction and scheduling object modeling stage: heterogeneous computing tasks from different processes are standardized and abstracted, and tasks of different sources and types are uniformly converted into scheduling objects SO containing process identifiers, device identifiers, queue positions, weight parameters, and execution states. i This results in a standardized set of scheduling objects Ω that eliminates differences in the underlying hardware. S2. In the time slice parameter calculation and scheduling strategy generation stage: Based on the set of scheduling objects Ω, define the system scheduling period T_cycle, and according to each scheduling object SO i Weight parameter W i Calculate the time slice length Δt i Generate a scheduling policy sequence SP containing the execution order and duration, thereby obtaining an ordered execution permission allocation scheme within a scheduling cycle; S3. In the scheduling control entity construction phase: Based on the scheduling policy sequence SP, a cross-process scheduling control base entity is constructed for real-time issuance of admission identifiers; S4. During the access admission and runtime constraint phase of the execution phase: Based on the scheduling control base entity, physical or logical access control and runtime constraints are implemented on task submission behavior to ensure that only admitted processes are allowed within the specified time slice parameter Δt. i Internal access accelerator; S5. During the status feedback and dynamic adjustment phase: Real-time monitoring of the actual execution time and resource utilization during the execution phase. i The feedback results are then used to dynamically adjust the time slice parameter Δt for the next cycle. i and weight parameter W i This forms a closed-loop adaptive scheduling, thereby achieving deterministic time-division resource isolation and fair sharing of heterogeneous accelerators under multi-task concurrency.

2. The unified time-division resource isolation method for heterogeneous accelerators according to claim 1, characterized in that, S1. In the task abstraction and scheduling object modeling phase, this includes: capturing heterogeneous computing call requests submitted by user processes; parsing the process identifier and target device identifier of the task; and constructing a scheduling object SO for each task, containing the process identifier, device identifier, queue position, weight parameters, and execution status. i ; and will schedule all SO objects i Insert it into the standardized scheduling object set Ω.

3. The unified time-division resource isolation method for heterogeneous accelerators according to claim 1, characterized in that, S2. In the time slice parameter calculation and scheduling strategy generation stage, the following steps are included: defining the system global scheduling period T_cycle; within the scheduling period T_cycle, according to each scheduling object SO... i Weight parameter W i The time slice length Δt for each scheduled object is calculated using a weighted proportional model. i , where Δt i = max(T_min, (W i / ΣW j )·T_cycle); and generate an ordered scheduling policy sequence SP = [(SO1, Δt1), (SO2, Δt2), …, (SO k , Δt k )).

4. The unified time-division resource isolation method for heterogeneous accelerators according to claim 1, characterized in that, S3. In the scheduling control entity construction phase, the following steps are included: constructing a shared memory data structure as the carrier for issuing scheduling instructions. This shared memory data structure contains the admission process identifier, the cycle start timestamp, and the status slot information of each process; the independent controller process reads the system monotonic clock to determine the start point of the current cycle; and according to the scheduling policy sequence SP, in each time slice Δt... i The starting point updates the admission process identifier through atomic operations; and triggers the wake-up mechanism of the controlled process through a named semaphore.

5. A unified time-division resource isolation method for heterogeneous accelerators according to claim 1, characterized in that, S4. During the access control and runtime constraint phase of the execution phase, at least one of the following three implementation schemes shall be adopted: The first implementation scheme is as follows: For the direct mapping model accelerator, the memory area corresponding to the accelerator instruction submission buffer is set to read-only. When a non-admitted process attempts to write, a page protection error signal is triggered. In the signal handling function, it is checked whether the current process is an admitted process. If they match, the write permission is temporarily restored and the single-step execution mode is enabled to immediately fall back the permission after completing a single write. Otherwise, the process is suspended and waits for the next admitted flag. The second implementation scheme is as follows: For the centralized management model accelerator, the driver layer intercepts system calls and injects microsecond-level delays into high-frequency task submission instructions. When the call counter reaches the preset trigger factor, the kernel sleep interface is called to force the current process to delay submission. The third implementation scheme: For user-space applications, symbol redirection is performed in the application programming interface call path. First, an admission check is performed to obtain a suggested rate-limiting sleep time. If this time is greater than zero, a microsecond-level sleep operation is executed. Subsequently, notifications are inserted before and after the original task to record the start and end points, and the timing results are pushed to an asynchronous timing thread pool for resource utilization. i Accumulate.

6. The unified time-division resource isolation method for heterogeneous accelerators according to claim 1, characterized in that, S5. During the status feedback and dynamic adjustment phase, this includes: real-time monitoring of the SO of each scheduling object. i The actual physical delay L from task submission trigger to completion i and resource utilization rate r i When the actual usage deviates from the expected time slice Δt i At that time, the weighting parameter W is adjusted according to the proportional coefficient. i If a process makes no requests within its allocated time slice, the time slice is reclaimed, and the corrected weight parameter W is applied. i and time slice parameter Δt i Feedback is fed back to the time slice parameter calculation and scheduling strategy generation stage to achieve closed-loop dynamic evolution.

7. A unified time-division resource isolation method for heterogeneous accelerators according to any one of claims 1 to 6, characterized in that, It also includes unified handling of graphics rendering and computational loads: by intercepting and analyzing the general instruction flow of the underlying drivers, graphics rendering tasks and computational tasks are incorporated into the same scheduling object set Ω and scheduling policy sequence SP, based on a unified weight parameter W. i Dynamically adjust the proportion of accelerator core usage for various tasks.

8. A unified time-division resource isolation system for heterogeneous accelerators, characterized in that, include: Task Abstraction and Scheduling Object Modeling Unit: This unit standardizes and abstracts heterogeneous computing tasks from different processes, transforming tasks of different origins and types into scheduling objects (SOs) that include process identifiers, device identifiers, queue positions, weight parameters, and execution states. i This results in a standardized set of scheduling objects Ω that eliminates differences in the underlying hardware. Time slice parameter calculation and scheduling strategy generation unit: Based on the set of scheduling objects Ω, it defines the system scheduling period T_cycle and generates scheduling strategies according to each scheduling object SO. i Weight parameter W i Calculate the time slice length Δt i Generate a scheduling policy sequence SP containing the execution order and duration, thereby obtaining an ordered execution permission allocation scheme within a scheduling cycle; Scheduling control entity construction unit: Based on the scheduling policy sequence SP, a cross-process scheduling control base entity is constructed for real-time issuance of admission identifiers; The access admission and runtime constraint unit during the execution phase: Based on the scheduling control base entity, it implements physical or logical admission control and runtime constraints on task submission behavior, thereby ensuring that only admitted processes are allowed within the specified time slice parameter Δt. i Internal access accelerator; Status feedback and dynamic adjustment unit: Real-time monitoring of actual execution time and resource utilization during the execution phase. i The feedback results are then used to dynamically adjust the time slice parameter Δt for the next cycle. i and weight parameter W i This forms a closed-loop adaptive scheduling, thereby achieving deterministic time-division resource isolation and fair sharing of heterogeneous accelerators under multi-task concurrency.

9. A unified time-division resource isolation system for heterogeneous accelerators according to claim 8, characterized in that, The task abstraction and scheduling object modeling unit specifically includes: capturing heterogeneous computing call requests submitted by user processes; parsing the process identifier and target device identifier of the task; and constructing a scheduling object SO for each task, containing the process identifier, device identifier, queue position, weight parameters, and execution status. i ; and will schedule all SO objects i Insert it into the standardized scheduling object set Ω.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of a unified time-division resource isolation method for heterogeneous accelerators according to any one of claims 1-7.

Images