Computing node resource perception and task execution method of heterogeneous computing power scheduling system

Abstract

The application provides a computing node resource sensing and task execution method of a heterogeneous computing power scheduling system, relates to the technical field of heterogeneous hardware resource management, and obtains the transmission throughput and congestion state of a network protocol stack in a computing node, and the resource occupation information of a heterogeneous computing device in the computing node and the interconnection bandwidth between devices; constructs resource state information of the computing node according to the transmission throughput, the congestion state, the resource occupation information and the interconnection bandwidth; performs atomic pre-occupation on target computing resources in a candidate task set based on the resource state information; when a computing core function of a heterogeneous computing device processes a large model training task, a target reasoning task is scheduled to the heterogeneous computing device of the corresponding computing node to run; after the target reasoning task is completed and the occupied resources are released, the execution of the large model training task is resumed. Instruction-level safe preemption and automatic merging and recovery of resources are realized.

Description

Technical Field

[0001] This application relates to the field of heterogeneous hardware resource management technology, and in particular to a method for computing node resource perception and task execution in a heterogeneous computing power scheduling system. Background Technology

[0002] With the development of artificial intelligence technology, computing tasks such as large model training and online inference are increasingly reliant on heterogeneous computing power clusters. In computing service scenarios, in order to improve the resource utilization of the underlying cluster, long-running large model training tasks and short-response online inference tasks may be deployed on the same computing node, which places demands on the computing node's resource awareness, task switching, and resource management capabilities.

[0003] Currently, some computing power scheduling schemes mainly rely on data collected periodically by the upper-level scheduling center to evaluate node status and allocate new computing tasks to the underlying hardware based on the evaluation results. When resource conflicts occur between mixed tasks of different priorities, the underlying system may use methods such as terminating the currently running process, pausing some tasks, or putting newly arrived tasks into a queuing state to complete task replacement.

[0004] However, when there is a time lag between the node status obtained by the upper-level scheduling center and the real-time status of the computing node, sudden task concurrency may increase the risk of redundant resource allocation or node memory overflow. When using process-level termination or long-term queuing to handle task conflicts, it may increase the cost of training task recovery or online inference response time. Furthermore, the storage space released after frequent task interruptions and iterations may be discretely distributed, affecting the reloading of subsequent larger tasks. Therefore, existing technologies face the technical challenge of balancing node stability, task switching overhead, and resource fragmentation efficiency. Summary of the Invention

[0005] The purpose of this application is to provide a method, device and storage medium for computing node resource perception and task execution in a heterogeneous computing power scheduling system, so as to solve the technical problems in the prior art that it is difficult to balance node stability, task switching loss and resource fragmentation efficiency.

[0006] In a first aspect, this application provides a method for computing node resource awareness and task execution in a heterogeneous computing power scheduling system, including: Obtain the transmission throughput and congestion status of the network protocol stack in the computing node, as well as the resource usage information of heterogeneous computing devices within the computing node and the interconnection bandwidth between devices; Resource status information of computing nodes is constructed based on transmission throughput, congestion status, resource occupancy information, and interconnection bandwidth; Based on resource status information, the target inference task and the target computing resources required by the target inference task are determined from the candidate task set. When the remaining computing resources of the computing node at the current time meet the requirements of the target computing resources, the target computing resources are atomically pre-occupied. When processing the computation kernel function of a large model training task on a heterogeneous computing device, the execution boundary of the computation kernel function is monitored. When the execution boundary is reached, the large model training task is suspended, and the target inference task is scheduled to run on the heterogeneous computing device of the corresponding computing node. After the target inference task is completed and the occupied resources are released, the idle resources in the computing node are scanned, merged, and the resource status information is updated in order to resume the execution of the large model training task.

[0007] Optionally, resource status information of the computing node is constructed based on transmission throughput, congestion status, resource occupancy information, and interconnection bandwidth, including: The product of transmission throughput and congestion status is used as the communication load; The difference between the total resource capacity and resource occupancy information of the computing node is taken as the remaining computing resources; Based on communication load, remaining computing resources, and interconnect bandwidth, resource status information of computing nodes is constructed.

[0008] Optionally, based on resource status information, the target inference task and the target computing resources required for the target inference task are determined from the candidate task set, including: Based on the task list of the candidate task set, determine multiple candidate tasks and the task requirements corresponding to each candidate task; Each candidate task whose required computing resources are less than or equal to the remaining computing resources in the resource status information is considered as a matching task. The target inference task is determined from multiple matching tasks based on a preset scheduling priority, and the target computing resources required by the target inference task are determined based on the configuration identifier of the target inference task.

[0009] Optionally, when the remaining computing resources of the computing node at the current moment meet the demand for the target computing resources, atomic pre-allocation of the target computing resources is performed, including: Calculate the difference between the remaining computing resources of the computing node at the current moment and the target computing resources; If the resource difference is greater than or equal to zero, it is determined that the remaining computing resources meet the requirements of the target computing resources and a lock status label is generated. The lock status label is written into the status register or management address space corresponding to the target computing resources, and the target computing resources are marked as pre-occupied. The pre-empted state is used to intercept call requests from other concurrent tasks to complete the atomic pre-emption of the target computing resources.

[0010] Optionally, when processing the computation kernel function of a large model training task on heterogeneous computing devices, the execution boundary of the computation kernel function is monitored. When the execution boundary is reached, the large model training task is suspended, and the target inference task is scheduled to run on the corresponding heterogeneous computing device, including: The execution queue of the large model training task determines the computation kernel function to be processed, and the machine instructions of the computation kernel function are parsed based on the preset instruction mapping rules to obtain the instruction execution sequence. Identify execution boundaries, including operation cycle end markers, from the instruction execution sequence, and generate interrupt suspension signals when multiple lines of low-level execution code reach the execution boundary on heterogeneous computing devices; Based on the interrupt suspension signal, the context state of the large model training task is saved to the local storage space to suspend the large model training task. After suspending the large model training task, the target inference task is scheduled to run on the heterogeneous computing device of the corresponding computing node.

[0011] Optionally, after the target inference task completes and releases the occupied resources, the idle resources within the computing node are scanned, merged, and the resource status information is updated, including: After the target inference task is completed and the occupied resources are released, multiple free resources within the computing node are retrieved, and based on the preset memory mapping table, the logical address range of the released occupied resources and the logical address range of each free resource are calculated. The logical address range of the released occupied resources is compared with the logical address range of each free resource by boundary value to determine the target free resource. When the logical address range of the target free resource is adjacent to the logical address range of the released occupied resources, the released occupied resources and the target free resources are combined into a continuous computing power block. Based on the computing power blocks, the resource status information is updated to obtain the updated resource status information.

[0012] Optionally, after updating the resource status information based on the computing power block to obtain the updated resource status information, the method further includes: Based on the updated resource status information, multiple discrete idle resources within the computing node are identified. The capacity of each discrete idle resource is summed to obtain the discrete total capacity, and the total node capacity of the computing node is determined. The ratio of the total discrete capacity to the total node capacity is used as the fragmentation index. When the fragmentation index is greater than the preset fragmentation threshold, a memory migration instruction is generated. Memory migration instructions are used to move multiple occupied resources within a computing node to a contiguous physical address range, so as to combine all discrete idle resources into a block of idle computing power with contiguous physical addresses. The idle computing power blocks are recorded in the updated resource status information to obtain the second updated resource status information, and the execution of the large model training task is resumed using the second updated resource status information.

[0013] Secondly, this application provides a computing node resource awareness and task execution system for a heterogeneous computing power scheduling system, comprising: The acquisition module is used to acquire the transmission throughput and congestion status of the network protocol stack in the computing node, as well as the resource usage information of heterogeneous computing devices within the computing node and the interconnection bandwidth between devices. The building module is used to construct the resource status information of computing nodes based on transmission throughput, congestion status, resource occupancy information, and interconnection bandwidth. The determination module is used to determine the target inference task and the target computing resources required by the target inference task from the candidate task set based on resource status information. When the remaining computing resources of the computing node at the current time meet the requirements of the target computing resources, the target computing resources are atomically pre-occupied. The monitoring module is used to monitor the execution boundary of the computation kernel function when the large model training task is processed on heterogeneous computing devices. When the execution boundary is reached, the large model training task is suspended and the target inference task is scheduled to run on the heterogeneous computing device of the corresponding computing node. The merging module is used to scan the idle resources within the computing node and merge them after the target inference task has finished running and released the occupied resources, and then update the resource status information so as to resume the execution of the large model training task.

[0014] Thirdly, this application provides an electronic device, comprising: Memory, used to store computer programs; The processor is used to implement the steps of the computing node resource awareness and task execution method of the heterogeneous computing power scheduling system as described in the first aspect above when executing computer programs.

[0015] Fourthly, this application provides a computer-readable storage medium storing a computer program, which, when executed by a processor, can implement the steps of the computing node resource awareness and task execution method of the heterogeneous computing power scheduling system described in the first aspect above.

[0016] This application provides a method for resource awareness and task execution in a heterogeneous computing power scheduling system. By acquiring the transmission throughput and congestion status of the network protocol stack in the computing node, as well as the resource occupancy information of heterogeneous computing devices within the computing node and the interconnection bandwidth between devices, it overcomes the limitations of existing technologies that overly rely on lagging data from the upper-level scheduling center. It eliminates the blind spots of single-dimensional status evaluation, avoids the risk of node memory overflow and crashes caused by disordered resource contention during periods of high concurrency of sudden tasks, and establishes a technical barrier to ensure the absolute stability of computing nodes. It abandons the outdated strategies of crudely killing running processes or forcing new tasks into long queues, achieving extremely low-loss switching between mixed tasks. It ensures that large model training tasks can obtain continuous and complete resource support for lossless resumption of execution.

[0017] Furthermore, this application parses machine instructions using preset instruction mapping rules to obtain the instruction execution sequence; then, it accurately identifies the execution boundary containing the end-of-operation cycle marker from the instruction execution sequence, and generates an interrupt suspension signal when multiple lines of low-level execution code on a heterogeneous computing device reach this execution boundary; finally, based on the interrupt suspension signal, the context state of the large model training task is securely saved to local storage to achieve suspension, and after suspending the large model training task, the target inference task is scheduled to run on the heterogeneous computing device of the corresponding computing node. This achieves a perfect balance between absolute preservation of training assets and agile response to inference requests. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 A flowchart illustrating a method for computing node resource awareness and task execution in a heterogeneous computing power scheduling system provided in this application embodiment; Figure 2 This is a schematic diagram of a heterogeneous system provided in an embodiment of this application; Figure 3 This is a schematic diagram of computing power block merging and physical address relocation provided in an embodiment of this application; Figure 4 This application provides a schematic diagram of the structure of a computing node resource awareness and task execution system for a heterogeneous computing power scheduling system. Figure 5 This is a schematic diagram of the hardware structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0020] In the context of large-scale model training and inference hybrid deployment scenarios, existing centralized scheduling is prone to memory overflow and crashes due to view lag, training losses and inference delays caused by abrupt process replacement, and the fragmentation of computing power leads to a sharp decline in carrying capacity.

[0021] This application constructs accurate local resource status information by real-time fusion of network protocol stack transmission throughput, congestion status, resource occupancy information of heterogeneous computing devices, and interconnection bandwidth between devices. Based on this real-time status, a local defense mechanism is established to atomically pre-allocate target computing resources only when the remaining computing resources truly meet the demand, fundamentally eliminating the risk of disorderly resource contention and node crashes under sudden concurrency. It breaks the black-box limitation of underlying hardware scheduling by probing and monitoring the execution boundary of the computation kernel function of large model training tasks, replacing the forced killing of process-level iterations with lossless flexible suspension, and achieving extremely low-loss smooth switching between long-term training and short-term online inference tasks. Finally, after the inference task is completed and the occupied resources are released, the application actively scans and merges idle resources within the node to completely eliminate unusable computing power fragments, thereby perfectly achieving efficient scheduling of mixed tasks and continuous reorganization and recovery of the overall system resource carrying capacity while ensuring the absolute stability of computing nodes.

[0022] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0023] The core of this application is to provide a method for computing node resource awareness and task execution in a heterogeneous computing power scheduling system. A flowchart of one specific implementation is shown below. Figure 1 As shown, the method includes: Step 101: Obtain the transmission throughput and congestion status of the network protocol stack in the computing node, as well as the resource usage information of heterogeneous computing devices within the computing node and the interconnection bandwidth between devices.

[0024] A computing node refers to the basic computing entity unit in a heterogeneous computing power scheduling architecture used to carry training and inference tasks. The resources in a computing node that participate in the scheduling decision in this application include core computing resources, device-side storage resources, host-side storage resources, and inter-device interconnect bandwidth resources.

[0025] Among them, computing core resources are used to characterize the number of allocatable computing units in GPUs, NPUs or other heterogeneous computing devices; device-side storage resources are used to characterize the video memory or on-chip addressable storage space of heterogeneous computing devices; host-side storage resources are used to characterize the main memory space in the host machine of the computing node that can be used for task context temporary storage and data paging; and device interconnect bandwidth resources are used to characterize the data transmission capability between heterogeneous computing devices and hosts or other heterogeneous computing devices.

[0026] Heterogeneous computing devices refer to underlying hardware chips or accelerator cards that contain different computing architectures. A network protocol stack refers to the software layer structure that handles the sending and receiving of network data packets. For example... Figure 2 As shown, a compute node contains a network protocol stack and heterogeneous computing devices, which may include graphics processors, neural network processors, or other acceleration hardware. The transmission throughput of the compute node's internal network protocol stack is obtained through the kernel network interface.

[0027] Transmission throughput refers to the amount of data successfully transmitted per unit of time. Congestion status is obtained by reading the register status of the network interface controller; this status can be quantified using at least one metric: packet loss rate, retransmission rate, or queue backlog length. Resource usage information of the heterogeneous computing devices is obtained by calling the underlying device driver interface.

[0028] Resource occupancy information consists of occupancy vectors corresponding to computing core resources, device-side storage resources, and host-side storage resources, respectively. For example, resource occupancy information may include the number of occupied computing cores, the occupied video memory capacity, and the occupied main memory capacity. The bus management interface is invoked to obtain the interconnect bandwidth between devices; interconnect bandwidth refers to the actual communication rate of the high-speed interconnect bus. By obtaining the occupancy status of different types of resources and the interconnect bandwidth between devices, subsequent resource status information can be calculated in a way that matches similar resources with similar resources.

[0029] Step 102: Construct the resource status information of the computing node based on the transmission throughput, congestion status, resource occupancy information and interconnection bandwidth.

[0030] The acquired multi-dimensional heterogeneous sensing data is structured and constructed. Construction refers to the process of combining and mapping scattered data according to a pre-defined data structure. The pre-defined data structure refers to a tensor matrix form pre-written in the code to store multi-dimensional variables. Local network throughput or partial computing power device occupancy cannot comprehensively and objectively reflect the overall true carrying capacity and status of computing nodes. Network layer data and underlying physical computing device layer data are extracted and sequentially filled into the pre-defined data structure according to a pre-defined dimensional order, thus integrating and constructing a structured resource status information.

[0031] Resource status information refers to a data set reflecting the current network status of computing nodes, the availability of computing cores, the availability of device-side storage, the availability of host-side storage, and the status of interconnect bandwidth between devices. Resource status information can be represented using multi-dimensional vectors or matrices, where different dimensions correspond to communication load, remaining computing core resources, remaining device-side storage resources, remaining host-side storage resources, and interconnect bandwidth, respectively. Different dimensions are not directly subtracted or merged; comparisons and calculations are only performed within the same resource type. This specific mapping and assignment process provides a quantifiable and unified data decision-making benchmark for subsequent high-precision scheduling and allocation, and localized resource defense.

[0032] Step 201: Use the product of transmission throughput and congestion status as the communication load.

[0033] Communication load refers to a comprehensive indicator used to measure the current network communication pressure and congestion risk of computing nodes. The preset network weight coefficient is a multiplier factor pre-configured based on the packet loss rate to amplify the impact of congestion. The current real-time packet loss rate of the network links is obtained, and a digital assessment value corresponding to the congestion state is calculated based on a preset exponential penalty mapping function. The specific mapping function is as follows: ,in Represents a digital assessment value. Represents the real-time packet loss rate. The preset penalty sensitivity constant is set to 10 in this embodiment. The calculated digital evaluation value is used as the preset network weight coefficient. The communication load is calculated by directly multiplying this preset network weight coefficient with the real-time value of the transmission throughput. The specific calculation formula is as follows: ,in Represents communication load. Represents the preset network weight coefficients. This represents transmission throughput. The product operation amplifies the calculated communication load value as network congestion worsens, thus providing a more sensitive and accurate reflection of hidden bottlenecks and pressures on the network communication link.

[0034] Step 202: Use the difference between the total resource capacity and resource occupancy information of the computing node as the remaining computing resources.

[0035] A compute node has a defined total resource capacity during initial configuration. This total resource capacity is a total capacity vector consistent with the resource usage information dimension, and can include the total number of allocatable compute cores within the compute node, the total device-side storage capacity, and the total host-side storage capacity.

[0036] Remaining computing resources refer to the vector of available resources on a computing node that can currently be used for the recovery of the target inference and training tasks. The total resource capacity vector is obtained by reading the node configuration information table. The remaining amount of each resource type is then obtained by subtracting the occupied amount of the same resource type from the resource occupancy information in the total resource capacity vector.

[0037] Specifically, the remaining computing resources are obtained by subtracting the computing core usage from the device-side storage usage, and the host-side storage usage from the host-side storage usage. The specific calculation formula is as follows: ,in Represents remaining computing resources. Represents total resource capacity. This represents resource usage information. The difference is used to determine the current remaining computing resources.

[0038] Step 203: Construct the resource status information of the computing node based on communication load, remaining computing resources, and interconnection bandwidth.

[0039] The calculated communication load, remaining computing resources, and real-time acquired interconnect bandwidth are used as different feature vectors. These vectors are then concatenated into arrays in the order of communication load, remaining computing resources, and interconnect bandwidth. The concatenated arrays are then encapsulated to construct a multi-dimensional data matrix format, ultimately yielding comprehensive resource status information representing the computing nodes. This multi-dimensional data matrix format preserves the independent characteristics of communication, computing power, and bus attributes while establishing structured relationships. This facilitates subsequent structural retrieval of computing resources by reading values ​​from specific dimensions of the matrix or performing dimensionality reduction comparison and matching in matrix operations.

[0040] Step 103: Based on the resource status information, determine the target inference task and the target computing resources required by the target inference task from the candidate task set. When the remaining computing resources of the computing node at the current time meet the requirements of the target computing resources, perform atomic pre-occupation of the target computing resources.

[0041] The candidate task set refers to the total number of task instructions currently waiting to be scheduled for execution in the local queue of the compute node. The target inference task refers to the online prediction task selected from the queue that urgently needs to be executed. The target computing resources refer to the resource requirement vector required to maintain the operation of the target inference task, which may include at least the number of compute cores, device-side storage capacity, and host-side temporary cache capacity required by the target inference task. When determining whether the remaining computing resources meet the target computing resource requirements, resources of the same dimension in the resource requirement vector and the remaining computing resource vector are compared separately; only when the remaining amount of each type of resource meets the corresponding requirement is it determined that the remaining computing resources of the compute node at the current moment meet the target computing resource requirements.

[0042] Atomic pre-emption refers to a low-level mechanism where the operation is indivisible and must be locked in a single, instantaneous process. In scenarios with sudden high concurrency, if an absolutely indivisible lock is not immediately implemented after resource verification confirms that the demand is met, the microsecond-level time difference can lead to other concurrent tasks preempting resources, causing node over-suppliing or even severe memory overflow and crashes. By performing atomic pre-emption through a kernel spinlock mechanism at the absolute instant that the remaining computing resources meet the demand, a robust and secure local real-time defense mechanism is provided.

[0043] Step 301: Based on the task list of the candidate task set, determine multiple candidate tasks and the task requirements corresponding to each candidate task.

[0044] The task list refers to the list of task descriptions stored at the head of the scheduling buffer queue. Task requirements specify the required memory bytes needed to run the corresponding task. By reading and parsing the task list of the candidate task set, the task identification information contained in the list is traversed to determine if it contains multiple candidate tasks currently in the queue. Simultaneously, the configuration file attached to each candidate task is extracted and parsed, and the task requirement information is extracted from the configuration file.

[0045] Step 302: Select each candidate task whose required computing resources are less than or equal to the remaining computing resources in the resource status information as a matching task.

[0046] A matching task refers to a task entity whose resource requirement does not exceed the node's current available surplus. Preliminary filtering and selection operations are performed, extracting the task requirement value for each candidate task one by one. This value is then compared with the remaining computing resources extracted from the resource status information matrix. During the numerical comparison, all candidate tasks whose required computing resources are less than or equal to the remaining computing resources are filtered out. Any candidate task satisfying this inequality is marked as a matching task.

[0047] Step 303: Determine the target inference task from multiple matching tasks based on the preset scheduling priority, and determine the target computing resources required by the target inference task based on the configuration identifier of the target inference task.

[0048] The preset scheduling priority refers to the scoring and ranking rules based on task urgency pre-set in the kernel. The configuration identifier refers to a specific resource request tag in the header of the code file. The weight parameters corresponding to the preset scheduling priority rules are read, and the waiting time and resource requirements of the multiple matching tasks selected in the previous step are weighted and summed for a comprehensive score and ranking. The task with the highest comprehensive score and the highest priority for execution is selected from the multiple matching tasks and designated as the target inference task. The configuration identifier carried within the target inference task is specifically read, parsed, and its byte size value is extracted to determine the actual target computing resources required for the execution of the target inference task.

[0049] Step 311: Calculate the resource difference between the remaining computing resources of the computing node at the current moment and the target computing resources.

[0050] Resource difference refers to the set of differences between the remaining computing resource vector and the target computing resource vector across the same resource type dimension. If the resource differences across all dimensions, including core computing resources, device-side storage resources, and host-side storage resources, are all greater than or equal to zero, then the remaining computing resources are deemed to meet the target computing resource requirements. The process involves extracting the currently measured remaining computing resource values ​​from memory, extracting the determined target computing resource values, and comparing the remaining and target computing resource values ​​by subtraction to accurately calculate the resource difference in capacity space. The specific calculation formula is as follows: ,in Represents the resource difference. Represents remaining computing resources. Represents the target computing resources.

[0051] Step 312: If the resource difference is greater than or equal to zero, determine that the remaining computing resources meet the requirements of the target computing resources and generate a lock status label. Write the lock status label into the status register or management address space corresponding to the target computing resources and mark the target computing resources as pre-occupied.

[0052] A locked status label is an identifier composed of hexadecimal characters representing an unreadable / unwritable state. A status register is a high-speed storage unit within the central processing unit (CPU) used to store runtime status words. The management address space is a memory-mapped region dedicated to the kernel. A pre-empted state refers to a resource that is physically unused but logically refused allocation. Logical operations are performed based on the calculated numerical results. If the resource difference is greater than or equal to zero, this logically proves that the current computing power inventory is indeed sufficient and available, confirming that the remaining computing resources truly meet the needs of the target computing resource. Once the condition is met, a locked status label is generated in memory. Through the hardware driver interface, the locked status label is forcibly written into the status register corresponding to the target computing resource, or written into the management address space, thereby marking the target computing resource as an unallocable pre-empted state at the lowest level.

[0053] Step 313: Use the pre-occupancy state to intercept call requests from other concurrent tasks to complete the atomic pre-occupancy of the target computing resources.

[0054] The underlying resource scheduling controller polls in real time to identify the pre-allocation status. A call request refers to a communication instruction sent by other queued tasks to the computing unit to allocate resources. Once the polling identifies this pre-allocation status, the underlying resource scheduling controller will discard the call request message to automatically intercept and directly reject any other concurrently arriving queued tasks' call requests for the target computing resource. By using the message discarding and interception mechanism, atomic pre-allocation of the target computing resource is achieved without interruption, avoiding resource overselling and system crashes caused by contention in high-concurrency scenarios.

[0055] Step 104: When processing the computation kernel function of the large model training task on the heterogeneous computing device, monitor the execution boundary of the computation kernel function. When the execution boundary is reached, suspend the large model training task and schedule the target inference task to run on the heterogeneous computing device of the corresponding computing node.

[0056] Large-scale model training tasks refer to deep learning processes involving massive parameter updates and backpropagation computations. A computation kernel function is the smallest parallel processing unit deployed to heterogeneous computing devices for massive matrix operations. The execution boundary refers to the instruction switching point within the computation kernel function that represents the complete end of an independent micro-operation cycle. By monitoring and suspending execution boundaries, we avoid sending forced termination signals to directly and brutally kill the running training process, preventing catastrophic losses due to the loss of long-term accumulated progress data. Through code injection probes, we find instruction-level safe execution boundaries, smoothly halt the process, and save the current state. This achieves lossless interception with absolute preservation of training assets while immediately freeing up computing resources for urgent inference requests, thus completing a very low-loss switch between the two.

[0057] Step 401: Based on the execution queue of the large model training task, determine the computation kernel function to be processed, and parse the machine instructions of the computation kernel function according to the preset instruction mapping rules to obtain the instruction execution sequence.

[0058] An execution queue is a linked list structure in the underlying hardware used to sequentially store pointers to kernel functions awaiting processing. A pre-defined instruction mapping rule refers to a pre-configured opcode lookup table defined by the chip manufacturer, mapping machine code to microcode. Machine instructions are binary code consisting of 0s and 1s that can be directly recognized and executed by a computer. An instruction execution sequence is a set of underlying code lines arranged chronologically and logically traceable. By accessing the execution queue at the bottom layer of a large model training task and reading the queue head pointer, the specific computational kernel function currently being processed at the hardware level of the heterogeneous computing device can be determined.

[0059] The system invokes a pre-defined instruction mapping rule. This rule contains a machine instruction disassembly dictionary for specific heterogeneous architectures, such as mapping hexadecimal low-level control machine code to specific instruction behaviors. By accessing the instruction registers of heterogeneous devices, it intercepts the issued binary code line by line, and by traversing the opcode lookup table and invoking the pre-defined instruction mapping rule, it extracts and reverse-translates the machine instructions for calculating kernel functions, such as the intercepted machine code starting with 0x8B, converting them into the corresponding data load instruction MOV and its addressing parameters. Finally, it concatenates the decompiled instructions according to the execution timestamp to obtain a traceable instruction execution sequence.

[0060] Taking a scenario where the heterogeneous computing device is a GPU as an example, during the training of a large model, the execution flow and command queue are submitted to the driver layer through the heterogeneous computing runtime interface. The command queue records the kernel function identifier, execution flow identifier, command buffer address, compiled code object address, and the resource handle corresponding to this submission. The underlying scheduler controller of the compute node reads the queue head pointer of the current execution flow through the driver layer command queue interface to determine the currently processed kernel function; then, it reads the compiled code object corresponding to the kernel function through the code object query interface provided by the driver layer, and parses the machine instructions in the code object into an instruction execution sequence according to the preset instruction mapping rules. For heterogeneous computing devices that do not expose the underlying machine code reading, the underlying scheduler controller can read the kernel function completion event, memory synchronization event, and barrier event output by the runtime interface, and convert these events into an event execution sequence corresponding to the instruction execution sequence according to the submission order.

[0061] Step 402: Identify the execution boundary, including the end-of-operation cycle marker, from the instruction execution sequence, and generate an interrupt suspension signal when multiple lines of low-level execution code are processed on a heterogeneous computing device and reach the execution boundary.

[0062] Execution boundary identifiers are used to characterize the boundary positions in the computation kernel function where it is appropriate to pause the training task. An execution boundary identifier can consist of a kernel function identifier, an execution flow identifier, a boundary type, a sequence number, and an instruction offset; the boundary type can include the end boundary of the computation kernel function, the memory synchronization completion boundary, the tensor write-back completion boundary, and the inter-device data transfer completion boundary.

[0063] The underlying scheduler scans the instruction execution sequence according to a preset instruction pattern or event pattern. When a synchronization instruction, memory barrier instruction, kernel function completion event, or tensor write-back completion event is identified, the location is determined as the execution boundary. When the execution flow advances to the execution boundary and there are no incomplete write-back operations, the underlying scheduler generates an interrupt suspension signal or a pause scheduling signal to prevent the training task from submitting the next computation kernel function.

[0064] To avoid data inconsistencies caused by pausing in the middle of the computation kernel function, the underlying scheduler performs boundary checks before generating an interrupt suspension signal. Boundary checks include at least confirming that the output tensor of the current computation kernel function has been written back, that memory copy commands in the corresponding execution flow that predate this execution boundary have been completed, that dirty page flags in the device-side cache have been synchronized to the video memory page table, and that there are no incomplete write operations pointing to the target computation resource. If any of these checks are not met, no interrupt suspension signal is generated, and the system continues to wait for the next execution boundary.

[0065] Step 403: Based on the interrupt suspension signal, save the context state of the large model training task to the local storage space to suspend the large model training task. After suspending the large model training task, schedule the target inference task to run on the heterogeneous computing device of the corresponding computing node.

[0066] The context state includes the process identifier, execution flow identifier, command queue pointer, current computation kernel identifier, program counter position, critical register snapshot, resource handle, video memory page table mapping, tensor dirty page bitmap, model parameter page index, and gradient state page index at the moment the large model training task is paused. Local storage space refers to the dynamic random access memory chip built into the node. After generating and capturing an interrupt suspension signal, the underlying scheduler freezes the execution flow corresponding to the training task, stops submitting subsequent computation kernel functions of the training task to heterogeneous computing devices, and writes the aforementioned context state into the local storage space of the computing node.

[0067] After the context state is saved, the underlying scheduling controller determines the GPU memory pages that need to be swapped out based on the tensor dirty page bitmap and allocates contiguous or mappable temporary storage areas in the host-side storage resources. Subsequently, the model parameter pages, gradient state pages, and intermediate activation pages are copied from the device-side storage resources to the host-side temporary storage area via the device-driven data transfer interface, and the mapping relationship between the device-side page addresses and the host-side temporary storage area addresses is recorded. After copying is complete, a pause flag is set for the swapped-out pages, and the device-side storage resources and computing core resources required for the target inference task are released.

[0068] After the target inference task completes and releases its occupied resources, the underlying scheduling controller reallocates computational core resources and device-side storage resources for the large model training task based on the updated resource status information. Then, according to the saved mapping relationship, it copies the model parameter pages, gradient state pages, and intermediate activation pages from the host-side temporary storage area back to the device-side storage resources. Subsequently, it restores the command queue pointer, execution flow identifier, program counter position, and critical register snapshot, and continues submitting subsequent computational kernel functions for the training task from after the saved execution boundary. If page copying fails, resources are insufficient, or execution flow recovery times out during the restoration process, the training task remains suspended, the pre-occupied tag corresponding to the target inference task is released, and the restoration process is re-executed based on the most recently fully saved context state.

[0069] Step 105: After the target inference task is completed and the occupied resources are released, scan the idle resources in the computing node, merge them and update the resource status information to resume the execution of the large model training task.

[0070] In this embodiment, occupied resources include computing core resources locked during the execution of the target inference task and storage resources related to the storage of data for the target inference task. The occupied resources involved in address range calculation and block merging in steps 501 to 514 refer to device-side or host-side storage blocks released by the target inference task. Idle resources refer to device-side or host-side storage blocks that are not currently allocated to any task. Computing core resources are released and reallocated through scheduling entries and do not participate in logical address range adjacency determination or physical address relocation.

[0071] The target inference task itself has a short lifespan. After its rapid completion, it sends a termination signal to the kernel to automatically return and release the resources it locked during execution. In hybrid scheduling scenarios, the extremely frequent preemption, suspension, and iteration of tasks can fragment the internal resource space of the device.

[0072] Merging resources refers to combining Figure 2As shown, after low-level scanning and combining, multiple fragmented physical or logical spaces are pieced together to form a large, block-shaped merged resource that can be redistributed as a whole. If the daemon process is not actively started to scan and merge idle resources, a large number of idle but fragmented computing power fragments will be generated, which are too large to be reloaded for training tasks due to their dispersed space. By scanning idle areas and updating page tables, fragmented holes can be pieced together into a continuous large space, clearing the way for the resumption of training tasks.

[0073] Step 501: After the target inference task is completed and the occupied resources are released, retrieve multiple free resources within the computing node, and calculate the logical address range of the released occupied resources and the logical address range of each free resource based on the preset memory mapping table.

[0074] A pre-defined memory mapping table refers to a page table data structure that is pre-established to maintain the translation association between virtual logical addresses and real physical memory pages. A logical address range refers to a continuous range of virtual addresses from the start addressing code to the end addressing code. When the target inference task declares completion and returns resources to the underlying environment, these are considered released resources.

[0075] The system then traverses the memory management linked list to retrieve multiple unused idle resources within the compute node that were left over from various task iterations. It calls the underlying preset memory mapping table. Based on the preset memory mapping table, it reads the start and end address pointers to accurately identify and calculate the start and end logical address ranges of the recently released occupied resources to obtain their logical address intervals. Similarly, it traverses the page table records to calculate the logical address interval for each idle resource retrieved at this time.

[0076] Step 502: Compare the boundary values ​​of the logical address range of the released occupied resources with the logical address range of each free resource to determine the target free resource. When the logical address range of the target free resource is adjacent to the logical address range of the released occupied resources, combine the released occupied resources and the target free resources into a continuous computing power block.

[0077] The target free resource refers to a free storage block that belongs to the same storage type as the released occupied resource and whose logical address range is adjacent to that of the released occupied resource. Boundary value comparison refers to comparing the start and end addresses of two address ranges under the same storage type to determine whether they are adjacent. A contiguous computing power block refers to... Figure 3 The computing power blocks within it have a seamlessly contiguous available memory space in terms of logical addresses. Combined with... Figure 3 As shown, the beginning and end boundary values ​​of the logical address range of the released occupied resources are compared mathematically with the beginning and end boundary values ​​of the logical address range of each free resource obtained through traversal.

[0078] like Figure 3 As shown, Figure 3 The code displays the target free resource and the released occupied resource. Through a subtraction comparison operation, if the absolute value of the difference between the tail logical address of a free resource and the head logical address of the released occupied resource equals the set addressing step size, it indicates that an occupied resource is immediately following the release, and the associated free resource is selected as the target free resource. Here, the set addressing step size refers to the smallest basic unit size used by the operating system's memory management unit for memory paging management, typically configured as the size of a standard page frame, such as 4KB or 2MB. When the difference between the start and end addresses of two memory blocks is exactly equal to the size of a standard page frame (i.e., the set addressing step size), it indicates that there are no fragmentation gaps in their logical address space, and they are absolutely adjacent.

[0079] When the logical address range of the target free resource is determined to be absolutely adjacent to the logical address range of the released occupied resource, the released occupied resource and the target free resource are combined into one by stitching and modifying the underlying memory mapping, thereby forming a larger and logically seamless continuous computing power block.

[0080] Step 503: Based on the computing power block, update the resource status information to obtain the updated resource status information.

[0081] The system captures the latest information of newly generated consecutive computing power blocks due to logical combination, and writes the changes in capacity and starting address of the latest consecutive computing power blocks into memory to overwrite the records. This updates the overall resource status information, resulting in updated resource status information that is more explicit and eliminates some virtual fragment records.

[0082] Step 511: Based on the updated resource status information, determine multiple discrete idle resources within the computing node, sum the capacities of each discrete idle resource to obtain the discrete total capacity, and determine the total node capacity of the computing node.

[0083] Discrete idle resources refer to, for example Figure 3 Discrete free resources are isolated, scattered blocks of idle memory whose logical addresses are not adjacent. The total discrete capacity refers to the sum of the byte sizes of all discrete free resources. The total node capacity refers to the maximum available physical memory value specified at the factory for the compute node.

[0084] To implement deep fragmentation defense, based on updated resource status information, a large number of discrete idle resources within the compute node were identified that could not be successfully combined due to non-adjacent logical addresses. The capacity of each discrete idle resource was extracted, and the capacities of each resource were summed using an addition algorithm. The summed capacity was then used as the total discrete capacity. The total node capacity of the compute node was determined by querying the node's hardware configuration file.

[0085] Step 512: Use the ratio of the total discrete capacity to the total node capacity as the fragmentation index. When the fragmentation index is greater than the preset fragmentation threshold, generate a memory migration instruction.

[0086] The fragmentation index is a ratio metric representing the severity of memory fragmentation and disconnections within a node. The preset fragmentation threshold is a pre-defined maximum tolerable percentage of memory fragmentation. Memory migration instructions are high-privilege driven operations that forcibly intervene in the underlying hardware control to move scattered physical data to one side. Using a division algorithm, the ratio of the total discrete capacity to the total node capacity is calculated, and this ratio is defined as the fragmentation index for assessing memory health. The specific calculation formula is as follows: ,in Represents the fragmentation index, Represents the discrete total capacity. This represents the total capacity of the node. The fragmentation index will be conditionally assessed. The preset fragmentation threshold can be set to 40%. When the fragmentation index exceeds the preset threshold, it indicates severe memory fragmentation, posing a risk that large tasks may be difficult to resume. In this case, a memory migration instruction will be automatically generated and dispatched to the underlying memory management unit.

[0087] Step 513: Use memory migration instructions to move multiple occupied resources within a computing node to a contiguous physical address range, so as to combine all discrete idle resources into a contiguous block of idle computing power.

[0088] A contiguous block of free computing power refers to, for example, a block of physical addresses. Figure 3 The idle computing power blocks in the system represent a large, contiguous section of available space at the physical hardware circuit level. Combined with... Figure 3 The active defragmentation mechanism shown receives and utilizes triggered memory migration instructions to perform proactive defragmentation and physical reorganization. This is applied to running data scattered throughout various parts of the computing node. Figure 3To lock occupied resources, a global soft interrupt is first sent to the kernel to suspend addressing and access to the relevant occupied resources. Then, the direct memory access controller is used to reverse-force the physical data inside the occupied resource and move it continuously to one side of the physical address range within the memory module. After the occupied resource is moved, the gaps that were originally interspersed are forcibly cleared and connected into a single block, concentrating the occupied space. In this way, all the previously scattered and discrete free resources are completely pieced together and combined into a large and complete block of free computing power with contiguous physical addresses.

[0089] Step 514: Record the idle computing power blocks into the updated resource status information to obtain the second updated resource status information, and use the second updated resource status information to resume the execution of the large model training task.

[0090] The updated resource status information refers to data records containing the latest healthy and continuous resource status, generated after deep physical relocation, compression, and reorganization. The specific physical address range and latest capacity information of the idle computing power blocks are recorded and loaded, overwriting the previously updated resource status information to obtain the updated resource status information. Guided by the continuous memory support of the updated resource status information, the previously frozen large model training task context state is retrieved from local storage based on the saved pointer address. The program counter is set to the interrupt location, and the large model training task is successfully woken up and resumed using the ample idle computing power blocks as a running platform. This completes the low-level computing power closed-loop self-healing management under extremely harsh heterogeneous conditions.

[0091] This application's embodiments overcome the limitations of existing technologies that overly rely on lagging data from upper-level scheduling centers by acquiring the transmission throughput and congestion status of the network protocol stack in the computing node, as well as the resource occupancy information of heterogeneous computing devices within the computing node and the interconnection bandwidth between devices. This eliminates the blind spots of single-dimensional status evaluation, avoids the risk of node memory overflow and crashes caused by disordered resource contention during periods of high concurrency of sudden tasks, and establishes a technical barrier to ensure the absolute stability of computing nodes. It abandons the outdated strategies of crudely killing running processes or forcing new tasks into long queues, achieving extremely low-loss switching between mixed tasks. This ensures that large model training tasks can obtain continuous and complete resource support for lossless resumption of execution.

[0092] Figure 4 This is a schematic diagram illustrating a specific implementation of a computing node resource awareness and task execution system for a heterogeneous computing power scheduling system provided in this application. (Refer to...) Figure 4 The system may include: The acquisition module 21 is used to acquire the transmission throughput and congestion status of the network protocol stack in the computing node, as well as the resource occupancy information of heterogeneous computing devices within the computing node and the interconnection bandwidth between devices. Module 22 is used to construct resource status information of computing nodes based on transmission throughput, congestion status, resource occupancy information and interconnection bandwidth; The determination module 23 is used to determine the target inference task and the target computing resources required by the target inference task from the candidate task set based on the resource status information. When the remaining computing resources of the computing node at the current time meet the requirements of the target computing resources, the target computing resources are atomically pre-occupied. The monitoring module 24 is used to monitor the execution boundary of the computation kernel function when the heterogeneous computing device is processing the computation kernel function of the large model training task. When the execution boundary is reached, the large model training task is suspended and the target inference task is scheduled to run in the heterogeneous computing device of the corresponding computing node. The merging module 25 is used to scan the idle resources in the computing node and merge them after the target inference task has finished running and released the occupied resources, and then update the resource status information so as to resume the execution of the large model training task.

[0093] This application provides a heterogeneous computing power scheduling system for computing node resource awareness and task execution, which is used to implement the aforementioned heterogeneous computing power scheduling system for computing node resource awareness and task execution. Therefore, the specific implementation of the heterogeneous computing power scheduling system for computing node resource awareness and task execution can be found in the embodiment section of the heterogeneous computing power scheduling system for computing node resource awareness and task execution, and its specific implementation can be referred to the description of the corresponding embodiments, which will not be repeated here.

[0094] Figure 5 A schematic diagram of the hardware structure of the electronic device provided in an embodiment of this application is shown.

[0095] This application also provides an electronic device, including: a memory for storing a computer program; and a processor for executing the computer program to implement the steps of a computing node resource awareness and task execution method of a heterogeneous computing power scheduling system as described above.

[0096] The electronic device may include a processor 510 and a memory 520 storing computer program instructions.

[0097] Specifically, the processor 510 may include a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits that can be configured to implement the embodiments of this application.

[0098] Memory 520 may include mass storage for data or instructions. For example, and not limitingly, memory 520 may include a hard disk drive (HDD), floppy disk drive, flash memory, optical disk, magneto-optical disk, magnetic tape, or Universal Serial Bus (USB) drive, or a combination of two or more of these. Where appropriate, memory 520 may include removable or non-removable (or fixed) media. Where appropriate, memory 520 may be internal or external to the integrated gateway disaster recovery device. In a particular embodiment, memory 520 is non-volatile solid-state memory.

[0099] Memory may include read-only memory (ROM), random access memory (RAM), disk storage media devices, optical storage media devices, flash memory devices, and electrical, optical, or other physical / tangible memory storage devices. Therefore, typically, memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., memory devices) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described with reference to the method according to the first aspect of this disclosure.

[0100] The processor 510 reads and executes computer program instructions stored in the memory 520 to implement the computing node resource awareness and task execution method of any heterogeneous computing power scheduling system in the above embodiments.

[0101] In one example, the electronic device may also include a communication interface 530 and a bus 540. Wherein, such as Figure 5 As shown, the processor 510, memory 520, and communication interface 530 are connected through bus 540 and complete communication with each other.

[0102] The communication interface 530 is mainly used to realize communication between various modules, devices, units and / or equipment in the embodiments of this application.

[0103] Bus 540 includes hardware, software, or both, that couples components of an online data traffic metering device together. For example, and not limitingly, the bus may include an Accelerated Graphics Port (AGP) or other graphics bus, an Enhanced Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), HyperTransport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an Infinite Bandwidth Interconnect, a Low Pin Count (LPC) bus, a memory bus, a Microchannel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local (VLB) bus, or other suitable buses, or combinations of two or more of these. Where appropriate, bus 540 may include one or more buses. Although specific buses are described and illustrated in embodiments of this application, any suitable bus or interconnect is contemplated herein.

[0104] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of a computing node resource awareness and task execution method for a heterogeneous computing power scheduling system as described above.

[0105] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as USB flash drives, read-only memory, random access memory, portable hard drives, magnetic disks, or optical disks.

[0106] Embodiments of the present invention also provide a computer program product, which includes a computer program that, when executed by a processor, implements the steps in the embodiments of the computing node resource awareness and task execution method of any of the heterogeneous computing power scheduling systems described above.

[0107] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0108] The foregoing has provided a detailed description of the computing node resource awareness and task execution method and system of a heterogeneous computing power scheduling system provided in this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are merely for the purpose of helping to understand the method and its core ideas. It should be noted that those skilled in the art can make various improvements and modifications to this application without departing from its principles, and these improvements and modifications also fall within the protection scope of this application.

Claims

1. A method for computing node resource awareness and task execution in a heterogeneous computing power scheduling system, characterized in that, include: The system obtains the transmission throughput and congestion status of the network protocol stack in the computing node, as well as the resource usage information of heterogeneous computing devices within the computing node and the interconnection bandwidth between devices. The resource status information of the computing node is constructed based on the transmission throughput, the congestion status, the resource occupancy information, and the interconnection bandwidth; Based on the resource status information, the target inference task and the target computing resources required by the target inference task are determined from the candidate task set. When the remaining computing resources of the computing node at the current time meet the requirements of the target computing resources, the target computing resources are atomically pre-occupied. When the heterogeneous computing device processes the computation kernel function of the large model training task, the execution boundary of the computation kernel function is monitored. When the execution boundary is reached, the large model training task is suspended, and the target inference task is scheduled to run in the heterogeneous computing device of the corresponding computing node. After the target inference task is completed and the occupied resources are released, the idle resources in the computing node are scanned, merged, and the resource status information is updated to resume the execution of the large model training task.

2. The method according to claim 1, characterized in that, The resource status information of the computing node is constructed based on the transmission throughput, the congestion state, the resource occupancy information, and the interconnection bandwidth, including: The product of the transmission throughput and the congestion state is used as the communication load; The difference between the total resource capacity of the computing node and the resource occupancy information is taken as the remaining computing resources; Based on the communication load, the remaining computing resources, and the interconnection bandwidth, the resource status information of the computing node is constructed.

3. The method according to claim 1, characterized in that, Based on the resource status information, the target inference task and the target computing resources required by the target inference task are determined from the candidate task set, including: Based on the task list of the candidate task set, multiple candidate tasks and the task requirements corresponding to each candidate task are determined. Each candidate task whose required computing resources are less than or equal to the remaining computing resources in the resource status information is considered as a matching task. The target inference task is determined from multiple matching tasks based on a preset scheduling priority, and the target computing resources required by the target inference task are determined based on the configuration identifier of the target inference task.

4. The method according to claim 1, characterized in that, When the remaining computing resources of the computing node at the current moment meet the demand of the target computing resources, atomic pre-allocation of the target computing resources is performed, including: Calculate the resource difference between the remaining computing resources of the computing node at the current moment and the target computing resources; If the resource difference is greater than or equal to zero, it is determined that the remaining computing resources meet the requirements of the target computing resources and a lock status label is generated. The lock status label is written into the status register or management address space corresponding to the target computing resources, and the target computing resources are marked as pre-occupied. The pre-occupancy state is used to intercept call requests from other concurrent tasks in order to complete the atomic pre-occupancy of the target computing resources.

5. The method according to claim 1, characterized in that, When the heterogeneous computing device processes the computation kernel function of a large model training task, the execution boundary of the computation kernel function is monitored. Upon reaching the execution boundary, the large model training task is suspended, and the target inference task is scheduled to run on the corresponding computing node of the heterogeneous computing device. This includes: Based on the execution queue of the large model training task, the computation kernel function to be processed is determined, and the machine instructions of the computation kernel function are parsed according to the preset instruction mapping rules to obtain the instruction execution sequence. An execution boundary, including an end-of-operation cycle identifier, is identified from the instruction execution sequence, and an interrupt suspension signal is generated when the heterogeneous computing device processes multiple lines of low-level execution code and reaches the execution boundary. Based on the interruption suspension signal, the context state of the large model training task is saved to the local storage space to suspend the large model training task. After suspending the large model training task, the target inference task is scheduled to run on the heterogeneous computing device of the corresponding computing node.

6. The method according to claim 1, characterized in that, After the target inference task completes and releases its occupied resources, the idle resources within the computing node are scanned, merged, and the resource status information is updated, including: After the target inference task is completed and the occupied resources are released, multiple free resources within the computing node are retrieved, and based on a preset memory mapping table, the logical address range of the released occupied resources and the logical address range of each free resource are calculated. The logical address range of the released occupied resources is compared with the logical address range of each free resource by boundary value to determine the target free resource. When the logical address range of the target free resource is adjacent to the logical address range of the released occupied resources, the released occupied resources and the target free resources are combined into a continuous computing power block. Based on the computing power block, the resource status information is updated to obtain the updated resource status information.

7. The method according to claim 6, characterized in that, After updating the resource status information based on the computing power block to obtain the updated resource status information, the method further includes: Based on the updated resource status information, multiple discrete idle resources within the computing node are determined, the capacity of each discrete idle resource is summed to obtain the discrete total capacity, and the total node capacity of the computing node is determined. The ratio of the discrete total capacity to the node total capacity is used as the fragmentation index. When the fragmentation index is greater than a preset fragmentation threshold, a memory migration instruction is generated. Using the memory migration instructions, multiple occupied resources within the computing node are moved to a continuous physical address range, so as to combine all discrete idle resources into a block of idle computing power with a continuous physical address. The idle computing power blocks are recorded in the updated resource status information to obtain the second updated resource status information, and the execution of the large model training task is restored using the second updated resource status information.

8. A computing node resource awareness and task execution system for a heterogeneous computing power scheduling system, characterized in that, include: The acquisition module is used to acquire the transmission throughput and congestion status of the network protocol stack in the computing node, as well as the resource occupancy information of the heterogeneous computing devices within the computing node and the interconnection bandwidth between the devices. A construction module is used to construct the resource status information of the computing node based on the transmission throughput, the congestion state, the resource occupancy information, and the interconnection bandwidth; The determination module is used to determine the target inference task and the target computing resources required by the target inference task from the candidate task set based on the resource status information, and to perform atomic pre-allocation of the target computing resources when the remaining computing resources of the computing node at the current time meet the requirements of the target computing resources; The monitoring module is used to monitor the execution boundary of the computation kernel function when the heterogeneous computing device is processing the large model training task, suspend the large model training task when the execution boundary is reached, and schedule the target inference task to run in the heterogeneous computing device of the corresponding computing node. The merging module is used to scan the idle resources within the computing node and merge them after the target inference task has finished running and released the occupied resources, and then update the resource status information so as to resume the execution of the large model training task.

9. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor is configured to implement the steps of the computing node resource awareness and task execution method of a heterogeneous computing power scheduling system as described in any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, which, when executed by a processor, enables a method for computing node resource awareness and task execution in a heterogeneous computing power scheduling system as described in any one of claims 1 to 7.

Images