A dynamic resource perception and adaptive computing task scheduling method and system for heterogeneous edge computing devices

Abstract

This invention relates to the fields of distributed computing and edge computing technology, providing a dynamic resource awareness and adaptive computing task scheduling method and system for heterogeneous edge computing devices. The method collects node runtime hardware status parameters through a sliding time window, including storage resource access conflict indicators, storage resource residency status, computing unit operating status, and data transmission link status. Based on these parameters, it performs node availability determination, excluding nodes from the scheduling candidate set when there is a risk of resource conflict or transmission bottleneck. A distributed scheduling structure is constructed based on the candidate node set, and tasks are adaptively partitioned and matched according to node resource status. When system load increases, a constraint control mechanism reduces the task allocation ratio of high-resource nodes and limits their upper limit, while increasing the participation of low-load nodes to control task distribution across multiple nodes. When a node malfunctions, intermediate state data is extracted and migrated to other nodes for continued execution. Furthermore, under network conditions that meet bandwidth and latency requirements, a remote memory access mechanism enables collaborative use of cross-node storage resources, thereby expanding computing resource capabilities.

Description

Technical Field

[0001] This invention relates to the fields of distributed computing, artificial intelligence computing, edge computing, and hardware resource monitoring, and particularly to a decentralized computing power scheduling method and system based on a hardware underlying operating characteristic perception and constraint control mechanism. Background Technology

[0002] In existing distributed computing scheduling, due to the high heterogeneity and unstable operating states of edge devices, traditional scheduling methods typically rely on static parameters such as memory capacity. This approach struggles to reflect the data exchange pressure within storage resources and the interference from host tasks, easily leading to resource allocation imbalances and localized network congestion. Furthermore, current technologies often treat the network merely as a data transmission channel, failing to fully utilize the cross-node resource collaboration capabilities in high-bandwidth network environments, resulting in the inability to effectively share storage resources across multiple nodes. Therefore, a scheduling method is needed that can dynamically constrain based on the underlying operating state and support cross-node resource collaboration. Summary of the Invention

[0003] This invention provides a distributed computing power scheduling method based on hardware operating characteristics. Its core technical features include: a perception and judgment mechanism: collecting the underlying hardware operating status based on a sliding time window and excluding nodes with conflict risks through judgment logic including hysteresis control; a constraint control mechanism: dynamically constraining scheduling priorities based on the overall system load status, reducing the task proportion of high-resource nodes, and controlling the distribution of tasks among multiple nodes; a fault tolerance and recovery mechanism: realizing computational state extraction and cross-node migration recovery based on a hardware architecture-independent intermediate representation; and a cross-node resource collaboration mechanism: under network conditions that meet bandwidth and latency requirements, enabling collaborative use of storage resources across multiple nodes through remote memory access. Attached Figure Description

[0004] Figure 1 This is a flowchart illustrating the system's global hierarchical structure and collaborative evolution of the present invention. Detailed Implementation

[0005] Example 1: Resource Conflict Determination. By monitoring memory access behavior and bandwidth usage, nodes are excluded when a conflict risk is detected, and a hysteresis mechanism is used to ensure system stability.

[0006] Example 2: Load-constrained scheduling. As system load increases, the proportion of tasks on high-resource nodes is gradually reduced to achieve a more balanced task distribution, thereby reducing the risk of congestion.

[0007] Example 3: State Transition Recovery. When a node fails, the intermediate computation state is extracted and migrated to other nodes to achieve continuous task execution.

[0008] Example 4: Cross-node storage resource collaboration. When network conditions are met, the storage resources of multiple nodes are logically integrated, and when node storage is insufficient, some intermediate data is transferred to other nodes through a remote memory access mechanism, thereby expanding the available computing resources.

Claims

1. A method and system for dynamic resource awareness and adaptive computing task scheduling for heterogeneous edge computing devices, characterized in that, include: S1 Hardware Operation Feature Acquisition and Judgment: Acquire the hardware status parameters of the node during the sliding time window, including: storage resource access conflict related indicators, storage resource residency status, computing unit operation status, and data transmission link status; perform node availability determination based on the parameters, characterized in that: the resource conflict risk is determined based on the storage resource access conflict indicators and data transmission link status; when a node has resource conflict risk or data transmission bottleneck risk, it is excluded from the scheduling candidate set; the determination process includes a hysteresis mechanism to ensure the stability of the node status under critical conditions; S2 Candidate Node Constraint Scheduling: A distributed scheduling structure is constructed based on the determined node set, and the scheduling process is subject to real-time dynamic constraints of the determined results; S3 Task Partitioning and Resource Adaptation: The computation task is divided into multiple subtasks, and adaptive matching is performed based on the resource residency status of candidate nodes to ensure that the resource requirements of each subtask do not exceed the actual available resource capacity of the target node. S4 Scheduling Priority Constraint Control Mechanism: Scheduling priorities are generated based on node capabilities, task requirements, and the overall system load. Its features include: reducing the task allocation ratio of high-resource nodes when the overall system load increases; limiting the upper limit of the proportion of high-resource nodes in the scheduling results, with this upper limit dynamically adjusted according to the system load; increasing the task participation of low-load nodes; controlling the distribution of tasks among multiple nodes; and the priority adjustment is based on real-time constraint control of the system's operating status, rather than relying on a fixed calculation model. S5 State Transition and Anomaly Recovery: When a node experiences a response timeout, heartbeat loss, or abnormal increase in resource consumption: extract intermediate computation state data and store it in a structured manner; record the computation progress index; migrate the state data to other nodes to continue execution; wherein the state data adopts a representation method independent of the underlying hardware architecture; S6 Cross-node storage resource collaboration mechanism: When the data transmission link is detected to meet the conditions of high bandwidth and low latency, the following actions are taken: Logically associate the available storage resources of multiple nodes to form a unified virtual resource space; when a node experiences a storage resource access conflict or insufficient capacity, transfer some intermediate data to the available storage resources of other nodes through a remote memory access mechanism; access and restore the remote storage data during subsequent calculations; thereby realizing the expansion and collaborative use of cross-node storage resources.

2. The method according to claim 1, characterized in that: The storage resource access conflict indicators include video memory paging frequency or cache replacement frequency.

3. The method according to claim 1, characterized in that: The data transmission link status includes the system's internal bus bandwidth utilization or the external network bandwidth utilization.

4. The method according to claim 1, characterized in that: The operating state of the computing unit includes thermal throttling or performance degradation.

5. The method according to claim 1, characterized in that: The change in scheduling priority satisfies the monotonicity constraint that varies with system load.

6. The method according to claim 1, characterized in that: The scheduling priority change is suppressed using a non-linear method.

7. The method according to claim 1, characterized in that: The task partitioning includes model parallelism, data parallelism, or pipeline parallelism.

8. The method according to claim 1, characterized in that: The state data includes intermediate tensor results and task execution indexes.

Images