A multi-architecture heterogeneous computing power unified scheduling method and system

Abstract

The application provides a multi-architecture heterogeneous computing power unified scheduling method and system, which comprises the following steps: collecting computing power node registration applications and reading hardware bottom feature information; performing standardized conversion and generating a global unique identifier record to a trusted computing power resource directory; realizing operator mapping through an instruction set conversion adaptation layer; performing multi-granularity load sampling, identifying idle computing power fragments and adding state labels, and constructing a global computing power resource pool; receiving a task request and constructing an attribute vector; constructing a correlation graph according to a detachable identifier and performing sub-task aggregation; adopting a two-stage hierarchical scheduling strategy of preliminary selection and selection to determine an optimal allocation scheme; adopting a bidding competition mechanism to realize distributed matching of a task group and a node and issue an instruction; monitoring the load in real time and releasing the resource after the task is completed; and storing the scheduling log in a distributed ledger. The application realizes heterogeneous computing power adaptation access without modification, automatic recycling and reuse of fragmented resources, and millisecond-level intelligent scheduling.

Description

Technical Field

[0001] This invention belongs to the technical field of computing power networks and heterogeneous resource scheduling, specifically relating to a unified scheduling method and system for multi-architecture heterogeneous computing power. Background Technology

[0002] With the explosive growth of applications such as large-scale AI model training, real-time inference services, and scientific computing, computing power demand is exhibiting significant characteristics of large scale, diversity, and high dynamism. However, the current management and scheduling of computing resources faces a series of deep-seated technical bottlenecks: First, there is the deep challenge of adapting to heterogeneous processor architectures. The underlying instruction sets, computing libraries, and runtime environments of graphics processors, neural network processors, and other hardware from different vendors are incompatible. Most existing solutions only adapt to a single processor architecture. To connect to various types of heterogeneous computing devices, customized modifications to the driver and operator layers must be performed for hardware from different vendors, resulting in long implementation cycles and high costs. Some solutions use containerization for coarse-grained isolation, but this cannot resolve the differences in instruction sets at the operator level, leading to severe performance degradation during cross-architecture scheduling.

[0003] Second, there is a lack of siloed computing resources and a global resource view. Computing resources are scattered across data centers in different regions and at different levels, with each data center operating independently, lacking a unified resource abstraction model and a global discovery mechanism. Traditional solutions use centralized databases to record resource information, but they cannot perceive the dynamic load changes of each node and the fragmented resource status in real time, resulting in generally low resource utilization.

[0004] Third, there is a lack of mechanisms for sensing and reclaiming discrete idle computing resources. A large amount of computing resources are idle or underutilized during nighttime, holidays, and task breaks. Existing monitoring solutions only collect coarse-grained average load indicators and cannot identify short-term idle windows at the millisecond or minute level, resulting in the waste of a large number of reclaimable discrete computing fragments.

[0005] Fourth, the scheduling complexity of heterogeneous tasks of various types. Real-world computing tasks exhibit distinct multi-type characteristics: some tasks possess strong atomicity and must be executed entirely on a single node; others contain numerous parallelizable subtasks, with multi-dimensional constraints such as data associations, control flow associations, and resource contention among these subtasks. Existing scheduling algorithms employ a uniform scheduling strategy for all tasks, failing to perform differentiated processing based on the inherent structural characteristics of each task.

[0006] Fifth, scheduling algorithms struggle to balance global optimization and real-time performance. While traditional mathematical programming-based scheduling algorithms can guarantee solution quality, their solution time increases exponentially in large-scale scenarios; and greedy and heuristic algorithms, though fast, are prone to getting trapped in local optima.

[0007] Sixth, the cross-domain scheduling link lacks a trusted auditing mechanism. Computing resources involve cross-regional and cross-ownership scheduling, and trust barriers exist between different participants. Centralized logs can be tampered with and are difficult to trace.

[0008] To address the aforementioned issues, there is an urgent need for an integrated technical solution that can systematically solve the problems of registering heterogeneous computing power with various architectures, recycling fragmented resources, differentiating scheduling of multiple types of tasks, and reliable evidence storage. Summary of the Invention

[0009] To address the aforementioned technical challenges, this invention proposes a unified scheduling method and system for heterogeneous computing power across multiple architectures. It achieves automated conversion and operator mapping across processor instruction sets via an instruction set conversion and adaptation layer, adapting to mainstream heterogeneous processors from multiple vendors without requiring hardware driver modifications. A global computing power resource catalog based on a unified resource feature vector is constructed, enabling unified abstraction and real-time perception of computing power resources across regions, levels, and ownership. Through periodic load sampling and fragmentation identification algorithms at various time scales, it automatically detects and reclaims discrete idle computing power fragments ranging from milliseconds to minutes. A multi-dimensional task association graph is established, classifying tasks as indivisible or divisible to achieve differentiated hierarchical scheduling strategies. A hierarchical scheduling architecture integrating adaptive learning and bidding competition mechanisms achieves near-globally optimal millisecond-level scheduling decisions in large-scale scenarios. Distributed ledger technology enables tamper-proof evidence storage of end-to-end scheduling behavior, supporting trusted settlement among multiple participants.

[0010] In a first aspect, the present invention provides the following technical solution: a unified scheduling method for heterogeneous computing power across multiple architectures, comprising: S1. The registration application of each computing power node is collected through the computing power access gateway deployed in front of each computing power node in each region, and the underlying hardware feature information of each computing power node is read. The underlying hardware feature information includes at least processor type, peak computing power, storage capacity and equipment manufacturer information. S2. Perform a standardized transformation on the underlying hardware feature information according to the unified computing power resource abstraction specification, generate a globally unique computing power resource identifier and record it in the trusted computing power resource catalog; S3. Perform automatic identification and low-level operator mapping on the native instruction set of different types of processors; perform multi-granularity time-domain load sampling on the connected computing power nodes, identify discrete idle computing power fragments based on load fluctuation characteristics and attach idle status labels, and uniformly store standardized metadata, global unique identifiers and idle status labels in the trusted computing power resource catalog to build a global unified computing power resource pool. S4. Receive the computing task request submitted by the user, parse the task attributes and construct a task attribute vector. The task attribute vector includes at least the computing resource requirement, storage resource requirement, bandwidth resource requirement, energy consumption requirement, task deadline and task priority weight. S5. Based on the task splittable identifier in the task attribute vector, the task is divided into indivisible task and splittable task. For splittable task, a multi-dimensional relationship graph is constructed and subtask aggregation is performed to group each subtask into several task groups. S6. Invoke the hierarchical scheduling strategy, and determine the optimal resource allocation scheme with the computing power resource pool and the task group as input; for the scalable task group determined to be allocated to multiple heterogeneous computing power nodes in the optimal resource allocation scheme, adopt a bidding competition mechanism to realize the distributed matching between the task group and the computing power node, generate scheduling instructions and send them to the corresponding computing power access gateway for execution. S7. Monitor the load status of computing nodes in real time during task execution, release the occupied computing resources and update the computing resource pool after the task is completed, and record the scheduling decision and execution process log in a distributed ledger storage method.

[0011] Compared with the prior art, the beneficial effects of the present invention are as follows: First, in terms of adapting to heterogeneous computing power across multiple architectures, this invention uses a computing power access gateway to uniformly collect registration applications, a hardware detection module to automatically read underlying hardware feature information, and an instruction set conversion adaptation layer to achieve automated operator mapping from the source instruction set to the target instruction set. This allows for adaptation to heterogeneous processors from multiple vendors without modifying hardware drivers, significantly reducing the implementation cycle and development cost of cross-architecture deployments.

[0012] Secondly, in terms of improving the utilization rate of computing resources, this invention performs periodic load sampling on connected computing nodes at three time scales: millisecond, second, and minute. It accumulates and quantifies the idle computing power during consecutive low-load periods into quantifiable fragment capacity values ​​and attaches structured status labels, making fragmented resources effective resources that can be perceived and retrieved by the scheduling system. When there are insufficient candidate nodes, multiple fragments are automatically aggregated into virtual nodes for scheduling, realizing the automated perception, recycling, and reuse of idle computing power, and significantly improving the overall utilization rate of existing computing resources.

[0013] Third, in terms of scheduling real-time performance and decision quality, this invention adopts a two-level serial scheduling architecture of initial selection and final selection. In the initial selection stage, candidate nodes are quickly screened through five-dimensional basic resource constraints to significantly reduce the search space. In the final selection stage, a comprehensive utility function is established that includes high-priority task guarantees, total latency minimization, and node load balancing. A flexible executor-judge algorithm is used to achieve millisecond-level near-global optimal scheduling decisions. Under the premise of ensuring the quality of the scheduling scheme, the real-time requirements of large-scale cross-domain computing power scheduling are fully met.

[0014] Fourth, in terms of collaborative scheduling of multi-type heterogeneous tasks, this invention constructs a directed acyclic graph for divisible tasks, which includes three dimensions: data association, control flow association, and resource competition association. The strongly associated subtasks are aggregated into task groups through spectral clustering algorithm. In the scheduling and execution stage, a bidding competition mechanism is adopted to realize the distributed matching between task groups and computing power nodes. Each node makes independent decisions based on a four-dimensional comprehensive bid considering computing time, transmission time, resource conflicts, and communication delay, which effectively solves the single-point decision bottleneck of centralized schedulers in large-scale distributed scenarios.

[0015] In summary, this invention systematically solves several existing technical bottlenecks through a complete integrated design encompassing unified hardware feature acquisition and abstraction, instruction set cross-architecture mapping, multi-granularity load sampling and fragment identification, multi-dimensional task association graphs and spectral clustering decomposition, hierarchical scheduling and reinforcement learning optimization, distributed matching for bidding competition, and distributed ledger storage. It provides a complete technical solution for the unified management and intelligent scheduling of large-scale wide-area heterogeneous computing networks.

[0016] Preferably, in step S2, the unified computing resource abstraction specification includes: For computing nodes Its resource feature vector is represented as: ; In the formula, For nodes Peak computing power For peak storage capacity, Peak available bandwidth, Peak available energy consumption For nodes Transmission delay, For nodes The processor instruction set type identifier; peak computing power Divided into central processing unit computing power and graphics processing power For nodes equipped only with a central processing unit, For nodes equipped with graphics processors, .

[0017] Preferably, in step S3, the step of automatically identifying and mapping the native instruction sets of different types of processors to low-level operators includes: Establish instruction set mapping relationship For the source processor instruction set Operators in ,Will Convert to a hardware-independent intermediate representation Then, through the target platform's backend compilation component, Generate as target instruction set Equivalent operator sequence on The equivalent operator sequence and They are functionally equivalent and yield the same calculation results. For operator sequences that are called more than a set threshold within a time window, a transformation cache is established. The conversion result is stored. When called later, the conversion cache is retrieved first. If a match is found, the cached target code is returned directly without triggering an immediate conversion. If a match is not found, an immediate conversion is performed and the result is written to the cache for later reuse.

[0018] Preferably, in step S3, the step of performing multi-granularity time-domain load sampling on the connected computing power nodes, identifying discretized idle computing power fragments based on load fluctuation characteristics, and attaching idle status tags includes: Periodic load sampling is performed on the connected computing power nodes using multiple time scales, including millisecond-level, second-level, and minute-level scales; for computing power nodes... During the time window Internal load sequence acquisition ,in, For the first Load rate of each sampling point ; If continuous at any scale The load rate of each sampling point is lower than the preset threshold. If so, the time period is determined to be a discretized fragment of idle computing power, and its available computing capacity is... for: ; In the formula, and These are the start and end indices of the consecutive low-load sampling points, respectively. For nodes Peak computing power; Add a state tag to each discretized idle computing power fragment, and the state tag includes at least: the start time. End time Available computing capacity Node identifier and processor type The status tag is serialized and stored in the fragment index table of the trusted computing power resource directory.

[0019] Preferably, in step S5, the step of constructing the task attribute vector includes: Task attribute vector Defined as: ; In the formula, To calculate resource requirements, Storage resource requirements, This refers to the bandwidth resource requirements. Energy consumption demand, The deadline for the task. This is a task-separable identifier. It is an indivisible task. For scalable tasks, As a task priority weight; The computing resource requirements Divided into central processing unit computing requirements and graphics processor computing requirements ,Task At the node On processing delay The calculation is as follows: ; In the formula, For the task The computational demands on the central processing unit. For the task The computational demands on graphics processing units (GPUs) For nodes The computing power of the central processing unit, For nodes The graphics processing unit's computing power; for tasks requiring only the central processing unit, For tasks that only require a graphics processor, ; priority weight Based on the urgency of the task Importance and waiting time Dynamic calculation:

[0020] in, , The first, second, and third weights are respectively. , , These represent the maximum values ​​of urgency, importance, and waiting time among all currently pending tasks. Introducing a time decay function Update effective priority ,in, These are the parameters for controlling the decay rate.

[0021] Preferably, in step S6, the step of constructing a multi-dimensional relationship graph for the separable tasks and performing subtask aggregation to group each subtask into several task groups includes: For tasks that can be decomposed, break them down into several atomic subtasks. ,in, The computational cost of the atomic subtask. The amount of data for atomic subtasks. Tolerable delay for atomic sub-tasks; Construct a directed acyclic graph , where the vertex set Represents the set of atomic subtasks, each vertex Corresponding atomic subtasks edge set Represents the multi-dimensional relationships between atomic subtasks, edges Composite weight for: ; In the formula, , The first, second, and third association weights are respectively. For data association weights, To control flow association weights, For resource competition-related weights; Constructing the weight matrix ,in Construct the degree matrix Calculate the normalized Laplace matrix Perform eigenvalue decomposition on the normalized Laplace matrix and select the first... The eigenvectors corresponding to the smallest non-zero eigenvalues ​​constitute the feature matrix. Each atomic subtask is mapped to a point in a low-dimensional feature space; Perform resource-constrained operations in a low-dimensional feature space. - Mean aggregation divides atomic subtasks into Task groups Each task group The following hard resource constraints must be met: ; In the formula, This refers to the upper limit of the central processing unit's computing power for a single computing node. This is the maximum memory capacity. Maximum delay; If, during the aggregation process, none of the current groups can accommodate a certain subtask without violating hard resource constraints, a new task group is created.

[0022] Preferably, in step S6, the hierarchical scheduling strategy includes: Preliminary selection stage: based on effective priority All tasks to be scheduled are sorted in descending order, and candidate computing nodes are selected for each task in turn; for each task... Iterate through the actual computing power nodes and virtual nodes formed by the aggregation of discrete idle computing power fragments in the global resource pool, and select nodes that simultaneously meet the following basic resource constraints to add to the candidate set. : ; In the formula, For nodes Peak computing power For the task The computing resource requirements, For peak storage capacity, Storage resource requirements, Peak available bandwidth, This refers to the bandwidth resource requirements. Peak available energy consumption Energy consumption demand, For nodes The remaining service time, The estimated execution time of the task on the node; If candidate set If the table is empty, the fragment index table in the trusted computing power resource directory is retrieved to determine whether there are aggregable discretized idle computing power fragments; if there are aggregable discretized idle computing power fragments, multiple fragments are aggregated into virtual nodes. The virtual node is then added to the candidate set, and its available computing capacity is the sum of the capacities of each fragment. ,in, For a collection of fragments, For the first The available computing capacity of each fragment; if it is still empty, the task enters the waiting queue and re-participates in the initial selection in the next scheduling cycle; Selection phase: The candidate set output from the initial selection phase As input, define decision variables , Indicates task Assigned to computing nodes Otherwise, the value is 0; an optimization model is established with the goal of maximizing the overall system utility: ; In the formula, For the task The effective priority, For nodes Peak computing power For the task To the node Communication delay, The estimated execution time of the task on the node. The computing resources already occupied by the node. These are the first and second weighting coefficients, respectively. For a set of tasks, For a set of computing power nodes, To optimize the function values ​​of the model; The constraints of the optimization model include: Computational capability constraints: ; Storage capacity constraints: ; Bandwidth constraints: ; Energy consumption constraints: ; Delay constraints: ; Task uniqueness constraint: ; In the formula, To calculate resource requirements, Storage resource requirements, This refers to the bandwidth resource requirements. Energy consumption demand, The deadline for the task. Peak available bandwidth, Peak available energy consumption This represents the peak storage capacity.

[0023] Preferably, the solution process of the optimization model includes: The computational resource scheduling problem of the optimization model is modeled as a Markov decision process. : state space : The task As a scheduling moment, at the scheduling moment state Includes remaining resource information and task queue information for each computing node: ; In the formula, For nodes Remaining computing power For the remaining storage capacity, For the remaining bandwidth, For the remaining energy consumption budget, The total number of computing nodes. It is the set of all task attribute vectors in the current waiting queue; Action space :action The resource allocation decision vector for the current task on each node; reward function for: ; In the formula, For state , The function value below, The entropy temperature coefficient For strategy In state Entropy below; Policy Network and value network Through the following objective function Joint optimization: ; In the formula, For strategy Induced trajectory distribution, Indicates adherence to the strategy Calculate the mathematical expectation of the resulting state-action trajectory; The value network employs a dual-Q network structure and updates by minimizing the soft Bellman residual: ; In the formula, Let the value network loss function be... For experience replay buffer, As a discount factor, For value networks The predicted cumulative reward for the current state and actions. In order to schedule Execute action The instant reward value returned later. The estimated state value for the target value network. This represents the mathematical expectation of the next state. This means randomly sampling a small batch of samples from the experience replay buffer and calculating the expected value. The policy network updates by minimizing the KL divergence: ; In the formula, Let the policy network loss function be... This represents the mathematical expectation of the action distribution output by the policy network. This indicates that the state is randomly sampled from the experience playback buffer. Find the expected value. For the current policy network in state Select action The probability; employing reparameterization techniques. Make the gradient propagable. It is Gaussian noise. This is a reparameterized sampling function; Network parameters adopt a soft update strategy: ; In the formula, This is the soft update coefficient. These are the parameters of the current value network and the current policy network, respectively. These are the parameters for the target value network and the target policy network, respectively.

[0024] Preferably, in step S6, the bidding competition mechanism includes: Taking the divisible task groups involving multiple heterogeneous computing nodes in the optimal resource allocation scheme as input, each task group... Each candidate computing node participates in resource bidding as a bidding unit. As the bidding party; Task Group For candidate computing power nodes Bid price The calculation is as follows: ; In the formula, For nodes Peak computing power For task groups China Atomic Sub-mission The computational load, For atomic sub-tasks The amount of data, For users to nodes Data transmission rate, This represents the resource conflict penalty coefficient. This is a resource conflict indicator function, when a node... Assigned task groups and current task groups If resource contention exists, use 1; otherwise, use 0. This is the communication delay penalty coefficient. For task groups To the node Communication delay; Each computing node selects the task group with the highest bid from all received bids for matching, and then updates the node's remaining resource status after matching: ; In the formula, For nodes The remaining computing power For nodes The remaining storage capacity; The bidding-allocation process is repeated until all task groups are allocated or all computing power node resources are exhausted; if there are unallocated task groups, the backfilling strategy is initiated, and the unallocated task groups are inserted into the waiting queue of the next scheduling cycle with lower priority.

[0025] Secondly, the present invention provides the following technical solution: a unified scheduling system for heterogeneous computing power across multiple architectures, the system comprising: The reading module is used to receive the registration application of each computing power node through the computing power access gateway deployed in front of each computing power node in each region, and to read the underlying hardware feature information of each computing power node. The underlying hardware feature information includes at least processor type, peak computing power, storage capacity and equipment manufacturer information. The conversion module is used to perform standardized conversion on the underlying hardware feature information according to the unified computing power resource abstraction specification, generate a globally unique computing power resource identifier and record it in the trusted computing power resource catalog. The mapping module is used to automatically identify and map the native instruction sets of different types of processors to low-level operators; to perform multi-granularity time-domain load sampling on connected computing power nodes; to identify discrete idle computing power fragments based on load fluctuation characteristics and attach idle status tags; and to uniformly store standardized metadata, globally unique identifiers and idle status tags in the trusted computing power resource directory to build a globally unified computing power resource pool. The parsing module is used to receive computing task requests submitted by users, parse task attributes and construct a task attribute vector. The task attribute vector includes at least the computing resource requirements, storage resource requirements, bandwidth resource requirements, energy consumption requirements, task deadline and task priority weight. The splitting module is used to classify tasks into indivisible tasks and divisible tasks according to the task splitting identifier in the task attribute vector. For divisible tasks, a multi-dimensional relationship graph is constructed and subtask aggregation is performed to group each subtask into several task groups. The scheduling module is used to invoke the hierarchical scheduling strategy, and determine the optimal resource allocation scheme with the computing power resource pool and the task group as input; for the divisible task group determined to be allocated to multiple heterogeneous computing power nodes in the optimal resource allocation scheme, a bidding competition mechanism is adopted to realize the distributed matching between the task group and the computing power node, generate scheduling instructions and send them to the corresponding computing power access gateway for execution; The evidence storage module is used to monitor the load status of computing nodes in real time during task execution, release the occupied computing resources and update the computing resource pool after the task is completed, and record the scheduling decision and execution process log in a distributed ledger evidence storage method.

[0026] Thirdly, the present invention provides the following technical solution: a computer, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the multi-architecture heterogeneous computing power unified scheduling method as described above.

[0027] Fourthly, the present invention provides the following technical solution: a storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the above-described unified scheduling method for multi-architecture heterogeneous computing power. Attached Figure Description

[0028] To more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0029] Figure 1 This is a flowchart of the multi-architecture heterogeneous computing power unified scheduling method provided in Embodiment 1 of the present invention; Figure 2 This is a structural block diagram of the multi-architecture heterogeneous computing power unified scheduling system provided in Embodiment 2 of the present invention; Figure 3 This is a schematic diagram of the hardware structure of a computer provided for another embodiment of the present invention.

[0030] The embodiments of the present invention will be further described below with reference to the accompanying drawings. Detailed Implementation

[0031] Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain embodiments of the present invention, and should not be construed as limiting the present invention.

[0032] Example 1 In Embodiment 1 of the present invention, as Figure 1 As shown, a unified scheduling method for heterogeneous computing power across multiple architectures includes: S1. The registration application of each computing power node is collected through the computing power access gateway deployed in front of each computing power node in each region, and the underlying hardware feature information of each computing power node is read. The underlying hardware feature information includes at least processor type, peak computing power, storage capacity and equipment manufacturer information. Specifically, in step S1, a lightweight agent program is first deployed on the computing power node side of each region. This agent program is responsible for establishing a communication connection with the computing power access gateway at the front end of the provincial platform. During the registration process, the agent program submits a registration data packet containing the node's identity certificate, geographical location tag, and network address information to the gateway. After verifying the validity of the certificate, the gateway establishes a long-lived connection session for the node and assigns a session token to ensure the legality and continuity of subsequent communication.

[0033] After registration and authentication are completed, the hardware detection module sends query commands to the device drivers of each node by calling the native management interfaces provided by various hardware vendors (such as the NVIDIA management library, Huawei Ascend computing language, and AMD open computing platform), automatically collecting low-level hardware characteristic information. During the collection process, the system dynamically selects the corresponding acquisition driver based on the node type, and sequentially obtains the processor type name, peak computing power (in GFLOPS), storage capacity (in MB), available bandwidth (in Mbps), and rated power consumption (in W) through interface functions, while recording the device manufacturer information. After each parameter is returned through the standardized interface, it is encapsulated into a structured hardware characteristic description file after format validation and outlier filtering, and temporarily stored in a message queue in a structured data format. This description file serves as the output data of step S1, for use in the next stage of the standardization conversion process.

[0034] By deploying standardized agent programs and calling the vendor's native management interface, the entire process of computing node registration and hardware feature collection is automated. This method eliminates the inaccuracies and inefficiencies caused by manual data entry, while ensuring the authentication of each node and the security of communication. The collected hardware feature information is comprehensive, covering all the basic parameters required for subsequent scheduling decisions, providing a reliable data foundation for unified resource abstraction.

[0035] S2. Perform a standardized transformation on the underlying hardware feature information according to the unified computing power resource abstraction specification, generate a globally unique computing power resource identifier and record it in the trusted computing power resource catalog; Specifically, step S2 defines a unified abstract specification for computing power resources. This specification defines the complete structure of the standard attribute set for computing power nodes, including attribute fields in six core dimensions, and clarifies the data type, legal value range, and unit of measurement for each field. Based on this specification, the system maps the hardware feature information of each node output in step S1 to the standard attribute fields one by one.

[0036] For computing nodes Its resource feature vector is represented as: ; In the formula, For nodes Peak computing power, measured in GFLOPS, represents the maximum number of floating-point operations that the node can perform per second. Peak storage capacity, in MB, represents the maximum available memory space for this node. Peak available bandwidth, measured in Mbps, represents the maximum data transfer rate between this node and the external network. Peak available power consumption, in watts (W), represents the maximum power consumption of this node during normal operation. For nodes The transmission delay, measured in milliseconds, is obtained through inter-node network probing. For nodes The processor instruction set type identifier is used to identify the processor architecture type (such as NVPTX, CANN, ROCm, etc.). peak computing power Divided into central processing unit computing power and graphics processing power For nodes equipped only with a central processing unit, For nodes equipped with graphics processors, .

[0037] After completing the attribute mapping, the system generates a globally unique computing resource identifier for each node. The identifier uses a unified encoding format, embedding information at three levels: region code, chip type code, and serial number, for example, urn:cn:compute:guangzhou:nvidia-a100:001. This ensures that identifiers from different regions and types of nodes do not conflict, and that the basic attribution information of the node can be parsed from the identifier. The converted resource feature vector and the generated identifier are written to the trusted computing resource directory. This directory adopts a hybrid storage architecture, establishing a primary index with the identifier as the primary key, and secondary indexes based on chip type, geographical location, and remaining computing power range, supporting efficient retrieval in subsequent scheduling steps.

[0038] By using a unified abstraction standard, heterogeneous hardware characteristics from different vendors and architectures are transformed into standardized attribute vector representations, eliminating differences at the hardware description level and making resources across different nodes comparable. This provides a unified quantitative basis for cross-architecture scheduling decisions. A globally unique identifier ensures the global identifiability of each node and the traceability of scheduling behavior. A hybrid index structure guarantees rapid retrieval of resource information in high-concurrency scheduling scenarios.

[0039] S3. Perform automatic identification and low-level operator mapping on the native instruction set of different types of processors; perform multi-granularity time-domain load sampling on the connected computing power nodes, identify discrete idle computing power fragments based on load fluctuation characteristics and attach idle status labels, and uniformly store standardized metadata, global unique identifiers and idle status labels in the trusted computing power resource catalog to build a global unified computing power resource pool. Specifically, this step achieves automated adaptation of the native instruction sets of various processor architectures through an instruction set conversion and adaptation layer. The adaptation layer consists of three core sub-components: a front-end parser, an intermediate representation converter, and a back-end code generator.

[0040] Establish instruction set mapping relationship For the source processor instruction set Operators in ,Will Convert to a hardware-independent intermediate representation Then, through the target platform's backend compilation component, Generate as target instruction set Equivalent operator sequence on The equivalent operator sequence and They are functionally equivalent and yield the same calculation results. For operator sequences that are called more than a set threshold within a time window, a transformation cache is established. The conversion result is stored. When called later, the conversion cache is retrieved first. If a match is found, the cached target code is returned directly without triggering an immediate conversion. If a match is not found, an immediate conversion is performed and the result is written to the cache for later reuse.

[0041] That is, the front-end parser interprets the source processor instruction set. Operators in Lexical and syntactic analysis are performed to extract meta-information such as operator types, parameter lists, and return value types, and an abstract syntax tree is constructed. An intermediate representation converter then transforms this abstract syntax tree into a hardware-platform-independent intermediate representation. The intermediate representation adopts the underlying virtual machine intermediate representation format, which is platform independent and offers rich optimization support. The conversion process includes three sub-steps: type mapping, control flow conversion, and data flow rearrangement.

[0042] The backend code generator depends on the target instruction set type. Select the corresponding code generation template and... Generate an equivalent sequence of operators on the target instruction set. The equivalent operator sequence and Functionally equivalent and yielding consistent computational results, the system maintains an instruction set mapping library, recording the mapping rules for various operators across different platforms. Specific mapping rules include: matrix multiplication operators are mapped to basic linear algebra subroutine library functions corresponding to each platform; convolution operators are mapped to deep learning library functions corresponding to each platform.

[0043] For operator sequences that are called more than a set threshold within a time window, a transformation cache is established. The transformation results are stored. The cache uses a Least Recently Used (LRU) eviction policy, and its capacity is set to a configurable value. Subsequent calls will first retrieve the transformation cache. If a cached target code is found, it is returned directly without triggering an immediate transformation. If a cached target code is not found, an immediate transformation is performed, and the result is written to the cache for later reuse.

[0044] A standardized operator mapping mechanism enables automatic instruction conversion between different processor architectures, allowing upper-layer applications to run on multiple architectures without needing to write adaptation code for specific hardware. This significantly reduces the technical costs and development cycle of cross-architecture deployment. The mapping rule base covers core operators in the deep learning field, ensuring functional completeness. The introduction of a conversion cache avoids the computational overhead of repeated conversions, greatly improving runtime performance under real-world workloads.

[0045] For multi-granularity time-domain load sampling and fragmented resource identification, periodic load data collection is performed on the connected computing nodes at three independent time scales. These multiple time scales include millisecond, second, and minute scales: the millisecond sampling channel collects the instantaneous load value of the node at a fixed short period (e.g., 100ms) to capture rapid fluctuations and transient spikes in the load; the second sampling channel collects load data at a medium period (e.g., 1s) to analyze short-term load trends; and the minute sampling channel collects load data at a long period (e.g., 60s) to identify long-term regular changes in the load, such as diurnal cycles.

[0046] At each sampling time, the system calculates the overall load rate based on the node type: for nodes equipped with a graphics processor, the load rate is the weighted average of the graphics processor utilization and video memory utilization, with weighting coefficients of 0.7 and 0.3, respectively; for nodes equipped only with a central processing unit (CPU), the load rate is the weighted average of the CPU utilization and memory utilization, with weighting coefficients of 0.6 and 0.4, respectively. The sampled data is stored in the local cache of each node in time-series format.

[0047] Periodic load sampling is performed on the connected computing power nodes using multiple time scales, including millisecond-level, second-level, and minute-level scales; for computing power nodes... During the time window Internal load sequence acquisition ,in, For the first Load rate of each sampling point ; If continuous at any scale The load rate of each sampling point is lower than the preset threshold. If so, the time period is determined to be a discretized fragment of idle computing power, and its available computing capacity is... for: ; In the formula, and These are the start and end indices of the consecutive low-load sampling points, respectively. For nodes Peak computing power; Add a state tag to each discretized idle computing power fragment, and the state tag includes at least: the start time. End time Available computing capacity Node identifier and processor type The status tag is serialized and stored in the fragment index table of the trusted computing power resource directory.

[0048] By employing three sampling channels with different granularities, both millisecond-level instantaneous idle time and minute-level periodic idle time can be captured simultaneously, avoiding the missed detection problems caused by single-granularity sampling. By quantifying idle computing power within continuous low-load time periods into specific numerical indicators, fragmented resources become effective resources that can be perceived, retrieved, and utilized by the scheduling system. The tagged storage method provides structured data support for subsequent resource aggregation queries. S4. Receive the computing task request submitted by the user, parse the task attributes and construct a task attribute vector. The task attribute vector includes at least the computing resource requirement, storage resource requirement, bandwidth resource requirement, energy consumption requirement, task deadline and task priority weight. Specifically, the system receives user-submitted computation task requests through a standardized API interface. The request data is in JSON structured format and includes fields such as task type identifier, code mirror address, resource requirement specifications, expected execution duration, and deadline. The task parsing module performs format and integrity verification on the request data, extracts key attribute fields, and converts them into a standardized task attribute vector.

[0049] The steps to construct a task attribute vector include: Task attribute vector Defined as: ; In the formula, To calculate resource requirements, Storage resource requirements, This refers to the bandwidth resource requirements. Energy consumption demand, The deadline for the task. This is a task-separable identifier. It is an indivisible task. For scalable tasks, As a task priority weight; The computing resource requirements Divided into central processing unit computing requirements and graphics processor computing requirements ,Task At the node On processing delay The calculation is as follows: ; In the formula, For the task The computational demands on the central processing unit. For the task The computational demands on graphics processing units (GPUs) For nodes The computing power of the central processing unit, For nodes The graphics processing unit's computing power; for tasks requiring only the central processing unit, For tasks that only require a graphics processor, ; Specifically, the formula means that the total processing latency of a task on a node is limited by the longer completion time of the CPU or GPU portion, and the maximum value of the two is taken.

[0050] priority weight Based on the urgency of the task Importance and waiting time Dynamic calculation:

[0051] in, , The first, second, and third weights are respectively. , , These represent the maximum values ​​of urgency, importance, and waiting time among all currently pending tasks. The formula means that the priority is obtained by weighted summation of the normalized values ​​of three dimensions: urgency, importance, and waiting time.

[0052] Introducing a time decay function Update effective priority ,in, These are the parameters for controlling the decay rate.

[0053] To further prevent low-priority tasks from being "starved" due to long waiting times, a time decay function is introduced. This mechanism ensures that the longer a task waits, the higher its effective priority, thus gaining a higher priority in subsequent scheduling.

[0054] By standardizing attribute vectors, diverse user-submitted task requests are transformed into unified quantitative descriptions, making different types of tasks comparable within the same framework. Separate descriptions of CPU and GPU requirements support more granular heterogeneous resource matching. Dynamic priority calculation comprehensively considers three dimensions: urgency, importance, and waiting time, and uses a time decay mechanism to prevent low-priority tasks from waiting for extended periods, improving scheduling fairness and task completion rates.

[0055] S5. Based on the task splittable identifier in the task attribute vector, the task is divided into indivisible task and splittable task. For splittable task, a multi-dimensional relationship graph is constructed and subtask aggregation is performed to group each subtask into several task groups. Specifically, the identifier can be split according to the task. Classify and process the tasks to be scheduled. If This task is identified as an indivisible task and enters the subsequent scheduling process directly as a whole scheduling unit, without performing the splitting and aggregation operations in this step. If This task is identified as a scalable task, and the following subtask splitting and aggregation process is executed: For tasks that can be decomposed, break them down into several atomic subtasks. ,in, The computational cost of the atomic subtask. The amount of data for atomic subtasks. Tolerable delay for atomic sub-tasks; Construct a directed acyclic graph , where the vertex set Represents the set of atomic subtasks, each vertex Corresponding atomic subtasks edge set Represents the multi-dimensional relationships between atomic subtasks, edges Composite weight for: ; In the formula, , The first, second, and third association weights are respectively. The data association weights are determined by subtasks. sub-tasks The normalized value of the amount of communication data between them reflects the density of data transmission. The control flow association weights are set to 1 if there is a sequential execution constraint, and 0 otherwise, reflecting the mandatory dependency relationship of the execution order. The resource competition correlation weights represent the intensity of competition among subtasks for shared hardware resources (such as GPU memory and computing cores), calculated based on the degree of overlap in the resource usage patterns of the subtasks. The values ​​of the three weight coefficients are dynamically adjusted according to the task type to adapt to the needs of different application scenarios; Constructing the weight matrix ,in Construct the degree matrix Calculate the normalized Laplace matrix Perform eigenvalue decomposition on the normalized Laplace matrix and select the first... The eigenvectors corresponding to the smallest non-zero eigenvalues ​​constitute the feature matrix. Each atomic subtask is mapped to a point in a low-dimensional feature space; Perform resource-constrained operations in a low-dimensional feature space. - Mean aggregation divides atomic subtasks into Task groups Each task group The following hard resource constraints must be met: ; In the formula, This refers to the upper limit of the central processing unit's computing power for a single computing node. This is the maximum memory capacity. Maximum delay; If, during the aggregation process, none of the current groups can accommodate a certain subtask without violating hard resource constraints, a new task group is created.

[0056] By decomposing large-scale complex tasks into independently schedulable subtask groups, the decision-making dimension of the scheduling problem is reduced from the task level to the group level, thus lowering the computational complexity of a single scheduling operation. Spectral clustering algorithms can effectively identify the inherent relationships between subtasks, aggregating subtasks with strong data dependencies and resource contention into the same group, reducing cross-node communication overhead and the probability of resource conflicts. The introduction of hard resource constraints ensures that each task group can be executed independently on any candidate node, improving the feasibility of the scheduling scheme.

[0057] S6. Invoke the hierarchical scheduling strategy, and determine the optimal resource allocation scheme with the computing power resource pool and the task group as input; for the scalable task group determined to be allocated to multiple heterogeneous computing power nodes in the optimal resource allocation scheme, adopt a bidding competition mechanism to realize the distributed matching between the task group and the computing power node, generate scheduling instructions and send them to the corresponding computing power access gateway for execution. Specifically, a two-level scheduling architecture is adopted here: an initial selection phase and a fine selection phase. The two phases are executed sequentially, gradually narrowing the decision-making scope and refining the allocation scheme. Taking a globally unified computing resource pool and task groups (including indivisible tasks) as input, the optimal resource allocation scheme is output.

[0058] Specifically, the hierarchical scheduling strategy includes: Preliminary selection stage: based on effective priority All tasks to be scheduled are sorted in descending order, and candidate computing nodes are selected for each task in turn; for each task... Iterate through the actual computing power nodes and virtual nodes formed by the aggregation of discrete idle computing power fragments in the global resource pool, and select nodes that simultaneously meet the following basic resource constraints to add to the candidate set. : ; In the formula, For nodes Peak computing power For the task The computing resource requirements, For peak storage capacity, Storage resource requirements, Peak available bandwidth, This refers to the bandwidth resource requirements. Peak available energy consumption Energy consumption demand, For nodes The remaining service time, The estimated execution time of the task on the node; If candidate set If the table is empty, the fragment index table in the trusted computing power resource directory is retrieved to determine whether there are aggregable discretized idle computing power fragments; if there are aggregable discretized idle computing power fragments, multiple fragments are aggregated into virtual nodes. The virtual node is then added to the candidate set, and its available computing capacity is the sum of the capacities of each fragment. ,in, For a collection of fragments, For the first The available computing capacity of each fragment; if it is still empty, the task enters the waiting queue and re-participates in the initial selection in the next scheduling cycle; Selection phase: The candidate set output from the initial selection phase As input, define decision variables , Indicates task Assigned to computing nodes Otherwise, the value is 0; an optimization model is established with the goal of maximizing the overall system utility: ; In the formula, For the task The effective priority, For nodes Peak computing power For the task To the node Communication delay, The estimated execution time of the task on the node. The computing resources already occupied by the node. These are the first and second weighting coefficients, respectively. For a set of tasks, For a set of computing power nodes, To optimize the function values ​​of the model; The constraints of the optimization model include: Computational capability constraints: ; Storage capacity constraints: ; Bandwidth constraints: ; Energy consumption constraints: ; Delay constraints: ; Task uniqueness constraint: ; In the formula, To calculate resource requirements, Storage resource requirements, This refers to the bandwidth resource requirements. Energy consumption demand, The deadline for the task. Peak available bandwidth, Peak available energy consumption This represents the peak storage capacity.

[0059] Specifically, the above constraints together ensure the feasibility of the allocation scheme: the first four constraints ensure that the resources of each node do not exceed the limit, the delay constraint ensures that the task is completed before the deadline, and the task uniqueness constraint ensures that each task is allocated to only one node.

[0060] The solution process of the optimization model includes: The computational resource scheduling problem of the optimization model is modeled as a Markov decision process. : state space : The task As a scheduling moment, at the scheduling moment state Includes remaining resource information and task queue information for each computing node: ; In the formula, For nodes Remaining computing power For the remaining storage capacity, For the remaining bandwidth, For the remaining energy consumption budget, The total number of computing nodes. It is the set of all task attribute vectors in the current waiting queue; Action space :action The resource allocation decision vector for the current task on each node; reward function for: ; In the formula, For state , The function value below, The entropy temperature coefficient For strategy In state Entropy below; Policy Network and value network Through the following objective function Joint optimization: ; In the formula, For strategy Induced trajectory distribution, Indicates adherence to the strategy Calculate the mathematical expectation of the resulting state-action trajectory; The value network employs a dual-Q network structure and updates by minimizing the soft Bellman residual: ; In the formula, Let the value network loss function be... For experience replay buffer, As a discount factor, For value networks The predicted cumulative reward for the current state and actions. In order to schedule Execute action The instant reward value returned later. The estimated state value for the target value network. This represents the mathematical expectation of the next state. This means randomly sampling a small batch of samples from the experience replay buffer and calculating the expected value. The policy network updates by minimizing the KL divergence: ; In the formula, Let the policy network loss function be... This represents the mathematical expectation of the action distribution output by the policy network. This indicates that the state is randomly sampled from the experience playback buffer. Find the expected value. For the current policy network in state Select action The probability; employing reparameterization techniques. Make the gradient propagable. It is Gaussian noise. This is a reparameterized sampling function; Network parameters adopt a soft update strategy: ; In the formula, This is the soft update coefficient. These are the parameters of the current value network and the current policy network, respectively. These are the parameters for the target value network and the target policy network, respectively.

[0061] Simultaneously, taking the divisible task groups involving multiple heterogeneous computing nodes in the optimal resource allocation scheme as input, a bidding competition mechanism is adopted to achieve distributed matching between task groups and computing nodes. This mechanism will assign each task group... Treating each candidate computing power node as a single bidding unit As the bidding party, a distributed negotiation process is implemented through multiple rounds of bidding and awarding. The bidding competition mechanism includes: Taking the divisible task groups involving multiple heterogeneous computing nodes in the optimal resource allocation scheme as input, each task group... Each candidate computing node participates in resource bidding as a bidding unit. As the bidding party; Task Group For candidate computing power nodes Bid price The calculation is as follows: ; In the formula, For nodes Peak computing power For task groups China Atomic Sub-mission The computational load, For atomic sub-tasks The amount of data, For users to nodes Data transmission rate, This represents the resource conflict penalty coefficient. This is a resource conflict indicator function, when a node... Assigned task groups and current task groups If resource contention exists, use 1; otherwise, use 0. This is the communication delay penalty coefficient. For task groups To the node Communication delay; Each computing node selects the task group with the highest bid from all received bids for matching, and then updates the node's remaining resource status after matching: ; In the formula, For nodes The remaining computing power For nodes The remaining storage capacity; The bidding-allocation process is repeated until all task groups are allocated or all computing power node resources are exhausted; if there are unallocated task groups, the backfilling strategy is initiated, and the unallocated task groups are inserted into the waiting queue of the next scheduling cycle with lower priority.

[0062] Specifically, after matching is completed, a scheduling instruction is generated. The scheduling instruction includes: task group identifier, target node identifier, resource allocation (computation, storage, bandwidth, energy consumption), execution order number, data dependencies (which prerequisite tasks need to be completed), and startup parameters. The scheduling instruction is digitally signed and then sent to the corresponding computing power access gateway for execution.

[0063] A distributed matching system between task groups and nodes is achieved through a bidding competition mechanism, avoiding the computational bottleneck of a centralized scheduler's single-point decision-making. Each node can make independent decisions based on its own resource status, improving the system's scalability and robustness. The bidding formula comprehensively considers four dimensions: computation time, transmission time, resource conflicts, and communication latency, making the matching result close to the global optimum. The backfilling strategy ensures that unassigned task groups are not lost but are postponed to the next cycle for further attempts.

[0064] S7. Monitor the load status of computing nodes in real time during task execution, release the occupied computing resources and update the computing resource pool after the task is completed, and record the scheduling decision and execution process log in a distributed ledger storage method.

[0065] Specifically, during task execution, the load monitoring module continuously collects load status indicators for each computing node at preset intervals (e.g., 1 second). For graphics processor nodes, the collected indicators include: graphics processor utilization (%), video memory usage (MB), video memory utilization (%), temperature (°C), power consumption (W), core frequency (MHz), video memory frequency (MHz), and throughput (MB / s). For central processing unit nodes, the collected indicators include: core utilization (%), memory usage (MB), memory utilization (%), disk I / O (MB / s), and network throughput (MB / s). The monitoring data is written to a time-series database in real time and displayed through a visual dashboard. The system has three levels of alarm thresholds. When any indicator exceeds the set percentage of the normal upper limit, the corresponding level of alarm is triggered, and in emergency situations, task migration or degradation operations are automatically performed.

[0066] Upon completion of the task (either normal completion or termination due to an anomaly), the resource reclamation module automatically performs the following release operations: stops the task container and releases computing resources (GPU / CPU computing cores); cleans up temporary storage files and releases storage resources; releases reserved network bandwidth; updates the remaining resource fields (remaining computing power, remaining storage capacity, remaining bandwidth, and remaining energy consumption budget) of each node in the global computing power resource pool; and merges and reorganizes the resource blocks released this time with the existing fragment records in the fragment index table.

[0067] After each scheduling decision and execution, the distributed ledger notarization module generates notarized transactions from the end-to-end scheduling events. Scheduling events include the following fields: task identifier, task group identifier, assigned node identifier, allocated resources (computing power / storage / bandwidth / energy consumption), scheduling decision timestamp, task start time, task completion time, and execution status (success / failure / timeout). Transaction data structure. for: ; in, For the scheduling event itself, The timestamp of the event. The hash value of the previous transaction. A digital signature for the transaction initiator. A tamper-proof data chain is formed by linking the hashes of previous transactions.

[0068] Multiple transactions are organized into a Merkle tree in chronological order, and the Merkle root is calculated. Each participating node performs a practical Byzantine fault-tolerant consensus confirmation on the Merkle root. This consensus mechanism can guarantee system consistency even if no more than one-third of the nodes are malicious. After consensus confirmation, the transaction is formally written to the distributed ledger. Each participating node maintains a complete copy of the scheduling log, which includes registration logs, scheduling logs, resource release logs, and execution status logs, supporting querying and auditing traceability by task identifier, node identifier, and time range.

[0069] Real-time load monitoring enables observability of the task execution process, and a three-level alarm mechanism allows for timely intervention in case of resource anomalies, ensuring system stability. Automated resource reclamation ensures that released resources are quickly reinstated into the scheduling scope, preventing resource leaks. Distributed ledger notarization, through encrypted hash chains and consensus mechanisms, guarantees the integrity and immutability of scheduling logs. The Merkle tree structure supports efficient integrity verification, meeting the auditing, supervision, and trusted settlement requirements in cross-domain computing power scheduling scenarios.

[0070] The method of this invention can be applied to the following typical scenarios: Scenario 1: Provincial-level computing power coordination and scheduling platform. The platform uniformly manages computing power nodes across the province equipped with heterogeneous processors from multiple vendors such as NVIDIA and Ascend. Through the registration process of this invention, it automatically discovers and registers each node, allocates computing power resources on demand through a hierarchical scheduling strategy, and automatically reclaims fragmented idle computing power during nighttime and off-peak hours. When enterprises submit large model training tasks, the system schedules hybrid architecture computing power in milliseconds, saving the investment in redundant computing power construction.

[0071] Scenario 2: Private Computing Cloud Across Multiple Campuses in Universities. This method allows universities to access heterogeneous computing clusters deployed across different campuses in a unified manner. The system senses the load of computing nodes on each campus, automatically breaks down large model training tasks into sub-task groups, and distributes them to heterogeneous computing nodes on different campuses for collaborative execution, reducing hardware maintenance investment and shortening model training cycles.

[0072] Scenario 3: AI Service Platform for SMEs. SMEs' own fragmented GPU servers can be registered and incorporated into a unified management system using this method. The system identifies idle computing power fragments on each node and adds tags, scheduling fragmented computing power resources on demand for lightweight inference tasks, thereby reducing the computing power costs of enterprise digital transformation.

[0073] The unified scheduling method for heterogeneous computing power across multiple architectures provided in Embodiment 1 of this invention has the following advantages: First, in terms of adapting to heterogeneous computing power across multiple architectures, this invention uses a computing power access gateway to uniformly collect registration applications, a hardware detection module to automatically read underlying hardware feature information, and an instruction set conversion adaptation layer to achieve automated operator mapping from the source instruction set to the target instruction set. This allows for adaptation to heterogeneous processors from multiple vendors without modifying hardware drivers, significantly reducing the implementation cycle and development cost of cross-architecture deployments.

[0074] Secondly, in terms of improving the utilization rate of computing resources, this invention performs periodic load sampling on connected computing nodes at three time scales: millisecond, second, and minute. It accumulates and quantifies the idle computing power during consecutive low-load periods into quantifiable fragment capacity values ​​and attaches structured status labels, making fragmented resources effective resources that can be perceived and retrieved by the scheduling system. When there are insufficient candidate nodes, multiple fragments are automatically aggregated into virtual nodes for scheduling, realizing the automated perception, recycling, and reuse of idle computing power, and significantly improving the overall utilization rate of existing computing resources.

[0075] Third, in terms of scheduling real-time performance and decision quality, this invention adopts a two-level serial scheduling architecture of initial selection and final selection. In the initial selection stage, candidate nodes are quickly screened through five-dimensional basic resource constraints to significantly reduce the search space. In the final selection stage, a comprehensive utility function is established that includes high-priority task guarantees, total latency minimization, and node load balancing. A flexible executor-judge algorithm is used to achieve millisecond-level near-global optimal scheduling decisions. Under the premise of ensuring the quality of the scheduling scheme, the real-time requirements of large-scale cross-domain computing power scheduling are fully met.

[0076] Fourth, in terms of collaborative scheduling of multi-type heterogeneous tasks, this invention constructs a directed acyclic graph for divisible tasks, which includes three dimensions: data association, control flow association, and resource competition association. The strongly associated subtasks are aggregated into task groups through spectral clustering algorithm. In the scheduling and execution stage, a bidding competition mechanism is adopted to realize the distributed matching between task groups and computing power nodes. Each node makes independent decisions based on a four-dimensional comprehensive bid considering computing time, transmission time, resource conflicts, and communication delay, which effectively solves the single-point decision bottleneck of centralized schedulers in large-scale distributed scenarios.

[0077] In summary, this invention systematically solves several existing technical bottlenecks through a complete integrated design encompassing unified hardware feature acquisition and abstraction, instruction set cross-architecture mapping, multi-granularity load sampling and fragment identification, multi-dimensional task association graphs and spectral clustering decomposition, hierarchical scheduling and reinforcement learning optimization, distributed matching for bidding competition, and distributed ledger storage. It provides a complete technical solution for the unified management and intelligent scheduling of large-scale wide-area heterogeneous computing networks.

[0078] Example 2 like Figure 2As shown, in Embodiment 2 of the present invention, a unified scheduling system for heterogeneous computing power across multiple architectures is provided. The system includes: The reading module 1 is used to receive the registration application of each computing power node through the computing power access gateway deployed in front of each computing power node in each region, and read the underlying hardware feature information of each computing power node. The underlying hardware feature information includes at least processor type, peak computing power, storage capacity and equipment manufacturer information. The conversion module 2 is used to perform standardized conversion on the underlying hardware feature information according to the unified computing power resource abstraction specification, generate a globally unique computing power resource identifier and record it in the trusted computing power resource catalog; The mapping module 3 is used to automatically identify and map the native instruction sets of different types of processors to the underlying operators; to perform multi-granularity time-domain load sampling on the connected computing power nodes; to identify discrete idle computing power fragments based on load fluctuation characteristics and attach idle status labels; and to uniformly store standardized metadata, globally unique identifiers and idle status labels in the trusted computing power resource directory to build a globally unified computing power resource pool. Parsing module 4 is used to receive computing task requests submitted by users, parse task attributes and construct task attribute vectors. The task attribute vectors include at least computing resource requirements, storage resource requirements, bandwidth resource requirements, energy consumption requirements, task deadline and task priority weight. The splitting module 5 is used to classify tasks into indivisible tasks and divisible tasks according to the task divisibility identifier in the task attribute vector. For divisible tasks, a multi-dimensional relationship graph is constructed and subtask aggregation is performed to group each subtask into several task groups. Scheduling module 6 is used to invoke a hierarchical scheduling strategy, using the computing power resource pool and the task group as inputs to determine the optimal resource allocation scheme; for the divisible task group determined to be allocated to multiple heterogeneous computing power nodes in the optimal resource allocation scheme, a bidding competition mechanism is adopted to realize the distributed matching between the task group and the computing power node, generate scheduling instructions and send them to the corresponding computing power access gateway for execution; The evidence storage module 7 is used to monitor the load status of computing nodes in real time during task execution, release the occupied computing resources and update the computing resource pool after the task is completed, and record the scheduling decision and execution process log in a distributed ledger evidence storage method.

[0079] In conversion module 2, the unified computing resource abstraction specification includes: For computing nodes Its resource feature vector is represented as: ; In the formula, For nodes Peak computing power For peak storage capacity, Peak available bandwidth, Peak available energy consumption For nodes Transmission delay, For nodes The processor instruction set type identifier; peak computing power Divided into central processing unit computing power and graphics processing power For nodes equipped only with a central processing unit, For nodes equipped with graphics processors, .

[0080] The mapping module 3 is used for: Establish instruction set mapping relationship For the source processor instruction set Operators in ,Will Convert to a hardware-independent intermediate representation Then, through the target platform's backend compilation component, Generate as target instruction set Equivalent operator sequence on The equivalent operator sequence and They are functionally equivalent and yield the same calculation results. For operator sequences that are called more than a set threshold within a time window, a transformation cache is established. The conversion result is stored. When called later, the conversion cache is retrieved first. If a match is found, the cached target code is returned directly without triggering an immediate conversion. If a match is not found, an immediate conversion is performed and the result is written to the cache for later reuse.

[0081] The mapping module 3 is further used for: Periodic load sampling is performed on the connected computing power nodes using multiple time scales, including millisecond-level, second-level, and minute-level scales; for computing power nodes... During the time window Internal load sequence acquisition ,in, For the first Load rate of each sampling point ; If continuous at any scale The load rate of each sampling point is lower than the preset threshold. If so, the time period is determined to be a discretized fragment of idle computing power, and its available computing capacity is... for: ; In the formula, and These are the start and end indices of the consecutive low-load sampling points, respectively. For nodes Peak computing power; Add a state tag to each discretized idle computing power fragment, and the state tag includes at least: the start time. End time Available computing capacity Node identifier and processor type The status tag is serialized and stored in the fragment index table of the trusted computing power resource directory.

[0082] The splitting module 5 is used for: Task attribute vector Defined as: ; In the formula, To calculate resource requirements, Storage resource requirements, This refers to the bandwidth resource requirements. Energy consumption demand, The deadline for the task. This is a task-separable identifier. It is an indivisible task. For scalable tasks, As a task priority weight; The computing resource requirements Divided into central processing unit computing requirements and graphics processor computing requirements ,Task At the node On processing delay The calculation is as follows: ; In the formula, For the task The computational demands on the central processing unit. For the task The computational demands on graphics processing units (GPUs) For nodes The computing power of the central processing unit, For nodes The graphics processing unit's computing power; for tasks requiring only the central processing unit, For tasks that only require a graphics processor, ; priority weight Based on the urgency of the task Importance and waiting time Dynamic calculation:

[0083] in, , The first, second, and third weights are respectively. , , These represent the maximum values ​​of urgency, importance, and waiting time among all currently pending tasks. Introducing a time decay function Update effective priority ,in, These are the parameters for controlling the decay rate.

[0084] The scheduling module 6 is used for: For tasks that can be decomposed, break them down into several atomic subtasks. ,in, The computational cost of the atomic subtask. The amount of data for atomic subtasks. Tolerable delay for atomic sub-tasks; Construct a directed acyclic graph , where the vertex set Represents the set of atomic subtasks, each vertex Corresponding atomic subtasks edge set Represents the multi-dimensional relationships between atomic subtasks, edges Composite weight for: ; In the formula, , The first, second, and third association weights are respectively. For data association weights, To control flow association weights, For resource competition-related weights; Constructing the weight matrix ,in Construct the degree matrix Calculate the normalized Laplace matrix Perform eigenvalue decomposition on the normalized Laplace matrix and select the first... The eigenvectors corresponding to the smallest non-zero eigenvalues ​​constitute the feature matrix. Each atomic subtask is mapped to a point in a low-dimensional feature space; Perform resource-constrained operations in a low-dimensional feature space. - Mean aggregation divides atomic subtasks into Task groups Each task group The following hard resource constraints must be met: ; In the formula, This refers to the upper limit of the central processing unit's computing power for a single computing node. This is the maximum memory capacity. Maximum delay; If, during the aggregation process, none of the current groups can accommodate a certain subtask without violating hard resource constraints, a new task group is created.

[0085] In the scheduling module 6, the hierarchical scheduling strategy includes: Preliminary selection stage: based on effective priority All tasks to be scheduled are sorted in descending order, and candidate computing nodes are selected for each task in turn; for each task... Iterate through the actual computing power nodes and virtual nodes formed by the aggregation of discrete idle computing power fragments in the global resource pool, and select nodes that simultaneously meet the following basic resource constraints to add to the candidate set. : ; In the formula, For nodes Peak computing power For the task The computing resource requirements, For peak storage capacity, Storage resource requirements, Peak available bandwidth, This refers to the bandwidth resource requirements. Peak available energy consumption Energy consumption demand, For nodes The remaining service time, The estimated execution time of the task on the node; If candidate set If the table is empty, the fragment index table in the trusted computing power resource directory is retrieved to determine whether there are aggregable discretized idle computing power fragments; if there are aggregable discretized idle computing power fragments, multiple fragments are aggregated into virtual nodes. The virtual node is then added to the candidate set, and its available computing capacity is the sum of the capacities of each fragment. ,in, For a collection of fragments, For the first The available computing capacity of each fragment; if it is still empty, the task enters the waiting queue and re-participates in the initial selection in the next scheduling cycle; Selection phase: The candidate set output from the initial selection phase As input, define decision variables , Indicates task Assigned to computing nodes Otherwise, the value is 0; an optimization model is established with the goal of maximizing the overall system utility: ; In the formula, For the task The effective priority, For nodes Peak computing power For the task To the node Communication delay, The estimated execution time of the task on the node. The computing resources already occupied by the node. These are the first and second weighting coefficients, respectively. For a set of tasks, For a set of computing power nodes, To optimize the function values ​​of the model; The constraints of the optimization model include: Computational capability constraints: ; Storage capacity constraints: ; Bandwidth constraints: ; Energy consumption constraints: ; Delay constraints: ; Task uniqueness constraint: ; In the formula, To calculate resource requirements, Storage resource requirements, This refers to the bandwidth resource requirements. Energy consumption demand, The deadline for the task. Peak available bandwidth, Peak available energy consumption This represents the peak storage capacity.

[0086] The solution process of the optimization model includes: The computational resource scheduling problem of the optimization model is modeled as a Markov decision process. : state space : The task As a scheduling moment, at the scheduling moment state Includes remaining resource information and task queue information for each computing node: ; In the formula, For nodes Remaining computing power For the remaining storage capacity, For the remaining bandwidth, For the remaining energy consumption budget, The total number of computing nodes. It is the set of all task attribute vectors in the current waiting queue; Action space :action The resource allocation decision vector for the current task on each node; reward function for: ; In the formula, For state , The function value below, The entropy temperature coefficient For strategy In state Entropy below; Policy Network and value network Through the following objective function Joint optimization: ; In the formula, For strategy Induced trajectory distribution, Indicates adherence to the strategy Calculate the mathematical expectation of the resulting state-action trajectory; The value network employs a dual-Q network structure and updates by minimizing the soft Bellman residual: ; In the formula, Let the value network loss function be... For experience replay buffer, As a discount factor, For value networks The predicted cumulative reward for the current state and actions. In order to schedule Execute action The instant reward value returned later. The estimated state value for the target value network. This represents the mathematical expectation of the next state. This means randomly sampling a small batch of samples from the experience replay buffer and calculating the expected value. The policy network updates by minimizing the KL divergence: ; In the formula, Let the policy network loss function be... This represents the mathematical expectation of the action distribution output by the policy network. This indicates that the state is randomly sampled from the experience playback buffer. Find the expected value. For the current policy network in state Select action The probability; employing reparameterization techniques. Make the gradient propagable. It is Gaussian noise. This is a reparameterized sampling function; Network parameters adopt a soft update strategy: ; In the formula, This is the soft update coefficient. These are the parameters of the current value network and the current policy network, respectively. These are the parameters for the target value network and the target policy network, respectively.

[0087] In scheduling module 6, the bidding competition mechanism includes: Taking the divisible task groups involving multiple heterogeneous computing nodes in the optimal resource allocation scheme as input, each task group... Each candidate computing node participates in resource bidding as a bidding unit. As the bidding party; Task Group For candidate computing power nodes Bid price The calculation is as follows: ; In the formula, For nodes Peak computing power For task groups China Atomic Sub-mission The computational load, For atomic sub-tasks The amount of data, For users to nodes Data transmission rate, This represents the resource conflict penalty coefficient. This is a resource conflict indicator function, when a node... Assigned task groups and current task groups If resource contention exists, use 1; otherwise, use 0. This is the communication delay penalty coefficient. For task groups To the node Communication delay; Each computing node selects the task group with the highest bid from all received bids for matching, and then updates the node's remaining resource status after matching: ; In the formula, For nodes The remaining computing power For nodes The remaining storage capacity; The bidding-allocation process is repeated until all task groups are allocated or all computing power node resources are exhausted; if there are unallocated task groups, the backfilling strategy is initiated, and the unallocated task groups are inserted into the waiting queue of the next scheduling cycle with lower priority.

[0088] In other embodiments of the present invention, the present invention provides the following technical solution: a computer, including a memory 102, a processor 101, and a computer program stored in the memory 102 and executable on the processor 101, wherein the processor 101 executes the computer program to implement the multi-architecture heterogeneous computing power unified scheduling method as described above.

[0089] Specifically, the processor 101 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits that can be configured to implement the embodiments of the present invention.

[0090] The memory 102 may include a large-capacity memory for data or instructions. For example, and not limitingly, the memory 102 may include a hard disk drive (HDD), a floppy disk drive, a solid-state drive (SSD), flash memory, an optical disk drive, a magneto-optical disk drive, magnetic tape, or a Universal Serial Bus (USB) drive, or a combination of two or more of these. Where appropriate, the memory 102 may include removable or non-removable (or fixed) media. Where appropriate, the memory 102 may be internal or external to a data processing device. In a particular embodiment, the memory 102 is non-volatile memory. In a particular embodiment, the memory 102 includes read-only memory (ROM) and random access memory (RAM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable read-only memory (PROM), an erasable read-only memory (EPROM), an electrically erasable read-only memory (EEPROM), an electrically alterable read-only memory (EAROM), or flash memory, or a combination of two or more of these. Where appropriate, the RAM can be Static Random-Access Memory (SRAM) or Dynamic Random-Access Memory (DRAM). DRAM can be Fast Page Mode Dynamic Random Access Memory (FPMDRAM), Extended Data Out Dynamic Random Access Memory (EDODRAM), Synchronous Dynamic Random-Access Memory (SDRAM), etc.

[0091] The memory 102 can be used to store or cache various data files that need to be processed and / or used for communication, as well as possible computer program instructions executed by the processor 101.

[0092] The processor 101 reads and executes the computer program instructions stored in the memory 102 to implement the above-mentioned unified scheduling method for heterogeneous computing power across multiple architectures.

[0093] In some embodiments, the computer may further include a communication interface 103 and a bus 100. For example, Figure 3 As shown, the processor 101, memory 102, and communication interface 103 are connected through bus 100 and communicate with each other.

[0094] The communication interface 103 is used to enable communication between the various modules, devices, units, and / or equipment in the embodiments of the present invention. The communication interface 103 can also enable data communication with other components such as external devices, image / data acquisition devices, databases, external storage, and image / data processing workstations.

[0095] Bus 100 includes hardware, software, or both, that couples components of a computer device together. Bus 100 includes, but is not limited to, at least one of the following: data bus, address bus, control bus, expansion bus, and local bus. For example, and not as a limitation, bus 100 may include an Accelerated Graphics Port (AGP) or other graphics bus, an Extended Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a Hyper Transport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an InfiniBand interconnect, a Low Pin Count (LPC) bus, a memory bus, a Micro Channel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local Bus (VLB) bus, or other suitable buses, or a combination of two or more of these. Where appropriate, bus 100 may include one or more buses. Although specific buses are described and illustrated in the embodiments of the present invention, the present invention is contemplated by any suitable bus or interconnect.

[0096] The computer can execute the multi-architecture heterogeneous computing power unified scheduling method of the present invention based on the acquisition of a multi-architecture heterogeneous computing power unified scheduling system, thereby realizing the unified scheduling of multi-architecture heterogeneous computing power.

[0097] In some further embodiments of the present invention, in conjunction with the above-described unified scheduling method for heterogeneous computing power across multiple architectures, the present invention provides the following technical solution: a storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the above-described unified scheduling method for heterogeneous computing power across multiple architectures.

[0098] Those skilled in the art will understand that the logic and / or steps represented in the flowcharts or otherwise described herein, for example, can be considered as a ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can mean any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device.

[0099] More specific examples of readable media (a non-exhaustive list) include: electrical connections (electronic devices) with one or more wires, portable computer disk drives (magnetic devices), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Furthermore, computer-readable media can even be paper or other suitable media on which the program can be printed, since the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in computer memory.

[0100] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0101] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0102] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these all fall within the protection scope of the present invention. Therefore, the protection scope of this invention patent should be determined by the appended claims.

Claims

1. A unified scheduling method for heterogeneous computing power across multiple architectures, characterized in that, include: S1. The registration application of each computing power node is collected through the computing power access gateway deployed in front of each computing power node in each region, and the underlying hardware feature information of each computing power node is read. The underlying hardware feature information includes at least processor type, peak computing power, storage capacity and equipment manufacturer information. S2. Perform a standardized transformation on the underlying hardware feature information according to the unified computing power resource abstraction specification, generate a globally unique computing power resource identifier and record it in the trusted computing power resource catalog; S3. Perform automatic identification and low-level operator mapping on the native instruction set of different types of processors; perform multi-granularity time-domain load sampling on the connected computing power nodes, identify discrete idle computing power fragments based on load fluctuation characteristics and attach idle status labels, and uniformly store standardized metadata, global unique identifiers and idle status labels in the trusted computing power resource catalog to build a global unified computing power resource pool. S4. Receive the computing task request submitted by the user, parse the task attributes and construct a task attribute vector. The task attribute vector includes at least the computing resource requirement, storage resource requirement, bandwidth resource requirement, energy consumption requirement, task deadline and task priority weight. S5. Based on the task splittable identifier in the task attribute vector, the task is divided into indivisible task and splittable task. For splittable task, a multi-dimensional relationship graph is constructed and subtask aggregation is performed to group each subtask into several task groups. S6. Invoke the hierarchical scheduling strategy and determine the optimal resource allocation scheme using the computing power resource pool and the task group as inputs; For the divisible task groups determined to be allocated to multiple heterogeneous computing power nodes in the optimal resource allocation scheme, a bidding competition mechanism is adopted to realize the distributed matching between task groups and computing power nodes, generate scheduling instructions and send them to the corresponding computing power access gateways for execution. S7. Monitor the load status of computing nodes in real time during task execution, release the occupied computing resources and update the computing resource pool after the task is completed, and record the scheduling decision and execution process log in a distributed ledger storage method.

2. The method for unified scheduling of heterogeneous computing power across multiple architectures according to claim 1, characterized in that, In step S2, the unified computing resource abstraction specification includes: For computing nodes Its resource feature vector is represented as: ； In the formula, For nodes Peak computing power For peak storage capacity, Peak available bandwidth, Peak available energy consumption For nodes Transmission delay, For nodes The processor instruction set type identifier; Peak computing power Divided into central processing unit computing power and graphics processing power For nodes equipped only with a central processing unit, For nodes equipped with graphics processors, .

3. The method for unified scheduling of heterogeneous computing power across multiple architectures according to claim 1, characterized in that, In step S3, the step of automatically identifying and mapping the native instruction sets of different types of processors to low-level operators includes: Establish instruction set mapping relationship For the source processor instruction set Operators in ,Will Convert to a hardware-independent intermediate representation Then, through the target platform's backend compilation component, Generate as target instruction set Equivalent operator sequence on The equivalent operator sequence and They are functionally equivalent and yield the same calculation results. For operator sequences that are called more than a set threshold within a time window, a transformation cache is established. The conversion result is stored. When called later, the conversion cache is retrieved first. If a match is found, the cached target code is returned directly without triggering an immediate conversion. If a match is not found, an immediate conversion is performed and the result is written to the cache for later reuse.

4. The method for unified scheduling of heterogeneous computing power across multiple architectures according to claim 1, characterized in that, In step S3, the step of performing multi-granularity time-domain load sampling on the connected computing power nodes, identifying discretized idle computing power fragments based on load fluctuation characteristics, and attaching idle status tags includes: Periodic load sampling is performed on the connected computing power nodes using multiple time scales, including millisecond-level, second-level, and minute-level scales; for computing power nodes... During the time window Internal load sequence acquisition ,in, For the first Load rate of each sampling point ; If continuous at any scale The load rate of each sampling point is lower than the preset threshold. If so, the time period is determined to be a discretized fragment of idle computing power, and its available computing capacity is... for: ； In the formula, and These are the start and end indices of the consecutive low-load sampling points, respectively. For nodes Peak computing power; Add a state tag to each discretized idle computing power fragment, and the state tag includes at least: the start time. End time Available computing capacity Node identifier and processor type The status tag is serialized and stored in the fragment index table of the trusted computing power resource directory.

5. The method for unified scheduling of heterogeneous computing power across multiple architectures according to claim 1, characterized in that, In step S5, the step of constructing the task attribute vector includes: Task attribute vector Defined as: ； In the formula, To calculate resource requirements, Storage resource requirements, This refers to the bandwidth resource requirements. Energy consumption demand, The deadline for the task. This is a task-separable identifier. It is an indivisible task. For scalable tasks, As a task priority weight; The computing resource requirements Divided into central processing unit computing requirements and graphics processor computing requirements ,Task At the node On processing delay The calculation is as follows: ； In the formula, For the task The computational demands on the central processing unit. For the task The computational demands on graphics processing units (GPUs) For nodes The computing power of the central processing unit, For nodes The graphics processing unit's computing power; for tasks requiring only the central processing unit, For tasks that only require a graphics processing unit, ; priority weight Based on the urgency of the task Importance and waiting time Dynamic calculation: in, , The first, second, and third weights are respectively. , , These represent the maximum values ​​of urgency, importance, and waiting time among all currently pending tasks. Introducing a time decay function Update effective priority ,in, These are the parameters for controlling the decay rate.

6. The method for unified scheduling of heterogeneous computing power across multiple architectures according to claim 1, characterized in that, In step S6, the step of constructing a multi-dimensional relationship graph for separable tasks and performing subtask aggregation to group each subtask into several task groups includes: For tasks that can be decomposed, break them down into several atomic subtasks. ,in, The computational cost of the atomic subtask. The amount of data for the atomic subtasks. Tolerable delay for atomic sub-tasks; Construct a directed acyclic graph , where the vertex set Represents the set of atomic subtasks, each vertex Corresponding atomic subtasks edge set Represents the multi-dimensional relationships between atomic subtasks, edges Composite weight for: ； In the formula, , The first, second, and third association weights are respectively. For data association weights, To control flow association weights, For resource competition-related weights; Constructing the weight matrix ,in Construct the degree matrix Calculate the normalized Laplace matrix Perform eigenvalue decomposition on the normalized Laplace matrix and select the first... The eigenvectors corresponding to the smallest non-zero eigenvalues ​​constitute the feature matrix. Each atomic subtask is mapped to a point in a low-dimensional feature space; Perform resource-constrained operations in a low-dimensional feature space. - Mean aggregation divides atomic subtasks into Task groups Each task group The following hard resource constraints must be met: ； In the formula, This refers to the upper limit of the central processing unit's computing power for a single computing node. This is the maximum memory capacity. Maximum delay; If, during the aggregation process, none of the current groups can accommodate a certain subtask without violating hard resource constraints, a new task group is created.

7. The method for unified scheduling of heterogeneous computing power across multiple architectures according to claim 1, characterized in that, In step S6, the hierarchical scheduling strategy includes: Preliminary selection stage: based on effective priority All tasks to be scheduled are sorted in descending order, and candidate computing nodes are selected for each task in turn; for each task... Iterate through the actual computing power nodes and virtual nodes formed by the aggregation of discrete idle computing power fragments in the global resource pool, and select nodes that simultaneously meet the following basic resource constraints to add to the candidate set. : ； In the formula, For nodes Peak computing power For the task The computing resource requirements, For peak storage capacity, Storage resource requirements, Peak available bandwidth, This refers to the bandwidth resource requirements. Peak available energy consumption Energy consumption demand, For nodes The remaining service time, The estimated execution time of the task on the node; If candidate set If the table is empty, the fragment index table in the trusted computing power resource directory is retrieved to determine whether there are aggregable discretized idle computing power fragments; if there are aggregable discretized idle computing power fragments, multiple fragments are aggregated into virtual nodes. The virtual node is then added to the candidate set, and its available computing capacity is the sum of the capacities of each fragment. ,in, For a collection of fragments, For the first The available computing capacity of each fragment; if it is still empty, the task enters the waiting queue and re-participates in the initial selection in the next scheduling cycle; Selection phase: The candidate set output from the initial selection phase As input, define decision variables , Indicates task Assigned to computing nodes Otherwise, the value is 0; an optimization model is established with the goal of maximizing the overall system utility: ； In the formula, For the task The effective priority, For nodes Peak computing power For the task To the node Communication delay, The estimated execution time of the task on the node. The computing resources already occupied by the node. These are the first and second weighting coefficients, respectively. For a set of tasks, A set of computing power nodes. To optimize the function values ​​of the model; The constraints of the optimization model include: Computational capability constraints: ; Storage capacity constraints: ; Bandwidth constraints: ; Energy consumption constraints: ; Delay constraints: ; Task uniqueness constraint: ; In the formula, To calculate resource requirements, Storage resource requirements, This refers to the bandwidth resource requirements. Energy consumption demand, The deadline for the task. Peak available bandwidth, Peak available energy consumption This represents the peak storage capacity.

8. The method for unified scheduling of heterogeneous computing power across multiple architectures according to claim 7, characterized in that, The solution process of the optimization model includes: The computational resource scheduling problem of the optimization model is modeled as a Markov decision process. : state space : The task As a scheduling moment, at the scheduling moment state Includes remaining resource information and task queue information for each computing node: ； In the formula, For nodes Remaining computing power For the remaining storage capacity, For the remaining bandwidth, For the remaining energy consumption budget, The total number of computing nodes. It is the set of all task attribute vectors in the current waiting queue; Action space :action The resource allocation decision vector for the current task on each node; reward function for: ； In the formula, For state , The function value below, The entropy temperature coefficient For strategy In state Entropy below; Policy Network and value network Through the following objective function Joint optimization: ； In the formula, For strategy Induced trajectory distribution, Indicates adherence to the strategy Calculate the mathematical expectation of the resulting state-action trajectory; The value network employs a dual-Q network structure and updates by minimizing the soft Bellman residual: ； In the formula, Let the value network loss function be... For experience replay buffer, As a discount factor, For value networks The predicted cumulative reward for the current state and actions. In order to schedule Execute action The instant reward value returned later. The estimated state value for the target value network. This represents the mathematical expectation of the next state. This means randomly sampling a small batch of samples from the experience replay buffer and calculating the expected value. The policy network updates by minimizing the KL divergence: ； In the formula, Let the policy network loss function be... This represents the mathematical expectation of the action distribution output by the policy network. This indicates a state randomly sampled from the experience playback buffer. Find the expected value. For the current policy network in state Select action The probability; employing reparameterization techniques Make the gradient propagable. It is Gaussian noise. This is a reparameterized sampling function; Network parameters adopt a soft update strategy: ； In the formula, This is the soft update coefficient. These are the parameters of the current value network and the current policy network, respectively. These are the parameters for the target value network and the target policy network, respectively.

9. The method for unified scheduling of heterogeneous computing power across multiple architectures according to claim 1, characterized in that, In step S6, the bidding competition mechanism includes: Taking the divisible task groups involving multiple heterogeneous computing nodes in the optimal resource allocation scheme as input, each task group... Each candidate computing node participates in resource bidding as a bidding unit. As the bidding party; Task Group For candidate computing power nodes Bid price The calculation is as follows: ； In the formula, For nodes Peak computing power For task groups China Atomic Sub-mission The computational load, For atomic sub-tasks The amount of data, For users to nodes Data transmission rate, This represents the resource conflict penalty coefficient. This is a resource conflict indicator function, when a node... Assigned task groups and current task groups If resource contention exists, use 1; otherwise, use 0. This is the communication delay penalty coefficient. For task groups To the node Communication delay; Each computing node selects the task group with the highest bid from all received bids for matching, and then updates the node's remaining resource status after matching: ； In the formula, For nodes The remaining computing power For nodes The remaining storage capacity; The bidding-allocation process is repeated until all task groups are allocated or all computing power node resources are exhausted; if there are unallocated task groups, the backfilling strategy is initiated, and the unallocated task groups are inserted into the waiting queue of the next scheduling cycle with lower priority.

10. A unified scheduling system for heterogeneous computing power across multiple architectures, characterized in that, The system includes: The reading module is used to receive the registration application of each computing power node through the computing power access gateway deployed in front of each computing power node in each region, and to read the underlying hardware feature information of each computing power node. The underlying hardware feature information includes at least processor type, peak computing power, storage capacity and equipment manufacturer information. The conversion module is used to perform standardized conversion on the underlying hardware feature information according to the unified computing power resource abstraction specification, generate a globally unique computing power resource identifier and record it in the trusted computing power resource catalog. The mapping module is used to automatically identify and map the native instruction sets of different types of processors to low-level operators; to perform multi-granularity time-domain load sampling on connected computing power nodes; to identify discrete idle computing power fragments based on load fluctuation characteristics and attach idle status tags; and to uniformly store standardized metadata, globally unique identifiers and idle status tags in the trusted computing power resource directory to build a globally unified computing power resource pool. The parsing module is used to receive computing task requests submitted by users, parse task attributes and construct a task attribute vector. The task attribute vector includes at least the computing resource requirements, storage resource requirements, bandwidth resource requirements, energy consumption requirements, task deadline and task priority weight. The splitting module is used to classify tasks into indivisible tasks and divisible tasks according to the task splitting identifier in the task attribute vector. For divisible tasks, a multi-dimensional relationship graph is constructed and subtask aggregation is performed to group each subtask into several task groups. The scheduling module is used to invoke the hierarchical scheduling strategy, and determine the optimal resource allocation scheme with the computing power resource pool and the task group as input; for the divisible task group determined to be allocated to multiple heterogeneous computing power nodes in the optimal resource allocation scheme, a bidding competition mechanism is adopted to realize the distributed matching between the task group and the computing power node, generate scheduling instructions and send them to the corresponding computing power access gateway for execution; The evidence storage module is used to monitor the load status of computing nodes in real time during task execution, release the occupied computing resources and update the computing resource pool after the task is completed, and record the scheduling decision and execution process log in a distributed ledger evidence storage method.

Images