Deterministic service-oriented resource scheduling method, apparatus and device, medium and program product

Abstract

The invention relates to a deterministic service-oriented resource scheduling method, apparatus and device, a medium and a program product. The method comprises the following steps: modeling a deterministic service into a directed acyclic graph; therefore, the processing flow of the deterministic service can be represented by the directed acyclic graph, so that the intelligent agent can identify the execution overhead of each task on the computing node according to the directed acyclic graph, and a basis is provided for subsequent scheduling decisions. Determining a real-time state of each computing node in the heterogeneous computing environment; therefore, the current data processing capacity of each computing node in the complex heterogeneous computing environment can be mastered in real time, and the task load of each computing node can be balanced conveniently. And based on the directed acyclic graph corresponding to the deterministic service and the real-time state of each computing node, outputting a scheduling decision of the deterministic service through the trained agent. Therefore, computing and communication resources can be dispatched cooperatively under a unified converged network architecture, so that the computing and communication resources can adapt to a complex and changeable network environment, and an optimal dispatching decision is output.

Description

Technical Field

[0001] This application relates to the field of industrial internet technology, and in particular to a resource scheduling method, apparatus, computer equipment, computer-readable storage medium, and computer program product for deterministic business. Background Technology

[0002] In modern energy systems, such as new power systems, intelligent oil and gas stations, and smart mines, multiple services typically need to run on a unified, converged network.

[0003] Providing reliable assurance for services with varying Quality of Service (QoS) requirements within a single converged network presents significant technical challenges. Existing solutions suffer from two core flaws. First, traditional network architectures cannot resolve resource contention conflicts and lack determinism. Second, computing and communication resources are fragmented, making end-to-end service assurance impossible.

[0004] Therefore, there is an urgent need for a new technical solution that can coordinate and schedule computing and communication resources under a unified converged network architecture. Summary of the Invention

[0005] Therefore, it is necessary to provide a resource scheduling method, apparatus, computer equipment, computer-readable storage medium, and computer program product that can coordinate the scheduling of computing and communication resources for deterministic services under a unified converged network architecture to address the above-mentioned technical problems.

[0006] Firstly, this application provides a resource scheduling method for deterministic business operations, the method comprising:

[0007] Model deterministic business processes as directed acyclic graphs;

[0008] Determine the real-time status of each computing node in a heterogeneous computing environment;

[0009] Based on the directed acyclic graph corresponding to the deterministic service and the real-time state of each computing node, the trained agent outputs the scheduling decision for the deterministic service.

[0010] In one embodiment, modeling the deterministic business as a directed acyclic graph includes:

[0011] The deterministic business is divided into multiple independent subtasks. Each subtask is a basic unit for scheduling and execution and has its own resource requirement attributes, including resource type and resource quantity.

[0012] Based on the direct dependencies between the subtasks, construct a set of directed arcs;

[0013] A directed acyclic graph (DAG) for the deterministic service is generated based on the set of subtasks and directed arcs, wherein each vertex in the DAG represents a subtask of the deterministic service, and the connections between the vertices in the DAG represent the direct dependencies between the subtasks.

[0014] In one embodiment, determining the real-time status of each computing node in a heterogeneous computing environment includes:

[0015] A multi-dimensional heterogeneous computing power model is constructed, which is used to quantitatively evaluate the processing capabilities of heterogeneous nodes of different types and from different manufacturers.

[0016] The real-time status of each computing node in a heterogeneous computing environment is evaluated by constructing a multi-dimensional heterogeneous computing power model. The real-time status includes: computing power, communication power, and storage power; wherein:

[0017] The computing power is associated with at least one of the following indicators: the logical operation capability, parallel computing capability, and neural network computing capability of the computing node;

[0018] The communication capability is associated with the network bandwidth and / or memory bandwidth of the computing node;

[0019] The storage capacity is related to the storage bandwidth of the computing node and / or the number of read / write operations per second.

[0020] In one embodiment, before the trained agent outputs the scheduling decision for the deterministic service based on the directed acyclic graph corresponding to the deterministic service and the real-time state of each computing node, the method further includes:

[0021] An intelligent agent is constructed, comprising: an environmental perception and control module and an intelligent agent decision-making module. The environmental perception and control module is used to perceive the environmental state, and the intelligent agent decision-making module adopts an actor network and critic network architecture to generate decision actions based on the environmental state. The environmental perception and control module is also used to execute the decision actions and return the updated environmental state and corresponding reward.

[0022] The environmental state includes: the real-time state of all computing nodes, and the completion status and dependencies of each subtask in the directed acyclic graph.

[0023] In one embodiment, after constructing the agent, the method further includes:

[0024] A reward function is set for the directed acyclic graph of deterministic services that have been scheduled and completed, and the agent decision-making module in the agent is trained according to the reward function;

[0025] The network parameters of the actor network are updated based on the generalized advantage estimation strategy and the objective function pruning strategy, and the network parameters of the critic network are updated based on the loss function, resulting in a trained agent.

[0026] In one embodiment, updating the network parameters of the actor network based on the generalized advantage estimation strategy and the objective function pruning strategy, and updating the network parameters of the critic network based on the loss function to obtain a trained agent, includes:

[0027] Initialize the network parameters of the actor network and the critic network, as well as the initial experience replay buffer, which is used to store training experience data;

[0028] Within each training cycle, for the directed acyclic graph of the deterministic business to be scheduled and the current system resource status, scheduling decisions for each subtask are generated, and the training experience data corresponding to each scheduling decision is stored in the experience replay buffer.

[0029] When the amount of stored training experience data is greater than a preset value, the advantage function estimate and state value target for each time step are determined based on the generalized advantage estimation strategy. Based on the advantage function estimate and the state value target, the network parameters of the actor network are updated by maximizing the pruning objective function of the proximal policy optimization algorithm.

[0030] The network parameters of the critic network are updated by minimizing the mean squared error loss function;

[0031] The network parameters of the actor network and the critic network are iteratively updated until the policy performance of the actor network converges, or the preset number of iterations is reached, to obtain a trained agent.

[0032] Secondly, this application also provides a resource scheduling apparatus for deterministic business operations, the apparatus comprising:

[0033] The Directed Acyclic Graph (DAG) building module is used to model deterministic business processes as directed acyclic graphs.

[0034] The node status determination module is used to determine the real-time status of each computing node in a heterogeneous computing environment.

[0035] The scheduling decision generation module is used to output the scheduling decision of the deterministic service through the trained agent, based on the directed acyclic graph corresponding to the deterministic service and the real-time state of each computing node.

[0036] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0037] Model deterministic business processes as directed acyclic graphs;

[0038] Determine the real-time status of each computing node in a heterogeneous computing environment;

[0039] Based on the directed acyclic graph corresponding to the deterministic service and the real-time state of each computing node, the trained agent outputs the scheduling decision for the deterministic service.

[0040] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, performs the following steps:

[0041] Model deterministic business processes as directed acyclic graphs;

[0042] Determine the real-time status of each computing node in a heterogeneous computing environment;

[0043] Based on the directed acyclic graph corresponding to the deterministic service and the real-time state of each computing node, the trained agent outputs the scheduling decision for the deterministic service.

[0044] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, performs the following steps:

[0045] Model deterministic business processes as directed acyclic graphs;

[0046] Determine the real-time status of each computing node in a heterogeneous computing environment;

[0047] Based on the directed acyclic graph corresponding to the deterministic service and the real-time state of each computing node, the trained agent outputs the scheduling decision for the deterministic service.

[0048] The aforementioned resource scheduling methods, devices, computer equipment, computer-readable storage media, and computer program products for deterministic services model deterministic services as directed acyclic graphs (DAGs). This allows the processing flow of deterministic services to be represented using DAGs, enabling intelligent agents to identify the execution overhead of each task on computing nodes based on the DAG, thus providing a basis for subsequent scheduling decisions. It also determines the real-time state of each computing node in a heterogeneous computing environment, allowing for real-time monitoring of the current data processing capabilities of each node in a complex heterogeneous computing environment, facilitating the balancing of workloads across nodes. Based on the DAG corresponding to the deterministic service and the real-time state of each computing node, a trained intelligent agent outputs scheduling decisions for the deterministic service. This enables the coordinated scheduling of computing and communication resources within a unified, converged network architecture, adapting to complex and ever-changing network environments and outputting optimal scheduling decisions. Attached Figure Description

[0049] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0050] Figure 1 This is a flowchart illustrating a resource scheduling method for deterministic business logic in one embodiment.

[0051] Figure 2 This is a schematic diagram of a directed acyclic graph of an energy deterministic service in one embodiment of this application;

[0052] Figure 3 This is a flowchart illustrating a resource scheduling method for deterministic business logic in another embodiment.

[0053] Figure 4 A schematic diagram of a resource scheduling framework based on an improved actor-critic algorithm provided for embodiments of this application;

[0054] Figure 5 A schematic diagram showing the comparison of the average latency of different algorithms provided in the embodiments of this application;

[0055] Figure 6 A schematic diagram showing the comparison of the average delay variance of different algorithms provided in the embodiments of this application;

[0056] Figure 7 A schematic diagram showing the comparison of resource utilization of different algorithms provided in the embodiments of this application;

[0057] Figure 8This is a structural block diagram of a resource scheduling device for deterministic services in one embodiment;

[0058] Figure 9 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0059] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0060] It should be noted that the terms "comprising" and "having," and any variations thereof, as used in this application, are intended to cover non-exclusive inclusion. The term "multiple" as used in this application refers to two or more. The term "and / or" as used in this application refers to one of the solutions, or any combination of multiple solutions.

[0061] In modern energy systems, such as new power systems, intelligent oil and gas stations, and smart mines, multiple services typically need to run on a unified, converged network. For example, hard real-time control services (like relay protection) have small data packets but require millisecond- or even microsecond-level latency and jitter tolerance. Monitoring and telemetry services, such as Supervisory Control and Data Acquisition (SCADA) systems, require highly reliable and continuously stable medium-to-high bandwidth. Production management services (like video surveillance) generate large volumes of data, demanding high bandwidth but not being sensitive to latency. Therefore, the industry urgently needs a new technological solution that can collaboratively schedule computing and communication resources within a unified, converged network architecture.

[0062] To address the aforementioned issues, this application aims to provide a resource scheduling method for deterministic services. By constructing a joint model of business processes and underlying resources, and utilizing an improved actor-critic algorithm, it intelligently allocates network and computing resources collaboratively for energy services. This resolves business conflict issues in converged bearer networks and ensures the safe and stable operation of critical infrastructure such as power grids and oil and gas.

[0063] This application aims to address two core challenges faced in unified and converged networks. First, there is the resource contention between heterogeneous services. High-bandwidth but latency-insensitive services (such as video surveillance) can crowd out resources for time-sensitive but low-data-volume hard real-time control services (such as relay protection), resulting in the latter's inability to effectively guarantee deterministic metrics such as latency and jitter. Second, there is the fragmentation of computing and communication resources. Because network resources and terminal device computing resources are managed and scheduled independently, "resource silos" are formed, making it impossible to collaboratively optimize the entire process of a service from data acquisition and network transmission to computational decision-making, and hindering the achievement of end-to-end Service Level Agreement (SLA) guarantees.

[0064] This application's embodiments explicitly model the entire business process (including computation and communication stages) as a Directed Acyclic Graph (DAG) and employ reinforcement learning algorithms to collaboratively schedule network and computing resources within a unified framework to find the globally optimal solution. This method yields significant benefits: it effectively ensures the determinism of critical services by planning conflict-free resource paths; it achieves true end-to-end SLA collaborative assurance by breaking down resource silos; and it makes it possible to carry multiple services on a single network, thereby improving resource utilization and significantly reducing the high costs associated with building independent private networks.

[0065] In one exemplary embodiment, such as Figure 1 As shown, a resource scheduling method for deterministic services is provided. The method in this embodiment may include the following steps S101 to S103. Wherein:

[0066] Step S101: Model the deterministic business as a directed acyclic graph.

[0067] Taking the energy industrial internet as an example, an end-to-end deterministic business, such as fault location, isolation, and recovery (FA) in a regional power grid, is usually not a single atomic operation, but a complex process composed of multiple sub-tasks with inherent temporal dependencies. For example, the aforementioned FA business can be decomposed into: a "data acquisition" sub-task executed in parallel by multiple Feeder Terminal Units (FTUs); a "fault analysis" sub-task executed on edge computing nodes after the relevant FTUs have completed data acquisition; a "isolation and recovery strategy generation" sub-task executed on the central station server after the fault analysis results are received; and a "control command execution" sub-task executed by multiple related FTUs or smart switches after receiving instructions from the central station.

[0068] To provide a formal description of such a complex business process, it can be abstracted into a Directed Acyclic Graph (DAG).

[0069] For example, deterministic business is divided into multiple independent subtasks. Each subtask is a basic unit for scheduling and execution and has its own resource requirement attributes, including resource type and resource quantity. Based on the direct dependencies between the subtasks, a set of directed arcs is constructed. Based on the set of subtasks and directed arcs, a directed acyclic graph (DAG) of the deterministic business is generated. Each vertex in the DAG represents a subtask of the deterministic business, and the connection relationships between the vertices in the DAG represent the direct dependencies between the subtasks.

[0070] For example, a deterministic business S can be modeled as a DAG, denoted as G=(V,E). Where: It is a set of vertices, representing n independent subtasks that constitute the business. Each subtask All are basic units of scheduling and execution, and each has its own resource requirement attributes (e.g., what type and quantity of computing, communication, or storage resources are needed, which can be quantified according to the computing power model in Part 1). E is a set of directed arcs, representing the direct dependencies (or preorder constraints) between subtasks. If there exists a curve from vertex E to E' in the graph... point to Directed arc ( , If )∈E, then it represents a subtask. It is a subtask The prerequisite tasks. It must be completed before it is finished. Only then can execution begin. Furthermore, this dependency can also be represented by an n×n adjacency matrix R, where the dependency is true if and only if ( , When )∈E, =1; otherwise, =0.

[0071] For example, Figure 2 This is a schematic diagram of a directed acyclic graph representing a deterministic energy service in one embodiment of this application, as shown below. Figure 2 As shown, the execution logic of this DAG business model follows strict pre-order constraints: any subtask Only when all its direct prerequisite tasks (i.e., the set { ∣( , When all tasks in ∈ E} have been completed, the subtask is... Once a task enters the ready state, it is eligible to be scheduled for execution on a specific computing node. By modeling the business process as a Directed Acyclic Graph (DAG), a clear mathematical structure is provided for subsequent automated scheduling. Furthermore, the scheduling algorithm can identify tasks currently in the ready state based on the topological structure of the DAG and make optimal matching decisions based on the task's resource requirements and the available resources of the node.

[0072] As an alternative example, in the heterogeneous environment of the energy industrial internet, the core problem of scheduling the aforementioned DAG business model can be defined as a multi-objective optimization problem. Its main objective is to achieve efficient utilization of system resources while ensuring that service level agreements (SLAs) are met.

[0073] In this embodiment, the primary optimization objective of scheduling is to meet the end-to-end latency constraints of the service. That is, for a given service DAG, an optimal scheduling scheme needs to be found (determining the scheduling of each subtask). (which compute node it executes on and its specific start time), resulting in the total time from the start of the first task in the DAG to the completion of the last task. It meets the preset SLA latency requirements. ,Right now ≤ This differs from traditional internet data centers, which aim to maximize total system throughput; energy scenarios focus more on the deterministic execution of individual critical business processes.

[0074] To quantify the above optimization objectives, a configurable, multi-objective cost function can be constructed. The ultimate goal of the scheduling algorithm is to find an optimal scheduling policy π that minimizes the value of this cost function. In one embodiment, this cost function can be expressed as:

[0075]

[0076] in, The total execution latency for the system to process all tasks. This represents resource utilization, including CPU, GPU, and memory utilization. a and b are the weight coefficients of the objective function, representing the system's optimization priorities for execution latency and resource utilization, respectively. The formula is:

[0077]

[0078] in, + + =1 represents the weight allocation of different resources. The above optimization objective aims to find a scheduling strategy that can achieve effective and balanced use of heterogeneous resources throughout the system while prioritizing low end-to-end latency for business operations.

[0079] Step S102: Determine the real-time status of each computing node in the heterogeneous computing environment.

[0080] The method in this embodiment can be applied to an energy industrial internet environment that includes multiple computing nodes. A typical system environment may include multiple geographically dispersed or centrally deployed computing nodes. These computing nodes may include, but are not limited to: edge computing devices deployed in remote unattended stations, servers in aggregation rooms, or data center servers in regional control centers. Due to differences in function, cost, and deployment time, the hardware configuration and processing capabilities of these computing nodes usually vary, constituting a heterogeneous computing environment.

[0081] For example, to achieve accurate subsequent collaborative scheduling of computing and network resources, a multi-dimensional heterogeneous computing power model needs to be constructed to provide a more precise and comprehensive quantitative description of the integrated business processing capabilities of computing nodes. This multi-dimensional heterogeneous computing power model can be used to quantitatively evaluate the processing capabilities of heterogeneous nodes from different types and vendors. It assesses the real-time status of each computing node in a heterogeneous computing environment, including computing power, communication power, and storage power. Specifically, computing power is related to at least one of the following indicators: logical operation capability, parallel computing capability, and neural network computing capability. Communication capability is related to the network bandwidth and / or memory bandwidth of the computing node. Storage power is related to the storage bandwidth and / or the number of read / write operations per second of the computing node.

[0082] It should be understood that different types of processing units (chips) have different efficiency advantages when handling different energy businesses. Therefore, the overall computing capabilities of computing nodes can be subdivided and weighted for modeling. For example, the Central Processing Unit (CPU) excels at handling logically complex, sequentially executed tasks, such as business logic judgments in SCADA systems. The Graphics Processing Unit (GPU) performs exceptionally well in large-scale parallel computing and is suitable for scenarios such as power system flow calculations and AI-assisted equipment status image analysis. The Neural Processing Unit / Tensor Processing Unit (NPU / TPU) is specifically optimized for neural network computing and is suitable for model inference tasks driven by Artificial Intelligence (AI), such as power grid fault identification and load forecasting.

[0083] Optionally, the computing power of a node can be described by three indicators: logical operation capability, parallel computing capability, and neural network computing capability. The calculation formula is as follows:

[0084]

[0085] in, This represents the computational power of node k. , These are configurable or trainable weight hyperparameters. These parameters can be adjusted to emphasize various computing capabilities according to different business scenarios, thereby dynamically reflecting the effective computing power of a node for a specific business. Metrics representing the CPU's logical operation capabilities, such as instruction execution speed; It is a metric for GPU parallel computing capabilities, such as floating-point operations per second (FLOPS). These are metrics related to the neural network computing capabilities of accelerators such as NPUs / TPUs. For example, the number of frames processed per second or the number of inferences performed under a specific benchmark model. This weighted combination method allows for a more accurate characterization of the overall computing power of a node.

[0086] As an example, in energy operations, data transmission efficiency is one of the key bottlenecks, encompassing both data interaction between nodes and external devices (network I / O) and data exchange between the node's internal computing units and memory (memory I / O). Therefore, the formula for calculating the communication capability of node k is as follows:

[0087]

[0088] in, This represents the overall communication capability of node k. This represents network bandwidth, reflecting the speed at which a node interacts with the external network for data. It refers to memory bandwidth, which reflects the rate at which data is transferred between memory and other components within a node. and This represents the weights. In large-scale distributed training scenarios, the frequent parameter exchanges between nodes place extremely high demands on network bandwidth, while the high-speed data transfer between memory and computing chips during model training depends on memory bandwidth.

[0089] Storage capacity is determined by storage bandwidth and the number of read / write operations per second (IOPS). During data storage and retrieval, storage bandwidth determines the data transfer speed, while IOPS reflects the frequency of read / write operations that the storage device can handle. The storage capacity of node k can be described using storage bandwidth and the number of IOPS:

[0090]

[0091] in, For the storage capacity of node k, , These are trainable hyperparameters. Storage bandwidth is the bandwidth, and IOPS represents the number of read and write operations per second. Taking large-scale data storage and analysis scenarios as an example, storage devices with high storage bandwidth and high IOPS can quickly respond to data read and write requests, ensuring the efficient execution of data analysis tasks.

[0092] In this embodiment, a multi-dimensional heterogeneous computing power model can map the processing capabilities of heterogeneous nodes of different types and from different vendors into a unified, multi-dimensional framework. This not only achieves a unified measurement of heterogeneous computing power in the energy industrial internet, but also provides a standard and reliable data foundation for subsequent refined, business-aware resource scheduling decisions.

[0093] Step S103: Based on the directed acyclic graph corresponding to the deterministic service and the real-time state of each computing node, the trained agent outputs the scheduling decision for the deterministic service.

[0094] In this embodiment, the trained agent can automatically learn the optimal scheduling decision for deterministic services based on the input directed acyclic graph and the real-time state of each computing node in the heterogeneous environment.

[0095] In the aforementioned resource scheduling method for deterministic services, the deterministic service is modeled as a directed acyclic graph (DAG). This allows the processing flow of the deterministic service to be represented using a DAG, enabling the agent to identify the execution cost of each task on the computing nodes and providing a basis for subsequent scheduling decisions. The real-time state of each computing node in the heterogeneous computing environment is determined, allowing for real-time monitoring of the current data processing capabilities of each node in the complex heterogeneous computing environment, facilitating the balancing of workloads across nodes. Based on the DAG corresponding to the deterministic service and the real-time state of each computing node, the trained agent outputs scheduling decisions for the deterministic service. This enables the coordinated scheduling of computing and communication resources within a unified, converged network architecture, adapting to complex and ever-changing network environments and outputting optimal scheduling decisions.

[0096] In another exemplary embodiment, such as Figure 3 As shown, a resource scheduling method for deterministic services is provided. The method in this embodiment may include the following steps S301 to S306. Wherein:

[0097] Step S301: Model the deterministic business as a directed acyclic graph.

[0098] Step S302: Determine the real-time status of each computing node in the heterogeneous computing environment.

[0099] For the specific implementation process and technical effects of steps S301 to S302 in this embodiment, please refer to [link / reference]. Figure 1 The relevant descriptions of steps S101 to S102 in the method embodiment shown will not be repeated here.

[0100] Step S303: Construct the intelligent agent.

[0101] In this embodiment, the intelligent decision-making agent within the intelligent agent employs a collaborative scheduling method based on improved actor-critic reinforcement learning. The intelligent agent includes an environment perception and control module and an intelligent agent decision-making module. The environment perception and control module perceives the environment state, while the intelligent decision-making agent, using an actor network and critic network architecture, generates decision actions based on the environment state. The environment perception and control module also executes the decision actions and returns the updated environment state and corresponding reward. The environment state includes the real-time state of all computing nodes, as well as the completion status and dependencies of each subtask in the directed acyclic graph.

[0102] Figure 4 A schematic diagram of the resource scheduling framework based on the improved actor-critic algorithm provided in this application embodiment is shown below. Figure 4As shown, it mainly includes two interactive modules: an environmental perception and control module and an agent decision-making module. Its basic workflow is as follows: the agent decision-making module generates a decision action based on the perceived environmental state; the environmental perception and control module executes the action and returns a new environmental state and corresponding reward; the agent continuously optimizes its decision-making strategy based on feedback, forming a closed loop of learning and optimization.

[0103] The environmental perception and control module includes at least a multi-dimensional heterogeneous computing power model and a directed acyclic graph (DAG) construction model. The multi-dimensional heterogeneous computing power model determines the real-time status of all computing nodes, including the availability of their computing, communication, and storage capabilities. The DAG construction model establishes the graph structure of the currently scheduled tasks, including the completion status and dependencies of each subtask. The scheduling controller is the interface connecting the intelligent agent to the physical / virtual environment. It is configured to perform the following functions:

[0104] Execution Action: Receive and parse scheduling actions from the agent decision-making module. In this embodiment, an action It can be defined as a tuple < , > indicates that the subtask is in the ready state. Assigned to compute nodes implement.

[0105] Perception Feedback: After performing an action, observe and collect changes in the environment to generate feedback information. This feedback information includes at least: the new system state. This includes the updated DAG graph state (e.g., task). (becoming in execution state) and compute nodes Resource usage status.

[0106] Instant rewards Or, the final reward R: The reward is a scalar signal that measures the quality of an action. In this embodiment, the reward is calculated directly based on the cost function defined above. For example, a small immediate reward (such as a term related to resource utilization) can be given after each scheduling step, and a final reward can be given after the entire DAG is completed, based on the final... and Calculate a total reward R = −J.

[0107] The agent decision-making module is the core of intelligent scheduling. Internally, it's built on an actor-critic framework and consists of two cooperating deep neural networks. The actor network is the policy network, responsible for determining the action to be taken based on the current state. The input system state is... This state It is a feature vector containing information about the DAG graph and resources. The output is an action or a probability distribution π in the action space. | This is used to select the most suitable scheduling action at the moment. The critic network is a value network whose function is to evaluate the quality of the actions chosen by the actor network given a particular state. Input system state. Output the value estimate V(to the current state). This value represents the change from the state. Starting from the current strategy, the sum of future cumulative rewards is expected to be obtained. The output of the critic network is not directly used for decision-making, but rather to generate a judgment signal to guide the update of the actor network.

[0108] To improve the training stability and convergence efficiency of the standard Actor-Critic algorithm on complex scheduling problems, this embodiment introduces the following two optimization mechanisms, which together constitute the core of the Proximal Policy Optimization (PPO) algorithm: (1) Policy update based on objective function pruning. When updating the actor network, PPO does not directly use gradient maximization for updating, but instead uses a pruned objective function. This mechanism limits the change in the new and old policies (i.e., limits the change in the objective function). / (2) Generalized Advantage Estimation (GAE): Advantage function A( , Used to measure actions Relative to the state How good is the average performance? This invention uses GAE to calculate the dominance function. By exponentially weighting the time-series difference errors over multiple future steps, GAE achieves a better balance between bias and variance, thus providing a more accurate and stable gradient signal for updating the actor network.

[0109] To apply the aforementioned reinforcement learning scheduling method, the scheduling problem must first be formalized as a Markov Decision Process (MDP). An MDP is defined by a triplet consisting of a state space (S), an action space (A), and a reward function (R).

[0110] The state space S provides all the information needed for the agent to make decisions. In this embodiment, a complete state St is a feature vector, which is concatenated from the following parts to comprehensively and accurately reflect the characteristics of the current system and the tasks to be scheduled:

[0111] Task status pending scheduling ( This section describes the attributes of several "ready" subtasks that need to be scheduled. For each ready subtask (a node in the DAG)... Its state includes resource constraints, such as the amount of computing, communication, and storage resources required to execute the task; the structure of this vector corresponds to the heterogeneous computing power model defined earlier; and execution constraints, such as the estimated maximum execution time. and data dependency information, such as communication constraints of preceding data streams. (Data volume, maximum latency, etc.)

[0112] Resource and environmental status ( This describes the real-time status of all available resources in the entire Energy Industrial Internet. This includes the available capabilities of each computing node k, i.e., the available computing power. , communication and storage Resources. Communication quality along critical paths in the network, such as end-to-end latency, jitter, and available bandwidth.

[0113] Task dependency state ( ): Describes the execution progress of the current business DAG, using a graph feature vector to represent which tasks in the current DAG have been completed, which are in the ready state, and the topological relationships between them.

[0114] Therefore, a complete state vector can be represented as S=[ , , This vector will be used as input to the actor network and critic network in the agent decision-making module.

[0115] Action space A defines the set of all possible decisions an agent can make at each decision-making moment. In this scheduling problem, an action... Defined as a scheduling decision, that is, a currently executable subtask. It is assigned to a specific heterogeneous computing node k for execution. =( ,k).

[0116] in, It is selected from the set of all currently ready tasks (i.e., all predecessor tasks have been completed) that are yet to be scheduled. It is selected from all available heterogeneous computing nodes. This is a high-dimensional discrete action space.

[0117] Step S304: Set a reward function for the directed acyclic graph of the deterministic business that has been scheduled and complete, and train the agent decision-making module in the agent according to the reward function.

[0118] In this embodiment, the reward function R is a scalar signal that guides the agent to learn the optimization direction, and is designed according to the cost function defined above. Since it is difficult to immediately assess the impact of a single task on the final end-to-end latency after scheduling, we mainly adopt a sparse reward mechanism, that is, after a complete business DAG is scheduled, a one-time reward is given based on the final execution result.

[0119] For a scheduled business DAG, the reward is... Defined as the inverse of the optimization objective, since the goal of reinforcement learning is to maximize the reward, while the original goal is to minimize a function. :

[0120]

[0121] in, This is the end-to-end total delay of the DAG. This represents the average resource utilization rate during its execution, with 'a' and 'b' representing the corresponding weights. This design directly guides the agent to learn strategies that reduce end-to-end latency while appropriately improving resource utilization. Furthermore, to encourage the agent to complete scheduling as quickly as possible, a small negative reward (e.g., ...) can be given at each scheduling step (each action executed). =-0.01).

[0122] Step S305: Update the network parameters of the actor network based on the generalized advantage estimation strategy and the objective function pruning strategy, and update the network parameters of the critic network based on the loss function to obtain the trained agent.

[0123] In this embodiment, the actor network and critic network within the agent decision-making module are trained and updated using an algorithm based on proximal policy optimization (PPO). The update mechanisms of the two networks will be described in detail below:

[0124] First, we learn and approximate the state value function V(s). Assume the input state... Output a scalar, namely the state value V( The critic network minimizes the mean squared error (MSE) loss function relative to the target value. This is used for updating. The target value is typically based on actual observed rewards and advantage estimates. Its loss function can be simplified as follows:

[0125]

[0126] in, It is the target value at time t, usually calculated by GAE.

[0127] The actor network is configured to learn an optimal scheduling strategy. (a|s) represents a probability distribution that generates a path to the action space A given state s. Input state The output determines the probability of selecting each possible action. The actor network uses the Clipped Objective (PPO) function to update the policy, ensuring the stability of policy learning. PPO guarantees training stability by limiting the magnitude of change between the old and new policies. Its core idea is to compute the current policy... Compared to the old strategy In action The probability ratio of :

[0128]

[0129] The optimization objective of the actor network is to maximize a pruned objective function. This function prevents excessive policy changes from occurring in a single update by limiting the range of probability ratios. Its formal expression is as follows:

[0130]

[0131] in, It is the importance sampling ratio, which reflects the ratio of the probability of the new strategy and the old strategy taking action at time t; It is the dominant function; It is a clipping function that restricts x to the interval Inside, that is, if If, then return a; if If so, then return b; These are pre-defined hyperparameters that define the range of pruning. This pruning operation limits the magnitude of policy updates, preventing excessive updates from damaging already learned good policies, thus improving the stability of the policy learning process while ensuring continuous policy optimization.

[0132] Combining GAE with the objective function pruning mechanism, the resulting optimization objective can be written as:

[0133]

[0134] This combination fully leverages the advantages of both approaches. On one hand, Generalized Dominance Estimation (GAE) provides a more accurate and less varianced estimate of the dominance function, enabling policy updates to be based on more reliable information. On the other hand, the objective function pruning mechanism limits the scope of policy updates, preventing algorithm instability caused by fluctuations in the dominance function estimate or over-updating of the policy. The synergistic effect between the two, in complex scenarios such as deep learning task scheduling, can more effectively reduce variance, improve the stability of policy learning, and enable the algorithm to perform better and more robustly approach the optimal policy.

[0135] For example, the overall training process of the collaborative scheduling method can be summarized as an iterative process including data acquisition and network updates: Initialize the network parameters of the actor network and the critic network, as well as an initial experience replay buffer, which stores training experience data; within each training cycle, for the directed acyclic graph of the deterministic business to be scheduled and the current system resource state, generate scheduling decisions for each subtask, and store the training experience data corresponding to each scheduling decision in the experience replay buffer; if the amount of stored training experience data exceeds a preset value, determine the advantage function estimate and state value objective for each time step based on the generalized advantage estimation strategy, and update the network parameters of the actor network by maximizing the pruning objective function of the near-end policy optimization algorithm based on the advantage function estimate and state value objective; update the network parameters of the critic network by minimizing the mean squared error loss function; iteratively update the network parameters of the actor network and the critic network until the policy performance of the actor network converges, or reaches a preset number of iterations, to obtain the trained agent. Specifically, it may include the following steps:

[0136] Step 1): Initialize the actor network Network parameters and the network of critics Network parameters In addition, an experience replay buffer is initialized to store training experience data.

[0137] Step 2): Interact with the environment and collect training data. Within a training cycle, repeat the following operations to collect data; start a new round: obtain a business DAG to be scheduled from the environment, and construct an initial state vector based on the DAG and the current system resource state. Execution round: In the current round, starting from the initial time t=0, for each decision step:

[0138] a) The actor network of the intelligent agent Based on the current state Output the probability distribution of actions and sample a scheduling action from it. b) Environmental interaction: The scheduling controller executes actions. After execution, the environment transitions to the next state. It returns a reward signal based on a preset reward function. c) Data storage: The transition sample tuple generated in this interaction ( , , , Store in the experience replay buffer. End round: The current round ends when all subtasks in the business DAG have been successfully scheduled and executed, or when the preset maximum number of decision steps has been reached.

[0139] Step 3): Update network parameters. Once a sufficient amount of trajectory data (e.g., one or more complete rounds) has accumulated in the experience replay buffer, perform a centralized update of the network parameters: calculate the learning objective, and for each trajectory acquired in the buffer, use the generalized advantage estimation (GAE) method to calculate the advantage function estimate at each time step t. and state value objectives .

[0140] Update the actor network to maximize the PPO clipping objective function mentioned earlier. (θ) to update the actor network The parameters. Update the commentator network: Use an optimizer (e.g., the Adam optimizer, by minimizing the mean squared error loss function mentioned above). To update the commentator network parameters .

[0141] Step 4): Iteration and Convergence. Clear the data in the experience replay buffer and repeat steps two and three. This iterative process continues until the policy performance of the actor network converges (e.g., the average reward no longer increases significantly over multiple consecutive rounds in the test environment), or the preset maximum number of training epochs is reached.

[0142] Through the above training process, the agent's decision-making module is ultimately able to learn a superior scheduling strategy, which is then embedded in the actor network. The parameters can be used for efficient and reliable real-time scheduling of new business DAGs during actual deployment.

[0143] Step S306: Based on the directed acyclic graph corresponding to the deterministic service and the real-time state of each computing node, the trained agent outputs the scheduling decision for the deterministic service.

[0144] For the specific implementation process and technical effects of step S306 in this embodiment, please refer to [link / reference needed]. Figure 1 The relevant description of step S103 in the method embodiment shown will not be repeated here.

[0145] This embodiment overcomes the limitations of traditional scheduling methods in complex heterogeneous environments by combining deep reinforcement learning with refined system modeling. It achieves end-to-end collaborative scheduling of computing and communication resources, breaking down resource silos and effectively guaranteeing end-to-end Service Level Agreements (SLAs). Existing technologies typically separate the scheduling of computing and network resources, easily leading to system bottlenecks. This invention models specific business processes as directed acyclic graphs (DAGs) and unifies the computing, communication, and storage capabilities of underlying heterogeneous nodes into a multi-dimensional computing power model, thereby constructing a global, cross-domain resource and business view for the scheduling agent. Therefore, when making decisions, the agent can jointly evaluate the execution cost of a subtask on a certain computing node with the transmission cost of the preceding and following data flows along the network path. This collaborative scheduling mechanism, which combines the business model with the underlying resource model, fundamentally solves the performance bottleneck problem caused by resource fragmentation and is the technical foundation for achieving end-to-end service level agreement (SLA) guarantees.

[0146] This embodiment also possesses dynamic adaptive and intelligent optimization capabilities, enabling robust adaptation to complex and ever-changing energy network environments. Existing scheduling methods based on fixed rules or analytical models suffer from poor flexibility and robustness when facing dynamic environments such as network congestion and changes in node load. The reinforcement learning mechanism employed in this invention learns a mapping policy π(A|S) from environmental state S to the optimal action A. When environmental factors such as network link quality and available node resources change in real time, these changes are perceived by the agent as new state vectors S. The agent can automatically output scheduling decisions adapted to the new state without modifying algorithms or rules, based on the learned policy. This endogenous, state-feedback-based dynamic adaptive capability allows the solution to continuously provide near-optimal scheduling performance in complex and ever-changing environments, significantly improving system stability and intelligence.

[0147] This embodiment balances the assurance of business deterministic SLAs with the optimal utilization of system resources. In mission-critical scenarios such as energy, ensuring business determinism (e.g., low latency) is the primary goal, but this does not mean reserving resources indefinitely and inefficiently. The advantage of this invention lies in its carefully designed reward function, which organically combines the two goals of "ensuring SLAs" and "improving resource utilization." During training, the agent will not only experience "business timeouts" (… Excessive spending will result in severe penalties, and will also be punished for "efficient use of resources" ( Higher efficiency leads to positive incentives. This guides agents to explore a "smarter" scheduling strategy: to create "green channels" that meet the SLAs of critical business flows, while also making the most of fragmented resources to support other non-critical business flows, thereby maximizing the overall resource efficiency and economy of the system without sacrificing security certainty.

[0148] This embodiment improves the stability and convergence of the algorithm, making advanced scheduling algorithms practical for engineering applications. Basic reinforcement learning algorithms often struggle with engineering implementation when dealing with high-dimensional and complex scheduling problems due to unstable training and slow convergence. This embodiment explicitly adopts the industry-leading proximal policy optimization algorithm framework and combines it with Generalized Dominance Estimation (GAE) technology. The PPO algorithm, through its core pruning objective function mechanism, effectively limits the step size of policy updates, avoiding training collapse caused by excessively large updates, and significantly improving the algorithm's stability and robustness. Simultaneously, GAE accelerates the model's convergence speed by providing more accurate, low-variance dominance function estimation. These specific improvements at the algorithm level ensure that the technical solution of this invention is not only theoretically advanced but also highly efficient and reliable in engineering practice, effectively solving complex scheduling problems in the real world.

[0149] For example, in order to evaluate the performance of the heterogeneous computing power scheduling method based on the improved actor-critic algorithm, comparative experiments were conducted to evaluate the performance differences between the traditional actor-critic algorithm and the greedy algorithm and the improved algorithm that introduces generalized advantage estimation (GAE) and objective function pruning mechanism in terms of average execution latency, average latency variance and resource utilization, so as to verify the effectiveness of the improved algorithm.

[0150] For example, Figure 5 This is a schematic diagram showing the comparison of the average latency of different algorithms provided in the embodiments of this application, as shown in the figure. Figure 5As shown, this paper compares the average execution latency (AEL) during training for traditional heuristic-based scheduling algorithms, the actor-critic algorithm, and an improved algorithm incorporating generalized advantage estimation (GAE) and objective function pruning mechanisms. In the early stages of training, due to the algorithms not yet converging, the AEL of all three algorithms is relatively high. Subsequently, as training progresses, the performance of the three algorithms varies. While the traditional algorithm shows a decreasing trend in AEL, the rate of decrease is relatively slow, and there are still some fluctuations during training. This indicates that the traditional heuristic-based scheduling algorithm is limited by its own constraints during optimization, making it difficult to quickly and stably reduce latency. The Actor-Critic (AC) algorithm decreases in AEL slightly faster than the traditional algorithm, but some repeated fluctuations occur in the middle of training. This indicates that during training, the coordination between policy updates and value estimation in the AC algorithm is unstable, leading to repeated fluctuations in latency optimization results. The improved algorithm, by introducing generalized advantage estimation (GAE) and objective function pruning mechanism, has a significantly faster rate of decrease in average latency and tends to stabilize after a relatively small number of training rounds. Its fluctuation range is much smaller than that of the first two algorithms, indicating that the improved algorithm can learn a better scheduling strategy more quickly during training, thereby effectively reducing the average execution latency of tasks and improving the response performance of task scheduling.

[0151] For example, Figure 6 This is a schematic diagram comparing the average delay variance of different algorithms provided in the embodiments of this application, as shown in the figure. Figure 6 As shown, this diagram compares the average latency variance of three algorithms. For each algorithm, six data points represent the average latency variance for every ten rounds starting from the 40th training round. Average latency variance reflects the fluctuation in task execution latency; a smaller variance indicates more stable task execution latency. As shown in the figure, the variance of the traditional algorithm fluctuates significantly within the range of 0.01-0.82, especially in the third and fourth data sets, where the variance values ​​reach 0.42 and 0.82 respectively, indicating poor stability of task execution latency and a significant risk of fluctuation. The variance of the AC algorithm is generally in the range of 0.06-1.41, which is an improvement over the traditional algorithm. However, the variance value of 1.41 in the first data set indicates that the task execution latency fluctuates significantly in the early stages of training. It gradually stabilizes as training progresses, but still has some instability. In contrast, the improved algorithm performs the best. Its variance rapidly decreases to the range of 0.004-0.68 from the second data set onwards, especially in the last three data sets, where the variance is below 0.01, almost approaching zero. This means that the improved algorithm has minimal latency fluctuation during task execution, providing a more stable service quality for the system and greatly reducing the risk of task execution anomalies caused by latency fluctuations. In terms of stability, it is significantly better than the traditional algorithm and the AC algorithm.

[0152] For example, Figure 7 This is a schematic diagram showing the comparison of resource utilization of different algorithms provided in the embodiments of this application, as shown below. Figure 7 As shown, this demonstrates the difference in resource utilization between the traditional actor-critic algorithm and the improved algorithm, comprehensively calculating CPU utilization, GPU utilization, and memory utilization. The comparison shows that the improved algorithm performs better in utilizing all types of resources. Regarding CPU utilization, the improved algorithm consistently outperforms the traditional algorithm in each training round, with a significant average improvement, indicating that it can more efficiently allocate CPU resources to meet the computational demands of the task. For GPU utilization, the improved algorithm also demonstrates an advantage, with its values ​​generally higher than the traditional algorithm and less fluctuation, indicating that the improved algorithm can better leverage the parallel computing capabilities of the GPU, improving the execution efficiency of graphics processing and complex computational tasks. In terms of memory utilization, the improved algorithm also significantly outperforms the traditional algorithm, with stable and high utilization values ​​ensuring efficient storage and retrieval of task data.

[0153] The analysis of the above experimental results shows that the improved actor-critic algorithm based on generalized advantage estimation (GAE) and objective function pruning mechanism outperforms the traditional actor-critic algorithm in terms of average execution latency, average latency variance, resource utilization, and convergence speed. This fully demonstrates that introducing GAE and objective function pruning mechanism can significantly improve the algorithm's performance, making it more efficient and stable in task scheduling in heterogeneous computing environments, and providing a more effective solution for the management and scheduling of heterogeneous computing resources in intelligent computing centers.

[0154] To further verify the impact of the weight coefficients a and b in the objective function on the performance of the scheduling strategy, we designed an additional set of sensitivity analysis experiments based on the previous experiments. This experiment aims to explore the effects of different weight configurations on average latency, resource utilization, and the final objective function value, thereby evaluating the adjustability and practical guiding role of the objective function.

[0155] The objective function includes average task execution latency and resource utilization, where 'a' and 'b' represent the preference weights for optimizing execution latency and resource utilization, respectively. Theoretically, different weight configurations should lead to different trade-offs between latency and resource utilization in the strategy.

[0156] Based on a fixed number of training rounds, task graph structure, and system resource parameters, this experiment sets the following four weight combinations:

[0157] 1) a:b = 1:0: The sole objective is to reduce latency;

[0158] 2) a:b = 1:1: Delay is as important as resource utilization;

[0159] 3) a:b = 3:1: Prioritizing delay while also considering resource utilization;

[0160] 4) a:b = 1:3: Prioritize resources, tolerate appropriate delays.

[0161] Each weight combination trains the agent separately, and finally collects the average latency and resource utilization rate, and calculates the objective function value accordingly. In order to ensure that the two indicators of latency and resource utilization rate are comparable, the average latency is scaled (i.e., divided by 150) to ensure that the setting of weight coefficients a and b has practical significance.

[0162] Table 1 shows the impact of different weight combinations.

[0163]

[0164] The experimental results show that different weight combinations do indeed lead to differences in the optimization direction of the scheduling strategy. The target value fluctuates with changes in weights, indicating that the optimization function has good adjustability and practical guiding ability. It also reflects a typical "resource-latency" trade-off under different weight combinations, allowing users to flexibly configure preferences according to specific business scenarios, thereby achieving more targeted scheduling optimization strategies.

[0165] It should be understood that although the steps in the flowcharts of the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages in other steps. It is understood that the steps in different embodiments can be freely combined as needed, and all non-contradictory solutions formed by such combinations are within the scope of protection of this application.

[0166] Based on the same inventive concept, this application also provides a resource scheduling apparatus for deterministic services, which implements the resource scheduling method for deterministic services described above. The solution provided by this apparatus is similar to the implementation described in the above method. Therefore, the specific limitations of one or more resource scheduling apparatus embodiments for deterministic services provided below can be found in the limitations of the resource scheduling method for deterministic services described above, and will not be repeated here.

[0167] In one exemplary embodiment, such as Figure 8 As shown, a resource scheduling device for deterministic business operations is provided, comprising: a directed acyclic graph construction module 801, a node state determination module 802, and a scheduling decision generation module 803, wherein:

[0168] The directed acyclic graph construction module 801 is used to model deterministic business as a directed acyclic graph;

[0169] The node status determination module 802 is used to determine the real-time status of each computing node in a heterogeneous computing environment.

[0170] The scheduling decision generation module 803 is used to output scheduling decisions for deterministic services through a trained agent based on the directed acyclic graph corresponding to the deterministic service and the real-time status of each computing node.

[0171] For example, the directed acyclic graph (DAG) construction module 801 is specifically used to: divide the deterministic business into multiple independent subtasks, where each subtask is a basic unit for scheduling and execution and has its own resource requirement attributes, including resource type and resource quantity; construct a set of directed arcs based on the direct dependencies between the subtasks; and generate a directed acyclic graph of the deterministic business based on the set of subtasks and directed arcs, wherein each vertex in the directed acyclic graph represents a subtask of the deterministic business, and the connection relationships between the vertices in the directed acyclic graph represent the direct dependencies between the subtasks.

[0172] For example, the node state determination module 802 is specifically used to: construct a multi-dimensional heterogeneous computing power model, which is used to quantitatively evaluate the processing capabilities of heterogeneous nodes of different types and from different manufacturers; evaluate the real-time state of each computing node in the heterogeneous computing environment through the constructed multi-dimensional heterogeneous computing power model, the real-time state including: computing power, communication power, and storage power; wherein: computing power is associated with at least one of the following indicators of computing node: logical operation power, parallel computing power, and neural network computing power; communication power is associated with computing node's network bandwidth and / or memory bandwidth; and storage power is associated with computing node's storage bandwidth and / or number of read / write operations per second.

[0173] For example, the above-mentioned device may further include: an intelligent agent construction module 804, used to construct an intelligent agent, the intelligent agent including: an environment perception and control module and an intelligent agent decision-making module, the environment perception and control module is used to perceive the environment state, the intelligent decision-making agent adopts an actor network and critic network architecture, and is used to generate decision actions according to the environment state; the environment perception and control module is also used to execute the decision actions and return the updated environment state and the corresponding reward; wherein, the environment state includes: the real-time state of all computing nodes, and the completion state and dependencies of each subtask in the directed acyclic graph.

[0174] For example, the above apparatus may further include: a training module 805, used to set a reward function for the directed acyclic graph of the deterministic business that has been scheduled, and to train the agent decision-making module in the agent according to the reward function; to update the network parameters of the actor network based on the generalized advantage estimation strategy and the objective function pruning strategy, and to update the network parameters of the critic network based on the loss function, so as to obtain the trained agent.

[0175] For example, the training module 805 is specifically used for: initializing the network parameters of the actor network and the critic network, and an initial experience replay buffer, which is used to store training experience data; in each training cycle, generating scheduling decisions for each subtask based on the directed acyclic graph of the deterministic business to be scheduled and the current system resource state, and storing the training experience data corresponding to each scheduling decision in the experience replay buffer; if the amount of stored training experience data is greater than a preset value, determining the advantage function estimate and state value objective for each time step based on the generalized advantage estimation strategy, and updating the network parameters of the actor network by maximizing the pruning objective function of the proximal policy optimization algorithm based on the advantage function estimate and state value objective; updating the network parameters of the critic network by minimizing the mean squared error loss function; iteratively updating the network parameters of the actor network and the critic network until the policy performance of the actor network converges or reaches a preset number of iterations, thus obtaining the trained agent.

[0176] The modules in the aforementioned resource scheduling device for deterministic business operations can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can invoke and execute the operations corresponding to each module.

[0177] In one exemplary embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 9As shown, the computer device includes a processor, memory, input / output interfaces, a communication interface, a display unit, and an input device. The processor, memory, and input / output interfaces are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interfaces. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The input / output interfaces are used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, Near Field Communication (NFC), or other technologies. When executed by the processor, the computer program implements a resource scheduling method for deterministic business operations. The display unit is used to form a visually visible image and can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the casing of the computer device, or external keyboards, touchpads, or mice, etc.

[0178] Those skilled in the art will understand that Figure 9 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0179] In one exemplary embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the method steps described above.

[0180] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0181] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0182] The above embodiments are merely illustrative of several implementation methods of this application, and their descriptions are relatively specific and detailed. However, they should not be construed as limiting the scope of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A resource scheduling method for deterministic business operations, characterized in that, The method includes: Model deterministic business processes as directed acyclic graphs; Determine the real-time status of each computing node in a heterogeneous computing environment; Based on the directed acyclic graph corresponding to the deterministic service and the real-time state of each computing node, the trained agent outputs the scheduling decision for the deterministic service.

2. The method according to claim 1, characterized in that, The step of modeling deterministic business as a directed acyclic graph includes: The deterministic business is divided into multiple independent subtasks. Each subtask is a basic unit for scheduling and execution and has its own resource requirement attributes, including resource type and resource quantity. Based on the direct dependencies between the subtasks, construct a set of directed arcs; A directed acyclic graph (DAG) for the deterministic service is generated based on the set of subtasks and directed arcs, wherein each vertex in the DAG represents a subtask of the deterministic service, and the connections between the vertices in the DAG represent the direct dependencies between the subtasks.

3. The method according to claim 2, characterized in that, Determining the real-time status of each computing node in a heterogeneous computing environment includes: A multi-dimensional heterogeneous computing power model is constructed, which is used to quantitatively evaluate the processing capabilities of heterogeneous nodes of different types and from different manufacturers. The real-time status of each computing node in a heterogeneous computing environment is evaluated by constructing a multi-dimensional heterogeneous computing power model. The real-time status includes: computing power, communication power, and storage power; wherein: The computing power is associated with at least one of the following indicators: the logical operation capability, parallel computing capability, and neural network computing capability of the computing node; The communication capability is associated with the network bandwidth and / or memory bandwidth of the computing node; The storage capacity is related to the storage bandwidth of the computing node and / or the number of read / write operations per second.

4. The method according to any one of claims 1 to 3, characterized in that, Before the trained agent outputs the scheduling decision for the deterministic service based on the directed acyclic graph corresponding to the deterministic service and the real-time state of each computing node, the method further includes: An intelligent agent is constructed, comprising: an environmental perception and control module and an intelligent agent decision-making module. The environmental perception and control module is used to perceive the environmental state, and the intelligent agent decision-making module adopts an actor network and critic network architecture to generate decision actions based on the environmental state. The environmental perception and control module is also used to execute the decision actions and return the updated environmental state and corresponding reward. The environmental state includes: the real-time state of all computing nodes, and the completion status and dependencies of each subtask in the directed acyclic graph.

5. The method according to claim 4, characterized in that, After constructing the agent, the method further includes: A reward function is set for the directed acyclic graph of deterministic services that have been scheduled and completed, and the agent decision-making module in the agent is trained according to the reward function; The network parameters of the actor network are updated based on the generalized advantage estimation strategy and the objective function pruning strategy, and the network parameters of the critic network are updated based on the loss function, resulting in a trained agent.

6. The method according to claim 5, characterized in that, The process of updating the network parameters of the actor network based on the generalized advantage estimation strategy and the objective function pruning strategy, and updating the network parameters of the critic network based on the loss function, to obtain the trained agent, includes: Initialize the network parameters of the actor network and the critic network, as well as the initial experience replay buffer, which is used to store training experience data; Within each training cycle, for the directed acyclic graph of the deterministic business to be scheduled and the current system resource status, scheduling decisions for each subtask are generated, and the training experience data corresponding to each scheduling decision is stored in the experience replay buffer. When the amount of stored training experience data is greater than a preset value, the advantage function estimate and state value target for each time step are determined based on the generalized advantage estimation strategy. Based on the advantage function estimate and the state value target, the network parameters of the actor network are updated by maximizing the pruning objective function of the proximal policy optimization algorithm. The network parameters of the critic network are updated by minimizing the mean squared error loss function; The network parameters of the actor network and the critic network are iteratively updated until the policy performance of the actor network converges, or the preset number of iterations is reached, to obtain a trained agent.

7. A resource scheduling device for deterministic business operations, characterized in that, The device includes: The Directed Acyclic Graph (DAG) building module is used to model deterministic business processes as directed acyclic graphs. The node status determination module is used to determine the real-time status of each computing node in a heterogeneous computing environment. The scheduling decision generation module is used to output the scheduling decision of the deterministic service through the trained agent, based on the directed acyclic graph corresponding to the deterministic service and the real-time state of each computing node.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images