Task scheduling method and system for computing power network

Abstract

The invention provides a task scheduling method and system for a computing power network, and the method comprises the steps: representing a task scheduling problem of the computing power network based on a Markov decision process, and determining the state, action and scheduling reward of each time slot; and taking each candidate scheduling scheme when the scheduling reward corresponding to each task is maximum as each target scheduling scheme based on the state of each time slot, so as to execute scheduling of each task according to each target scheduling scheme. According to the invention, each task is scheduled by adopting multi-agent reinforcement learning, real-time dynamic task scheduling in the computing power network can be efficiently realized, and the computing power resource allocation efficiency and the computing power resource utilization rate of the real-time dynamic task are effectively improved, so that the network overhead is reduced, and the network performance is improved.

Description

Technical Field

[0001] The present invention relates to the field of computing power networks and reinforcement learning technologies, and in particular to a task scheduling method and system for a computing power network. Background Art

[0002] The Industrial Internet of Things (IIoT) is a crucial component of future industrial systems. By the end of 2024, the number of connections between machines will have increased from 5.6 billion in 2016 to 27 billion. With the rapid development of artificial intelligence (AI) and large-scale models, the integration of AI models with IIoT systems has become a key research area, providing unprecedented opportunities for optimizing industrial processes and improving operational efficiency. It is well known that AI models require vast amounts of data, and data volume is growing at an unprecedented rate. It is estimated that global data volume will reach 163 zettabytes in 2025, with a compound annual growth rate of 20%. This exponential growth in data has led to a severe shortage of computing power in traditional data centers. Some large-scale heterogeneous computing resources may be distributed across the cloud, edge, and edge devices. Isolated computing resources lead to low resource utilization and poor service performance, necessitating the integration of edge and edge computing resources to meet future communication needs.

[0003] Fortunately, recent advances in computing power networks (CPNs) offer an effective solution. CPNs leverage networks to connect widely distributed computing resources, achieving a deep integration of computing and networking. Through unified resource coordination and network control, they perceive computing resources of different types and locations and provide them to users in a unified manner, effectively improving service performance and resource utilization. Given these advantages, they have garnered widespread attention from both industry and academia. Task scheduling is a key method for computing power networks to optimize resource allocation and load balancing, ensuring efficient utilization of computing resources and shortening response times. Many studies have employed deep reinforcement learning (DRL) or heuristic algorithms to implement task scheduling in dynamic environments. However, due to the significant increase in computing power requirements caused by the large number of AI applications in industrial IoT environments, which fluctuate in real time, existing task scheduling methods are unable to quickly adapt to the dynamically changing task volume and resource requirements, resulting in uneven load or resource waste.

[0004] Moreover, with the deployment of IIoT in smart factories, privacy and security issues and challenges have also become the focus of research. However, existing task scheduling methods do not consider the reliability and security issues between IoT devices. Summary of the Invention

[0005] In view of this, an embodiment of the present invention provides a task scheduling method and system for a computing power network to eliminate or improve one or more defects in the prior art.

[0006] One aspect of the present invention provides a method for scheduling tasks in a computing network, the method comprising the following steps:

[0007] The task scheduling problem of the computing power network is characterized based on the Markov decision process, and the state, action and scheduling reward of each time slot are determined, wherein the state includes the task publishing requirements of each task and the task forwarding requirements of each node, the initial task queue length and the updated reputation score; the action includes all candidate scheduling schemes for each task, and the candidate scheduling schemes include a source node for receiving tasks published by the task publisher and at least one destination node for receiving tasks forwarded by the source node and processing the tasks; the scheduling reward includes a reward based on the task delay of each task, the task completion rate and the immediate cost of each task publisher, the task delay includes the delay obtained based on the task publishing requirements, the task forwarding requirements of the source node and the initial task queue length of at least one destination node, and the immediate cost includes the cost obtained based on the task publishing requirements of each task published by the task publisher, the task forwarding requirements of each source node and the updated reputation score of each destination node;

[0008] Based on the status of each time slot, each candidate scheduling scheme when the scheduling reward corresponding to each task is maximum is used as each target scheduling scheme, so as to execute the scheduling of each task according to the each target scheduling scheme.

[0009] In some embodiments of the present invention, the method further comprises:

[0010] In each time slot, the updated reputation score, communication capability, and computing capability of each node in the computing power network in the previous time slot are obtained, and the trust score of each node is obtained based on the updated reputation score, communication capability, and computing capability of each node in the previous time slot. A trusted node set is formed by multiple nodes whose trust scores are greater than a preset trust threshold; the candidate scheduling scheme is determined based on the trusted node set.

[0011] In some embodiments of the present invention, the updated reputation score is obtained based on influencing factors, including timeliness of interaction between the task publisher and the destination node, positive interaction or negative interaction, and credibility of the task publisher.

[0012] In some embodiments of the present invention, the updated reputation score is obtained based on the reputation score of the destination node by each task publisher that interacts with the destination node and the credibility of each task publisher, wherein the reputation score is obtained by taking the weighted average of each historical interaction evaluation score and the corresponding freshness weighting coefficient and each interaction weighting coefficient given to the destination node by the task publisher that interacted with the destination node at least once before the corresponding time slot, the interaction is a positive interaction or a negative interaction, the interaction weight coefficient corresponding to the negative interaction is greater than the interaction weight coefficient corresponding to the positive interaction, and the freshness weighting coefficient is obtained based on historical interaction rounds and historical interaction times; the credibility is obtained by the reputation score of each destination node interacting with any one of the task publishers by each task publisher that interacts with the destination node.

[0013] In some embodiments of the present invention, the task publishing requirements include the transmission power, size, bandwidth resource requirements and computing resource requirements of the published tasks; the task forwarding requirements include the transmission power, size, bandwidth resource requirements and computing resource requirements of the forwarded tasks; the initial task queue length is obtained based on the initial task queue length of the previous time slot, the total computing load of all newly arrived forwarded tasks in the task queue and the total computing load of processing all forwarded tasks.

[0014] In some embodiments of the present invention, the task delay includes a first transmission delay from the task publisher to the source node, a second transmission delay from the source node to at least one destination node, a queuing delay and a task processing delay, wherein the first transmission delay is obtained based on the transmission rate obtained from the transmission power and bandwidth resource requirements of the published task received by the source node, the second transmission delay is obtained based on the transmission rate obtained from the transmission power and bandwidth resource requirements of the forwarding task received by each of the at least one destination nodes, the queuing delay is obtained based on the initial task queue length of each of the at least one destination nodes, the computing resource requirements of each task in the initial task queue, and the computing resource requirements and size of each forwarding task received, and the task processing delay is obtained based on the size and computing resource requirements of the forwarding task received by each of the at least one destination nodes.

[0015] In some embodiments of the present invention, the task completion rate is the number of tasks completed within a preset task delay threshold.

[0016] In some embodiments of the present invention, the instantaneous cost includes task transmission cost and task processing cost. The task transmission cost is obtained based on the size and bandwidth resource requirements of each published task received by each source node and the size of each forwarded task received by each destination node. The task processing cost is obtained based on the size and computing resource requirements of each forwarded task received by each destination node and the updated reputation score of each destination node.

[0017] Another aspect of the present invention provides a task scheduling system for a computing power network, the system comprising: a computer device, the computer device comprising a processor and a memory, the memory storing computer instructions, the processor being used to execute the computer instructions stored in the memory, and when the computer instructions are executed by the processor, the system implements the steps of the aforementioned method.

[0018] Another aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, which is used to implement the steps of the aforementioned method when executed by a processor.

[0019] Another aspect of the present invention provides a computer program product comprising computer instructions, which implement the steps of the above method when executed by a processor.

[0020] The task scheduling method and system for computing networks of the present invention utilize multi-agent reinforcement learning to schedule individual tasks. This method efficiently implements real-time dynamic task scheduling within the computing network and effectively improves the efficiency of computing resource allocation and utilization for real-time dynamic tasks, thereby reducing network overhead and enhancing network performance. Furthermore, by determining candidate and optimal scheduling strategies for each task based on a set of trusted nodes, the risk of privacy leakage of user task data can be effectively reduced, improving privacy and security, and enhancing system reliability and stability.

[0021] Additional advantages, objects, and features of the present invention will be set forth in part in the following description and will become apparent to those skilled in the art upon examination of the following or may be learned from practice of the present invention. The objects and other advantages of the present invention may be realized and obtained by the structures particularly pointed out in the description and drawings.

[0022] Those skilled in the art will understand that the purposes and advantages that can be achieved by the present invention are not limited to the above specific descriptions, and the above and other purposes that can be achieved by the present invention will be more clearly understood based on the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The drawings described herein are used to provide a further understanding of the present invention, constitute a part of this application, and do not constitute a limitation of the present invention.

[0024] Figure 1 This is a schematic diagram of the architecture of a task scheduling system for a computing network in one embodiment of the present invention;

[0025] Figure 2 Schematic diagram of a task scheduling method for a computing network according to an embodiment of the present invention;

[0026] Figure 3 Schematic diagram of the principle of agent reinforcement learning in one embodiment of the present invention;

[0027] Figure 4 2 is a flow chart of a method for scheduling tasks in a computing network according to another embodiment of the present invention. DETAILED DESCRIPTION

[0028] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the embodiments and the accompanying drawings. Here, the exemplary embodiments of the present invention and their descriptions are used to explain the present invention, but are not intended to limit the present invention.

[0029] It should also be noted that, in order to avoid obscuring the present invention due to unnecessary details, the accompanying drawings only show structures and / or processing steps closely related to the solutions according to the present invention, while other details that are not closely related to the present invention are omitted.

[0030] It should be emphasized that the term "include / comprises" when used herein refers to the existence of features, elements, steps or components, but does not exclude the existence or addition of one or more other features, elements, steps or components.

[0031] It should also be noted that, unless otherwise specified, the term "connection" herein may refer not only to a direct connection but also to an indirect connection involving an intermediate.

[0032] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. In the accompanying drawings, the same reference numerals represent the same or similar components, or the same or similar steps.

[0033] Figure 1 FIG. 1 is a schematic diagram of the architecture of a task scheduling system for a computing network according to an embodiment of the present invention. Figure 1As shown, the system architecture used in the present invention can include an application layer and a blockchain layer. The application layer includes an Industrial Internet of Things (IIoT) network and a computing network. The IIoT network includes multiple IIoT devices. The computing network can be a computing network established based on the IIoT environment and includes base stations with edge computing capabilities distributed at the edge and cloud computing centers distributed in the cloud. The IIoT devices serve as end nodes, the base stations serve as edge nodes, and the cloud computing centers serve as cloud nodes. Each node is interconnected and communicates with each other via a wireless communication network. End nodes are task issuers, typically devices such as sensors, and do not have their own computing resources. Therefore, when encountering tasks that require computing resources, the task needs to be quickly transferred to a closer or nearest edge node in the computing network for processing. Both edge nodes and cloud nodes have computing resources for processing computing tasks and are therefore also referred to as computing network nodes (CPN nodes). The difference is that in this computing network, cloud nodes have more computing resources than edge nodes and can handle more complex computing tasks. For task scheduling in an IIoT network environment, real-time task data processing, high reliability, and resource utilization efficiency are crucial. The computing power network can also be a general network that does not specifically distinguish between edge nodes and cloud nodes and includes multiple computing power nodes that are interconnected through a wireless network and have their own computing power resources.

[0034] In this invention, Multi-Agent Deep Reinforcement Learning (MADRL) technology is used to treat each CPN node and each task publisher as an independent agent, achieving unified multi-agent task scheduling. The task scheduling of the present invention includes two scheduling phases. In the first scheduling phase, each task publisher publishes different computing tasks at different time slots and, as an agent, makes scheduling decisions, deciding which edge node (or general CPN node) to request computing resources from and which edge node (or general CPN node) to transmit the published computing tasks to for execution. In the second scheduling phase, after receiving the computing tasks from the end nodes, the edge nodes (or general CPN nodes) located at the edge side make another decision as agents, deciding whether to process the computing tasks locally or forward them to another edge node or cloud node (or other general CPN node) for processing. This is done to prevent too many computing tasks from being forwarded to the same edge node (or general CPN node) at the same time, resulting in computing tasks not being processed in a timely manner. In addition, with the gradual development of AI technology and large models, the task size and computing resource requirements generated by actual applications in the industrial field are also increasing. Some tasks with large computing resource requirements need to be transferred to cloud nodes (or general CPN nodes with more computing resources) for processing or transferred to multiple edge nodes (or multiple general CPN nodes) for joint processing.

[0035] After an edge node or cloud node (or general CPN node), i.e., a destination node, completes a computing task, the task publisher will evaluate the destination node's processing quality for the computing task and, based on the node's interaction history and past reputation, give the node an updated reputation score. Finally, the updated reputation score is stored in the blockchain layer, ensuring accuracy and immutability, and is maintained by all nodes in the computing network. Therefore, the present invention establishes and maintains a reputation model in the blockchain so that all nodes in the computing network can obtain the updated reputation score of each node in the previous time slot within each time slot and promptly update the reputation score of each node after the task execution in each time slot is completed.

[0036] Figure 2 FIG. 1 is a flow chart of a task scheduling method for a computing network according to an embodiment of the present invention. Figure 2 As shown, the method includes the following steps:

[0037] Step S210, characterizes the task scheduling problem of the computing power network based on the Markov decision process, and determines the state, action and scheduling reward of each time slot, wherein the state includes the task publishing requirements of each task and the task forwarding requirements of each node, the initial task queue length and the updated reputation score; the action includes all candidate scheduling schemes for each task, and the candidate scheduling schemes include a source node for receiving tasks published by the task publisher and at least one destination node for receiving tasks forwarded by the source node and processing the tasks; the scheduling reward includes a reward based on the task delay of each task, the task completion rate and the immediate cost of each task publisher, the task delay includes the delay obtained based on the task publishing requirements, the task forwarding requirements of the source node and the initial task queue length of at least one destination node, and the immediate cost includes the cost obtained based on the task publishing requirements of each task published by the task publisher, the task forwarding requirements of each source node and the updated reputation score of each destination node.

[0038] Task publishers are selfish and rational, each selecting a CPN node to minimize its long-term cost Ci. This method defines the task scheduling process in a computing network as a total scheduling process, including a first-stage scheduling phase in which each CPN node receives tasks issued by each task publisher, and a second-stage scheduling phase in which each CPN node forwards each received task to its corresponding CPN node for processing. In the second scheduling phase, the CPN node that receives a task from a task publisher can process the task itself or forward it to one or more other CPN nodes for joint processing. For example, if forwarding a task to two other CPN nodes for joint processing, part of the task data can be forwarded to one CPN node, and the rest to the other CPN node. The CPN node that receives the task issued by the task publisher is considered the source node, and the CPN node that receives and processes the task issued by the task publisher or forwarded by other CPN nodes is considered the destination node. If the source node makes a decision to process the task received from the task publisher locally, the source node is converted to a destination node, or is regarded as a destination node, that is, when the source node and the destination node are the same or different, the task received by the destination node can be regarded as forwarded by the source node. In addition, in the industrial Internet of Things environment, the task completion rate Ti and the delay (task delay) Di are key indicators for evaluating the quality of computing services provided by CPN nodes. It is a challenging task to achieve effective task scheduling, that is, high task completion rate and low delay, to improve the service quality of the entire computing power network environment while maintaining low costs for task publishers. At any time, the cost and delay experienced by each task publisher depends on the current state of the CPN environment and the operations of other task publishers in the industrial Internet of Things network environment, and is independent of the previous state. In the next time period or time slot, the CPN environment randomly switches to a new state. Therefore, the present invention models the task scheduling problem of the computing power network as a Markov Decision Process (MDP) with state, action and reward. Among them, the action can be expressed as To express, where π i Contains all candidate scheduling schemes or strategies for a task published by task publisher i, Represents the number of task publishers in the network. When determining candidate scheduling strategies for a task, the constraint that the channel bandwidth allocated to the task during its wireless transmission is less than the total channel bandwidth of the computing network must be met. Rewards are determined using a reward function that is influenced by factors such as optimizable task latency, task completion rate, and cost, assigning specific weights to these factors. This transforms the optimization problem of minimizing cost and task latency while maximizing task completion rate into the optimization problem of maximizing the reward function over a long period of time using the optimal scheduling strategy.

[0039] In some embodiments, the task publishing requirements include the transmission power, size, bandwidth resource requirements, and computing resource requirements of the published task; the task forwarding requirements include the transmission power, size, bandwidth resource requirements, and computing resource requirements of the forwarded task; and the initial task queue length is obtained based on the initial task queue length of the previous time slot, the total computing load of all newly arrived forwarding tasks in the task queue, and the total computing load of processing all forwarding tasks. The total computing load of all newly arrived forwarding tasks can be obtained based on the size and computing resource requirements of each newly arrived forwarding task, and the total computing load of processing all forwarding tasks can be obtained based on the size and computing resource requirements of each forwarding task.

[0040] Specifically, each CPN node has a task queue. Tasks that have arrived but have not yet been executed and processed need to queue in the task queue and wait for processing by the CPN node. Tasks arriving in the task queue can include tasks received directly from task publishers and tasks forwarded by other CPN nodes. Therefore, the total computational load of all newly arrived tasks in the task queue of CPN node n as the destination node in time slot t (t = 1, 2, ...) is θ n (t) can be expressed as:

[0041]

[0042] Among them, i represents the serial number of the task publisher, It also represents the set of all task publishers in the network. represents the task μ published by task publisher i i the size of (t), Indicates that the task μ published by task publisher i i (t) Scheduling decision from source node to destination node, τ i represents the task μ published by task publisher i i (t) The computing resource requirements. After the time slot t ends, the changes in the task queue are as follows:

[0043]

[0044] in, represents the initial task queue length of CPN node n in time slot t, Ω n (t) represents the total computational load of all tasks processed by CPN node n in time slot t (the calculation formula is the same as θ n (t) similar), Indicates the upper limit of the task queue length of CPN node n. When the changed task queue length exceeds the upper limit of the task queue length of CPN node n, the excess task data will be discarded. The changed task queue length of CPN node n in time slot t As the initial task queue length for the next time slot.

[0045] In some embodiments, the task delay includes a first transmission delay from the task publisher to the source node, a second transmission delay from the source node to at least one destination node, a queuing delay, and a task processing delay, wherein the first transmission delay is obtained based on the transmission rate obtained from the transmission power and bandwidth resource requirements of the published task received by the source node, the second transmission delay is obtained based on the transmission rate obtained from the transmission power and bandwidth resource requirements of the forwarding task received by each of the at least one destination nodes, the queuing delay is obtained based on the initial task queue length of each of the at least one destination nodes, the computing resource requirements of each task in the initial task queue, and the computing resource requirements and size of each forwarding task received, and the task processing delay is obtained based on the size and computing resource requirements of the forwarding task received by each of the at least one destination nodes.

[0046] Specifically, in the first scheduling phase, task publisher i will publish task μ in time slot t. i (t) The stage of transmission to the selected CPN node m as the source node, the scheduling decision of the first scheduling stage can be defined as Considering the complexity and variability of the actual network environment, and the existence of path loss and Rayleigh fading, according to the Shannon formula, task publisher i publishes task μ to CPN node m in time slot t. i Transmission rate at (t) It can be calculated as:

[0047]

[0048] Among them, w i (t) represents the channel bandwidth resource requirement sent by task publisher i in time slot t; p i Indicates that task publisher i transmits task μ i (t) is the transmission power of task publisher i or task μ i (t) is the transmission power; h is the channel gain, represents the path loss determined by the network environment and distance, σ 2 represents the noise power, represents the set of all CPN nodes in the computing network, and w represents the total channel bandwidth resources of the computing network. Therefore, in time slot t, task publisher i sends task μ to CPN node m i The first transmission delay It can be calculated as:

[0049]

[0050] In the second scheduling phase, the source node will receive the task μ in time slot t. i (t) Forwarding to at least one selected destination node (CPN node n), the scheduling decision of the second scheduling stage can be defined as CPN node m transmits the received task directly to at least one adjacent CPN node n through the wireless network. This point-to-point transmission method reduces the dependence on intermediate nodes. In time slot t, when a task μ i (t) The transmission rate from CPN node m to CPN node n when it is scheduled from CPN node m to CPN node n It can be expressed as follows:

[0051]

[0052] Among them, w m (t) represents the channel bandwidth resource requirement sent by CPN node m in time slot t, p m represents the transmission task μ of CPN node m i (t), i.e., the transmission power of the source node or task μ i Therefore, in time slot t, CPN node m forwards task μ to CPN node n. i The second transmission delay It can be calculated as:

[0053]

[0054] Computation task μ i (t) After arriving at the task queue of CPN node n, it may need to wait in line until it is its turn to be processed. The waiting delay is composed of two parts: one is the delay caused by processing the tasks accumulated previously (i.e., in the previous time slot) in the task queue; It can be expressed as follows:

[0055]

[0056] Among them, f n (t) represents the computing power of CPN node n, that is, all the computing resources it has; τ represents the computing resource requirements of each task in the initial task queue (tasks issued by each task publisher, which may include tasks issued by task publisher i). The other part is when there are multiple new tasks in the task queue of CPN node n within time slot t, task μ i (t) needs to wait for processing in the queue order, that is, task μi (t) Average waiting time before being processed It can be calculated as:

[0057]

[0058]

[0059] Among them, q j,n (t) represents all newly arrived tasks in the task queue of CPN node n in time slot t, except for task μ i The total computational load of each new task other than (t), j represents the set of all task publishers The serial numbers of each task publisher except task publisher i, represents the size of the task published by task publisher j, τ j represents the computing resource requirements of the task issued by task publisher j. That is, the queuing delay can be and The added value.

[0060] Task processing also has a certain delay, task processing delay It is mainly related to the computing resources and size required for the task and the computing power of the corresponding destination node. The calculation formula can be:

[0061]

[0062] Therefore, in this method, the scheduling task μ i (t) The generated task delay may be the sum of the first transmission delay, the second transmission delay, the queuing delay and the task processing delay.

[0063] Step S220 , based on the status of each time slot, each candidate scheduling scheme when the scheduling reward corresponding to each task is maximum is used as each target scheduling scheme, so as to execute the scheduling of each task according to the each target scheduling scheme.

[0064] Figure 3 FIG. 1 is a schematic diagram of the principle of agent reinforcement learning according to an embodiment of the present invention. Figure 3As shown, this method employs a multi-agent deep reinforcement learning (MADRL) algorithm to solve the aforementioned reward maximization optimization problem. Specifically, each CPN node acts as an agent. Through intelligent decision-making and continuous interaction with the computing network environment, it adjusts its task scheduling strategy or plan to obtain higher rewards. In other words, each agent optimizes the computing network's task completion rate, task delay, and task cost by continuously making scheduling decisions in consecutive time slots. Each CPN node determines the relevant actions for task scheduling, and the computing network environment provides feedback to the CPN node based on its performance in handling the scheduled tasks. In the pursuit of higher rewards, each agent gradually develops more effective task scheduling strategies or plans, and their ultimate rewards converge. Based on the above steps, by using a multi-agent deep reinforcement learning algorithm for task scheduling, efficient real-time dynamic task scheduling can be achieved in the computing network, effectively improving the efficiency and utilization of computing resource allocation for real-time dynamic tasks, thereby reducing network overhead and improving network performance.

[0065] Figure 4 FIG. 1 is a flow chart of a task scheduling method for a computing network in another embodiment of the present invention. Figure 4 As shown, in this embodiment, the method further includes the following steps:

[0066] In step S200, within each time slot, the updated reputation score, communication capability, and computing capability of each node in the computing network are obtained for the previous time slot. A trust score for each node is then determined based on the updated reputation score, communication capability, and computing capability of each node in the previous time slot. A trusted node set is formed from a plurality of nodes whose trust scores exceed a preset trust threshold. Accordingly, the candidate scheduling solutions in step S210 are determined based on this trusted node set.

[0067] In step S200, at the beginning of each time slot, all CPN nodes first obtain the updated reputation score, communication capacity, and computing capacity of each CPN node from the previous time slot, stored in the blockchain layer. Based on the updated reputation score, communication capacity, and computing capacity of each CPN node from the previous time slot, they jointly select a certain number of trusted nodes to form a trusted node set M. Based on this trusted node set M, candidate scheduling solutions for each task are generated. In other words, the source and destination nodes for each task scheduling must be selected from within the trusted node set M. In this method, the scheduling of each task requires the task issuer to pass through the source and destination nodes, resulting in a multi-level flow of task data. By scheduling tasks between trusted nodes, the risk of privacy leakage caused by the multi-level flow of user task data can be effectively reduced, privacy and security can be improved, and the reliability and stability of the system can be enhanced.

[0068] Specifically, all CPN nodes in the computing network vote based on their updated reputation scores, communication capabilities, and computing capabilities from the previous time slot. This voting result generates a comprehensive score for each CPN node, i.e., the trust score of each CPN node. The trust scores of each CPN node are sorted from high to low, and the top M CPN nodes with the highest trust scores, or M CPN nodes with trust scores greater than a preset trust threshold, are selected from the sorted results to form the trusted node set M. The trusted node set determined in each time slot changes as the updated reputation scores, communication capabilities, and computing capabilities of each CPN node in the computing network change.

[0069] In some embodiments, the updated reputation score is obtained based on influencing factors, including timeliness of interaction between the task publisher and the destination node, positive interaction or negative interaction, and credibility (ie, reputation) of the task publisher.

[0070] Furthermore, the updated reputation score is obtained based on the reputation score of the destination node by each task publisher that interacts with the destination node and the credibility of each task publisher, wherein the reputation score is obtained by taking the weighted average of each historical interaction evaluation score and the corresponding freshness weighting coefficient and each interaction weighting coefficient given to the destination node by the task publisher that interacted with the destination node at least once before the corresponding time slot, the interaction is a positive interaction or a negative interaction, the interaction weight coefficient corresponding to the negative interaction is greater than the interaction weight coefficient corresponding to the positive interaction, and the freshness weighting coefficient is obtained based on historical interaction rounds and historical interaction times; the credibility is obtained by the reputation score of each destination node interacting with any one of the task publishers by each task publisher that interacts with the destination node.

[0071] Specifically, regarding the impact of interaction timeliness, the quality of service and reliability of each CPN node will change over time. The latest historical interaction can more accurately reflect the service situation of the CPN node, so the weight of the latest historical interaction is relatively large. Let α i,n (t) represent the number of historical interactions between task publisher i (i = 1, 2,...) and CPN node n (n = 1, 2,...) before time slot t, and e i,n (t ′ ) represent the historical interaction score of the t ′ -th (t ′ = 1, 2,...) historical interaction between task publisher i and CPN node n, that is, after the task published by the task publisher is processed by CPN node n each time, the historical interaction evaluation score corresponding to the historical interaction evaluation made by the task publisher on the computing service provided by this node. Then, in time slot t, the freshness weighting coefficient of e i,n (t ′ )(t ′ < t) can be expressed as:

[0072]

[0073] where t ′ represents the historical interaction round.

[0074] Regarding the impact of positive or negative interactions, a CPN node may gradually build a positive reputation, but may also quickly lose its reputation due to improper behavior. To prevent the CPN node from vacillating between good behavior and bad behavior, it is set that the weight of negative interaction (negative interaction) in reputation evaluation is higher than the weight of positive interaction (positive interaction). Use ε i,n (t) to represent the reputation score value of task publisher i for CPN node n in time slot t, and its calculation formula can be:

[0075] <​​​​​Regarding the influence of the reputation of the task publisher, the task publisher may be malicious, and the untrusted task publisher may overestimate or underestimate the computing service of the CPN node for his own purpose. Therefore, it is necessary to weight the reputation score value according to the credibility of the user (task publisher) to limit the influence of untrusted users. By comparing the service evaluation of the task publisher i and other task publishers other than task publisher i on the same CPN node n among all task publishers interacting with the same CPN node n, the average reputation score value of all task publishers other than task publisher i for CPN node n in time slot t is calculated. It can be expressed as:

[0078]

[0079] in, represents the set of all task publishers that interact with CPN node n and have corresponding reputation score values, Representing a collection The size of i ′ Representing a collection Any other task publisher except task publisher i, ε i′,n (t) represents the task publisher i ′ For the reputation score value of CPN node n. In time slot t, the cosine function is used to define the reputation score of task publisher i and all task publishers i ′ The similarity coefficient Sim between service evaluations of the same CPN node n i,i′ (t), the credibility of task publisher i, can be as follows:

[0080]

[0081] in, represents the set of CPN nodes n that interact with task publisher i. It should be noted that the set The set corresponding to different task publishers i in It may also be different.

[0082] Based on the above factors, the updated reputation score of CPN node n in time slot t can be calculated according to the following formula:

[0083]

[0084] Among them, ε n (t) represents the updated reputation score value of CPN node n.

[0085] CPN node n uploads the local state information of each time slot to the blockchain layer. The local state information includes the updated reputation score ε n(t), communication capability W n (t) and computing power F n (t), where W n (t) and F n (t) can be expressed as follows:

[0086]

[0087] in, represents the transmission rate when task publisher i publishes a task to CPN node m in time slot t, represents the transmission rate when CPN node m forwards a task to CPN node n at time slot t. The updated reputation scores, communication capabilities, and computing power of each CPN node in the computing network are maintained at the blockchain layer, making them immune to attacks and alterations. This allows for decentralized security verification of interactions between CPN nodes and users (i.e., task publishers), thereby providing an intelligent, trusted, and reliable network environment for task scheduling. By introducing CPN node reputation scores to evaluate the computing services they provide, unreliability caused by factors such as the difficulty of managing heterogeneous CPN nodes and potential laziness and dishonesty among nodes can be significantly mitigated.

[0088] In some embodiments, the immediate cost includes task transmission cost and task processing cost. The task transmission cost is obtained based on the size and bandwidth resource requirements of each published task received by each source node and the size of each forwarded task received by each destination node. The task processing cost is obtained based on the size and computing resource requirements of each forwarded task received by each destination node and the updated reputation score of each destination node.

[0089] Specifically, when a task publisher publishes a task and a CPN node provides the corresponding service through effective task scheduling, the task publisher needs to pay the corresponding fee for the service provided. This fee is the cost of the task publisher, including the payment required for the calculation and processing of the CPN node n and the payment fee for transmission through the wireless link. Therefore, for each task publisher i, its goal is to minimize the payment for the calculation of each task while meeting the delay requirements of each task it publishes. Therefore, the instantaneous cost c of task publisher i in time slot t can be expressed as i,t It is defined as the following formula:

[0090]

[0091] Among them, M represents the set of trusted nodes, h n represents the unit payment fee for CPN node n or destination node to perform a bit task, ζ wl Represents the bandwidth payment for transmitting one data bit over a wireless link.

[0092] In some embodiments, the task completion rate is the number of tasks completed within a preset task latency threshold. In other words, for a computing power network, the goal of achieving effective task scheduling is to complete the scheduling and processing of as many tasks as possible within the preset task latency threshold while meeting the latency requirements of each task publisher, i.e., achieving a high task completion rate.

[0093] Corresponding to the above method, the present invention also provides a task scheduling system for a computing power network, which includes a computer device, wherein the computer device includes a processor and a memory, wherein the memory stores computer instructions, and the processor is used to execute the computer instructions stored in the memory. When the computer instructions are executed by the processor, the system implements the steps of the above method.

[0094] In summary, the task scheduling method and system for a computing network of the present invention utilizes multi-agent reinforcement learning to schedule individual tasks. This method can efficiently implement real-time dynamic task scheduling within the computing network and effectively improve the efficiency of computing resource allocation and utilization for real-time dynamic tasks, thereby reducing network overhead and improving network performance. Furthermore, by determining candidate scheduling strategies and optimal scheduling strategies for each task based on a set of trusted nodes, the risk of privacy leakage of user task data can be effectively reduced, privacy and security can be improved, and the reliability and stability of the system can be enhanced.

[0095] An embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the aforementioned method. The computer-readable storage medium may be a tangible storage medium, such as a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a floppy disk, a hard disk, a removable storage disk, a CD-ROM, or any other form of storage medium known in the art.

[0096] An embodiment of the present invention further provides a computer program product, comprising computer instructions, which implement the steps of the aforementioned method when executed by a processor.

[0097] It should be understood by those skilled in the art that the various exemplary components, systems and methods described in conjunction with the embodiments disclosed herein can be implemented in hardware, software or a combination of the two. Whether it is specifically performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention. When implemented in hardware, it can be, for example, an electronic circuit, an application specific integrated circuit (ASIC), appropriate firmware, a plug-in, a function card, etc. When implemented in software, the elements of the present invention are programs or code segments that are used to perform the required tasks. The program or code segment can be stored in a machine-readable medium, or transmitted on a transmission medium or a communication link via a data signal carried in a carrier.

[0098] It should be understood that the present invention is not limited to the specific configurations and processes described above and illustrated in the figures. For the sake of brevity, a detailed description of known methods is omitted. In the above embodiments, several specific steps are described and illustrated as examples. However, the method of the present invention is not limited to the specific steps described and illustrated. Those skilled in the art may make various changes, modifications, and additions, or change the order of the steps after understanding the spirit of the present invention.

[0099] In the present invention, features described and / or illustrated for one embodiment may be used in the same or similar manner in one or more other embodiments, and / or combined with or replace features of other embodiments.

[0100] The foregoing description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Those skilled in the art will readily appreciate that various modifications and variations to the present invention are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention are intended to be within the scope of protection of the present invention.

Claims

1. A task scheduling method for a computing network, characterized in that: The method comprises: The task scheduling problem of the computing power network is characterized based on the Markov decision process, and the state, action and scheduling reward of each time slot are determined, wherein the state includes the task publishing requirements of each task and the task forwarding requirements of each node, the initial task queue length and the updated reputation score; the action includes all candidate scheduling schemes for each task, and the candidate scheduling schemes include a source node for receiving tasks published by the task publisher and at least one destination node for receiving tasks forwarded by the source node and processing the tasks; the scheduling reward includes a reward based on the task delay of each task, the task completion rate and the immediate cost of each task publisher, the task delay includes the delay obtained based on the task publishing requirements, the task forwarding requirements of the source node and the initial task queue length of at least one destination node, and the immediate cost includes the cost obtained based on the task publishing requirements of each task published by the task publisher, the task forwarding requirements of each source node and the updated reputation score of each destination node; Based on the status of each time slot, each candidate scheduling scheme when the scheduling reward corresponding to each task is maximum is used as each target scheduling scheme, so as to execute the scheduling of each task according to the each target scheduling scheme.

2. The method according to claim 1, characterized in that The method further comprises: In each time slot, the updated reputation score, communication capability, and computing capability of each node in the computing power network in the previous time slot are obtained, and the trust score of each node is obtained based on the updated reputation score, communication capability, and computing capability of each node in the previous time slot. A trusted node set is formed by multiple nodes whose trust scores are greater than a preset trust threshold; the candidate scheduling scheme is determined based on the trusted node set.

3. The method according to claim 1 or 2, characterized in that The updated reputation score is obtained based on influencing factors, including the timeliness of interaction between the task publisher and the destination node, positive interaction or negative interaction, and the credibility of the task publisher.

4. The method according to claim 3, characterized in that The updated reputation score is obtained based on the reputation score of the destination node by each task publisher that interacts with the destination node and the credibility of each task publisher, wherein the reputation score is obtained by taking the weighted average of each historical interaction evaluation score and the corresponding freshness weighting coefficient and each interaction weighting coefficient given to the destination node by the task publisher that interacted with the destination node at least once before the corresponding time slot, the interaction is a positive interaction or a negative interaction, the interaction weight coefficient corresponding to the negative interaction is greater than the interaction weight coefficient corresponding to the positive interaction, and the freshness weighting coefficient is obtained based on the historical interaction rounds and the historical number of interactions; the credibility is obtained by the reputation score of each destination node interacting with any one of the task publishers by each task publisher that interacts with the destination node.

5. The method according to claim 1, wherein The task publishing requirements include the transmission power, size, bandwidth resource requirements and computing resource requirements of the published tasks; the task forwarding requirements include the transmission power, size, bandwidth resource requirements and computing resource requirements of the forwarded tasks; the initial task queue length is based on the initial task queue length of the previous time slot, the total computing load of all newly arrived forwarded tasks in the task queue and the total computing load of processing all forwarded tasks.

6. The method according to claim 5, characterized in that The task delay includes a first transmission delay from the task publisher to the source node, a second transmission delay from the source node to at least one destination node, a queuing waiting delay and a task processing delay, wherein the first transmission delay is obtained based on the transmission rate obtained by the transmission power and bandwidth resource requirements of the published task received by the source node, the second transmission delay is obtained based on the transmission rate obtained by the transmission power and bandwidth resource requirements of the forwarding task received by each of the at least one destination nodes, the queuing waiting delay is obtained based on the initial task queue length of each of the at least one destination nodes, the computing resource requirements of each task in the initial task queue, and the computing resource requirements and size of each forwarding task received, and the task processing delay is obtained based on the size and computing resource requirements of the forwarding task received by each of the at least one destination nodes.

7. The method according to claim 1 or 6, characterized in that The task completion rate is the number of tasks completed within a preset task delay threshold.

8. The method according to claim 5, characterized in that The instantaneous cost includes task transmission cost and task processing cost. The task transmission cost is obtained based on the size and bandwidth resource requirements of each published task received by each source node and the size of each forwarded task received by each destination node. The task processing cost is obtained based on the size and computing resource requirements of each forwarded task received by each destination node and the updated reputation score of each destination node.

9. A task scheduling system for a computing network, comprising a processor, a memory, and computer instructions stored in the memory, characterized in that: The processor is configured to execute the computer instructions. When the computer instructions are executed, the system implements the steps of the method according to any one of claims 1 to 8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

Images