An Adaptive Weighted Task Scheduling Method and System Based on Temporal Difference Learning

Abstract

This invention discloses an adaptive weighted task scheduling method and system based on temporal difference learning. The method includes: defining and initializing the cluster and each node in the cluster based on a node information structure; receiving and parsing adaptive weighted task scheduling requests in real time to obtain the parsed tasks; performing circuit breaker filtering on each node to determine non-circuit breaker nodes; and calculating the weight W of the non-circuit breaker nodes based on a three-dimensional weight calculation model. i Filter out weights W i The non-circuit-breaker node with the highest value is selected as the task execution node. The parsed task is sent to the task execution node for execution, and the execution result is returned. The execution result is verified, and the node reputation value and node status of the task execution node are updated dynamically according to the verification result and the node reputation value update formula. By applying the method and system provided by this invention, the efficiency and system stability of adaptive task weighting task scheduling and allocation are improved.

Description

Technical Field

[0001] This application relates to the fields of distributed computing and load balancing technology, and more specifically, to an adaptive weighted task scheduling method and system based on temporal difference learning. Background Technology

[0002] 1. Limitations of static scheduling strategies

[0003] Traditional static scheduling algorithms (such as round-robin and random allocation) assign tasks based on fixed rules, failing to detect fluctuations in node computing power (such as changes in CPU / GPU load) and dynamic changes in network latency (such as network congestion). In blockchain AI training, static scheduling may assign computationally intensive tasks to nodes with temporarily insufficient computing power, leading to task timeouts or failures and a significant reduction in system resource utilization.

[0004] 2. The lag in credit assessment mechanisms

[0005] Traditional reputation systems use simple accumulation or averaging algorithms, failing to consider the temporal correlation of node behavior. For example, malicious nodes can evade detection by using a "brushing-dormant-re-attack" pattern. Traditional systems require multiple instances of abnormal behavior to trigger a penalty, resulting in slower response times.

[0006] 3. Lack of fault tolerance mechanism

[0007] Existing systems lack dynamic circuit breaker mechanisms, making it impossible to isolate faulty nodes in a timely manner when they encounter DDoS attacks or hardware failures. Traditional algorithms suffer from high task failure rates and large average scheduling delays when nodes fail.

[0008] Therefore, it is necessary to introduce a new method and system to calculate the weights of non-circuit-breaker nodes based on a three-dimensional weight calculation model. Through temporal differential learning (TD-Learning) and a circuit breaker mechanism, the node reputation values ​​of each node in the cluster can be quickly adjusted after a single anomaly. Targeted rewards or penalties can be applied to task nodes, and circuit breaker periods can be set. This addresses the technical problems in existing technologies, such as static scheduling assigning high-computation tasks to nodes with insufficient computing power, penalties being triggered only after multiple abnormal behaviors, leading to a decrease in response speed, and the inability to promptly isolate faulty nodes when they encounter DDoS attacks or hardware failures. This will improve the efficiency of adaptive task weighting, task scheduling, and allocation, as well as the system stability. Summary of the Invention

[0009] To address the aforementioned technical issues, this invention provides an adaptive weighted task scheduling method and system based on temporal differential learning. Based on a defined node information structure and a three-dimensional weight calculation model, the weights of non-circuit-breaker nodes are calculated. Through temporal differential learning (TD-Learning) and a circuit breaker mechanism, the node reputation values ​​of each node in the cluster are quickly adjusted after a single anomaly. Targeted rewards or penalties are applied to task nodes, and circuit breaker periods are set. This solves the technical problems in existing technologies, such as static scheduling assigning high-computation tasks to nodes with insufficient temporary computing power, penalties being triggered only after multiple abnormal behaviors leading to decreased response speed, and the inability to promptly isolate faulty nodes when they encounter DDoS attacks or hardware failures. Therefore, this invention improves the efficiency and system stability of adaptive task weighted task scheduling and allocation.

[0010] This invention provides an adaptive weighted task scheduling method based on temporal difference learning. The method includes: S1, defining the node information structure: defining and constructing a node information structure according to the adaptive weighted task scheduling process based on temporal difference learning; S2, node initialization: initializing the cluster and each node in the cluster based on the node information structure, and acquiring the real-time computing power, network latency, node reputation value, peak computing power of all nodes in the network, and minimum latency of all nodes in the network in real time; S3, filtering nodes and executing tasks: receiving and parsing the adaptive weighted task scheduling request in real time, obtaining the parsed tasks, filtering each node for circuit breaker periods, determining non-circuit-breaker nodes, and calculating the weight W of the non-circuit-breaker nodes based on a three-dimensional weight calculation model. i The weight W of the non-circuit-breaker node i Sort by size and filter out the weights W. i The non-circuit-breaker node with the largest value is selected as the task execution node. The parsed task is sent to the task execution node for execution, and the execution result is returned. S4, Node information update: The execution result is verified, and the node reputation value and node status of the task execution node are updated according to the verification result and the node reputation value dynamic update formula.

[0011] Preferably, the node information structure includes: real-time computing power (FLOPS), network latency (ms), node reputation value, last update timestamp, number of consecutive failures, circuit breaker expiration timestamp, global maximum computing power, single task latency, global minimum latency, node real-time computing power, peak computing power of all nodes in the network, node network round-trip latency, minimum latency of all nodes in the network, and task reward value; wherein, the node reputation value is greater than or equal to 0 and less than or equal to 100;

[0012] Preferably, in step S2, the step of initializing the cluster and each node in the cluster based on the node information structure specifically includes: S211, based on the node information structure, obtaining the real-time computing power, network latency, node reputation value, and node latency of each node in the cluster, determining the peak computing power of the entire network nodes and the minimum latency of the entire network nodes according to the real-time computing power and network latency of each node, and setting the consecutive failure count and circuit breaker expiration timestamp of each node in the cluster to zero; S212, setting the single task latency of each node to a value greater than or equal to the single task latency threshold, wherein the single task latency threshold is preset by the operator according to the requirements of adaptive weight task execution; S213, setting the initial value of the task reward value and the consecutive failure count of each node in the cluster to zero.

[0013] Preferably, in step S211, the step of obtaining the real-time computing power, network latency, node reputation value, and node latency of each node in the cluster based on the node information structure, and determining the peak computing power and minimum latency of the entire network nodes based on the real-time computing power and network latency of each node, further includes: S211-1, node computing power acquisition: performing 1024×1024 matrix multiplication operations on each node in the cluster according to a preset number of tests, calculating and obtaining the number of floating-point operations per second for each node performing the 1024×1024 matrix multiplication operation each time, and the average number of floating-point operations per second after performing the preset number of tests, taking the average number of floating-point operations per second after performing the preset number of tests as the real-time computing power of each node, and determining the peak computing power of the entire network nodes based on the real-time computing power of each node; S211-2, node latency acquisition: sending a preset number of ICMP Echo requests to each node at a preset test time, recording each ICMP request of each node. The round-trip time of Echo requests is used to remove outliers from all recorded round-trip times. The median of all round-trip times after removing outliers is used as the network latency of each node. The minimum latency of all nodes in the network is determined based on the network latency of each node.

[0014] Preferably, in step S3, the step of filtering each node for circuit breaker periods and determining non-circuit-breaker nodes further includes: obtaining the circuit breaker expiration timestamp of each node in the cluster, determining the circuit breaker attribute of the node based on the circuit breaker expiration timestamp, wherein the circuit breaker attribute includes circuit-breaker nodes and non-circuit-breaker nodes; if the circuit breaker expiration timestamp of the node is greater than or equal to the current time, then the node is a circuit-breaker node; if the circuit breaker expiration timestamp of the node is less than the current time, then the node is a non-circuit-breaker node; wherein, the weight of the circuit-breaker node is zero, and task allocation will be suspended when the node is determined to be a circuit-breaker node.

[0015] Preferably, in step S3, the three-dimensional weight calculation model is: Among them, W i W represents the weight of the node. i ∈[0,100], α, β, and γ are all weighting coefficients, where α is the computing power dimension of the weighting coefficients, β is the network latency dimension of the weighting coefficients, and γ is the reputation dimension of the weighting coefficients. α + β + γ = 1, FLOPS i For the real-time computing power of nodes, FLOPS max RTT represents the peak computing power of all network nodes. i RTT is the round-trip time for the node network. min Reputation is the minimum latency across all nodes in the network. i This represents the node's reputation value.

[0016] Preferably, in step S4, the step of verifying the execution result and updating the node reputation value and node status of the task execution node according to the verification result and the node reputation value dynamic update formula includes:

[0017] The execution results include successful task execution, failed task execution, and malicious behavior;

[0018] The verification results include verification passed, verification failed, and malicious behavior;

[0019] S41, when the execution result is that the task was successfully executed, the verification result is that the verification passed, and a reward is sent to the task execution node, and the task reward value R of the task execution node is set. i (t) Add the success reward value, and dynamically update the formula based on the node reputation value and the current node reputation value of the task execution node. i (t) Update the node reputation value of the task execution node. i (t+1);

[0020] S42, when the execution result is task execution failure, the verification result is verification failure, then the consecutive failure count of the task execution node is incremented by 1, and the weight of the task execution node is updated according to the updated consecutive failure count of the task execution node or the single task delay, and the circuit breaker period is set.

[0021] The task reward value R of the task execution node i (t) Subtract the first failure penalty value, and simultaneously update the formula dynamically based on the node reputation value and the current node reputation value of the task execution node. i (t) Update the node reputation value of the task execution node.i (t+1);

[0022] Return to step S3, select a new task execution node, send the parsed task to the new task execution node for execution, and return the execution result;

[0023] S43, when the execution result is malicious behavior, the verification result is malicious behavior, then the task reward value R of the task execution node is reduced. i (t) Subtract the second failure penalty value, and dynamically update the node reputation value based on the node reputation value and the current node reputation value of the task execution node. i (t) Update the node reputation value of the task execution node. i (t+1), and set the weight of the task execution node to 0, mark the task execution node as a circuit breaker node, and set the circuit breaker period of the task execution node according to the preset circuit breaker period;

[0024] Return to step S3, select a new task execution node, send the parsed task to the new task execution node for execution, and return the execution result;

[0025] in,

[0026] If the task execution node updates its node reputation value i If (t+1) is greater than 100, then the updated node reputation value of the task execution node will be set. i The value of (t+1) is set to 100, if the task execution node updates the node reputation value. i If (t+1) is less than 0, then the updated node reputation value of the task execution node will be... i The value of (t+1) is set to 0;

[0027] The success reward value, the first failure penalty value, and the second failure penalty value are all preset according to the needs of task scheduling, and the second failure penalty value is greater than the first failure penalty value.

[0028] Preferably, the dynamic update formula for the node reputation value is: Reputation i (t+1)=Reputation i (t)+η·(R i (t)-Reputation i (t)); where, Reputation i (t+1) represents the updated node reputation value of the task execution node.i (t) represents the current node reputation value of the task execution node, R i (t) represents the task reward value of the task execution node, and η represents the learning rate of the task reward value, which is used to dynamically adjust the reward or penalty intensity for the task execution node, where η∈[0.1,0.3].

[0029] Preferably, in step S42, the step of updating the weight of the task execution node based on the updated consecutive failure count or single task delay, and setting the circuit breaker period, further includes: if the updated consecutive failure count of the task execution node is greater than or equal to 3, or the single task delay is greater than the single task delay threshold, then the weight of the task execution node is set to 0, the task execution node is marked as a circuit breaker node, and the circuit breaker period of the task execution node is set based on the circuit breaker period calculation formula, and the circuit breaker expiration timestamp of the task execution node is updated; wherein, the circuit breaker period calculation formula is: T = 10·e failedcount In the formula, T is the set circuit breaker period, and failedcount is the number of consecutive failures after the update of the task execution node.

[0030] Accordingly, this invention also provides an adaptive weighted task scheduling system based on temporal difference learning, the system comprising a structure definition module, a node information acquisition module, a node filtering module, and a node information update module; wherein,

[0031] The structure definition module is used for defining the node information structure: defining and constructing the node information structure according to the process of adaptive weighted task scheduling based on temporal difference learning;

[0032] The node information acquisition module is used for node initialization: based on the node information structure, it initializes the cluster and each node in the cluster, and obtains the real-time computing power, network latency, node reputation value, peak computing power of all nodes and minimum latency of all nodes in the network in real time.

[0033] The node filtering module is used to filter nodes and execute tasks: it receives and parses adaptive weighted task scheduling requests in real time, obtains the parsed tasks, filters each node for circuit breaker periods, determines non-circuit-breaker nodes, and calculates the weight W of the non-circuit-breaker nodes based on a three-dimensional weight calculation model. i The weight W of the non-circuit-breaker node i Sort by size and filter out the weights W. i The non-circuit-breaker node with the largest value is selected as the task execution node. The parsed task is sent to the task execution node for execution, and the execution result is returned.

[0034] The node information update module is used for node information update: verifying the execution result, and updating the node reputation value and node status of the task execution node according to the verification result and the node reputation value dynamic update formula.

[0035] This invention, by applying the above technical solutions, realizes a defined node information structure that can collect real-time node information such as computing power, network latency, and circuit breaker expiration timestamps of each node in the cluster. Based on the real-time acquired node information, a three-dimensional weight calculation model is used to calculate the weight of non-circuit-breaker nodes, select the task execution node with the largest weight value, and quickly adjust the node reputation value of each node in the cluster after a single anomaly through temporal differential learning (TD-Learning) and a circuit breaker mechanism. Targeted rewards or penalties are applied to task nodes, and circuit breaker periods and node attributes are set. This solves the technical problems of static scheduling that assigns high-computation tasks to nodes with insufficient computing power temporarily, penalties that are triggered only after multiple abnormal behaviors, leading to a decrease in response speed, and the inability to isolate faulty nodes in a timely manner when nodes encounter DDoS attacks or hardware failures. This improves the efficiency and system stability of adaptive task weighting task scheduling and allocation. Attached Figure Description

[0036] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0037] Figure 1 A flowchart illustrating the adaptive weighted task scheduling method based on temporal difference learning proposed in an embodiment of the present invention is shown.

[0038] Figure 2 The diagram shows a schematic of the structure of an adaptive weighted task scheduling system based on temporal difference learning proposed in an embodiment of the present invention. Detailed Implementation

[0039] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0040] This invention provides an adaptive weighted task scheduling method based on temporal difference learning, such as... Figure 1 As shown, the method includes the following steps:

[0041] S1, Node Information Structure Definition: Define and construct the node information structure based on the process of adaptive weighted task scheduling based on temporal difference learning.

[0042] In this embodiment, the node information structure includes:

[0043] Real-time computing power (FLOPS), network latency (ms), node reputation value, last update timestamp, number of consecutive failures, circuit breaker expiration timestamp, global maximum computing power, single task latency, global minimum latency, node real-time computing power, peak computing power of all nodes in the network, node network round-trip latency, minimum latency of all nodes in the network, and task reward value.

[0044] Wherein, the node reputation value is greater than or equal to 0 and less than or equal to 100;

[0045] S2, Node initialization: Based on the node information structure, initialize the cluster and each node in the cluster, and obtain the real-time computing power, network latency, node reputation value, peak computing power of all nodes and minimum latency of all nodes in the network in real time.

[0046] In this embodiment, step S2, the step of initializing the cluster and each node in the cluster based on the node information structure, specifically includes:

[0047] S211, based on the node information structure, obtain the real-time computing power, network latency, node reputation value and node latency of each node in the cluster, determine the peak computing power of the entire network nodes and the minimum latency of the entire network nodes according to the real-time computing power and network latency of each node, and set the number of consecutive failures and the circuit breaker expiration timestamp of each node in the cluster to zero.

[0048] S212, set the single task delay of each node to a value greater than or equal to the single task delay threshold, wherein the single task delay threshold is preset by the operator according to the requirements of adaptive weight task execution;

[0049] S213 sets the initial value of the task reward value and the number of consecutive failures for each node in the cluster to zero.

[0050] In this embodiment, step S211, which involves obtaining the real-time computing power, network latency, node reputation value, and node latency of each node in the cluster based on the node information structure, and determining the peak computing power and minimum latency of the entire network nodes based on the real-time computing power and network latency of each node, further includes:

[0051] S211-1, Node computing power acquisition: Perform 1024×1024 matrix multiplication operations on each node in the cluster according to a preset number of tests, calculate and obtain the number of floating-point operations per second for each node performing the 1024×1024 matrix multiplication operation each time, and the average number of floating-point operations per second after performing the preset number of tests, take the average number of floating-point operations per second after performing the preset number of tests as the real-time computing power of each node, and determine the peak computing power of the entire network nodes based on the real-time computing power of each node;

[0052] S211-2, Node latency collection: At a preset test time, send a preset number of ICMP Echo requests to each node, record the round-trip time of each ICMP Echo request sent and returned by each node, remove outliers from all recorded round-trip times, and take the median value of all round-trip times after removing outliers as the network latency of each node. Determine the minimum latency of all nodes in the network based on the network latency of each node.

[0053] S3, Filtering Nodes and Executing Tasks: Receive and parse adaptive weighted task scheduling requests in real time, obtain the parsed tasks, filter each node for circuit breaker periods, determine non-circuit breaker nodes, and calculate the weight W of the non-circuit breaker nodes based on a three-dimensional weight calculation model. i The weight W of the non-circuit-breaker node i Sort by size and filter out the weights W. i The non-circuit-breaker node with the largest value is selected as the task execution node. The parsed task is sent to the task execution node for execution, and the execution result is returned.

[0054] In this embodiment, step S3, the step of filtering each node for the circuit breaker period to determine the non-circuit breaker node, further includes:

[0055] Obtain the circuit breaker expiration timestamps of each node in the cluster, and determine the circuit breaker attribute of the node based on the circuit breaker expiration timestamps. The circuit breaker attribute includes circuit breaker nodes and non-circuit breaker nodes.

[0056] If the circuit breaker expiration timestamp of the node is greater than or equal to the current time, then the node is a circuit breaker node; if the circuit breaker expiration timestamp of the node is less than the current time, then the node is a non-circuit breaker node.

[0057] The weight of the circuit breaker node is zero, and task allocation will be suspended when a node is identified as a circuit breaker node.

[0058] In this embodiment, the three-dimensional weight calculation model in step S3 is:

[0059] Among them, Wi W represents the weight of the node. i ∈[0,100], α, β, and γ are all weighting coefficients, where α is the computing power dimension of the weighting coefficients, β is the network latency dimension of the weighting coefficients, and γ is the reputation dimension of the weighting coefficients. α + β + γ = 1, FLOPS i For the real-time computing power of nodes, FLOPS max RTT represents the peak computing power of all network nodes. i RTT is the round-trip time for the node network. min Reputation is the minimum latency across all nodes in the network. i Let i be the node's reputation value, and i be the node's ID.

[0060] S4. Node Information Update: Verify the execution result, and update the node reputation value and node status of the task execution node according to the verification result and the node reputation value dynamic update formula.

[0061] In this embodiment, step S4, which involves verifying the execution result and updating the node reputation value and node status of the task execution node based on the verification result and the node reputation value dynamic update formula, includes:

[0062] The execution results include successful task execution, failed task execution, and malicious behavior;

[0063] The verification results include verification passed, verification failed, and malicious behavior;

[0064] S41, when the execution result is that the task was successfully executed, the verification result is that the verification passed, and a reward is sent to the task execution node, and the task reward value R of the task execution node is set. i (t) Add the success reward value, and dynamically update the formula based on the node reputation value and the current node reputation value of the task execution node. i (t) Update the node reputation value of the task execution node. i (t+1);

[0065] S42, when the execution result is task execution failure, the verification result is verification failure, then the consecutive failure count of the task execution node is incremented by 1, and the weight of the task execution node is updated according to the updated consecutive failure count of the task execution node or the single task delay, and the circuit breaker period is set.

[0066] The task reward value R of the task execution node i (t) Subtract the first failure penalty value, and simultaneously update the formula dynamically based on the node reputation value and the current node reputation value of the task execution node.i (t) Update the node reputation value of the task execution node. i (t+1);

[0067] Return to step S3, select a new task execution node, send the parsed task to the new task execution node for execution, and return the execution result;

[0068] S43, when the execution result is malicious behavior, the verification result is malicious behavior, then the task reward value R of the task execution node is reduced. i (t) Subtract the second failure penalty value, and dynamically update the node reputation value based on the node reputation value and the current node reputation value of the task execution node. i (t) Update the node reputation value of the task execution node. i (t+1), and set the weight of the task execution node to 0, mark the task execution node as a circuit breaker node, and set the circuit breaker period of the task execution node according to the preset circuit breaker period;

[0069] Return to step S3, select a new task execution node, send the parsed task to the new task execution node for execution, and return the execution result;

[0070] in,

[0071] If the task execution node updates its node reputation value i If (t+1) is greater than 100, then the updated node reputation value of the task execution node will be set. i The value of (t+1) is set to 100, if the task execution node updates the node reputation value. i If (t+1) is less than 0, then the updated node reputation value of the task execution node will be... i The value of (t+1) is set to 0;

[0072] The success reward value, the first failure penalty value, and the second failure penalty value are all preset according to the needs of task scheduling, and the second failure penalty value is greater than the first failure penalty value.

[0073] In this embodiment, the dynamic update formula for the node reputation value is: Reputation i (t+1)=Reputation i (t)+η·(R i (t)-Reputation i (t));

[0074] Among them, Reputation i (t+1) represents the updated node reputation value of the task execution node. i (t) represents the current node reputation value of the task execution node, R i (t) represents the task reward value of the task execution node, and η represents the learning rate of the task reward value, which is used to dynamically adjust the reward or penalty intensity for the task execution node, where η∈[0.1,0.3].

[0075] In this embodiment, step S42, which involves updating the weight of the task execution node based on the updated consecutive failure count or single task delay, and setting the circuit breaker period, further includes:

[0076] If the number of consecutive failures of the updated task execution node is greater than or equal to 3, or the single task delay is greater than the single task delay threshold, then the weight of the task execution node is set to 0, the task execution node is marked as a circuit breaker node, the circuit breaker period of the task execution node is set based on the circuit breaker period calculation formula, and the circuit breaker expiration timestamp of the task execution node is updated.

[0077] in,

[0078] The formula for calculating the circuit breaker period is: T = 10·e failedcount In the formula, T is the set circuit breaker period, and failedcount is the number of consecutive failures after the update of the task execution node.

[0079] To enable those skilled in the art to better understand the technical solutions in this step, further examples and explanations are provided regarding the three-dimensional weight calculation model and its parameters.

[0080] The three-dimensional weight calculation model The parameters can be set as follows: for example, computing power dimension α = 0.5, network latency dimension β = 0.3, and reputation dimension γ = 0.2. These three values ​​can be dynamically adjusted through a smart contract. These three parameters represent the weight coefficients of the three core dimensions, and their numerical settings reflect the importance ranking of each dimension in the system. α, β, and γ are normalization coefficients, summing to 1, used to quantify the contribution ratio of different dimensions to the final weight.

[0081] FLOPS i The real-time computing power of the nodes was obtained through matrix operation testing, with an error rate of <1%.

[0082] FLOPS max The peak computing power of all nodes in the network is updated in real time via smart contracts.

[0083] RTT i The network round-trip latency for nodes is determined by the median of the test results of sending three ping requests to each node within 5 minutes during actual acquisition.

[0084] RTT min This represents the minimum latency across all nodes in the network.

[0085] Reputation i The node reputation value ranges from 0 to 100 and is dynamically updated using a node reputation value dynamic update formula based on temporal difference learning (TD-Learning).

[0086] By using a multi-dimensional normalized score (0-100 range), a fair comparison of computing power, network, and reputation is achieved, thereby improving the rationality of task allocation.

[0087] The dynamic update formula for the node reputation value is: Reputation i (t+1)=Reputation i (t)+η·(R i (t)-Reputation i (t));

[0088] Here, if the node reputation value obtained in real time is the updated value, then the Reputation i The value is equal to the updated node reputation value. i (t+1), if the real-time node reputation value is the one before the update, then Reputation i The value is equal to the node reputation value before the update. i (t),

[0089] η∈[0.1,0.3], where η is the learning rate. The default value can be set to 0.2, which is used to dynamically adjust the rate to balance response speed and stability.

[0090] R i (t): Task reward value, where the determination of malicious behavior needs to be linked with the reputation update mechanism. The punishment intensity is dynamically adjusted through η. For example, when high-frequency malicious behavior is detected, the η value is automatically increased to accelerate reputation decay and strengthen the deterrent effect.

[0091] The success reward value, the first failure penalty value, and the second failure penalty value are all task reward values. In practical applications, the success reward value, the first failure penalty value, and the second failure penalty value can be set. For example, the success reward value can be set to 10, the first failure penalty value can be set to 20, and the second failure penalty value can be set to 50.

[0092] Boundary constraints: The updated node reputation value is automatically limited to the range [0, 100]. In the node reputation value update mechanism, it is theoretically possible for the updated reputation value to exceed 100, such as the cumulative effect of consecutive high-reward tasks, the reverse correction of historical reputation penalties, or abnormal parameter configuration. However, it is usually forcibly limited to the range [0, 100] through a boundary constraint mechanism. The boundary constraint method can be mathematical truncation or a tiered reward decay mechanism.

[0093] By applying this technical solution, compared with the traditional moving average algorithm, the detection delay for malicious behavior is reduced from 3 anomalies to 1, and the attack detection rate is also greatly improved.

[0094] Dynamic circuit breaker mechanism:

[0095] Triggering conditions: Three consecutive task failures; or a single task delay exceeding a threshold (e.g., 500ms).

[0096] Circuit breaker operation: Immediately reduce the node's weight to 0 and suspend task allocation;

[0097] Circuit breaker period T = 10·e failedcount Minutes, for example, when there are 3 consecutive failures, T = 10e 3 The circuit breaker period is approximately 200 minutes. The more failures there are, the longer the circuit breaker period extends exponentially.

[0098] Through the dynamic circuit breaker mechanism, the isolation time of faulty nodes is reduced from an average of 10 minutes in traditional algorithms to the second level, thereby improving the system's resistance to DDoS attacks.

[0099] Algorithm Derivation:

[0100] 1. Proof of weight normalization

[0101] because (When RTTi≥RTTmin). And α+β+γ=1, so Wi∈[0,1], ensuring that the weight values ​​are within the normalized interval, supporting fair comparison across nodes.

[0102] 2. Proof of convergence of TD-Learning

[0103] According to reinforcement learning theory, when the learning rate satisfies: and The TD-Learning algorithm converges to the optimal value function, meaning that the reputation value eventually stably reflects the true performance of the node, proving the theoretical reliability of the mechanism.

[0104] Computing power acquisition.

[0105] · Test method: Perform 1024×1024 matrix multiplication operations and calculate the floating-point operations per second (FLOPS).

[0106] · Precision control: Repeat the test 3 times, take the average value, and control the error rate within 1%.

[0107] · Update frequency: Actively report when the node goes online each time, or automatically refresh once per hour.

[0108] Latency collection.

[0109] · Test method: Send 3 ICMP Echo requests (ping) to the node every 5 minutes and record the round-trip time.

[0110] · Data processing: Eliminate outliers and take the median as the current RTT value.

[0111] · Storage mechanism: Retain the latency data of the last 1 hour for dynamic threshold judgment.

[0112] When obtaining the global maximum computing power, set the first obtained global maximum computing power as the maximum computing power maxFlops, and obtain the latest global maximum computing power _flops of the entire cluster in real time. If _flops > maxFlops, then maxFlops = _flops;

[0113] When obtaining the global minimum latency, set the first obtained network latency as the minimum latency minRtt, and obtain the latest network latency _rtt of the entire cluster in real time. If _rtt < minRtt, then minRtt = _rtt.

[0114] By applying the above technical solutions, it is possible to achieve real-time collection of node information such as the computing power, network latency, and fuse expiration timestamp of each node in the cluster based on the defined node information structure, calculate the weights of non-fused nodes using a three-dimensional weight calculation model based on the real-time obtained node information, select the task execution node with the largest weight value, and quickly adjust the node reputation values of each node in the cluster through temporal difference learning (TD-Learning) and the fuse mechanism after a single exception, and conduct targeted rewards or punishments, set fuse periods and node attributes for task nodes, solving the technical problems in the prior art such as static scheduling allocating high-computation tasks to nodes with temporarily insufficient computing power, the response speed decreasing due to cumulative multiple abnormal behaviors triggering punishment, and the inability to isolate faulty nodes in a timely manner when a node encounters a DDOS attack or hardware failure, thereby improving the efficiency of adaptive task weight task scheduling and allocation and the system stability.

[0115] Furthermore, node selection is optimized through multi-dimensional indicator evaluation and an adaptive circuit breaker mechanism. The contract maintains information such as computing power, network latency, and reputation value for each node, dynamically updating the overall network performance benchmark. A weighted scoring model is used to calculate node weights, and an exponential circuit breaker penalty (circuit breaker period 10×e) is triggered when a node fails ≥3 consecutive times. n (Minutes), during which the node weight resets to zero. Reputation increases upon task success and decreases upon failure, creating a positive incentive mechanism. This design ensures that highly reliable nodes receive more task allocations, effectively improving the overall stability and efficiency of the system.

[0116] The learning rate controls the speed and magnitude of reputation value updates. The system is initialized with a given node reputation value, constrained to the range of 0-100. Each time, based on the task results, the reputation value is dynamically adjusted using gradient descent, while ensuring that the updated reputation value always remains within the valid range of 0-100. This design allows the reputation system to respond quickly to changes in node performance while avoiding drastic fluctuations in reputation values ​​due to single extreme events, achieving a smooth evaluation of long-term node performance.

[0117] Corresponding to the adaptive weighted task scheduling method based on temporal difference learning described in the embodiments of the present invention, the present invention also discloses an adaptive weighted task scheduling system based on temporal difference learning, such as... Figure 2 As shown, the system includes a structure definition module, a node information acquisition module, a node filtering module, and a node information update module;

[0118] in,

[0119] The structure definition module is used for defining the node information structure: defining and constructing the node information structure according to the process of adaptive weighted task scheduling based on temporal difference learning;

[0120] The node information acquisition module is used for node initialization: based on the node information structure, it initializes the cluster and each node in the cluster, and obtains the real-time computing power, network latency, node reputation value, peak computing power of all nodes and minimum latency of all nodes in the network in real time.

[0121] The node filtering module is used to filter nodes and execute tasks: it receives and parses adaptive weighted task scheduling requests in real time, obtains the parsed tasks, filters each node for circuit breaker periods, determines non-circuit-breaker nodes, and calculates the weight W of the non-circuit-breaker nodes based on a three-dimensional weight calculation model. i The weight W of the non-circuit-breaker node i Sort by size and filter out the weights W. i The non-circuit-breaker node with the largest value is selected as the task execution node. The parsed task is sent to the task execution node for execution, and the execution result is returned.

[0122] The node information update module is used for node information update: verifying the execution result, and updating the node reputation value and node status of the task execution node according to the verification result and the node reputation value dynamic update formula.

[0123] The various embodiments in this specification are described in a related manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.

[0124] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention are included within the scope of protection of the present invention.

Claims

1. An adaptive weighted task scheduling method based on temporal difference learning, applied to a cluster containing multiple nodes, characterized in that... The method includes: S1, Node Information Structure Definition: Define and construct the node information structure based on the process of adaptive weighted task scheduling based on temporal difference learning; S2, Node initialization: Based on the node information structure, initialize the cluster and each node in the cluster, and obtain the real-time computing power, network latency, node reputation value, peak computing power of all nodes and minimum latency of all nodes in the network in real time. S3, Node Filtering and Task Execution: Receive and parse adaptive weighted task scheduling requests in real time, obtain the parsed tasks, filter each node for circuit breaker periods, determine non-circuit breaker nodes, and calculate the weights of the non-circuit breaker nodes based on a three-dimensional weight calculation model. The weights of the non-circuit-breaker nodes Sort by size and filter by weight. The non-circuit-breaker node with the largest value is selected as the task execution node. The parsed task is sent to the task execution node for execution, and the execution result is returned. S4. Node Information Update: Verify the execution result, and update the node reputation value and node status of the task execution node according to the verification result and the node reputation value dynamic update formula; In step S4, the step of verifying the execution result and updating the node reputation value and node status of the task execution node according to the verification result and the node reputation value dynamic update formula includes: The execution results include successful task execution, failed task execution, and malicious behavior; The verification results include verification passed, verification failed, and malicious behavior; S41, when the execution result is that the task was successfully executed, the verification result is that the verification passed, and a reward is sent to the task execution node, and the task reward value of the task execution node is... In addition to the success reward value, the system dynamically updates the formula based on the node reputation value and the current node reputation value of the task execution node. Update the node reputation value of the task execution node. ; S42, when the execution result is task execution failure, the verification result is verification failure, then the consecutive failure count of the task execution node is incremented by 1, and the weight of the task execution node is updated according to the updated consecutive failure count of the task execution node or the single task delay, and the circuit breaker period is set. The task reward value of the task execution node Subtract the first failure penalty value, and simultaneously update the formula dynamically based on the node reputation value and the current node reputation value of the task execution node. Update the node reputation value of the task execution node. ; Return to step S3, select a new task execution node, send the parsed task to the new task execution node for execution, and return the execution result; S43, when the execution result is malicious behavior, the verification result is malicious behavior, then the task reward value of the task execution node is... Subtract the second failure penalty value, and dynamically update the formula based on the node reputation value and the current node reputation value of the task execution node. Update the node reputation value of the task execution node. The weight of the task execution node is set to 0, the task execution node is marked as a circuit breaker node, and the circuit breaker period of the task execution node is set according to the preset circuit breaker period. Return to step S3, select a new task execution node, send the parsed task to the new task execution node for execution, and return the execution result; in, If the node reputation value of the task execution node is updated If the value is greater than 100, then the updated node reputation value of the task execution node will be used. The value is set to 100, if the task execution node updates the node reputation value. If the value is less than 0, then the node reputation value of the task execution node will be updated. The value is set to 0; The success reward value, the first failure penalty value, and the second failure penalty value are all preset according to the needs of task scheduling, and the second failure penalty value is greater than the first failure penalty value.

2. The method as described in claim 1, characterized in that, The node information structure includes: Real-time computing power (FLOPS), network latency (ms), node reputation value, last update timestamp, number of consecutive failures, circuit breaker expiration timestamp, global maximum computing power, single task latency, global minimum latency, node real-time computing power, peak computing power of all nodes in the network, node network round-trip latency, minimum latency of all nodes in the network, and task reward value. The node reputation value is greater than or equal to 0 and less than or equal to 100.

3. The method as described in claim 1, characterized in that, Step S2, the step of initializing the cluster and each node in the cluster based on the node information structure, specifically includes: S211, based on the node information structure, obtain the real-time computing power, network latency, node reputation value and node latency of each node in the cluster, determine the peak computing power of the entire network nodes and the minimum latency of the entire network nodes according to the real-time computing power and network latency of each node, and set the number of consecutive failures and the circuit breaker expiration timestamp of each node in the cluster to zero. S212, set the single task delay of each node to a value greater than or equal to the single task delay threshold, wherein the single task delay threshold is preset by the operator according to the requirements of adaptive weight task execution; S213 sets the initial value of the task reward value and the number of consecutive failures for each node in the cluster to zero.

4. The method as described in claim 3, characterized in that, In step S211, the step of obtaining the real-time computing power, network latency, node reputation value, and node latency of each node in the cluster based on the node information structure, and determining the peak computing power and minimum latency of the entire network nodes based on the real-time computing power and network latency of each node, further includes: S211-1, Node computing power acquisition: Perform 1024×1024 matrix multiplication operations on each node in the cluster according to a preset number of tests, calculate and obtain the number of floating-point operations per second for each node performing the 1024×1024 matrix multiplication operation each time, and the average number of floating-point operations per second after performing the preset number of tests, take the average number of floating-point operations per second after performing the preset number of tests as the real-time computing power of each node, and determine the peak computing power of the entire network nodes based on the real-time computing power of each node; S211-2, Node latency collection: At a preset test time, send a preset number of ICMP Echo requests to each node, record the round-trip time of each ICMP Echo request sent and returned by each node, remove outliers from all recorded round-trip times, and take the median value of all round-trip times after removing outliers as the network latency of each node. Determine the minimum latency of all nodes in the network based on the network latency of each node.

5. The method as described in claim 1, characterized in that, In step S3, the step of filtering each node by its circuit breaker period to determine the non-circuit breaker nodes further includes: Obtain the circuit breaker expiration timestamps of each node in the cluster, and determine the circuit breaker attribute of the node based on the circuit breaker expiration timestamps. The circuit breaker attribute includes circuit breaker nodes and non-circuit breaker nodes. If the circuit breaker expiration timestamp of the node is greater than or equal to the current time, then the node is a circuit breaker node; if the circuit breaker expiration timestamp of the node is less than the current time, then the node is a non-circuit breaker node. The weight of the circuit breaker node is zero, and task allocation will be suspended when a node is identified as a circuit breaker node.

6. The method as described in claim 1, characterized in that, In step S3, the three-dimensional weight calculation model is as follows: , in, The weight of the node. α, β, and γ are all weighting coefficients. The computing power dimension in the weighting coefficients. The network latency dimension is part of the weighting coefficients. The reputation dimension in the weighting coefficients. , For the real-time computing power of the node, This represents the peak computing power of all nodes in the network. For the round-trip latency of the node network, This represents the minimum latency across all nodes in the network. This represents the node's reputation value.

7. The method as described in claim 1, characterized in that, The formula for dynamically updating the node reputation value is: ; in, The updated node reputation value for the task execution node. The current node reputation value of the task execution node. The task reward value for the task execution node. The learning rate, representing the task reward value, is used to dynamically adjust the reward or penalty intensity for the task execution nodes. .

8. The method as described in claim 1, characterized in that, In step S42, the step of updating the weight of the task execution node based on the updated consecutive failure count or single task delay, and setting the circuit breaker period, further includes: If the number of consecutive failures of the updated task execution node is greater than or equal to 3, or the single task delay is greater than the single task delay threshold, then the weight of the task execution node is set to 0, the task execution node is marked as a circuit breaker node, the circuit breaker period of the task execution node is set based on the circuit breaker period calculation formula, and the circuit breaker expiration timestamp of the task execution node is updated. in, The formula for calculating the circuit breaker period is as follows: In the formula, The circuit breaker period is set. This represents the updated consecutive failure count for the task execution node.

9. A system for implementing the adaptive weighted task scheduling method based on temporal difference learning as described in claim 1, characterized in that, The system includes a structure definition module, a node information acquisition module, a node filtering module, and a node information update module; in, The structure definition module is used for defining the node information structure: defining and constructing the node information structure according to the process of adaptive weighted task scheduling based on temporal difference learning; The node information acquisition module is used for node initialization: based on the node information structure, it initializes the cluster and each node in the cluster, and obtains the real-time computing power, network latency, node reputation value, peak computing power of all nodes and minimum latency of all nodes in the network in real time. The node filtering module is used to filter nodes and execute tasks: it receives and parses adaptive weighted task scheduling requests in real time, obtains the parsed tasks, filters each node for circuit breaker periods, determines non-circuit breaker nodes, and calculates the weights of the non-circuit breaker nodes based on a three-dimensional weight calculation model. The weights of the non-circuit-breaker nodes Sort by size and filter by weight. The non-circuit-breaker node with the largest value is selected as the task execution node. The parsed task is sent to the task execution node for execution, and the execution result is returned. The node information update module is used for node information update: verifying the execution result, and updating the node reputation value and node status of the task execution node according to the verification result and the node reputation value dynamic update formula.

Images