Edge computing scheduling optimization method based on actor-critic model

Abstract

The present invention relates to the field of edge computing technology, specifically to an edge computing scheduling optimization method based on an actor-critic model, comprising the following steps: recording resource change calculation standard deviations, weighting feature priorities based on latency overflows, generating prediction results, extracting load trends to generate instantaneous curves, dividing acquisition rates into intervals, generating adjustment benchmarks, screening high-latency actions, normalizing gradients, dynamically adjusting learning rates, and generating optimization solutions. The present invention enhances bottleneck identification by adjusting feature priorities based on weighted latency and overflow rate changes, simultaneously extracts load trends and resource directions, constructs instantaneous resource curves, and improves change prediction accuracy. It also strengthens load regulation by dividing states based on rate and adjusting thresholds, screening actions and re-prioritizing them, and improving scheduling response. The normalized gradient differential dynamically adjusts the learning rate, accelerating strategy convergence.

Description

Technical Field

[0001] The present invention relates to the field of edge computing technology, and in particular to an edge computing scheduling optimization method based on an actor-critic model. Background Art

[0002] The field of edge computing technology encompasses a technical system that migrates computing tasks and data processing functions from centralized data centers to network edge nodes. The core of this technology lies in reducing data transmission latency, improving system response speed and service quality, and effectively alleviating the load pressure on central nodes by deploying distributed computing resources. Edge computing architecture typically consists of three layers: terminal devices, edge nodes, and cloud centers. By processing information close to the data source and optimizing resource allocation and energy management, it is widely used in smart manufacturing, Internet of Vehicles, smart cities, the Internet of Things, and real-time interactive application scenarios. Its overall technical field systematically covers areas such as edge node computing resource management, task distribution and scheduling strategies, data synchronization and consistency assurance, and edge intelligent model deployment and inference optimization.

[0003] Among them, the edge computing scheduling optimization method based on the actor-critic model refers to a technical solution that uses the actor-critic algorithm structure in the field of reinforcement learning to achieve parallel updates of decision-making and evaluation for computing resource allocation and task scheduling optimization problems in an edge computing environment. The technical matters targeted by this patent subject cover the optimal scheduling decision-making under the condition of limited resources of edge nodes in the case of multi-task concurrency. By establishing an environmental state representation, designing an action generation mechanism based on policy gradients, and a performance evaluation module based on value functions, the resource scheduling strategy of edge computing nodes is optimized in a joint training manner. Specifically, the state observation is input into the neural network, the probability distribution of scheduling actions is output, and the reward function is combined to design a guidance strategy and synchronize the update of the value function, thereby completing the optimization process of task scheduling in the edge environment.

[0004] Existing technologies often rely on static value analysis to detect node resource status changes, lacking a mechanism to extract continuous features of resource fluctuations. This results in delayed feature recognition during node resource fluctuations, hindering the precise formulation of scheduling policies. Load trend extraction typically relies on a single point-in-time load value, failing to effectively reflect the directionality and continuity of load evolution, resulting in inaccurate predictions of resource utilization trends. Load status interval demarcation relies on fixed thresholds and lacks a dynamic rate-of-change adjustment process, making it difficult to adapt to scenarios with drastic node load fluctuations and prone to either slow or overreaction. Action selection relies primarily on static priority settings, failing to dynamically select the optimal action based on real-time load status, resulting in insufficient scheduling responsiveness. Local policy updates typically employ a uniform step size adjustment, failing to refine learning rate changes based on local gradient trends. This reduces the policy's adaptability to load fluctuations and can easily lead to delayed or ineffective policy updates. In high-concurrency, rapidly changing load scenarios, this can lead to decreased node resource utilization efficiency and increased service response latency. Summary of the Invention

[0005] The purpose of the present invention is to solve the shortcomings of the existing technology and propose an edge computing scheduling optimization method based on the actor-critic model.

[0006] To achieve the above objectives, the present invention adopts the following technical solution: an edge computing scheduling optimization method based on the actor-critic model, comprising the following steps:

[0007] S1: Synchronously record changes in node CPU utilization, memory usage, and bandwidth usage, calculate standard deviations and change amplitudes, calculate weighted ratios based on response delay and overflow rate changes, adjust feature priorities based on sensitivity thresholds, and generate feature sensitivity prediction results.

[0008] S2: Based on the feature sensitivity prediction results, call the real-time CPU and memory usage, set the weighted average of resource weights, extract the load fluctuation trend, build the instantaneous resource curve and generate the instantaneous resource utilization trend;

[0009] S3: collecting time slice load values according to the instantaneous resource utilization trend, extracting a change rate data set, dividing the load state intervals, adjusting and comparing with the node basic load threshold, and generating a dynamic load adjustment benchmark;

[0010] S4: Determine the instantaneous load range according to the dynamic load adjustment benchmark, filter the sensitivity response action set, filter the high-latency actions based on the delay label, and form a scheduling action screening list;

[0011] S5: Extract the policy parameter update batch according to the scheduling action screening list, obtain the local gradient change, perform normalized difference processing, dynamically adjust the learning rate amplitude based on the difference mean and the threshold, and generate a scheduling optimization plan.

[0012] As a further solution of the present invention, the feature sensitivity prediction results include the CPU utilization change range, memory occupancy change range, bandwidth occupancy change range, task response delay change, overflow rate change, and feature priority adjustment coefficient; the instantaneous resource utilization trend includes load fluctuation trend, resource utilization direction change, and instantaneous resource utilization curve; the dynamic load adjustment benchmark includes load change rate distribution, load state interval division, and basic load threshold adjustment basis; the scheduling action screening list includes sensitivity response action set, action execution delay statistics, and action execution priority sorting; the scheduling optimization scheme includes strategy parameter update batches, local gradient change vectors, normalized difference set mean, and learning rate adjustment range.

[0013] As a further solution of the present invention, the specific steps of S1 are:

[0014] S101: Obtain the node CPU utilization, node memory occupancy, and node bandwidth occupancy, calculate the dynamic weight change rate based on the periodic parameter value after continuous sampling, and call the ratio of the square mean of the change rate deviation to the number of nodes to generate the node parameter change standard deviation;

[0015] S102: Based on the standard deviation of the node parameter change, the change rates of CPU utilization, memory occupancy, and bandwidth occupancy are normalized, and the call delay change and overflow rate change are multiplied and accumulated with the normalized amplitudes respectively, and the node parameter sensitivity result is obtained based on the node number ratio;

[0016] S103: Based on the node parameter sensitivity result, call the node feature sensitivity threshold, compare the sensitivity coefficient with the threshold, adjust the node feature priority state, establish a mapping relationship between the standard deviation change range and the feature priority, and obtain the feature sensitivity prediction result.

[0017] As a further solution of the present invention, the dynamic weight change rate calculation formula is specifically:

[0018] ;

[0019] in, Representative Node In time The dynamic weight change rate, Representative Node In time The CPU utilization, memory usage or bandwidth usage sampling value, represents the time interval between consecutive samplings, Represents the total time length of the cycle parameters, Represents the number of nodes, Representative Node The effective sampling interval within the cycle.

[0020] As a further solution of the present invention, the specific steps of S2 are:

[0021] S201: Based on the feature sensitivity prediction result, call the real-time CPU utilization and memory occupancy of the node, set the node resource weight coefficient, take the weighted average of the CPU utilization and memory occupancy respectively, calculate the weighted weight coefficient of the node resource utilization, and generate the node load fluctuation trend;

[0022] S202: Based on the node load fluctuation trend, the change in node resource utilization per unit time is called, and the upper and lower bounds are compared according to the set load change interval reference value to determine the resource utilization direction, extract the positive and negative signs of the change and establish a time series to obtain the node resource direction change;

[0023] S203: According to the node load fluctuation trend and the node resource direction change, call the node time series order, integrate the change amplitude and direction information, generate instantaneous resource data points for each node, draw a resource change curve based on the time series, and obtain the instantaneous resource utilization trend.

[0024] As a further solution of the present invention, the weight coefficient calculation formula after weighting the node resource utilization is specifically:

[0025] ;

[0026] in, Represents the weight coefficient of the node resource utilization, Represents the weighted coefficient of the node CPU utilization, Represents the weighted coefficient of the node memory usage, Represents the value of the node's current real-time CPU utilization. Represents the value of the node's current real-time memory usage. Represents the total number of nodes currently participating in the statistics.

[0027] As a further solution of the present invention, the specific steps of S3 are:

[0028] S301: Based on the instantaneous resource utilization trend, collect the time slice load values of the node within the time period, arrange the time slice load values, extract the unit time load change based on the difference between adjacent time slices, integrate the change value set, and generate a node load change rate data set;

[0029] S302: Based on the node load change rate data set, set a load state interval according to a change rate range, call the load change rate data and compare it with the interval boundary, classify the node load state according to the result, and obtain a node load state interval set;

[0030] S303: Based on the node load state interval set, call the node basic load threshold benchmark, calculate the load deviation value based on the comparison between the node load state and the benchmark, integrate the node deviation data to generate an adjustment set, and obtain a dynamic load adjustment benchmark.

[0031] As a further solution of the present invention, the load deviation value calculation formula is specifically:

[0032] ;

[0033] in, Represents the node load deviation value, Represents the actual load data of the node, Represents the node basic load threshold data, Represents the average of the basic load thresholds of all nodes, Represent all The sum of the absolute values of the differences between the actual load and the benchmark load of each node.

[0034] As a further solution of the present invention, the specific steps of S4 are:

[0035] S401: calling the dynamic load adjustment benchmark, comparing the node instantaneous load value with the upper and lower bounds of the adjustment benchmark interval, determining the node load position, extracting the node interval mark information, integrating the node position mark data set, and generating the node instantaneous load position mark value;

[0036] S402: Based on the instantaneous load position tag value of the node, filter the actions corresponding to the sensitivity response flag in the action pool, call the action delay tag to accumulate the delay times, filter the action set with the highest delay times, integrate the ranking information, and obtain the optimal set of sensitive actions;

[0037] S403: Based on the preferred set of sensitive actions, the original execution priority of the action is called, the priority order is readjusted according to the ranking of the number of delays, the action priority weight is allocated, the node action priority change information is integrated, and a scheduling action screening list is obtained.

[0038] As a further solution of the present invention, the specific steps of S5 are:

[0039] S501: Extracting local policy parameter update batches from the returned data according to the scheduling action screening list, arranging the batch node policy parameter change values, extracting local gradient changes, integrating node change data, and generating a local gradient change vector set;

[0040] S502: Based on the local gradient change vector set, extract batch change vector data, perform normalized difference operation on the local gradient change vector and the batch change vector, integrate the node normalized difference values to form a node normalized difference sequence, and obtain a normalized difference set;

[0041] S503: Based on the normalized difference set, the normalized difference mean is compared with the upper and lower thresholds to determine the direction of mean deviation, adjust the corresponding node learning rate amplitude, integrate the node learning rate change information, and obtain a scheduling optimization plan.

[0042] Compared with the prior art, the advantages and positive effects of the present invention are:

[0043] In the present invention, feature priorities are adjusted by weighting delay and overflow rate changes, bottleneck identification is enhanced, load trends and resource directions are extracted synchronously, instantaneous resource curves are constructed, and the accuracy of change prediction is improved. States are divided and thresholds are adjusted according to rates, load regulation is strengthened, actions are screened and priorities are rearranged, scheduling response is improved, and normalized gradient differences are used to dynamically adjust the learning rate, thereby accelerating strategy convergence. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0045] Figure 1 Schematic diagram of the steps of the present invention. DETAILED DESCRIPTION

[0046] The technical solution of the present invention is described below in conjunction with the accompanying drawings.

[0047] In the embodiments of the present invention, words such as "exemplarily" and "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as an "exemplary" in the present invention should not be interpreted as being preferred or advantageous over other embodiments or designs. Rather, the use of the word "exemplary" is intended to present concepts in a concrete manner. Furthermore, in the embodiments of the present invention, "and / or" can mean both or either of the two.

[0048] In the embodiments of the present invention, the terms "image" and "picture" may sometimes be used interchangeably. It should be noted that, when the distinction is not emphasized, the meanings they convey are the same. The terms "of," "corresponding," and "corresponding" may sometimes be used interchangeably. It should be noted that, when the distinction is not emphasized, the meanings they convey are the same.

[0049] In the embodiments of the present invention, sometimes a subscript such as W1 may be written as a non-subscript such as W1. When the difference is not emphasized, the meanings to be expressed are the same.

[0050] In order to make the technical problems, technical solutions and advantages to be solved by the present invention clearer, a detailed description will be given below with reference to the accompanying drawings and specific embodiments.

[0051] See also Figure 1 ,The edge computing scheduling optimization method based on the actor-critic model includes the following steps:

[0052] S1: Obtain the node CPU utilization, node memory occupancy, and node bandwidth occupancy, record the parameter values in two consecutive sampling periods through time synchronization, calculate the standard deviation based on the value change rate and extract the change amplitude, calculate the sensitivity coefficient after normalizing the change amplitude of each standard deviation, call the node task response delay change and overflow rate change and the standard deviation change amplitude to calculate the weighted ratio, adjust the node feature priority according to the set sensitivity threshold, and generate the feature sensitivity prediction result;

[0053] S2: Based on the feature sensitivity prediction results, the real-time CPU utilization and memory occupancy of the node are called, and the node resource weight coefficient is set for weighted averaging. The load fluctuation trend within a unit time is extracted and generated. The node resource utilization direction is determined based on the load change interval. The load trend and direction change information are integrated to construct the node instantaneous resource curve and generate the instantaneous resource utilization trend.

[0054] S3: Based on the instantaneous resource utilization trend, the load values of the node in the time slice are collected, and the load change rate data set is extracted. The load state intervals are divided according to the change rate set. The node basic load threshold benchmark is used to adjust and compare the differentiated state intervals to generate a dynamic load adjustment benchmark.

[0055] S4: Call the dynamic load adjustment benchmark to determine the position of the node's instantaneous load value within the adjustment benchmark range, filter the action set with the most sensitive response in the action pool, select the action with the largest number of delays based on the action execution delay label, reallocate the action execution priority, and generate a scheduling action screening list;

[0056] S5: Extract the local policy parameter update batch from the action execution return data according to the scheduling action screening list, obtain the local policy gradient change vector within the batch, perform normalized difference operation between the gradient change vector and the batch change vector, compare the normalized difference set mean with the upper and lower limit thresholds, adjust the learning rate amplitude, and generate a scheduling optimization plan.

[0057] The feature sensitivity prediction results include the change range of CPU utilization, memory occupancy, bandwidth occupancy, task response delay, overflow rate, and feature priority adjustment coefficient. The instantaneous resource utilization trend includes load fluctuation trend, resource utilization direction change, and instantaneous resource utilization curve. The dynamic load adjustment benchmark includes load change rate distribution, load state interval division, and basic load threshold adjustment basis. The scheduling action screening list includes sensitivity response action set, action execution delay statistics, and action execution priority sorting. The scheduling optimization plan includes strategy parameter update batches, local gradient change vectors, normalized difference set mean, and learning rate adjustment range.

[0058] The specific steps of S1 are:

[0059] S101: Obtain the node CPU utilization, node memory occupancy, and node bandwidth occupancy, calculate the dynamic weight change rate based on the periodic parameter value after continuous sampling, and call the ratio of the square mean of the change rate deviation to the number of nodes to generate the node parameter change standard deviation;

[0060] The dynamic weight change rate calculation formula is as follows:

[0061] ;

[0062] in, Representative Node In time The dynamic weight change rate, Representative Node In time The CPU utilization, memory usage or bandwidth usage sampling value, represents the time interval between consecutive samplings, Represents the total time length of the cycle parameters, Represents the number of nodes, Representative Node The effective sampling interval within the cycle;

[0063] node In time CPU utilization The system monitoring tool collects data in real time, with a sampling period of 5 seconds and a time interval of Second, For nodes exist The CPU utilization monitoring value in seconds, for example , ; Cycle parameters Seconds, number of nodes , the effective sampling interval of each node Derived from system log statistics, for example Second, Second, Seconds, total Seconds; Substituting into the formula we get:

[0064] ;

[0065] Parameter setting basis: Seconds are defined by the default sampling frequency of the monitoring system, is the total time of the actual effective sampling times of all nodes in the cycle, The seconds are set by the period parameter of the system configuration; and Obtained in real time through the performance monitoring interface, the unit is percentage; calculation results Representation node The weighted change rate of , whose value is used as a component of the node change rate set and is used for the subsequent calculation of the mean squared deviation;

[0066] Explanation of numerical results: Reflection Node In time The dynamic change amplitude relative to the previous time point combines the difference between the current and historical values, the normalization adjustment of the historical values, and the dynamic weights of all node sampling intervals within the period. This result is an element of the node change rate set and participates in the subsequent standard deviation calculation;

[0067] Non-numeric data quantification: CPU utilization is directly obtained through system performance counters without additional quantification; effective sampling interval Calculated by the difference between the timestamps of two consecutive valid samples recorded in the system log, such as the timestamp Second, seconds, then Second;

[0068] Verification of reasonable parameter range: CPU utilization The realistic and reasonable range is 0% to 100%, and the time interval In monitoring systems, typically 1 to 60 seconds, total No more than (3600 seconds × 3 = 10800 seconds). The actual value of 14 seconds meets the constraint.

[0069] Formula derivation process: first calculate the absolute difference of the numerator , the denominator , get the basic change rate ; Then calculate the dynamic weight item , and the final weighted change rate is ;

[0070] This result shows that the node The rate of change of is affected by its own historical value and the global sampling interval distribution within the period. Its value is used as the input item of the node change rate set, which is used to subsequently calculate the ratio of the squared mean of the deviation to the number of nodes, and finally generate the standard deviation of the node parameter change.

[0071] S102: Based on the standard deviation of node parameter changes, the change rates of CPU utilization, memory occupancy, and bandwidth occupancy are normalized. The change in call delay and overflow rate are multiplied and accumulated with the normalized amplitudes respectively. The node parameter sensitivity result is obtained based on the node number ratio.

[0072] Based on the standard deviation of node parameter changes, each node's CPU utilization, memory usage, and bandwidth usage change rates are normalized. The average change rate for all nodes is subtracted from the change rate obtained for each sampling, and then normalized using the standard deviation of the node parameter change. For example, if a node's CPU change rate is +2% in a certain sampling, the average CPU change rate is +1.5%, and the standard deviation of the CPU change is 0.2%, then the normalized result is (2% - 1.5%) / 0.2% = 2.5. Following this method, each change rate is normalized. After completion, the node's latency change and overflow rate change are read. For example, if the node latency change is +10ms, the overflow rate change is +0. 5%, multiplied by the corresponding normalized amplitude respectively. For example, the delay change multiplied by the normalized CPU change of 2.5 is 25, and the overflow rate change multiplied by the bandwidth change of 2.0 is 1. The delay product and overflow rate product of each node are added up. For example, the total delay of 10 nodes is 250, and the total overflow rate is 10. Then divide them by the number of nodes 10 to obtain the delay sensitivity of 25 and the overflow rate sensitivity of 1, as the node parameter sensitivity results. The setting of the node parameter change standard deviation refers to the stability of the node historical data. Usually, stable nodes are set to a change of less than 1%, and nodes with obvious fluctuations are set to a change of more than 2%. When the change rate is abnormal during normalization, if the normalization amplitude exceeds ±3, it is identified as a high fluctuation node.

[0073] S103: Based on the node parameter sensitivity result, the node feature sensitivity threshold is called, the sensitivity coefficient is compared with the threshold, the node feature priority state is adjusted, a mapping relationship between the standard deviation change range and the feature priority is established, and the feature sensitivity prediction result is obtained;

[0074] According to the sensitivity results of node parameters, the node feature sensitivity threshold is set. The delay change sensitivity threshold is set to ±20, and the overflow rate change sensitivity threshold is set to ±2. The threshold setting is based on the node business level. For example, the threshold of important business nodes is set to ±15, the threshold of ordinary nodes is set to ±20, and the threshold of test nodes is set to ±25. The specific setting is determined based on the node's past operation data and the system service quality requirements. For example, the delay change of important nodes within 5ms is considered normal, and the change of more than 10ms is considered a sensitive abnormality. Therefore, the sensitivity threshold is set to ±15. Ordinary nodes are allowed to change slightly larger, and the threshold is set to ±20. When the node sensitivity exceeds the positive and negative thresholds, Adjust the node feature priority status. If the delay sensitivity of 25 exceeds the normal node threshold of 20, the node priority will be raised by one level. For example, the original priority is 2, and it is raised to 1. At the same time, the change in the node standard deviation is recorded. For example, if the standard deviation of a node is 0.15%, it will be archived according to the mapping relationship σ<0.2% corresponding to priority 1, σ∈[0.2%,0.5%) corresponding to priority 2, and σ≥0.5% corresponding to priority 3. The node with σ=0.15% will be finally set to priority 1. Finally, based on the node feature sensitivity and the change in standard deviation, the node feature sensitivity prediction result is output, indicating that the sensitivity of the node will maintain a high level of fluctuation in the future cycle.

[0075] The specific steps of S2 are:

[0076] S201: Based on the feature sensitivity prediction result, call the real-time CPU utilization and memory occupancy of the node, set the node resource weight coefficient, take the weighted average of the CPU utilization and memory occupancy respectively, calculate the weighted weight coefficient of the node resource utilization, and generate the node load fluctuation trend;

[0077] The calculation formula for the weight coefficient after weighting the node resource utilization is as follows:

[0078] ;

[0079] in, Represents the weight coefficient of the node resource utilization, Represents the weighted coefficient of the node CPU utilization, Represents the weighted coefficient of the node memory usage, Represents the value of the node's current real-time CPU utilization. Represents the value of the node's current real-time memory usage. Represents the total number of nodes currently participating in the statistics;

[0080] Parameter definition and data source:

[0081] : CPU utilization weighting factor. This factor, derived from historical data analysis, is used to balance the impact of CPU utilization on the overall node resource weight. The specific value is set to 0.6, based on average data and taking into account the importance of CPU to node performance.

[0082] : Weight coefficient for memory usage. This coefficient reflects the impact of memory usage on node performance. The value is set to 0.4, also based on historical data analysis, considering that the relative impact of memory usage on performance is slightly lower than that of CPU.

[0083] : The current real-time CPU utilization of the node. This data is obtained through the real-time monitoring system. For example, the measured value is 70%.

[0084] : The current real-time memory usage of the node. This is also obtained through the real-time monitoring system and the measured value is set to 55%.

[0085] : The total number of nodes currently participating in the statistics. This value is calculated in real time. For example, there are currently 10 nodes running.

[0086] Formula derivation process and calculation examples:

[0087] Calculate the weighted average CPU and memory usage:

[0088] ;

[0089] ;

[0090] Compute the sum of squares and absolute values:

[0091] ;

[0092] ;

[0093] Substitute the above values into the formula for W:

[0094] ;

[0095] Explanation of numerical results: This result indicates that the average resource load weight coefficient of the current node group, after weighing CPU and memory utilization, is 14.35. This value reflects the average resource load of the node at a given moment. Higher values indicate relatively concentrated resource usage and may require load balancing.

[0096] S202: Based on the node load fluctuation trend, the change in node resource utilization per unit time is called, and the upper and lower bounds are compared with the set load change interval benchmark value to determine the resource utilization direction. The positive and negative signs of the change are extracted and a time series is established to obtain the node resource direction change;

[0097] Based on the node load fluctuation trend, the node resource utilization change per unit time is called, and the upper and lower bounds are compared according to the set load change interval benchmark value. The load change benchmark value is set according to the actual business carrying capacity of the node. For example, the benchmark value is set to ±5% for ordinary business nodes, ±3% for important business nodes, and ±10% for test nodes. Node A is an important computing node and the benchmark value is set to ±3%. The set of node A changes {1.5%, -2.5%, 1.5%, 2%} is compared one by one. If the change is greater than +3%, it is marked as a significant positive increase. If the change is less than -3%, it is marked as a significant negative decrease. If the change is within the range of [-3%, +3%], it is marked In order to ensure stability, the actual comparison found that 1.5%, -2.5%, 1.5%, and 2% all fall within the range of [-3%, +3%] and are all marked as stable. At the same time, the positive and negative signs of the changes are recorded. If the change is positive, it is marked as +1, and if it is negative, it is marked as -1. The stable change is marked as 0. The change sign sequence of node A is {+1, -1, +1, +1}. Subsequently, the node time series is established in the order of sampling time, and each change sign is associated with the timestamp to form a node resource direction change sequence. For example, if the corresponding timestamps are T1, T2, T3, and T4, {(T1, +1), (T2, -1), (T3, +1), (T4, +1)} are obtained to generate the node resource direction change.

[0098] S203: Based on the node load fluctuation trend and the node resource direction change, call the node time series sequence, integrate the change amplitude and direction information, generate instantaneous resource data points for each node, draw a resource change curve based on the time series, and obtain the instantaneous resource utilization trend;

[0099] According to the node load fluctuation trend and the change in the node resource direction, the node sampling time series order is read, and the change amplitude and change direction information are integrated for each moment. For example, the change of node A at time T1 is 1.5% in the direction of +1, the change is -2.5% in the direction of -1 at T2, the change is 1.5% in the direction of +1 at T3, and the change is 2% in the direction of +1 at T4. Instantaneous resource data points are established one by one. The data point content contains three parts: timestamp, change amplitude, and change direction, such as (T1, 1.5%, +1), (T2, 2.5%, -1), (T3, 1.5%, +1), and (T4, 2%, +1). All instantaneous resource data points are sorted in chronological order to form a complete time series set. Based on this time series data, a resource change curve is drawn. The horizontal axis is the time axis, the vertical axis is the resource change amplitude, and the change direction is divided into positive and negative. For example, positive changes are drawn in the positive direction, and negative changes are drawn in the negative direction. The change curve formed by node A from T1 to T4 rises from T1, falls at T2, rises again at T3, and continues to rise at T4. The instantaneous resource utilization trend of node A is obtained, which is a major upward growth trend. If there are more than three consecutive positive changes, the node trend is considered to be continuous growth. If there are more than three negative changes, it is considered to be continuous decline. Otherwise, it is considered to be a fluctuating state. Node A has three consecutive positive changes. According to the above rules, its instantaneous resource utilization trend is considered to be continuous growth.

[0100] The specific steps of S3 are:

[0101] S301: Based on the instantaneous resource utilization trend, collect the time slice load values of the node within the time period, arrange the time slice load values, extract the unit time load change based on the difference between adjacent time slices, integrate the change value set, and generate a node load change rate data set;

[0102] According to the instantaneous resource utilization trend, the corresponding node set is first selected. For each node, the time slice load value within the node time period is collected at the set time interval. For example, load data is collected every 10 seconds. The load values continuously collected by node A in a time period are recorded as 50%, 52%, 49%, 53%, 55%, and 54%. Then, all the collected time slice load values are arranged in chronological order to form a time series {T1: 50%, T2: 52%, T3: 49%, T4: 53%, T5: 55%, T6: 54%}. The difference extraction is performed based on the load values of adjacent time slices, that is, the load value of the current time slice is subtracted from the load value of the previous time slice to obtain the load change per unit time. For example, T2-T1=52%-50%= 2%, T3-T2=49%-52%=-3%, T4-T3=53%-49%=4%, T5-T4=55%-53%=2%, T6-T5=54%-55%=-1%. All load changes are sorted out to form a change set of node A {+2%, -3%, +4%, +2%, -1%}. Then all change data are integrated to establish a node load change rate data set, where each change rate data contains a time slice identifier, change value, and change direction flag, for example (T2, 2%, +1), (T3, -3%, -1), (T4, 4%, +1), (T5, 2%, +1), (T6, -1%, -1), and finally the generation of the node load change rate data set is completed.

[0103] S302: Based on the node load change rate data set, set the load state interval according to the change rate range, call the load change rate data and compare it with the interval boundary, classify the node load state according to the result, and obtain the node load state interval set;

[0104] Based on the node load change rate data set, for each node's collected change rate set, set the load state interval. For example, the change rate range is divided into: [-5%, -2%) as the decline interval, [-2%, +2%] as the stable interval, (2%, 5%] as the rise interval, greater than 5% is a sharp rise interval, less than -5% is a sharp decline interval. The interval division is set according to the node business stability requirements. For example, nodes with high real-time requirements use a narrower interval setting, and ordinary nodes use a regular interval setting. For the change amount set of node A {+2%, -3%, +4%, +2%, - 1%} are compared separately. For example, +2% falls in the rising interval of (2%, 5%], -3% falls in the falling interval of [-5%, -2%), +4% falls in the rising interval of (2%, 5%], +2% falls in the rising interval of (2%, 5%], and -1% falls in the stable interval of [-2%, +2%]. After comparing the change rate with the interval boundary one by one, the node load status is classified accordingly and marked as {rising, falling, rising, rising, stable} respectively. Finally, the load status interval set of node A is sorted out as {rising, falling, rising, rising, stable}, which is used as the basis for subsequent load adjustment actions.

[0105] S303: Based on the node load status interval set, call the node basic load threshold benchmark, compare the node load status with the benchmark, calculate the load deviation value, integrate the node deviation data to generate an adjustment set, and obtain a dynamic load adjustment benchmark;

[0106] The specific calculation formula for load deviation value is:

[0107] ;

[0108] in, Represents the node load deviation value, Represents the actual load data of the node, Represents the node basic load threshold data, Represents the average of the basic load thresholds of all nodes, Represent all The sum of the absolute values of the differences between the actual load and the benchmark load of each node;

[0109] Parameter Definition

[0110] :node The actual load data of the monitoring data is displayed. GB.

[0111] :node The basic load threshold is obtained by analyzing historical data and node load. GB.

[0112] : The average of all node basic load thresholds. GB, based on the load threshold average of all nodes in the network.

[0113] :node The actual load and basic load threshold. These data come from the network A monitoring system for each node.

[0114] : The total number of nodes, set to .

[0115] Specific parameter values and calculation methods

[0116] calculate :

[0117] ;

[0118] calculate And find the root:

[0119] ;

[0120] ;

[0121] Calculate the adjustment factor :

[0122] ;

[0123] Sum :

[0124] Assume that in this network, the statistical mean of the deviations is approximately GB, total number of nodes ,but:

[0125] ;

[0126] Calculate the denominator of the formula:

[0127] ;

[0128] Substitute the values into the main formula:

[0129] ;

[0130] Interpretation of the results

[0131] This result shows that the node The load deviation value is approximately , which means that the node The actual load is close to the average network load, but slightly higher. This deviation is calculated by The difference from the average baseline is calculated, taking into account the load deviation of the entire network. This value is used for further network load regulation to help maintain network balance and efficiency.

[0132] The specific steps of S4 are:

[0133] S401: Invoke the dynamic load adjustment benchmark, compare the node instantaneous load value with the upper and lower bounds of the adjustment benchmark interval, determine the node load position, extract the node interval mark information, integrate the node position mark data set, and generate the node instantaneous load position mark value;

[0134] The dynamic load adjustment benchmark is called. Based on the comparison of the node's instantaneous load value with the upper and lower bounds of the adjustment benchmark interval, the current instantaneous load value of the node is first obtained. Assume that the current instantaneous load value of node A is 53%, and the dynamic load adjustment benchmark of node A is [50%, 55%]. Therefore, the current instantaneous load value is compared with the adjustment benchmark interval. Node A's load value is between the upper and lower bounds, so its load position can be determined to be in the normal range. The load position flag information of node A is extracted as "normal range". This flag information is then compared with other nodes, and the load position flag data set of all nodes is integrated. For example, the load value of node B is 56%, which exceeds the upper limit of 55% and is therefore marked as "high load" interval. The load value of node C is 48%, which is below the lower limit of 50% and is marked as "low load" interval. The final result is a node load position flag value dataset, such as {node A: normal range, node B: high load, node C: low load}, which provides a basis for subsequent decision-making.

[0135] S402: Based on the node instantaneous load position tag value, filter the actions corresponding to the sensitivity response flag in the action pool, call the action delay tag to accumulate the delay times, filter the action set with the highest delay times, integrate the ranking information, and obtain the optimal set of sensitive actions;

[0136] Based on the instantaneous load position tag value of the node, the action pool corresponding to the sensitivity response flag is first read. The action pool contains multiple action response types, each of which has an associated delay tag. The delay tag is used to mark the delay when the action response is executed. For example, if the delay tag of an action "adjust load" is 3 times, it means that the action has been executed 3 times with a delay. The current load position of node A is marked as "normal range". The actions in the action pool are filtered. According to the cumulative delay times of the delay tags, the actions with the largest number of delays are selected for optimization. For example, action A has accumulated 5 delays, action B has accumulated 2 delays, and action C has accumulated 8 delays. When filtering, the actions with the highest cumulative delay times are first selected. For example, action C has 8 delays, action A has 5 delays, and action B has 2 delays. The delay number ranking information is integrated to obtain a sensitive action priority set, such as {action C: 8 times, action A: 5 times, action B: 2 times}. This set indicates that under the current load state, actions with higher delay times are prioritized to ensure faster and more effective resource adjustment.

[0137] S403: Based on the sensitive action selection set, the original execution priority of the action is called, the priority order is readjusted according to the delay number ranking, the action priority weight is assigned, the node action priority change information is integrated, and a scheduling action screening list is obtained;

[0138] Based on the preferred set of sensitive actions, the original execution priority of each action is first read. The action priority is set based on the action's impact on system load regulation. For example, the original execution priority of action A is 2, the original execution priority of action B is 3, and the original execution priority of action C is 1. The priority order is re-adjusted based on the number of delays. Action C has the most delays and ranks first, so its priority is adjusted to 1. Action A is second and is adjusted to 2. Action B has the fewest delays and is adjusted to 3. The adjusted priority is {action C: priority 1, action A: priority 2, action B: priority 3}. Next, priority weights are assigned to each action, and the weights are assigned in descending order from high to low priority. For example, action C has a weight of 0.6, action A has a weight of 0.3, and action B has a weight of 0.1. The node action priority change information is integrated to finally generate a scheduling action filter list, for example, {action C: weight 0.6, action A: weight 0.3, action B: weight 0.1}. This list is used by the scheduling system to determine which actions should be executed first, thereby improving load regulation efficiency.

[0139] The specific steps of S5 are:

[0140] S501: Filter the list of scheduling actions, extract the local policy parameter update batches in the returned data, arrange the policy parameter change values of the batch nodes, extract the local gradient change, integrate the node change data, and generate a local gradient change vector set;

[0141] According to the scheduling action screening list, the local policy parameter update batches in the returned data are extracted. First, the local policy parameter update records corresponding to each node in the current node set are determined. For example, node A is updated in batches 1, 3, and 5, and node B is updated in batches 2, 4, and 6. When arranging the batch node policy parameter change values, the policy parameter change amounts of each batch node are arranged in chronological order. For example, the change value of node A in batch 1 is +0.05, the change value of batch 3 is +0.02, and the change value of batch 5 is -0.01. The change value of node B in batch 2 is +0.03, the change value of batch 4 is -0.02, and the change value of batch 6 is +0.04. The local gradient change amount is extracted, that is, the difference between the policy change amounts of two adjacent batches is taken, for example The local gradient change of node A from the 1st batch to the 3rd batch is +0.02-+0.05=-0.03, and the local gradient change from the 3rd batch to the 5th batch is -0.01-+0.02=-0.03. Similarly, the local gradient change of node B from the 2nd batch to the 4th batch is -0.02-+0.03=-0.05, and from the 4th batch to the 6th batch is +0.04-(-0.02)=+0.06. All node change data are integrated, and the local gradient change extracted from each node is formed into a vector set. For example, the local gradient change vector of node A is {-0.03, -0.03}, and that of node B is {-0.05, +0.06}. The local gradient change vector set is generated for subsequent normalization processing.

[0142] S502: Based on the local gradient change vector set, extract batch change vector data, perform normalized difference operation on the local gradient change vector and the batch change vector, integrate the node normalized difference values, form a node normalized difference sequence, and obtain a normalized difference set;

[0143] Based on the local gradient change vector set, batch change vector data is extracted. For example, the batch change vector of node A is {+0.05, +0.02, -0.01}, and that of node B is {+0.03, -0.02, +0.04}. For each node, a normalized difference operation is performed between the local gradient change vector and the batch change vector. The normalization process is to subtract the mean of the corresponding batch change from each local gradient change and then divide it by the maximum absolute value range. For example, the batch change mean of node A is (+0.05+0.02-0.01) / 3=+0.02, and the maximum absolute value is 0.05. Therefore, the normalized difference of the first local gradient change of node A is (-0.03-0.02) / 0.05=-1, and the second local gradient change is (-0.03-0.02) / 0.05=-1. The normalized difference of the local gradient change is (-0.03-0.02) / 0.05=-1. The same is true for node B. The batch change mean is (+0.03-0.02+0.04) / 3=+0.0167, and the maximum absolute value is 0.04. The normalized difference of the first local gradient change is (-0.05-0.0167) / 0.04=-1.6667, and the second is (+0.06-0.0167) / 0.04=1.0833. The normalized difference values of the nodes are integrated separately to form a node normalized difference sequence. For example, the normalized difference sequence of node A is {-1, -1}, and the normalized difference sequence of node B is {-1.6667, 1.0833}. Finally, the overall normalized difference set is summarized for subsequent judgment analysis.

[0144] S503: Based on the normalized difference set, the normalized difference mean is compared with the upper and lower thresholds to determine the direction of mean deviation, adjust the corresponding node learning rate amplitude, integrate the node learning rate change information, and obtain a scheduling optimization plan;

[0145] According to the normalized difference set, first calculate the mean of the normalized difference sequence of each node. For example, the normalized difference sequence of node A is {-1, -1}, with a mean of -1, and the normalized difference sequence of node B is {-1.6667, 1.0833}, with a mean of (-1.6667+1.0833) / 2=-0.2917. Set the upper and lower limit thresholds of the normalized difference mean, for example, ±0.5. The threshold setting is based on the system scheduling stability standard. If the node deviation is within the range of [-0.5, 0.5], the load regulation is considered stable. If it exceeds the range, it is considered that adjustment is required. Compare the nodes. The mean and upper and lower limit thresholds show that the mean of node A, -1, is lower than the lower limit of -0.5, and the mean of node B, -0.2917, is within the range. It is determined that the mean deviation of node A is too negative, and node B is normal and does not need to be adjusted. The learning rate amplitude of node A is adjusted, and the original learning rate setting of 0.01 is increased by 20%. The adjusted learning rate is 0.012, and the learning rate of node B remains unchanged. The learning rate change information of all nodes is integrated to finally obtain the scheduling optimization plan. For example, the learning rate of node A is adjusted to 0.012, and the learning rate of node B remains at 0.01, forming an updated learning rate scheduling list.

[0146] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present invention should be included in the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection of the claims.

Claims

1. Edge computing scheduling optimization method based on actor-critic model, characterized by: The following steps are involved: S1: Synchronously record changes in node CPU utilization, memory usage, and bandwidth usage, calculate standard deviations and change amplitudes, calculate weighted ratios based on response delay and overflow rate changes, adjust feature priorities based on sensitivity thresholds, and generate feature sensitivity prediction results. S2: Based on the feature sensitivity prediction results, call the real-time CPU and memory usage, set the weighted average of resource weights, extract the load fluctuation trend, build the instantaneous resource curve and generate the instantaneous resource utilization trend; S3: Collect time slice load values according to the instantaneous resource utilization trend, extract the change rate data set, divide the load state interval, refer to the node basic load threshold adjustment comparison, and generate a dynamic load adjustment benchmark; the specific steps of S3 are: S301: Based on the instantaneous resource utilization trend, collect the time slice load values of the node within the time period, arrange the time slice load values, extract the unit time load change based on the difference between adjacent time slices, integrate the change value set, and generate a node load change rate data set; S302: Based on the node load change rate data set, set a load state interval according to a change rate range, call the load change rate data and compare it with the interval boundary, classify the node load state according to the result, and obtain a node load state interval set; S303: Based on the node load state interval set, call the node basic load threshold benchmark, calculate the load deviation value based on the comparison between the node load state and the benchmark, integrate the node deviation data to generate an adjustment set, and obtain a dynamic load adjustment benchmark; S4: Determine the instantaneous load range according to the dynamic load adjustment benchmark, filter the sensitivity response action set, filter the high-latency actions based on the delay label, and form a scheduling action screening list; S5: Extract the policy parameter update batch according to the scheduling action screening list, obtain the local gradient change, perform normalized difference processing, dynamically adjust the learning rate amplitude based on the difference mean and the threshold, and generate a scheduling optimization plan.

2. The edge computing scheduling optimization method based on the actor-critic model according to claim 1 is characterized in that: The feature sensitivity prediction results include the CPU utilization change range, memory occupancy change range, bandwidth occupancy change range, task response delay change, overflow rate change, and feature priority adjustment coefficient. The instantaneous resource utilization trend includes load fluctuation trend, resource utilization direction change, and instantaneous resource utilization curve. The dynamic load adjustment benchmark includes load change rate distribution, load state interval division, and basic load threshold adjustment basis. The scheduling action screening list includes sensitivity response action set, action execution delay statistics, and action execution priority sorting. The scheduling optimization scheme includes strategy parameter update batches, local gradient change vectors, normalized difference set mean, and learning rate adjustment range.

3. The edge computing scheduling optimization method based on the actor-critic model according to claim 1 is characterized in that: The specific steps of S1 are: S101: Obtain the node CPU utilization, node memory occupancy, and node bandwidth occupancy, calculate the dynamic weight change rate based on the periodic parameter value after continuous sampling, and call the ratio of the square mean of the change rate deviation to the number of nodes to generate the node parameter change standard deviation; S102: Based on the standard deviation of the node parameter change, the change rates of CPU utilization, memory occupancy, and bandwidth occupancy are normalized, and the call delay change and overflow rate change are multiplied and accumulated with the normalized amplitudes respectively, and the node parameter sensitivity result is obtained based on the node number ratio; S103: Based on the node parameter sensitivity result, call the node feature sensitivity threshold, compare the sensitivity coefficient with the threshold, adjust the node feature priority state, establish a mapping relationship between the standard deviation change range and the feature priority, and obtain the feature sensitivity prediction result.

4. The edge computing scheduling optimization method based on the actor-critic model according to claim 3 is characterized in that: The dynamic weight change rate calculation formula is specifically: ； in, Representative Node In time The dynamic weight change rate, Representative Node In time The CPU utilization, memory usage or bandwidth usage sampling value, represents the time interval between consecutive samplings, Represents the total time length of the cycle parameters, Represents the number of nodes, Representative Node The effective sampling interval within the cycle.

5. The edge computing scheduling optimization method based on the actor-critic model according to claim 1 is characterized in that: The specific steps of S2 are: S201: Based on the feature sensitivity prediction result, call the real-time CPU utilization and memory occupancy of the node, set the node resource weight coefficient, take the weighted average of the CPU utilization and memory occupancy respectively, calculate the weighted weight coefficient of the node resource utilization, and generate the node load fluctuation trend; S202: Based on the node load fluctuation trend, the change in node resource utilization per unit time is called, and the upper and lower bounds are compared according to the set load change interval reference value to determine the resource utilization direction, extract the positive and negative signs of the change and establish a time series to obtain the node resource direction change; S203: According to the node load fluctuation trend and the node resource direction change, call the node time series order, integrate the change amplitude and direction information, generate instantaneous resource data points for each node, draw a resource change curve based on the time series, and obtain the instantaneous resource utilization trend.

6. The edge computing scheduling optimization method based on the actor-critic model according to claim 5 is characterized in that: The specific calculation formula of the weight coefficient after the weighted node resource utilization is: ； in, Represents the weight coefficient of the node resource utilization, Represents the weighted coefficient of the node CPU utilization, Represents the weighted coefficient of the node memory usage, Represents the value of the node's current real-time CPU utilization. Represents the value of the node's current real-time memory usage. Represents the total number of nodes currently participating in the statistics.

7. The edge computing scheduling optimization method based on the actor-critic model according to claim 1 is characterized in that: The load deviation value calculation formula is specifically: ； in, Represents the node load deviation value, Represents the actual load data of the node, Represents the node basic load threshold data, Represents the average of the basic load thresholds of all nodes, Represent all The sum of the absolute values of the differences between the actual load and the benchmark load of each node.

8. The edge computing scheduling optimization method based on the actor-critic model according to claim 1 is characterized in that: The specific steps of S4 are: S401: calling the dynamic load adjustment benchmark, comparing the node instantaneous load value with the upper and lower bounds of the adjustment benchmark interval, determining the node load position, extracting the node interval mark information, integrating the node position mark data set, and generating the node instantaneous load position mark value; S402: Based on the instantaneous load position tag value of the node, filter the actions corresponding to the sensitivity response flag in the action pool, call the action delay tag to accumulate the delay times, filter the action set with the highest delay times, integrate the ranking information, and obtain the optimal set of sensitive actions; S403: Based on the preferred set of sensitive actions, the original execution priority of the action is called, the priority order is readjusted according to the ranking of the number of delays, the action priority weight is allocated, the node action priority change information is integrated, and a scheduling action screening list is obtained.

9. The edge computing scheduling optimization method based on the actor-critic model according to claim 1 is characterized in that: The specific steps of S5 are: S501: Extracting local policy parameter update batches from the returned data according to the scheduling action screening list, arranging the batch node policy parameter change values, extracting local gradient changes, integrating node change data, and generating a local gradient change vector set; S502: Based on the local gradient change vector set, extract batch change vector data, perform normalized difference operation on the local gradient change vector and the batch change vector, integrate the node normalized difference values to form a node normalized difference sequence, and obtain a normalized difference set; S503: Based on the normalized difference set, the normalized difference mean is compared with the upper and lower thresholds to determine the direction of mean deviation, adjust the corresponding node learning rate amplitude, integrate the node learning rate change information, and obtain a scheduling optimization plan.

Images