Intelligent scheduling method and system based on distributed computing resources

Abstract

The application relates to the technical field of distributed computing resource scheduling, and particularly discloses a distributed computing resource intelligent scheduling method and system, which collects historical load of edge nodes to form a time sequence queue, reconstructs the time sequence queue into a high-dimensional phase point set, calculates a load phase trace divergence coefficient as a chaos discrimination value, compares the chaos discrimination value with an ascending switching threshold value and a descending switching threshold value in a hysteresis comparison interval, and updates a chaos mode or a predictable mode flag in combination with a current mode flag; in the chaos mode, resource reservation or task driving is executed according to comparison between real-time load and high and low water level threshold values; in the predictable mode, a load estimation value is deduced through deep lightweight convolution mapping, and scheduling is executed according to the same threshold value; the application can adaptively switch a scheduling strategy according to a load chaos degree, avoids invalid prediction overhead, suppresses mode jitter, and is suitable for high-dynamic distributed environments such as edge computing.

Description

Technical Field

[0001] This invention relates to the field of distributed computing resource scheduling technology, and specifically to a method and system for intelligent scheduling of distributed computing resources. Background Technology

[0002] In edge computing and cloud collaborative inference scenarios, such as distributed systems consisting of autonomous driving roadside computing units and a central cloud, real-time scheduling of computing tasks is typically required on resource-constrained edge nodes. To rationally allocate resources and ensure service quality during load fluctuations, existing scheduling methods often employ load prediction techniques: collecting historical load data from edge nodes, using a long short-term memory network time-series prediction model to estimate the load of nodes in the next few seconds, and pre-emptively reserving resources or evicting low-priority tasks based on the estimated value to avoid resource contention during peak load periods.

[0003] When edge computing node loads are in a highly chaotic state (i.e., load timing is difficult to predict in the short term), how can we simultaneously avoid the ineffective consumption of computing resources caused by continuously running deep learning prediction models and their online retraining, and the disruption of scheduling continuity caused by frequent mode jitter due to single threshold switching? By introducing a load trajectory divergence coefficient to quantify the degree of chaos, and combining it with a hysteresis comparison interval to achieve a smooth transition between prediction mode and threshold mode, the system can proactively abandon high-overhead deep predictions and fall back to lightweight threshold scheduling during chaotic periods, while maintaining stability at the mode switching boundary. This solves the dual problems of existing methods becoming "inefficient with more predictions" and "instable with more switching" in dynamic extreme environments. Summary of the Invention

[0004] The purpose of this invention is to provide a method and system for intelligent scheduling of distributed computing resources to solve the problems mentioned above.

[0005] The objective of this invention can be achieved through the following technical solutions: The intelligent scheduling method based on distributed computing resources includes the following steps: S1: Collect a fixed number of recent historical load sampling points on the edge computing nodes to form a load time sequence queue; S2: For the load time-series queue, the mutual information method is used to calculate the delay parameter, and the false nearest neighbor method is used to calculate the embedding dimension. Based on the delay parameter and the embedding dimension, the load time-series queue is reconstructed into a set of phase points in a high-dimensional phase space. S3: In high-dimensional phase space, by tracking the distance divergence rate between adjacent phase points, the load phase trace divergence coefficient is estimated, and the load phase trace divergence coefficient is used as a chaos discrimination value to characterize the current degree of load chaos. S4: Compare the chaos discrimination value with the preset chaos threshold, and generate chaos mode flags and predictable mode flags based on the comparison results; S5: When a chaotic mode flag is generated, resource reservation or task eviction is determined based on the comparison results between the real-time load sampling points and the preset high-water level threshold and low-water level threshold. When a predictable mode flag is generated, the most recent consecutive load sampling points in the load time sequence queue are used as input, and the load prediction value at future time is deduced through the preset deep lightweight convolution mapping function. Resource reservation or task eviction is then performed based on the load prediction value.

[0006] As a further aspect of the present invention: the reconstructing of the load time-series queue into a set of phase points in a high-dimensional phase space based on delay parameters and embedding dimensions specifically includes: Extract a sequence of continuous load sampling points from the load time sequence queue, calculate the mutual information value at each candidate delay time, and select the delay parameter corresponding to the first drop of the mutual information value to a local minimum. Starting from the initial value of the embedding dimension, the dimension is increased successively. The proportion of false nearest neighbors in each dimension is calculated, and the embedding dimension corresponding to the first time the proportion is lower than the preset threshold is selected. Based on the delay parameter and the embedding dimension, each load sampling point is combined with its subsequent sampling points at the time interval delay parameter to form a high-dimensional phase point, and all high-dimensional phase points constitute a phase point set.

[0007] As a further aspect of the present invention: the calculation of the mutual information value at each candidate delay time specifically includes: The continuous load sampling point sequence is arranged in ascending order of value. A fixed number of value intervals are divided using the equal frequency binning method. The probability of the sampling point in each value interval is then counted. For each candidate delay time, each sampling point in the original sequence and its subsequent sampling point after the candidate delay time are combined to form a sampling point pair. The sampling point pair is mapped to a two-dimensional interval grid according to the numerical interval to which each sampling point belongs, and the joint occurrence probability in each grid is calculated. Based on the occurrence probability of each numerical interval and the joint occurrence probability of each two-dimensional interval grid, the mutual information value under the candidate delay time is calculated by summation.

[0008] As a further aspect of the present invention: the estimation of the load phase trace divergence coefficient in a high-dimensional phase space by tracking the distance divergence rate between adjacent phase points specifically includes: For each high-dimensional phase point in the set of phase points, find the nearest neighbor phase point that does not overlap in time, and record the Euclidean distance of each pair of adjacent phase points at the initial time. In subsequent consecutive time steps, the evolution sequence of the Euclidean distance between each pair of adjacent phase points is tracked with the step size, and the natural logarithm of each evolution sequence is taken and then fitted with a straight line with the corresponding step size. The slopes of all fitted lines are arithmetically averaged, and this arithmetic mean is used as the load phase trace divergence coefficient.

[0009] As a further aspect of the present invention: the step of taking the natural logarithm of each evolutionary sequence and then fitting it with a straight line to the corresponding step size specifically includes: Remove mutation distance values ​​caused by the re-pairing of adjacent phase points in each evolutionary sequence, and retain continuous step size and log distance data pairs; For all retained data pairs, the median fitting method is used to search for the slope of the line that minimizes the sum of the absolute residuals; The slope obtained from the search is used as the slope of the fitted line of the evolution sequence.

[0010] As a further aspect of the present invention: the comparison of the chaos discrimination value with a preset chaos threshold specifically includes: Define a hysteresis comparison interval, which includes an upward switching threshold and a downward switching threshold, wherein the upward switching threshold is greater than the downward switching threshold. When the current mode flag is non-chaotic mode and the chaos discrimination value exceeds the rising switching threshold, the mode flag will be updated to the chaotic mode flag. When the current mode flag is chaotic and the chaos discrimination value is lower than the descent switching threshold, the mode flag will be updated to a predictable mode flag.

[0011] As a further aspect of the present invention: the decision to perform resource reservation or task eviction specifically includes: The real-time load sampling point is compared with the high water level threshold. If the real-time load sampling point is higher than the high water level threshold, the task with the latest arrival time and the shortest resource occupation time is selected from all currently executing tasks as the task to be evicted. The task is immediately terminated and the resources it occupies are released. If the real-time load sampling point is lower than the low water level threshold, select the highest priority task from the waiting queue whose resource requirement is less than the current available resource amount, allocate resources to it, and start its execution. If the real-time load sampling point is between the low water level threshold and the high water level threshold, the current resource allocation status remains unchanged.

[0012] As a further aspect of the present invention: the step of performing resource reservation or task eviction based on load estimates specifically includes: Arrange the most recent consecutive load sampling points in chronological order into a one-dimensional input sequence. Perform three depthwise separable convolution operations on the one-dimensional input sequence in sequence. After each convolution, a threshold linear unit activation is performed to obtain the output feature sequence. Perform a fully connected mapping on the last element of the output feature sequence to obtain a scalar value as the load estimate for future time steps; The estimated load is compared with the high-water level threshold and the low-water level threshold, and the corresponding operation is performed using the same resource reservation or task eviction decision rules as under the chaos mode flag.

[0013] A distributed computing resource intelligent scheduling system includes: Historical load acquisition module: Collects a fixed number of recent historical load sampling points on edge computing nodes to form a load time sequence queue; Phase space reconstruction module: For the load time sequence queue, the mutual information method is used to calculate the delay parameter, and the false nearest neighbor method is used to calculate the embedding dimension. Based on the delay parameter and the embedding dimension, the load time sequence queue is reconstructed into a set of phase points in a high-dimensional phase space. Chaos discrimination calculation module: In high-dimensional phase space, by tracking the distance divergence rate between adjacent phase points, the load phase trace divergence coefficient is estimated, and the load phase trace divergence coefficient is used as a chaos discrimination value to characterize the current degree of load chaos. Pattern flag generation module: compares the chaos discrimination value with the preset chaos threshold, and generates chaos pattern flags and predictable pattern flags based on the comparison results; Resource scheduling execution module: When a chaotic mode flag is generated, the module determines whether to reserve resources or evict tasks based on the comparison results between the real-time load sampling points and the preset high-water level threshold and low-water level threshold. When a predictable mode flag is generated, the module takes the most recent consecutive load sampling points in the load time sequence queue as input, and uses a preset deep lightweight convolution mapping function to deduce the load estimate for future time moments, and then performs resource reservation or task eviction based on the load estimate.

[0014] The beneficial effects of this invention are: (1) By calculating the load trajectory divergence coefficient and setting the hysteresis comparison interval, the scheduling mode based on instantaneous thresholds can be automatically switched when the load is highly chaotic and cannot be predicted in the short term, thus avoiding the continuous operation of deep convolutional prediction networks and their potential retraining process in unpredictable scenarios. Since the scheduling in chaotic mode only relies on the simple comparison between real-time sampling points and two thresholds, it no longer occupies the originally tight central processing unit and memory resources on edge nodes for meaningless load inference, thus effectively reducing the additional computational burden caused by invalid predictions, enabling edge nodes to concentrate their limited computing power on the execution of actual tasks; (2) A hysteresis comparison interval is constructed by using an ascending switching threshold and a descending switching threshold. This ensures that switching the mode flag from a predictable mode to a chaotic mode requires the chaotic discrimination value to exceed the higher ascending threshold, while switching back from a chaotic mode to a predictable mode requires the chaotic discrimination value to be lower than the lower descending threshold. This design maintains the current mode flag unchanged when the chaotic discrimination value fluctuates around the threshold, avoiding repeated mode flag jumps caused by instantaneous load disturbances. Consequently, resource reservation or task eviction policies will not frequently switch between two drastically different scheduling rules, thereby reducing erroneous task evictions or invalid resource reservations caused by mode switching and improving the continuity and stability of the scheduling policy at the execution level. Attached Figure Description

[0015] Figure 1 This is a flowchart of the method of the present invention; Figure 2 This is a system block diagram of the present invention. Detailed Implementation

[0016] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0017] Please see Figure 1 As shown, this invention is an intelligent scheduling method based on distributed computing resources, comprising the following steps: S1: Collect a fixed number of recent historical load sampling points on the edge computing nodes to form a load time sequence queue; S2: For the load time-series queue, the mutual information method is used to calculate the delay parameter, and the false nearest neighbor method is used to calculate the embedding dimension. Based on the delay parameter and the embedding dimension, the load time-series queue is reconstructed into a set of phase points in a high-dimensional phase space. S3: In high-dimensional phase space, by tracking the distance divergence rate between adjacent phase points, the load phase trace divergence coefficient is estimated, and the load phase trace divergence coefficient is used as a chaos discrimination value to characterize the current degree of load chaos. S4: Compare the chaos discrimination value with the preset chaos threshold, and generate chaos mode flags and predictable mode flags based on the comparison results; S5: When a chaotic mode flag is generated, resource reservation or task eviction is determined based on the comparison results between the real-time load sampling points and the preset high-water level threshold and low-water level threshold. When a predictable mode flag is generated, the most recent consecutive load sampling points in the load time sequence queue are used as input, and the load prediction value at future time is deduced through the preset deep lightweight convolution mapping function. Resource reservation or task eviction is then performed based on the load prediction value.

[0018] In S1, a fixed number of recent historical load sampling points are collected from the edge computing nodes to form a load time-series queue, specifically including: Deploy a persistent monitoring process on the edge computing node. This process periodically collects the load data of the edge computing node by calling the performance counter interface provided by the operating system kernel and the pseudo file interface in the file system that records resource usage.

[0019] Specifically, a fixed sampling period of 100 milliseconds is set. At the end of each sampling period, the monitoring process reads the average utilization percentage of the central processing unit in the last 100 milliseconds, the percentage of the ratio of the currently occupied bytes of physical memory to the total number of bytes, the percentage of network bandwidth utilization converted from the total number of bytes received and sent by the network interface in the last 100 milliseconds, and the number of tasks waiting to be executed in the task ready queue on the edge computing node.

[0020] The four percentage values ​​mentioned above are weighted and summed with the number of tasks to obtain a comprehensive load value, which is then used as a historical load sampling point. The monitoring process maintains a fixed-length circular queue in memory, with a length of 512. Whenever a new historical load sampling point is generated, the monitoring process writes the sampling point to the current write pointer position of the circular queue, then moves the write pointer forward one position. If the write pointer reaches the tail of the queue, it automatically wraps back to the head of the queue, thus overwriting the oldest sampling point. After 512 consecutive sampling periods, the circular queue is full, and the 512 historical load sampling points stored therein are arranged in chronological order from oldest to newest, forming a load time-series queue. Thereafter, the monitoring process continues to run, updating the load time-series queue every time a new sampling point is collected, always ensuring that the queue stores historical load sampling points from the most recent 512 sampling periods.

[0021] In S2, for the load time-series queue, the mutual information method is used to calculate the delay parameter, and the spurious nearest neighbor method is used to calculate the embedding dimension. Based on the delay parameter and the embedding dimension, the load time-series queue is reconstructed into a set of phase points in a high-dimensional phase space, specifically including: Extract all historical load sampling points from the load timing queue obtained in step S1 and arrange them in chronological order to form a continuous load sampling point sequence. To calculate the delay parameter, it is necessary to traverse a series of candidate delay times, each candidate delay time taking the value of any positive integer between 1 and half the length of the load timing queue. For each candidate delay time, calculate the mutual information value under that candidate delay time as follows: rearrange all sampling points in the continuous load sampling point sequence in ascending order of value, and use the equal-frequency binning method to divide the sorted sequence into a fixed number of value intervals, which is 16, with each value interval containing an equal number of sampling points. Count the number of times the sampling points in the original sequence appear in each value interval, and divide by the total number of sampling points to obtain the probability of occurrence of each value interval. For the current candidate delay time, pair each sampling point in the continuous load sampling point sequence with its successor sampling point that is delayed by the candidate delay time in time, forming a sampling point pair, ignoring those pairs where the successor sampling point does not exist. Based on the numerical intervals to which each sampling point and its successor sampling points belong, each sampling point pair is mapped to a two-dimensional interval grid. The rows of this grid correspond to the numerical interval numbers of the sampling points, and the columns correspond to the numerical interval numbers of the successor sampling points, resulting in a total of 16 x 16, or 256 grids. The frequency of each sampling point pair within each grid is counted, and this count is divided by the total number of sampling point pairs to obtain the joint occurrence probability of each two-dimensional interval grid. For each numerical interval, the occurrence probability of that interval is multiplied by the logarithm of that probability (base 2) to obtain the self-information contribution value of that interval. The self-information contribution values ​​of all numerical intervals are then summed and their negative values ​​are taken to obtain the edge entropy of the load time sequence queue. For each two-dimensional interval grid, the joint occurrence probability of that grid is multiplied by the logarithm of that joint occurrence probability (base 2) to obtain the joint information contribution value of that grid. The joint information contribution values ​​of all grids are then summed and their negative values ​​are taken to obtain the joint entropy of the load time sequence queue. Subtracting the edge entropy from this joint entropy yields the mutual information value at the current candidate delay time. After calculating the mutual information value corresponding to all candidate delay times in sequence, check them one by one starting from the delay parameter equal to 1. When it is found that the mutual information value corresponding to a certain delay time is smaller than the mutual information value corresponding to the next delay time, and the mutual information value decreases relative to the mutual information value corresponding to the previous delay time, take the delay time as the delay parameter corresponding to the first decrease to the local minimum value, and denote it as the delay parameter.

[0022] Calculate the embedding dimension: Starting with an initial value of 1, increment by 1 for each subsequent embedding dimension attempt. For each currently attempted embedding dimension, based on the determined delay parameter and the embedding dimension, combine each sample point in the continuous load sampling point sequence with its subsequent sample points that are time-intervald by 1x the delay parameter, 2x the delay parameter, and so on, up to the embedding dimension minus 1x the delay parameter, to form a high-dimensional phase point. For each high-dimensional phase point, find the phase point with the closest Euclidean distance among its subsequent phase points and calculate this nearest distance. Simultaneously, treat the last component of the current high-dimensional phase point (i.e., the sample point with the largest offset in the original sequence) as a point in an incremental dimension and calculate the distance between adjacent phase points in this incremental dimension. If the ratio of this nearest distance to the distance between adjacent phase points in the incremental dimension is greater than a preset threshold (set to 10), then the current phase point is determined to have a false nearest neighbor. Calculate the proportion of false nearest neighbors among all high-dimensional phase points. When this proportion first falls below the preset threshold, the currently attempted embedding dimension is taken as the final embedding dimension. The preset threshold is set to 5%.

[0023] Based on the calculated delay parameters and embedding dimension, the load time series queue is reconstructed into a set of phase points in a high-dimensional phase space. Specifically, starting from the first sampling point in the load time series queue, each sampling point is sequentially taken as a starting point. This starting point is then combined with subsequent sampling points spaced at intervals of 1x, 2x, and up to the embedding dimension minus 1x the delay parameter to form a high-dimensional phase point. All such high-dimensional phase points are arranged in chronological order, forming a set of phase points in the high-dimensional phase space. Each phase point in this set represents a state of the load time series in the phase space, used for subsequent chaos discrimination calculations.

[0024] In S3, in the high-dimensional phase space, the load phase trace divergence coefficient is estimated by tracking the distance divergence rate between adjacent phase points. This load phase trace divergence coefficient is then used as a chaos criterion characterizing the current degree of load chaos, specifically including: The set of phase points obtained in step S2 contains a total of M high-dimensional phase points, which are sequentially labeled as the 1st phase point, the 2nd phase point, and so on, up to the Mth phase point, in chronological order. For each high-dimensional phase point, the nearest neighbor phase point whose time does not coincide with its own is found. Specifically, for the Mth phase point... The nth phase point, from the nth phase point Within the range from the first phase point to the Mth phase point, calculate the Euclidean distance between that phase point and every subsequent phase point. The Euclidean distance is calculated as follows: subtract the component values ​​of each phase point in each dimension, square the result, sum the squared values ​​of all dimensions, and then take the square root of the sum. Among all calculated Euclidean distances, select the subsequent phase point corresponding to the minimum value as the Mth phase point. The nearest neighbor of the i-th phase point. Record the i-th phase point. The Euclidean distance between the nth phase point and its nearest neighbor at the initial time, which is the nth phase point. The time position of each phase point is recorded as the initial distance.

[0025] Track the evolution sequence of the Euclidean distance between each pair of adjacent phase points with the step size. For the first... A pair of phase points, consisting of a phase point and its nearest neighbor, is defined with a maximum tracking step size of 50. In subsequent steps 1, 2, and up to the maximum tracking step size, the Euclidean distance between the new phase points after each phase point evolves forward by the corresponding step size is calculated. Specifically, in the ... step( (Values ​​range from 1 to 50), the first Each phase point moves along the time direction in phase space. The nth phase point interval is obtained to get the nth phase point interval. Each phase point; simultaneously move its nearest neighbor phase point as well. Given a phase point interval of n, obtain the corresponding phase points; calculate the Euclidean distance between these two new phase points, denoted as the nth phase point. The step distance. Thus, for each pair of initial adjacent phase points, an evolutionary sequence consisting of 50 distance values ​​is obtained, with each distance value corresponding to a step size.

[0026] A linear fit is required for each evolutionary sequence. The goal of the linear fit is to find a straight line that describes the relationship between the step size and the natural logarithm of the distance. Let the step size be the independent variable and the natural logarithm of the distance be the dependent variable; the slope of the fitted line represents the divergence rate of the evolutionary sequence. The specific fitting process is as follows: First, remove the mutation distance values ​​in each evolutionary sequence caused by the re-pairing of adjacent phase points. The mutation distance value refers to the distance between the first and second phase points when tracing to a certain step. When a phase point moves and its nearest neighbor is no longer its nearest neighbor, the distance value will change abruptly. To identify these abrupt changes in distance values, the ratio of the distance between two adjacent steps is calculated. If the ratio is greater than 5 or less than 0.2, the distance value corresponding to that step is considered an abrupt change and is removed from the evolutionary sequence. The remaining step size and the corresponding natural logarithm of the distance are retained to form continuous valid data pairs.

[0027] For all retained valid data pairs, the slope of the straight line is calculated using the median fitting method. The goal of the median fitting method is to find a straight line that minimizes the sum of the absolute residuals of all data points. Let there be N valid data pairs, where the th pair is... The step size of each data pair is The natural logarithm of the distance is The equation of the line to be fitted is: ,in The slope The intercept is given. The sum of absolute residuals is defined as: the actual values ​​of all data points. with fitted value The sum of the absolute values ​​of the differences. The slope is determined through a search method. The algorithm iterates through the possible slopes with a step size of 0.001, where the lower bound is -1 and the upper bound is 1. For each candidate slope... Calculate the intercept that minimizes the sum of the absolute residuals. ,intercept The value is for all data points. The median. Then calculate the candidate slope. The sum of the corresponding absolute residuals. After the traversal is complete, select the slope that minimizes the sum of the absolute residuals. The slope of the fitted straight line for this evolutionary sequence.

[0028] After fitting all phase pairs, a set of slopes is obtained, with each slope corresponding to a pair of initial adjacent phase points. These slopes are summed and then divided by the total number of slopes to obtain the arithmetic mean, which is used as the load phase trace divergence coefficient. This coefficient is a real value representing the degree of divergence of the current load time sequence trajectory in phase space. A large load phase trace divergence coefficient indicates that the distance between adjacent phase points increases exponentially with time, and the load exhibits chaotic characteristics; a coefficient close to zero or negative indicates that the load is predictable.

[0029] The above calculation involves two mathematical formulas. The first is the formula for calculating the Euclidean distance, which is used for two embedding dimensions. The phase points, and their component sequences are respectively and The Euclidean distance d is calculated as follows: ;in, and These represent the two phase points at the th phase. The values ​​in each dimension, and the subscript of the summation symbol. Starting from 1, we take the embedding dimension m, sum the squares of the differences between each dimension, and then take the square root. The second objective is to minimize the sum of the absolute residuals during the fitting process, expressed mathematically as: in, Indicates candidate slope The sum of the corresponding absolute residuals, For the first The natural logarithm of the distance between each data point For the corresponding step size, This indicates the median operation. This represents the total number of valid data pairs. The search enables... smallest The value represents the slope of the fitted line. The meanings of all parameters in this formula have been defined in the above text.

[0030] In S4, the chaos discrimination value is compared with a preset chaos threshold. Based on the comparison result, a chaos mode flag and a predictable mode flag are generated, specifically including: A hysteresis comparison interval is set for comparing chaotic discrimination values. This hysteresis comparison interval includes two thresholds: an rising switching threshold and a falling switching threshold, where the value of the rising switching threshold is greater than the value of the falling switching threshold. Specifically, these two thresholds are pre-determined based on the load characteristics of the application scenario where the edge computing node is located. In a typical implementation, when the load trace divergence coefficient calculated in step S3 is between 0 and 0.1, the rising switching threshold is set to 0.05, and the falling switching threshold is set to 0.02. The difference between the rising and falling switching thresholds is the hysteresis width, which is 0.03 in this case. The purpose of this hysteresis width is to prevent the chaotic discrimination value from fluctuating frequently around a single threshold, causing the mode flag to switch repeatedly.

[0031] Initialize a mode flag variable to record the current scheduling mode. When the edge computing node starts, the initial value of the mode flag is set to the predictable mode flag. This mode flag variable is stored in the node's memory and can be read and updated by subsequent steps.

[0032] Within each scheduling cycle, the chaos discrimination value obtained in step S3 is compared with the two thresholds mentioned above, and a decision is made on whether to update the mode flag based on the current state of the mode flag. The specific judgment logic is divided into the following three cases.

[0033] Scenario 1: When the current mode flag is a non-chaotic mode (i.e., a predictable mode flag), check if the chaos discrimination value exceeds the rising switching threshold. If the chaos discrimination value is greater than the rising switching threshold, it indicates that the chaos level of the load has risen to an unacceptable level, and the mode flag is updated to a chaotic mode flag. If the chaos discrimination value does not exceed the rising switching threshold, the current predictable mode flag remains unchanged.

[0034] The second scenario: When the current mode flag is a chaotic mode flag, check if the chaos discrimination value is lower than the descent switching threshold. If the chaos discrimination value is lower than the descent switching threshold, it indicates that the chaos level of the load has been reduced to a predictable range, and the mode flag is updated to a predictable mode flag. If the chaos discrimination value is not lower than the descent switching threshold, the current chaotic mode flag remains unchanged.

[0035] The third scenario: When the chaos discrimination value is between the decreasing switching threshold and the increasing switching threshold, the current mode flag remains unchanged regardless of its current state, and no update operation is performed.

[0036] Through the aforementioned hysteresis comparison logic, the mode flag only changes when the chaos discrimination value clearly crosses a relatively wide switching boundary, thus avoiding frequent flag jumps caused by instantaneous load fluctuations. The updated mode flag will be used as input to step S5 to determine which resource scheduling strategy to adopt subsequently.

[0037] In S5, when a chaotic mode flag is generated, resource reservation or task eviction is determined based on the comparison between real-time load sampling points and preset high-water level and low-water level thresholds. When a predictable mode flag is generated, the most recent consecutive load sampling points in the load time-series queue are used as input. A preset depthwise lightweight convolutional mapping function is used to deduce the load estimate for future time moments. Resource reservation or task eviction is then performed based on the load estimate, specifically including: Two thresholds are pre-defined for comparison: a high-water level threshold and a low-water level threshold, where the high-water level threshold is greater than the low-water level threshold. These two thresholds are determined based on the resource capacity and performance targets of the edge computing nodes. In a typical implementation, the theoretical maximum value of the overall load is 100, with the high-water level threshold set to 85 and the low-water level threshold set to 40. A real-time load sampling point higher than 85 indicates resource overload, lower than 40 indicates resource idleness, and values ​​in between indicate normal load.

[0038] When the mode flag output in step S4 is a chaotic mode flag, a threshold comparison scheduling method based on real-time load sampling points is adopted. Specifically, in each scheduling cycle, the latest real-time load sampling point is obtained, and the value of this real-time load sampling point is compared with the high-water mark threshold and the low-water mark threshold sequentially. If the value of the real-time load sampling point is higher than the high-water mark threshold, it indicates that the current edge computing node is in an overloaded state and resources need to be released. At this time, from all the tasks currently being executed on the node, a task that meets the following two conditions is searched: it arrived at the node the latest, that is, it was recently assigned to the node; and the resource duration occupied from the start of execution to the current moment (i.e., the task execution time) is the shortest among all the tasks currently being executed. The found task is identified as a task to be evicted, its execution process is immediately terminated, and all resources such as CPU time slice and memory space occupied by it are released. If the value of the real-time load sampling point is lower than the low-water mark threshold, it indicates that the current edge computing node is in a light-load state and can accept new tasks. At this point, from the waiting queue (i.e., the set of tasks that have not yet been assigned to any node and are in a queue), all tasks whose resource requirements are less than the available resources of the current node are selected. Then, the task with the highest priority (the smaller the priority value, the higher the priority) is selected from these tasks, the required resources are allocated to this task, and execution is started. If the value of the real-time load sampling point is between the low watermark threshold and the high watermark threshold (including the case equal to the threshold), the running state of all current tasks remains unchanged, and neither eviction nor the start of a new task is performed.

[0039] When the pattern flag output in step S4 is a predictable pattern flag, a load prediction method based on a depthwise lightweight convolution mapping function is adopted. First, several recent consecutive historical load sampling points are extracted from the load time-series queue maintained in step S1, with the number of consecutive sampling points fixed at 32. These 32 sampling points are arranged into a one-dimensional input sequence in chronological order. Then, three depthwise separable convolution operations are performed sequentially on this one-dimensional input sequence. Each depthwise separable convolution operation consists of two steps: the first step is to perform depthwise convolution independently on each position of the input sequence, that is, to convolve each input channel separately, with a convolution kernel size of 3; the second step is pointwise convolution, that is, to linearly combine the outputs of each channel using a 1x1 convolution kernel. After each depthwise separable convolution operation, a threshold linear unit activation operation is performed. The threshold linear unit activation is calculated as follows: for each value of the convolution output, if the value is greater than 0, the original value is kept unchanged; if the value is less than or equal to 0, the value is multiplied by 0.01 as the output. After three convolutions and three activations, an output feature sequence is obtained, with the same length as the input sequence, consisting of 32 feature values. The last element of this output feature sequence (the 32nd feature value) is extracted, and a fully connected mapping operation is performed on this element. Specifically, the fully connected mapping is performed by multiplying the last feature value by a preset weight coefficient and adding a preset bias coefficient to obtain a scalar value, which represents the estimated load at future time steps. The weight coefficient and bias coefficient are determined through offline training and remain fixed after training. After obtaining the estimated load, this estimated load is compared with the aforementioned high-water mark and low-water mark thresholds, using the same judgment rules as under the chaotic mode flag: if the estimated load is higher than the high-water mark threshold, the task with the latest arrival time and shortest resource usage time is selected from the currently executing tasks and evicted; if the estimated load is lower than the low-water mark threshold, the task with the highest priority and resource requirement less than the current idle resource amount is selected from the waiting queue and started for execution; if the estimated load is between the low-water mark threshold and the high-water mark threshold, the current resource allocation state remains unchanged. Thus, in the predictable mode, resource reservation or task eviction is performed in advance based on the estimated future load to avoid impending overload or fully utilize idle resources.

[0040] Please see Figure 2 As shown, the intelligent scheduling system based on distributed computing resources includes: Historical load acquisition module: Collects a fixed number of recent historical load sampling points on edge computing nodes to form a load time sequence queue; Phase space reconstruction module: For the load time sequence queue, the mutual information method is used to calculate the delay parameter, and the false nearest neighbor method is used to calculate the embedding dimension. Based on the delay parameter and the embedding dimension, the load time sequence queue is reconstructed into a set of phase points in a high-dimensional phase space. Chaos discrimination calculation module: In high-dimensional phase space, by tracking the distance divergence rate between adjacent phase points, the load phase trace divergence coefficient is estimated, and the load phase trace divergence coefficient is used as a chaos discrimination value to characterize the current degree of load chaos. Pattern flag generation module: compares the chaos discrimination value with the preset chaos threshold, and generates chaos pattern flags and predictable pattern flags based on the comparison results; Resource scheduling execution module: When a chaotic mode flag is generated, the module determines whether to reserve resources or evict tasks based on the comparison results between the real-time load sampling points and the preset high-water level threshold and low-water level threshold. When a predictable mode flag is generated, the module takes the most recent consecutive load sampling points in the load time sequence queue as input, and uses a preset deep lightweight convolution mapping function to deduce the load estimate for future time moments, and then performs resource reservation or task eviction based on the load estimate.

[0041] The working principle of this invention is as follows: On edge computing nodes, a resident monitoring process periodically collects CPU utilization, memory usage, network bandwidth utilization, and task ready queue length. These are weighted and summed to obtain a comprehensive load value, which is then stored in a fixed-length circular queue to form a load time-series queue. The mutual information method is used to calculate the latency parameter of this load time-series queue, and the spurious nearest neighbor method is used to calculate the embedding dimension. Based on this, the load time-series queue is reconstructed into a set of phase points in a high-dimensional phase space. In the high-dimensional phase space, the nearest neighbor phase point with non-overlapping time is found for each phase point. The evolution sequence of the Euclidean distance between each pair of adjacent phase points with the step size is tracked. After eliminating abrupt distances, a median straight line is fitted between the step size and the natural logarithm of the distance. The arithmetic mean of the resulting slopes is used as the load phase trace divergence coefficient and the chaos discrimination value. The chaos discrimination value is compared with a preset hysteresis comparison interval. The system compares the rising and falling switching thresholds and updates the flag to either a chaotic mode flag or a predictable mode flag based on the current mode flag and the comparison result. If the flag is chaotic, the real-time load sampling point is compared with the high-water level threshold and the low-water level threshold. If the flag is higher than the high-water level threshold, the task that arrived last and occupied the least resources is evicted. If the flag is lower than the low-water level threshold, the task with the highest priority and resource requirement less than the available space is selected from the waiting queue and started for execution. If the flag is between the two, the state remains unchanged. If the flag is predictable, the most recent 32 sampling points are extracted from the load time-series queue as a one-dimensional input sequence. After passing through three depthwise separable convolutions and threshold linear unit activations, the last element of the output feature sequence is taken for fully connected mapping to obtain the load prediction value. Then, resource reservation or task eviction is performed according to the same high-water level threshold and low-water level threshold comparison rules.

[0042] The foregoing has provided a detailed description of one embodiment of the present invention, but this description is merely a preferred embodiment and should not be construed as limiting the scope of the invention. All equivalent variations and modifications made within the scope of the claims of this invention should still fall within the patent coverage of this invention.

Claims

1. A method for intelligent scheduling of distributed computing resources, characterized in that, Includes the following steps: S1: Collect a fixed number of recent historical load sampling points on the edge computing nodes to form a load time sequence queue; S2: For the load time-series queue, the mutual information method is used to calculate the delay parameter, and the false nearest neighbor method is used to calculate the embedding dimension. Based on the delay parameter and the embedding dimension, the load time-series queue is reconstructed into a set of phase points in a high-dimensional phase space. S3: In a high-dimensional phase space, the load phase trace divergence coefficient is calculated by tracking the distance divergence rate between adjacent phase points. This load phase trace divergence coefficient is then used as a chaos criterion to characterize the current degree of load chaos. The calculation of the load phase trace divergence coefficient specifically includes: For each high-dimensional phase point in the set of phase points, find the nearest neighbor phase point that does not overlap in time, and record the Euclidean distance of each pair of adjacent phase points at the initial time. In subsequent consecutive time steps, the evolution sequence of the Euclidean distance between each pair of adjacent phase points is tracked with the step size, and the natural logarithm of each evolution sequence is taken and then fitted with a straight line with the corresponding step size. The slopes of all fitted lines are arithmetically averaged, and the arithmetic mean is used as the load trace divergence coefficient. S4: Compare the chaos discrimination value with the preset chaos threshold, and generate chaos mode flags and predictable mode flags based on the comparison results; S5: When a chaotic mode flag is generated, resource reservation or task eviction is determined based on the comparison results between the real-time load sampling points and the preset high-water level threshold and low-water level threshold. When a predictable mode flag is generated, the most recent consecutive load sampling points in the load time sequence queue are used as input, and the load prediction value at future time is deduced through the preset deep lightweight convolution mapping function. Resource reservation or task eviction is then performed based on the load prediction value.

2. The intelligent scheduling method based on distributed computing resources according to claim 1, characterized in that, The process of reconstructing the load time-series queue into a set of phase points in a high-dimensional phase space based on delay parameters and embedding dimensions specifically includes: Extract a sequence of continuous load sampling points from the load time sequence queue, calculate the mutual information value at each candidate delay time, and select the delay parameter corresponding to the first drop of the mutual information value to a local minimum. Starting from the initial value of the embedding dimension, the dimension is increased successively. The proportion of false nearest neighbors in each dimension is calculated, and the embedding dimension corresponding to the first time the proportion is lower than the preset threshold is selected. Based on the delay parameter and the embedding dimension, each load sampling point is combined with its subsequent sampling points at the time interval delay parameter to form a high-dimensional phase point, and all high-dimensional phase points constitute a phase point set.

3. The intelligent scheduling method based on distributed computing resources according to claim 2, characterized in that, The calculation of the mutual information value for each candidate delay time specifically includes: The continuous load sampling point sequence is arranged in ascending order of value. A fixed number of value intervals are divided using the equal frequency binning method. The probability of the sampling point in each value interval is then counted. For each candidate delay time, each sampling point in the original sequence and its subsequent sampling point after the candidate delay time are combined to form a sampling point pair. The sampling point pair is mapped to a two-dimensional interval grid according to the numerical interval to which each sampling point belongs, and the joint occurrence probability in each grid is calculated. Based on the occurrence probability of each numerical interval and the joint occurrence probability of each two-dimensional interval grid, the mutual information value under the candidate delay time is calculated by summation.

4. The intelligent scheduling method based on distributed computing resources according to claim 1, characterized in that, The step of taking the natural logarithm of each evolutionary sequence and fitting it with a straight line to the corresponding step size specifically includes: Remove mutation distance values ​​caused by the re-pairing of adjacent phase points in each evolutionary sequence, and retain continuous step size and log distance data pairs; For all retained data pairs, the median fitting method is used to search for the slope of the line that minimizes the sum of the absolute residuals; The slope obtained from the search is used as the slope of the fitted line of the evolution sequence.

5. The intelligent scheduling method based on distributed computing resources according to claim 1, characterized in that, The comparison of the chaos discrimination value with the preset chaos threshold specifically includes: Define a hysteresis comparison interval, which includes an upward switching threshold and a downward switching threshold, wherein the upward switching threshold is greater than the downward switching threshold. When the current mode flag is non-chaotic mode and the chaos discrimination value exceeds the rising switching threshold, the mode flag will be updated to the chaotic mode flag. When the current mode flag is chaotic and the chaos discrimination value is lower than the descent switching threshold, the mode flag will be updated to a predictable mode flag.

6. The intelligent scheduling method based on distributed computing resources according to claim 1, characterized in that, The decision to implement resource reservation or task eviction specifically includes: The real-time load sampling point is compared with the high water level threshold. If the real-time load sampling point is higher than the high water level threshold, the task with the latest arrival time and the shortest resource occupation time is selected from all currently executing tasks as the task to be evicted. The task is immediately terminated and the resources it occupies are released. If the real-time load sampling point is lower than the low water level threshold, select the highest priority task from the waiting queue whose resource requirement is less than the current available resource amount, allocate resources to it, and start its execution. If the real-time load sampling point is between the low water level threshold and the high water level threshold, the current resource allocation status remains unchanged.

7. The intelligent scheduling method based on distributed computing resources according to claim 1, characterized in that, The process of reserving resources or evicting tasks based on load estimates specifically includes: Arrange the most recent consecutive load sampling points in chronological order into a one-dimensional input sequence. Perform three depthwise separable convolution operations on the one-dimensional input sequence in sequence. After each convolution, a threshold linear unit activation is performed to obtain the output feature sequence. Perform a fully connected mapping on the last element of the output feature sequence to obtain a scalar value as the load estimate for future time steps; The estimated load is compared with the high-water level threshold and the low-water level threshold, and the corresponding operation is performed using the same resource reservation or task eviction decision rules as under the chaos mode flag.

8. A distributed computing resource intelligent scheduling system, characterized in that, The method for executing the intelligent scheduling method based on distributed computing resources according to any one of claims 1-7 includes: Historical load acquisition module: Collects a fixed number of recent historical load sampling points on edge computing nodes to form a load time sequence queue; Phase space reconstruction module: For the load time sequence queue, the mutual information method is used to calculate the delay parameter, and the false nearest neighbor method is used to calculate the embedding dimension. Based on the delay parameter and the embedding dimension, the load time sequence queue is reconstructed into a set of phase points in a high-dimensional phase space. Chaos discrimination calculation module: In high-dimensional phase space, by tracking the distance divergence rate between adjacent phase points, the load phase trace divergence coefficient is calculated. This load phase trace divergence coefficient is used as a chaos discrimination value characterizing the current degree of load chaos. The calculation of the load phase trace divergence coefficient specifically includes: For each high-dimensional phase point in the set of phase points, find the nearest neighbor phase point that does not overlap in time, and record the Euclidean distance of each pair of adjacent phase points at the initial time. In subsequent consecutive time steps, the evolution sequence of the Euclidean distance between each pair of adjacent phase points is tracked with the step size, and the natural logarithm of each evolution sequence is taken and then fitted with a straight line with the corresponding step size. The slopes of all fitted lines are arithmetically averaged, and the arithmetic mean is used as the load trace divergence coefficient. Pattern flag generation module: compares the chaos discrimination value with the preset chaos threshold, and generates chaos pattern flags and predictable pattern flags based on the comparison results; Resource scheduling execution module: When a chaotic mode flag is generated, the module determines whether to reserve resources or evict tasks based on the comparison results between the real-time load sampling points and the preset high-water level threshold and low-water level threshold. When a predictable mode flag is generated, the module takes the most recent consecutive load sampling points in the load time sequence queue as input, and uses a preset deep lightweight convolution mapping function to deduce the load estimate for future time moments, and then performs resource reservation or task eviction based on the load estimate.

Images