Dynamic scheduling system and method for container calculation

Abstract

The invention relates to the technical field of task scheduling, in particular to a container computing-oriented dynamic scheduling system and method, and the system comprises a resource state modeling module which obtains CPU, memory and network bandwidth parameters and analyzes multi-node resource characteristics based on resource use data of nodes in a container cluster. According to the invention, through dynamic resource modeling, analysis of probability change of multi-node resources and accurate prediction of resource fluctuation, a scheduling system can rapidly deal with dynamic resource adjustment, real-time resource comparison is combined, refined load optimization and reallocation are executed, global resource balance is enhanced, resource waste and node overload conditions are avoided, and the scheduling efficiency is improved. In the task scheduling process, matching is carried out based on the priority of tasks and resource multi-dimensional data, scheduling intelligence and high efficiency are ensured, loads are monitored in real time, task distribution is flexibly adjusted, the loads are prevented from being concentrated in a few nodes, task migration among the nodes is optimized, the bottleneck is eliminated, and the system stability and the overall performance are remarkably improved.

Description

Technical Field

[0001] The present invention relates to the field of task scheduling technology, and in particular to a dynamic scheduling system and method for container computing. Background Art

[0002] The field of task scheduling technology mainly involves the management and allocation of computing resources, with the goal of ensuring the effective execution of computing tasks through reasonable resource scheduling and allocation strategies. Task scheduling technology is widely used in distributed systems, cloud computing, parallel computing and other scenarios. Through the dynamic allocation and optimization of computing tasks, it maximizes the utilization of resources and improves the overall efficiency and performance of the system. In the modern computing environment, the task scheduling system needs to have the ability to handle complex tasks, such as load balancing, resource contention resolution, task dependency management, etc., in order to cope with diverse application needs.

[0003] Among them, the dynamic scheduling system for container computing refers to managing the computing tasks and resource allocation in the container through a dynamic scheduling mechanism in a containerized environment. The main purpose of this system is to improve the utilization of computing resources and the overall operating efficiency of the system. It is particularly suitable for highly scalable cloud-native application scenarios. It can flexibly adjust the scheduling and resource allocation of containers according to the real-time changes of tasks to ensure the stability and efficient operation of the system.

[0004] Existing task scheduling methods are deficient in resource usage prediction and response, and it is difficult to effectively capture the dynamic changes in resource demand between nodes, resulting in unbalanced resource allocation, overloading of some nodes, and serious resource waste. The lack of real-time difference analysis and dynamic resource optimization mechanisms makes it difficult to adjust loads and redistribute tasks in a timely manner, resulting in load concentration and bottlenecks, limiting the overall performance and stability of the system. For example, in containerized computing, uneven load distribution between nodes may lead to response delays and stagnation of some services, seriously affecting computing efficiency and user experience. Summary of the invention

[0005] The purpose of the present invention is to solve the shortcomings of the prior art and to propose a dynamic scheduling system and method for container computing.

[0006] In order to achieve the above object, the present invention adopts the following technical solution: A dynamic scheduling system for container computing includes:

[0007] The resource status modeling module obtains CPU, memory, and network bandwidth parameters based on the resource usage data of nodes in the container cluster, analyzes the multi-node resource characteristics, generates multi-node probability distribution, and establishes a dynamic resource model to calculate the probability value of node resource changes;

[0008] The load deviation optimization module compares the node resources and load data based on the node resource change probability value, analyzes the differences between nodes, performs resource reallocation, corrects the global equilibrium state, and obtains the resource scheduling optimization value;

[0009] The intelligent scheduling strategy module evaluates the task matching degree based on the resource scheduling optimization value, combined with node resources, bandwidth and delay data, and sorts the task priorities, selects nodes to execute tasks, and generates intelligent scheduling allocation for tasks;

[0010] The self-matching strategy adjustment module monitors multi-node resources in real time based on the intelligent scheduling and allocation of tasks, records bandwidth and load changes, compares resource differences of the current task execution node, calculates resource matching changes between multiple nodes, and if load deviation is found, performs task migration or load adjustment between nodes to generate node adaptive load adjustment values;

[0011] The bottleneck prevention scheduling module identifies the node load bottleneck based on the node adaptive load adjustment value, obtains the bottleneck node, migrates some tasks to the idle node, reallocates the tasks based on the task and node characteristics, and generates the bottleneck node load migration value.

[0012] The dynamic resource model includes multi-node probability distribution of CPU parameters, memory parameters, and network bandwidth parameters; the node resource change probability value includes CPU change probability, memory change probability, and network bandwidth change probability; the resource scheduling optimization value includes a global balance correction value, a resource reallocation value, and a node load difference value; the task intelligent scheduling allocation includes task priority, task matching evaluation value, and execution node selection; the node adaptive load adjustment value includes task migration value, load adjustment value, and resource matching change value; the bottleneck node load migration value includes a bottleneck node identification value and a node characteristic matching value.

[0013] As a further solution of the present invention, the step of calculating the probability value of node resource change is specifically:

[0014] Based on the resource usage data of multiple nodes in the container cluster, the resource characteristic parameters of CPU, memory, and network bandwidth are extracted to build a node resource usage matrix.

[0015] Extracting resource usage frequency data of each node from the node resource usage matrix, calculating the resource usage statistical distribution of multiple nodes through the frequency data, analyzing the resource characteristics among multiple nodes by integrating CPU, memory and network bandwidth parameters, and constructing a resource characteristic probability distribution model through multivariate analysis to obtain a multi-node resource characteristic probability distribution matrix;

[0016] The probability integral calculation is performed on the data of the multi-node resource characteristic probability distribution matrix using the formula:

[0017]

[0018] Calculate and generate the probability value of node resource change;

[0019] Among them, P i represents the resource change probability value of node i, N represents the total number of nodes, and U ij represents the resource usage frequency of nodes i and j, R ij represents the resource characteristic relationship coefficient between node i and node j, M ij represents the resource interaction influence matrix between node i and node j, γ is the interaction parameter, and ∈ is a constant used to avoid the denominator being zero.

[0020] As a further solution of the present invention, the step of obtaining the resource scheduling optimization value is specifically as follows:

[0021] Based on the node resource change probability value, calling multiple node load data, analyzing the comparison between node resource usage and load data, and calculating the difference between resource allocation and load, generating a node resource difference analysis value;

[0022] Based on the node resource difference analysis value, resource characteristic parameters of multiple nodes are extracted, the intersection of resources between nodes is analyzed, and resources are reallocated according to the matching between resource differences and characteristic parameters to obtain node resource reallocation results;

[0023] Using the node resource reallocation results and combining the load data of multiple nodes, the formula is adopted:

[0024]

[0025] Calculate the global balance correction value and obtain the resource scheduling optimization value;

[0026] Among them, O represents the resource scheduling optimization value, N represents the total number of nodes, and D i represents the resource reallocation value of node i, L i represents the load data of node i, T i represents the resource characteristics of node i, V i represents the resource change of node i, α represents the weight parameter of the resource reallocation value, which is the influence degree of resource allocation, β represents the weight parameter of the square value of the node load, which is the influence degree of load change, γ represents the adjustment coefficient of the influence of node resource characteristics on the correction value, which is the difference in characteristics between nodes, and δ is a constant used to avoid the denominator being zero.

[0027] As a further solution of the present invention, the steps of obtaining the task intelligent scheduling allocation are specifically:

[0028] Based on the resource scheduling optimization value, call the node resource parameters, bandwidth and delay data, evaluate the current node resources and task requirements, and calculate the impact of node resources and bandwidth on task adaptability to generate a task matching evaluation result;

[0029] Utilizing the task matching evaluation result, based on the node resource usage and task adaptability, extracting the adaptability parameters of the node resources, calculating the adaptation priorities of multiple nodes, and sorting the tasks according to the priority values ​​to obtain the task priority sorting results;

[0030] Based on the task priority ranking results, combined with the performance characteristics of the node and the resource adaptability parameters, the formula is adopted:

[0031]

[0032] Calculate the task allocation weight, select the optimal node for task allocation, and generate intelligent task scheduling allocation;

[0033] Among them, W i represents the task allocation weight of node i, P i represents the performance index of node i, D req Represents the performance requirement of the task, which is the square root of the reciprocal of the absolute value of the difference with the node performance plus one, and is used to evaluate the adaptability of the node performance to the task requirement. i represents the current available resources of node i, B i Represents the total resource capacity of node i, which is used to calculate the utilization of node resources and indicates the ratio of the current resources of the node to the total resources.

[0034] As a further solution of the present invention, the step of obtaining the node adaptive load adjustment value is specifically:

[0035] Based on the intelligent task scheduling and allocation, the resource usage of multiple nodes is monitored in real time, the bandwidth and load data of multiple nodes are recorded, the dynamic change characteristics of the bandwidth and load of each node are extracted, and the resource differences of the nodes are calculated to obtain the multi-node resource monitoring results;

[0036] By using the multi-node resource monitoring results, the bandwidth and load characteristics of the current task execution node and other nodes are compared, the resource changes of the multiple nodes are analyzed, the resource matching degree changes between the nodes are calculated, and the resource matching degree change values ​​between the nodes are obtained;

[0037] Based on the change value of the resource matching degree between the nodes, it is determined whether the load difference between the nodes exceeds the set threshold. If it exceeds the threshold, task migration or load adjustment is performed, using the formula:

[0038]

[0039] Calculate and generate node adaptive load adjustment value;

[0040] Among them, ΔL represents the adaptive load adjustment value, R1 represents the resource utilization rate of node 1, R2 represents the resource utilization rate of node 2, σ R Represents the standard deviation of resource usage.

[0041] As a further solution of the present invention, the step of obtaining the bottleneck node load migration value is specifically as follows:

[0042] Based on the node adaptive load adjustment value, monitor the load status of all nodes, extract the load data of each node, analyze whether the load parameter exceeds the preset load threshold, identify the bottleneck node exceeding the threshold, and generate the bottleneck node identification result;

[0043] Using the bottleneck node identification result, the nodes suitable for task migration in the bottleneck nodes are screened, and the resource characteristics and task requirements between the bottleneck nodes and the idle nodes are analyzed in combination with the resource parameters of the idle nodes, the task migration target of the bottleneck node is determined, and the task migration matching result of the bottleneck node is obtained;

[0044] The bottleneck node task migration matching result is called, and based on the resource difference characteristics of the bottleneck node and the idle node, the formula is adopted:

[0045]

[0046] Calculate the load migration amount of the bottleneck node and obtain the load migration value T of the bottleneck node;

[0047] Among them, L b Represents the current load value of the bottleneck node, L f Represents the current load value of the idle node, R b Represents the total resources of the bottleneck node, R f Represents the total amount of resources of idle nodes, and K represents the adjustment coefficient obtained based on previous data, which is used to match the differences in resource requirements under differentiated environments.

[0048] A dynamic scheduling method for container computing is provided, wherein the dynamic scheduling method for container computing is performed based on the dynamic scheduling system for container computing, and comprises the following steps:

[0049] S1: Based on the resource usage data of multiple nodes in the container cluster, calculate and obtain the CPU resource usage value, memory usage, and network bandwidth demand value, generate the CPU, memory, and network bandwidth distribution probability values ​​of multiple nodes, establish a node dynamic resource model, and obtain the node resource change probability value;

[0050] S2: Based on the node resource change probability value, compare the load data of CPU resources, memory usage and network bandwidth between nodes, calculate the multi-node load and resource difference, perform resource reallocation, correct the multi-node resource load balancing state, obtain the resource reallocation balance value, analyze the node resource matching situation, and generate the resource scheduling optimization value;

[0051] S3: Based on the resource scheduling optimization value, combined with the CPU resource usage value, memory usage, and network bandwidth requirement value, a matching operation is performed on the task requirements, the task and node resource matching value is calculated, and the node with the highest matching degree is selected to execute the priority task, and the task intelligent scheduling allocation result is obtained;

[0052] S4: Based on the results of the intelligent scheduling and allocation of the tasks, monitor the changes in the node CPU resource usage, memory usage, and network bandwidth demand values, analyze the changes in resource matching between nodes, detect load deviations, perform task migration and load redistribution, obtain node adaptive load adjustment values, identify bottleneck nodes, migrate some tasks to idle nodes, and obtain bottleneck node load migration.

[0053] Compared with the prior art, the advantages and positive effects of the present invention are:

[0054] In the present invention, dynamic resource modeling is used to analyze the probability changes of multi-node resources and accurately predict resource fluctuations, so that the scheduling system can quickly respond to dynamic resource adjustments. Combined with real-time resource comparison, detailed load optimization and redistribution are performed to enhance global resource balance and avoid resource waste and node overload. During task scheduling, task priority is matched with multi-dimensional resource data to ensure the intelligence and efficiency of scheduling. By real-time monitoring of load, task distribution is flexibly adjusted to avoid load concentration on a few nodes, optimize task migration between nodes, eliminate bottlenecks, and significantly improve system stability and overall performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 is a system flow chart of the present invention;

[0056] Figure 2 A flowchart of the steps of calculating the probability value of node resource change of the present invention;

[0057] Figure 3 A flow chart of the steps for obtaining the resource scheduling optimization value of the present invention;

[0058] Figure 4 A flowchart of the steps for obtaining the intelligent scheduling and allocation of tasks of the present invention;

[0059] Figure 5 A flowchart of the steps for obtaining the node adaptive load adjustment value of the present invention;

[0060] Figure 6 This is a flow chart of the steps for obtaining the bottleneck node load migration value of the present invention. DETAILED DESCRIPTION

[0061] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0062] In the description of the present invention, it should be understood that the terms "length", "width", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside" and the like indicate positions or positional relationships based on the positions or positional relationships shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, in the description of the present invention, "multiple" means two or more, unless otherwise clearly and specifically defined.

[0063] Embodiment 1

[0064] See also Figure 1 The present invention provides a technical solution: a dynamic scheduling system for container computing includes:

[0065] The resource status modeling module obtains CPU, memory, and network bandwidth parameters based on the resource usage data of nodes in the container cluster, analyzes the multi-node resource characteristics, generates multi-node probability distribution, and establishes a dynamic resource model to calculate the probability value of node resource changes;

[0066] The load deviation optimization module compares node resources and load data based on the probability value of node resource changes, analyzes the differences between nodes, performs resource reallocation, corrects the global equilibrium state, and obtains the resource scheduling optimization value;

[0067] The intelligent scheduling strategy module evaluates the task matching degree based on the resource scheduling optimization value, combined with node resources, bandwidth and delay data, and sorts the task priorities, selects nodes to execute tasks, and generates intelligent task scheduling allocation;

[0068] The self-matching strategy adjustment module is based on intelligent task scheduling and allocation, monitors multi-node resources in real time, records bandwidth and load changes, compares resource differences of the current task execution node, calculates resource matching changes between multiple nodes, and performs task migration or load adjustment between nodes if load deviation is found, and generates node adaptive load adjustment values;

[0069] The bottleneck prevention scheduling module identifies the node load bottleneck based on the node adaptive load adjustment value, obtains the bottleneck node, migrates some tasks to the idle nodes, reallocates the tasks based on the task and node characteristics, and generates the bottleneck node load migration value.

[0070] The dynamic resource model includes the multi-node probability distribution of CPU parameters, memory parameters, and network bandwidth parameters. The node resource change probability values ​​include the CPU change probability, memory change probability, and network bandwidth change probability. The resource scheduling optimization values ​​include the global balance correction value, the resource reallocation value, and the node load difference value. The task intelligent scheduling allocation includes task priority, task matching evaluation value, and execution node selection. The node adaptive load adjustment value includes the task migration value, load adjustment value, and resource matching change value. The bottleneck node load migration value includes the bottleneck node identification value and the node characteristic matching value.

[0071] See also Figure 2 ,The specific steps for calculating the probability value of node resource change are:

[0072] Based on the resource usage data of multiple nodes in the container cluster, the resource characteristic parameters of CPU, memory, and network bandwidth are extracted to build a node resource usage matrix.

[0073] Based on the resource usage data of each node in the container cluster, through meticulous data collection and preliminary analysis, core resource characteristic parameters such as CPU, memory, and network bandwidth are extracted. These parameters are obtained in real time from the node's resource monitoring system to ensure the accuracy and real-time nature of the data. Data collection covers the performance indicators of nodes under different load and time conditions. Data processing at this stage includes data cleaning and preliminary classification to remove outliers and atypical data points. Then, the extracted data is used to construct a resource usage matrix for each node. The matrix reflects the performance of each node at different times and different resource utilization rates. The mutual influence and resource sharing between nodes are taken into account during the matrix construction process.

[0074] Extract the resource usage frequency data of each node from the node resource usage matrix, calculate the statistical distribution of resource usage of multiple nodes through the frequency data, analyze the resource characteristics among multiple nodes by integrating CPU, memory and network bandwidth parameters, and construct a resource characteristic probability distribution model through multivariate analysis to obtain the probability distribution matrix of multi-node resource characteristics;

[0075] The resource usage frequency data of each node is extracted from the generated node resource usage matrix. These data are processed through precise statistical analysis methods to calculate the statistical distribution of resource usage of each node. The calculation of statistical distribution takes into account the interdependence and influence of various resources. These influencing factors include the physical location of the node, the quality of network connection and the type of task. Through the integrated analysis of CPU, memory and network bandwidth parameters, the resource differences between multiple nodes can be identified. This analysis is not limited to the usage of a single resource, but through multivariate analysis, a characteristic probability distribution model containing multiple resource dimensions is constructed. This model can predict the resource requirements and bottlenecks that different nodes may face in the future. The final multi-node resource characteristic probability distribution matrix provides a scientific basis for resource allocation and optimization.

[0076] The probability integral calculation is performed on the data of the probability distribution matrix of multi-node resource characteristics using the formula:

[0077]

[0078] Calculate and generate the probability value of node resource change;

[0079] Among them, P i represents the resource change probability value of node i, N represents the total number of nodes, and U ij represents the resource usage frequency of nodes i and j, R ij represents the resource characteristic relationship coefficient between node i and node j, M ij represents the resource interaction influence matrix between node i and node j, γ is the interaction parameter, and ∈ is a constant used to avoid the denominator being zero.

[0080] formula:

[0081]

[0082] The benefit of the formula is that it improves the accuracy and efficiency of resource allocation by considering the interactions between nodes and their respective resource utilization.

[0083] Detailed explanation of the formula and the process of formula calculation and derivation:

[0084] Assume there are three nodes, the interaction parameter γ between nodes is set to 0.1, and the resource usage frequency U between nodes ij are 0.8, 0.5, and 0.3 respectively, and the resource characteristic relationship coefficient R between nodes ij Set to 1.5, 1.2, 0.9, the resource interaction influence matrix M ij The value is 10, 20, 30, the constant ∈ is set to 1, and the resource change probability value P of each node is calculated by substituting it into the formula i , the calculation process is as follows:

[0085]

[0086]

[0087]

[0088] P1=0.00304+0.000336+0.0000493

[0089] P1=0.0034253

[0090] The result shows that the resource change probability value of the first node under the current resource configuration and interaction environment is 0.0034253, which means that the resource status of this node is less likely to change in the short term, so corresponding resource scheduling or optimization measures can be taken to improve the overall performance and resource utilization of the node.

[0091] See also Figure 3 ,The specific steps for obtaining the resource scheduling optimization value are:

[0092] Based on the probability value of node resource change, call the multi-node load data, analyze the comparison between node resource usage and load data, calculate the difference between resource allocation and load, and generate the node resource difference analysis value;

[0093] Based on the comparative analysis of the probability value of node resource change and the load data of each node, by accurately calculating the difference between the resources and load of each node, it is first necessary to extract the relevant resource usage data and load data from the node resource management system, including CPU utilization, memory usage and network bandwidth usage. The data of each node is monitored and recorded in real time, and then the difference analysis algorithm is used to compare and analyze these data. During the analysis process, by calculating the deviation between resource utilization and load demand, it is determined which nodes’ resource usage exceeds or falls short of expectations, thereby calculating the difference between resource allocation and load, and obtaining the node resource difference analysis value. This result will directly affect the next step of resource reallocation strategy, ensure the effective use of resources between nodes, and avoid resource waste or shortage.

[0094] Based on the node resource difference analysis value, the resource characteristic parameters of multiple nodes are extracted, the intersection of resources between nodes is analyzed, and the resources are reallocated according to the matching between resource differences and characteristic parameters to obtain the node resource reallocation result;

[0095] Based on the node resource difference analysis value, the reallocation of resources between nodes is performed. First, the resource demand status of each node is evaluated according to the difference analysis value. Combined with the operating characteristics of the node, such as processing power and storage capacity, a resource optimization algorithm is used to adjust the resource allocation. In this process, the workload of each node and the prediction of future tasks are considered in detail to optimize the resource configuration. The reallocation algorithm is dynamically adjusted according to the deviation between resource utilization and load demand. The reallocation result will be updated in the system to support efficient resource management and obtain the node resource reallocation result. This result is the premise of resource scheduling optimization, ensuring the rational flow of resources in the cluster and improving the overall efficiency.

[0096] Using the node resource reallocation results and combined with the load data of multiple nodes, the formula is adopted:

[0097]

[0098] Calculate the global balance correction value and obtain the resource scheduling optimization value;

[0099] Among them, O represents the resource scheduling optimization value, N represents the total number of nodes, and D i represents the resource reallocation value of node i, L i represents the load data of node i, T i represents the resource characteristics of node i, V i represents the resource change of node i, α represents the weight parameter of the resource reallocation value, which is the influence degree of resource allocation, β represents the weight parameter of the square value of the node load, which is the influence degree of load change, γ represents the adjustment coefficient of the influence of node resource characteristics on the correction value, which is the difference in characteristics between nodes, and δ is a constant used to avoid the denominator being zero.

[0100] formula:

[0101]

[0102] The formula is beneficial because it introduces multiple parameters (α, β, γ1, T i ,V i ,δ) enhances the flexibility and adaptability of the model, and can dynamically adjust resource allocation according to the actual resource usage and load requirements of different nodes to achieve the optimal resource utilization efficiency.

[0103] Detailed explanation of the formula and the process of formula calculation and derivation:

[0104] Assume that there are 3 nodes, the resource reallocation value D1 of node 1 is 120, the load data L1 is 75, the resource characteristic T1 is 0.8, and the resource change V1 is 0.05; the weight parameters α is 0.5, β is 2, γ is 1.5, and δ is 0.01. Calculate the global balance correction value of node 1:

[0105]

[0106] The result shows that the resource scheduling optimization value of node 1 after considering its resource characteristics, load demand and resource change is 46164, which indicates that node 1 needs to give priority to resource reallocation to meet its high load demand and thus maintain the global balance of the system.

[0107] See also Figure 4 ,The specific steps for obtaining task intelligent scheduling allocation are:

[0108] Based on the resource scheduling optimization value, call the node resource parameters, bandwidth and delay data to evaluate the current node resources and task requirements, and calculate the impact of node resources and bandwidth on task adaptability to generate task matching evaluation results;

[0109] Based on the resource scheduling optimization value and detailed node data, the current status of each node is first evaluated, which involves pulling resource usage data, bandwidth and delay information from each node. These data are obtained through a real-time monitoring system to ensure the timeliness and accuracy of the data. During the evaluation process, the resource usage of each node is compared with the preset task requirements, and an algorithm is used to quantify the fitness of each node. The calculation of fitness depends on the performance indicators of the node and the resource consumption of the task requirements. This step is a prerequisite for task scheduling, because only by correctly evaluating the current status of the node can tasks and resources be effectively matched, thereby optimizing the overall resource utilization efficiency. The task matching evaluation results generated by this process will directly affect the decision of task allocation.

[0110] Using the task matching evaluation results, based on the node resource usage and task adaptability, the adaptability parameters of the node resources are extracted, the adaptation priorities of multiple nodes are calculated, and the tasks are sorted according to the priority values ​​to obtain the task priority sorting results;

[0111] According to the task matching evaluation results, the next task is to prioritize tasks and select nodes. In this process, the resource adaptability parameters of each node and the current task queue are examined in detail, and tasks are sorted through a priority algorithm, which involves complex weight allocation and resource demand comparison. The calculation of task priority not only considers the urgency of the task, but also takes into account the load capacity and previous performance of the node. Tasks with high priority will be assigned to nodes with sufficient resources to ensure that the tasks can be executed quickly and stably. This process not only improves resource utilization efficiency, but also ensures the timeliness and accuracy of task execution. The resource adaptability parameters contained in the algorithm are supported by real-time monitoring data, ensuring that the data support for decision-making is real-time and accurate.

[0112] Based on the task priority sorting results, combined with the performance characteristics of the node and the resource adaptability parameters, the formula is used:

[0113]

[0114] Calculate the task allocation weight, select the optimal node for task allocation, and generate intelligent task scheduling allocation;

[0115] Among them, W i represents the task allocation weight of node i, P i represents the performance index of node i, D req Represents the performance requirement of the task, which is the square root of the reciprocal of the absolute value of the difference with the node performance plus one, and is used to evaluate the adaptability of the node performance to the task requirement. i represents the current available resources of node i, B i Represents the total resource capacity of node i, which is used to calculate the utilization of node resources and indicates the ratio of the current resources of the node to the total resources.

[0116] formula:

[0117]

[0118] Detailed explanation of the formula and the process of formula calculation and derivation:

[0119] Consider a specific example, let the performance index P of node i be i is 500 units, while the performance requirement of the task is D req Set to 450 units. Here P i and D req The value is obtained from the real-time monitoring data of the system, reflecting the current processing capacity of the node and the processing capacity required by the task. Calculate the difference between the two and their absolute value:

[0120] |P i -D req |=|500-450|=50

[0121] Add one and take the square root of the reciprocal to get:

[0122]

[0123] The current available resource amount C of node i i Set to 200GB, total resource capacity B i These values ​​are obtained through real-time data from the resource management system, reflecting the total capacity and current idle status of node resources. Calculate resource utilization:

[0124]

[0125] Multiply these two calculation results to get the task allocation weight of node i:

[0126] W i =0.140×0.4=0.056

[0127] The result shows that the weight value of node i for the current task is 0.056, which reflects the adaptability of node i to the current task in terms of performance and resource applicability. The higher this value is, the more suitable the node is for performing the task. By comparing the W of multiple nodes i value, the most suitable node can be selected for task allocation, thereby optimizing resource usage and task execution efficiency.

[0128] See also Figure 5 , the specific steps for obtaining the node adaptive load adjustment value are:

[0129] Based on intelligent task scheduling and allocation, it monitors the resource usage of multiple nodes in real time, records the bandwidth and load data of multiple nodes, extracts the dynamic change characteristics of the bandwidth and load of each node, and calculates the resource differences of the nodes to obtain the multi-node resource monitoring results;

[0130] Based on the data allocated by intelligent task scheduling, the usage of multi-node resources is monitored in real time. The changing characteristics of bandwidth and load are key factors. This requires the system to accurately record the real-time data of each node and make dynamic comparisons. By regularly monitoring the resource utilization and bandwidth occupancy of each node, the system can detect any significant load changes or deviations in resource demand, thereby providing early warning of potential resource bottlenecks or uneven distribution. This method can effectively avoid performance degradation or system crashes caused by node overload. Through comprehensive analysis of real-time data, the system can not only record current usage, but also make intelligent predictions based on previous data trends, thereby achieving better resource allocation strategies.

[0131] Using the multi-node resource monitoring results, compare the bandwidth and load characteristics of the current task execution node with other nodes, analyze the resource changes of multiple nodes, calculate the resource matching degree changes between nodes, and obtain the resource matching degree change value between nodes;

[0132] Based on the multi-node resource monitoring results, the system will compare the resource usage of the current task execution node with other nodes. This analysis is not only based on real-time monitoring data, but also includes in-depth mining of previous data of each node. Through this method, it is possible to identify which nodes may affect the efficiency of task execution due to insufficient resources, or which nodes are at risk of overload. Furthermore, the system uses an algorithm to compare the resource matching degree between nodes and identify the best resource allocation plan to ensure that each task can run on the most suitable node, thereby optimizing the performance and resource utilization of the overall network. This strategy can significantly improve task processing speed and system response time.

[0133] Based on the change value of resource matching between nodes, determine whether the load difference between nodes exceeds the set threshold. If it exceeds the threshold, perform task migration or load adjustment using the formula:

[0134]

[0135] Calculate and generate node adaptive load adjustment value;

[0136] Among them, ΔL represents the adaptive load adjustment value, R1 represents the resource utilization rate of node 1, and R2

[0137] represents the resource utilization rate of node 2, σ R Represents the standard deviation of resource usage.

[0138] formula:

[0139]

[0140] Detailed explanation of the formula and the process of formula calculation: Parameters R1 and R2 represent the resource utilization of the two nodes. In actual scenarios, these data are continuously collected by the resource monitoring system. For example, the average CPU utilization of node 1 in a certain period of time is 65%, that is, R1 = 65%. The average CPU utilization of node 2 in the same period of time is 50%, that is, R2 = 50%.

[0141] Parameter σ R is the standard deviation of resource utilization, which is used to reflect the variation of resource utilization of all monitoring nodes. Assume that in the resource utilization data of multiple nodes, the standard deviation is calculated to be 15%.

[0142] Substitute specific values ​​into the formula for calculation. First, calculate the absolute value of the difference in resource utilization between the two nodes:

[0143] |R1-R2|=|65%-50%|=15%

[0144] Next, calculate the square root of the standard deviation:

[0145]

[0146] Finally, the adaptive load adjustment value ΔL is calculated using the above values:

[0147]

[0148] The result shows that the load difference between nodes is 3.88, which means that the load of node one is 3.88 standard deviation units higher than that of node two, indicating a significant imbalance and suggesting that task migration or load adjustment is needed to achieve a more balanced resource usage state.

[0149] See also Figure 6 ,The specific steps for obtaining the bottleneck node load migration value are:

[0150] Based on the node adaptive load adjustment value, monitor the load status of all nodes, extract the load data of each node, analyze whether the load parameters exceed the preset load threshold, identify the bottleneck nodes that exceed the threshold, and generate the bottleneck node identification results;

[0151] Monitor the real-time load status of all nodes. For each node, use the real-time data acquisition system to collect load data. Through the threshold analysis algorithm, analyze the load data of each node, identify overloaded bottleneck nodes, call the preset load balancing strategy, analyze and identify the nodes that need to be processed immediately, ensure that the system operation efficiency is not affected, and obtain a detailed list of bottleneck nodes and their load status.

[0152] Using the bottleneck node identification results, we screen out nodes in the bottleneck nodes that are suitable for task migration. We combine the resource parameters of the spare nodes to analyze the resource characteristics and task requirements between the bottleneck nodes and the spare nodes, determine the task migration target of the bottleneck nodes, and obtain the bottleneck node task migration matching results.

[0153] According to the load data of the bottleneck node, combined with the resource allocation log and real-time monitoring data in the system, the resource matching degree between tasks and nodes is analyzed, and the dynamic resource scheduling algorithm is used to optimize resource allocation to ensure that tasks can be executed in a balanced manner within the range allowed by resources. By comparing task requirements with node capabilities, it is determined which specific tasks should be migrated from bottleneck nodes to idle nodes to achieve resource optimization.

[0154] Call the bottleneck node task migration matching result, based on the resource difference characteristics of the bottleneck node and the idle node, using the formula:

[0155]

[0156] Calculate the load migration amount of the bottleneck node and obtain the load migration value T of the bottleneck node;

[0157] Among them, Lb Represents the current load value of the bottleneck node, L f Represents the current load value of the idle node, R b Represents the total resources of the bottleneck node, R f Represents the total amount of resources of idle nodes, and K represents the adjustment coefficient obtained based on previous data, which is used to match the differences in resource requirements under differentiated environments.

[0158] formula:

[0159]

[0160] Detailed explanation of the formula and the process of formula calculation: The formula is used to calculate the load migration amount from the bottleneck node to the idle node. Parameter L b and L f Represent the current load values ​​of bottleneck nodes and idle nodes respectively. These data are obtained through the real-time monitoring system of the nodes. b and R f It is the total amount of resources of bottleneck nodes and idle nodes, including CPU, memory, etc., which is read through the system resource management tool. K is the adjustment coefficient obtained through data analysis based on historical load and resource usage data, which is used to balance the differences in resource requirements under different environments.

[0161] Set specific values ​​for example derivation: Assume that the current load of the bottleneck node is L b = 950 units, the current load of the idle node is L f = 300 units, the total bottleneck node resources are R b = 1500 units, the total amount of idle node resources is R f = 1200 units. The adjustment factor K is set to 0.8 based on past data analysis, reflecting the optimization results of load migration efficiency in history.

[0162] The steps in the calculation formula are as follows:

[0163] 1. Calculate the absolute value of the load difference:

[0164] |L b -L f |=|950-300|=650

[0165] 2. Calculate the total node resources:

[0166] R b +R f =1500+1200=2700

[0167] 3. Apply the formula to calculate the load migration amount:

[0168]

[0169] The results show that about 0.006 units of load should be migrated from the bottleneck node to the idle node to reduce the workload of the bottleneck node and optimize the performance of the entire system. This load migration reflects the optimal migration strategy based on the current resources and load conditions, which means that the resources of the idle nodes can be effectively utilized to reduce the pressure on the bottleneck nodes.

[0170] A dynamic scheduling method for container computing is provided. The dynamic scheduling method for container computing is performed based on the dynamic scheduling system for container computing, and includes the following steps:

[0171] S1: Based on the resource usage data of multiple nodes in the container cluster, calculate and obtain the CPU resource usage value, memory usage, and network bandwidth demand value, generate the CPU, memory, and network bandwidth distribution probability values ​​of multiple nodes, establish a node dynamic resource model, and obtain the node resource change probability value;

[0172] S2: Based on the probability value of node resource changes, compare the load data of CPU resources, memory usage and network bandwidth between nodes, calculate the load and resource differences of multiple nodes, perform resource reallocation, correct the load balancing status of multiple node resources, obtain the resource reallocation balance value, analyze the node resource matching situation, and generate the resource scheduling optimization value;

[0173] S3: Based on the resource scheduling optimization value, combined with the CPU resource usage value, memory usage, and network bandwidth requirement value, the task requirements are matched, the task and node resource matching value is calculated, and the node with the highest matching degree is selected to execute the priority task, and the task intelligent scheduling allocation result is obtained;

[0174] S4: Based on the results of intelligent task scheduling and allocation, monitor the changes in node CPU resource usage, memory usage, and network bandwidth demand values, analyze the changes in resource matching between nodes, detect load deviations, perform task migration and load redistribution, obtain node adaptive load adjustment values, identify bottleneck nodes, migrate some tasks to idle nodes, and obtain bottleneck node load migration values.

[0175] The above are only preferred embodiments of the present invention and are not intended to limit the present invention in other forms. Any technician familiar with the profession may use the technical contents disclosed above to change or modify them into equivalent embodiments with equivalent changes and apply them to other fields. However, any simple modification, equivalent change and modification made to the above embodiments based on the technical essence of the present invention without departing from the technical solution of the present invention still falls within the protection scope of the technical solution of the present invention.

Claims

1. A dynamic scheduling system for container computing, characterized in that: The system comprises: The resource status modeling module obtains CPU, memory, and network bandwidth parameters based on the resource usage data of nodes in the container cluster, analyzes the multi-node resource characteristics, generates multi-node probability distribution, and establishes a dynamic resource model to calculate the probability value of node resource changes; The load deviation optimization module compares the node resources and load data based on the node resource change probability value, analyzes the differences between nodes, performs resource reallocation, corrects the global equilibrium state, and obtains the resource scheduling optimization value; The intelligent scheduling strategy module evaluates the task matching degree based on the resource scheduling optimization value, combined with node resources, bandwidth and delay data, and sorts the task priorities, selects nodes to execute tasks, and generates intelligent scheduling allocation for tasks; The self-matching strategy adjustment module monitors multi-node resources in real time based on the intelligent scheduling and allocation of tasks, records bandwidth and load changes, compares resource differences of the current task execution node, calculates resource matching changes between multiple nodes, and if load deviation is found, performs task migration or load adjustment between nodes to generate node adaptive load adjustment values; The bottleneck prevention scheduling module identifies the node load bottleneck based on the node adaptive load adjustment value, obtains the bottleneck node, migrates some tasks to the idle node, reallocates the tasks based on the task and node characteristics, and generates the bottleneck node load migration value.

2. The dynamic scheduling system for container computing according to claim 1, characterized in that: The dynamic resource model includes multi-node probability distribution of CPU parameters, memory parameters, and network bandwidth parameters; the node resource change probability value includes CPU change probability, memory change probability, and network bandwidth change probability; the resource scheduling optimization value includes a global balance correction value, a resource reallocation value, and a node load difference value; the task intelligent scheduling allocation includes task priority, task matching evaluation value, and execution node selection; the node adaptive load adjustment value includes task migration value, load adjustment value, and resource matching change value; the bottleneck node load migration value includes a bottleneck node identification value and a node characteristic matching value.

3. The dynamic scheduling system for container computing according to claim 2, characterized in that: The step of calculating the node resource change probability value is specifically as follows: Based on the resource usage data of multiple nodes in the container cluster, the resource characteristic parameters of CPU, memory, and network bandwidth are extracted to build a node resource usage matrix. Extracting resource usage frequency data of each node from the node resource usage matrix, calculating the resource usage statistical distribution of multiple nodes through the frequency data, analyzing the resource characteristics among multiple nodes by integrating CPU, memory and network bandwidth parameters, and constructing a resource characteristic probability distribution model through multivariate analysis to obtain a multi-node resource characteristic probability distribution matrix; The probability integral calculation is performed on the data of the multi-node resource characteristic probability distribution matrix using the formula: Calculate and generate the probability value of node resource change; Among them, P i represents the resource change probability value of node i, N represents the total number of nodes, and U ij represents the resource usage frequency of nodes i and j, R ij represents the resource characteristic relationship coefficient between node i and node j, M ij represents the resource interaction influence matrix between node i and node j, γ is the interaction parameter, and ∈ is a constant used to avoid the denominator being zero.

4. The dynamic scheduling system for container computing according to claim 3, characterized in that: The steps for obtaining the resource scheduling optimization value are specifically as follows: Based on the node resource change probability value, calling multiple node load data, analyzing the comparison between node resource usage and load data, and calculating the difference between resource allocation and load, generating a node resource difference analysis value; Based on the node resource difference analysis value, resource characteristic parameters of multiple nodes are extracted, the intersection of resources between nodes is analyzed, and resources are reallocated according to the matching between resource differences and characteristic parameters to obtain node resource reallocation results; Using the node resource reallocation results and combining the load data of multiple nodes, the formula is adopted: Calculate the global balance correction value and obtain the resource scheduling optimization value; Among them, O represents the resource scheduling optimization value, N represents the total number of nodes, and D i represents the resource reallocation value of node i, L i represents the load data of node i, T i represents the resource characteristics of node i, V i represents the resource change of node i, α represents the weight parameter of the resource reallocation value, which is the influence degree of resource allocation, β represents the weight parameter of the square value of the node load, which is the influence degree of load change, γ represents the adjustment coefficient of the influence of node resource characteristics on the correction value, which is the difference in characteristics between nodes, and δ is a constant used to avoid the denominator being zero.

5. The dynamic scheduling system for container computing according to claim 4, characterized in that: The steps for obtaining the task intelligent scheduling allocation are specifically as follows: Based on the resource scheduling optimization value, call the node resource parameters, bandwidth and delay data, evaluate the current node resources and task requirements, and calculate the impact of node resources and bandwidth on task adaptability to generate a task matching evaluation result; Utilizing the task matching evaluation result, based on the node resource usage and task adaptability, extracting the adaptability parameters of the node resources, calculating the adaptation priorities of multiple nodes, and sorting the tasks according to the priority values ​​to obtain the task priority sorting results; Based on the task priority ranking results, combined with the performance characteristics of the node and the resource adaptability parameters, the formula is adopted: Calculate the task allocation weight, select the optimal node for task allocation, and generate intelligent task scheduling allocation; Among them, W i represents the task allocation weight of node i, P i represents the performance index of node i, D req Represents the performance requirement of the task, which is the square root of the reciprocal of the absolute value of the difference with the node performance plus one, and is used to evaluate the adaptability of the node performance to the task requirement. i represents the current available resources of node i, B i Represents the total resource capacity of node i, which is used to calculate the utilization of node resources and indicates the ratio of the current resources of the node to the total resources.

6. The dynamic scheduling system for container computing according to claim 5, characterized in that: The step of obtaining the node adaptive load adjustment value is specifically as follows: Based on the intelligent task scheduling and allocation, the resource usage of multiple nodes is monitored in real time, the bandwidth and load data of multiple nodes are recorded, the dynamic change characteristics of the bandwidth and load of each node are extracted, and the resource differences of the nodes are calculated to obtain the multi-node resource monitoring results; By using the multi-node resource monitoring results, the bandwidth and load characteristics of the current task execution node and other nodes are compared, the resource changes of the multiple nodes are analyzed, the resource matching degree changes between the nodes are calculated, and the resource matching degree change values ​​between the nodes are obtained; Based on the change value of the resource matching degree between the nodes, it is determined whether the load difference between the nodes exceeds the set threshold. If it exceeds the threshold, task migration or load adjustment is performed, using the formula: Calculate and generate node adaptive load adjustment value; Among them, ΔL represents the adaptive load adjustment value, R1 represents the resource utilization rate of node 1, R2 represents the resource utilization rate of node 2, σ R Represents the standard deviation of resource usage.

7. The dynamic scheduling system for container computing according to claim 6, characterized in that: The steps for obtaining the bottleneck node load migration value are specifically as follows: Based on the node adaptive load adjustment value, monitor the load status of all nodes, extract the load data of each node, analyze whether the load parameter exceeds the preset load threshold, identify the bottleneck node exceeding the threshold, and generate the bottleneck node identification result; Using the bottleneck node identification result, the nodes suitable for task migration in the bottleneck nodes are screened, and the resource characteristics and task requirements between the bottleneck nodes and the idle nodes are analyzed in combination with the resource parameters of the idle nodes, the task migration target of the bottleneck node is determined, and the task migration matching result of the bottleneck node is obtained; The bottleneck node task migration matching result is called, and based on the resource difference characteristics of the bottleneck node and the idle node, the formula is adopted: Calculate the load migration amount of the bottleneck node and obtain the load migration value T of the bottleneck node; Among them, L b Represents the current load value of the bottleneck node, L f Represents the current load value of the idle node, R b Represents the total resources of the bottleneck node, R f Represents the total amount of resources of idle nodes, and K represents the adjustment coefficient obtained based on previous data, which is used to match the differences in resource requirements under differentiated environments.

8. A dynamic scheduling method for container computing, characterized in that: The dynamic scheduling system for container computing according to any one of claims 1 to 7 is executed, comprising the following steps: Based on the resource usage data of multiple nodes in the container cluster, the CPU resource usage value, memory usage, and network bandwidth demand value are calculated, the CPU, memory, and network bandwidth distribution probability values ​​of multiple nodes are generated, a node dynamic resource model is established, and the probability value of node resource change is obtained; Based on the node resource change probability value, compare the load data of CPU resources, memory usage and network bandwidth between nodes, calculate the multi-node load and resource differences, perform resource reallocation, correct the multi-node resource load balancing state, obtain the resource reallocation balance value, analyze the node resource matching situation, and generate the resource scheduling optimization value; Based on the resource scheduling optimization value, combined with the CPU resource usage value, memory usage, and network bandwidth requirement value, a matching operation is performed on the task requirements, the task and node resource matching value is calculated, and the node with the highest matching degree is selected to execute the priority task, and the task intelligent scheduling allocation result is obtained; Based on the results of the intelligent task scheduling and allocation, monitor the changes in node CPU resource usage, memory usage, and network bandwidth demand values, analyze the changes in resource matching between nodes, detect load deviations, perform task migration and load redistribution, obtain node adaptive load adjustment values, identify bottleneck nodes, migrate some tasks to idle nodes, and obtain bottleneck node load migration.

Images