Distributed task load balancing scheduling method and system

Abstract

The invention relates to the technical field of load balancing, and discloses a distributed task load balancing scheduling method and system, and the method comprises the steps: obtaining a performance demand index of a to-be-executed task, and distributing the to-be-executed task for a server node in a server cluster network topology according to the performance demand index of the to-be-executed task; obtaining a historical load change rate of the server nodes, and determining a load threshold value corresponding to each server node according to the historical load change rate of the server nodes; and obtaining real-time performance indexes of the server nodes, determining load score values according to the real-time performance indexes of the server nodes, screening out overload server nodes of which the load score values exceed a load threshold value, and carrying out load balancing scheduling on tasks in the overload server nodes. According to the method, the optimal distribution node of each task can be accurately calculated, a refined task scheduling strategy of a server cluster network is provided, and the load balance degree is effectively improved.

Description

Technical Field

[0001] This application relates to the technical field of load balancing, and more specifically, to a distributed task load balancing scheduling method and system. Background Art

[0002] The cloud platform is constructed in different application forms, and digital resources are stored in the service platform. Resource users can access the cloud platform, and efficiently use the cloud service platform to obtain relevant resources without being restricted by time and location. The load balancing of the cloud platform server nodes refers to reasonably distributing network requests or computing tasks to multiple server nodes through specific technologies and strategies to ensure the balanced distribution of resources and improve the performance, reliability, and scalability of the system.

[0003] However, due to the high dynamicity and heterogeneity of the cloud platform environment, and different tasks having different resource requirements for server nodes, the existing technologies cannot meet the diverse needs of tasks during the load balancing scheduling process of server nodes, resulting in unreasonable task allocation and low scheduling efficiency. Summary of the Invention

[0004] The present invention provides a distributed task load balancing scheduling method and system to solve the problems of unreasonable task allocation and low scheduling efficiency in the load balancing scheduling of server nodes in the prior art. The method includes: obtaining the performance requirement indicators of the task to be executed, and allocating the task to be executed to the server nodes in the server cluster network topology according to the performance requirement indicators of the task to be executed; obtaining the historical load change rate of the server nodes, and determining the load threshold corresponding to each server node according to the historical load change rate of the server nodes; obtaining the real-time performance indicators of the server nodes, determining the load score value according to the real-time performance indicators of the server nodes, screening out the overloaded server nodes whose load score values exceed the load threshold, and performing load balancing scheduling on the tasks in the overloaded server nodes.

[0005] Further, the step of allocating the task to be executed to the server nodes in the server cluster network topology according to the performance requirement indicators of the task to be executed includes: obtaining the performance requirement indicators of the task to be executed, where the performance requirement indicators include CPU requirement indicators, IO requirement indicators, network requirement indicators, and memory requirement indicators; establishing a center point, and evenly drawing rays from the center point to the surrounding as index axes; performing normalization processing on the performance requirement indicators of the task to be executed, and filling the normalized performance requirement indicators into the index axes; connecting the normalized performance requirement indicators on the index axes in sequence to obtain a radar chart of the performance requirement indicators of the task to be executed, and allocating the task to be executed to the server nodes according to the radar chart of the performance requirement indicators of the task to be executed.

[0006] Further, allocating the to-be-executed tasks to server nodes according to the radar chart of performance requirement indicators of the to-be-executed tasks includes: obtaining the node performance indicators of the current server node, where the node performance indicators include CPU resource indicators, IO resource indicators, network resource indicators, and memory resource indicators; establishing a radar chart of node resource indicators according to the node performance indicators, and calculating the matching degree between the radar chart of node resource indicators of each server node and the radar chart of performance requirement indicators of the to-be-executed tasks; and allocating the to-be-executed tasks to server nodes according to the matching degree between the radar chart of node resource indicators of each server node and the radar chart of performance requirement indicators of the to-be-executed tasks.

[0007] Further, calculating the matching degree between the radar chart of node resource indicators of each server node and the radar chart of performance requirement indicators of the to-be-executed tasks includes: making the center points of the radar chart of node resource indicators and the radar chart of performance requirement indicators of the to-be-executed tasks coincide, and aligning the vertices of the two radar charts to obtain a coincident chart of the radar chart of node resource indicators and the radar chart of performance requirement indicators; calculating the absolute distance difference between the corresponding vertices of the radar chart of node resource indicators and the radar chart of performance requirement indicators in the coincident chart, and determining the average absolute distance difference according to the absolute distance differences of all vertices in the coincident chart; calculating the cosine value of the angle between the corresponding sides of the radar chart of node resource indicators and the radar chart of performance requirement indicators in the coincident chart, and determining the average cosine value according to all the cosine values of the angles in the coincident chart; and multiplying the average absolute distance difference by the average cosine value to obtain the matching degree between the radar chart of node resource indicators of each server node and the radar chart of performance requirement indicators of the to-be-executed tasks.

[0008] Further, determining the load threshold corresponding to each server node according to the historical load change rate of the server node includes: clustering each server node according to the historical load change rate of the server node, obtaining the preset load threshold of the server node, and determining the clustering center of the clustering cluster corresponding to the server node according to the clustering result; and correcting the preset load threshold according to the clustering center of the clustering cluster corresponding to the server node to obtain the corrected load threshold.

[0009] Further, clustering each server node according to the historical load change rate of the server node includes: establishing a sample data set according to the historical load change rate of all server nodes, and randomly selecting k initial clustering centers of the sample data set; calculating the Manhattan distance from the sample data in the sample data set to the initial clustering centers, and dividing each server node into the corresponding clustering cluster according to the Manhattan distance from the sample data in the sample data set to the initial clustering centers; calculating the average value of the sample data in each clustering cluster, and recalculating the clustering center according to the average value of the sample data in each clustering cluster; repeating the above steps iteratively until the clustering center no longer changes or the number of iterations reaches the preset maximum number of iterations to obtain the clustering result of the server nodes.

[0010] Further, determining the load score value according to the real-time performance metrics of the server nodes includes: obtaining the real-time performance metrics of each server node in the historical server cluster network topology, where the real-time performance metrics include CPU operation metrics, IO occupancy metrics, network operation metrics, and memory occupancy metrics, and establishing a training sample set based on the real-time performance metrics of each server node in the historical server cluster network topology; performing expert manual scoring on the real-time performance metrics in the training sample set to obtain the load score codes with manual annotations; establishing a deep neural network model to perform load score coding on the real-time performance metrics in the training sample set, and calculating the loss value between the load score codes output by the deep neural network and the load capacity score codes with manual annotations; iteratively training the parameters of the deep neural network by minimizing the loss value to obtain a deep neural network model that can encode the real-time performance metrics of the server nodes into corresponding load score values; inputting the real-time performance metrics of the current server node into the trained deep neural network model to obtain the corresponding load score value.

[0011] Further, performing load balancing scheduling on the tasks in the overloaded server nodes includes: detecting the surrounding nodes of the overloaded server nodes in the server cluster network topology, and calculating the replacement matching degree between the surrounding nodes and the tasks to be offloaded; when the replacement matching degree between the surrounding nodes and the tasks to be offloaded is greater than a first preset threshold, transferring the tasks to be offloaded to the corresponding surrounding nodes.

[0012] Further, calculating the replacement matching degree between the surrounding nodes and the tasks to be offloaded includes: calculating the replacement matching degree between the surrounding nodes and the tasks to be offloaded according to the replacement matching degree calculation formula, and the replacement matching degree calculation formula is , where is the replacement matching degree, is the node load value of the i-th surrounding node, is the corrected load threshold of the i-th surrounding node, is the network distance value between the i-th surrounding node and the overloaded server node, is the matching degree between the node resource index radar chart of the i-th surrounding node and the performance requirement index radar chart of the task to be offloaded, is the preset standard matching degree, is the range adjustment coefficient.

[0013] To achieve the above object, the present invention further provides a distributed task load balancing scheduling system, including: a task module, configured to obtain the performance requirement indicators of the tasks to be executed, and allocate the tasks to be executed to the server nodes in the server cluster network topology according to the performance requirement indicators of the tasks to be executed; a threshold module, configured to obtain the historical load change rate of the server nodes, and determine the load threshold corresponding to each server node according to the historical load change rate of the server nodes; a scheduling module, configured to obtain the real-time performance indicators of the server nodes, determine the load score value according to the real-time performance indicators of the server nodes, filter out the overloaded server nodes whose load score value exceeds the load threshold, and perform load balancing scheduling on the tasks in the overloaded server nodes.

[0014] The beneficial effects of the present invention are as follows. By applying the above technical solutions, the present invention can allocate corresponding and suitable server nodes for the tasks to be executed with different performance requirements, accurately calculate the optimal allocation nodes for each task, simultaneously monitor the performance indicators of each server node in real time, timely detect the overloaded server nodes and allocate the overloaded tasks therein to the optimal and suitable nodes in the server cluster network topology, provide a refined task scheduling strategy for the server cluster network, and effectively improve the load balancing degree. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] To more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative efforts.

[0016] Figure 1 FIG. shows the overall flowchart of a distributed task load balancing scheduling method proposed by an embodiment of the present invention; Figure 2 FIG. shows the structural schematic diagram of a distributed task load balancing scheduling system proposed by an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0017] The following will clearly and completely describe the technical solutions in the embodiments of the present application with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some embodiments of the present application, rather than all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

[0018] The embodiments of the present application provide a distributed task load balancing scheduling method, as Figure 1 shown, including: S101. Obtain the performance requirement indicators of the task to be executed, and allocate the task to be executed to the server nodes in the server cluster network topology according to the performance requirement indicators of the task to be executed. In some embodiments of the present application, the step of allocating the task to be executed to the server nodes in the server cluster network topology according to the performance requirement indicators of the task to be executed includes: obtaining the performance requirement indicators of the task to be executed, where the performance requirement indicators include CPU requirement indicators, IO requirement indicators, network requirement indicators, and memory requirement indicators; establishing a center point, and evenly drawing rays from the center point in all directions as index axes; performing normalization processing on the performance requirement indicators of the task to be executed, and filling the normalized performance requirement indicators into the index axes; connecting the normalized performance requirement indicators on the index axes in sequence to obtain a radar chart of the performance requirement indicators of the task to be executed, and allocating the task to be executed to the server nodes according to the radar chart of the performance requirement indicators of the task to be executed.

[0019] In this embodiment, by detecting the CPU requirement indicators, IO requirement indicators, network requirement indicators, and memory requirement indicators of the task to be executed, and integrating the above four indicators into the radar chart of the performance requirement indicators, the multi-dimensional requirements of the task to be executed are comprehensively reflected, so that the corresponding and suitable server nodes can be allocated through the radar chart of the performance requirement indicators.

[0020] In some embodiments of the present application, the step of allocating the task to be executed to the server nodes according to the radar chart of the performance requirement indicators of the task to be executed includes: obtaining the node performance indicators of the current server node, where the node performance indicators include CPU resource indicators, IO resource indicators, network resource indicators, and memory resource indicators; establishing a radar chart of the node resource indicators according to the node performance indicators, and calculating the matching degree between the radar chart of the node resource indicators of each server node and the radar chart of the performance requirement indicators of the task to be executed; allocating the task to be executed to the server nodes according to the matching degree between the radar chart of the node resource indicators of each server node and the radar chart of the performance requirement indicators of the task to be executed.

[0021] In this embodiment, the node performance indicators of each server node are detected in sequence, and the node performance indicators are integrated into the radar chart of the node resource indicators, so as to calculate the matching degree between the radar chart of the node resource indicators of each server node and the radar chart of the performance requirement indicators, and allocate the task to be executed to the server according to the matching degree.

[0022] In some embodiments of the present application, calculating the matching degree between the node resource index radar chart of each server node and the performance requirement index radar chart of the to-be-executed task includes: overlapping the center points of the node resource index radar chart and the performance requirement index radar chart of the to-be-executed task, and aligning the vertices of the two radar charts to obtain an overlapping chart of the node resource index radar chart and the performance requirement index radar chart; calculating the absolute distance difference between the corresponding vertices of the node resource index radar chart and the performance requirement index radar chart in the overlapping chart, and determining the average absolute distance difference according to the absolute distance differences of all vertices in the overlapping chart; calculating the cosine value of the included angle between the corresponding sides of the node resource index radar chart and the performance requirement index radar chart in the overlapping chart, and determining the average cosine value according to all the cosine values of the included angles in the overlapping chart; multiplying the average absolute distance difference by the average cosine value to obtain the matching degree between the node resource index radar chart and the performance requirement index radar chart of the to-be-executed task.

[0023] In this embodiment, the center points of the two radar charts are overlapped, and the vertices of each index are aligned according to CPU requirement index - CPU resource index, IO requirement index - IO resource index, network requirement index - network resource index, and memory requirement index - memory resource index to obtain an overlapping chart. The edges connecting two adjacent vertices of the node resource index radar chart in the overlapping chart and the edges connecting the corresponding vertices of the performance requirement index radar chart are extracted, and the cosine value of the acute angle formed by the two edges is calculated to obtain the cosine value of the included angle. The average cosine value of all the cosine values of the included angles in the overlapping chart is calculated. The resource deviation between the node resources and the resources required for the to-be-executed task is calculated more accurately through the average cosine value, and at the same time, the matching degree between the node resource index radar chart and the performance requirement index radar chart of the to-be-executed task is calculated in combination with the average absolute distance difference.

[0024] S102. Obtain the historical load change rate of the server node, and determine the load threshold corresponding to each server node according to the historical load change rate of the server node. In some embodiments of the present application, determining the load threshold corresponding to each server node according to the historical load change rate of the server node includes: clustering each server node according to the historical load change rate of the server node to obtain the preset load threshold of the server node, and determining the clustering center of the clustering cluster corresponding to the server node according to the clustering result; correcting the preset load threshold according to the clustering center of the clustering cluster corresponding to the server node to obtain the corrected load threshold.

[0025] Clustering each server node according to the historical load change rate of the server node includes: establishing a sample data set according to the historical load change rates of all server nodes, and randomly selecting k initial clustering centers from the sample data set; calculating the Manhattan distance from the sample data in the sample data set to the initial clustering centers, and dividing each server node into the corresponding clustering cluster according to the Manhattan distance from the sample data in the sample data set to the initial clustering centers; calculating the average value of the sample data in each clustering cluster, and recalculating the clustering centers according to the average value of the sample data in each clustering cluster; repeating the above steps iteratively until the clustering centers no longer change or the number of iterations reaches the preset maximum number of iterations, to obtain the clustering result of the server nodes.

[0026] In this embodiment, a preset load threshold is manually set for each server node. Based on the k-means clustering algorithm, each server node is clustered according to the historical load change rate of the server node, the clustering centers of the corresponding clustering clusters of each server node are extracted, the clustering centers are normalized, and their values are limited between [0.5 - 1.5]. The normalized clustering centers are set as correction coefficients, and the correction coefficients are multiplied by the preset load thresholds of the server nodes to obtain the corrected load thresholds.

[0027] S103, obtaining the real-time performance metrics of the server nodes, determining the load score values according to the real-time performance metrics of the server nodes, screening out the overloaded server nodes whose load score values exceed the load threshold, and performing load balancing scheduling on the tasks in the overloaded server nodes.

[0028] In some embodiments of the present application, determining the load score value according to the real-time performance metrics of the server nodes includes: obtaining the real-time performance metrics of each server node in the historical server cluster network topology, where the real-time performance metrics include CPU operation metrics, IO occupancy metrics, network operation metrics, and memory occupancy metrics, and establishing a training sample set according to the real-time performance metrics of each server node in the historical server cluster network topology; performing expert manual scoring on the real-time performance metrics in the training sample set to obtain the manually labeled load score codes; establishing a deep neural network model to perform load score coding on the real-time performance metrics in the training sample set, and calculating the loss value between the load score codes output by the deep neural network and the manually labeled load capacity score codes; iteratively training the parameters of the deep neural network by minimizing the loss value to obtain a deep neural network model that can encode the real-time performance metrics of the server nodes into the corresponding load score values; inputting the real-time performance metrics of the current server node into the trained deep neural network model to obtain the corresponding load score value.

[0029] In this embodiment, the historical operation records of the cloud platform are collected to obtain the real-time performance indicators of each server node in the historical server cluster network topology. The real-time performance indicators are vectorized to obtain a training sample set. A deep neural network model is established to perform load scoring encoding on the vectorized real-time performance indicators. The loss value between the load scoring encoding output by the network and the load scoring encoding manually annotated by actual computer experts is calculated. The parameters of the deep neural network are iteratively trained by minimizing the loss value, so as to obtain a trained deep neural network model and output the load scoring value corresponding to the current server node.

[0030] In some embodiments of the present application, the load balancing scheduling of tasks in the overloaded server nodes includes: detecting the surrounding nodes of the overloaded server nodes in the server cluster network topology, and calculating the replacement matching degree between the surrounding nodes and the tasks to be unloaded; when the replacement matching degree between the surrounding nodes and the tasks to be unloaded is greater than a first preset threshold, transfer the tasks to be unloaded to the corresponding surrounding nodes.

[0031] In this embodiment, after detecting the overloaded server nodes, the task with the largest load occupancy rate is extracted as the task to be unloaded.

[0032] In some embodiments of the present application, the calculation of the replacement matching degree between the surrounding nodes and the tasks to be unloaded includes: calculating the replacement matching degree between the surrounding nodes and the tasks to be unloaded according to the replacement matching degree calculation formula, and the replacement matching degree calculation formula is , where is the replacement matching degree, is the node load value of the i-th surrounding node, is the corrected load threshold of the i-th surrounding node, is the network distance value between the i-th surrounding node and the overloaded server node, is the matching degree between the node resource index radar chart of the i-th surrounding node and the performance requirement index radar chart of the task to be unloaded, is the preset standard matching degree, is the range adjustment coefficient, is the natural exponential function.

[0033] In this embodiment, first, the nodes within the first preset range of the server node are used as the surrounding nodes. By the node load utilization rate of the surrounding nodes corresponding to the overloaded server node is obtained. By the distance influence coefficient between the overloaded server node and the surrounding nodes is obtained. The smaller the network distance, the higher this value. By Calculate the matching degree between the surrounding nodes and the task to be offloaded, and comprehensively determine whether the surrounding nodes can accept the scheduling of the overloaded server node by combining the above parameters. If there are no surrounding nodes that meet the conditions, gradually expand the range of the surrounding nodes until surrounding nodes that meet the conditions appear, so as to achieve the load balancing of the server cluster network.

[0034] To achieve the above object, the present invention also provides a distributed task load balancing scheduling system, including: a task module, which is used to obtain the performance requirement indicators of the task to be executed, and allocate the task to be executed to the server nodes in the server cluster network topology according to the performance requirement indicators of the task to be executed; a threshold module, which is used to obtain the historical load change rate of the server node, and determine the load threshold corresponding to each server node according to the historical load change rate of the server node; a scheduling module, which is used to obtain the real-time performance indicators of the server node, determine the load score value according to the real-time performance indicators of the server node, screen out the overloaded server nodes whose load score values exceed the load threshold, and perform load balancing scheduling on the tasks in the overloaded server nodes.

[0035] By applying the above technical solutions, the present invention obtains the performance requirement indicators of the task to be executed, and allocates the task to be executed to the server nodes in the server cluster network topology according to the performance requirement indicators of the task to be executed; obtains the historical load change rate of the server node, and determines the load threshold corresponding to each server node according to the historical load change rate of the server node; obtains the real-time performance indicators of the server node, determines the load score value according to the real-time performance indicators of the server node, screens out the overloaded server nodes whose load score values exceed the load threshold, and performs load balancing scheduling on the tasks in the overloaded server nodes. The present invention can accurately calculate the optimal allocation nodes of each task, provide a refined task scheduling strategy for the server cluster network, and effectively improve the load balancing degree.

[0036] Through the description of the above embodiments, those skilled in the art can clearly understand that the present invention can be implemented by hardware or by means of software plus a necessary general hardware platform. Based on such an understanding, the technical solution of the present invention can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.), including several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in various implementation scenarios of the present invention.

[0037] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the various embodiments of the present application.

Claims

1. A distributed task load balancing and scheduling method, characterized in that The method includes: Obtaining the performance requirement indicators of the task to be executed, and allocating the task to be executed to the server nodes in the server cluster network topology according to the performance requirement indicators of the task to be executed; Obtaining the historical load change rate of the server nodes, and determining the load threshold corresponding to each server node according to the historical load change rate of the server nodes; Obtaining the real-time performance indicators of the server nodes, determining the load score value according to the real-time performance indicators of the server nodes, screening out the overloaded server nodes whose load score values exceed the load threshold, and performing load balancing scheduling on the tasks in the overloaded server nodes.

2. The distributed task load balancing and scheduling method according to claim 1, wherein The allocating the task to be executed to the server nodes in the server cluster network topology according to the performance requirement indicators of the task to be executed includes: Obtaining the performance requirement indicators of the task to be executed, where the performance requirement indicators include CPU requirement indicators, IO requirement indicators, network requirement indicators, and memory requirement indicators; Establishing a center point, and evenly drawing rays from the center point to the surroundings as the index axes; Performing normalization processing on the performance requirement indicators of the task to be executed, and filling the normalized performance requirement indicators into the index axes; Connecting the normalized performance requirement indicators on the index axes in sequence to obtain a radar chart of the performance requirement indicators of the task to be executed, and allocating the task to be executed to the server nodes according to the radar chart of the performance requirement indicators of the task to be executed.

3. The distributed task load balancing scheduling method according to claim 2, wherein The allocating the task to be executed to the server nodes according to the radar chart of the performance requirement indicators of the task to be executed includes: Obtaining the node performance indicators of the current server node, where the node performance indicators include CPU resource indicators, IO resource indicators, network resource indicators, and memory resource indicators; Establishing a radar chart of the node resource indicators according to the node performance indicators, and calculating the matching degree between the radar chart of the node resource indicators of each server node and the radar chart of the performance requirement indicators of the task to be executed; Allocating the task to be executed to the server nodes according to the matching degree between the radar chart of the node resource indicators of each server node and the radar chart of the performance requirement indicators of the task to be executed.

4. The distributed task load balancing scheduling method according to claim 3, wherein The calculating the matching degree between the radar chart of the node resource indicators of each server node and the radar chart of the performance requirement indicators of the task to be executed includes: Coinciding the center points of the radar chart of the node resource indicators and the radar chart of the performance requirement indicators of the task to be executed, and aligning the vertices of the two radar charts to obtain a coincidence chart of the radar chart of the node resource indicators and the radar chart of the performance requirement indicators; Calculating the absolute distance difference between the corresponding vertices of the radar chart of the node resource indicators and the radar chart of the performance requirement indicators in the coincidence chart, and determining the average absolute distance difference according to the absolute distance differences of all the vertices in the coincidence chart; Calculating the cosine value of the included angle between the corresponding sides of the radar chart of the node resource indicators and the radar chart of the performance requirement indicators in the coincidence chart, and determining the average cosine value according to all the cosine values in the coincidence chart; Multiplying the average absolute distance difference by the average cosine value to obtain the matching degree between the radar chart of the node resource indicators and the radar chart of the performance requirement indicators of the task to be executed.

5. The distributed task load balancing scheduling method according to claim 4, wherein The determining the load threshold corresponding to each server node according to the historical load change rate of the server nodes includes: Cluster each server node according to the historical load change rate of the server node, obtain the preset load threshold of the server node, and determine the cluster center of the cluster corresponding to the server node according to the clustering result; Modify the preset load threshold according to the cluster center of the cluster corresponding to the server node to obtain the modified load threshold.

6. The distributed task load balancing scheduling method according to claim 5, wherein The clustering of each server node according to the historical load change rate of the server node includes: Establish a sample data set according to the historical load change rate of all server nodes, and randomly select k initial cluster centers of the sample data set; Calculate the Manhattan distance from the sample data in the sample data set to the initial cluster center, and divide each server node into the corresponding cluster according to the Manhattan distance from the sample data in the sample data set to the initial cluster center; Calculate the average value of the sample data in each cluster, and recalculate the cluster center according to the average value of the sample data in each cluster; Repeat the above steps until the cluster center no longer changes or the number of iterations reaches the preset maximum number of iterations to obtain the clustering result of the server node.

7. The distributed task load balancing and scheduling method according to claim 1, wherein The determination of the load score value according to the real-time performance index of the server node includes: Obtain the real-time performance indexes of each server node in the historical server cluster network topology. The real-time performance indexes include CPU operation indexes, IO occupancy indexes, network operation indexes, and memory occupancy indexes, and establish a training sample set according to the real-time performance indexes of each server node in the historical server cluster network topology; Perform expert manual scoring on the real-time performance indexes in the training sample set to obtain the load score code with manual annotation; Establish a deep neural network model to perform load score coding on the real-time performance indexes in the training sample set, and calculate the loss value between the load score code output by the deep neural network and the load capacity score code with manual annotation; Iteratively train the parameters of the deep neural network by minimizing the loss value to obtain a deep neural network model that can encode the real-time performance indexes of the server node into the corresponding load score value; Input the real-time performance indexes of the current server node into the trained deep neural network model to obtain the corresponding load score value.

8. The distributed task load balancing and scheduling method according to claim 5, characterized in that The load balancing scheduling of the tasks in the overloaded server node includes: Detect the surrounding nodes of the overloaded server node in the server cluster network topology, and calculate the replacement matching degree between the surrounding nodes and the task to be unloaded; When the replacement matching degree between the surrounding node and the task to be unloaded is greater than the first preset threshold, transfer the task to be unloaded to the corresponding surrounding node.

9. The distributed task load balancing scheduling method according to claim 8, characterized in that, The calculation of the replacement matching degree between the surrounding node and the task to be unloaded includes: Calculate the replacement matching degree between the surrounding node and the task to be unloaded according to the replacement matching degree calculation formula. The replacement matching degree calculation formula is ， Among them, is the replacement matching degree, is the node load value of the i-th surrounding node, is the corrected load threshold of the i-th surrounding node, is the network distance value between the i-th surrounding node and the overloaded server node, is the matching degree between the node resource index radar chart of the i-th surrounding node and the performance requirement index radar chart of the task to be unloaded, is the preset standard matching degree, is the range adjustment coefficient.

10. A distributed task load balancing and scheduling system, characterized in that including: A task module for obtaining the performance requirement indexes of the task to be executed and allocating the task to be executed to the server nodes in the server cluster network topology according to the performance requirement indexes of the task to be executed; A threshold module for obtaining the historical load change rate of the server node and determining the load threshold corresponding to each server node according to the historical load change rate of the server node; The scheduling module is used to obtain the real-time performance metrics of server nodes, determine the load score value according to the real-time performance metrics of server nodes, filter out the overloaded server nodes whose load score value exceeds the load threshold, and perform load balancing scheduling on the tasks in the overloaded server nodes.

Images