A server cluster load balancing method based on edge computing

Abstract

The present invention belongs to the technical field of server cluster load balancing. The present invention discloses a server cluster load balancing method based on edge computing, including: collecting task data, transmission data between nodes and load data of server nodes; constructing a load topology map based on the load data and the transmission data, performing multi-dimensional aggregation on the load data based on the load topology map, and obtaining a node status description; using an analysis algorithm to perform priority evaluation on the task data to obtain a task score; performing optimal task matching on the node status description based on the task score to obtain an optimal allocation plan, predicting the optimal allocation plan based on a constructed load prediction model to obtain an overload allocation, performing local task migration on the overload allocation, and obtaining a local optimization plan; the overall processing capability of the server cluster is greatly improved.

Description

Technical Field

[0001] The present invention relates to the technical field of server cluster load balancing, and more specifically, to a server cluster load balancing method based on edge computing. Background Art

[0002] With the rapid development of edge computing, more and more intelligent terminal devices need to process large amounts of data in real time. In order to alleviate the network congestion and high latency problems in the centralized cloud computing model, edge computing processes data at nodes close to the data source, significantly improving data processing efficiency and user experience. However, in the edge computing environment, the data traffic generated by terminal devices is usually dynamic and uncertain, for example: The distribution of data traffic at different times and locations may vary greatly. Intelligent transportation systems generate a large amount of data during peak hours, while the load is lower during non-peak hours. Different devices or applications have different requirements for computing, storage and bandwidth resources. Real-time video analysis requires high computing power, while simple temperature monitoring has lower resource requirements. These problems lead to serious uneven load distribution among edge nodes, with some nodes being overloaded while other nodes have idle resources. Traditional load balancing methods mostly adopt a centralized scheduling mode, which mainly relies on central servers or the cloud to uniformly manage load distribution. This process may cause higher communication delays and cannot meet the real-time requirements of edge computing. Traditional methods usually allocate tasks based on static rules, which makes it difficult to cope with dynamically changing load environments. In multi-node collaborative computing, traditional methods lack comprehensive consideration of resource collaboration and data dependencies between nodes, resulting in global suboptimal scheduling and a huge increase in resource consumption.

[0003] In view of this, the present invention proposes a server cluster load balancing method based on edge computing to solve the above problems. Summary of the invention

[0004] In order to overcome the above-mentioned defects of the prior art and to achieve the above-mentioned purpose, the present invention provides the following technical solution: a server cluster load balancing method based on edge computing, comprising:

[0005] S1, collects task data, transmission data between nodes and load data of server nodes;

[0006] S2. construct a load topology map based on the load data and the transmission data, and perform multi-dimensional aggregation on the load data based on the load topology map to obtain a node status description;

[0007] S3, use analytical algorithms to evaluate the priority of task data and obtain task scores;

[0008] S4. Based on the task score, the node status description is optimally matched to obtain the optimal allocation plan. The optimal allocation plan is predicted based on the constructed load prediction model to obtain the overload allocation. Local task migration is performed on the overload allocation to obtain a local optimization plan.

[0009] Furthermore, the load data includes: CPU utilization, memory usage, task processing volume and disk read and write rate; the task data includes: data volume and computational complexity; the transmission data includes: network bandwidth and latency.

[0010] Furthermore, the method of constructing a load topology diagram based on the load data and the transmission data includes:

[0011] Each load data is used as a graph node, the transmission data represents the connection relationship between graph nodes, and the edge weights are initialized with network bandwidth and latency. The formula for weight initialization is:

[0012] ,in, Representative The graph node and The edge weights between the nodes in the graph are Representative The graph node and The delay between graph nodes, Representative The graph node and The network bandwidth between graph nodes, represents the delay weight, Represents the network bandwidth weight. A graph is constructed based on graph nodes, connection relationships, and node edge weights to obtain a load topology graph.

[0013] Furthermore, the method of multi-dimensionally aggregating load data based on the load topology diagram includes:

[0014] zUse load data as the initial feature of each node, and construct a load adjacency matrix and a load weight matrix based on the load topology graph. The load adjacency matrix is ​​a matrix that describes the connection relationship between graph nodes, and the load weight matrix is ​​a left diagonal matrix that describes the sum of the edge weights of all nodes that have direct edges connected to the graph nodes; initialize the result set to be empty, preset the weighting ratio, perform feature traversal on the initial features based on the weighting ratio, record the traversed features as features to be processed, and amplify the features to be processed of each node by the weighting ratio to obtain the feature amplification value; replace the value of the feature to be processed in each initial feature with the feature amplification value to obtain the extended feature; use the clustering analysis algorithm to cluster the extended features to obtain feature clustering clusters, calculate the feature entropy of the feature clustering clusters, obtain the feature entropy of the feature to be processed, add the feature entropy to the feature weight set, repeat until the traversal is completed, and output the result set at this time as the feature weight set; perform feature convolution on the initial features based on the feature weight set to obtain a description of the node state.

[0015] Furthermore, the formula for performing feature convolution on the initial features is:

[0016] ;in Represents the initial feature Convolution features after layer convolution, represents the feature weight set, represents the characteristic entropy, represents the load weight matrix, represents the load adjacency matrix, Initial feature pass Convolution features after layer convolution, Representative The weight matrix of the layer, Represents a non-linear activation function.

[0017] Furthermore, the method of performing priority evaluation on task data includes:

[0018] Priority criteria are constructed based on task data. The priority criteria include: the larger the data volume, the higher the priority; the higher the computational complexity, the higher the priority; a criterion selection domain is preset, and the priority criteria are compared in pairs. A random selection algorithm is used to select values ​​within the criterion selection domain as element values ​​to construct an initial criterion matrix. The elements in the initial criterion matrix represent the relative importance of different criteria; the criterion index is initialized to be infinitesimal, the evaluation weight set is initialized to be empty, the initial criterion matrix is ​​verified for consistency, the sum of each column in the initial criterion matrix is ​​calculated to obtain the column sum, each column element in the initial criterion matrix is ​​divided by the column sum of the corresponding column to obtain an average criterion matrix, and each row element of the average criterion matrix is ​​averaged to obtain the criteria. feature vector; multiply the criterion feature vector by the initial criterion matrix to obtain a weight matrix, divide each element in the weight matrix by the element at the corresponding position in the criterion feature vector to obtain an approximate feature vector, select the element with the largest value in the approximate feature vector as the maximum feature, calculate the index based on the maximum feature to obtain a consistency index, when the consistency index is greater than the criterion index, use the consistency index at this time as the new criterion index, use the criterion feature vector at this time as the new evaluation weight set, repeat until the value of the criterion index no longer changes, and output the evaluation weight set at this time as the task rating index; perform task evaluation on the task data based on the task rating index to obtain a task score.

[0019] Furthermore, the method of performing the most appropriate task matching on the node load characteristics includes:

[0020] Sort the task data from large to small according to the task score to obtain the task sequence. Use the task sequence as the distribution order of the task data. Use the random matching algorithm to pair the task data with the node load characteristics to obtain M task matching schemes. Each pairing satisfies the task constraint condition: the data volume of the task data is less than the task processing volume in the node load characteristics.

[0021] Each task matching scheme is regarded as a chromosome, and the genes in the chromosome correspond to the pairing status of the task data and the node load characteristics; for each chromosome, the fitness of each chromosome is calculated according to the fitness calculation formula, the optimal chromosome is initialized to be empty, the optimal fitness is initialized to be infinitesimal, the trophy value is preset, and the number of chromosomes equal to the trophy value is randomly selected based on the trophy value. The fitness of the selected chromosomes is compared, and the chromosome with the largest fitness value is taken as the parent chromosome. When the number of parent chromosomes is consistent with the trophy value, it stops, and all parent chromosomes constitute the parent chromosome set;

[0022] The permutation and combination algorithm is used to pair the chromosomes in the parent chromosome set in pairs to obtain the parent combination; a gene in the parent combination is randomly selected as an exchange node, the gene after the exchange node in the parent combination is exchanged to obtain the daughter chromosome, the fitness of each daughter chromosome is calculated based on the fitness calculation formula, and the daughter chromosome with a fitness greater than or equal to the parent combination is taken as the preferred daughter; the permutation and combination algorithm is used to pair the preferred daughters in pairs to obtain the second-generation combination, a gene in the second-generation combination is randomly selected as a mutation node, the task data of the mutation node in the second-generation combination is replaced to obtain the mutant chromosome, the fitness of each mutant chromosome is calculated based on the fitness calculation formula, the mutant chromosome with a fitness greater than or equal to the second-generation combination is taken as the preferred variant, and the preferred variant with the largest fitness is selected as the candidate individual. When the fitness of the candidate individual is greater than the optimal fitness, the candidate individual is taken as the new optimal chromosome, the fitness of the candidate individual is taken as the new optimal fitness, and the preferred variant is taken as the new parent chromosome, and repeated until the value of the optimal fitness no longer changes. The optimal chromosome output at this time is the most suitable allocation scheme.

[0023] Furthermore, the calculation formula of the fitness is:

[0024] ;in, represents fitness, represents the number of genes in the chromosome (the total number of node load features), Representative The total task score of the task data matched by the node load characteristics, Representative The total amount of task data matched by the node load characteristics, Representative The task processing capacity of the node load characteristics, Representative The computational complexity of task data matching the node load characteristics, Representative The disk read and write rates of each node load characteristic.

[0025] Furthermore, the load prediction model is constructed in the following manner:

[0026] Collect historical node data and corresponding load evaluations. The historical node data includes: historical node load data and historical task allocation data. The data type of the historical node load data is consistent with the data type of the load data, and the data type of the historical task allocation data is consistent with the data type of the task data. Use the historical node data and load evaluation as training sample sets, use the linear regression model as the initial model, use the training sample set to train the linear regression model, use the historical node data as input data of the load prediction model, and use the predicted load evaluation as output data of the load prediction model. Use minimizing the error between the actual load evaluation and the predicted load evaluation as the training goal, use the mean square error function as the loss function of the load prediction model, and stop training when the loss function converges to obtain the load prediction model.

[0027] Furthermore, the method of performing local task migration on overload distribution includes:

[0028] Based on the load topology graph, the graph nodes with directly connected edges in the load topology graph corresponding to each overload allocation are used as cutting edges, and the load topology graph is cut with the cutting edges to obtain the overload local graph, and the task data assigned to all graph nodes in the overload local graph are counted to obtain the local tasks; B ants are initialized, and the value of B is consistent with the data volume of the local task, and a local task is assigned to each ant. Each ant is randomly assigned to a different graph node in the overload local graph, and the pheromone absorption rate of each ant is initialized. The ratio of the node edge weight of each edge in the overload local graph to the sum of all node edge weights is calculated to obtain the initial concentration value, and the initial pheromone concentration is assigned to each edge in the overload local graph based on the initial concentration value;

[0029] Each ant moves in the overloaded local graph, and selects the next position to which the ant will move based on the initial pheromone concentration of each edge. The ant moves towards the edge with the highest initial pheromone concentration. When the graph node information of each ant changes, the pheromone concentration of the edge passed by each ant is reduced based on the pheromone absorption rate. The ant information on each graph node is counted at fixed time intervals, and each graph node in the overloaded local graph is predicted based on the constructed load prediction model to obtain the overload assessment of each graph node. The graph nodes with overload assessment greater than or equal to the overload warning threshold will drive the ants with the lowest task scores to migrate the graph nodes. When all ants stop moving, the local tasks represented by the ants on each graph node in the overloaded local graph are the local optimization solutions.

[0030] Technical effects and advantages of a server cluster load balancing method based on edge computing in the present invention:

[0031] The present invention can accurately reflect the interdependence and complex relationships between nodes by constructing a load topology map, and use the load topology map to perform multi-dimensional aggregation of load data, provide a comprehensive state representation for subsequent task allocation, greatly improve the system's responsiveness, and ensure the global optimization effect of load balancing; by performing priority evaluation on task data, it not only provides a scientific basis for task allocation, but also improves the transparency and accuracy of the decision-making process, and ensures the rationality and fairness of the task scheduling process; by performing task allocation based on the preferred genetic algorithm, the task allocation order and matching process are optimized, ensuring that each task can be reasonably allocated to a node with a lower load according to its computing requirements, effectively avoiding excessive concentration or idleness of node resources, thereby improving the overall processing efficiency of the cluster; through local task migration of overload allocation, the performance degradation caused by overload of a single node is avoided, the distribution of tasks between nodes is flexibly adjusted, and the resource utilization and processing capacity of the server cluster are further optimized. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 A schematic diagram of a server cluster load balancing method based on edge computing of the present invention;

[0033] Figure 2 This is a schematic diagram of a server cluster load balancing system based on edge computing of the present invention. DETAILED DESCRIPTION

[0034] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0035] Example 1

[0036] See also Figure 1 As shown, the server cluster load balancing method based on edge computing described in this embodiment includes:

[0037] S1, collects task data, transmission data between nodes and load data of server nodes;

[0038] S2. construct a load topology map based on the load data and the transmission data, and perform multi-dimensional aggregation on the load data based on the load topology map to obtain a node status description;

[0039] S3, use analytical algorithms to evaluate the priority of task data and obtain task scores;

[0040] S4. Based on the task score, the node status description is optimally matched to obtain the optimal allocation plan. The optimal allocation plan is predicted based on the constructed load prediction model to obtain the overload allocation. Local task migration is performed on the overload allocation to obtain a local optimization plan.

[0041] Load data includes: CPU utilization, memory utilization, task processing volume and disk read and write rate; task data includes: data volume and computational complexity; transmission data includes: network bandwidth and latency; CPU utilization refers to the ratio of the current CPU load to its maximum processing capacity, reflecting the processor's workload; memory utilization indicates the ratio of used memory to total memory in the system, affecting program operation and data processing speed; task processing volume is the maximum amount of tasks processed simultaneously by a node; disk read and write rate is the rate of disk read and write operations, usually expressed as the amount of disk read and write data per unit time; task computational complexity refers to the amount of computing resources required to perform a task; network bandwidth refers to the amount of data transmitted over the network between different nodes per unit time, which determines the rate of data transmission; latency refers to the time required for data to reach the target from the source, affecting network response speed and real-time communication quality.

[0042] The methods for constructing a load topology diagram based on load data and transmission data include:

[0043] Each load data is used as a graph node, the transmission data represents the connection relationship between graph nodes, and the edge weights are initialized with network bandwidth and latency. The formula for weight initialization is:

[0044] ,in, Representative The graph node and The edge weights between the nodes in the graph are Representative The graph node and The delay between graph nodes, Representative The graph node and The network bandwidth between graph nodes, Represents the delay weight, which is used to balance the impact of delay on the node edge weight. Represents the network bandwidth weight, which is used to balance the impact of network bandwidth on node edge weight. The graph is constructed based on graph nodes, connection relationships, and node edge weights to obtain a load topology graph.

[0045] The methods for multi-dimensional aggregation of load data based on the load topology diagram include:

[0046] Using load data as the initial feature of each node, a load adjacency matrix and a load weight matrix are constructed based on the load topology graph. The load adjacency matrix is ​​a matrix that describes the connection relationship between graph nodes. When there is a direct edge between graph nodes, the element value in the load adjacency matrix is ​​1, and when there is no direct edge between graph nodes, the element value in the load adjacency matrix is ​​0; the load weight matrix is ​​a left diagonal matrix that describes the sum of the edge weights of all nodes that have direct edges with graph nodes; the initial result set is empty, and the weighting ratio is preset to measure the performance of each feature in the initial feature, based on The initial features are traversed at a weighted ratio, and the traversed features are recorded as features to be processed. The features to be processed of each node are amplified by the weighted ratio (the weighted ratio is multiplied by the value of the feature to be processed) to obtain the feature amplification value; the value of the feature to be processed in each initial feature is replaced by the feature amplification value to obtain the extended feature; the extended features are clustered using a clustering analysis algorithm to obtain feature clustering clusters. Commonly used clustering analysis algorithms include K-Means clustering algorithm and coagulation layer clustering algorithm; feature entropy is calculated for the feature clustering clusters. The formula for calculating feature entropy is:

[0047] ;in, represents the feature entropy of the feature to be processed, Represents the number of clusters of feature clusters, Representative The probability density of feature clusters; add the feature entropy to the feature weight set, repeat until the traversal is completed, and the result set output at this time is the feature weight set; perform feature convolution on the initial features based on the feature weight set, and the formula for feature convolution is:

[0048] ;in Represents the initial feature Convolution features after layer convolution, represents the feature weight set, represents the characteristic entropy, represents the load weight matrix, represents the load adjacency matrix, Initial feature pass Convolution features after layer convolution, Representative The weight matrix of the layer, Represents a nonlinear activation function. Common nonlinear activation functions include the ReLU function. The initial features are The convolution features after layer convolution are the node load features. Multi-dimensional aggregation of load data can capture the complex interaction relationships between nodes, provide a comprehensive state representation for the follow-up, and greatly improve the system's responsiveness and global optimization effects.

[0049] Ways to prioritize task data include:

[0050] Priority criteria are constructed based on task data. Priority criteria include: the larger the data volume, the higher the priority; the higher the computational complexity, the higher the priority; preset a criterion selection domain, compare the priority criteria in pairs, and use a random selection algorithm to select values ​​within the criterion selection domain as element values ​​to construct an initial criterion matrix. Based on the random selection algorithm in the priority criterion selection domain, the task score is effectively weighted under multiple criteria, ensuring the rationality and fairness of the task scheduling process; the elements in the initial criterion matrix represent the relative importance of different criteria; the criterion index is initialized to be infinitesimal, the initial evaluation weight set is initialized to be empty, the consistency of the initial criterion matrix is ​​verified, the sum of each column in the initial criterion matrix is ​​calculated to obtain the column sum, the elements of each column in the initial criterion matrix are divided by the column sum of the corresponding column to obtain the average criterion matrix, and the elements of each row of the average criterion matrix are averaged to obtain the criterion feature vector; the criterion feature vector is multiplied by the initial criterion matrix to obtain the weight matrix, each element in the weight matrix is ​​divided by the element in the corresponding position in the criterion feature vector to obtain the approximate feature vector, the element with the largest value in the approximate feature vector is selected as the maximum feature, and the indicator calculation is performed based on the maximum feature. The formula for indicator calculation is: ;in, Represents the consistency index, Represents the largest feature, The number of categories representing the priority criteria. When the consistency index is greater than the criterion index, the consistency index at this time is used as the new criterion index, and the criterion feature vector at this time is used as the new evaluation weight set. Repeat until the value of the criterion index no longer changes, and output the evaluation weight set at this time as the task rating index. Task evaluation is performed on the task data based on the task rating index. The formula for task evaluation is: ;in, Represents the task score, Represents the first row of elements in the task rating index, Data volume, Represents the second row of elements in the task rating index, Represents computational complexity; task scoring takes into account two key factors: data volume and computational complexity, and obtains a reasonable set of evaluation weights through consistency verification and matrix operations. This evaluation mechanism not only provides a scientific basis for task allocation, but also improves the transparency and accuracy of the decision-making process;

[0051] The preferred criterion selection domain of this embodiment is in the range of [2,9], assuming that the initial criterion matrix is: ,in, The amount of data represents the importance of computational complexity , represents that the computational complexity is twice the importance of the data volume; then the criterion eigenvector is: ,The task scoring based on the criterion feature vector calculation will take into account the importance of both data volume and computational complexity, and improve the transparency and scientificity of subsequent decision-making.

[0052] Ways to optimally match node load characteristics to tasks include:

[0053] Sort the task data from large to small according to the task score to obtain the task sequence, use the task sequence as the distribution order of the task data, use the random matching algorithm to pair the task data with the node load characteristics, and obtain M task matching schemes, where M is an integer greater than zero, and each pairing satisfies the task constraint condition: the data volume of the task data is less than the task processing volume in the node load characteristics;

[0054] Each task matching scheme is regarded as a chromosome, and the genes in the chromosome correspond to the pairing status of the task data and the node load characteristics; for each chromosome, the fitness of each chromosome is calculated according to the fitness calculation formula, which is:

[0055] ;in, represents fitness, represents the number of genes in the chromosome (the total number of node load features), Representative The total task score of the task data matched by the node load characteristics, Representative The total amount of task data matched by the node load characteristics, Representative The task processing capacity of the node load characteristics, Representative The computational complexity of task data matching the node load characteristics, Representative The disk read and write rate of the node load characteristics; initialize the optimal chromosome to be empty, initialize the optimal fitness to be infinitesimal, preset the trophy value, randomly select chromosomes with the trophy value based on the trophy value, compare the fitness of the selected chromosomes, and take the chromosome with the largest fitness value as the parent chromosome. When the number of parent chromosomes is consistent with the trophy value, stop, and all parent chromosomes constitute the parent chromosome set;

[0056] The permutation and combination algorithm is used to pair the chromosomes in the parent chromosome set in pairs to obtain the parent combination; a gene in the parent combination is randomly selected as an exchange node, and the genes after the exchange node in the parent combination are exchanged to obtain the offspring chromosome. The fitness of each offspring chromosome is calculated based on the fitness calculation formula, and the offspring chromosome with a fitness greater than or equal to the parent combination is taken as the preferred offspring; the permutation and combination algorithm is used to pair the preferred offspring in pairs to obtain the second-generation combination, and a gene in the second-generation combination is randomly selected as a mutation node, and the task data of the mutation node in the second-generation combination is replaced to obtain the mutation chromosome, and the fitness calculation formula is used to calculate each mutation chromosome The fitness of the second generation combination is selected as the preferred variant, and the preferred variant with the largest fitness is selected as the candidate individual. When the fitness of the candidate individual is greater than the optimal fitness, the candidate individual is used as the new optimal chromosome, and the fitness of the candidate individual is used as the new optimal fitness. The preferred variant is used as the new parent chromosome, and the process is repeated until the value of the optimal fitness no longer changes. The optimal chromosome at this time is output as the optimal allocation plan. Through the optimal allocation plan, those nodes with uneven loads or idle resources can be adjusted to avoid centralized processing of tasks or waste of resources, thereby accelerating the overall task processing speed;

[0057] Collect historical node data and corresponding load evaluations. The historical node data includes: historical node load data and historical task allocation data. The data type of the historical node load data is consistent with the data type of the load data, and the data type of the historical task allocation data is consistent with the data type of the task data. Use the historical node data and load evaluation as training sample sets, use the linear regression model as the initial model, use the training sample set to train the linear regression model, use the historical node data as input data of the load prediction model, and use the predicted load evaluation as output data of the load prediction model. Use minimization of the error between the actual load evaluation and the predicted load evaluation as the training goal, use the mean square error function as the loss function of the load prediction model, and when the loss function reaches convergence, stop training to obtain the load prediction model. The load evaluation corresponding to the historical scenario data is obtained based on expert evaluation.

[0058] Based on the constructed load prediction model, the optimal allocation plan is predicted one by one to obtain the load evaluation value of each task allocation, and the overload warning threshold is preset. The task allocation with a load evaluation value greater than or equal to the overload warning threshold is used as the overload allocation. The overload allocation is the task matching when the server node processes the task and the server node is overloaded. When the server node runs under overload conditions, it will reduce the server life and even cause service downtime.

[0059] The methods of local task migration for overload distribution include:

[0060] Based on the load topology graph, the graph nodes with directly connected edges in the load topology graph corresponding to each overload allocation are used as cutting edges, and the load topology graph is cut with the cutting edges to obtain the overload local graph. The task data assigned to all graph nodes in the overload local graph are counted to obtain the local tasks; B ants are initialized, and the value of B is consistent with the data volume of the local task. Local tasks are assigned to each ant, and each ant is randomly assigned to different graph nodes in the overload local graph. The pheromone absorption rate of each ant is initialized. The pheromone absorption rate is a minimum value to ensure that the number of iterations is sufficient. The node edge weight of each edge in the overload local graph is calculated as the ratio of the sum of all node edge weights to obtain the initial concentration value, and the initial pheromone concentration is assigned to each edge in the overload local graph based on the initial concentration value;

[0061] Each ant moves in the overloaded local graph, and selects the next position to which the ant will move based on the initial pheromone concentration of each edge. The ant moves towards the edge with the highest initial pheromone concentration. When the graph node information of each ant changes, the pheromone concentration of the edge passed by each ant is reduced based on the pheromone absorption rate. The ant information on each graph node is counted at fixed time intervals, and each graph node in the overloaded local graph is predicted based on the constructed load prediction model to obtain the overload assessment of each graph node. The graph nodes with overload assessment greater than or equal to the overload warning threshold will drive the ants with the lowest task scores to migrate the graph nodes. When all ants stop moving, the local tasks represented by the ants on each graph node in the overloaded local graph are the local optimization solutions.

[0062] This embodiment can accurately reflect the interdependence and complex relationships between nodes by constructing a load topology map, and use the load topology map to perform multi-dimensional aggregation of load data, providing a comprehensive state representation for subsequent task allocation, greatly improving the system's responsiveness, and ensuring the global optimization effect of load balancing; by performing priority evaluation on task data, it not only provides a scientific basis for task allocation, but also improves the transparency and accuracy of the decision-making process, ensuring the rationality and fairness of the task scheduling process; by performing task allocation based on the preferred genetic algorithm, it optimizes the task allocation order and matching process, ensuring that each task can be reasonably allocated to a node with a lower load according to its computing requirements, effectively avoiding excessive concentration or idleness of node resources, thereby improving the overall processing efficiency of the cluster; through local task migration of overload allocation, the performance degradation caused by overload of a single node is avoided, and the distribution of tasks between nodes is flexibly adjusted to further optimize the resource utilization and processing capacity of the server cluster.

[0063] Example 2

[0064] See also Figure 2As shown, the part not described in detail in this embodiment is described in Example 1, and a server cluster load balancing system based on edge computing is provided, including:

[0065] Data collection module: collects task data, transmission data between nodes and load data of server nodes;

[0066] Data processing module: constructs a load topology map based on load data and transmission data, performs multi-dimensional aggregation on load data based on the load topology map, and obtains a description of node status;

[0067] Task evaluation module: Uses analysis algorithms to evaluate the priority of task data and obtain task scores;

[0068] Task allocation module: It includes a global allocation unit and a local optimization unit. The global allocation unit performs the best task matching on the node status description based on the task score to obtain the best allocation plan. The best allocation plan is predicted based on the built load prediction model to obtain the overload allocation. The local optimization unit performs local task migration on the overload allocation to obtain the local optimization plan.

[0069] The modules are connected via wired and / or wireless means to achieve data transmission between the modules.

[0070] Example 3

[0071] This embodiment discloses an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the operation mode of the server cluster load balancing method based on edge computing provided above is implemented.

[0072] Since the electronic device introduced in this embodiment is an electronic device used to implement a server cluster load balancing method based on edge computing in the embodiment of the present application, based on the server cluster load balancing method based on edge computing introduced in the embodiment of the present application, the technical personnel of this field can understand the specific implementation of the electronic device of this embodiment and its various variations, so how the electronic device implements the method in the embodiment of the present application is not introduced in detail here. As long as the technical personnel of this field implement the electronic device used in the server cluster load balancing method based on edge computing in the embodiment of the present application, it belongs to the scope of protection of this application.

[0073] The above formulas are all dimensionless and numerical calculations. The formula is a formula for the most recent real situation obtained by collecting a large amount of data and performing software simulation. The preset parameters and thresholds in the formula are set by technicians in this field according to actual conditions.

[0074] The above is only a preferred embodiment of the present invention, and the protection scope of the present invention is not limited to the above embodiments. All technical solutions under the concept of the present invention belong to the protection scope of the present invention. It should be pointed out that for ordinary technical users in this technical field, some improvements and modifications without departing from the principle of the present invention should also be regarded as the protection scope of the present invention.

Claims

1. A server cluster load balancing method based on edge computing, characterized in that: include: S1, collects task data, transmission data between nodes and load data of server nodes; S2. construct a load topology map based on the load data and the transmission data, and perform multi-dimensional aggregation on the load data based on the load topology map to obtain a node status description; S3, use analytical algorithms to evaluate the priority of task data and obtain task scores; S4. Perform the most suitable task matching on the node status description based on the task score to obtain the most suitable allocation plan. Predict the most suitable allocation plan based on the constructed load prediction model to obtain the overload allocation. Perform local task migration on the overload allocation to obtain the local optimization plan. The method of constructing a load topology diagram based on load data and transmission data includes: Each load data is used as a graph node, the transmission data represents the connection relationship between graph nodes, and the edge weights are initialized with network bandwidth and latency. The formula for weight initialization is: Among them, Wei ij represents the node edge weight between the i-th graph node and the j-th graph node, Lat ij represents the delay between the i-th graph node and the j-th graph node, Baw ij represents the network bandwidth between the i-th graph node and the j-th graph node, α represents the delay weight, and β represents the network bandwidth weight. The graph is constructed based on the graph nodes, connection relationships, and node edge weights to obtain the load topology graph; The method of multi-dimensionally aggregating load data based on the load topology diagram includes: Take the load data as the initial feature of each node, and construct the load adjacency matrix and load weight matrix based on the load topology graph. The load adjacency matrix is ​​a matrix that describes the connection relationship between the graph nodes, and the load weight matrix is ​​a left diagonal matrix that describes the sum of the edge weights of all nodes that have direct edges connected to the graph nodes; initialize the result set to be empty, preset the weighting ratio, perform feature traversal on the initial features based on the weighting ratio, record the traversed features as the features to be processed, and amplify the features of the features to be processed of each node by the weighting ratio to obtain the feature amplification value; replace the value of the feature to be processed in each initial feature with the feature amplification value to obtain the extended feature; cluster the extended features using a clustering analysis algorithm to obtain feature clustering clusters, calculate the feature entropy of the feature clustering clusters, obtain the feature entropy of the feature to be processed, add the feature entropy to the feature weight set, repeat until the traversal is completed, and output the result set at this time as the feature weight set; perform feature convolution on the initial features based on the feature weight set to obtain a description of the node state; The method of evaluating the priority of task data includes: Priority criteria are constructed based on task data. The priority criteria include: the larger the data volume, the higher the priority; the higher the computational complexity, the higher the priority; a criterion selection domain is preset, and the priority criteria are compared in pairs. A random selection algorithm is used to select values ​​within the criterion selection domain as element values ​​to construct an initial criterion matrix. The elements in the initial criterion matrix represent the relative importance of different criteria; the criterion index is initialized to be infinitesimal, the evaluation weight set is initialized to be empty, the initial criterion matrix is ​​verified for consistency, the sum of each column in the initial criterion matrix is ​​calculated to obtain the column sum, each column element in the initial criterion matrix is ​​divided by the column sum of the corresponding column to obtain an average criterion matrix, and each row element of the average criterion matrix is ​​averaged to obtain the criteria. feature vector; multiply the criterion feature vector by the initial criterion matrix to obtain a weight matrix, divide each element in the weight matrix by the element at the corresponding position in the criterion feature vector to obtain an approximate feature vector, select the element with the largest value in the approximate feature vector as the maximum feature, calculate the index based on the maximum feature to obtain a consistency index, when the consistency index is greater than the criterion index, use the consistency index at this time as the new criterion index, use the criterion feature vector at this time as the new evaluation weight set, repeat until the value of the criterion index no longer changes, and output the evaluation weight set at this time as the task rating index; perform task evaluation on the task data based on the task rating index to obtain a task score.

2. The server cluster load balancing method based on edge computing according to claim 1 is characterized in that: The load data includes: CPU utilization, memory usage, task processing volume and disk read and write rate; the task data includes: data volume and computational complexity; the transmission data includes: network bandwidth and latency.

3. The server cluster load balancing method based on edge computing according to claim 2 is characterized in that: The formula for performing feature convolution on the initial features is: Among them, Char L+1 represents the convolution feature after the initial feature passes through L layers of convolution, N represents the feature weight set, ρ represents the feature entropy, D represents the load weight matrix, A represents the load adjacency matrix, Char L The convolutional features after the initial features are convolved by L-1 layers, W L represents the weight matrix of the Lth layer, and σ represents the nonlinear activation function.

4. The server cluster load balancing method based on edge computing according to claim 3 is characterized in that: The method of performing the most suitable task matching on the node status description includes: Sort the task data from large to small according to the task score to obtain the task sequence. Use the task sequence as the distribution order of the task data. Use the random matching algorithm to pair the task data with the node state description to obtain M task matching schemes. Each pairing satisfies the task constraint condition: the data volume of the task data is less than the task processing volume in the node state description. Each task matching scheme is regarded as a chromosome, and the genes in the chromosome correspond to the pairing status described by the task data and the node status; for each chromosome, the fitness of each chromosome is calculated according to the fitness calculation formula, the optimal chromosome is initialized to be empty, the optimal fitness is initialized to be infinitesimal, the trophy value is preset, and the number of chromosomes equal to the trophy value is randomly selected based on the trophy value, and the fitness of the selected chromosomes is compared, and the chromosome with the largest fitness value is taken as the parent chromosome. When the number of parent chromosomes is consistent with the trophy value, it stops, and all parent chromosomes constitute the parent chromosome set; The permutation and combination algorithm is used to pair the chromosomes in the parent chromosome set in pairs to obtain the parent combination; a gene in the parent combination is randomly selected as an exchange node, the gene after the exchange node in the parent combination is exchanged to obtain the daughter chromosome, the fitness of each daughter chromosome is calculated based on the fitness calculation formula, and the daughter chromosome with a fitness greater than or equal to the parent combination is taken as the preferred daughter; the permutation and combination algorithm is used to pair the preferred daughters in pairs to obtain the second-generation combination, a gene in the second-generation combination is randomly selected as a mutation node, the task data of the mutation node in the second-generation combination is replaced to obtain the mutant chromosome, the fitness of each mutant chromosome is calculated based on the fitness calculation formula, the mutant chromosome with a fitness greater than or equal to the second-generation combination is taken as the preferred variant, and the preferred variant with the largest fitness is selected as the candidate individual. When the fitness of the candidate individual is greater than the optimal fitness, the candidate individual is taken as the new optimal chromosome, the fitness of the candidate individual is taken as the new optimal fitness, and the preferred variant is taken as the new parent chromosome, and repeated until the value of the optimal fitness no longer changes. The optimal chromosome output at this time is the most suitable allocation scheme.

5. The server cluster load balancing method based on edge computing according to claim 4 is characterized in that: The calculation formula of the fitness is: Among them, Fix represents fitness, Max represents the number of genes in the chromosome, the total number of node state descriptions, and Grade b Represents the total task score of the task data matched by the b-th node state description, Tas b Represents the total amount of task data matched by the b-th node state description, Tma b Represents the task processing amount described by the b-th node state, read b represents the computational complexity of the task data matched by the b-th node state description, cal b Represents the disk read and write rate described by the b-th node status.

6. The server cluster load balancing method based on edge computing according to claim 5 is characterized in that: The load prediction model is constructed in the following manner: Collect historical node data and corresponding load evaluations. The historical node data includes: historical node load data and historical task allocation data. The data type of the historical node load data is consistent with the data type of the load data, and the data type of the historical task allocation data is consistent with the data type of the task data. Use the historical node data and load evaluation as training sample sets, use the linear regression model as the initial model, use the training sample set to train the linear regression model, use the historical node data as input data of the load prediction model, and use the predicted load evaluation as output data of the load prediction model. Use minimizing the error between the actual load evaluation and the predicted load evaluation as the training goal, use the mean square error function as the loss function of the load prediction model, and stop training when the loss function converges to obtain the load prediction model.

7. The server cluster load balancing method based on edge computing according to claim 6 is characterized in that: The method of performing local task migration for overload distribution includes: Based on the load topology graph, the graph nodes with directly connected edges in the load topology graph corresponding to each overload allocation are used as cutting edges, and the load topology graph is cut with the cutting edges to obtain the overload local graph, and the task data assigned to all graph nodes in the overload local graph are counted to obtain the local tasks; B ants are initialized, and the value of B is consistent with the data volume of the local task, and a local task is assigned to each ant. Each ant is randomly assigned to a different graph node in the overload local graph, and the pheromone absorption rate of each ant is initialized. The ratio of the node edge weight of each edge in the overload local graph to the sum of all node edge weights is calculated to obtain the initial concentration value, and the initial pheromone concentration is assigned to each edge in the overload local graph based on the initial concentration value; Each ant moves in the overloaded local graph, and selects the next position to which the ant will move based on the initial pheromone concentration of each edge. The ant moves towards the edge with the highest initial pheromone concentration. When the graph node information of each ant changes, the pheromone concentration of the edge passed by each ant is reduced based on the pheromone absorption rate. The ant information on each graph node is counted at fixed time intervals, and each graph node in the overloaded local graph is predicted based on the constructed load prediction model to obtain the overload assessment of each graph node. The graph nodes with overload assessment greater than or equal to the overload warning threshold will drive the ants with the lowest task scores to migrate the graph nodes. When all ants stop moving, the local tasks represented by the ants on each graph node in the overloaded local graph are the local optimization solutions.

Images