Adaptive load balancing method and system based on multiple models

Abstract

The present application relates to the technical field of load balancing, and discloses a multi-model-based adaptive load balancing method and system. The method includes: obtaining a multi-dimensional resource information data set of multiple server nodes and performing implicit fuzzy K-means clustering to obtain a fuzzy partitioning result; generating a target similarity matrix based on the fuzzy partitioning result and constructing an effective load topology map; performing historical load records and network topology analysis to generate dynamic load prediction results; modeling multiple server nodes in the effective load topology map as a multi-agent cooperative game, obtaining the cooperation strategy of each server node and establishing a multi-model load balancing mechanism; performing load distribution calculation on the dynamic load prediction result based on the multi-model load balancing mechanism to obtain an initial load distribution strategy; performing convex optimization processing on the initial load distribution strategy to obtain an optimal load distribution plan. The present application improves the accuracy of adaptive load balancing.

Description

Technical Field

[0001] The present application relates to the technical field of load balancing, and in particular to a multi-model-based adaptive load balancing method and system. Background Art

[0002] In today's highly complex and dynamically changing network environment, server cluster load balancing has become one of the key technologies. Traditional load balancing methods mainly rely on static rules and single models, and often show limitations when facing multi-dimensional resource demand changes and uneven load distribution. With the development of technologies such as cloud computing, big data, and artificial intelligence, the resource utilization of server nodes has become more diverse and unpredictable, making it difficult for traditional methods to effectively deal with complex load distribution problems.

[0003] Existing load balancing methods usually ignore the interdependence between server nodes and the influence of network topology. They only rely on a single resource utilization indicator to distribute load, which can easily lead to overload of some nodes or waste of resources. In addition, many methods lack the ability to dynamically predict load change trends and cannot adjust load distribution strategies in time to adapt to load fluctuations. This static allocation mechanism that lacks predictability is often slow to respond to sudden traffic and resource bottlenecks, resulting in reduced service quality and poor user experience. Summary of the invention

[0004] The present application provides a multi-model based adaptive load balancing method and system for improving the accuracy of adaptive load balancing.

[0005] In a first aspect, the present application provides an adaptive load balancing method based on multiple models, and the adaptive load balancing method based on multiple models includes:

[0006] Acquire a multidimensional resource information data set of multiple server nodes, and perform implicit fuzzy K-means clustering on the multidimensional resource information data set to obtain a fuzzy partition result of each server node;

[0007] Generating a target similarity matrix for each server node according to the fuzzy partitioning result, and constructing a valid load topology graph of the plurality of server nodes according to the target similarity matrix;

[0008] Performing historical load records and network topology analysis on the effective load topology diagram to generate dynamic load prediction results;

[0009] Modeling multiple server nodes in the effective load topology graph as a multi-agent cooperative game, obtaining a cooperation strategy for each server node, and establishing a multi-model load balancing mechanism according to the cooperation strategy;

[0010] Perform load distribution calculation on the dynamic load prediction result based on the multi-model load balancing mechanism to obtain an initial load distribution strategy;

[0011] The initial load distribution strategy is subjected to convex optimization processing to obtain an optimal load distribution solution for the multiple server nodes.

[0012] In a second aspect, the present application provides an adaptive load balancing system based on multiple models, and the adaptive load balancing system based on multiple models includes:

[0013] An acquisition module, used to acquire a multidimensional resource information data set of multiple server nodes, and perform implicit fuzzy K-means clustering on the multidimensional resource information data set to obtain a fuzzy partition result of each server node;

[0014] A construction module, used to generate a target similarity matrix for each server node according to the fuzzy partitioning result, and to construct a valid load topology map of the plurality of server nodes according to the target similarity matrix;

[0015] An analysis module, used for performing historical load recording and network topology analysis on the effective load topology diagram to generate a dynamic load prediction result;

[0016] A modeling module, used for modeling multiple server nodes in the effective load topology diagram as a multi-agent cooperative game, obtaining a cooperation strategy for each server node, and establishing a multi-model load balancing mechanism according to the cooperation strategy;

[0017] A calculation module, used for performing load distribution calculation on the dynamic load prediction result based on the multi-model load balancing mechanism to obtain an initial load distribution strategy;

[0018] The processing module is used to perform convex optimization processing on the initial load distribution strategy to obtain an optimal load distribution solution for the multiple server nodes.

[0019] The third aspect of the present application provides a multi-model-based adaptive load balancing device, comprising: a memory and at least one processor, wherein the memory stores instructions; the at least one processor calls the instructions in the memory so that the multi-model-based adaptive load balancing device executes the above-mentioned multi-model-based adaptive load balancing method.

[0020] A fourth aspect of the present application provides a computer-readable storage medium, in which instructions are stored. When the computer-readable storage medium is run on a computer, the computer executes the above-mentioned multi-model-based adaptive load balancing method.

[0021] In the technical solution provided by the present application, by obtaining the multi-dimensional resource information of multiple server nodes, implicit fuzzy K-means clustering is performed, which effectively reflects the resource usage of the server nodes. The iterative update of the membership matrix and the cluster center makes the clustering results more accurate, can dynamically adjust the classification of the server nodes, and improves the accuracy of load balancing. By calculating the comprehensive membership vector and cosine similarity of the server nodes, generating the target similarity matrix, and constructing the effective load topology map, the similarity and dependency between the server nodes are accurately reflected. The load topology is optimized using the community discovery algorithm and the minimum spanning tree algorithm to ensure the minimum cost of the load distribution path, improve the load transmission efficiency and overall network performance. The dynamic load prediction is performed using historical load data and network topology features, and the future load changes are modeled through the gated cyclic unit network, and the prediction results are more accurate and timely. Based on the multi-agent cooperative game model and the deep reinforcement learning method, the load distribution strategy is dynamically adjusted, so that the load balancing mechanism can adapt to the changing load demand, and improve the adaptability and robustness of the system. The alternating direction multiplier method and the interior point method are used for load distribution optimization, which solves the non-convex constraint problem and ensures the global optimality of the load distribution scheme. Through load sensitivity evaluation and robustness optimization, the impact of load changes on performance is considered to ensure the fairness and stability of the load distribution strategy under different load conditions. The multi-model adaptive load balancing method comprehensively considers the multi-dimensional resource usage of server nodes, historical load data and network topology characteristics, and can maintain efficient resource utilization in a complex and changing environment. It improves the system's response speed and service quality, reduces resource waste and overload, and enhances user experience and service stability. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0023] Figure 1 A schematic diagram of an embodiment of an adaptive load balancing method based on multiple models in an embodiment of the present application;

[0024] Figure 2 A schematic diagram of an embodiment of an adaptive load balancing system based on multiple models in an embodiment of the present application. DETAILED DESCRIPTION

[0025] The embodiment of the present application provides a method and system for adaptive load balancing based on multiple models. The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments described here can be implemented in an order other than the content illustrated or described here. In addition, the terms "including" or "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0026] For ease of understanding, the specific process of the embodiment of the present application is described below. Figure 1 , an embodiment of the adaptive load balancing method based on multiple models in the embodiment of the present application includes:

[0027] Step S101, obtaining a multidimensional resource information data set of multiple server nodes, and performing implicit fuzzy K-means clustering on the multidimensional resource information data set to obtain a fuzzy partition result of each server node;

[0028] It is understandable that the execution subject of the present application may be a multi-model based adaptive load balancing system, or a terminal or a server, which is not specifically limited here. The present application embodiment is described by taking a server as the execution subject as an example.

[0029] Specifically, the CPU utilization, memory utilization, I / O utilization and network bandwidth utilization of multiple server nodes are obtained to construct a multidimensional resource information data set. By performing feature scaling processing on the multidimensional resource information data set of each server node, the data of different dimensions are compared on the same scale to obtain a normalized multidimensional resource information data set. According to the normalized multidimensional resource information data set, the clustering center and membership matrix of the implicit fuzzy K-means clustering algorithm are initialized, the initial clustering center is selected, and the membership of each data point to the clustering center is randomly assigned to establish an initial membership matrix. According to the clustering center and the membership matrix, the Euclidean distance between each server node and each clustering center is calculated. This distance metric is used to evaluate the similarity of each server node relative to the clustering center, and the membership matrix is ​​updated according to the distance, so that the membership matrix reflects the fuzzy membership relationship of the server node. Through this iterative calculation, the new clustering center is recalculated each time according to the updated membership matrix. These new clustering centers are the center points that best represent each class under the current membership matrix. This process is repeated until the cluster center and the membership matrix converge, that is, after multiple iterations, the location of the cluster center and the value of the membership matrix no longer change significantly, and the fuzzy partition result of each server node is obtained. Based on the implicit fuzzy K-means clustering method, by repeatedly updating the cluster center and the membership matrix, the multi-dimensional resource information of the server node is effectively clustered, and finally an accurate and dynamic server node fuzzy partition result is obtained.

[0030] Step S102: generating a target similarity matrix for each server node according to the fuzzy partitioning result, and constructing a valid load topology map of multiple server nodes according to the target similarity matrix;

[0031] Specifically, according to the fuzzy partitioning results, the weighted average of the membership of each server node in each cluster is calculated to obtain the comprehensive membership vector of each server node. Through calculation, the fuzzy membership of the server nodes in different clusters is integrated to form a comprehensive membership representation, so as to more accurately reflect the load characteristics of each server node. According to the comprehensive membership vector, the cosine similarity between any two server nodes is calculated to obtain the server node similarity matrix. The server node similarity matrix is ​​threshold filtered to obtain a binary target similarity matrix. By setting a similarity threshold, the similarity values ​​above the threshold are retained as 1, and the rest are set to 0, so as to obtain a binary similarity matrix and simplify the representation of similarity. According to the target similarity matrix, a similarity weighted undirected graph between server nodes is constructed to obtain the initial load topology graph. The edge weights in the graph are determined by the similarity between the server nodes, reflecting the load correlation between the nodes. The initial load topology graph is processed by the community discovery algorithm to obtain the community division result of the server nodes. By identifying the community structure in the initial topology graph, the server nodes with high similarity are divided into the same community, which is convenient for subsequent load balancing operations. After obtaining the community division results, the load balancing capability of the server nodes in each community is evaluated according to the community division results to obtain the community load balancing capability vector. This vector reflects the load balancing capability of each node in each community. According to the community load balancing capability vector, the community in the initial load topology map is selected for load balancing nodes to obtain a community load balancing node set. By selecting nodes with strong load balancing capabilities in the community, the load balancing effect of the entire community can be improved. On this basis, according to the community load balancing node set, the initial load topology map is updated with edge weights to obtain an inter-community load balancing topology map. The updated topology map more accurately reflects the load balancing relationship between communities. The inter-community load balancing topology map is processed by the minimum spanning tree algorithm to obtain a minimum cost load balancing path set. The inter-community load balancing path is optimized by minimizing the sum of edge weights. According to the minimum cost load balancing path set, an effective load topology map covering all server nodes is constructed. This map ensures that all server nodes can efficiently perform load balancing, thereby optimizing the overall performance of the system.

[0032] Step S103: Perform historical load record and network topology analysis on the effective load topology map to generate dynamic load prediction results;

[0033] Specifically, according to the effective load topology map, the historical load data of multiple server nodes are obtained and a historical load time series data set is constructed. By collecting the load data of each server node at different times, a time series data set reflecting the load change trend is formed. The historical load time series data set is subjected to feature extraction processing to obtain a set of historical load feature vectors. By analyzing the characteristics of the time series data, such as seasonality, trend and periodicity, a set of feature vectors representing the historical load situation is extracted, and a historical load prediction model is created based on the set of historical load feature vectors. According to the effective load topology map, the network delay and bandwidth data between the server nodes are obtained and a network topology feature matrix is ​​constructed to obtain the network connection characteristics between the server nodes. The network topology feature matrix is ​​subjected to spectral clustering analysis to obtain the network topology community structure of the server nodes. Spectral clustering analysis is a method for clustering through the Laplace matrix of a graph, which can effectively identify the community structure in the network, and calculate the centrality index of the server nodes in each community according to the network topology community structure to obtain the community centrality vector. The centrality index can reflect the importance of each node in the network. According to the community centrality vector, the load transfer impact of the server nodes in each community is evaluated to obtain the community load transfer impact matrix. By analyzing the centrality and connection relationship of each node, the influence of load transfer between nodes is evaluated to form the community load transfer impact matrix. The historical load prediction model and the community load transfer impact matrix are input into the gated recurrent unit network for dynamic load evolution modeling to obtain a dynamic load prediction model. The gated recurrent unit network is a deep learning model for processing time series data, which can effectively capture the time dependence and dynamic evolution characteristics of load data. According to the dynamic load prediction model, the load change trend of each server node in the future is predicted to obtain the load change trend prediction result. According to the load change trend prediction result and the community load transfer impact matrix, a dynamic load prediction result covering all server nodes is generated.

[0034] Step S104: Model multiple server nodes in the effective load topology diagram as a multi-agent cooperative game, obtain the cooperation strategy of each server node, and establish a multi-model load balancing mechanism according to the cooperation strategy;

[0035] Specifically, according to the effective load topology, each server node is modeled as an agent to construct a multi-agent system model. In this model, each server node is regarded as an agent that can independently process the load and cooperate with other agents. The load processing capacity and resource utilization of each agent are quantitatively evaluated to obtain the agent state feature vector. These feature vectors include key indicators such as CPU utilization, memory utilization, I / O utilization and network bandwidth utilization of each agent, which fully reflect the operation status and resource usage of the agent. According to the agent state feature vector, the cooperative game matrix between agents is constructed, and the game profit function is defined. The cooperative game matrix reflects the cooperative relationship and possible cooperative benefits between agents, and the game profit function is used to quantify the benefits of agents under different cooperative strategies. The Nash equilibrium of the cooperative game matrix is ​​solved by the gradient descent algorithm with adaptive learning rate to obtain the cooperative strategy of each agent. Through this optimization method, a stable cooperative strategy is found to maximize the benefits of each agent in the game while maintaining the overall equilibrium of the system. After obtaining the cooperation strategies of each agent, a load balancing cooperation mechanism between agents is constructed according to the cooperation strategies, and deep reinforcement learning training is performed to obtain the load balancing strategy evaluation network and the load balancing strategy generation network. Through a large number of simulations and actual operations, deep reinforcement learning training gradually optimizes the load balancing strategy of the agent, so that it can effectively distribute and adjust the load under different load conditions. The load balancing strategy evaluation network is used to evaluate the effect of the current load balancing strategy, while the load balancing strategy generation network is used to generate new load balancing strategies. The load balancing strategy evaluation network and the load balancing strategy generation network are combined into an Actor-Critic architecture to construct a load balancing agent model. In the Actor-Critic architecture, the Actor is responsible for selecting and executing the load balancing strategy, while the Critic is responsible for evaluating the effect of the selected strategy and guiding the Actor to optimize the strategy. In this way, the load balancing agent model can continuously learn and improve its load balancing strategy, achieve efficient load management and distribution, and ensure that the system can maintain efficient and stable operation under various load conditions.

[0036] Step S105: performing load distribution calculation on the dynamic load prediction result based on the multi-model load balancing mechanism to obtain an initial load distribution strategy;

[0037] Specifically, according to the dynamic load prediction results, the initial load demand matrix and the initial load supply matrix of the server nodes are constructed. By analyzing the load prediction data of each server node, the load demand and supply of each node in the future period are determined to form the initial load demand and supply matrix. The initial load demand matrix and the initial load supply matrix are standardized to obtain the standardized load demand matrix and the standardized load supply matrix. According to the standardized load demand matrix and the standardized load supply matrix, a load distribution optimization model is constructed, and the load distribution utility function is defined. The load distribution optimization model is used to determine how to distribute the load among the server nodes to maximize the performance and resource utilization of the overall system, while the load distribution utility function is used to quantify the effects of different load distribution strategies. The alternating direction multiplier method is used to solve the load distribution optimization model to obtain the initial load distribution matrix. The alternating direction multiplier method is a commonly used optimization algorithm that decomposes complex optimization problems into multiple sub-problems and solves them step by step through iteration to find the global optimal solution. According to the load balancing agent model in the multi-model load balancing mechanism, the initial load distribution matrix is ​​subjected to game equilibrium analysis to obtain the game equilibrium load distribution matrix. Game equilibrium analysis determines a stable load distribution strategy by considering the mutual influence and cooperation relationship of each node to maximize the benefits of each node in the game. According to the load balancing strategy evaluation network in the multi-model load balancing mechanism, the game balancing load distribution matrix is ​​evaluated to obtain the load distribution strategy evaluation result. The load balancing strategy evaluation network is used to evaluate the effect of the current load distribution strategy and identify possible optimization space. According to the load distribution strategy evaluation result, the game balancing load distribution matrix is ​​improved to obtain the improved load distribution matrix. By combining the strategy evaluation results, the current load distribution strategy is adjusted and optimized to further improve the load balancing performance of the system. According to the improved load distribution matrix, the load distribution strategy between server nodes is generated to obtain the first load distribution strategy set. This strategy set reflects the optimal load distribution scheme between server nodes under the current load prediction and optimization conditions. The first load distribution strategy set is used as the input of the load balancing strategy generation network and a dynamic load distribution strategy is generated to obtain the final initial load distribution strategy. The load balancing strategy generation network generates a dynamically adjusted load distribution strategy based on the input strategy set through a deep learning model, thereby ensuring that it can respond to load changes in real time during actual operation and maintain efficient operation of the system.

[0038] Step S106: Perform convex optimization processing on the initial load distribution strategy to obtain an optimal load distribution solution for multiple server nodes.

[0039] Specifically, according to the initial load distribution strategy, the objective function and constraints of the load distribution optimization problem are constructed to obtain the load distribution optimization model. The model comprehensively describes the load distribution problem by defining the objectives of load distribution, such as minimizing response time or maximizing resource utilization, and a series of constraints, such as the maximum processing capacity of each server node and network bandwidth limit. The load distribution optimization model is subjected to convexity analysis, and non-convex constraints are converted into convex constraints to obtain a convex optimization load distribution model. Convexity analysis makes it easier to solve the optimization problem by converting non-convex constraints into convex constraints, ensuring the existence of the global optimal solution. The convex optimization load distribution model is solved by the interior point method to obtain the optimal solution vector of the load distribution strategy. The interior point method is an effective convex optimization algorithm that can quickly and accurately solve the load distribution optimization problem by gradually approaching the optimal solution in the feasible domain. According to the optimal solution vector, the optimal load distribution strategy between server nodes is generated to obtain the second load distribution strategy set. This strategy set is based on the initial load distribution strategy, and the optimal solution obtained after convex optimization processing can effectively improve the load balancing performance of the system. A sensitivity analysis is performed on the second load distribution strategy set to evaluate the impact of the load changes of each server node on the load distribution performance, and the load sensitivity evaluation results are obtained. Sensitivity analysis helps identify potential bottlenecks and risk points in load distribution by evaluating the impact of the load changes of each node on the overall system performance. According to the load sensitivity evaluation results, the second load distribution strategy set is robustly optimized to obtain a robust load distribution strategy set. Robust optimization adjusts the load distribution strategy so that it can remain stable and efficient in the face of load fluctuations and uncertainties, thereby improving the system's anti-interference ability. The robust load distribution strategy set is used as a constraint to construct a load distribution fairness optimization model and define load distribution fairness indicators. The fairness optimization model aims to ensure that the load is distributed more fairly among the server nodes to avoid the situation where some nodes are overloaded while other nodes are idle. The load distribution fairness optimization model is solved by a sequential convex programming algorithm to obtain the optimal load distribution strategy that takes into account both load distribution performance and fairness. The sequential convex programming algorithm decomposes the non-convex optimization problem into a series of convex optimization sub-problems, gradually solves them, and finally finds the global optimal solution.

[0040] In the embodiment of the present application, by obtaining the multidimensional resource information of multiple server nodes, implicit fuzzy K-means clustering is performed, which effectively reflects the resource usage of the server nodes. The iterative update of the membership matrix and the cluster center makes the clustering result more accurate, can dynamically adjust the classification of the server nodes, and improves the accuracy of load balancing. By calculating the comprehensive membership vector and cosine similarity of the server nodes, generating the target similarity matrix, and constructing the effective load topology map, the similarity and dependency between the server nodes are accurately reflected. The load topology is optimized using the community discovery algorithm and the minimum spanning tree algorithm to ensure the minimum cost of the load distribution path, improve the load transmission efficiency and overall network performance. The dynamic load prediction is performed using historical load data and network topology features, and the future load changes are modeled through the gated cyclic unit network, and the prediction results are more accurate and timely. Based on the multi-agent cooperative game model and the deep reinforcement learning method, the load distribution strategy is dynamically adjusted, so that the load balancing mechanism can adapt to the changing load demand, and improve the adaptability and robustness of the system. The alternating direction multiplier method and the interior point method are used for load distribution optimization, which solves the non-convex constraint problem and ensures the global optimality of the load distribution scheme. Through load sensitivity evaluation and robustness optimization, the impact of load changes on performance is considered to ensure the fairness and stability of the load distribution strategy under different load conditions. The multi-model adaptive load balancing method comprehensively considers the multi-dimensional resource usage of server nodes, historical load data and network topology characteristics, and can maintain efficient resource utilization in a complex and changing environment. It improves the system's response speed and service quality, reduces resource waste and overload, and enhances user experience and service stability.

[0041] In a specific embodiment, the process of executing step S101 may specifically include the following steps:

[0042] (1) Obtain the CPU utilization, memory utilization, I / O utilization, and network bandwidth utilization of multiple server nodes and construct a multi-dimensional resource information dataset;

[0043] (2) performing feature scaling processing on the multidimensional resource information dataset to obtain a normalized multidimensional resource information dataset;

[0044] (3) Initializing the cluster center and membership matrix of the implicit fuzzy K-means clustering algorithm based on the normalized multidimensional resource information data set;

[0045] (4) Based on the cluster centers and the membership matrix, calculate the Euclidean distance between each server node and each cluster center, and update the membership matrix;

[0046] (5) Based on the updated membership matrix, calculate the new cluster center until the cluster center and the membership matrix converge, and obtain the fuzzy partition result of each server node.

[0047] Specifically, the utilization data of four key resources are obtained from multiple server nodes, including CPU utilization, memory utilization, I / O utilization and network bandwidth utilization. By collecting these data, a multidimensional resource information data set is constructed, and each data point represents the resource usage of a server node. Assuming that data is collected from N server nodes, the data set can be represented as a matrix with N rows and 4 columns, where each row represents a server node and each column represents CPU utilization, memory utilization, I / O utilization and network bandwidth utilization. The multidimensional resource information data set is subjected to feature scaling processing, and each feature is scaled to the same range, usually between 0 and 1, to eliminate the influence of different feature scales, and obtain a normalized multidimensional resource information data set. The cluster center and membership matrix of the implicit fuzzy K-means clustering algorithm are initialized. The cluster center refers to the center point of each cluster, and the membership matrix indicates the degree to which each data point belongs to each cluster. Assume that the server nodes are divided into K clusters. The initialized cluster center can randomly select K data points as the initial center. The membership matrix is ​​a matrix with N rows and K columns, which represents the membership of each server node to each cluster center. During initialization, a non-negative matrix that satisfies the sum of each row is 1 can be randomly generated. Based on the initialized cluster center and the membership matrix, the Euclidean distance between each server node and each cluster center is calculated, and the membership matrix is ​​updated. The calculation formula of the Euclidean distance is as follows:

[0048]

[0049] Among them, d ij represents the Euclidean distance between the i-th server node and the j-th cluster center, X′ il represents the I-th feature value of the i-th server node in the normalized dataset, C jl Represents the Ith eigenvalue of the jth cluster center. According to the Euclidean distance, the membership matrix can be updated, and the new membership calculation formula is as follows:

[0050]

[0051] Among them, u ij Indicates the membership of the i-th server node to the j-th cluster center, m is the fuzzy index of the membership matrix, which is usually greater than 1 and is used to control the fuzziness of the membership. According to the updated membership matrix, the new cluster center is calculated. The calculation formula of the new cluster center is as follows:

[0052]

[0053] Among them, C jl represents the I-th eigenvalue of the j-th cluster center, u ijrepresents the updated membership, and m is the fuzzy index of the membership matrix. The membership matrix and cluster centers are updated iteratively until the cluster centers and membership matrix converge, that is, after multiple iterations, the positions of the cluster centers and the values ​​of the membership matrix no longer change significantly, thereby obtaining the fuzzy partitioning results of each server node.

[0054] In a specific embodiment, the process of executing step S102 may specifically include the following steps:

[0055] (1) Based on the fuzzy partitioning results, the weighted average membership of each server node in each cluster is calculated to obtain the comprehensive membership vector of each server node;

[0056] (2) Based on the comprehensive membership vector, the cosine similarity between any two server nodes is calculated to obtain the server node similarity matrix;

[0057] (3) Perform threshold filtering on the server node similarity matrix to obtain a binary target similarity matrix;

[0058] (4) Based on the target similarity matrix, a similarity weighted undirected graph between server nodes is constructed to obtain the initial load topology graph;

[0059] (5) The initial load topology map is processed through the community discovery algorithm to obtain the community division results of the server nodes;

[0060] (6) Based on the community division results, the load balancing capability of the server nodes in each community is evaluated to obtain the community load balancing capability vector;

[0061] (7) Selecting load balancing nodes for the communities in the initial load topology graph according to the community load balancing capability vector to obtain a community load balancing node set;

[0062] (8) According to the community load balancing node set, the initial load topology graph is updated with edge weights to obtain an inter-community load balancing topology graph;

[0063] (9) Processing the inter-community load balancing topology graph through the minimum spanning tree algorithm to obtain the minimum cost load balancing path set;

[0064] (10) Based on the minimum cost load balancing path set, a valid load topology map covering all server nodes is constructed.

[0065] Specifically, according to the fuzzy partitioning results, the weighted average of the membership of each server node in each cluster is calculated to obtain the comprehensive membership vector of each server node. Assume that there are N server nodes and K cluster centers, and the membership matrix is ​​U, where U ijRepresents the membership of the i-th server node to the j-th cluster center, and the comprehensive membership vector V i It can be expressed as:

[0066]

[0067] Among them, V i Represents the comprehensive membership vector of the i-th server node. Based on the comprehensive membership vector, the cosine similarity between any two server nodes is calculated to obtain the server node similarity matrix. The calculation formula of cosine similarity is as follows:

[0068]

[0069] Among them, sim(V i ,V j ) represents the cosine similarity between the i-th and j-th server nodes, V i ·V j Represents the dot product of two vectors, ∥V i ∥ and ∥V j∥ respectively represent the modulus of the vector. Threshold filtering is performed on the server node similarity matrix to obtain a binary target similarity matrix. By setting a similarity threshold, the similarity values ​​above the threshold are retained as 1, and the rest are set to 0, to obtain a binary similarity matrix, thereby simplifying the representation of similarity. According to the target similarity matrix, a similarity weighted undirected graph between server nodes is constructed to obtain the initial load topology graph. In this graph, nodes represent servers, edges represent the similarity between servers, and the weight of the edge is determined by the similarity value. The initial load topology graph is processed by the community discovery algorithm to obtain the community division result of the server nodes. The community discovery algorithm is used to identify the community structure in the initial topology graph and divide the server nodes with high similarity into the same community, thereby facilitating subsequent load balancing operations. According to the community division results, the load balancing capability of the server nodes in each community is evaluated to obtain the community load balancing capability vector. This vector reflects the load balancing capability of each node in each community and provides data support for optimizing load distribution. According to the community load balancing capability vector, load balancing nodes are selected for the communities in the initial load topology graph to obtain a community load balancing node set. By selecting nodes with strong load balancing capabilities in the community, the load balancing effect of the entire community can be improved. On this basis, the initial load topology map is updated with edge weights according to the community load balancing node set to obtain the inter-community load balancing topology map. The updated topology map more accurately reflects the load balancing relationship between communities. The inter-community load balancing topology map is processed by the minimum spanning tree algorithm to obtain the minimum cost load balancing path set. The minimum spanning tree algorithm optimizes the load balancing path between communities by selecting the path set with the smallest sum of edge weights. Based on the minimum cost load balancing path set, an effective load topology map covering all server nodes is constructed. This map ensures that all server nodes can efficiently perform load balancing, thereby optimizing the overall performance of the system.

[0070] In a specific embodiment, the process of executing step S103 may specifically include the following steps:

[0071] (1) According to the effective load topology diagram, the historical load data of multiple server nodes are obtained and a historical load time series data set is constructed;

[0072] (2) Perform feature extraction on the historical load time series data set to obtain a set of historical load feature vectors, and create a historical load prediction model based on the set of historical load feature vectors;

[0073] (3) According to the payload topology graph, obtain the network delay and bandwidth data between server nodes and construct a network topology feature matrix;

[0074] (4) Perform spectral clustering analysis on the network topology feature matrix to obtain the network topology community structure of the server nodes, and calculate the centrality index of the server nodes in each community based on the network topology community structure to obtain the community centrality vector;

[0075] (5) Based on the community centrality vector, the load transfer impact of the server nodes in each community is evaluated to obtain the community load transfer impact matrix;

[0076] (6) Inputting the historical load forecasting model and the community load transfer influence matrix into the gated recurrent unit network to perform dynamic load evolution modeling and obtain a dynamic load forecasting model;

[0077] (7) According to the dynamic load prediction model, the load change trend of each server node in the future period is predicted to obtain the load change trend prediction result;

[0078] (8) Based on the load change trend prediction results and the community load transfer impact matrix, a dynamic load prediction result covering all server nodes is generated.

[0079] Specifically, according to the effective load topology map, the historical load data of multiple server nodes are obtained and a historical load time series data set is constructed. Assume that there are N server nodes, and each node has its load data at different time points, such as CPU utilization, memory utilization, I / O utilization, and network bandwidth utilization. By collecting and organizing these data, a historical load time series data set is constructed, in which each row represents the load data at a time point, and each column represents a server node. The historical load time series data set is subjected to feature extraction processing to extract key load features and form a historical load feature vector set. These feature vectors may include the average value, variance, peak value, periodicity, etc. of the load. Through these features, the law of load change can be better understood. A historical load prediction model is created based on the historical load feature vector set. For example, methods based on time series analysis (such as ARIMA model), machine learning algorithms (such as support vector machine, random forest) or deep learning algorithms (such as LSTM, GRU), etc. By training historical data, the model can predict the load situation in the future. According to the effective load topology map, the network delay and bandwidth data between server nodes are obtained and a network topology feature matrix is ​​constructed. This matrix is ​​used to represent the connection relationship and communication cost between server nodes, where each element represents the delay or bandwidth between two nodes. By performing spectral clustering analysis on the network topology feature matrix, server nodes with similar network characteristics are divided into the same community, thereby obtaining the network topology community structure of the server nodes. According to the network topology community structure, the centrality index of the server nodes in each community is calculated to obtain the community centrality vector. The centrality index can be used to measure the importance of each node in the network. Common centrality indexes include degree centrality, betweenness centrality, and closeness centrality. Through these indicators, server nodes with important positions in the community can be identified. According to the community centrality vector, the load transfer impact of the server nodes in each community is evaluated to obtain the community load transfer impact matrix. This matrix is ​​used to represent the load transfer path and impact degree in the community. By analyzing the load transfer between different nodes, the needs and challenges of load balancing can be better understood. The historical load prediction model and the community load transfer impact matrix are input into the gated recurrent unit (GRU) network for dynamic load evolution modeling to obtain a dynamic load prediction model. GRU is a commonly used recurrent neural network that can effectively process time series data and capture the dynamic change law of load. By combining historical load data and load transfer effects, future load changes can be predicted more accurately. Based on the dynamic load prediction model, the load change trend of each server node in the future is predicted to obtain the load change trend prediction result. This result can be used to guide load balancing decisions, so that the system can prepare in advance to deal with load changes, thereby improving the stability and response speed of the system.Based on the load change trend prediction results and the community load transfer impact matrix, a dynamic load prediction result covering all server nodes is generated. By comprehensively considering the load change trend and load transfer impact, load balancing can be better achieved to ensure the resource utilization of each server node and the overall performance of the system. These data are input into the GRU network for dynamic load evolution modeling, predicting future load change trends, and finally generating dynamic load prediction results covering all server nodes.

[0080] In a specific embodiment, the process of executing step S104 may specifically include the following steps:

[0081] (1) Based on the payload topology, each server node is modeled as an agent to construct a multi-agent system model;

[0082] (2) Quantitatively evaluate the load processing capacity and resource utilization of each agent to obtain the agent state feature vector. Based on the agent state feature vector, construct the cooperative game matrix between agents and define the game payoff function.

[0083] (3) Using the gradient descent algorithm with adaptive learning rate to solve the Nash equilibrium of the cooperative game matrix, we can obtain the cooperative strategy of each intelligent agent.

[0084] (4) Based on the cooperation strategy, a load balancing cooperation mechanism between intelligent agents is constructed and deep reinforcement learning training is performed to obtain a load balancing strategy evaluation network and a load balancing strategy generation network;

[0085] (5) The load balancing strategy evaluation network and the load balancing strategy generation network are combined into an Actor-Critic architecture to construct a load balancing intelligent agent model.

[0086] Specifically, according to the effective load topology, each server node is modeled as an agent, and a multi-agent system model is constructed. In this model, each server node is regarded as an independent agent, and these agents can achieve load balancing by cooperating with each other. The load processing capacity and resource utilization of each agent are quantitatively evaluated to obtain the agent state feature vector, including key indicators such as CPU utilization, memory utilization, I / O utilization, and network bandwidth utilization. According to the agent state feature vector, a cooperative game matrix between agents is constructed, and the game payoff function is defined. The element A in the game matrix ij represents the cooperative relationship between agent i and agent j, and the game profit function is used to quantify the benefits under different cooperation strategies. Assume that the profit function R i It is expressed as:

[0087] R i =∑ j Aij ·f(S i ,S j );

[0088] Among them, f(S i ,S j ) is a function that represents the impact of the state characteristics of agent i and agent j on the cooperative benefits. The Nash equilibrium is solved for the cooperative game matrix using the gradient descent algorithm with an adaptive learning rate to obtain the cooperative strategies of each agent. Nash equilibrium is a game theory solution that indicates that in an equilibrium state, no agent can improve its benefits by unilaterally changing its strategy. Through the gradient descent algorithm, the strategy of each agent can be gradually adjusted so that the overall system gradually tends to an equilibrium state. The adaptive learning rate can dynamically adjust the learning rate according to the current gradient information, thereby accelerating the convergence speed and avoiding falling into a local optimal solution. According to the cooperative strategies of each agent, a load balancing cooperation mechanism between agents is constructed, and deep reinforcement learning training is performed to obtain a load balancing strategy evaluation network and a load balancing strategy generation network. In deep reinforcement learning, the evaluation network is used to evaluate the effect of the current load balancing strategy, and the generation network is used to generate a new load balancing strategy. The evaluation network inputs the current system state and strategy, and outputs an evaluation value to measure the pros and cons of the strategy. The generation network inputs the current system state and outputs a new load balancing strategy. The load balancing strategy evaluation network and the load balancing strategy generation network are combined into an Actor-Critic architecture to build a load balancing agent model. In the Actor-Critic architecture, the Actor is responsible for selecting and executing the load balancing strategy, and the Critic is responsible for evaluating the effect of the selected strategy and guiding the Actor to optimize the strategy. In this way, the agent can continuously learn and improve its load balancing strategy, achieve efficient load management and distribution, and ensure that the system can maintain efficient and stable operation under various load conditions.

[0089] In a specific embodiment, the process of executing step S105 may specifically include the following steps:

[0090] (1) Based on the dynamic load prediction results, construct the initial load demand matrix and initial load supply matrix of the server node;

[0091] (2) normalizing the initial load demand matrix and the initial load supply matrix to obtain a standardized load demand matrix and a standardized load supply matrix;

[0092] (3) Based on the standardized load demand matrix and the standardized load supply matrix, a load distribution optimization model is constructed and a load distribution utility function is defined;

[0093] (4) Using the alternating direction multiplier method to solve the load distribution optimization model, the initial load distribution matrix is ​​obtained;

[0094] (5) According to the load balancing agent model in the multi-model load balancing mechanism, a game equilibrium analysis is performed on the initial load distribution matrix to obtain a game equilibrium load distribution matrix;

[0095] (6) According to the load balancing strategy evaluation network in the multi-model load balancing mechanism, a strategy evaluation is performed on the game balancing load distribution matrix to obtain a load distribution strategy evaluation result;

[0096] (7) According to the load distribution strategy evaluation results, the game equilibrium load distribution matrix is ​​improved to obtain an improved load distribution matrix;

[0097] (8) generating a load distribution strategy among server nodes according to the improved load distribution matrix, and obtaining a first load distribution strategy set;

[0098] (9) The first load distribution strategy set is used as an input of the load balancing strategy generation network and a dynamic load distribution strategy is generated to obtain a final initial load distribution strategy.

[0099] Specifically, based on the dynamic load prediction results, the initial load demand matrix and initial load supply matrix of the server nodes are constructed. Assuming that there are N server nodes, the load demand and supply of each node in the future can be expressed as a demand matrix and a supply matrix, respectively. Based on these matrices, we can fully understand the load conditions of each node in different time periods. The initial load demand matrix and the initial load supply matrix are standardized to obtain a standardized load demand matrix and a standardized load supply matrix. The purpose of standardization is to scale each feature to the same scale for easy comparison and optimization. Based on the standardized load demand matrix and the standardized load supply matrix, a load distribution optimization model is constructed, and the load distribution utility function is defined. The goal of the load distribution optimization model is to maximize the overall utility of the system while satisfying the demand and supply constraints of each node. Assume that the utility function U is expressed as:

[0100]

[0101] Among them, w ij represents the weight between node i and node j, reflecting the cooperative relationship between them. i j ′ and S i j ′ represent the standardized demand and supply values, respectively. Used to measure the effectiveness of load distribution.

[0102] The alternating direction multiplier method is used to solve the load distribution optimization model and obtain the initial load distribution matrix. The alternating direction multiplier method is a commonly used optimization algorithm that decomposes complex optimization problems into multiple sub-problems and solves them step by step through iteration to find the global optimal solution. According to the load balancing agent model in the multi-model load balancing mechanism, the initial load distribution matrix is ​​subjected to game equilibrium analysis to obtain the game equilibrium load distribution matrix. Game equilibrium analysis determines a stable load distribution strategy by considering the mutual influence between nodes, so that no node can improve its own benefits by unilaterally changing the strategy. According to the load balancing strategy evaluation network in the multi-model load balancing mechanism, the game equilibrium load distribution matrix is ​​subjected to strategy evaluation to obtain the load distribution strategy evaluation result. The load balancing strategy evaluation network inputs the current system state and strategy and outputs an evaluation value to measure the pros and cons of the strategy. According to the load distribution strategy evaluation result, the game equilibrium load distribution matrix is ​​strategically improved to obtain the improved load distribution matrix. By combining the strategy evaluation result, the current load distribution strategy is adjusted and optimized to further improve the load balancing performance of the system. According to the improved load distribution matrix, the load distribution strategy between server nodes is generated to obtain the first load distribution strategy set. This strategy set is based on the initial load distribution strategy, and is the optimal solution obtained through game equilibrium and strategy improvement, which can effectively improve the load balancing effect of the system. The first load distribution strategy set is used as the input of the load balancing strategy generation network, and a dynamic load distribution strategy is generated to obtain the final initial load distribution strategy. The load balancing strategy generation network generates a dynamically adjusted load distribution strategy based on the input strategy set through a deep learning model, thereby ensuring that it can respond to load changes in real time during actual operation and maintain efficient operation of the system.

[0103] In a specific embodiment, the process of executing step S106 may specifically include the following steps:

[0104] (1) According to the initial load distribution strategy, the objective function and constraints of the load distribution optimization problem are constructed to obtain the load distribution optimization model;

[0105] (2) Perform convexity analysis on the load distribution optimization model, transform non-convex constraints into convex constraints, and obtain a convex optimization load distribution model;

[0106] (3) The convex optimization load distribution model is solved using the interior point method to obtain the optimal solution vector of the load distribution strategy;

[0107] (4) generating an optimal load distribution strategy among the server nodes according to the optimal solution vector, and obtaining a second load distribution strategy set;

[0108] (5) performing a sensitivity analysis on the second load distribution strategy set to evaluate the impact of load changes on each server node on load distribution performance and obtain a load sensitivity evaluation result;

[0109] (6) performing robustness optimization on the second load distribution strategy set according to the load sensitivity evaluation result to obtain a robust load distribution strategy set;

[0110] (7) Taking the robust load distribution strategy set as a constraint, a load distribution fairness optimization model is constructed and the load distribution fairness index is defined;

[0111] (8) A sequential convex programming algorithm is used to solve the load distribution fairness optimization model and obtain the optimal load distribution strategy that takes into account both load distribution performance and fairness.

[0112] Specifically, according to the initial load distribution strategy, the objective function and constraints of the load distribution optimization problem are constructed to obtain the load distribution optimization model. Assume that there are N server nodes, and the load demand and supply of each node are represented by vectors D and S respectively, where D i represents the load demand of the i-th node, S i represents the load supply of the first node. The initial load distribution strategy is represented by the matrix X, where X ij represents the load distributed from the i-th node to the j-th node. The objective function aims to minimize the load distribution cost of the overall system, which is defined as:

[0113]

[0114] Among them, c ij represents the cost of distributing load from the i-th node to the j-th node. The constraints include that the load demand and supply of each node must be balanced, that is:

[0115]

[0116] Through these constraints, it is ensured that the load distribution of each node is within its supply capacity and meets its load demand. The load distribution optimization model is subjected to convexity analysis, and non-convex constraints are converted into convex constraints to obtain a convex optimization load distribution model. The purpose of convexity analysis is to ensure that the optimization problem has a unique global optimal solution so that the optimization algorithm can be effectively solved. Non-convex constraints may include complex restrictions in load distribution. These non-convex constraints are converted into convex constraints through relaxation techniques or other mathematical means. For example, by introducing auxiliary variables and Lagrange multipliers, complex constraints are restated in convex form. The convex optimization load distribution model is solved by the interior point method to obtain the optimal solution vector of the load distribution strategy. The interior point method is a commonly used optimization algorithm. By gradually approaching the optimal solution in the feasible domain, the global optimal solution can be quickly found. The interior point method effectively handles the constraints by introducing a barrier function so that the optimization process always remains in the feasible domain. According to the optimal solution vector, the optimal load distribution strategy between server nodes is generated to obtain the second load distribution strategy set. This strategy set is based on the initial load distribution strategy. The optimal solution obtained after convex optimization can effectively improve the load distribution efficiency of the system. A sensitivity analysis is performed on the second load distribution strategy set to evaluate the impact of the load changes of each server node on the load distribution performance, and the load sensitivity evaluation results are obtained. Sensitivity analysis helps identify potential bottlenecks and risk points in load distribution by analyzing the impact of the load changes of each node on the overall system performance. For example, by slightly perturbing the load demand and supply of each node, the change in the total cost of the system is observed, thereby evaluating the robustness of the system. According to the load sensitivity evaluation results, the second load distribution strategy set is robustly optimized to obtain a robust load distribution strategy set. The purpose of robust optimization is to make the system stable and efficient in the face of load fluctuations and uncertainties. By introducing redundancy and buffers, it is ensured that the system can cope with unexpected load changes. For example, a certain amount of redundant capacity can be reserved in the load distribution strategy so that the system can still operate normally when the load suddenly increases. The robust load distribution strategy set is used as a constraint condition to construct a load distribution fairness optimization model and define the load distribution fairness index. The purpose of the fairness optimization model is to ensure that the load is distributed more fairly among the server nodes to avoid the situation where some nodes are overloaded while other nodes are idle. Fairness indicators such as the Gini coefficient or Theil index can be used to quantify the fairness of load distribution. By introducing fairness constraints, it is ensured that the load distribution process meets both performance requirements and fairness. The sequential convex programming algorithm is used to solve the load distribution fairness optimization model to obtain the optimal load distribution strategy that takes into account both load distribution performance and fairness. The sequential convex programming algorithm decomposes the non-convex optimization problem into a series of convex optimization sub-problems, gradually solves them, and finally finds the global optimal solution. In this way, the fairness of load distribution can be achieved while meeting performance requirements.

[0117] The above describes the adaptive load balancing method based on multiple models in the embodiment of the present application. The following describes the adaptive load balancing system based on multiple models in the embodiment of the present application. Figure 2 In the embodiment of the present application, an embodiment of the adaptive load balancing system based on multiple models includes:

[0118] The acquisition module 201 is used to acquire a multidimensional resource information data set of multiple server nodes, and perform implicit fuzzy K-means clustering on the multidimensional resource information data set to obtain a fuzzy partition result of each server node;

[0119] A construction module 202 is used to generate a target similarity matrix for each server node according to the fuzzy partitioning result, and to construct a valid load topology map of multiple server nodes according to the target similarity matrix;

[0120] The analysis module 203 is used to perform historical load records and network topology analysis on the effective load topology diagram to generate dynamic load prediction results;

[0121] A modeling module 204 is used to model multiple server nodes in the effective load topology diagram as a multi-agent cooperative game, obtain a cooperation strategy for each server node, and establish a multi-model load balancing mechanism according to the cooperation strategy;

[0122] The calculation module 205 is used to perform load distribution calculation on the dynamic load prediction result based on the multi-model load balancing mechanism to obtain an initial load distribution strategy;

[0123] The processing module 206 is used to perform convex optimization processing on the initial load distribution strategy to obtain an optimal load distribution solution for multiple server nodes.

[0124] Through the cooperation of the above components, implicit fuzzy K-means clustering is performed by obtaining multi-dimensional resource information of multiple server nodes, which effectively reflects the resource usage of server nodes. The iterative update of the membership matrix and cluster center makes the clustering results more accurate, and can dynamically adjust the classification of server nodes, thereby improving the accuracy of load balancing. By calculating the comprehensive membership vector and cosine similarity of the server nodes, the target similarity matrix is ​​generated, and the effective load topology map is constructed, which accurately reflects the similarity and dependency between server nodes. The load topology is optimized using the community discovery algorithm and the minimum spanning tree algorithm to ensure the minimum cost of the load distribution path, improve the load transmission efficiency and overall network performance. The dynamic load prediction is performed using historical load data and network topology characteristics, and the future load changes are modeled through the gated recurrent unit network, so that the prediction results are more accurate and timely. Based on the multi-agent cooperative game model and deep reinforcement learning method, the load distribution strategy is dynamically adjusted, so that the load balancing mechanism can adapt to the changing load demand, and improve the adaptability and robustness of the system. The alternating direction multiplier method and the interior point method are used for load distribution optimization, which solves the non-convex constraint problem and ensures the global optimality of the load distribution scheme. Through load sensitivity evaluation and robustness optimization, the impact of load changes on performance is considered to ensure the fairness and stability of the load distribution strategy under different load conditions. The multi-model adaptive load balancing method comprehensively considers the multi-dimensional resource usage of server nodes, historical load data and network topology characteristics, and can maintain efficient resource utilization in a complex and changing environment. It improves the system's response speed and service quality, reduces resource waste and overload, and enhances user experience and service stability.

[0125] The present application also provides an adaptive load balancing device based on multiple models, wherein the adaptive load balancing device based on multiple models includes a memory and a processor, wherein computer-readable instructions are stored in the memory, and when the computer-readable instructions are executed by the processor, the processor executes the steps of the adaptive load balancing method based on multiple models in the above-mentioned embodiments.

[0126] The present application also provides a computer-readable storage medium, which may be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium. Instructions are stored in the computer-readable storage medium. When the instructions are executed on a computer, the computer executes the steps of the multi-model-based adaptive load balancing method.

[0127] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described systems, systems and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0128] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program code.

[0129] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A multi-model based adaptive load balancing method, characterized in that: The method comprises: Acquire a multidimensional resource information data set of multiple server nodes, and perform implicit fuzzy K-means clustering on the multidimensional resource information data set to obtain a fuzzy partition result of each server node; According to the fuzzy partitioning result, the weighted average of the membership of each server node in each cluster is calculated to obtain a comprehensive membership vector of each server node; according to the comprehensive membership vector, the cosine similarity between any two server nodes is calculated to obtain a server node similarity matrix; the server node similarity matrix is ​​subjected to threshold filtering to obtain a binary target similarity matrix, and a valid load topology map of the multiple server nodes is constructed according to the target similarity matrix; Performing historical load records and network topology analysis on the effective load topology diagram to generate dynamic load prediction results; Modeling multiple server nodes in the effective load topology graph as a multi-agent cooperative game, obtaining a cooperation strategy for each server node, and establishing a multi-model load balancing mechanism according to the cooperation strategy; Perform load distribution calculation on the dynamic load prediction result based on the multi-model load balancing mechanism to obtain an initial load distribution strategy; The initial load distribution strategy is subjected to convex optimization processing to obtain an optimal load distribution solution for the multiple server nodes.

2. The multi-model based adaptive load balancing method according to claim 1, characterized in that: The method of acquiring a multidimensional resource information data set of multiple server nodes and performing implicit fuzzy K-means clustering on the multidimensional resource information data set to obtain a fuzzy partition result of each server node includes: Obtain CPU utilization, memory utilization, I / O utilization, and network bandwidth utilization of multiple server nodes to build a multi-dimensional resource information dataset; Performing feature scaling processing on the multidimensional resource information data set to obtain a normalized multidimensional resource information data set; Initializing the cluster center and membership matrix of the implicit fuzzy K-means clustering algorithm according to the normalized multidimensional resource information data set; According to the cluster centers and the membership matrix, the Euclidean distance between each server node and each cluster center is calculated, and the membership matrix is ​​updated; According to the updated membership matrix, the new cluster center is calculated until the cluster center and the membership matrix converge, and the fuzzy partition result of each server node is obtained.

3. The multi-model based adaptive load balancing method according to claim 1, characterized in that: The constructing the effective load topology graph of the plurality of server nodes according to the target similarity matrix comprises: According to the target similarity matrix, a similarity weighted undirected graph between server nodes is constructed to obtain an initial load topology graph; The initial load topology graph is processed by a community discovery algorithm to obtain a community division result of the server node; According to the community division result, the load balancing capability of the server nodes in each community is evaluated to obtain a community load balancing capability vector; Selecting load balancing nodes for the community in the initial load topology diagram according to the community load balancing capability vector to obtain a community load balancing node set; According to the community load balancing node set, the initial load topology graph is updated with edge weights to obtain an inter-community load balancing topology graph; Processing the inter-community load balancing topology graph by a minimum spanning tree algorithm to obtain a minimum cost load balancing path set; Based on the minimum cost load balancing path set, a valid load topology diagram covering all server nodes is constructed.

4. The multi-model based adaptive load balancing method according to claim 1, characterized in that: The performing of historical load recording and network topology analysis on the effective load topology diagram to generate a dynamic load prediction result includes: According to the effective load topology diagram, obtaining historical load data of the plurality of server nodes and constructing a historical load time series data set; Performing feature extraction processing on the historical load time series data set to obtain a historical load feature vector set, and creating a historical load prediction model based on the historical load feature vector set; According to the effective load topology graph, obtaining network delay and bandwidth data between server nodes and constructing a network topology feature matrix; Performing spectral clustering analysis on the network topology feature matrix to obtain a network topology community structure of server nodes, and calculating the centrality index of the server nodes in each community based on the network topology community structure to obtain a community centrality vector; According to the community centrality vector, load transfer impact assessment is performed on the server nodes in each community to obtain a community load transfer impact matrix; Inputting the historical load forecasting model and the community load transfer influence matrix into a gated recurrent unit network to perform dynamic load evolution modeling to obtain a dynamic load forecasting model; According to the dynamic load prediction model, the load change trend of each server node in the future period is predicted to obtain the load change trend prediction result; A dynamic load prediction result covering all server nodes is generated according to the load change trend prediction result and the community load transfer influence matrix.

5. The multi-model based adaptive load balancing method according to claim 1, characterized in that: The method of modeling the multiple server nodes in the effective load topology graph as a multi-agent cooperative game, obtaining a cooperation strategy for each server node, and establishing a multi-model load balancing mechanism according to the cooperation strategy includes: According to the payload topology diagram, each server node is modeled as an agent to construct a multi-agent system model; Quantitatively evaluate the load processing capacity and resource utilization of each agent to obtain the agent state feature vector, and construct a cooperative game matrix between agents based on the agent state feature vector, and define the game profit function; The cooperative game matrix is ​​solved by using a gradient descent algorithm with an adaptive learning rate to obtain the cooperation strategy of each intelligent agent; According to the cooperation strategy, a load balancing cooperation mechanism between intelligent agents is constructed and deep reinforcement learning training is performed to obtain a load balancing strategy evaluation network and a load balancing strategy generation network; The load balancing strategy evaluation network and the load balancing strategy generation network are combined into an Actor-Critic architecture to construct a load balancing intelligent agent model.

6. The multi-model based adaptive load balancing method according to claim 5, characterized in that: The performing load distribution calculation on the dynamic load prediction result based on the multi-model load balancing mechanism to obtain an initial load distribution strategy includes: According to the dynamic load prediction result, construct an initial load demand matrix and an initial load supply matrix of the server node; Performing standardization processing on the initial load demand matrix and the initial load supply matrix to obtain a standardized load demand matrix and a standardized load supply matrix; According to the standardized load demand matrix and the standardized load supply matrix, a load distribution optimization model is constructed, and a load distribution utility function is defined; The load distribution optimization model is solved by using the alternating direction multiplier method to obtain the initial load distribution matrix. According to the load balancing agent model in the multi-model load balancing mechanism, a game equilibrium analysis is performed on the initial load distribution matrix to obtain a game equilibrium load distribution matrix; According to the load balancing strategy evaluation network in the multi-model load balancing mechanism, a strategy evaluation is performed on the game balancing load distribution matrix to obtain a load distribution strategy evaluation result; According to the load distribution strategy evaluation result, the game balance load distribution matrix is ​​improved strategically to obtain an improved load distribution matrix; Generate a load distribution strategy between server nodes according to the improved load distribution matrix to obtain a first load distribution strategy set; The first load distribution strategy set is used as an input of a load balancing strategy generation network to generate a dynamic load distribution strategy, thereby obtaining a final initial load distribution strategy.

7. The multi-model based adaptive load balancing method according to claim 6, characterized in that: The performing convex optimization processing on the initial load distribution strategy to obtain an optimal load distribution scheme for the multiple server nodes includes: According to the initial load distribution strategy, construct the objective function and constraint conditions of the load distribution optimization problem to obtain a load distribution optimization model; Performing convexity analysis on the load distribution optimization model, converting non-convex constraints into convex constraints, and obtaining a convex optimization load distribution model; The convex optimization load distribution model is solved by using the interior point method to obtain the optimal solution vector of the load distribution strategy; Generate an optimal load distribution strategy among server nodes according to the optimal solution vector to obtain a second load distribution strategy set; Performing a sensitivity analysis on the second load distribution strategy set to evaluate the impact of load changes on the load distribution performance of each server node and obtain a load sensitivity evaluation result; According to the load sensitivity evaluation result, performing robustness optimization on the second load distribution strategy set to obtain a robust load distribution strategy set; Taking the robust load distribution strategy set as a constraint condition, constructing a load distribution fairness optimization model, and defining a load distribution fairness index; The load distribution fairness optimization model is solved by using a sequential convex programming algorithm to obtain an optimal load distribution strategy that takes both load distribution performance and fairness into consideration.

8. A multi-model based adaptive load balancing system, used to execute the multi-model based adaptive load balancing method according to any one of claims 1 to 7, characterized in that: The system comprises: An acquisition module, used to acquire a multidimensional resource information data set of multiple server nodes, and perform implicit fuzzy K-means clustering on the multidimensional resource information data set to obtain a fuzzy partition result of each server node; A construction module is used to calculate the weighted average of the membership of each server node in each cluster according to the fuzzy partitioning result to obtain a comprehensive membership vector of each server node; calculate the cosine similarity between any two server nodes according to the comprehensive membership vector to obtain a server node similarity matrix; perform threshold filtering on the server node similarity matrix to obtain a binary target similarity matrix, and construct a valid load topology map of the multiple server nodes according to the target similarity matrix; An analysis module, used for performing historical load recording and network topology analysis on the effective load topology diagram to generate a dynamic load prediction result; A modeling module, used for modeling multiple server nodes in the effective load topology diagram as a multi-agent cooperative game, obtaining a cooperation strategy for each server node, and establishing a multi-model load balancing mechanism according to the cooperation strategy; A calculation module, used for performing load distribution calculation on the dynamic load prediction result based on the multi-model load balancing mechanism to obtain an initial load distribution strategy; The processing module is used to perform convex optimization processing on the initial load distribution strategy to obtain an optimal load distribution solution for the multiple server nodes.

9. An adaptive load balancing device based on multiple models, characterized in that: The multi-model based adaptive load balancing device comprises: a memory and at least one processor, wherein instructions are stored in the memory; The at least one processor calls the instruction in the memory to enable the multi-model-based adaptive load balancing device to execute the multi-model-based adaptive load balancing method as described in any one of claims 1-7.

10. A computer-readable storage medium having instructions stored thereon, characterized in that: When the instructions are executed by the processor, the multi-model based adaptive load balancing method as described in any one of claims 1 to 7 is implemented.

Images