Load balancing method and device, electronic equipment, storage medium and program product
By constructing performance indicator functions in multi-agent systems and determining the optimal strategy function using reinforcement learning algorithms, the problems of lack of flexibility and request loss of existing load balancing solutions are solved, and adaptive load balancing and network performance optimization are achieved.
Patent Information
- Application Number
- CN202510076852.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-01-17
- Publication Date
- 2025-05-02
AI Technical Summary
Existing load balancing solutions lack flexibility, make it difficult to assign tasks based on the server's load-bearing capacity, and request loss is prone to occur in the event of a server failure.
By obtaining the status information of each server node in a multi-agent system, building performance indicator functions, and using reinforcement learning algorithms to update iteratively to determine the optimal strategy function, thereby achieving load balancing.
It realizes adaptive adjustment of load balancing strategies according to the real-time status of each server node, optimizes network traffic management, improves resource utilization and network performance, and avoids the problem of request loss.
Smart Images
Figure CN119917281A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of network technology, and in particular to a load balancing method, device, electronic device, computer storage medium and computer program product. Background Art
[0002] Load balancing is an indispensable part of modern network architecture, which improves system performance and reliability by distributing requests to multiple servers. Load balancing products can reasonably distribute requests according to load balancing algorithms, maximize the use of system resources, and avoid wasting system resources.
[0003] In the related technology, the solutions to achieve load balancing mainly include the following: 1) Polling: The polling algorithm aims to distribute requests to the background servers in the order in which the requests arrive. 2) Weighted polling: Requests can be distributed to different servers according to different weights based on different hardware configurations. 3) Random strategy: A random function can be set to randomly take a value and map the request to the corresponding server. 4) Hash algorithm: By setting a hash function, the Internet Protocol (IP) address of the client is mapped to a hash value, and the hash value is modulo the number of servers to obtain the result, which is the serial number of the server to which the request is mapped.
[0004] However, the above-mentioned load balancing solutions lack flexibility. For example, since different servers have different hardware configurations, simple polling methods or random strategies are difficult to allocate tasks according to the server's carrying capacity; and load balancing solutions based on hash algorithms are difficult to cope with server failure scenarios. If a server goes down, the server to which the request should fall cannot obtain the request, which can easily lead to request loss. Summary of the invention
[0005] The present application provides a load balancing method, device, electronic device, computer storage medium and computer program product.
[0006] The technical solution of this application is implemented as follows:
[0007] The present application provides a load balancing method, which is applied to a multi-agent system, wherein the multi-agent system includes a plurality of server nodes running in a distributed manner, and the method includes:
[0008] Obtaining state information of each server node in the multi-agent system at a first moment;
[0009] Determine, based on the state information at the first moment, a total first state difference between each server node and all adjacent server nodes;
[0010] Constructing a performance indicator function of each server node according to the total difference of the first state corresponding to each server node;
[0011] Iteratively updating the performance indicator function of each server node until a stop iteration condition is met, and determining an optimal strategy function based on the performance indicator function obtained in the last iteration;
[0012] The multi-agent system is load balanced according to the optimal strategy function.
[0013] The present application provides a load balancing device, which is applied to a multi-agent system, wherein the multi-agent system includes a plurality of server nodes running in a distributed manner, and the device includes:
[0014] An acquisition module, used to acquire the state information of each server node in the multi-agent system at a first moment;
[0015] a determination module, configured to determine a total first state difference between each server node and all adjacent server nodes according to the state information at the first moment;
[0016] A construction module, configured to construct a performance indicator function of each server node according to the total difference of the first state corresponding to each server node;
[0017] An iteration module, used for iteratively updating the performance indicator function of each server node until a stop iteration condition is met, and determining an optimal strategy function according to the performance indicator function obtained by the last iteration;
[0018] A balancing module is used to load balance the multi-agent system according to the optimal strategy function.
[0019] The present application provides an electronic device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the load balancing method provided by one or more of the aforementioned technical solutions when executing the program.
[0020] The present application provides a computer storage medium, which stores a computer program; after the computer program is executed, it can implement the load balancing method provided by one or more of the above-mentioned technical solutions.
[0021] The present application provides a computer program product, including a computer program, which, when executed by a processor, implements the load balancing method provided by one or more of the aforementioned technical solutions.
[0022] The embodiments of the present application provide a load balancing method, apparatus, electronic device, computer storage medium and computer program product, which are applied to a multi-agent system, wherein the multi-agent system includes multiple server nodes running in a distributed manner, and the method includes: obtaining status information of each server node in the multi-agent system at a first moment; determining the total difference of the first state between each server node and all adjacent server nodes based on the status information at the first moment; constructing a performance indicator function of each server node based on the total difference of the first state corresponding to each server node; iteratively updating the performance indicator function of each server node until a stop iteration condition is met, and determining an optimal strategy function based on the performance indicator function obtained from the last iteration; and load balancing the multi-agent system according to the optimal strategy function.
[0023] It can be seen that in the embodiment of the present application, each server node in the multi-agent system is used as an intelligent agent, and a performance indicator function is constructed based on the total difference in state between each server node and all adjacent server nodes. The optimal strategy function is obtained by iteratively updating the performance indicator function, and the optimal strategy function is used to perform load balancing so that the state of each server node is consistent, thereby achieving load balancing, that is, it can adaptively adjust its own load balancing strategy according to the real-time state of each server node, optimize network traffic management, and improve resource utilization and network performance. BRIEF DESCRIPTION OF THE DRAWINGS
[0024] Figure 1 A flow chart of a load balancing method provided in an embodiment of the present application;
[0025] Figure 2 A schematic diagram of iterative training of an agent provided in an embodiment of the present application;
[0026] Figure 3 A schematic diagram of the composition structure of the load balancing device provided in an embodiment of the present application;
[0027] Figure 4 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0028] The technical solution in this application will be clearly and completely described below in conjunction with the drawings in this application.
[0029] The present application will be further described in detail below in conjunction with the accompanying drawings and Examples. It should be understood that the embodiments provided herein are only used to explain the present application and are not intended to limit the present application. In addition, the embodiments provided below are partial embodiments for implementing the present application, rather than providing all embodiments for implementing the present application. In the absence of conflict, the technical solutions recorded in the present application can be implemented in any combination.
[0030] It should be noted that, in the embodiments of the present application, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a method or system including a series of elements includes not only the elements explicitly recorded, but also includes other elements not explicitly listed, or also includes elements inherent to the implementation of the method or system. In the absence of further restrictions, an element defined by the sentence "includes a ..." does not exclude the presence of other related elements (such as steps in a method or units in a system, for example, a unit may be a part of a circuit, a part of a processor, a part of a program or software, etc.) in the method or system including the element.
[0031] For example, the load balancing method provided in the present application includes a series of steps, but the load balancing method provided in the present application is not limited to the recorded steps. Similarly, the load balancing device provided in the present application includes a series of modules, but the load balancing device provided in the present application is not limited to including the modules explicitly recorded, and may also include modules that need to be set up to obtain relevant information or perform processing based on information.
[0032] In some embodiments of the present application, the load balancing method can be implemented based on a processor in a load balancing device, and the above-mentioned processor can be at least one of an application-specific integrated circuit (ASIC), a digital signal processor (DSP), a digital signal processing device (DSPD), a programmable logic device (PLD), a field programmable gate array (FPGA), a central processing unit (CPU), a controller, a microcontroller, and a microprocessor.
[0033] In the embodiments of the present application, the load balancing method can be applied to various scenarios where load balancing of multiple servers is required, with the aim of evenly distributing traffic or tasks to each server and improving system processing performance and reliability.
[0034] Figure 1 A flow chart of a load balancing method provided in an embodiment of the present application, such as Figure 1 As shown, the process may include:
[0035] Step 100: Obtain the status information of each server node in the multi-agent system at the first moment.
[0036] In the embodiment of the present application, the executor of the load balancing method is a multi-agent system; here, the multi-agent system may be pre-built, which may include multiple server nodes running in a distributed manner.
[0037] It should be noted that the server nodes in the multi-agent system can communicate with each other according to the network topology and exchange their respective status information; according to graph theory, the network topology conditions between the server nodes can satisfy the minimum spanning tree.
[0038] In the embodiment of the present application, the server node is regarded as a single agent in the multi-agent system; the process of load balancing the multi-agent system involves the consistency theory of the multi-agent system; wherein the multi-agent system is a complex system formed by a group of agents with specific functions coupled through a communication network. The consistency theory of the multi-agent system refers to the fact that adjacent agents in the communication network communicate information and use a preset control protocol to complete the update of their own states, so that all agents finally reach a unified state.
[0039] Exemplarily, the first moment may represent any moment in the operation process of each server node; the status information may include the load information and configuration information of the server node itself, the load information may include CPU usage, memory usage, disk usage, etc., and the configuration information may include communication protocol, IP address, port number, etc.
[0040] In the embodiment of the present application, there is no specific limitation on the method of obtaining the status information. For example, the status information may be obtained through log information or other methods.
[0041] Step 101: Determine the total first state difference between each server node and all adjacent server nodes based on the state information at the first moment.
[0042] In an embodiment of the present application, after obtaining the status information of each server node at the first moment according to the above steps, the total difference in the first state between each server node and all adjacent server nodes can be determined based on the status information of each server node at the first moment.
[0043] Exemplarily, according to the above content, each server node in the multi-agent system operates in a distributed manner, and thus each server node adjacent to each server node can be determined based on the network topology of each server node.
[0044] In some embodiments, determining the total first state difference between each server node and all adjacent server nodes based on the state information at the first moment may include determining the state difference between each server node and each adjacent server node based on the state information at the first moment; for each server node, accumulating each state difference corresponding to the server node to obtain the total first state difference between each server node and all adjacent server nodes.
[0045] In an embodiment of the present application, after determining the server nodes adjacent to each server node based on the network topology of each server node, the state information of each server node and each adjacent server node can be subtracted to obtain the state difference between each server node and each adjacent server node. Then, the state difference values corresponding to each server node are accumulated to obtain the first total state difference between each server node and all adjacent server nodes.
[0046] For example, the above process can be expressed by formula (1):
[0047]
[0048] Among them, e i (k) represents the total state difference of server node i at time k, that is, when time k is the first time, the total state difference of the first time is e i (k); j represents one of the server nodes adjacent to server node i, x j (k) represents the status information of server node j at time k, represents the set of adjacent server nodes of server node i, which includes all server nodes adjacent to server node i, a ij It represents the weight of the edge between server node i and server node j, which can be understood as the weight of the communication link between the two server nodes. The weight can represent the distribution ratio of network requests. The larger the weight, the more network requests will be allocated.
[0049] For example, according to formula (1), when e i When (k) is 0, it means that the load of server node i and the adjacent server node j have reached an agreement. i (k), it means that the system has achieved load balancing.
[0050] Step 102: constructing a performance indicator function of each server node according to the total difference of the first state corresponding to each server node.
[0051] In the embodiment of the present application, after obtaining the first state total difference corresponding to each server node according to the above steps, a performance indicator function of each server node can be constructed according to the first state total difference.
[0052] Here, the performance indicator function represents the cumulative reward function of each server node; in some embodiments, the performance indicator function of each server node is constructed according to the total difference of the first state corresponding to each server node, including: obtaining the policy function of each server node at the first moment; constructing the performance indicator function of each server node according to the total difference of the first state corresponding to each server node and the policy function at the first moment.
[0053] Exemplarily, the total difference of the first state corresponding to each server node refers to the total difference of the state corresponding to each server node at the first moment, corresponding to the above e i (k); The strategy function of each server node at the first moment is used to characterize the behavior information of each server node at the first moment, which can be expressed as u i (k).
[0054] In one embodiment, the total difference value e according to the first state may be i (k) and the policy function u i (k) The performance index function constructed is expressed as Q i (e i (k),u i (k)), abbreviated as Q i ; Among them, the performance indicator function Q i The cumulative reward function of each server node at different times can be expressed as shown in formula (2):
[0055]
[0056] Among them, r i (e i (t),u i (t)) represents the reward function of server node i at time t, also known as the utility function, which is used to judge the direct benefit of the behavior information of server node i at time t. Here, the reward function r i (e i (t),u i (t)) can be expressed by formula (3):
[0057]
[0058] Among them, T represents the transposition symbol, Q ii ≥0, R ii >0, and Q ii and R ii is a non-negative definite constant matrix used to define the reward function, which is convenient for the subsequent performance index function Q i Estimates.
[0059] Step 103: Iteratively update the performance indicator function of each server node until the iteration stop condition is met, and determine the optimal strategy function based on the performance indicator function obtained in the last iteration.
[0060] Exemplarily, the condition for stopping iteration may be that the difference between the performance indicator functions obtained from two previous iterations is less than a stopping iteration threshold.
[0061] In an embodiment of the present application, a reinforcement learning algorithm may be used to iteratively update the performance indicator function of each server node until the difference between the performance indicator functions obtained from two previous iterations is less than a stop iteration threshold.
[0062] Understandably, in the field of reinforcement learning, the benefits obtained by the agent for completing each behavior are calculated by the reward function, and its control goal is that the agent's control strategy can maximize the cumulative reward function (value function). The process of the agent training its own control strategy based on the environment and interaction data is iterative. Figure 2 The iterative idea of the agent training process is demonstrated. However, due to the different definitions of the value function, in the field of optimal control, the control goal is to make the value function (performance indicator function Q i ) minimized.
[0063] Here, the process of interaction between the agent and the environment is to make the agent reach the state of the next moment according to the given initial strategy function under the condition that the state information of the current moment has been initialized, and further complete the update of the strategy function of the current moment according to the state values of the previous and next moments and the defined reward function and performance indicator function, and then make the agent reach the state of the next moment according to the updated strategy function, and repeat this cycle until the optimal strategy function is finally solved. This process is the interaction between the agent and the environment.
[0064] In some embodiments, iteratively updating the performance indicator function of each server node using a reinforcement learning algorithm may include: determining the state information of each server node at a second moment; determining the total difference of the second state between each server node and all adjacent server nodes based on the state information at the second moment; determining the reward function corresponding to each server node based on the total difference of the first state and the total difference of the second state corresponding to each server node; and iteratively updating the performance indicator function of each server node based on the reward function corresponding to each server node.
[0065] Exemplarily, determining the status information of each server node at the second moment may include: obtaining the behavior information of each server node at the first moment; using a kinetic equation to process the status information and behavior information of each server node at the first moment to obtain the status information of each server node at the second moment.
[0066] Here, the second moment is the next moment of the first moment; for example, if the first moment is represented as k, the second moment can be represented as k+1.
[0067] Exemplarily, the behavior information may refer to a load balancing strategy adopted by the server node at the first moment, for example, when receiving network traffic, whether to choose to distribute the network traffic to the server node.
[0068] It can be understood that since each server node in the multi-agent system can adjust its own state information and behavior information, and the adjustment process is constantly changing with time, the embodiment of the present application can use the dynamic equation to process the state information and behavior information of each server node at the first moment to obtain the state information of each server node at the second moment. This process can be expressed by formula (4):
[0069] x i (k+1)=Ax i (k)+B i u i (k),i=1,2,...,N (4)
[0070] Where N represents the number of server nodes, x i (k) represents the state information of server node i at time k, through the behavior information u i (k) Adjust its own status information to x i (k+1), that is, x i (k+1) is the state information of server node i at time k+1; parameters A and B i represents the parameter of server node i. Here, time k represents the first time, and k+1 represents the next time after the first time, that is, the second time.
[0071] Furthermore, by combining the above formula (1) and formula (4), the change process of each server node adjusting its own state can be derived, as shown in formula (5):
[0072]
[0073] Among them, e i (k+1) represents the total difference of the second state corresponding to the server node i at time k+1.
[0074] Furthermore, the reward function corresponding to each server node can be determined according to the total difference of the first state and the total difference of the second state corresponding to each server node; the performance indicator function of each server node is iteratively updated according to the reward function corresponding to each server node; the process is exemplified below.
[0075] In the embodiment of the present application, a policy function is first initialized for each server node. And initialize the status information x of each server node i (0), set the iteration index l and the stop iteration threshold ρ. The iteration index is used to record the iteration rounds, and its initial value is 0. The stop iteration threshold is used to determine the stop iteration condition.
[0076] Further, according to the total difference value e of the first state corresponding to each server node i (k) and the strategy function at the first moment Represents the performance index function of each server node at the first moment According to the total difference value e of the second state corresponding to each server node i (k+1) Strategy function at the second moment Represents the performance index function of each server node at the second moment Then, based on the performance indicator functions at these two moments, determine the reward function corresponding to each server node The corresponding process can be expressed by formula (6):
[0077]
[0078] For example, after obtaining the reward function After that, the parameters of the performance indicator function can be adjusted so that the reward function The estimated value is infinitely close to the true value, and the performance indicator function of this iteration is obtained.
[0079] Furthermore, the difference between the performance indicator function of this iteration and the performance indicator function before adjustment is calculated, and whether the difference is less than the stop iteration threshold is determined according to formula (7):
[0080]
[0081] Exemplarily, when it is determined that the difference is less than the stop iteration threshold, the performance indicator function of this iteration can be solved by using the gradient descent method to obtain the optimal strategy function corresponding to each server node. The optimal strategy function is used to achieve the optimal load scheduling of the server node. The corresponding process is shown in formula (8);
[0082]
[0083] It should be noted that the above formula (8) can be used to complete the strategy function The improvement of i represents the learning rate of server node i, Represents the performance indicator function with respect to the policy function The first-order partial derivative of .
[0084] Exemplarily, when it is determined that the difference is greater than or equal to the stop iteration threshold, l=l+1 is set, and the performance indicator function is continued to be iteratively updated until the difference between the performance indicator functions obtained from the previous and next two iterations is less than the stop iteration threshold, that is, formula (7) is satisfied.
[0085] In the embodiment of the present application, based on the above-mentioned reinforcement learning algorithm, when all server nodes in the multi-agent system satisfy formula (7), that is, the iteration condition is stopped, it means that all server nodes have learned the optimal policy function. At this time, the performance indicator function obtained according to the last iteration can be obtained to determine the optimal policy function; here, the optimal policy function can be obtained by calculating the gradient of the performance indicator function obtained from the last iteration.
[0086] It can be seen that in the embodiment of the present application, a reinforcement learning algorithm is used to enable each server node in the system to have adaptive capabilities, and to reasonably adjust its own load strategy according to the real-time network conditions and its own status. In addition, the reinforcement learning algorithm updates the strategy function of each server node in an iterative manner to ensure that the final optimal strategy function is more stable and achieves more reliable load balancing.
[0087] Step 104: Load balance the multi-agent system according to the optimal strategy function.
[0088] In the embodiment of the present application, after obtaining the optimal strategy function corresponding to each server node, the multi-agent system can be load balanced according to the optimal strategy function.
[0089] In some embodiments, load balancing the multi-agent system according to the optimal strategy function may include: allocating to-be-processed data to each server node according to the optimal strategy function to complete load balancing of the multi-agent system.
[0090] Exemplarily, the data to be processed may be data that needs to be processed and is sent from outside the multi-agent system. Here, there is no specific limitation on the type of data to be processed. For example, it may be network traffic, computing tasks, or network requests.
[0091] In an embodiment of the present application, when any server node in the multi-agent system receives data to be processed, the data to be processed can be scheduled to the target server node according to its corresponding optimal strategy function, thereby achieving load balancing of the multi-agent system.
[0092] It can be seen that in the embodiments of the present application, the consistency theory of multi-agent systems and the reinforcement learning method are applied to network load balancing. In view of the complex network conditions often encountered in traditional networks and virtual networks, the load balancing method can adaptively adjust its own load balancing strategy according to the real-time network status, optimize network traffic management, reduce network congestion, and significantly improve resource utilization and network performance.
[0093] Figure 3 FIG. 1 is a schematic diagram of the composition structure of the load balancing device of an embodiment of the present application, which is applied to a multi-agent system. The multi-agent system includes multiple server nodes running in a distributed manner, such as Figure 3 As shown, the device includes: an acquisition module 300, a determination module 301, a construction module 302, an iteration module 303 and a balance module 304, wherein:
[0094] An acquisition module 300 is used to acquire the state information of each server node in the multi-agent system at a first moment;
[0095] A determination module 301, configured to determine a total first state difference between each server node and all adjacent server nodes according to the state information at the first moment;
[0096] A construction module 302, configured to construct a performance indicator function of each server node according to the total difference of the first state corresponding to each server node;
[0097] Iteration module 303, used for iteratively updating the performance indicator function of each server node until the iteration stop condition is met, and determining the optimal strategy function according to the performance indicator function obtained by the last iteration;
[0098] The balancing module 304 is used to load balance the multi-agent system according to the optimal strategy function.
[0099] In some embodiments, the iteration module 303 is further configured to:
[0100] The performance indicator function of each server node is iteratively updated using a reinforcement learning algorithm until the difference between the performance indicator functions obtained from two previous iterations is less than a stopping iteration threshold.
[0101] In some embodiments, the iteration module 303 is further configured to:
[0102] Determine the state information of each server node at a second moment; the second moment is a moment after the first moment;
[0103] Determine, based on the state information at the second moment, a total difference in the second state between each server node and all adjacent server nodes;
[0104] Determine the reward function corresponding to each server node according to the total difference of the first state and the total difference of the second state corresponding to each server node;
[0105] According to the reward function corresponding to each server node, the performance indicator function of each server node is iteratively updated.
[0106] In some embodiments, the iteration module 303 is further configured to:
[0107] Obtaining behavior information of each server node at a first moment;
[0108] The state information and behavior information of each server node at the first moment are processed using a dynamic equation to obtain the state information of each server node at the second moment.
[0109] In some embodiments, the determining module 301 is further configured to:
[0110] Based on the status information at the first moment, the status difference between each server node and each adjacent server node is determined; for each server node, the status difference values corresponding to the server node are accumulated to obtain the first total status difference between each server node and all adjacent server nodes.
[0111] In some embodiments, the building module 302 is further used to:
[0112] Obtaining a policy function of each server node at a first moment;
[0113] A performance indicator function of each server node is constructed according to the first state total difference corresponding to each server node and the policy function at the first moment.
[0114] In some embodiments, the balancing module 304 is further configured to:
[0115] According to the optimal strategy function, the data to be processed is allocated to each server node to complete the load balancing of the multi-agent system.
[0116] In practical applications, the acquisition module 300, determination module 301, construction module 302, iteration module 303 and balancing module 304 can all be implemented by a processor located in an electronic device, and the processor can be at least one of an ASIC, a DSP, a DSPD, a PLD, an FPGA, a CPU, a controller, a microcontroller, and a microprocessor.
[0117] In addition, each functional module in this embodiment can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above integrated unit can be implemented in the form of hardware or software functional modules.
[0118] If the integrated unit is implemented in the form of a software function module and is not sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this embodiment is essentially or the part that contributes to the relevant technology or the whole or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) or a processor to perform all or part of the steps of the method of this embodiment. The aforementioned storage medium includes: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, and other media that can store program codes.
[0119] Specifically, a computer program instruction corresponding to a load balancing method in this embodiment can be stored on a storage medium such as a CD, a hard disk, or a USB flash drive. When the computer program instruction corresponding to a load balancing method in the storage medium is read or executed by an electronic device, any load balancing method in the aforementioned embodiments is implemented.
[0120] Based on the same technical concept as the above embodiments, see Figure 4 , which shows an electronic device 400 provided in an embodiment of the present application, which may include: a memory 401 and a processor 402; wherein,
[0121] Memory 401, used for storing computer programs and data;
[0122] The processor 402 is used to execute the computer program stored in the memory to implement any one of the load balancing methods in the foregoing embodiments.
[0123] In practical applications, the memory 401 may be a volatile memory, such as RAM; or a non-volatile memory, such as ROM, flash memory, hard disk drive (HDD) or solid-state drive (SSD); or a combination of the above types of memory, and provide instructions and data to the processor 402.
[0124] The processor 402 may be at least one of ASIC, DSP, DSPD, PLD, FPGA, CPU, controller, microcontroller, and microprocessor. It is understandable that for different load balancing devices, the electronic device used to implement the processor function may also be other, which is not specifically limited in the embodiments of the present application.
[0125] In some embodiments, the embodiments of the present application further provide a computer program product, including a computer program, which implements any one of the load balancing methods of the aforementioned embodiments when executed by a processor.
[0126] In some embodiments, the functions or modules included in the device provided in the embodiments of the present application can be used to execute the method described in the above method embodiments. The specific implementation can refer to the description of the above method embodiments. For the sake of brevity, it will not be repeated here.
[0127] The above description of various embodiments tends to emphasize the differences between the various embodiments. The same or similar aspects can be referenced to each other, and for the sake of brevity, they will not be repeated herein.
[0128] The methods disclosed in the various method embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments.
[0129] The features disclosed in the various product embodiments provided in this application can be arbitrarily combined without conflict to obtain new product embodiments.
[0130] The features disclosed in the various method or device embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments or device embodiments.
[0131] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of hardware embodiments, software embodiments, or embodiments in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage and optical storage, etc.) that contain computer-usable program code.
[0132] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0133] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.
[0134] The above are only preferred embodiments of the present application and are not intended to limit the protection scope of the present application.
Claims
1. A load balancing method, characterized in that: Applied to a multi-agent system, the multi-agent system comprising a plurality of server nodes operating in a distributed manner, the method comprising: Obtaining state information of each server node in the multi-agent system at a first moment; Determine, based on the state information at the first moment, a total first state difference between each server node and all adjacent server nodes; Constructing a performance indicator function of each server node according to the total difference of the first state corresponding to each server node; Iteratively updating the performance indicator function of each server node until a stop iteration condition is met, and determining an optimal strategy function based on the performance indicator function obtained in the last iteration; The multi-agent system is load balanced according to the optimal strategy function.
2. The method according to claim 1, characterized in that The iterative updating of the performance indicator function of each server node until a stop iteration condition is met includes: The performance indicator function of each server node is iteratively updated using a reinforcement learning algorithm until the difference between the performance indicator functions obtained from two previous iterations is less than a stopping iteration threshold.
3. The method according to claim 2, characterized in that The iterative updating of the performance indicator function of each server node by using a reinforcement learning algorithm includes: Determine the state information of each server node at a second moment; the second moment is a moment after the first moment; Determine, based on the state information at the second moment, a total difference in the second state between each server node and all adjacent server nodes; Determine the reward function corresponding to each server node according to the total difference of the first state and the total difference of the second state corresponding to each server node; According to the reward function corresponding to each server node, the performance indicator function of each server node is iteratively updated.
4. The method according to claim 3, characterized in that The determining the status information of each server node at the second moment includes: Obtaining behavior information of each server node at a first moment; The state information and behavior information of each server node at the first moment are processed using a dynamic equation to obtain the state information of each server node at the second moment.
5. The method according to claim 1, characterized in that The determining, based on the state information at the first moment, a first state total difference between each server node and all adjacent server nodes comprises: Determine, based on the state information at the first moment, a state difference between each server node and each adjacent server node; For each of the server nodes, each state difference value corresponding to the server node is accumulated to obtain a first state total difference value between each of the server nodes and all adjacent server nodes.
6. The method according to claim 1, characterized in that The step of constructing a performance indicator function of each server node according to the total difference of the first state corresponding to each server node includes: Obtaining a policy function of each server node at a first moment; A performance indicator function of each server node is constructed according to the first state total difference corresponding to each server node and the policy function at the first moment.
7. The method according to claim 1, characterized in that The load balancing of the multi-agent system according to the optimal strategy function comprises: According to the optimal strategy function, the data to be processed is allocated to each server node to complete the load balancing of the multi-agent system.
8. A load balancing device, characterized in that: Applied to a multi-agent system, the multi-agent system includes a plurality of server nodes running in a distributed manner, the device includes: An acquisition module, used to acquire the state information of each server node in the multi-agent system at a first moment; a determination module, configured to determine a total first state difference between each server node and all adjacent server nodes according to the state information at the first moment; A construction module, configured to construct a performance indicator function of each server node according to the total difference of the first state corresponding to each server node; An iteration module, used for iteratively updating the performance indicator function of each server node until a stop iteration condition is met, and determining an optimal strategy function according to the performance indicator function obtained by the last iteration; A balancing module is used to load balance the multi-agent system according to the optimal strategy function.
9. An electronic device, characterized in that: The device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, and the method according to any one of claims 1 to 7 is implemented when the processor executes the program.
10. A computer storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.
11. A computer program product, comprising a computer program, characterized in that The computer program implements the method according to any one of claims 1 to 7 when executed by a processor.