Load balancing monitoring method and system based on virtual node consistent hashing algorithm

Abstract

The embodiment of the application provides a load balancing monitoring method and system based on a virtual node consistent hash algorithm, and belongs to the technical field of power system networking communication. The monitoring method comprises the following steps: obtaining a client IP node and a server, mapping the client IP node and the server into a hash space by using a hash algorithm to form a virtual circular ring; calculating the total load of each server on the virtual circular ring according to formula (1); calculating the balanced load of each server according to formula (2); respectively judging whether the total load and the balanced load are greater than or equal to the corresponding preset threshold value; and issuing an alarm in the case that the total load or the balanced load is greater than or equal to the corresponding preset threshold value. The monitoring method calculates the load of each server through the constructed hash ring, realizes the monitoring of the load condition of each server, can timely issue an alarm information when the server load is too high or the balance is too poor, and improves the operation efficiency of the intranet server.

Description

Technical Field

[0001] This invention relates to the field of power system network communication technology, specifically to a load balancing monitoring method and system based on a virtual node consistent hashing algorithm. Background Technology

[0002] IT information dispatching work faces difficulties such as the difficulty in unified collection and monitoring of operational data, and the difficulty in effectively commanding emergency work. Maintenance work also faces difficulties in troubleshooting.

[0003] For intranets, multiple servers often share the access needs of intranet users. However, due to the lack of server load monitoring, significant differences in load between different servers frequently occur during intranet operation, leading to intranet communication congestion. Summary of the Invention

[0004] The purpose of this invention is to provide a load balancing monitoring method and system based on a virtual node consistent hashing algorithm, which can improve the operating efficiency of intranet servers.

[0005] To achieve the above objectives, embodiments of the present invention provide a load balancing monitoring method based on a virtual node consistent hashing algorithm, comprising:

[0006] Obtain the customer's IP node and server, and use a hash algorithm to map the customer's IP node and server to a hash space to form a virtual ring;

[0007] The total load of each server on the virtual ring is calculated according to formula (1).

[0008]

[0009] in, The total load of the server, l new To monitor the new load on the system, Let s be the load of the nth node of the server, and s be the number of nodes of the server.

[0010] The load balancing of each server on the virtual ring is calculated according to formula (2).

[0011]

[0012] in, For the balanced load;

[0013] Determine whether the total load and the balanced load are greater than or equal to the corresponding preset thresholds;

[0014] If the total load or balanced load is determined to be greater than or equal to the corresponding preset threshold, an alarm will be issued.

[0015] Optionally, the client IP nodes and servers are obtained, and a hash algorithm is used to map the client IP nodes and servers to a hash space to form a virtual ring, including:

[0016] Determine the type of hash function and the range of the hash space;

[0017] The number of nodes to be mapped is determined based on the number of customer IP nodes and servers;

[0018] The padding process is performed based on the number of nodes to be mapped and the range of the hash space;

[0019] Divide the hash space after padding into blocks and calculate the hash value of each block.

[0020] The nodes to be mapped are mapped to each block in the hash space to obtain the virtual ring.

[0021] Optionally, the hash space has a range of 512, and the hash function is the SHA-256 function.

[0022] Optionally, the number of nodes to be mapped is determined based on the number of client IP nodes and servers, including:

[0023] The device name, management IP address, serial number, and device installation location of the hardware devices corresponding to the client IP node and the server node are sequentially associated and combined using the first string.

[0024] The device uptime, device online time, and most recent startup time are associated and combined using a second string with a timestamp.

[0025] Combine the device's load type and master device status information using a third string;

[0026] The results of the first string association combination, the second string association combination, and the third string association combination are combined and associated to obtain the association information;

[0027] The associated information is input into the autoencoder to obtain the feature representation of the corresponding hidden layer;

[0028] The feature representation is hashed using a hash function to obtain the corresponding hash value;

[0029] The nodes to be mapped are assigned to each block in the hash space based on the hash value.

[0030] Optionally, the association information is input into the autoencoder to obtain the feature representation of the corresponding hidden layer, including:

[0031] The feature representation is determined according to formula (3).

[0032] ξ=f(Wx+b), (3)

[0033] Where ξ is the feature representation, f is the activation function of the autoencoder, W is the weight matrix, x is the association information input to the autoencoder, and b is the bias vector.

[0034] Optionally, the hash space after padding is divided into blocks, and the hash value of each block is calculated separately, including:

[0035] The hash value is calculated according to formula (4).

[0036]

[0037] Among them, H (i) H is the hash value of the i-th block. (i-1) Let i be the hash value of the (i-1)th block. For block M (i) The compression function.

[0038] On the other hand, the present invention also provides a load balancing monitoring system based on a virtual node consistent hashing algorithm, the monitoring system including a processor configured to execute any of the monitoring methods described above.

[0039] In another aspect, the present invention also provides a computer-readable storage medium storing instructions for being read by a machine to cause the machine to perform any of the monitoring methods described above.

[0040] Through the above technical solution, the load balancing monitoring method and system based on virtual nodes to execute hash algorithms provided by this invention constructs a hash ring to set the positions of each server and user IP node, then calculates the balanced load and total load of each server, and finally combines the threshold judgments of the balanced load and total load to achieve the assessment of the load status of each server in the current intranet. Compared with the prior art, the monitoring method provided by this application calculates the load of each server by constructing a hash ring, realizing the monitoring of the load status of each server, so that alarm information can be issued in a timely manner when the server load is too high or the balance is poor, thereby improving the operating efficiency of intranet servers.

[0041] Other features and advantages of the embodiments of the present invention will be described in detail in the following detailed description section. Attached Figure Description

[0042] The accompanying drawings are provided to further illustrate embodiments of the present invention and form part of the specification. They are used together with the following detailed description to explain the embodiments of the present invention, but do not constitute a limitation thereof. In the drawings:

[0043] Figure 1 This is a flowchart of a load balancing monitoring method based on virtual nodes to perform hash algorithms according to an embodiment of the present invention;

[0044] Figure 2 This is a flowchart of a method for constructing a virtual ring according to an embodiment of the present invention;

[0045] Figure 3 This is a flowchart of a method for determining a node to be mapped according to an embodiment of the present invention;

[0046] Figure 4 This is an example diagram of a virtual ring according to one embodiment of the present invention. Detailed Implementation

[0047] The specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are for illustration and explanation only and are not intended to limit the scope of the present invention.

[0048] like Figure 1 The diagram shows a flowchart of a load balancing monitoring method based on a virtual node consistent hashing algorithm according to an embodiment of the present invention. Figure 1 In this context, the monitoring method may include:

[0049] In step S10, the client IP node and server are obtained, and a hash algorithm is used to map the client IP node and server into a hash space to form a virtual ring;

[0050] In step S11, the total load of each server on the virtual ring is calculated according to formula (1).

[0051]

[0052] in, For the total load of the server, l new To monitor the new load on the system, Let s be the load on the nth node of the server, and s be the number of nodes on the server.

[0053] In step S12, the load balancing of each server on the virtual ring is calculated according to formula (2).

[0054]

[0055] in, To balance the load;

[0056] In step S13, it is determined whether the total load and the balanced load are greater than or equal to the corresponding preset thresholds.

[0057] In step S14, if the total load or balanced load is determined to be greater than or equal to the corresponding preset threshold, an alarm is issued.

[0058] In such Figure 1 In the method shown, step S10 is used to obtain the client IP node and server, and construct a virtual ring. The server is used to support the access needs of the client IP node. The specific method for constructing the virtual ring can be various methods known to those skilled in the art. In one example of the present invention, the method for constructing the virtual ring may include, for example... Figure 2 The steps shown are described. Figure 2 In this context, the method may include:

[0059] In step S20, the type of hash function and the range of the hash space are determined. The type of hash function can be, but is not limited to, various types known to those skilled in the art; correspondingly, the range of the hash space can vary depending on the type of hash function. In one example of the present invention, the hash function can be the SHA-256 function, and correspondingly, the range of the hash space can be 512, that is, 0 to 2^32. 32 -1.

[0060] In step S21, the number of nodes to be mapped is determined based on the number of client IP nodes and servers. In this embodiment, typically one client IP node or server is considered as one node to be mapped. However, considering the differences in load capacity among servers, if the same method is uniformly applied to the subsequent allocation of client IP nodes for servers with larger and smaller load capacities, the system will obviously fail to achieve load balancing. Therefore, in one example of the present invention, when determining the number of nodes to be mapped, servers with larger load capacities can be allocated as multiple nodes to be mapped. Specifically, a unit load capacity can be preset first, and then servers with larger load capacities can be split into multiple nodes to be mapped based on this unit load capacity. This allows for equivalent calculation of servers with larger and smaller load capacities when calculating load balancing and total load in subsequent calculations, reducing computational complexity and improving system response efficiency.

[0061] Because the locations of each server and client IP node on the virtual ring need to be allocated based on their own attributes, this ensures a balanced and reasonable allocation of the nearest server when the virtual ring subsequently allocates access servers. Under this premise, when determining the node to be mapped, it is necessary to consider the attributes of the server and client IP nodes. These attributes can be of various types known to those skilled in the art. In a preferred embodiment of this invention, these attributes can be divided into three categories: first, attributes of the hardware device itself, namely: device name, management IP address, serial number, and device installation location; second, the hardware device's operating time, namely: device operating time, device online time, and most recent startup time; and third, the device's current status, namely: device load type and master device status information. Corresponding to these three types of attributes, step S21 can further include, for example... Figure 3 The steps shown are described. Figure 3 In this context, step S21 may further include the following steps:

[0062] In step S30, the device name, management IP address, serial number, and device installation location of the hardware devices corresponding to the client IP node and the server node are sequentially associated and combined using the first string.

[0063] In step S31, the device running time, device online time, and most recent startup time are combined and associated with a second string using timestamps;

[0064] In step S32, the device load type and master device status information are combined using a third string association.

[0065] In step S33, the results of the first string association combination, the second string association combination, and the third string association combination are combined and associated to obtain the association information;

[0066] In step S34, the association information is input into the autoencoder to obtain the feature representation of the corresponding hidden layer. After determining the association information, since the association information is long and contains various types of encodings, the feature information of the association information can be obtained through the autoencoder, thus facilitating subsequent hash calculation. Specifically, step S34 can be performed by determining the feature representation according to formula (3).

[0067] ξ=f(Wx+b), (3)

[0068] Where ξ is the feature representation, f is the activation function of the autoencoder, W is the weight matrix, x is the association information input to the autoencoder, and b is the bias vector.

[0069] In step S35, the feature representation is hashed using a hash function to obtain the corresponding hash value;

[0070] In step S36, the nodes to be mapped are assigned to each block in the hash space based on their hash values.

[0071] In step S22, padding is performed based on the number of nodes to be mapped and the range of the hash space. Since the number of client IP nodes and servers may not be able to achieve complete coverage, i.e., the number of user IP nodes and servers is generally less than 512, virtual nodes need to be padded to ensure that the nodes to be mapped meet the requirements.

[0072] In step S23, the hash space after padding is divided into blocks, and the hash value of each block is calculated. Specifically, the hash value can be calculated using formula (4).

[0073]

[0074] Among them, H (i) H is the hash value of the i-th block. (i-1) Let i be the hash value of the (i-1)th block. For block M (i) The compression function.

[0075] In step S24, the nodes to be mapped are mapped to each block in the hash space to obtain a virtual ring.

[0076] After the virtual ring is constructed through step S10 (for example) Figure 4 As shown in the diagram (where A, B, C, and D represent server nodes), when any user needs to access the server, a hash calculation is first performed on that user to obtain the corresponding hash code. Then, the corresponding position is found on the virtual ring. Next, based on the currently found position, the nearest node representing the server is searched clockwise. Since each server node is evenly distributed after hash calculation, subsequent node allocation ensures both finding the nearest server and an even distribution of nearby servers, thus avoiding imbalances.

[0077] On the other hand, the present invention also provides a load balancing monitoring system based on a virtual node consistent hashing algorithm, the monitoring system including a processor configured to execute any of the monitoring methods described above.

[0078] In another aspect, the present invention also provides a computer-readable storage medium storing instructions for being read by a machine to cause the machine to perform any of the monitoring methods described above.

[0079] Through the above technical solution, the load balancing monitoring method and system based on virtual nodes to execute hash algorithms provided by this invention constructs a hash ring to set the positions of each server and user IP node, then calculates the balanced load and total load of each server, and finally combines the threshold judgments of the balanced load and total load to achieve the assessment of the load status of each server in the current intranet. Compared with the prior art, the monitoring method provided by this application calculates the load of each server by constructing a hash ring, realizing the monitoring of the load status of each server, so that alarm information can be issued in a timely manner when the server load is too high or the balance is poor, thereby improving the operating efficiency of intranet servers.

[0080] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0081] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0082] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0083] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0084] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0085] Memory may include non-persistent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0086] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0087] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0088] The above are merely embodiments of this application and are not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A load balancing monitoring method based on a virtual node consistent hashing algorithm, characterized in that, The monitoring method includes: Obtain the customer's IP node and server, and use a hash algorithm to map the customer's IP node and server to a hash space to form a virtual ring; Calculate the total load of each server on the virtual ring according to formula (1). ，（1） in, The total load of the server. To monitor the new load on the system, For the server's first The load of each node, The number of nodes on the server; The load balancing of each server on the virtual ring is calculated according to formula (2). ，（2） in, For the balanced load; Determine whether the total load and the balanced load are greater than or equal to the corresponding preset thresholds; If the total load or balanced load is determined to be greater than or equal to the corresponding preset threshold, an alarm will be issued. Obtain the client's IP address and server information, and use a hash algorithm to map the client's IP address and server information to a hash space to form a virtual ring, including: Determine the type of hash function and the range of the hash space; The number of nodes to be mapped is determined based on the number of customer IP nodes and servers, including: presetting a unit load capacity, and then splitting servers with large load capacity into multiple nodes to be mapped based on the unit load capacity; The padding process is performed based on the number of nodes to be mapped and the range of the hash space; Divide the hash space after padding into blocks and calculate the hash value of each block. The nodes to be mapped are mapped to each block in the hash space to obtain the virtual ring; The number of nodes to be mapped is determined based on the number of client IP nodes and servers, including: The device name, management IP address, serial number, and device installation location of the hardware devices corresponding to the client IP node and the server node are sequentially associated and combined using the first string. The device uptime, device online time, and most recent startup time are associated and combined using a second string with a timestamp. Combine the device's load type and master device status information using a third string; The results of the first string association combination, the second string association combination, and the third string association combination are combined and associated to obtain the association information; The associated information is input into the autoencoder to obtain the feature representation of the corresponding hidden layer; The feature representation is hashed using a hash function to obtain the corresponding hash value; The nodes to be mapped are assigned to each block in the hash space based on the hash value.

2. The monitoring method according to claim 1, characterized in that, The hash space has a range of 512, and the hash function is the SHA-256 function.

3. The monitoring method according to claim 1, characterized in that, The associated information is input into the autoencoder to obtain the feature representation of the corresponding hidden layer, including: The feature representation is determined according to formula (3). ，（3） in, The feature is represented as follows. This is the activation function of the autoencoder. This is the weight matrix. To input the association information of the auto encoder, This is the bias vector.

4. The monitoring method according to claim 1, characterized in that, The padded hash space is divided into blocks, and the hash value of each block is calculated, including: The hash value is calculated according to formula (4). ，（4） in, For the first The hash value of each block, For the first The hash value of each block, For block The compression function.

5. A load balancing monitoring system based on a virtual node consistent hashing algorithm, characterized in that, The monitoring system includes a processor configured to perform the monitoring method as described in any one of claims 1 to 4.

6. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores instructions that are read by a machine to cause the machine to perform the monitoring method as described in claims 1 to 4.

Images