Load balancing method and system for a server cluster
By using request allocation based on session ID and generating custom SessionIDs, the problem of balancing session persistence and load balancing in Nginx load balancing strategies is solved, achieving uniform traffic distribution and session continuity, and reducing system complexity and overhead.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- NORTH CLOUD VIEW TECH (BEIJING) CO LTD
- Filing Date
- 2025-09-25
- Publication Date
- 2026-07-24
AI Technical Summary
In existing technologies, Nginx's load balancing strategy struggles to balance server load and session persistence, leading to load imbalance or session discontinuity, which increases system hardware and software overhead and complexity.
By using a request allocation method based on session ID, it ensures that requests from the same user are always forwarded to the same backend service node. Combined with the default load balancing strategy, it achieves a balance between session persistence and load balancing. Custom SessionID generation and hash calculation are used to control the hash result and avoid load skew.
This ensures that requests from the same user are always forwarded to the same backend service node, maintaining session continuity. At the same time, it reduces the dependence on distributed session storage, lowers system complexity and network overhead, and improves load balancing efficiency and user experience.
Smart Images
Figure CN121125732B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of server performance optimization technology, and in particular to a load balancing method and system for server clusters. Background Technology
[0002] Against the backdrop of accelerated global digital transformation, the user base of information systems is growing exponentially, and traditional single-node web server architectures can no longer support millions of concurrent requests. Multi-node distributed web server clusters have become the standard solution in the industry for dealing with high concurrency and high availability requirements. Nginx (engine x), with its high-performance event-driven architecture (a single instance can handle 50,000+ concurrent connections) and intelligent traffic scheduling capabilities, has become an indispensable traffic scheduling hub in modern web server clusters by achieving precise request distribution through reverse proxy (supporting 7 load balancing algorithms), dynamic health checks (failover latency <200ms), and static / dynamic content separation optimization.
[0003] To achieve effective traffic scheduling, Nginx uses a round-robin strategy for the simplest load balancing by default; under the round-robin strategy, each request is distributed to different backend servers sequentially. Existing technologies also offer several auxiliary methods to improve user access efficiency. For example, to ensure that requests from the same user are always sent to the same server, an IP hash strategy can be used to distribute requests, ensuring that requests from the same client IP address are assigned to the same server. Additionally, a least-connections strategy can be used to distribute the load, assigning new requests to the backend server with the fewest connections; or different weights can be assigned to backend servers, with higher-performing servers receiving more requests, thus achieving load balancing based on server processing capacity.
[0004] However, existing load balancing methods that rely on client IP addresses to allocate requests can lead to a concentration of requests from numerous different users accessing the same gateway (e.g., enterprise users) onto the same server, resulting in load skew. Furthermore, this method cannot dynamically adjust for load imbalances already present in the server cluster. On the other hand, if requests are allocated independently, such as through round-robin, least connections, or weighted strategies, the lack of session persistence means that multiple requests from the same user may be distributed across different servers, actually reducing efficiency and user experience. Using external session storage mechanisms to optimize these strategies requires distributed storage and synchronization management, significantly increasing system hardware and software overhead, impacting system cost, complexity, and performance. Summary of the Invention
[0005] Based on the above analysis, the embodiments of the present invention aim to provide a load balancing method and system for server clusters, so as to solve the problem that the load balancing strategy in the prior art is difficult to balance server load and session persistence.
[0006] On one hand, embodiments of the present invention provide a load balancing method for a server cluster, the method comprising the following steps:
[0007] Receive the client's access request, extract the SessionID from the access request, and determine the validity period of the SessionID;
[0008] When the SessionID does not exist or has expired, the Web application server is notified to generate a new SessionID for the access request, and the service node number specified by the current first load balancing strategy is obtained. The hash result of the new SessionID is then determined to point to the service node number.
[0009] If the hash result of the new SessionID does not point to the service node number, the new SessionID for the access request is generated repeatedly until the hash result of the new SessionID points to the service node number.
[0010] When the SessionID already exists, or when the hash result of the new SessionID points to the service node number, the Nginx server executes the default load balancing strategy.
[0011] Based on a further improvement of the above method, the default load balancing strategy is a hash strategy based on SessionID, and the hash strategy performs the same hash calculation as when determining the hash result of the new SessionID.
[0012] Based on a further improvement to the above method, the first load balancing strategy is a round-robin strategy, a least-connections strategy, or a weight allocation strategy; wherein,
[0013] The polling strategy selects the next service node as the designated service node in an average distribution manner.
[0014] The least connections strategy selects the service node with the lowest current load as the designated service node;
[0015] The weight allocation strategy selects the service node with the lightest overall load as the designated service node.
[0016] Based on a further improvement of the above method, the Nginx server is deployed as a proxy server between multiple clients and multiple service nodes.
[0017] A further improvement to the above method is that the SessionID is extracted from the Cookie in the access request header.
[0018] A further improvement to the above method is characterized in that the Web application server is a Tomcat server.
[0019] Based on a further improvement of the above method, a service for custom-generating new SessionIDs is configured in the Tomcat server.
[0020] On the other hand, embodiments of the present invention provide a load balancing system for a server cluster, the system comprising:
[0021] The SessionID extraction module is used to receive access requests from clients, extract the SessionID from the access requests, and determine the timeliness of the SessionID.
[0022] The new SessionID generation module is used to notify the Web application server to generate a new SessionID for the access request when the SessionID does not exist or has expired, and to obtain the service node number specified by the current first load balancing strategy, and determine whether the hash result of the new SessionID points to the service node number.
[0023] The collision matching module is used to repeatedly generate a new SessionID for the access request when the hash result of the new SessionID does not point to the service node number, until the hash result of the new SessionID points to the service node number.
[0024] The load balancing module is used to enable the Nginx server to execute the default load balancing strategy when the SessionID already exists in the Nginx server or the hash result of the new SessionID points to the service node number.
[0025] Based on further improvements to the above system, the Nginx server is deployed as a proxy server between multiple clients and multiple service nodes.
[0026] Based on further improvements to the above system, the default load balancing strategy is a hash strategy based on SessionID, and the hash strategy performs the same hash calculation as when determining the hash result of the new SessionID.
[0027] Compared with the prior art, the embodiments of the present invention provide a load balancing method and system for server clusters. By allocating requests based on session ID, it ensures that requests from the same user are always forwarded to the same backend service node, while ensuring that the default load balancing strategy actually executed is effectively directed to the service node selected / specified by the first load balancing strategy, thus achieving traffic distribution that balances session persistence and load balancing.
[0028] In this invention, the above-described technical solutions can be combined with each other to achieve more preferred combinations. Other features and advantages of this invention will be set forth in the following description, and some advantages may become apparent from the description or be learned by practicing the invention. The objects and other advantages of this invention can be realized and obtained from what is particularly pointed out in the description and drawings. Attached Figure Description
[0029] The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Throughout the drawings, the same reference numerals denote the same parts.
[0030] Figure 1 This is a schematic diagram of a load balancing method for a server cluster in one embodiment of the present invention;
[0031] Figure 2 This is a schematic diagram of the system service architecture in one embodiment of the present invention. Detailed Implementation
[0032] Preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings, which form part of this application and are used together with the embodiments of the present invention to illustrate the principles of the present invention, but are not intended to limit the scope of the present invention.
[0033] Nginx provides a high-performance, flexible, and scalable solution for web servers and proxy servers, and is adopted by the vast majority of large server clusters. Nginx also provides some default load balancing solutions for server clusters, but existing solutions do not provide ideal support for session persistence. IP hashing strategies, which can partially support session persistence, struggle to achieve ideal load balancing. Existing technologies also include solutions that store session IDs on intermediate servers and perform load balancing based on the hash value of these session IDs. However, this requires additional hardware and software overhead, resulting in high system costs. Furthermore, session IDs are randomly generated, and their generation strategy only guarantees uniqueness, not the uniformity of hash results (especially when the hashing strategy is unclear). In extreme cases, a large number of different session IDs may be assigned to the same server due to identical hash results, leading to load skew.
[0034] In view of this, a specific embodiment of the present invention discloses a load balancing method for a server cluster, such as... Figure 1 As shown, the method includes the following steps:
[0035] S11, Receive the client's access request, extract the SessionID from the access request, and determine the validity period of the SessionID;
[0036] S12, when the SessionID does not exist or has expired, notify the Web application server to generate a new SessionID for the access request, obtain the service node number specified by the current first load balancing strategy, and determine whether the hash result of the SessionID points to the service node number.
[0037] S13, when the hash result of the new SessionID does not point to the service node number, the new SessionID of the access request is generated repeatedly until the hash result of the new SessionID points to the service node number;
[0038] S14, when the SessionID already exists, or the hash result of the new SessionID points to the service node number, the Nginx server executes the default load balancing strategy.
[0039] The technical solution of this invention provides a load balancing scheme that balances session persistence. By allocating requests based on session IDs, it ensures that requests from the same user are always forwarded to the same backend service node. This scheme effectively maintains user state consistency while reducing reliance on distributed session storage, thus lowering the complexity and network overhead of inter-cluster session synchronization. Furthermore, this scheme allows the default load balancing strategy to be effectively directed to the service node selected / specified by the first load balancing strategy, thereby simultaneously gaining the advantages of both load balancing strategies.
[0040] Based on further improvements to the above method, the first load balancing strategy is a round-robin strategy, a least-connections strategy, or a weighted allocation strategy. The first load balancing strategy is used to balance the load among multiple service nodes. It sequentially selects the next service node through an average distribution method using a round-robin strategy, selects the service node with the lowest current load through a least-connections strategy, or selects the service node with the lightest overall load through a weighted allocation strategy. This ensures that requests to be allocated are reasonably and evenly distributed to the specified service nodes, ensuring load balancing across nodes, avoiding overload on a single server node, and thus improving the overall stability and throughput of the cluster service.
[0041] like Figure 2As shown, the system is divided into multiple layers. The Nginx server, acting as a proxy server, is deployed between n clients and N service nodes (forming a server cluster). The first load balancing module is implemented by a service at the lowest level of the entire system. This service can be either a standalone service or a business service. This service only needs to provide an interface to calculate the number of the service node to which the next request should be assigned, and then return the number of the service node selected by the first load balancing strategy as the specified service node number. The calculation method can be either round-robin or based on server load. A simple round-robin strategy can be implemented as follows: The system service uses two variables to control the return result of the round-robin strategy. The first variable records the total number of allocable service nodes, which generally does not change after being determined based on the cluster configuration. All service nodes are sequentially numbered starting from 0 according to the cluster configuration. In a cluster with a total of N service nodes, the first service node is numbered 0, ..., and the last service node is numbered N-1. The second variable counts the requests that call the interface (i.e., request the allocation of service nodes), with an initial value of 0. When the calculation interface provided by the system service is requested again, the count of the second variable is incremented by 1. Then, it is checked whether the current value of the second variable is equal to the total number of service nodes. If they are not equal, the return value of (second variable - 1) is used as the designated service node number; if they are equal, the return value of (second variable - 1) is used as the designated service node number, and the second variable is reassigned to 0. In this way, the returned designated service node numbers cycle sequentially between 0 and N-1, ensuring that the number of requests allocated to each service node is the same.
[0042] Based on a further improvement of the above method, the default load balancing strategy is a hash strategy based on SessionID, which is the same hash calculation performed when judging the hash result in steps S12 and S13. This method allows all access requests to be distributed among service nodes based on the hash result of SessionID. On the one hand, it ensures that the same SessionID is assigned to the same service node, achieving session persistence and improving service response efficiency and user experience. On the other hand, it ensures that the hash result based on SessionID points to the service node selected / specified by the first load balancer. That is, the distribution of access requests also satisfies the uniformity and rationality of the round-robin strategy, the least connections strategy, or the weight distribution strategy, so that the load balancing scheme can have the advantages of both load balancing strategies.
[0043] A further improvement to the above method is that the SessionID is extracted from the Cookie in the access request header.
[0044] Based on a further improvement of the above method, the web application server is a Tomcat server.
[0045] Normally, the initial access request initiated by a user client generates a random but unique SessionID from the Tomcat server. The Tomcat server saves this SessionID and returns it to the client. The client also stores this SessionID in its browser cookie. Subsequent access requests from the same client based on the same session will include this cookie in the request header, helping the Nginx server to send consecutive requests to the same service node to maintain session persistence. If the session is closed by the client, or if no consecutive access requests are initiated for an extended period, the Tomcat server will mark the SessionID as expired and will no longer enforce session persistence for that SessionID. When it receives a non-existent or expired SessionID, it will treat the access request as a new access request.
[0046] This invention provides a hashing strategy based on SessionID. However, since the generation of SessionID is random, its corresponding hash result is also random. Relying solely on the hashing strategy cannot achieve true load balancing for the server cluster. The generation of SessionID is completely random, and the hash calculation is irreversible; both are uncontrollable data processing methods, and neither algorithm can be directly controlled to immediately obtain the desired result. Therefore, this invention further utilizes a collision vulnerability to repeatedly generate new SessionIDs and repeatedly calculate the hash result. By using a sufficient number of new SessionIDs to collide and obtain the desired hash result, a new SessionID matching the ideal hash result (i.e., the specified service node number) can be obtained in a largely controllable manner, avoiding the randomness of SessionIDs from affecting reasonable load distribution.
[0047] A further improvement to the above method is to configure a custom service / program method for repeatedly generating SessionIDs in the Tomcat server. For example, a Java class named `CustomSessionIdGenerator` can be written, defining a function to repeatedly generate SessionIDs until a hash collision is satisfied. The class directory of this custom class can be set to `org.apache.catalina.util.CustomSessionIdGenerator`, and references to this class directory and the custom class method can be added to the `context.xml` file in the Tomcat configuration directory to implement customized SessionID generation in the Tomcat server.
[0048] Furthermore, for example, a typical hash calculation can be a modulo / remainder calculation. In a cluster with a total of N service nodes, the SessionID string is modulo N to obtain the remainder of the SessionID divided by N. This remainder is used as the hash result. It is then checked whether this remainder is the same as the service node number specified by the current first load balancing strategy. If they are the same, the access request is directly sent to the corresponding service node for processing according to the remainder / number. If they are different, a new SessionID is generated and the modulo / remainder calculation is repeated until the remainder is the same as the service node number. Then, the access request is sent to the corresponding service node for processing according to the remainder / number. For example, if the remainder is 0, the access request is sent to the first service node with the number 0, and so on, until the remainder is N-1, at which point the access request is sent to the last service node with the number N-1.
[0049] Compared with existing methods, the embodiments of the present invention provide a traffic scheduling scheme that balances session persistence and load balancing, and have the following significant advantages:
[0050] 1. Ensure even distribution of traffic: The hash result of the new SessionID is consistent with the goal of the first load balancing strategy, thereby ensuring that new access requests are evenly distributed to different service nodes.
[0051] 2. Ensure session stickiness: The SessionID that has been saved and has not expired remains unchanged, ensuring that continuous requests from the same session are sent to the same service node, effectively achieving session persistence.
[0052] 3. High code extensibility: It provides a service / program method for customizing and repeatedly generating SessionIDs. The system can be extended by simply configuring a reference in the Tomcat server.
[0053] On the other hand, embodiments of the present invention provide a load balancing system for a server cluster, the system comprising:
[0054] The SessionID extraction module is used to receive access requests from clients, extract the SessionID from the access requests, and determine the timeliness of the SessionID.
[0055] The new SessionID generation module is used to notify the Web application server to generate a new SessionID for the access request when the SessionID does not exist or has expired, and to obtain the service node number specified by the current first load balancing strategy, and determine whether the hash result of the new SessionID points to the service node number.
[0056] The collision matching module is used to repeatedly generate a new SessionID for the access request when the hash result of the new SessionID does not point to the service node number, until the hash result of the new SessionID points to the service node number.
[0057] The load balancing module is used to enable the Nginx server to execute the default load balancing strategy when the SessionID already exists or the hash result of the new SessionID points to the service node number.
[0058] Based on further improvements to the above system, the default load balancing strategy is a hash strategy based on SessionID, and the hash strategy performs the same hash calculation as when determining the hash result of the new SessionID.
[0059] Through the technical solutions in the above examples, this embodiment of the invention provides a load balancing method and system for server clusters. By allocating requests based on session IDs, it ensures that requests from the same user are always forwarded to the same backend service node. At the same time, it ensures that the default load balancing strategy is effectively directed to the service node selected / specified by the first load balancing strategy, thus achieving traffic distribution that balances session persistence and load balancing.
[0060] Those skilled in the art will understand that all or part of the processes of the methods described in the above embodiments can be implemented by a computer program instructing related hardware, and the program can be stored in a computer-readable storage medium. The computer-readable storage medium may be a disk, optical disk, read-only memory, or random access memory, etc.
[0061] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.
Claims
1. A load balancing method for a server cluster, characterized in that, The method includes the following steps: Receive the client's access request, extract the SessionID from the access request, and determine the validity period of the SessionID; When the SessionID does not exist or has expired, the Web application server is notified to generate a new SessionID for the access request, and the service node number specified by the current first load balancing strategy is obtained. The hash result of the new SessionID is then determined to point to the service node number. If the hash result of the new SessionID does not point to the service node number, the new SessionID for the access request is generated repeatedly until the hash result of the new SessionID points to the service node number. When the SessionID already exists, or when the hash result of the new SessionID points to the service node number, the Nginx server executes the default load balancing strategy.
2. The method according to claim 1, characterized in that, The default load balancing strategy is a hash strategy based on SessionID, which performs the same hash calculation as when determining the hash result of the new SessionID.
3. The method according to claim 1, characterized in that, The first load balancing strategy is a round-robin strategy, a least-connections strategy, or a weight allocation strategy; wherein, The polling strategy selects the next service node as the designated service node in an average distribution manner. The least connections strategy selects the service node with the lowest current load as the designated service node; The weight allocation strategy selects the service node with the lightest overall load as the designated service node.
4. The method according to claim 1, characterized in that, The Nginx server is deployed as a proxy server between multiple clients and multiple service nodes.
5. The method according to claim 1, characterized in that, In the method, the SessionID is extracted from the Cookie in the access request header.
6. The method according to claim 1, characterized in that, The web application server is a Tomcat server.
7. The method according to claim 6, characterized in that, Configure a custom service to repeatedly generate new SessionIDs in the Tomcat server.
8. A load balancing system for a server cluster, characterized in that, The system includes: The SessionID extraction module is used to receive access requests from clients, extract the SessionID from the access requests, and determine the timeliness of the SessionID. The new SessionID generation module is used to notify the Web application server to generate a new SessionID for the access request when the SessionID does not exist or has expired, and to obtain the service node number specified by the current first load balancing strategy, and determine whether the hash result of the new SessionID points to the service node number. The collision matching module is used to repeatedly generate a new SessionID for the access request when the hash result of the new SessionID does not point to the service node number, until the hash result of the new SessionID points to the service node number. The load balancing module is used to enable the Nginx server to execute the default load balancing strategy when the SessionID already exists in the Nginx server or the hash result of the new SessionID points to the service node number.
9. The system according to claim 8, characterized in that, The Nginx server is deployed as a proxy server between multiple clients and multiple service nodes.
10. The system according to claim 8, characterized in that, The default load balancing strategy is a hash strategy based on SessionID, which performs the same hash calculation as when determining the hash result of the new SessionID.