Load balancing method, device and equipment and computer readable storage medium

Abstract

The invention discloses a load balancing method, device and equipment and a computer readable storage medium, and is applied to the technical field of computers, and the method comprises the steps: determining a target feature of each node in a back-end server cluster based on back-end server data, NGINX server data and NGINX access log data; performing standardization processing based on each target feature, and performing combination to obtain a combined feature; generating an optimal load balancing weight strategy through a dynamic load balancing model based on the combined features, and carrying out load balancing processing on nodes in the back-end server cluster based on the optimal load balancing weight strategy; the dynamic load balancing model is a machine learning model based on a gradient lifting decision tree and a model for dynamically determining node weights by a deep Q network. According to the load balancing method, the load balancing weight strategy is determined based on the machine learning model of the gradient lifting decision tree and the deep Q network, so that the weight has the characteristics of predictability and real-time performance, and the accuracy of load balancing is improved.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to a load balancing method, apparatus, device, and computer-readable storage medium. Background Technology

[0002] Because existing technologies rely on a static, round-robin algorithm for traffic distribution decisions, and their eviction policies are passively triggered and lack predictability, they cannot perceive or adapt to the macro-level trends of system operation. When the system faces a traffic surge, causing indicators such as CPU load and P95 latency to exceed thresholds, a fixed adjustment strategy that does not consider historical patterns is passively triggered. This makes its behavior disconnected from the overall system state, preventing proactive congestion avoidance through predictive traffic scheduling.

[0003] It is evident that improving the accuracy of load balancing is a technical problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0004] In view of this, the purpose of the present invention is to provide a load balancing method, apparatus, device and computer-readable storage medium, which solves the technical problem of inaccurate load balancing in the prior art.

[0005] To address the aforementioned technical problems, this invention provides a load balancing method, comprising: Align the backend server data, NGINX server data, and NGINX access log data to obtain the aligned data. The target features of each node in the backend server cluster are determined based on the aligned data; wherein, the target features include node local features and node global features; Each target feature is standardized to obtain a standardized feature, and the standardized features of all nodes are combined to obtain a combined feature; Based on the combined features, an optimal load balancing weight strategy is generated through a dynamic load balancing model, and load balancing is performed on the nodes in the backend server cluster based on the optimal load balancing weight strategy; wherein, the dynamic load balancing model is a model that dynamically determines the node weights using a gradient boosting decision tree machine learning model and a deep Q-network.

[0006] Optionally, based on the combined features, an optimal load balancing weight strategy is generated through a dynamic load balancing model, and load balancing is performed on the nodes in the backend server cluster based on the optimal load balancing weight strategy, including: Based on the combined features, the machine learning model based on gradient boosting decision tree is used to predict the optimal weight distribution in the current state; The combined features and the optimal weight distribution are concatenated to obtain the concatenated features; Based on the splicing features, a weight adjustment vector is generated using the deep Q-network. The optimal load balancing weight strategy is determined based on the weight adjustment vector and the optimal weight distribution.

[0007] Optionally, based on the splicing features, a weight adjustment vector is generated using the deep Q-network, including: Define an action space; wherein, the actions in the action space are K predefined, discretized N-dimensional adjustment vectors; The Q value corresponding to each action in the action space is determined based on the splicing features; The weight adjustment vector is determined based on the Q value for each action.

[0008] Optionally, based on the combined features, an optimal load balancing weight strategy is generated through a dynamic load balancing model, and load balancing is performed on the nodes in the backend server cluster based on the optimal load balancing weight strategy, including: Determine the currently effective load balancing weight strategy, determine the change between the latest optimal load balancing weight strategy and the currently effective load balancing weight strategy; The change value is compared with a set change threshold; When the change value is less than or equal to the change threshold, a regular hot update is performed based on the optimal load balancing weight strategy to perform load balancing processing on the nodes in the backend server cluster; wherein, the regular hot update is a strategy of applying the latest optimal load balancing weight strategy to all traffic. When the change value exceeds the change threshold, the optimal load balancing weight strategy is determined, and the currently effective load balancing weight strategy is adjusted using a gray-scale adjustment mechanism to obtain the adjusted optimal load balancing weight strategy. Load balancing is then performed based on the adjusted optimal load balancing weight strategy. The gray-scale adjustment mechanism involves a first proportion of traffic being load balanced based on the optimal load balancing weight strategy, and a second proportion of traffic being load balanced based on the currently effective load balancing weight strategy.

[0009] Optionally, when the change value is greater than the change threshold, based on the optimal load balancing weight strategy, the currently effective load balancing weight strategy is adjusted using a gray-scale adjustment mechanism to obtain the adjusted optimal load balancing weight strategy. After performing load balancing processing based on the adjusted optimal load balancing weight strategy, the method further includes: Determine the first performance parameter corresponding to the first proportion of traffic and the second performance parameter corresponding to the second proportion of traffic; Based on the first performance parameter and the second performance parameter, determine whether the performance of the optimal load balancing weight strategy is better than the performance of the currently effective load balancing weight strategy. When it is determined that the performance of the optimal load balancing weight strategy is better than the performance of the currently effective load balancing weight strategy, load balancing is determined to be performed based on the optimal load balancing weight strategy. When it is determined that the performance of the optimal load balancing weight strategy is not better than the performance of the currently effective load balancing weight strategy, load balancing is determined to be performed based on the currently effective load balancing weight strategy.

[0010] Optionally, after generating an optimal load balancing weight strategy based on the combined characteristics using a dynamic load balancing model, and performing load balancing processing on the nodes in the backend server cluster based on the optimal load balancing weight strategy, the method further includes: Obtain state policy data pairs; wherein, the state policy data pairs are data composed of the combined features of each period and the finally effective load balancing weight policy; The current gradient boosting decision tree-based machine learning model is trained based on the state policy data to obtain the latest gradient boosting decision tree-based machine learning model, and the latest gradient boosting decision tree-based machine learning model is used as the gradient boosting decision tree-based machine learning model in the dynamic load balancing model.

[0011] Optionally, after generating an optimal load balancing weight strategy based on the combined characteristics using a dynamic load balancing model, and performing load balancing processing on the nodes in the backend server cluster based on the optimal load balancing weight strategy, the method further includes: A replay pool is constructed based on state transition tuples; wherein, the data in the state transition tuples includes concatenated features, weight adjustment vectors, and reward values; the concatenated features are features corresponding to each time period obtained by concatenating combined features and the optimal weight distribution determined by a machine learning model based on boosting decision trees; the weight adjustment vectors are vectors corresponding to each time period determined based on the deep Q-network; the reward values ​​are reward values ​​determined based on a reward function and the performance parameter scores of the previous period; the reward function is a function constructed based on a set number of performance parameters. Based on the data in the replay pool, the evaluation network of the deep Q-network is updated by using an optimizer and a loss function to minimize the gap between the predicted Q-value and the target Q-value.

[0012] The present invention also provides a load balancing device, comprising: The alignment module is used to align backend server data, NGINX server data, and NGINX access log data to obtain aligned data. The target feature determination module is used to determine the target features of each node in the backend server cluster based on the aligned data; wherein, the target features include node local features and node global features; The combination module is used to standardize each of the target features to obtain standardized features, and to combine the standardized features of all nodes to obtain combined features; The load balancing module is used to generate an optimal load balancing weight strategy based on the combined features through a dynamic load balancing model, and to perform load balancing processing on the nodes in the backend server cluster based on the optimal load balancing weight strategy; wherein, the dynamic load balancing model is a model that dynamically determines the node weights using a gradient boosting decision tree machine learning model and a deep Q-network.

[0013] The present invention also provides a load balancing device, comprising: Memory, used to store computer programs; A processor for executing the computer program to implement the steps of the load balancing method described above.

[0014] The present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the above-described load balancing method.

[0015] The present invention also provides a computer program product, including a computer program / instructions, which, when executed by a processor, implement the steps of the above-described load balancing method.

[0016] As can be seen, this invention aligns backend server data, NGINX server data, and NGINX access log data to obtain aligned data; based on the aligned data, it determines the target features of each node in the backend server cluster; wherein, the target features include node local features and node global features; each target feature is standardized to obtain standardized features, and the standardized features of all nodes are combined to obtain combined features; based on the combined features, an optimal load balancing weight strategy is generated through a dynamic load balancing model, and load balancing is performed on the nodes in the backend server cluster based on the optimal load balancing weight strategy; wherein, the dynamic load balancing model is a model that dynamically determines node weights using a gradient-based decision tree machine learning model and a deep Q-network. Compared to current load balancing based on a single metric, this invention can determine multiple metrics for load balancing based on backend server data, NGINX server data, and NGINX access log data. It also adopts an XGBoost (a machine learning model based on gradient boosting decision trees) - DQN (deep Q-network) collaborative architecture, where XGBoost predicts stable long-term load balancing weights, and DQN then fine-tunes these long-term load balancing weights based on real-time status. Therefore, the strategy of this invention is both predictive and real-time, thereby improving the accuracy of load balancing.

[0017] In addition, the present invention also provides a load balancing device, equipment, and computer-readable storage medium, which also have the above-mentioned beneficial effects. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0019] Figure 1 A flowchart of a load balancing method provided in an embodiment of the present invention; Figure 2 An architecture diagram of a dynamic load balancing model provided in an embodiment of the present invention; Figure 3 This is a schematic diagram illustrating the verification and execution of a load balancing weight strategy according to an embodiment of the present invention. Figure 4 A flowchart illustrating a load balancing method provided in an embodiment of the present invention; Figure 5 This is a schematic diagram of the structure of a load balancing device provided in an embodiment of the present invention; Figure 6This is a schematic diagram of the structure of a load balancing device provided in an embodiment of the present invention. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0021] Some terms that appear in the description of the embodiments of this application are subject to the following interpretation: NGINX (Nginx, load balancer): A high-performance open-source HTTP (Hypertext Transfer Protocol) server, reverse proxy server, and load balancer. In this invention, it specifically refers to its upstream load balancing module. The method of this invention aims to perform real-time, intelligent, and dynamic optimization of the weight allocation of the backend servers in this module.

[0022] XGBoost (eXtreme Gradient Boosting, an efficient machine learning algorithm based on Gradient Boosting Decision Tree (GBDT)): An efficient machine learning algorithm based on Gradient Boosting Decision Tree (GBDT). In this scheme, it is used as a "baseline policy generator," predicting a long-term stable and high-performance base weight distribution based on massive historical data and current macro-level characteristics (such as time period and total QPS).

[0023] DQN (Deep Q-Network): A reinforcement learning algorithm that combines deep neural networks with Q-Learning. In this scheme, it is used as a "real-time dynamic fine-tuner," which innovatively uses the XGBoost baseline as part of its state input, learning what "adjustment action" to perform on the baseline in a specific real-time state to obtain the maximum compound reward.

[0024] P95 Latency (95th percentile latency): A network service performance metric. It refers to the response time of 95% of requests within a statistical period being lower than this value. Compared to the mean latency, P95 more sensitively reflects the performance jitter of long-tail requests and is a key indicator for measuring real user experience and Quality of Service (QoS).

[0025] The `split_clients` module (NGINX split_clients Module, traffic splitting module): A standard NGINX module that splits client requests into different backend groups based on a preset percentage (e.g., 5% and 95%) according to specific variables (such as client IP hashes). In this solution, it is the core technology tool for implementing "policy-validation-based canary rollout," used to import a small proportion of real traffic into a new policy group `upstream_gray` for A / B testing.

[0026] Composite Reward Function: A multi-objective reward function designed in reinforcement learning. In this scheme, the function not only includes scores representing performance (throughput, latency), but also innovatively introduces a "system imbalance penalty" (i.e., CPU standard deviation) representing cluster balance, forcing the DQN model to consider system load balance while pursuing high performance.

[0027] Please refer to Figure 1 , Figure 1 A flowchart illustrating a load balancing method provided in an embodiment of the present invention. The method may include: S101 aligns the backend server data, NGINX server data, and NGINX access log data to obtain aligned data.

[0028] Each step in this embodiment can be executed by a designated electronic device, which can be a server, a portable terminal, or other form. The backend server data in this embodiment is the data of the actual running business system in the backend. The NGINX server data in this embodiment is the data of the NGINX cluster. This embodiment can collect operational data in real time and completely from the NGINX cluster and the backend server cluster, and perform cleaning and alignment to provide a high-quality data source for subsequent steps. This embodiment can be implemented using the Prometheus (an open-source system monitoring and alerting system) service deployed on a monitoring server. On all backend server nodes, the Node Exporter (node ​​monitoring tool) service is deployed. Prometheus actively pulls the node_cpu_seconds_total (total CPU usage time), node_memory_MemAvailable_bytes (available memory), node_network_receive_bytes_total (total number of bytes received via the network), and node_network_transmit_bytes_total (total number of bytes sent via the network) metrics from each node at 15-second scraping intervals (scrape_interval). Deploying the Nginx-Exporter (monitoring) service on the NGINX server, Prometheus also pulls nginx_upstream_responses_total (total number of responses from each upstream node (NGINX server), distinguishing status codes), nginx_upstream_response_time_seconds_bucket (upstream node response time histogram), and nginx_upstream_failures_total (number of failed connections from upstream nodes) every 15 seconds. node_cpu_seconds_total, node_memory_MemAvailable_bytes (memory), and network I / O metrics (number of bytes received and sent) are chosen as core parameters because these three types of resources constitute the physical performance foundation of a web server's handling of HTTP requests—CPU determines computing and concurrent processing capabilities, memory determines connection persistence and cache capacity, and network bandwidth determines the throughput limit. This combination constitutes a "minimum complete set" for describing the health of server nodes, effectively solving the problem of biased decision-making caused by existing technologies that rely on a single dimension (such as only looking at the number of connections).From a technical standpoint, other parameters (such as disk I / O, system load average, or the number of TCP connection states) can be used to replace or supplement these parameters, and the number of parameters can be increased or decreased. However, reducing parameters (such as removing memory metrics) may prevent the model from recognizing specific overload scenarios like "memory leaks but low CPU usage" (i.e., "blind spots"). Blindly increasing parameters, while improving detail awareness, will significantly increase the dimensionality of the state vector, leading to increased training difficulty and slower convergence. Therefore, the four metrics selected in this solution represent the optimal balance between "feature effectiveness" and "computational complexity" for high-performance web server scenarios. "Upstream Node" specifically refers to the backend server nodes proxied by NGINX. These are the computing resources that actually receive and process user business requests. Here, "upstream" is a relative concept compared to NGINX (the reverse proxy server). In a reverse proxy architecture, NGINX sits between the client and the application server, forwarding requests to a group of backend servers, which are called "upstream nodes." Therefore, not every server node "owns" an upstream node; rather, these N backend servers themselves are the upstream nodes corresponding to NGINX. One NGINX server (or cluster) corresponds to the entire backend server cluster configured in the upstream module, and traffic is distributed to these upstream nodes according to algorithm weights. This embodiment can be implemented by deploying Filebeat (a lightweight delivery tool for forwarding and centralizing log data) on all NGINX servers. Filebeat is configured to monitor NGINX's access logs (access.log) in real time. To ensure data integrity, the log format is customized to include JSON format containing request_time (total request time), upstream_response_time (upstream response time), upstream_addr (upstream server IP), status (HTTP status code), and remote_addr (client IP). Filebeat sends the collected log lines to a centralized Kafka cluster's nginx-log-raw topic in real time. This provides actual status information for the business. This implementation can periodically (every second) query the data obtained from the Prometheus API. Its core processing logic is: based on the http_user_agent or remote_addr field in the log, discard all health check requests from internal monitoring probes. Log stream data (such as QPS, P95 latency, error code count) and metric stream data (such as CPU, memory) are aggregated into a unified time window, generating a structured data stream containing all relevant data within that time window, measured in seconds.

[0029] S102, determine the target features of each node in the backend server cluster based on the aligned data; wherein, the target features include node local features and node global features.

[0030] In this embodiment, node local features refer to characteristics that describe each individual node, while node global features refer to characteristics that describe the overall characteristics of nodes in the backend server cluster. Node local features in this embodiment include average CPU utilization, average memory usage, average total network bandwidth, average requests per second, P95 response time, and error rate; node global features can include the sum of the average requests per second for all nodes. This embodiment can define a sliding time window, with a window size... seconds, sliding step The module is triggered every 30 seconds, pulling data from the past 60 seconds from the nginx-data-aligned topic for calculation to determine the target features, i.e. Node i in Average CPU utilization within the system. Node i in Average memory usage within the system. Node i in Average total network bandwidth (received + transmitted) within the network. Node i in The average number of requests per second within the system. Node i in The P95 response time (calculated based on upstream_response_time). Node i in Internal 5xx error rate (number of 5xx responses / total number of responses). Global characteristics: : All N nodes The sum of all data points. It's understandable that while backend server metrics are collected, the "quality of service" from the backend server's perspective is often inaccurate (for example, when a server crashes or experiences network congestion, it cannot report errors, but NGINX logs faithfully record "timeout" or "502 error"). Therefore, data from the NGINX side must be combined to construct complete data and thus determine accurate target characteristics.

[0031] The average CPU utilization corresponds to `node_cpu_seconds_total` (total CPU usage time). The calculation logic is as follows: since the collected time is cumulative, the "utilization rate" equals the value at the end of the current window minus the value at the beginning, then divided by the window duration. That is: The average memory usage corresponds to the node_memory_MemAvailable_bytes mentioned above; the calculation logic is as follows: The average network bandwidth corresponds to node_network_receive_bytes_total and transmit mentioned above; the calculation logic is the same: calculate the increment (rate) per unit time. (QPS) (P95 delay) (Error Rate): Corresponds to the cleaned NGINX log stream. Calculation Logic: Requests are assigned to specific nodes based on the upstream_addr field in the logs, and a count or quantile value is calculated. In this example, the upstream_addr field in the logs is first parsed to calculate the number of requests per second (Q_i) belonging to each backend node i. Then, the Q_i of all N nodes are summed to obtain QPS_{total}. Simply put, it is the total number of valid log entries collected within the current time window, representing the total traffic pressure received by the system. Choosing the sum of QPS as a global feature allows the AI ​​model to judge the "macro load level." Metrics of a single node (such as high CPU usage on a server) only reflect a local situation, while the sum of QPS reflects the total external request pressure on the entire system. This is crucial for decision-making: for example, when node CPU usage increases, if the total QPS is also high, the model will judge this as a normal traffic peak and tend to distribute traffic evenly; if the total QPS is low but node CPU usage remains high, the model can judge that the node may have deadlocked or failed and tend to shut down the node. It provides the AI ​​with the "environmental context" needed for decision-making. Common alternatives to choosing other global features include "global average CPU utilization," "global average P95 latency," or "global total network bandwidth." However, in this approach, choosing the total QPS is optimal because it is the "cause" (input) leading to changes in system load, while CPU and latency are the "results" (performance metrics) after the system has been under stress. Using the "cause" as a macro-level feature input helps XGBoost and DQN more accurately predict the system's capacity based on the current traffic volume, thereby developing a more reasonable baseline strategy.

[0032] S103, standardize each target feature to obtain standardized features, and combine the standardized features of all nodes to obtain combined features.

[0033] This embodiment does not limit the specific method of standardization. For example, this embodiment can use the Z-score (standard score) standardization method to process all features in the target features; or this embodiment can use Min-Max normalization (compressing the data to the 0-1 range) or logarithmic transformation (Log). It should be noted that the input data of this invention contains multiple physical dimensions: CPU is a percentage (0-1), response time is milliseconds (0-1000+), network bandwidth is bytes, and error rate is a very small number (0-0.01). If input directly into the model without processing, features with huge values ​​(such as bandwidth) will dominate the gradient descent direction, causing the model to ignore small but critical features (such as error rate), and even leading to gradient explosion or model non-convergence. Therefore, Z-score is the optimal standardization method in this scheme. However, in heavy load scenarios, transient performance spikes (outliers) often occur. The Min-Max method is greatly affected by extreme values, which can cause the data in the normal range to be compressed too much. The Z-score standardization used in this scheme can preserve the distribution characteristics of the data and center the input value to near 0, which is most conducive to the efficient operation of the DQN activation function (ReLU). The calculation formula for standardization in this embodiment is as follows: ;in, and These are the historical mean and historical standard deviation of the corresponding features. These two parameters are stored in a key-value database (such as Redis) and are calculated and updated daily by a background task at the lowest traffic hour (fixed at 03:00) based on data from the past 24 hours to adapt to cyclical changes in business operations. This embodiment combines the standardized features of all N nodes with the global feature to construct a fixed-dimensional state vector. Its structure is as follows: This structure allows the DQN model not only to perceive "heavy system load," but also to accurately pinpoint "where the load is heavy" and "why it is heavy." For example, the model can clearly distinguish between "Node 1 experiencing latency due to excessive CPU usage (requiring deweighting)" and "Node 2 experiencing packet loss due to full network bandwidth (requiring traffic offloading)" through vector numerical differences. Simultaneously, by combining global QPS, the model can determine whether the current situation is a "global traffic surge" (preferring balanced load balancing) or a "single point of failure" (preferring circuit breaker isolation). This level of granularity is unattainable with traditional vector construction methods based on average values ​​or single dimensions, laying the data foundation for subsequent precise adjustments to specific nodes.

[0034] S104. Based on the combined features, the optimal load balancing weight strategy is generated through a dynamic load balancing model, and the load balancing of nodes in the backend server cluster is performed based on the optimal load balancing weight strategy. The dynamic load balancing model is a model that dynamically determines the node weights using a gradient-based decision tree machine learning model and a deep Q-network.

[0035] The dynamic load balancing model in this embodiment includes a machine learning model based on a gradient boosting decision tree and a deep Q-network. When determining the weights, the combined features are first input into the machine learning model based on the gradient boosting decision tree. The input of the deep Q-network is constructed based on the output of the machine learning model based on the gradient boosting decision tree. The input is then input into the deep Q-network. Finally, the output of the deep Q-network and the output of the machine learning model based on the gradient boosting decision tree determine the optimal load balancing weight strategy.

[0036] It should be further explained that, based on any of the above embodiments, the above-mentioned generation of an optimal load balancing weight strategy through a dynamic load balancing model based on combined features, and the load balancing processing of nodes in the backend server cluster based on the optimal load balancing weight strategy, may include: S1041, based on combined features, uses a gradient boosting decision tree-based machine learning model to predict the optimal weight distribution in the current state.

[0037] For easier understanding, please refer to Figure 2 , Figure 2 This is an architecture diagram of a dynamic load balancing model provided in an embodiment of the present invention, receiving a state vector. This embodiment utilizes an innovative collaborative architecture of XGBoost and DQN (Deep Q-Network) to generate an optimal load balancing weight strategy. The XGBoost in this example can be a pre-trained offline XGBoost model with fixed parameters: n_estimators=100, max_depth=5, learning_rate=0.1, gamma=0.1, lambda=0.1. Two additional contextual features can be concatenated to the state vector (combined features): Hour_Of_Day (current hour, 0-23) and Day_Of_Week (current day of the week, 0-6), both of which are One-Hot encoded. Based on these combined and contextual features, the XGBoost model predicts a long-term optimal weight distribution under the current macro-level conditions. It's important to understand that "long-term" here is relative to the real-time fine-tuning of DQN. XGBoost makes predictions based on macro-level features (such as "Monday morning rush hour" or "total QPS of 50,000"), which have low change frequencies (minute-level or hour-level). Therefore, it generates a relatively stable baseline strategy within the current business period. Its role is to set a "robust tone" for the system, preventing DQN from losing its way due to excessive focus on second-level network jitter. It can be understood as "strategic planning," while DQN is "tactical adjustment." With the same server configuration, the optimal distribution strategy is completely different during "low traffic at night" and "high concurrency during peak periods" (e.g., energy saving may be prioritized during off-peak periods, while extreme throughput balance may be prioritized during peak periods). Limiting the "current macro-state" (input time and total load) allows the model to retrieve the best-performing strategy in similar historical scenarios. In subsequent offline retraining, only historically validated and highly-rated weight strategies can be selected as "labels" for the training data. XGBoost learns from this data to establish a mapping of the "optimal weights under the macro-state." Therefore, during real-time prediction, it is essentially "recalling" historical expert experience and outputting the statistically most probable optimal solution. The output weight strategy strictly corresponds to N backend business server nodes. The larger the value of the weight w_i, the higher the proportion of traffic (request probability) that NGINX allocates to node i. For example, if w_1=0.8 and w_2=0.2, it means that Node 1 will handle approximately 80% of user requests. This is usually because XGBoost determines from historical data that Node 1 has stronger hardware performance, or that its processing power is more abundant at the current time.

[0038] S1042, the combined features and the optimal weight distribution are concatenated to obtain the concatenated features.

[0039] In this embodiment, the output of the machine learning model of the gradient boosting decision tree is an N-dimensional baseline weight vector (optimal weight distribution). ;in and This vector It is passed to the DQN real-time tuner. This represents the baseline weight value of the first backend service server node; the subscript b represents the Baseline, and the subscript 1 represents the node number; N represents the total number of backend server nodes. This embodiment's DQN status input... Innovatively designed as a state space That is, features are combined in real time. and optimal weight distribution This is achieved by concatenating the baseline policy W_{base}(t) generated by XGBoost. This allows DQN's decisions to be based on both "what the current environment is" and "what experts recommend." The design is based on the concept of "residual learning," aiming to reduce the learning difficulty of the model and improve its stability. If only the real-time state (combined features) V(t) is input, DQN needs to figure out a complete weight allocation scheme from scratch, which is very difficult and prone to initial policy oscillations. By concatenating the baseline policy W_{base}(t) (optimal weight distribution) generated by XGBoost, it is equivalent to providing DQN with a "standard reference answer." In this way, the task of DQN is simplified from "finding the optimal weights" to "fine-tuning the standard answer W_{base}(t) based on the current real-time failure V(t)." This design allows the model to retain the macroscopic stability of the baseline policy while giving it the flexibility to handle microscopic bursts of traffic, greatly accelerating the convergence speed.

[0040] S1043 uses a deep Q-network to generate weight adjustment vectors based on splicing features.

[0041] This embodiment is based on a DQN (Deep Q Network) model, which is used to make real-time and fine-grained adjustments to the XGBoost baseline strategy to cope with sudden traffic and short-term fluctuations.

[0042] S1044 determines the optimal load balancing weight strategy based on the weight adjustment vector and the optimal weight distribution.

[0043] This step will set the XGBoost baseline. Adjustment actions selected with DQN Integrate into the final strategy. For example, This embodiment can perform weight correction (ReLU): to prevent weights from becoming negative, the weights are adjusted accordingly. Perform a ReLU operation on each element i: Weight normalization is performed: To ensure that the sum of all weights is 1, normalization is performed. This embodiment ultimately outputs the optimal load balancing weight strategy in N dimensions. It can also be sent to the policy verification and execution module.

[0044] This embodiment provides a specific method for determining the optimal load balancing weight strategy using a machine learning model based on gradient boosting decision trees and a dynamic load balancing model composed of deep Q-networks, thereby improving the accuracy of determining the optimal load balancing weight strategy.

[0045] It should be further noted that, based on any of the above embodiments, the generation of weight adjustment vectors using a deep Q-network based on splicing features can include: Step 1: Define the action space; where the actions in the action space are K predefined, discretized N-dimensional adjustment vectors.

[0046] In this embodiment, the actions are K predefined, discretized N-dimensional adjustment vectors. .by Taking a node as an example, define One action: (Maintain baseline); (Significantly increases the weight of node 1); (Significantly increases the weight of node 2); (Significantly increases the weight of node 3); (Significantly reduce the weight of node 1); (Significantly reduces the weight of node 2); (Significantly reduces the weight of node 3); The adjustment vector is a set of pre-defined numerical increments, such as [+0.05, -0.025, -0.025]. It's a tool for adjusting the traffic allocation weights of backend business server nodes. Specifically, it represents how much traffic the DQN model believes should be increased for a certain node based on the current baseline weights, while reducing traffic for other nodes to maintain overall weight balance. N=3 nodes still refer to the backend business server nodes. In the entire load balancing system, the physical meaning of weight is to control the proportion of traffic distributed to the backend. Therefore, all parameters involving dimension N always correspond to the N actual backend servers processing the business, not the NGINX proxy servers themselves. K represents the total number of actions, that is, how many choices the model has. In this example, K=7 means there are 7 choices. Each action is defined as a specific adjustment scheme. For example, "Action 1" is bound to the vector "weight node 1", while "Action 0" is bound to "remain unchanged". This definition method transforms the complex continuous adjustment problem into a simple discrete choice problem, facilitating rapid decision-making by the model.

[0047] Step 2: Determine the Q value corresponding to each action in the action space based on the splicing features.

[0048] In this embodiment, the DQN Q-network adopts a 3-layer fully connected structure (input layer dimension, hidden layer 1 dimension 64, hidden layer 2 dimension 64, output layer dimension), and the activation function is ReLU (Rectified Linear Unit). The model calculates the Q-value of each action based on the input and uses an epsilon-greedy strategy to select actions. The initial exploration rate is 0.9, multiplied by a decay factor at each step, decreasing to a minimum of 0.01. This embodiment can use other structures, such as using a recurrent neural network (LSTM) to capture time series features, or a convolutional neural network (CNN) to handle more complex inputs. However, a 3-layer fully connected structure is chosen in this scheme because it has low computational cost and extremely fast inference speed, which can meet the stringent requirements of millisecond-level real-time response for server load balancing. This embodiment can also use other types of activation functions. Common alternatives include Tanh (hyperbolic tangent function), Sigmoid (logistic function), or LeakyReLU (leaking corrected linear unit). However, ReLU is chosen as the optimal choice in this scheme because ReLU is simple to compute and can effectively alleviate the gradient vanishing problem in deep neural networks. In this embodiment, the Q-value is calculated based on the Bellman equation fitted to the neural network in reinforcement learning. The Q-value represents the expected long-term total reward after performing a specific action in the current state. The initial epsilon value is set to 0.9 to force high-intensity exploration during the system's cold start phase. Initially, the model has no experience, and setting it to 0.9 means that the system spends 90% of its time randomly trying various weight adjustments, thereby quickly and extensively collecting feedback data from different states and preventing the model from getting trapped in local optima. The decay factor is reduced to a minimum of 0.01 to achieve a smooth transition from exploration to execution. As the number of training steps increases, the model gradually learns the patterns. At this point, it is necessary to reduce random attempts by decaying epsilon, allowing the model to execute more of the strategies it deems correct. Maintaining a minimum value of 0.01 instead of reducing it to 0 is to retain 1% flexibility, allowing the system to try new strategies when facing sudden changes in the network environment, thus maintaining adaptability to dynamic environments.

[0049] Step 3: Determine the weight adjustment vector based on the Q value of each action.

[0050] In this embodiment, the model compares the Q-values ​​of all possible actions during decision-making, typically selecting the action with the highest Q-value, i.e., choosing the strategy evaluated as the best. The epsilon-greedy strategy has a direct logical relationship with the Q-value: this strategy determines whether the model selects the optimal action based on the Q-value or ignores the Q-value and randomly selects an action (exploration). Specifically, there is a 1-epsilon probability of selecting the action with the highest Q-value, and an epsilon probability of random trial, thus balancing the use of experience and the exploration of new knowledge. Designing the actions as predefined discrete adjustment vectors (Delta W) is to transform the complex continuous variable control problem into a relatively simple discrete classification problem. Compared to directly outputting continuous weight values, allowing the model to choose from a limited number of options (such as increasing, decreasing, or maintaining) significantly reduces the difficulty of model training and improves the convergence speed and decision stability.

[0051] It should be further explained that, based on any of the above embodiments, the above-mentioned generation of an optimal load balancing weight strategy through a dynamic load balancing model based on combined features, and the load balancing processing of nodes in the backend server cluster based on the optimal load balancing weight strategy, may include: S1: Determine the currently effective load balancing weight strategy, determine the latest optimal load balancing weight strategy, and the change between the current and effective load balancing weight strategies.

[0052] This embodiment maintains the currently active online weighting strategy in memory. When the new strategy from step three is received... First, calculate the L1 norm (sum of absolute changes) between the two to determine the change value. .

[0053] S2: Compare the change value with the set change threshold.

[0054] This embodiment does not limit the specific change threshold. For example, the change threshold in this embodiment can be... In real-world production environments, this value is typically set between 0.10 and 0.20 (i.e., 10% to 20%). In this embodiment, setting it to 0.10 means that if the total weight change of all nodes exceeds 10% compared to the old strategy, the system considers this a drastic adjustment. This threshold is set to balance efficiency and safety: small fluctuations are considered normal dynamic fine-tuning, while large mutations are considered high-risk operations that must be validated.

[0055] S3: When the change value is less than or equal to the change threshold, determine the optimal load balancing weight strategy to perform regular hot update and load balancing on the nodes in the backend server cluster; where regular hot update is the strategy of applying the latest optimal load balancing weight strategy to all traffic.

[0056] In this embodiment, if : Determined as a routine fine-tuning, strategy The system is considered safe and performs a regular hot update. In this embodiment, the regular hot update utilizes the interface technology of NGINX Plus or OpenResty; the core innovation of this invention is the canary adjustment mechanism based on split_clients combined with AI (artificial intelligence) scoring for automated verification. The difference between the two is: a regular hot update immediately applies the new weights to 100% of production traffic, which is fast but risky and only suitable for fine-tuning; the canary adjustment mechanism applies the new weights only to a very small percentage (e.g., 5%) of traffic for isolated testing, which is slower but extremely secure, specifically designed to handle significant policy changes and prevent crashes caused by decision-making errors. This embodiment targets an NGINX Plus or OpenResty environment because the standard open-source version of NGINX does not support dynamically modifying the weights of upstream servers without restarting the process. If the standard version is used, each adjustment requires reloading the configuration file, which causes connection interruptions and performance degradation, failing to meet the "real-time dynamic" requirement of this invention. Therefore, this solution must rely on the dynamic API functionality provided by NGINX Plus or the Lua shared memory programming capabilities provided by OpenResty to achieve lossless millisecond-level weight updates. (1) For NGINX Plus: The system calls the REST API interface of NGINX Plus to send messages to all NGINX instances via http / upstreams / <upstream_name> / servers / <server_id> The path sends a PATCH request, dynamically modifying the weight parameter to match... Consistent. Each weight value w_i in W_final(t) will be mapped to the JSON format required by the NGINX Plus API. The weight of node 1 is 0.2, so the system calls the API to change the weight parameter of node 1 to 0.2. (2) For OpenResty: The system modifies the weight values ​​stored in the shared memory dictionary (ngx.shared.dict) of Lua through a dedicated HTTP interface. The NGINX balancer_by_lua script reads this shared memory with each request to realize the real-time update of weights. Each time a new request comes in, it does not read the static weight in the configuration file, but reads the latest W_final(t) data in the shared memory, thereby realizing the dynamic allocation of traffic. After the update is completed, Updated to .

[0057] S4: When the change value is greater than the change threshold, determine the optimal load balancing weight strategy, and use the gray-scale adjustment mechanism to adjust the effective load balancing weight strategy to obtain the adjusted optimal load balancing weight strategy. Perform load balancing processing based on the adjusted optimal load balancing weight strategy. Among them, the gray-scale adjustment mechanism is that the first proportion of traffic is load balanced based on the optimal load balancing weight strategy, and the second proportion of traffic is load balanced based on the effective load balancing weight strategy.

[0058] In this embodiment, if If a major policy change is detected, the system must initiate a phased rollout mechanism for verification. For example, 5% of the traffic during the phased rollout may be moved to the new policy, while 95% of the stable traffic may remain under the old policy. For easier understanding, please refer to [link / reference needed]. Figure 3 , Figure 3 This is a schematic diagram illustrating the verification and execution of a load balancing weight strategy provided in an embodiment of the present invention.

[0059] It should be further explained that, based on any of the above embodiments, when the change value is greater than the change threshold, an optimal load balancing weight strategy is determined, and the currently effective load balancing weight strategy is adjusted using a gray-scale adjustment mechanism to obtain the adjusted optimal load balancing weight strategy. After performing load balancing processing based on the adjusted optimal load balancing weight strategy, the process may further include: Step 1: Determine the first performance parameter corresponding to the first proportion of traffic and the second performance parameter corresponding to the second proportion of traffic.

[0060] Specifically, this embodiment can dynamically modify the NGINX configuration to define two upstream server groups: upstream_stable (whose server directive's weight parameter is configured as follows). ) and upstream_gray (whose server directive's weight parameter is configured as ) Based on the old weights stored in memory and the newly calculated weights, a completely new configuration file is written, overwriting the original configuration. This behavior of automatically rewriting the configuration file based on real-time data is called dynamic configuration generation in systems engineering. The upstream_stable group will never use the new strategy. The specific Chinese process is described as follows: The system first reads the currently used old strategy (denoted as W_old) and uses it to configure the first server group, "Stable Group"; at the same time, it reads the new strategy just generated by AI (denoted as W_new) and uses it to configure the second server group, "Gray Group". The purpose of this is to create a parallel universe: the vast majority of users access the "Stable Group" (experience the old strategy, ensuring security), and a very small number of users access the "Gray Group" (experience the new strategy, used for testing). If the stable group with the old strategy is not retained, the benchmark for comparison is lost, and the experience of most users cannot be guaranteed when the test fails. Then, using NGINX's split_clients module, traffic is split based on the client IP hash (remote_addr). The generated configuration snippet is as follows: split_clients remote_addr traffic_split { 5% "gray"; `"stable";}proxy_passhttp: / / upstream_traffic_split;` This configuration takes effect through hot reloading by sending a HUP signal to the NGINX master process (executing the `nginx -sreload` command). At this point, 5% of the gray-scale traffic enters the new policy, while 95% of the stable traffic remains under the old policy. Understandably, 5% is an empirically safe threshold that provides sufficient real request samples (for observing P95 latency and error rate under the new policy) without causing devastating damage to the overall business if the policy fails. If the percentage is too low (e.g., 1%), the amount of log data collected will be too small, leading to huge fluctuations in the statistically calculated P95 latency, making it impossible to draw accurate conclusions; if the percentage is too high (e.g., 20%), if the AI ​​model predicts incorrectly, one-fifth of user requests will be affected, posing too high a risk. This percentage can be adjusted. In massive clusters with extremely high traffic, the percentage can be reduced to 1% or even lower, as the large traffic base still ensures the statistical significance of the data; while in test environments with lower traffic, the percentage can be appropriately increased to 10% to speed up verification.

[0061] This step can start a system. The verification period is seconds. During this period, the data acquisition module in step one analyzes the traffic_split variable in the logs to determine the core performance indicators of the two traffic groups: stable95% and gray5%. contrast and contrast Independent monitoring and comparison should be conducted.

[0062] Step 2: Based on the first performance parameter and the second performance parameter, determine whether the performance of the optimal load balancing weight strategy is better than the performance of the currently active load balancing weight strategy.

[0063] In this embodiment, the first performance parameter can be: and This is the second performance parameter. and This is the first performance parameter. The judgment condition is: in Within the period, determine and (That is, the performance of the new strategy is no worse than that of the old strategy, and a 5% jitter tolerance is allowed) is true.

[0064] Step 3: When it is determined that the performance of the optimal load balancing weight strategy is better than the performance of the currently effective load balancing weight strategy, determine to perform load balancing based on the optimal load balancing weight strategy.

[0065] If true, then determine Verification successful. The system has restored its NGINX configuration and is redirecting 100% of traffic. and set .

[0066] Step 4: When it is determined that the performance of the optimal load balancing weight strategy is not better than the performance of the currently effective load balancing weight strategy, determine to perform load balancing based on the currently effective load balancing weight strategy.

[0067] If this embodiment determines that the above conditions are not met, then it makes a determination. This is a poor strategy. The system has been restored to its NGINX configuration, and 100% traffic remains. Simultaneously, a "rollback" signal can be sent to the subsequent closed-loop self-learning module, along with the reason for this failure. (State-Action Pair).

[0068] It should be further explained that, based on any of the above embodiments, after generating the optimal load balancing weight strategy based on the combined features through the dynamic load balancing model, and performing load balancing processing on the nodes in the backend server cluster based on the optimal load balancing weight strategy, it may also include: obtaining state strategy data pairs; wherein, the state strategy data pairs are data composed of the combined features of each period and the finally effective load balancing weight strategy; training the current gradient boosting decision tree-based machine learning model based on the state strategy data pairs to obtain the latest gradient boosting decision tree-based machine learning model, and using the latest gradient boosting decision tree-based machine learning model as the gradient boosting decision tree-based machine learning model in the dynamic load balancing model. This embodiment is XGBoost offline retraining, mainly including: (1) Task triggering: This module is implemented through a timed task (CRON), which is fixed to be executed at the lowest system load of each day (e.g., 03:00). (2) Data filtering: This module retrieves all data from the production database in the past 30 days. Periodic state-policy data pairs The system only selects strategies that have been successfully validated (promoted) in the above-mentioned gray-scale adjustment mechanism steps, and those that have resulted in positive performance scores after execution in the regular update steps. Strategies for improvement.

[0069] Positive performance score P(t) = QPS / (P95 latency × (1 + error rate)). The so-called "improvement" means that after executing a certain strategy, the P value of the system increases (for example, QPS increases, or latency and error rate decrease). The data screening logic here is: only feed high-quality historical data that actually makes the system faster and more stable to XGBoost for learning, to avoid the model learning incorrect experience. (3) Model retraining: Retrain the XGBoost model using the selected high-quality dataset. (4) Model deployment: The newly trained model file will automatically replace the model file currently in use, thereby updating the system's "baseline strategy" cognition and realizing long-term adaptive optimization.

[0070] It should be further explained that, based on any of the above embodiments, after generating the optimal load balancing weight strategy through a dynamic load balancing model based on combined features, and performing load balancing processing on the nodes in the backend server cluster based on the optimal load balancing weight strategy, it may further include: constructing a replay pool based on state transition tuples; wherein, the data in the state transition tuples includes concatenated features, weight adjustment vectors, and reward values; the concatenated features are the features corresponding to each time period obtained by concatenating the combined features and the optimal weight distribution determined by the machine learning model based on the boosting decision tree; the weight adjustment vector is the vector corresponding to each time period determined based on the deep Q-network; the reward value is the reward value determined based on the reward function and the performance parameter score of the previous period; the reward function is a function constructed based on a set number of performance parameters; based on the data in the replay pool, the evaluation network of the deep Q-network is updated by minimizing the difference between the predicted Q-value and the target Q-value using an optimizer and a loss function. This embodiment is the process of online training of DQN, which can maintain an experience replay pool (Replay Buffer) with a capacity of 1e6. Experience storage: This module continuously receives and stores state transition tuples. . , Corresponding to the splicing features and action controls in the above steps, To be based on the characteristics of the new combination The reward value is obtained using a composite reward function. Specifically, when a "rollback" signal is received, it will be forcibly changed to a value related to this. corresponding Change to a strong negative penalty value And increase the priority of the tuple in the sampling. (2) Model training: Continuously sample a batch (batch_size = 64) of tuples randomly from the replay pool. Using the Adam optimizer (learning rate 1e-4) and Huber loss function, minimize the predicted Q value. With the target Q value The difference between them is used to update the DQN's evaluation network.

[0071] Predict Q value It is calculated by the evaluation network of DQN through forward propagation. Its input is the current state vector. (Including real-time metrics and baseline strategies), the network performs layer-by-layer calculations, outputting a score for each possible action. Only the action actually executed is extracted from this score. The corresponding score is used as the predicted value. The objective function (Huber loss function) calculates the difference between this "predicted score" and the "target score". The goal of training is to minimize the gap between (the actual reward plus the expected future return) and (the difference between the actual reward and the expected future return). (3) Calculation of the target Q value: ;wherein discount factor Set to 0.99. (This is the target calculation method for Double DQN to improve stability); It is a variable representing the next moment. Any action that may be performed. This means: let the current evaluation network determine the next state. Which action scores the highest (i.e., select the optimal action)? The target network is a copy of a network with fixed parameters that remains unchanged for a period of time. Its role is to calculate the value of the "optimal action". Simply put, "the evaluation network is responsible for picking actions, and the target network is responsible for calculating scores", which can avoid the model overestimating itself. (4) Network synchronization: every Each training step (learning iterations) involves completely copying the parameters of the evaluation network to the target network. Each time the model retrieves a batch of data (64 data points) from the experience replay pool and completes a parameter update (gradient descent), it counts as one training step. Synchronization occurs every 500 steps, meaning the model only copies the latest parameters to the target network after every 500 self-corrections. This is done to keep the training target relatively static, preventing it from changing too rapidly; otherwise, the model would fail to converge due to chasing a constantly changing target.

[0072] It should be noted that the symbolic reward in this invention innovatively combines a "performance gain term" (a P-value composed of throughput, latency, and error rate) with a "system imbalance penalty term" (an I-value composed of CPU standard deviation). This specific combination formula, designed to balance high performance and high stability in Web load balancing scenarios, is not a standard formula commonly used in the industry. Furthermore, it is uniquely designed to interact with the "policy verification module": it can receive a "rollback signal" when a gray-scale test fails and forcibly inject strong negative penalty data accordingly. This specific closed-loop mechanism, which directly transforms the A / B test results from the business layer into training signals for the underlying model, is a logic specifically designed by this invention to address the security issues of AI deployment. The direct result of the calculation is a specific reward value Rt (a scalar value). The system extracts QPS, latency, error rate, and CPU load data from the new state V(t+1), substitutes them into the reward function formula, and calculates a score representing the "degree of good or bad" (e.g., +2.5 or -10 points). This value is the core feedback signal in reinforcement learning. It tells the DQN model whether the adjustment made in the previous step improved the system performance (gained a positive score) or caused the system to deteriorate (gained a negative score), thus guiding the model to correct the parameters.

[0073] (1) Composite performance score : ; in and It is the global P95 latency and global 5xx error rate obtained.

[0074] (2) System imbalance penalty .

[0075] ; That is, the standard deviation of CPU utilization of N nodes.

[0076] (3) Final reward : ; in, This is the performance score from the previous cycle. (Performance gain weight) is fixed at , (Balanced penalty weight) is fixed at 0.3. Used for training DQN.

[0077] The load balancing method provided in this invention, compared with the current load balancing based on a single indicator, can determine multiple indicators for load balancing based on backend server data, NGINX server data, and NGINX access log data. It also adopts an XGBoost-DQN collaborative architecture, in which XGBoost predicts a stable long-term baseline, and DQN then fine-tunes the baseline based on the real-time status. Therefore, the strategy of this invention has both predictive and real-time characteristics, thereby improving the accuracy of load balancing.

[0078] Traditional algorithms rely on a single dimension (or no dimension) for decision-making, lacking the comprehensive perception and prediction capabilities for complex traffic patterns and multi-dimensional server states, making it difficult to adapt to the rapidly changing load characteristics in current business scenarios. This invention addresses the problems in NGINX and similar load balancers: existing static algorithms (such as round-robin) make one-sided decisions and cannot perceive real-time load; simple dynamic algorithms (such as least connections) have a single decision dimension and are prone to load oscillations. Simultaneously, it addresses the issues of existing rule-based threshold-based tuning schemes being slow to respond and difficult to maintain; and existing AI load balancing schemes lacking security verification mechanisms and easily impacting production environment stability due to poor strategy exploration. This invention proposes a hybrid AI dynamic load balancing system and method based on XGBoost-DQN collaborative decision-making and strategy verification. The aim is to improve the throughput and response performance of server clusters, ensure the safety and smoothness of strategy switching, and achieve high robustness and intelligent adaptive capabilities under varying traffic patterns.

[0079] For a clearer understanding of this invention, please refer to the following details. Figure 4 , Figure 4 A flowchart illustrating a load balancing method provided in an embodiment of the present invention may specifically include: S201, through the data acquisition and preprocessing module, collects backend server metric data, NGINX server data, and NGINX access log data in real time, and performs cleaning, filtering, and timestamp alignment.

[0080] S202, through the dynamic feature engineering module, performs sliding window aggregation based on aligned data, calculates and standardizes multidimensional features, and then combines and outputs real-time combined features.

[0081] S203 receives real-time combined features through a hybrid AI decision module, executes XGBoost-DQN collaborative decision-making based on a dynamic load balancing model, and generates the optimal load balancing weight strategy.

[0082] S204 receives the optimal load balancing weight policy through the policy verification and execution module, and decides whether to execute a regular hot update or start a gray-scale adjustment mechanism based on the change magnitude of the policy and the currently effective weight policy, according to the policy verification and execution module.

[0083] S205, through the closed-loop self-learning module, performs online and offline iterative optimization of the model parameters in the hybrid AI decision-making module based on the execution feedback of S204 and the composite reward function defined in S203.

[0084] Specifically, the system and method described in this invention are deployed in an NGINX cluster with N=3 backend servers (Node 1, Node 2, Node 3). The relevant system parameters are set as follows: data collection interval 15 seconds, sliding window. seconds, sliding step Seconds, change threshold Gray-scale verification cycle Seconds, strong negative penalty .

[0085] At 14:30:00 on a certain workday, the currently effective online policy of the system is: This means that Node 1 is handling 50% of the traffic. At this point, the business traffic suddenly increases, causing the load on Node 1 to rise sharply.

[0086] Step 1: Through the data acquisition and preprocessing module.

[0087] This module worked continuously from 14:30:00 to 14:30:30. Prometheus collected a sharp increase in node_cpu_seconds_total for Node 1, and the Nginx-Exporter report showed a P95 latency for Node 1 in nginx_upstream_response_time_seconds_bucket. The 5xx error rate spiked to 800ms, and the nginx_upstream_responses_total{status=~"5xx"} count increased. Filebeat logs, after being cleaned and aligned by Flink, confirmed the 5xx error rate on Node 1. It rose to 5%.

[0088] Step 2: Through the dynamic feature engineering module.

[0089] At 14:30:30, this module is triggered to process the data in the window from 14:29:30 to 14:30:30. The module calculates the... The average is 90%. It takes 800ms. The figure was 5%; while the metrics for Node 2 and Node 3 were both at healthy levels (e.g., The module uses Z-score standardization (based on historical data). These raw features are processed to construct and output a state vector containing standardized metrics for all nodes. .

[0090] Step 3: Through the hybrid AI decision-making module.

[0091] This module received .

[0092] 3.1 XGBoost Baseline Policy Generator: XGBoost Model Reception And contextual features (Hour=14, Day=Weekday). Based on historical data, the model identifies the current peak business period and Node 1 as overloaded, predicting that the load on Node 1 should be reduced. Output baseline strategy: .

[0093] 3.2 DQN Real-Time Tuner: The DQN receives the spliced ​​state. . High latency and high error rate in and ) as input to DQN, enabling it to perceive the difference The suggested more urgent situation. After the Q-network computation of DQN, an adjustment vector aimed at further reducing the weights of Node 1 is selected through an epsilon-greedy strategy. .

[0094] 3.3 Final Strategy Fusion: System Calculation.

[0095] ; After ReLU correction and normalization, the final weight policy is output. .

[0096] Step 4: Pass through the policy verification and execution module.

[0097] This module received and with Compare them.

[0098] 4.1 Strategy change magnitude detection: Calculate the L1 norm.

[0099] .

[0100] 4.2 Change in execution decision: Detected This value is greater than the preset threshold. The system determines this to be a major policy change, requiring a phased rollout mechanism to be triggered instead of a regular hot update.

[0101] 4.4 Gray-scale adjustment and strategy verification: (a) The module dynamically generates the NGINX configuration, defining upstream_stable (weight is...). ) and upstream_gray (weight is ).

[0102] (b) The module applies the split_clients configuration (5% traffic to gray, 95% to stable) and executes nginx -s reload.

[0103] (c) System entry The verification period is in seconds (from 14:30:30 to 14:35:30). During this period, the system independently monitors the performance of the gray and stable traffic groups.

[0104] (d) (This case simulation verification failed) During the verification period, monitoring revealed... The strategy was too aggressive, causing the load on Node 2 and Node 3 to increase too rapidly in an instant, resulting in a high P95 latency for upstream_gray traffic. Instead, it rose to 850ms, exceeding the latency of upstream_stable. ).

[0105] (e) Verification decision: At 14:35:30, the system determines... The conditions are not met. Strategy validation failed.

[0106] (f) Rollback: The system automatically restores the NGINX configuration, redirecting 100% of traffic back to upstream_stable (i.e., Strategy), It was discarded. Simultaneously, the module sent a message containing... to step five. The "rollback" signal.

[0107] Step 5: Through the closed-loop self-learning module.

[0108] This module received the "rollback" signal from step four.

[0109] 5.1 DQN Online Training Submodule: The module identifies the state... Execute action at time This caused the verification to fail. The system then forcibly injected a high-priority tuple into the experience replay pool. ,in In subsequent DQN training, the model will quickly learn to perform similar tasks. In this state, you should avoid choosing (Overly aggressive adjustments) thereby improved the model's safety.

[0110] 5.2 XGBoost Offline Retraining Submodule: When executed at 03:00 the next day, due to the strategy at 14:30:30... The verification was deemed unsuccessful. The data pairs are not used for XGBoost retraining, ensuring that the baseline model only learns those policies that have been validated as "high-quality" by real online traffic.

[0111] The beneficial effects of this invention are as follows: (1) XGBoost, as a baseline policy generator, predicts a stable long-period weight based on macroscopic features and historical data; DQN, as a real-time dynamic fine-tuner, innovatively includes a real-time state vector and the baseline policy generated by XGBoost in its state space, i.e., . The decision objective of DQN is to learn the "optimal adjustment vector" for the baseline, and the final policy is formed by the fusion of and . This architecture allows DQN's exploration and learning to be based on the "expert experience" of XGBoost, which greatly improves the robustness and convergence speed of the decision, and realizes the organic combination of long-period stability and short-period adaptation. Therefore, the policy of this invention has both predictive and real-time characteristics.

[0112] (2) This solution implements a proactive A / B testing-based strategy verification. When the change in the strategy generated by the hybrid AI decision-making module exceeds a threshold, the strategy verification and execution module utilizes NGINX's split_clients function to split a small proportion (e.g., 5%) of real traffic to the upstream_gray using the new strategy, while keeping the mainstream traffic in the upstream_stable of the old strategy. During the verification period, the system compares the core performance indicators (P95 latency, error rate) of the two upstream groups in real time. Only when the new strategy is verified to be superior to or no worse than the old strategy will it be fully implemented; otherwise, it will be rolled back immediately. This fundamentally eliminates the risk of inferior AI strategies impacting the production environment and ensures the security of the system.

[0113] (3) First, the reward function of DQN is designed to be composite, where is the standard deviation of the CPU utilization of the backend server, i.e., the "system imbalance penalty term". This forces the AI ​​to maintain the load balance of the cluster while pursuing high performance, and avoid the "pseudo-balanced" state. More importantly, this scheme tightly couples policy verification with self-learning: when the gray-scale adjustment mechanism determines that the policy verification has failed and executes rollback, it will send a rollback signal to the closed-loop self-learning module. The module will then forcibly inject a strong negative penalty associated with the failure (state-action) pair into the experience replay pool of DQN, so that the AI ​​can quickly learn from the "dangerous decision" that leads to system instability, and achieve a truly safe, closed-loop, and intelligent self-evolution.

[0114] (4) Because the present invention uses a real-time state vector containing multi-dimensional indicators such as CPU, P95 latency, and error rate as the basis for AI decision-making, its decision-making can comprehensively and accurately assess the overall health status of each node, overcome the one-sidedness of traditional solutions, realize accurate perception and intelligent decision-making of backend load, and fundamentally avoid node overload caused by insufficient information.

[0115] The load balancing device provided in the embodiments of the present invention will be described below. The load balancing device described below and the load balancing method described above can be referred to each other.

[0116] Please refer to the details. Figure 5 , Figure 5 A schematic diagram of a load balancing device provided in an embodiment of the present invention may include: Alignment module 100 is used to align backend server data, NGINX server data, and NGINX access log data to obtain aligned data; The target feature determination module 200 is used to determine the target features of each node in the backend server cluster based on the aligned data; wherein, the target features include node local features and node global features; The combination module 300 is used to standardize each of the target features to obtain standardized features, and to combine the standardized features of all nodes to obtain combined features; The load balancing module 400 is used to generate an optimal load balancing weight strategy based on the combined features through a dynamic load balancing model, and to perform load balancing processing on the nodes in the backend server cluster based on the optimal load balancing weight strategy; wherein, the dynamic load balancing model is a model that dynamically determines the node weights using a gradient boosting decision tree machine learning model and a deep Q-network.

[0117] Furthermore, based on any of the above embodiments, the load balancing module 400 may include: The optimal weight distribution determination module is used to predict the optimal weight distribution in the current state based on the combined features using the gradient boosting decision tree-based machine learning model; wherein, the gradient boosting decision tree-based machine learning model is a model trained based on the combined features and the corresponding actual weight distribution during the training phase. The splicing unit is used to splice the combined features and the optimal weight distribution to obtain the spliced ​​features; A weight adjustment vector determination unit is used to generate a weight adjustment vector based on the splicing features using the deep Q-network. The optimal load balancing weight strategy determination unit is used to determine the optimal load balancing weight strategy based on the weight adjustment vector and the optimal weight distribution.

[0118] Furthermore, based on any of the above embodiments, the weight adjustment vector determination unit may include: An action space determination subunit is used to define the action space; wherein, the actions in the action space are K predefined, discretized N-dimensional adjustment vectors; Q-value determination subunit is used to determine the Q-value corresponding to each action in the action space based on the splicing features; A weight adjustment vector determination subunit is used to determine the weight adjustment vector based on the Q value for each action.

[0119] Furthermore, based on any of the above embodiments, the load balancing module 400 may include: The change value determination unit is used to determine the currently effective load balancing weight strategy, and to determine the change value between the latest optimal load balancing weight strategy and the currently effective load balancing weight strategy. The comparison unit is used to compare the change value with a set change threshold. A regular hot update unit is used to determine, when the change value is less than or equal to the change threshold, to perform a regular hot update based on the optimal load balancing weight strategy to perform load balancing processing on the nodes in the backend server cluster; wherein, the regular hot update is a strategy of applying the latest optimal load balancing weight strategy to all traffic. A grayscale adjustment unit is used to determine, when the change value is greater than the change threshold, the optimal load balancing weight strategy, and adjust the currently effective load balancing weight strategy using a grayscale adjustment mechanism to obtain the adjusted optimal load balancing weight strategy, and perform load balancing processing based on the adjusted optimal load balancing weight strategy; wherein, the grayscale adjustment mechanism is that a first proportion of traffic is load balanced based on the optimal load balancing weight strategy, and a second proportion of traffic is load balanced based on the currently effective load balancing weight strategy.

[0120] Furthermore, based on any of the above embodiments, the load balancing device may further include: A performance parameter determination module is used to determine a first performance parameter corresponding to the first proportion of traffic and a second performance parameter corresponding to the second proportion of traffic. A performance comparison module is used to determine, based on the first performance parameter and the second performance parameter, whether the performance of the optimal load balancing weight strategy is better than the performance of the currently effective load balancing weight strategy. A load balancing module is configured to determine load balancing based on the optimal load balancing weight strategy when the performance of the optimal load balancing weight strategy is determined to be better than the performance of the currently effective load balancing weight strategy. The module for determining load balancing based on the currently effective load balancing weight strategy is used to determine load balancing based on the currently effective load balancing weight strategy when it is determined that the performance of the optimal load balancing weight strategy is not better than the performance of the currently effective load balancing weight strategy.

[0121] Furthermore, based on any of the above embodiments, the load balancing device may further include: The state policy data pair acquisition module is used to acquire state policy data pairs; wherein, the state policy data pair is data composed of the combined features of each period and the finally effective load balancing weight policy; The gradient boosting decision tree-based machine learning model adjustment module is used to train the current gradient boosting decision tree-based machine learning model based on the state policy data to obtain the latest gradient boosting decision tree-based machine learning model, and to use the latest gradient boosting decision tree-based machine learning model as the gradient boosting decision tree-based machine learning model in the dynamic load balancing model.

[0122] Furthermore, based on any of the above embodiments, the load balancing device may further include: The replay pool determination module is used to construct a replay pool based on state transition tuples. The data in the state transition tuples include concatenated features, weight adjustment vectors, and reward values. The concatenated features are features corresponding to each time period obtained by concatenating combined features and the optimal weight distribution determined by a machine learning model based on a boosting decision tree. The weight adjustment vector is a vector corresponding to each time period determined based on the deep Q-network. The reward value is a reward value determined based on a reward function and the performance parameter score of the previous period. The reward function is a function constructed based on a set number of performance parameters. The deep Q-network update module is used to update the evaluation network of the deep Q-network based on the data in the replay pool by using an optimizer and a loss function to minimize the gap between the predicted Q-value and the target Q-value.

[0123] It should be noted that the order of the modules and units in the above-mentioned load balancing device can be changed without affecting the logic.

[0124] This invention provides a load balancing device, which may include: an alignment module 100 for aligning backend server data, NGINX server data, and NGINX access log data to obtain aligned data; a target feature determination module 200 for determining the target features of each node in the backend server cluster based on the aligned data; wherein the target features include node local features and node global features; a combination module 300 for standardizing each target feature to obtain standardized features, and combining the standardized features of all nodes to obtain combined features; and a load balancing module 400 for generating an optimal load balancing weight strategy based on the combined features using a dynamic load balancing model, and performing load balancing on the nodes in the backend server cluster based on the optimal load balancing weight strategy; wherein the dynamic load balancing model is a model that dynamically determines node weights using a gradient boosting decision tree-based machine learning model and a deep Q-network. This invention can determine multiple indicators for load balancing based on backend server data, NGINX server data, and NGINX access log data. It also adopts an XGBoost-DQN collaborative architecture, in which XGBoost predicts a stable long-term baseline, and DQN then fine-tunes the baseline based on the real-time status. Therefore, the strategy of this invention combines predictability and real-time performance, thereby improving the accuracy of load balancing.

[0125] The following describes a load balancing device provided by an embodiment of the present invention. The load balancing device described below and the load balancing method described above can be referred to in correspondence.

[0126] Please refer to Figure 6 , Figure 6 A schematic diagram of a load balancing device provided in an embodiment of the present invention may include: Memory 10 is used to store computer programs; Processor 20 is used to execute computer programs to implement the load balancing method described above.

[0127] The memory 10, processor 20, and communication interface 30 all communicate with each other through the communication bus 40.

[0128] In this embodiment of the invention, the memory 10 is used to store one or more programs. The programs may include program code, which includes computer operation instructions. In this embodiment of the invention, the memory 10 may store programs for implementing the following functions: Align the backend server data, NGINX server data, and NGINX access log data to obtain the aligned data. The target features of each node in the backend server cluster are determined based on the aligned data; the target features include node local features and node global features. Each target feature is standardized to obtain standardized features, and the standardized features of all nodes are combined to obtain combined features; Based on the combined features, an optimal load balancing weight strategy is generated through a dynamic load balancing model, and load balancing is performed on the nodes in the backend server cluster based on the optimal load balancing weight strategy. The dynamic load balancing model is a model that dynamically determines the node weights using a gradient-based decision tree machine learning model and a deep Q-network.

[0129] In one possible implementation, the memory 10 may include a program storage area and a data storage area, wherein the program storage area may store the operating system and applications required for at least one function; and the data storage area may store data created during use.

[0130] Furthermore, memory 10 may include read-only memory and random access memory, providing instructions and data to the processor. A portion of the memory may also include NVRAM. The memory stores operating systems and operating instructions, executable modules, or data structures, or subsets thereof, or extended sets thereof, wherein the operating instructions may include various operating instructions for implementing various operations. The operating system may include various system programs for implementing various basic tasks and handling hardware-based tasks.

[0131] Processor 20 can be a central processing unit (CPU), an application-specific integrated circuit, a digital signal processor, a field-programmable gate array, or other programmable logic device. Processor 20 can be a microprocessor or any conventional processor. Processor 20 can call programs stored in memory 10.

[0132] The communication interface 30 can be an interface for the communication module, used to connect with other devices or systems.

[0133] Of course, it should be noted that, Figure 6 The structure shown does not constitute a limitation on the load balancing device in the embodiments of the present invention. In practical applications, the load balancing device may include more than Figure 6 More or fewer components as shown, or combinations of certain components.

[0134] The following describes the computer-readable storage medium provided in the embodiments of the present invention. The computer-readable storage medium described below can be referred to in correspondence with the load balancing method described above.

[0135] The present invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the load balancing method described above.

[0136] The computer-readable storage medium may include various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0137] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section.

[0138] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0139] Finally, it should be noted that in this document, relationships such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0140] The above provides a detailed description of a load balancing method, apparatus, device, and computer-readable storage medium provided by the present invention. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A load balancing method, characterized in that, include: Align the backend server data, NGINX server data, and NGINX access log data to obtain the aligned data. The target features of each node in the backend server cluster are determined based on the aligned data; wherein, the target features include node local features and node global features; Each target feature is standardized to obtain a standardized feature, and the standardized features of all nodes are combined to obtain a combined feature; Based on the combined features, an optimal load balancing weight strategy is generated through a dynamic load balancing model, and load balancing is performed on the nodes in the backend server cluster based on the optimal load balancing weight strategy; wherein, the dynamic load balancing model is a model that dynamically determines the node weights using a gradient boosting decision tree machine learning model and a deep Q-network.

2. The load balancing method according to claim 1, characterized in that, Based on the combined features, an optimal load balancing weight strategy is generated through a dynamic load balancing model, and load balancing processing is performed on the nodes in the backend server cluster based on the optimal load balancing weight strategy, including: Based on the combined features, the gradient boosting decision tree-based machine learning model is used to predict the optimal weight distribution in the current state; wherein, the gradient boosting decision tree-based machine learning model is a model trained based on the combined features and the corresponding actual weight distribution during the training phase. The combined features and the optimal weight distribution are concatenated to obtain the concatenated features; Based on the splicing features, a weight adjustment vector is generated using the deep Q-network. The optimal load balancing weight strategy is determined based on the weight adjustment vector and the optimal weight distribution.

3. The load balancing method according to claim 2, characterized in that, Based on the splicing features, the deep Q-network is used to generate a weight adjustment vector, including: Define an action space; wherein the actions in the action space are K predefined, discretized N-dimensional adjustment vectors; The Q value corresponding to each action in the action space is determined based on the splicing features; The weight adjustment vector is determined based on the Q value for each action.

4. The load balancing method according to any one of claims 1 to 3, characterized in that, Based on the combined features, an optimal load balancing weight strategy is generated through a dynamic load balancing model, and load balancing processing is performed on the nodes in the backend server cluster based on the optimal load balancing weight strategy, including: Determine the currently effective load balancing weight strategy, determine the change between the latest optimal load balancing weight strategy and the currently effective load balancing weight strategy; The change value is compared with a set change threshold; When the change value is less than or equal to the change threshold, a regular hot update is performed based on the optimal load balancing weight strategy to perform load balancing processing on the nodes in the backend server cluster; wherein, the regular hot update is a strategy of applying the latest optimal load balancing weight strategy to all traffic. When the change value exceeds the change threshold, the optimal load balancing weight strategy is determined, and the currently effective load balancing weight strategy is adjusted using a gray-scale adjustment mechanism to obtain the adjusted optimal load balancing weight strategy. Load balancing is then performed based on the adjusted optimal load balancing weight strategy. The gray-scale adjustment mechanism involves a first proportion of traffic being load balanced based on the optimal load balancing weight strategy, and a second proportion of traffic being load balanced based on the currently effective load balancing weight strategy.

5. The load balancing method according to claim 4, characterized in that, When the change value exceeds the change threshold, based on the optimal load balancing weight strategy, the currently effective load balancing weight strategy is adjusted using a gray-scale adjustment mechanism to obtain the adjusted optimal load balancing weight strategy. After performing load balancing processing based on the adjusted optimal load balancing weight strategy, the process further includes: Determine the first performance parameter corresponding to the first proportion of traffic and the second performance parameter corresponding to the second proportion of traffic; Based on the first performance parameter and the second performance parameter, determine whether the performance of the optimal load balancing weight strategy is better than the performance of the currently effective load balancing weight strategy. When it is determined that the performance of the optimal load balancing weight strategy is better than the performance of the currently effective load balancing weight strategy, load balancing is determined to be performed based on the optimal load balancing weight strategy. When it is determined that the performance of the optimal load balancing weight strategy is not better than the performance of the currently effective load balancing weight strategy, load balancing is determined to be performed based on the currently effective load balancing weight strategy.

6. The load balancing method according to claim 1, characterized in that, After generating an optimal load balancing weight strategy based on the combined characteristics using a dynamic load balancing model, and performing load balancing processing on the nodes in the backend server cluster based on the optimal load balancing weight strategy, the process further includes: Obtain state policy data pairs; wherein, the state policy data pairs are data composed of the combined features of each period and the finally effective load balancing weight policy; The current gradient boosting decision tree-based machine learning model is trained based on the state policy data to obtain the latest gradient boosting decision tree-based machine learning model, and the latest gradient boosting decision tree-based machine learning model is used as the gradient boosting decision tree-based machine learning model in the dynamic load balancing model.

7. The load balancing method according to claim 1, characterized in that, After generating an optimal load balancing weight strategy based on the combined characteristics using a dynamic load balancing model, and performing load balancing processing on the nodes in the backend server cluster based on the optimal load balancing weight strategy, the process further includes: A replay pool is constructed based on state transition tuples; wherein, the data in the state transition tuples includes concatenated features, weight adjustment vectors, and reward values; the concatenated features are features corresponding to each time period obtained by concatenating combined features and the optimal weight distribution determined by a machine learning model based on boosting decision trees; the weight adjustment vectors are vectors corresponding to each time period determined based on the deep Q-network; the reward values ​​are reward values ​​determined based on a reward function and the performance parameter scores of the previous period; the reward function is a function constructed based on a set number of performance parameters. Based on the data in the replay pool, the evaluation network of the deep Q-network is updated by using an optimizer and a loss function to minimize the gap between the predicted Q-value and the target Q-value.

8. A load balancing device, characterized in that, include: The alignment module is used to align backend server data, NGINX server data, and NGINX access log data to obtain aligned data. The target feature determination module is used to determine the target features of each node in the backend server cluster based on the aligned data; wherein, the target features include node local features and node global features; The combination module is used to standardize each of the target features to obtain standardized features, and to combine the standardized features of all nodes to obtain combined features; The load balancing module is used to generate an optimal load balancing weight strategy based on the combined features through a dynamic load balancing model, and to perform load balancing processing on the nodes in the backend server cluster based on the optimal load balancing weight strategy; wherein, the dynamic load balancing model is a model that dynamically determines the node weights using a gradient boosting decision tree machine learning model and a deep Q-network.

9. A load balancing device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the steps of the load balancing method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the load balancing method as described in any one of claims 1 to 7.

Images