Automatic extension method and system for TCP long connection server

Abstract

The invention provides a TCP long connection server automatic expansion method and system. The method comprises the following steps: acquiring and monitoring various load indexes of a server in real time; receiving and analyzing the real-time load data, and judging whether resources need to be expanded or reduced currently according to a preset threshold value; based on the current load trend and historical data, performing short-term load prediction by adopting an extended decision algorithm; when expansion or reduction conditions are met, expansion or reduction operation is started, and a capacity expansion and reduction decision instruction is output; configuring a plurality of load balancing strategies, and distributing the received TCP long connection request to each service instance; and dynamically adjusting the size of a connection pool in the TCP long connection service cluster according to the system load condition and the capacity expansion and contraction decision instruction, and executing TCP connection load balancing operation after resource expansion or reduction. According to the invention, dynamic adjustment of resources is realized through technical means of load monitoring, automatic expansion, load balancing, dynamic connection management and the like, so as to meet service requirements under different load conditions.

Description

Technical Field

[0001] The present invention relates to the field of computer network technology, and in particular to a TCP long connection server automatic expansion method for use in the fields of instant messaging systems, Internet of Things platforms, real-time data stream services, and message middleware in microservice architectures. Background Art

[0002] As a core, connection-oriented, reliable transmission protocol in the Internet, TCP (Transmission Control Protocol) plays a vital role in modern network applications that require continuous data transmission. In particular, in business areas such as instant messaging, online games, IoT platforms, and live broadcast systems, the advantages of TCP long connections are particularly significant. It can maintain the session state during the communication process, effectively avoid the waste of resources caused by frequent connection reconstruction, and significantly improve transmission efficiency and user experience.

[0003] However, with the continuous expansion of business scale and the sharp increase in the number of users, TCP persistent connection servers are facing a series of severe technical challenges. The first challenge is the ability to handle high concurrent loads. The server needs to maintain a large number of long-term connected sessions at the same time. When the number of concurrent connections increases sharply, the server's hardware resources (such as CPU, memory, etc.) will be quickly consumed, which may lead to a decrease in response speed or even connection failure. This places extremely high demands on the server's hardware configuration and architecture design.

[0004] Secondly, resource utilization and operation and maintenance costs are also urgent issues to be addressed. To cope with load peaks, traditional methods tend to reserve a large amount of resources to ensure system stability. However, in actual operations, traffic often shows dynamic characteristics, especially in business scenarios such as instant messaging and online live broadcasts, where the difference between peak and trough traffic is huge. This fixed resource allocation method not only leads to low resource utilization, but also significantly increases operation and maintenance costs.

[0005] Furthermore, the scalability and high availability of the system are also crucial. In the long connection scenario, the server needs to be able to dynamically adapt to traffic changes and ensure stable operation when some instances fail or the load is too high. Therefore, the system needs to have excellent elastic scaling capabilities and fault recovery mechanisms to meet users' service needs under different load conditions.

[0006] At present, there are many technical solutions for the automatic expansion of TCP long connection servers, but all of them have certain limitations. The manual expansion solution relies on manual decision-making, and has disadvantages such as untimely response, resource waste and human error; the fixed resource pool solution cannot dynamically respond to traffic changes, resulting in low resource utilization; although the resource scheduling solution based on load balancing can allocate traffic, it cannot achieve automatic expansion of resources. This solution usually requires manual or semi-automatic expansion of instances in combination with the cloud API or the automation script of the operation and maintenance platform. Therefore, although it can improve high availability, it cannot effectively save resources, and it cannot automatically reduce resources according to load fluctuations; and although the automatic expansion solution of the cloud platform has achieved automated resource management to a certain extent, its response speed may not be flexible enough, and the expansion strategy is usually based on simple threshold judgment, which is difficult to meet complex dynamic adjustment needs.

[0007] Specifically, the prior art has the following major defects:

[0008] Inability to dynamically scale in real time: Most existing solutions trigger resource expansion based on fixed thresholds, with a low degree of automation, making it difficult to flexibly respond to traffic bursts or real-time load changes.

[0009] Resource waste and management complexity: Fixed resource pools or manual expansion methods result in a large amount of idle resources during low loads, and insufficient resources during peak hours, resulting in low overall resource utilization. At the same time, the cloud platform's automatic expansion lacks specific optimization for TCP long connections and cannot effectively manage the resource configuration required for a large number of long-term connections.

[0010] Inflexible connection management: Although dynamic connection pools and load balancing solutions can allocate the number of connections, the management mechanism for idle connections and timed connections is not perfect, making it difficult to effectively control the life cycle of connections, resulting in large fluctuations in system load.

[0011] In summary, the existing technology still has many deficiencies in the automatic expansion of TCP persistent connection servers, and cannot meet the growing business needs and user experience requirements. Therefore, an improved TCP persistent connection server automatic expansion method is urgently needed to overcome the limitations of the existing technology, achieve efficient resource utilization, timely load response, and flexible control of connection management. Summary of the invention

[0012] In order to solve the problems existing in the prior art, the purpose of the present invention is to provide a TCP long connection server automatic expansion method and system, which realizes dynamic adjustment of resources through technical means such as load monitoring, automatic expansion, load balancing and dynamic connection management to meet service requirements under different load conditions.

[0013] The present invention achieves the above-mentioned purpose through the following technical solutions:

[0014] A TCP long connection server automatic expansion method, comprising:

[0015] Load monitoring steps: collect and monitor various load indicators of the server in real time to comprehensively evaluate the load status of the server;

[0016] Scaling decision step: Receive and analyze the real-time load data provided by the load monitoring step, and determine whether resources need to be expanded or reduced based on the preset threshold; use the expansion decision algorithm to make short-term load forecasts based on the current load trend and historical data; initiate the expansion or reduction operation when the expansion or reduction conditions are met, and output the expansion or reduction decision instructions;

[0017] Load balancing step: Configure and apply multiple load balancing strategies to evenly distribute the received TCP persistent connection requests to each service instance to balance the load of each instance;

[0018] Service cluster management steps: According to the system load and expansion and contraction decision instructions, dynamically adjust the connection pool size in the TCP persistent connection service cluster, including the adjustment of the minimum and maximum number of connections, and perform TCP connection load balancing operations after resources are expanded or reduced.

[0019] According to a TCP long connection server automatic expansion method provided by the present invention, the load index includes:

[0020] CPU usage, used to evaluate the computing pressure of the server;

[0021] Memory usage, used to determine the system's memory usage;

[0022] The current number of connections is used to count the number of current TCP long connections.

[0023] According to a TCP long connection server automatic expansion method provided by the present invention, the short-term load prediction is performed using an expansion decision algorithm, including:

[0024] Collect historical load data;

[0025] Using the ARIMA time series model or other time series forecasting techniques, the collected historical load data is trained and learned to build a forecasting model for predicting future load changes;

[0026] According to the constructed prediction model, the current load trend data is input to generate the load prediction results for a period of time in the future, which include the predicted load peak, load growth trend and possible load fluctuation range;

[0027] Based on the load forecast results and the preset expansion strategy, the system automatically calculates and determines the number of service instances that need to be expanded and the best expansion time.

[0028] When the expansion conditions are met, the expansion operation is triggered based on the calculation results, and at the same time, an expansion instruction is sent to the service cluster module through a preset communication mechanism to dynamically increase the service instance.

[0029] According to a TCP persistent connection server automatic expansion method provided by the present invention, the configuration and application of multiple load balancing strategies include:

[0030] Polling scheduling strategy: Allocate new TCP connection requests to each service instance in turn according to the preset order; in which, under long-term operation, the number of connections handled by each service instance tends to be balanced to avoid a situation where one instance is overloaded while other instances are idle;

[0031] Minimum connection number strategy: monitors the current number of connections of each service instance in real time, and assigns new TCP connection requests to the service instance with the least current number of connections to balance the load of each instance;

[0032] Connection health check-based strategy: Perform health checks on each service instance regularly or on demand, and evaluate the health of the instance by sending probe requests or monitoring key performance indicators.

[0033] According to a TCP long connection server automatic expansion method provided by the present invention, when it is detected that an instance is unavailable or the response delay exceeds a preset threshold, the instance is automatically excluded from the load balancing distribution list until it recovers to a healthy state and then re-added to the distribution list, thereby ensuring that TCP connection requests are only allocated to healthy and available service instances.

[0034] According to a TCP long connection server automatic expansion method provided by the present invention, when building a prediction model, the following steps are specifically included:

[0035] Clean the collected historical load data, remove outliers, fill missing values, and perform smoothing. At the same time, perform seasonal adjustments or trend decomposition on the data to extract long-term trends, seasonal changes, and random fluctuations in the load data.

[0036] Extract features related to load prediction based on the preprocessed data;

[0037] Select ARIMA model parameters or other time series forecasting techniques based on the characteristics of historical load data and forecasting requirements; build an initial forecasting model using the extracted features and the selected model, and set the model’s training parameters;

[0038] The historical load data after preprocessing and feature extraction is input into the initial prediction model, and the model parameters are adjusted through iterative optimization algorithm to enable the model to learn the inherent laws and trends of the historical load data;

[0039] Use the rolling window verification method unique to time series to verify the trained prediction model and evaluate the prediction accuracy and stability of the model;

[0040] The verified and optimized prediction model is deployed to the load balancing control system for real-time or periodic prediction of future load changes. At the same time, a model update mechanism is established to regularly collect new load data and retrain and update the model.

[0041] According to a TCP persistent connection server automatic expansion method provided by the present invention, the connection pool size in the TCP persistent connection service cluster is dynamically adjusted according to the system load and the instructions of the expansion and contraction decision step, including:

[0042] According to the expansion and contraction decision instructions and the current system load, calculate the target value of the connection pool adjustment, including the new minimum and maximum number of connections; based on the calculated target value, dynamically adjust the connection pool configuration of each service instance in the TCP long connection service cluster; adjustment effect monitoring and feedback: After the connection pool adjustment is executed, continuously monitor the system load and connection pool usage to evaluate whether the adjustment effect meets expectations.

[0043] According to a TCP long connection server automatic expansion method provided by the present invention, the following steps are further performed:

[0044] Build a connection pool usage efficiency evaluation model. Based on the collected data, the model uses machine learning algorithms or statistical methods to analyze the relationship between connection pool configuration and service instance performance, evaluate resource utilization and service response speed under the current connection pool configuration, and identify improper configuration or inefficiency.

[0045] Based on the results of the efficiency evaluation model based on the adaptive adjustment algorithm, the fine-tuning values ​​of the connection pool configuration parameters are automatically calculated; based on the output of the adaptive adjustment algorithm, an adjustment strategy is formulated, including the timing, frequency, amplitude of the adjustment and the verification method after the adjustment; the adjustment strategy is automatically executed, and the fine-tuned configuration parameters are applied to the corresponding service instances through the configuration management system, API calls or other automated means.

[0046] According to a TCP persistent connection server automatic expansion method provided by the present invention, after resources are expanded or reduced, a TCP connection load balancing operation is performed, comprising:

[0047] After the resource expansion or reduction operation is completed, the resource status information of the load balancer is updated synchronously immediately;

[0048] Use connection migration strategies to smoothly migrate existing TCP connections from old service instances to new service instances when resources change, or reallocate existing connections to avoid overloading certain instances.

[0049] Dynamically adjust the parameters of the load balancing algorithm based on the number and performance of service instances after resource expansion or reduction;

[0050] After the resources change, the newly added service instance is immediately checked for health to ensure that it can process connection requests normally. At the same time, the health status of all service instances is continuously monitored. Once a failure or performance degradation is found, it is removed from the load balancer and the affected connections are reallocated to other healthy instances.

[0051] Dynamically adjust the connection pool configuration of each service instance, including the minimum and maximum number of connections, based on resource expansion or reduction;

[0052] After the load balancing operation is executed, the overall performance indicators of the system are continuously monitored, and the load balancing strategy, connection migration strategy, and connection pool configuration are fine-tuned and optimized based on the monitoring results to improve the stability and efficiency of the system.

[0053] A TCP long connection server automatic expansion system, comprising:

[0054] The load monitoring module is used to collect and monitor various load indicators of the server in real time to comprehensively evaluate the load status of the server;

[0055] The expansion and contraction decision module is used to receive and analyze the real-time load data provided by the load monitoring step, and determine whether resources need to be expanded or reduced according to the preset threshold; based on the current load trend and historical data, the expansion decision algorithm is used to perform short-term load forecasting; when the expansion or reduction conditions are met, the expansion or reduction operation is initiated, and the expansion or contraction decision instruction is output;

[0056] The load balancing module is used to configure and apply multiple load balancing strategies to evenly distribute the received TCP long connection requests to each service instance to balance the load of each instance;

[0057] The service cluster management module is used to dynamically adjust the connection pool size in the TCP persistent connection service cluster according to the system load and expansion and contraction decision instructions, including the adjustment of the minimum and maximum number of connections, and perform TCP connection load balancing operations after resources are expanded or reduced.

[0058] It can be seen that compared with the prior art, the present invention has the following beneficial effects:

[0059] 1. The present invention uses advanced monitoring technology to monitor the load of the TCP long connection server in real time and accurately, including key indicators such as the number of connections, CPU usage, and memory occupancy. The present invention combines machine learning algorithms to conduct in-depth analysis and prediction of load data, and can identify the trend of load changes in advance, providing a scientific basis for resource adjustment. This intelligent monitoring and prediction mechanism enables the system to respond quickly to load changes, effectively avoiding service interruptions or performance degradation caused by load surges.

[0060] 2. The present invention designs an efficient dynamic load balancing algorithm that can automatically distribute new connections to instances with lighter loads according to the current load conditions of service instances. Through this flexible load balancing strategy, efficient resource utilization and load balancing are achieved, and the overall throughput and response speed of the system are improved.

[0061] 3. The present invention realizes the automatic management of resources, and can automatically trigger the expansion or reduction of resources according to the load conditions. Through deep integration with the cloud platform, it can quickly and flexibly adjust the number and configuration of server instances to meet the ever-changing business needs. This automatic and flexible resource management method significantly reduces resource costs and avoids waste and idleness of resources.

[0062] 4. Since the present invention can respond to load changes in real time and automatically adjust resource allocation, it effectively avoids service interruption or performance degradation caused by insufficient or overloaded resources. At the same time, flexible load balancing strategies and automated resource management also improve the system's fault tolerance and recovery capabilities, ensuring the high availability of the system.

[0063] In summary, the present invention significantly improves the response capability of the TCP long connection server in load change scenarios through intelligent load monitoring and prediction, flexible dynamic load balancing strategy, and automated and elastic resource expansion and reduction technology, effectively reduces resource costs and greatly improves system reliability, and provides more efficient and stable service support for modern Internet applications.

[0064] The present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] Figure 1 It is a flow chart of an embodiment of a method for automatically extending a TCP long connection server of the present invention.

[0066] Figure 2 It is a schematic diagram of an embodiment of a TCP long connection server automatic expansion system of the present invention. DETAILED DESCRIPTION

[0067] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0068] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0069] See also Figure 1 This embodiment provides a method for automatically extending a TCP persistent connection server, comprising the following steps:

[0070] S1-Load monitoring step: collect and monitor various load indicators of the server in real time to comprehensively evaluate the load status of the server.

[0071] S2-Scaling decision step: Receive and analyze the real-time load data provided by the load monitoring step, and determine whether resources need to be expanded or reduced based on the preset threshold; based on the current load trend and historical data, use the expansion decision algorithm to perform short-term load forecasting; when the expansion or reduction conditions are met, start the expansion or reduction operation, and output the expansion or reduction decision instructions.

[0072] S3-Load balancing step: Configure and apply multiple load balancing strategies to evenly distribute the received TCP long connection requests to each service instance to balance the load of each instance.

[0073] S4-Service cluster management step: According to the system load and expansion and contraction decision instructions, dynamically adjust the connection pool size in the TCP persistent connection service cluster, including the adjustment of the minimum and maximum number of connections, and perform TCP connection load balancing operations after resource expansion or contraction. For example, when the load is large, the size of the connection pool can be increased to avoid connection requests being rejected; when the load is reduced, the connection pool can be reduced to reduce resource usage.

[0074] In this embodiment, the load indicators include: CPU usage, which is used to evaluate the computing pressure of the server; memory usage, which is used to determine the memory usage of the system; and the current number of connections, which is used to count the number of current TCP long connections.

[0075] In the scaling decision step, the preset threshold judgment includes: when the monitoring data exceeds the preset threshold (for example, the CPU usage exceeds 80%, or the number of connections reaches 90% of the upper limit), the system will trigger the expansion decision.

[0076] In the scaling decision step, an expansion decision algorithm is used to perform short-term load forecasting, including:

[0077] Collect historical load data, including but not limited to key performance indicators such as the number of TCP connections, CPU usage, memory usage, network bandwidth usage, and response time.

[0078] The collected historical load data is trained and learned using the ARIMA time series model or other time series forecasting techniques to build a forecasting model for predicting future load changes. Based on the constructed forecasting model, the current load trend data is input to generate load forecast results for a period of time in the future, which include the predicted load peak, load growth trend, and possible load fluctuation range.

[0079] According to the load forecast results, combined with the preset expansion strategies, such as resource reservation ratio, expansion cost-benefit analysis, etc., the number of service instances that need to be expanded and the best time to expand are automatically calculated and determined; when the expansion conditions are met, based on the calculation results, the expansion operation is triggered, and at the same time, the expansion instructions are sent to the service cluster module through the preset communication mechanism to dynamically increase the service instances, so as to effectively cope with the future increase in load and ensure the stability and availability of the service.

[0080] In this embodiment, multiple load balancing strategies are configured and applied, including:

[0081] Polling scheduling strategy: New TCP connection requests are assigned to each service instance in turn according to a preset order; in which, under long-term operation, the number of connections handled by each service instance tends to be balanced to avoid a situation where one instance is overloaded while other instances are idle.

[0082] Minimum connection number strategy: monitors the current number of connections of each service instance in real time, and assigns new TCP connection requests to the service instance with the least current number of connections to balance the load of each instance. This is especially effective in scenarios where the processing capabilities of each instance vary or the load fluctuates greatly.

[0083] Connection health check-based strategy: Perform health checks on each service instance regularly or on demand, and evaluate the instance's operating status by sending probe requests or monitoring key performance indicators. When an instance is detected to be unavailable or the response delay exceeds the preset threshold, the instance is automatically excluded from the load balancing allocation list until it recovers and then re-joins the allocation list, thereby ensuring that TCP connection requests are only allocated to healthy and available service instances, improving the overall stability and reliability of the service.

[0084] In this embodiment, when constructing a prediction model, the following steps are specifically included:

[0085] The collected historical load data is cleaned to remove outliers, fill in missing values, and perform smoothing. At the same time, the data is seasonally adjusted or trend decomposed to extract long-term trends, seasonal changes, and random fluctuations in the load data.

[0086] Based on the preprocessed data, features related to load forecasting are extracted, including but not limited to statistical features of time series (such as mean, variance, autocorrelation coefficient, etc.), frequency domain features of time series (such as spectrum analysis results), and other features selected based on business experience.

[0087] According to the characteristics of historical load data and forecasting requirements, select ARIMA model parameters or other time series forecasting technologies, such as long short-term memory network (LSTM), support vector machine (SVM) or random forest; use the extracted features and the selected model to build the initial forecasting model, and set the model training parameters, such as learning rate, number of iterations, etc.

[0088] The historical load data after preprocessing and feature extraction is input into the initial prediction model, and the model parameters are adjusted through iterative optimization algorithms (such as gradient descent, genetic algorithm, etc.) so that the model can learn the inherent laws and trends of the historical load data.

[0089] Use methods such as holdout, cross-validation, or time series-specific rolling window validation to validate the trained prediction model and evaluate the prediction accuracy and stability of the model; deploy the verified and optimized prediction model to the load balancing control system for real-time or periodic prediction of future load changes; at the same time, establish a model update mechanism to regularly collect new load data, retrain and update the model to adapt to changes in load patterns and developments in business needs.

[0090] In the service cluster management step, the connection pool size in the TCP persistent connection service cluster is dynamically adjusted according to the system load and the instructions of the expansion and contraction decision step, including:

[0091] Based on the scaling decision instructions and the current system load, including but not limited to indicators such as the number of TCP connections, request processing rate, response time, and resource utilization of service instances.

[0092] According to the expansion and contraction decision instructions and the current load of the system, the target value of the connection pool adjustment is calculated. Based on the calculated target value, the connection pool configuration of each service instance in the TCP persistent connection service cluster is dynamically adjusted, including modifying the configuration file, restarting the service instance, or calling the service interface, so as to realize the instant adjustment of the connection pool size.

[0093] Adjustment effect monitoring and feedback: After the connection pool adjustment is executed, the system load and connection pool usage are continuously monitored to evaluate whether the adjustment effect meets expectations, such as the connection establishment success rate, connection idle rate, and service response time. Based on the monitoring results, further adjustments are made if necessary or a re-evaluation of the expansion and contraction decision module is triggered.

[0094] In the service cluster management step, after resources are expanded or reduced, TCP connection load balancing operations are performed, including:

[0095] After the resource expansion or reduction operation is completed, the resource status information of the load balancer is immediately updated synchronously, including the IP address, port number, connection pool status, etc. of the newly added or removed service instance, to ensure that the load balancer has the latest service instance list and status.

[0096] Use connection migration strategies to smoothly migrate existing TCP connections from old service instances to new service instances when resources change, or reallocate existing connections to avoid overloading certain instances. The connection migration strategy should consider connection persistence, session state consistency, and minimize the impact on clients and services;

[0097] According to the number and performance of service instances after resource expansion or reduction, dynamically adjust the parameters of the load balancing algorithm or select a more appropriate algorithm, such as polling, minimum number of connections, consistent hashing, etc.

[0098] After the resources change, the newly added service instance is immediately checked for health to ensure that it can handle connection requests normally. At the same time, the health status of all service instances is continuously monitored. Once a failure or performance degradation is found, it is removed from the load balancer and the affected connections are reallocated to other healthy instances.

[0099] Dynamically adjust the connection pool configuration of each service instance, including the minimum and maximum number of connections, based on resource expansion or reduction. After the load balancing operation is executed, continuously monitor the overall performance indicators of the system, and fine-tune and optimize the load balancing strategy, connection migration strategy, and connection pool configuration based on the monitoring results to improve the stability and efficiency of the system.

[0100] In this embodiment, the following is also performed:

[0101] Build a connection pool usage efficiency evaluation model. Based on the collected data, the model uses machine learning algorithms or statistical methods to analyze the relationship between connection pool configuration and service instance performance, evaluate resource utilization and service response speed under the current connection pool configuration, and identify improper configuration or inefficiency.

[0102] Based on the results of the efficiency evaluation model based on the adaptive adjustment algorithm, the fine-tuning values ​​of the connection pool configuration parameters are automatically calculated, including increasing or decreasing the minimum / maximum number of connections, adjusting the connection timeout, optimizing the connection idle time, etc.; based on the output of the adaptive adjustment algorithm, an adjustment strategy is formulated, including the timing, frequency, amplitude of the adjustment, and the verification method after the adjustment; the adjustment strategy is automatically executed, and the fine-tuned configuration parameters are applied to the corresponding service instances through the configuration management system, API calls or other automated means.

[0103] Specifically, when starting an expansion or reduction operation, assume that there is a service cluster containing three instances: Instance A, Instance B, and Instance C. The system monitors the following data:

[0104] Extensions:

[0105] Assuming the number of connections:

[0106] Instance A: 70 connections

[0107] Instance B: 80 connections

[0108] Instance C: 90 connections (close to the upper limit)

[0109] Triggering expansion: When the average CPU usage exceeds 80% or the number of connections approaches the upper limit, the system triggers expansion.

[0110] Execute scaling: The system starts a new instance (Instance D) and configures the load balancer to distribute traffic to the new instance.

[0111] After expansion:

[0112] Instance A: 60 connections

[0113] Instance B: 60 connections

[0114] Instance C: 60 connections

[0115] Instance D: 60 connections (new instance)

[0116] reduce:

[0117] Assuming the number of connections:

[0118] Instance A: 40 connections

[0119] Instance B: 40 connections

[0120] Instance C: 30 connections

[0121] Instance D: 10 connections

[0122] Trigger reduction: The average CPU usage is less than 30% or the number of connections is less than 30. The system determines that so many instances are no longer needed and prepares to destroy the machine with the lowest load.

[0123] Execution reduction: The system no longer allocates new traffic to Instance D and sends new connection instances (Instance A, Instance B, Instance C) to the client. After all clients are connected to the new service instance, Instance D is destroyed to release resources.

[0124] After reduction:

[0125] Instance A: 40 connections

[0126] Instance B: 40 connections

[0127] Instance C: 40 connections.

[0128] When allocating new TCP connection requests to the service instance with the least number of current connections, assume that there is a service cluster consisting of three instances: Instance A, Instance B, and Instance C. The current number of connections for each instance is as follows:

[0129] Instance A: The current number of connections is 5

[0130] Instance B: The current number of connections is 3

[0131] Instance C: The current number of connections is 8

[0132] Assume that a new request needs to be assigned to a service instance. When using the minimum number of connections load balancing strategy, the new request will be assigned to the instance with the least number of connections to achieve load balancing. The new request will be assigned to Instance B, and the current number of connections will become 4.

[0133] A TCP long connection server automatic expansion system embodiment

[0134] like Figure 2 As shown, this embodiment provides a TCP long connection server automatic expansion system, including:

[0135] The load monitoring module is used to collect and monitor various load indicators of the server in real time to comprehensively evaluate the load status of the server;

[0136] The expansion and contraction decision module is used to receive and analyze the real-time load data provided by the load monitoring step, and determine whether resources need to be expanded or reduced according to the preset threshold; based on the current load trend and historical data, the expansion decision algorithm is used to perform short-term load forecasting; when the expansion or reduction conditions are met, the expansion or reduction operation is initiated, and the expansion or contraction decision instruction is output;

[0137] The load balancing module is used to configure and apply multiple load balancing strategies to evenly distribute the received TCP long connection requests to each service instance to balance the load of each instance;

[0138] The service cluster management module is used to dynamically adjust the connection pool size in the TCP persistent connection service cluster according to the system load and expansion and contraction decision instructions, including the adjustment of the minimum and maximum number of connections, and perform TCP connection load balancing operations after resources are expanded or reduced.

[0139] In actual applications, in high-concurrency scenarios, the specific workflow of the system is as follows:

[0140] Load monitoring: The load monitoring module detects that the number of connections is increasing rapidly and the CPU usage has exceeded the preset threshold.

[0141] Scaling decision: The scaling decision module analyzes historical data and real-time load data and predicts that the high load will continue for a period of time, so it decides to add 3 service instances. Three new instances are launched through Kubernetes.

[0142] Service cluster module: After the service is started, TCP long connection balancing operation is performed, and after completion, service registration is performed (such as using ETCD for service registration). The load monitoring module monitors through service discovery (such as using ETCD for service discovery).

[0143] Load balancing: The load balancing module allocates connection addresses according to the load situation

[0144] Load reduction: When the load monitoring module detects that the number of connections and CPU usage are gradually decreasing, the expansion decision module decides to reduce two instances, and the load balancing no longer allocates new connections to the reduced instances. The reduced service instance notifies the client to connect to the new service instance, and disconnects the TCP long connection after completion. Finally, the service instance is closed.

[0145] Among them, in the design of the load balancing module, the key is to balance the load of each service instance by intelligently allocating connection requests to ensure stability and performance in high-concurrency scenarios. Based on the resource utilization of each instance (such as CPU and memory), priority is given to instances with sufficient resources. Suppose we have a service cluster containing 3 instances: Instance A, Instance B, and Instance C. The load of each instance includes: Instance A: CPU utilization 80%; Instance B: CPU utilization 85%; Instance C: CPU utilization 66%. Here, the address of Instance C will be allocated to allow the client to connect to Instance C.

[0146] In summary, the present invention significantly improves the response capability of the TCP long connection server in load change scenarios through intelligent load monitoring and prediction, flexible dynamic load balancing strategy, and automated and elastic resource expansion and reduction technology, effectively reduces resource costs and greatly improves system reliability, and provides more efficient and stable service support for modern Internet applications.

[0147] Furthermore, the present invention uses advanced monitoring technology to monitor the load of the TCP long connection server in real time and accurately, including key indicators such as the number of connections, CPU usage, and memory occupancy. The present invention combines machine learning algorithms to conduct in-depth analysis and prediction of load data, and can identify the trend of load changes in advance, providing a scientific basis for resource adjustment. This intelligent monitoring and prediction mechanism enables the system to respond quickly to load changes, effectively avoiding service interruptions or performance degradation caused by load surges.

[0148] Furthermore, the present invention designs an efficient dynamic load balancing algorithm, which can automatically distribute new connections to instances with lighter loads according to the current load conditions of service instances. Through this flexible load balancing strategy, efficient resource utilization and load balancing are achieved, and the overall throughput and response speed of the system are improved.

[0149] Furthermore, the present invention realizes the automatic management of resources, can automatically trigger the expansion or reduction of resources according to the load situation, and through deep integration with the cloud platform, can quickly and flexibly adjust the number and configuration of server instances to meet the ever-changing business needs. This automatic and flexible resource management method significantly reduces resource costs and avoids waste and idleness of resources.

[0150] Furthermore, since the present invention can respond to load changes in real time and automatically adjust resource allocation, it effectively avoids service interruption or performance degradation caused by insufficient or overloaded resources. At the same time, flexible load balancing strategies and automated resource management also improve the system's fault tolerance and recovery capabilities, ensuring the high availability of the system.

[0151] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0152] The above-mentioned embodiments are only preferred embodiments of the present invention and cannot be used to limit the scope of protection of the present invention. Any non-substantial changes and substitutions made by technicians in this field on the basis of the present invention shall fall within the scope of protection required by the present invention.

Claims

1. A method for automatically extending a TCP long connection server, characterized in that: The following steps are involved: Load monitoring steps: collect and monitor various load indicators of the server in real time to comprehensively evaluate the load status of the server; Scaling decision step: Receive and analyze the real-time load data provided by the load monitoring step, and determine whether resources need to be expanded or reduced based on the preset threshold; use the expansion decision algorithm to make short-term load forecasts based on the current load trend and historical data; initiate the expansion or reduction operation when the expansion or reduction conditions are met, and output the expansion or reduction decision instructions; Load balancing step: Configure and apply multiple load balancing strategies to evenly distribute the received TCP persistent connection requests to each service instance to balance the load of each instance; Service cluster management steps: According to the system load and expansion and contraction decision instructions, dynamically adjust the connection pool size in the TCP persistent connection service cluster, including the adjustment of the minimum and maximum number of connections, and perform TCP connection load balancing operations after resources are expanded or reduced.

2. The method according to claim 1, characterized in that: The load indicators include: CPU usage, used to evaluate the computing pressure of the server; Memory usage, used to determine the system's memory usage; The current number of connections is used to count the number of current TCP long connections.

3. The method according to claim 1, characterized in that The short-term load forecasting using an extended decision algorithm includes: Collect historical load data; Using the ARIMA time series model or other time series forecasting techniques, the collected historical load data is trained and learned to build a forecasting model for predicting future load changes; According to the constructed prediction model, the current load trend data is input to generate the load prediction results for a period of time in the future, which include the predicted load peak, load growth trend and possible load fluctuation range; Based on the load forecast results and the preset expansion strategy, the system automatically calculates and determines the number of service instances that need to be expanded and the best expansion time. When the expansion conditions are met, the expansion operation is triggered based on the calculation results, and at the same time, an expansion instruction is sent to the service cluster module through a preset communication mechanism to dynamically increase the service instance.

4. The method according to claim 1, characterized in that The configuration and application of multiple load balancing strategies include: Polling scheduling strategy: Allocate new TCP connection requests to each service instance in turn according to the preset order; in which, under long-term operation, the number of connections handled by each service instance tends to be balanced to avoid a situation where one instance is overloaded while other instances are idle; Minimum connection number strategy: monitors the current number of connections of each service instance in real time, and assigns new TCP connection requests to the service instance with the least current number of connections to balance the load of each instance; Connection health check-based strategy: Perform health checks on each service instance regularly or on demand, and evaluate the health of the instance by sending probe requests or monitoring key performance indicators.

5. The method according to claim 4, characterized in that: When an instance is detected to be unavailable or the response delay exceeds the preset threshold, the instance is automatically excluded from the load balancing allocation list until it recovers and then re-added to the allocation list, thus ensuring that TCP connection requests are only allocated to healthy and available service instances.

6. The method according to claim 3, characterized in that: When building a prediction model, the following steps are included: Clean the collected historical load data, remove outliers, fill missing values, and perform smoothing. At the same time, perform seasonal adjustments or trend decomposition on the data to extract long-term trends, seasonal changes, and random fluctuations in the load data. Extract features related to load prediction based on the preprocessed data; Select ARIMA model parameters or other time series forecasting techniques based on the characteristics of historical load data and forecasting requirements; build an initial forecasting model using the extracted features and the selected model, and set the model’s training parameters; The historical load data after preprocessing and feature extraction is input into the initial prediction model, and the model parameters are adjusted through iterative optimization algorithm to enable the model to learn the inherent laws and trends of the historical load data; Use the rolling window verification method unique to time series to verify the trained prediction model and evaluate the prediction accuracy and stability of the model; The verified and optimized prediction model is deployed to the load balancing control system for real-time or periodic prediction of future load changes. At the same time, a model update mechanism is established to regularly collect new load data and retrain and update the model.

7. The method according to claim 1, characterized in that The dynamically adjusting the connection pool size in the TCP persistent connection service cluster according to the system load and the instructions of the expansion and contraction decision step includes: According to the expansion and contraction decision instructions and the current system load, calculate the target value of the connection pool adjustment, including the new minimum and maximum number of connections; based on the calculated target value, dynamically adjust the connection pool configuration of each service instance in the TCP long connection service cluster; adjustment effect monitoring and feedback: After the connection pool adjustment is executed, continuously monitor the system load and connection pool usage to evaluate whether the adjustment effect meets expectations.

8. The method according to any one of claims 1 to 7, characterized in that: Also execute: Build a connection pool usage efficiency evaluation model. Based on the collected data, the model uses machine learning algorithms or statistical methods to analyze the relationship between connection pool configuration and service instance performance, evaluate resource utilization and service response speed under the current connection pool configuration, and identify improper configuration or inefficiency. Automatically calculate the fine-tuning values ​​of connection pool configuration parameters based on the results of the efficiency evaluation model based on the adaptive adjustment algorithm; Based on the output of the adaptive adjustment algorithm, formulate an adjustment strategy, including the timing, frequency, magnitude of the adjustment, and the verification method after the adjustment; Automated execution of adjustment strategies is achieved, and fine-tuned configuration parameters are applied to corresponding service instances through configuration management systems, API calls, or other automated means.

9. The method according to any one of claims 1 to 7, characterized in that: After the resources are expanded or reduced, the TCP connection load balancing operation is performed, including: After the resource expansion or reduction operation is completed, the resource status information of the load balancer is updated synchronously immediately; Use connection migration strategies to smoothly migrate existing TCP connections from old service instances to new service instances when resources change, or reallocate existing connections to avoid overloading certain instances. Dynamically adjust the parameters of the load balancing algorithm based on the number and performance of service instances after resource expansion or reduction; After the resources change, the newly added service instance is immediately checked for health to ensure that it can process connection requests normally. At the same time, the health status of all service instances is continuously monitored. Once a failure or performance degradation is found, it is removed from the load balancer and the affected connections are reallocated to other healthy instances. Dynamically adjust the connection pool configuration of each service instance, including the minimum and maximum number of connections, based on resource expansion or reduction; After the load balancing operation is executed, the overall performance indicators of the system are continuously monitored, and the load balancing strategy, connection migration strategy, and connection pool configuration are fine-tuned and optimized based on the monitoring results to improve the stability and efficiency of the system.

10. A TCP long connection server automatic expansion system, characterized in that: include: The load monitoring module is used to collect and monitor various load indicators of the server in real time to comprehensively evaluate the load status of the server; The expansion and contraction decision module is used to receive and analyze the real-time load data provided by the load monitoring step, and determine whether resources need to be expanded or reduced according to the preset threshold; based on the current load trend and historical data, the expansion decision algorithm is used to perform short-term load forecasting; when the expansion or reduction conditions are met, the expansion or reduction operation is initiated, and the expansion or contraction decision instruction is output; The load balancing module is used to configure and apply multiple load balancing strategies to evenly distribute the received TCP long connection requests to each service instance to balance the load of each instance; The service cluster management module is used to dynamically adjust the connection pool size in the TCP persistent connection service cluster according to the system load and expansion and contraction decision instructions, including the adjustment of the minimum and maximum number of connections, and perform TCP connection load balancing operations after resources are expanded or reduced.

Images