Netty adaptive connection method and system based on EventLoop affinity and PID control

Abstract

The invention provides a Netty adaptive connection method and system based on EventLoop affinity and PID (Proportion Integration Differentiation) control, and relates to the field of computer network communication. The method comprises the steps of obtaining a service thread request queue, determining an EventLoop thread of a current service, and determining available connection resources in a hierarchical architecture connection pool according to a local priority principle; the hierarchical architecture connection pool is a hierarchical architecture connection pool based on EventLoop affinity, and comprises a thread local pool and a global shared pool; periodically calculating a pressure value representing a system load state based on the multi-dimensional index of the service request queue; based on the pressure value, determining a resource capacity variation through a feedback control algorithm; and on the basis of the capacity variation, performing capacity expansion or capacity reduction operation of connection resources on the thread local pool according to a preset rule, and recovering redundant connections generated by capacity reduction or idle to the global shared pool. The method and the device are used in a Netty self-adaptive connection process based on EventLoop affinity and PID control, and the technical problem of multi-thread lock competition existing in existing Netty connection is solved.

Description

Technical Field

[0001] This application relates to the field of computer network communication, and in particular to a Netty adaptive connection method and system based on EventLoop affinity and PID control. Background Technology

[0002] Netty, as a high-performance, asynchronous event-driven network application framework, is widely used in high-concurrency scenarios such as remote procedure call frameworks, API gateways, and instant messaging systems. In such systems, TCP connection management is a core factor affecting throughput and system stability. Currently, Netty's connection management technology mainly adopts two schemes. The first is a fixed-size connection pool scheme, which pre-creates a fixed number of connections and places them in the pool when the system starts. When a business thread needs to communicate, it obtains a connection from the pool; after use, it returns the connection. Its core logic is to assume that business traffic is relatively stable and avoid the overhead of frequently establishing connections by reusing fixed resources. The second is a threshold-based dynamic connection pool scheme, which adds elastic scaling capabilities to the fixed connection pool. Typically, a minimum and maximum number of connections are set, and a single metric, such as the current number of active connections, is monitored. When the number of active connections exceeds the set threshold, expansion is triggered; when it falls below the threshold, shrinkage is triggered. Although the above two schemes can work normally under normal traffic, in complex scenarios with ultra-high concurrency and drastic traffic fluctuations, there are serious multi-threaded lock contention problems. Existing connection pool implementations typically employ a globally shared queue structure. However, in Netty's Reactor model, multiple EventLoop threads concurrently access the same connection pool to acquire or return connections, leading to contention for the same lock. This high-frequency lock contention not only causes severe CPU context switching but also increases I / O latency, contradicting Netty's high-performance design principles of "lock-free" and "event-driven" architecture. Therefore, a method is urgently needed to address the technical problem of multi-threaded lock contention in existing Netty connections. Summary of the Invention

[0003] This application provides a Netty adaptive connection method and system based on EventLoop affinity and PID control, which solves the technical problem of multi-threaded lock contention in existing Netty connections.

[0004] To achieve the above objectives, this application adopts the following technical solution: Firstly, a Netty adaptive connection method based on EventLoop affinity and PID control is provided, including: obtaining the business thread request queue and determining the current business's EventLoop thread; determining available connection resources in the hierarchical connection pool according to the local priority principle based on the current business's EventLoop thread; the hierarchical connection pool is a hierarchical connection pool based on EventLoop affinity, including a thread-local pool and a global shared pool; periodically calculating the pressure value characterizing the system load state based on multi-dimensional indicators of the business request queue; the multi-dimensional indicators refer to the backlog rate of the business request queue within a preset period, TCP RTT latency, and the task backlog of the EventLoop thread; determining the resource capacity change amount through a feedback control algorithm based on the pressure value; and performing expansion or contraction operations on the connection resources of the thread-local pool according to preset rules based on the capacity change amount, and recycling the excess connections generated by contraction or idleness to the global shared pool.

[0005] In conjunction with the first aspect mentioned above, in one possible implementation, the EventLoop thread based on the current business determines the available connection resources in the layered architecture connection pool according to the local priority principle. This includes: the EventLoop thread based on the current business obtains the target available connection from the thread-local pool bound to the EventLoop thread in a lock-free manner; when there is no available connection in the thread-local pool, it reads an available connection from the global shared pool and migrates the read available connection to the thread-local pool.

[0006] In conjunction with the first aspect mentioned above, in one possible implementation, the resource capacity change is determined based on the pressure value through a feedback control algorithm, including: calculating the current load deviation value based on the pressure value; performing proportional, integral, and derivative operations on the load deviation value to generate a capacity adjustment output value; and determining the resource capacity change of the thread-local pool based on the capacity adjustment output value.

[0007] In conjunction with the first aspect mentioned above, in one possible implementation, a pressure value representing the system load status is periodically calculated based on multi-dimensional indicators of the business request queue. This includes: collecting the backlog rate of the business request queue, TCP RTT latency, and task backlog of the EventLoop thread at a preset period; normalizing the backlog rate, TCP RTT latency, and task backlog respectively, and weighting them according to preset weight coefficients to obtain the pressure value representing the system load.

[0008] In conjunction with the first aspect mentioned above, in one possible implementation, based on the capacity change, the thread-local pool is expanded or shrunk according to preset rules, and excess connections generated during shrinkage or idleness are recycled to the global shared pool. This includes: determining the target capacity of the thread-local pool based on the capacity change; when the number of existing connections in the thread-local pool is lower than the target capacity, creating new connections according to a preset increment and adding them to the thread-local pool; when the number of existing connections in the thread-local pool is higher than the target capacity, removing the excess connections from the thread-local pool and recycling them to the global shared pool.

[0009] In conjunction with the first aspect above, in one possible implementation, when the number of existing connections in the thread-local pool is lower than the target capacity, new connections are created and added to the thread-local pool according to a preset increment, including: when creating a new connection, marking the connection as a gray-scale preheating state; when the connection is in the preheating state, only allocating low-priority service requests, or detecting the RTT variance of the network link in the gray-scale preheating state through an idle probe packet; when the RTT variance is detected to be less than a preset threshold, promoting the connection from the preheating state to the active state and adding it to the thread-local pool or the global shared pool.

[0010] In conjunction with the first aspect above, in one possible implementation, when the number of existing connections in the thread-local pool exceeds the target capacity, the excess connections are removed from the thread-local pool and recycled to the global shared pool. This includes: when it is determined that the thread-local pool needs to be scaled down, marking the connections to be removed as pre-destructed; connections in the pre-destructed state refuse to accept new requests and wait for existing in-transit requests to be processed within the emptying window; when the emptying window ends and there are still uncompleted requests, the connection is forcibly closed; otherwise, the connection is gracefully closed and removed from the thread-local pool or the global shared pool.

[0011] In conjunction with the first aspect mentioned above, in one possible implementation, after performing expansion or contraction operations on the thread-local pool according to preset rules based on the capacity change, and reclaiming the excess connections generated by contraction or idleness to the global shared pool, the method further includes: monitoring Netty's I / O exception events to calculate the connection read / write exception rate; actively sending heartbeat probe packets and calculating the heartbeat RTT and its jitter to determine the RTT metric and RTT jitter metric; based on the RTT metric and RTT jitter metric, when the RTT standard deviation of a connection within a preset period increases and reaches a preset jitter threshold, downgrading the connection from the thread-local pool to the global shared pool or entering an isolation observation state.

[0012] Secondly, a Netty adaptive connection system based on EventLoop affinity and PID control is provided, including: a communication unit and a processing unit; The system comprises the following components: a communication unit for acquiring the business thread request queue and determining the current business's EventLoop thread; a processing unit for determining available connection resources in the hierarchical connection pool based on the current business's EventLoop thread and prioritizing local connections; a hierarchical connection pool based on EventLoop affinity, comprising a thread-local pool and a global shared pool; a pressure value representing the system load status based on multi-dimensional metrics of the business request queue; multi-dimensional metrics including the backlog rate of the business request queue within a preset period, TCP RTT latency, and the amount of tasks accumulated by the EventLoop thread; a resource capacity change determined through a feedback control algorithm based on the pressure value; and a capacity change performed on the thread-local pool according to preset rules for expanding or shrinking connection resources, and reclaiming excess connections generated during shrinkage or idle periods to the global shared pool.

[0013] In conjunction with the second aspect above, in one possible implementation, the processing unit is further configured to: determine the target capacity of the thread-local pool based on the capacity change; when the number of existing connections in the thread-local pool is lower than the target capacity, create new connections according to a preset increment and add them to the thread-local pool; when the number of existing connections in the thread-local pool is higher than the target capacity, remove the excess connections from the thread-local pool and reclaim them to the global shared pool.

[0014] This application provides a Netty adaptive connection method and system based on EventLoop affinity and PID control. By constructing a hierarchical connection pool architecture based on EventLoop affinity and combining it with a PID adaptive control mechanism driven by multi-dimensional load indicators, the system reduces multi-threaded lock contention in high-concurrency scenarios, enabling lock-free acceleration of the connection acquisition process. Simultaneously, the system dynamically calculates pressure values ​​based on multi-source information such as business queue backlog rate, TCP RTT latency, and EventLoop task backlog, and accordingly performs precise scaling of the thread-local pool. This allows the number of connections to adaptively adjust with traffic changes, ensuring peak throughput while avoiding resource waste. Furthermore, through hierarchical distribution of connections between the local pool and the global pool, connection resources are efficiently reused, reducing the overhead of frequent connection establishment and destruction. This comprehensively improves the performance stability and resource utilization of Netty connection management, solving the technical problem of multi-threaded lock contention in existing Netty connections.

[0015] It should be understood that the descriptions of technical features, technical solutions, beneficial effects, or similar language in this application do not imply that all features and advantages can be achieved in any single embodiment. Rather, it is understood that the description of a feature or beneficial effect means that a specific technical feature, technical solution, or beneficial effect is included in at least one embodiment. Therefore, the descriptions of technical features, technical solutions, or beneficial effects in this specification do not necessarily refer to the same embodiment. Furthermore, the technical features, technical solutions, and beneficial effects described in this embodiment can be combined in any suitable manner. Those skilled in the art will understand that embodiments can be implemented without one or more specific technical features, technical solutions, or beneficial effects of a particular embodiment. In other embodiments, additional technical features and beneficial effects may be identified in specific embodiments that do not embody all embodiments. Attached Figure Description

[0016] Figure 1 A system architecture diagram of a Netty adaptive connection system based on EventLoop affinity and PID control is provided for embodiments of this application; Figure 2 A flowchart illustrating a Netty adaptive connection method based on EventLoop affinity and PID control, provided for an embodiment of this application; Figure 3 A flowchart illustrating another Netty adaptive connection method based on EventLoop affinity and PID control provided for embodiments of this application; Figure 4 A flowchart illustrating another Netty adaptive connection method based on EventLoop affinity and PID control provided for embodiments of this application; Figure 5 This is a flowchart illustrating another Netty adaptive connection method based on EventLoop affinity and PID control, provided for an embodiment of this application. Detailed Implementation

[0017] In the description of this application, unless otherwise stated, " / " means "or," for example, A / B can mean A or B. The "and / or" in this document is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, and B alone. Furthermore, "at least one" means one or more, and "multiple" means two or more. The terms "first," "second," etc., do not limit the quantity or order of execution, and "first," "second," etc., do not necessarily imply differences.

[0018] It should be noted that, in this application, the terms "exemplary" or "for example" are used to indicate that something is being described as an example, illustration, or illustration. Any embodiment or design described as "exemplary" or "for example" in this application should not be construed as being more preferred or advantageous than other embodiments or design solutions. Specifically, the use of terms such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0019] The Netty adaptive connection method based on EventLoop affinity and PID control provided in this application embodiment can be applied to, for example... Figure 1 In the Netty adaptive connection system based on EventLoop affinity and PID control shown, such as Figure 1 As shown, the system includes: a data acquisition module 101 and an electronic device 102; Among them, the data acquisition module 101 is used to acquire the business thread request queue and determine the EventLoop thread of the current business. Electronic device 102 is used to determine available connection resources in a hierarchical connection pool based on the EventLoop thread of the current business, according to the principle of local priority. The hierarchical connection pool is a hierarchical connection pool based on EventLoop affinity, including a thread-local pool and a global shared pool. Based on the multi-dimensional indicators of the business request queue, it periodically calculates the pressure value that characterizes the system load status. The multi-dimensional indicators refer to the backlog rate of the business request queue within a preset period, the TCP RTT latency, and the task backlog of the EventLoop thread. Based on the pressure value, the resource capacity change is determined through a feedback control algorithm. Based on the capacity change, the thread-local pool is expanded or shrunk according to preset rules, and the excess connections generated by shrinking or idle are recycled to the global shared pool.

[0020] In one possible implementation, the electronic device is also used to: determine the target capacity of the thread-local pool based on the capacity change; when the number of existing connections in the thread-local pool is lower than the target capacity, create new connections according to a preset increment and add them to the thread-local pool; when the number of existing connections in the thread-local pool is higher than the target capacity, remove the excess connections from the thread-local pool and reclaim them to the global shared pool.

[0021] To address the technical problem of multi-threaded lock contention in existing Netty connections, this application provides a Netty adaptive connection method based on EventLoop affinity and PID control. The method includes: acquiring the business thread request queue and determining the current business's EventLoop thread; determining available connection resources in a hierarchical connection pool based on the current business's EventLoop thread, following a local priority principle; the hierarchical connection pool is an EventLoop affinity-based hierarchical connection pool, including a thread-local pool and a global shared pool; periodically calculating pressure values ​​characterizing the system load state based on multi-dimensional indicators of the business request queue; the multi-dimensional indicators refer to the backlog rate of the business request queue within a preset period, TCPRTT latency, and the task backlog of the EventLoop thread; determining the resource capacity change based on the pressure value using a feedback control algorithm; and performing expansion or contraction operations on the thread-local pool according to preset rules based on the capacity change, and reclaiming excess connections generated by contraction or idleness to the global shared pool.

[0022] Figure 2 A flowchart illustrating the Netty adaptive connection method based on EventLoop affinity and PID control provided in this application embodiment is shown below. Figure 2 As shown, the method includes: S201. Obtain the business thread request queue and determine the EventLoop thread of the current business.

[0023] Among them, the business thread request queue refers to the data structure used in Netty applications to cache pending business requests. The EventLoop thread is the core thread in the Netty Reactor model responsible for I / O event processing and task scheduling. Each business request is bound to the corresponding EventLoop thread for execution after submission.

[0024] In this embodiment, the execution context of the current business request is clarified by recording the EventLoop thread to which the business request belongs when it enters the Netty processing chain and synchronously obtaining the business thread request queue currently corresponding to the EventLoop thread.

[0025] It should be noted that the determination of the EventLoop thread does not depend on additional thread mapping relationships, but is directly based on the thread binding mechanism inside the Netty framework, in order to avoid introducing additional scheduling overhead.

[0026] As an example, in an RPC framework scenario, when a business request is initiated by the client and arrives at the Netty server, the request will be dispatched to a fixed EventLoop thread, and the request queue corresponding to that thread will serve as the current business thread's request queue.

[0027] This step clarifies the binding relationship between the business request and the EventLoop thread in the initial stage of request processing, providing accurate contextual information for subsequent selection of connection resources based on thread affinity.

[0028] S202. Based on the EventLoop thread of the current business, determine the available connection resources in the layered architecture connection pool according to the principle of local priority.

[0029] The layered architecture connection pool is an EventLoop affinity-based layered architecture connection pool, which includes a thread-local pool corresponding one-to-one with the EventLoop thread and a global shared pool for connection reuse; the local priority principle of this application means that connection resources are preferentially obtained from the thread-local pool bound to the current EventLoop thread.

[0030] In this embodiment of the application, after determining the EventLoop thread corresponding to the current service, the system first attempts to obtain an available connection from the thread local pool corresponding to the EventLoop thread in a lock-free manner; only when there is no available connection in the thread local pool will it access the global shared pool to obtain connection resources.

[0031] Based on the above steps, this step adopts a fixed binding relationship between the thread-local pool and the EventLoop thread, thereby avoiding multiple EventLoop threads competing for the same connection pool lock at the same time, improving the concurrency efficiency of the connection acquisition process, effectively reducing thread competition and synchronization overhead in the connection acquisition process, and improving Netty's connection access efficiency and overall throughput in high-concurrency scenarios.

[0032] S203. Based on the multi-dimensional indicators of the business request queue, periodically calculate the pressure value that characterizes the system load status.

[0033] Among them, multidimensional indicators refer to load characteristics that reflect the system's operating status from different perspectives, including the backlog rate of the business request queue within a preset period, TCP RTT latency, and the task backlog of the EventLoop thread. The pressure value is used to comprehensively characterize the current system load status.

[0034] In this embodiment, the system collects the backlog rate of the service request queue, the TCP RTT latency, and the task backlog of the EventLoop thread at a preset period; the backlog rate, TCP RTT latency, and task backlog are normalized respectively, and weighted according to preset weight coefficients to obtain the pressure value representing the system load.

[0035] It should be noted that TCP RTT latency is used to reflect the real-time status of the network transport layer, while the task backlog of the EventLoop thread is used to characterize the processing pressure of the I / O thread. Together with the business request queue backlog rate, they constitute a comprehensive description of the system load.

[0036] As an example, in scenarios of network jitter or a sudden surge in requests, the backlog rate of the business request queue and the TCP RTT latency will rise synchronously, and the calculated pressure value can reflect the trend of system load changes in a timely manner.

[0037] Based on the above steps, this step can avoid the load judgment distortion caused by relying on a single indicator, and enable the system to have higher perception accuracy for complex operating states.

[0038] S204. Based on the pressure value, the change in resource capacity is determined through a feedback control algorithm.

[0039] Among them, the feedback control algorithm refers to the control method that dynamically adjusts resource configuration based on the deviation between the current pressure value and the target load state. The resource capacity change is used to indicate the adjustment range of the number of connections in the thread local pool.

[0040] In this embodiment of the application, the system calculates the load deviation based on the current pressure value and generates a capacity adjustment output through proportional, integral and derivative operations, thereby determining the amount of capacity change of the connected resources.

[0041] It should be noted that the feedback control algorithm can comprehensively consider the current load status and historical trends, avoiding frequent oscillations or over-adjustment of connection resource scale.

[0042] Based on the above steps, the smoothness and stability of the connection resource adjustment process can be achieved, making the changes in connection scale more consistent with the actual load characteristics of the system.

[0043] S205. Based on the capacity change, perform expansion or contraction operations on the connection resources of the thread-local pool according to preset rules, and reclaim the excess connections generated by contraction or idleness to the global shared pool.

[0044] Among them, expansion or contraction operations refer to creating or reclaiming connection resources in the thread-local pool according to the target capacity change, while the global shared pool is used to uniformly manage reusable connection resources.

[0045] In this embodiment, when the capacity change indicates that expansion is needed, the system creates new connections and adds them to the thread-local pool according to preset rules; when the capacity change indicates that reduction is needed, connections exceeding the target capacity are removed from the thread-local pool and recycled to the global shared pool.

[0046] Based on the above steps, it is possible to improve connection reuse rate and reduce overall system resource consumption while ensuring that connection resources dynamically adapt to load changes.

[0047] This application's embodiments combine EventLoop thread affinity with a multi-dimensional load-aware feedback control mechanism to achieve precise allocation and adaptive adjustment of Netty connection resources. While effectively reducing multi-threaded lock contention, it improves the system's performance stability and resource utilization efficiency in high-concurrency and traffic fluctuation scenarios, solving the technical problem of multi-threaded lock contention in existing Netty connections.

[0048] In one possible implementation of the embodiments of this application, combined with Figure 2 ,like Figure 3 As shown, the above S202 can be specifically implemented through the following S301 and S302, which are explained in detail below: S301. Based on the current business EventLoop thread, obtain the target available connection from the thread-local pool bound to the EventLoop thread in a lock-free manner.

[0049] The thread-local pool refers to a connection resource pool that is bound one-to-one with the EventLoop thread. It is used to store connection objects that can only be accessed by the corresponding EventLoop thread. The target available connection refers to the connection resource that is active and can be directly used for the current business request.

[0050] In this embodiment of the application, when a business request is assigned to a specific EventLoop thread, the system directly accesses the thread-local pool bound to that EventLoop thread and obtains the target available connection from it through a lock-free data structure to complete the network communication of the business request.

[0051] It should be noted that since the thread-local pool is only accessed by the corresponding EventLoop thread, there is no need to introduce synchronization locks or atomic operations during the connection acquisition process, thus avoiding lock contention issues caused by concurrent access by multiple threads.

[0052] As an example, in Netty's Reactor model, each EventLoop thread only accesses its own thread-local pool when processing I / O events, thereby achieving low-latency response during the connection acquisition process in high-concurrency RPC call scenarios.

[0053] Based on the above steps, this process can achieve rapid acquisition of connection resources in most business request scenarios, reduce synchronization overhead during connection access, and improve the processing efficiency of the EventLoop thread.

[0054] S302. When there are no available connections in the thread-local pool, read available connections from the global shared pool and migrate the read available connections to the thread-local pool.

[0055] The global shared pool refers to the connection pool structure used to uniformly manage reusable connection resources, and migration refers to rebinding the connection obtained from the global shared pool to the thread-local pool corresponding to the current EventLoop thread.

[0056] In this embodiment, when no available connection is detected in the thread-local pool, the system reads an available connection from the global shared pool and updates the ownership of the connection to the current EventLoop thread, thereby adding it to the corresponding thread-local pool for reuse in subsequent requests.

[0057] It should be noted that the global shared pool is only accessed when local pool resources are insufficient, in order to reduce its access frequency and thus reduce the risk of concurrency competition caused by multiple EventLoop threads accessing the global shared pool at the same time.

[0058] As an example, in the event that a sudden surge in traffic causes some EventLoop thread-local pool connections to run out, the system can quickly replenish connection resources through the global shared pool and stably incorporate the replenished connections into the thread-local pool management.

[0059] Based on the above steps, this process can ensure full utilization of connection resources while maintaining the stability and independence of the thread-local pool and avoiding frequent cross-thread access to global connection resources.

[0060] This application embodiment achieves efficient allocation and reuse of connection resources among EventLoop threads by prioritizing the acquisition of connections from the thread-local pool and introducing a global shared pool as a supplement when necessary. This reduces lock contention while improving the overall performance and scalability of Netty connection management in high-concurrency scenarios.

[0061] In one possible implementation of the embodiments of this application, combined with Figure 2 ,like Figure 4 As shown, the above S204 can be specifically implemented through the following S401 to S403, which are explained in detail below: S401. Calculate the current load deviation value based on the pressure value.

[0062] The load deviation value refers to the difference between the currently calculated system pressure value and the preset target pressure value, which is used to characterize the degree of deviation of the current operating state of the system from the expected load state.

[0063] In this embodiment of the application, the system compares the pressure value calculated in real time with the preset target pressure threshold in each control cycle, and calculates the current load deviation value accordingly, which is used as the input parameter for subsequent feedback control calculation.

[0064] It should be noted that the target pressure value can be configured based on the system's service level agreement or historical operating data to ensure that the load regulation process has a clear control objective and a stable reference benchmark.

[0065] As an example, in a high-concurrency RPC call scenario, when the backlog rate of the business request queue and the backlog of EventLoop thread tasks increase, the calculated pressure value will be greater than the target pressure value, thus resulting in a positive load deviation value.

[0066] Based on the above steps, this step can transform the complex system load state into quantifiable deviation parameters, providing a clear calculation basis for subsequent capacity regulation based on feedback control.

[0067] S402. Perform proportional, integral, and derivative calculations on the load deviation value to generate a capacity regulation output value.

[0068] The proportional term reflects the immediate impact of the current load deviation, the integral term accumulates historical deviation changes, and the derivative term reflects the load change trend, thus forming the core of the feedback control algorithm.

[0069] In this embodiment, the system inputs the load deviation value into the proportional, integral, and differential calculation modules respectively, and performs proportional weighting on the current value, historical cumulative value, and rate of change of the deviation according to their importance to generate a capacity adjustment output value for guiding the adjustment of connection resources.

[0070] It should be noted that by simultaneously introducing proportional, integral, and differential terms, the frequent oscillations or response lag caused by adjusting based on a single threshold can be effectively avoided, thereby improving the stability and sensitivity of the capacity adjustment process.

[0071] As an example, the proportion term Satisfy the following formula:

[0072] in, For the target load, This represents the current load.

[0073] As an example, the integral term Satisfy the following formula:

[0074] in, This represents the deviation of the proportional term.

[0075] As an example, differential terms Satisfy the following formula:

[0076] in, For the deviation of the proportional term, This indicates the load deviation value recorded in the previous control cycle.

[0077] It should also be noted that the derivative term is used to characterize the trend and rate of change of load deviation. When a rapid increase in load deviation is detected within adjacent control cycles, i.e., when the derivative term value is large, it indicates that the system load is in a rapid increase phase. At this time, even if the current load has not yet exceeded the preset target threshold, the feedback control algorithm can still trigger the expansion operation of connection resources in advance based on the output result of the derivative term, thereby achieving predictive expansion in response to load changes. In this way, the system can preemptively increase connection resources before a surge in service requests, reduce response latency during load surges, and avoid connection acquisition failures or performance jitter caused by expansion lag.

[0078] Based on the above steps, a comprehensive adjustment result can be generated according to the current state, historical trend and rate of change of the system load, making the connection resource adjustment process more accurate and controllable.

[0079] S403. Determine the change in resource capacity of the thread-local pool based on the capacity adjustment output value.

[0080] Among them, the change in resource capacity refers to the increase or decrease in the number of connections that the thread-local pool needs to execute within the current control cycle, which is used to guide the expansion or contraction of connection resources.

[0081] In this embodiment, the system determines the number of connections that the thread-local pool needs to increase or decrease based on the size and sign of the capacity adjustment output value, and uses the result as a direct control parameter for subsequent connection resource management strategies.

[0082] It should be noted that, based on the original parameter adjustments, the maximum adjustment range and minimum adjustment granularity of the capacity change can be set to avoid drastic changes in the number of thread-local pool connections due to instantaneous load fluctuations.

[0083] Based on the above steps, this step can transform the abstract feedback control output into specific executable connection resource adjustment instructions, thereby realizing the feasibility and controllability of the connection pool capacity adjustment process.

[0084] This application embodiment transforms the system load state into a load deviation value and introduces a feedback control mechanism combining proportional, integral, and derivative methods to achieve smooth adjustment of the thread-local connection pool capacity, effectively avoiding frequent fluctuations in connection resources and improving the stability and responsiveness of Netty connection management under complex load scenarios.

[0085] In one possible implementation of the embodiments of this application, combined with Figure 2 ,like Figure 5 As shown, the above S205 can be implemented through the following S501 to S503, which are explained in detail below: S501. Determine the target capacity of the thread-local pool based on the capacity change.

[0086] The target capacity refers to the upper limit of the number of connections that the thread-local pool should maintain within the current control cycle, which is used to characterize the reasonable configuration scale of connection resources under the current load state.

[0087] In this embodiment, the system combines the current capacity of the thread-local pool with the capacity change output by the feedback control algorithm to calculate the target capacity of the thread-local pool in the next adjustment cycle.

[0088] It should be noted that the target capacity can be constrained by the minimum and maximum capacity limits of the thread-local pool to avoid a decrease in throughput due to too small a number of connections or a waste of resources due to too large a number of connections.

[0089] As an example, when the capacity change is positive, the system increases the corresponding number of threads in the original thread-local pool to obtain the new target capacity; when the capacity change is negative, the target capacity is reduced accordingly.

[0090] Based on the above steps, the abstract capacity adjustment result can be transformed into a clear capacity control target, making the connection resource adjustment process of the thread-local pool executable and consistent.

[0091] S502. When the number of existing connections in the thread-local pool is lower than the target capacity, create new connections according to the preset increment and add them to the thread-local pool.

[0092] The preset increment refers to the upper limit of the number of new connections allowed within a single adjustment cycle, which is used to control the pace and magnitude of the capacity expansion process.

[0093] In this embodiment, when the number of existing connections in the thread-local pool is less than the target capacity, the system gradually creates new connections according to a preset incremental rule. When a new connection is created, the connection is marked as a gray-scale preheating state. When the connection is in the preheating state, only low-priority service requests are allocated, or the RTT variance of the network link in the gray-scale preheating state is detected by an idle probe packet. When the RTT variance is detected to be less than a preset threshold, the connection is promoted from the preheating state to the active state and added to the thread-local pool or the global shared pool.

[0094] Based on the above steps, this step introduces a pre-defined incremental control and connection gray-scale warm-up mechanism during the capacity expansion process. This avoids the instantaneous resource jitter and link instability caused by creating and deploying a large number of new connections at once when the load increases. Simultaneously, before newly created connections officially carry core business requests, their network link quality is verified through low-priority requests or idle probing. This ensures that unstable connections are identified and isolated before entering an active state, thereby reducing the impact of abnormal connections on overall service quality. Furthermore, this approach ensures that the increase in connection resources is consistent with the actual load growth rhythm, improving the reliability of connection availability and the smoothness of system operation in high-concurrency scenarios.

[0095] S503. When the number of existing connections in the thread-local pool exceeds the target capacity, the excess connections are removed from the thread-local pool and recycled to the global shared pool.

[0096] The excess connections refer to the number of reclaimable connection resources in the thread-local pool that exceeds the target capacity.

[0097] In this embodiment, when the number of existing connections in the thread-local pool exceeds the target capacity, the system selects the excess connections from the thread-local pool according to a preset recycling strategy. When it is determined that the thread-local pool needs to be scaled down, the connections to be removed are marked as pre-destructed. Connections in the pre-destructed state refuse to accept new requests and wait for existing in-transit requests to be processed within the emptying window. When the emptying window ends and there are still uncompleted requests, the connection is forcibly closed; otherwise, the connection is gracefully closed and removed from the thread-local pool or the global shared pool.

[0098] It should be noted that the reclaimed connections are preferentially selected from those with longer idle times or lower loads, in order to reduce the impact on connections that are currently processing business requests.

[0099] Based on the above steps, by introducing a pre-destruction state and a drain window mechanism during the scaling-down process, the request interruption, TCP handshake storm, and service jitter issues caused by directly closing connections are avoided. Connections are restricted from receiving new requests before entering the recycling process and wait for in-transit requests to complete naturally, thus ensuring the continuity and correctness of business processing. Simultaneously, by prioritizing the recycling of connections with longer idle times or lower loads, active connections are protected, reducing the impact of scaling-down operations on system throughput and response latency. This approach makes the connection resource recycling process more gentle and controllable, effectively improving system stability and connection resource reuse efficiency in load reduction scenarios.

[0100] This application's embodiments map capacity changes to the target capacity of the thread-local pool and combine them with controlled expansion and recycling strategies to achieve smooth adjustment of connection resources under different load conditions. While ensuring system throughput, it improves the overall utilization efficiency of connection resources and solves the problems of lag and oscillation in the expansion and contraction algorithms and coarse connection lifecycle management in existing Netty connection pools.

[0101] The foregoing primarily describes the solutions of the embodiments of this application from the perspective of device implementation. It is understood that each device, such as the Netty adaptive connection system based on EventLoop affinity and PID control, includes at least one of the hardware structures and software modules corresponding to the execution of each function in order to achieve the aforementioned functions. Those skilled in the art should readily recognize that, based on the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed in a hardware or computer software-driven manner depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0102] This application embodiment can divide the Netty adaptive connection system based on EventLoop affinity and PID control into functional units according to the above method example. For example, each function can be divided into a separate functional unit, or two or more functions can be integrated into one processing unit. The integrated unit can be implemented in hardware or as a software functional unit. It should be noted that the unit division in this application embodiment is illustrative and only represents one logical functional division; other division methods may be used in actual implementation.

[0103] In implementation, each step of the method provided in this embodiment can be completed by integrated logic circuits in the processor or by instructions in software form. The steps of the method disclosed in the embodiments of this application can be directly manifested as being executed by a hardware processor, or being executed by a combination of hardware and software modules in the processor.

[0104] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented using software programs, implementation can be, in whole or in part, in the form of a computer program product. This computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the flow or function according to the embodiments of this application is generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, computer instructions can be transmitted from one website, computer, server, or data center to another via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer or a data storage device containing one or more servers, data centers, etc., that can be integrated with the medium. The available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state disks (SSDs)).

[0105] Although this application has been described herein in conjunction with various embodiments, those skilled in the art, by reviewing the accompanying drawings, disclosure, and appended claims, will understand and implement other variations of the disclosed embodiments in carrying out the claimed application. In the claims, the word "comprising" does not exclude other components or steps, and "a" or "an" does not exclude multiple instances. A single processor or other unit can implement several functions listed in the claims. While different dependent claims may recite certain measures, this does not mean that these measures cannot be combined to produce good results.

[0106] Although this application has been described in conjunction with specific features and embodiments, it is obvious that various modifications and combinations can be made thereto without departing from the spirit and scope of this application. Accordingly, this specification and drawings are merely exemplary illustrations of this application as defined by the appended claims, and are considered to cover any and all modifications, variations, combinations, or equivalents within the scope of this application. Clearly, those skilled in the art can make various alterations and modifications to this application without departing from the spirit and scope of this application. Thus, if such modifications and modifications of this application fall within the scope of the claims of this application and their equivalents, this application is also intended to include such modifications and modifications.

Claims

1. A Netty adaptive connection method based on EventLoop affinity and PID control, characterized in that, include: Obtain the request queue of the business thread and determine the EventLoop thread of the current business; Based on the EventLoop thread of the current business, available connection resources are determined in the hierarchical architecture connection pool according to the local priority principle; The layered connection pool is an EventLoop affinity-based layered connection pool, including a thread-local pool and a globally shared pool; Based on multi-dimensional metrics of the business request queue, pressure values ​​that characterize the system load status are periodically calculated; the multi-dimensional metrics refer to the backlog rate of the business request queue, TCP RTT latency, and task backlog of the EventLoop thread within a preset period. Based on the pressure value, the change in resource capacity is determined through a feedback control algorithm; Based on the capacity change, the thread-local pool is expanded or reduced in size according to preset rules, and any excess connections generated during shrinkage or idleness are recycled to the global shared pool.

2. The Netty adaptive connection method for EventLoop affinity and PID control according to claim 1, characterized in that, The EventLoop thread based on the current service determines available connection resources in the hierarchical connection pool according to the local priority principle, including: Based on the current business EventLoop thread, the target available connection is obtained from the thread-local pool bound to the EventLoop thread in a lock-free manner; When there are no available connections in the thread-local pool, an available connection is read from the global shared pool and then migrated to the thread-local pool.

3. The Netty adaptive connection method for EventLoop affinity and PID control according to claim 1, characterized in that, The determination of resource capacity change based on the pressure value using a feedback control algorithm includes: Calculate the current load deviation value based on the pressure value; Perform proportional, integral, and derivative calculations on the load deviation value to generate a capacity regulation output value; The change in resource capacity of the thread-local pool is determined based on the capacity adjustment output value.

4. The Netty adaptive connection method for EventLoop affinity and PID control according to claim 1, characterized in that, The multi-dimensional metrics based on the business request queue periodically calculate pressure values ​​that characterize the system load status, including: Collect the backlog rate of the business request queue, TCP RTT latency, and task backlog of the EventLoop thread at preset intervals; The backlog rate, the TCP RTT latency, and the task backlog are normalized and then weighted according to preset weighting coefficients to obtain a pressure value that characterizes the system load.

5. The Netty adaptive connection method for EventLoop affinity and PID control according to claim 1, characterized in that, The step of scaling up or down the thread-local pool according to preset rules based on the capacity change, and reclaiming excess connections generated during scaling down or idle periods to the global shared pool, includes: The target capacity of the thread-local pool is determined based on the aforementioned capacity change. When the number of existing connections in the thread-local pool is lower than the target capacity, new connections are created and added to the thread-local pool according to a preset increment. When the number of existing connections in the thread-local pool exceeds the target capacity, the excess connections are removed from the thread-local pool and recycled to the global shared pool.

6. The Netty adaptive connection method based on EventLoop affinity and PID control according to claim 5, characterized in that, The step of creating new connections and adding them to the thread-local pool according to a preset increment when the number of existing connections in the thread-local pool is lower than the target capacity includes: When a new connection is created, the connection is marked as a grayscale preheating state; When a link is in a preheating state, only low-priority service requests are allocated, or the RTT variance of the link network in the gray-scale preheating state is detected by using idle probe packets. When the RTT variance is detected to be less than a preset threshold, the connection is promoted from the warm-up state to the active state and added to the thread-local pool or the global shared pool.

7. The Netty adaptive connection method based on EventLoop affinity and PID control according to claim 5, characterized in that, The step of removing and reclaiming excess connections from the thread-local pool to the global shared pool when the number of existing connections in the thread-local pool exceeds the target capacity includes: When it is determined that the thread-local pool needs to be scaled down, the connections that need to be removed are marked as pre-destructed. Connections in a pre-destruction state refuse to accept new requests and wait for existing in-transit requests to be processed during the emptying window. When the emptying window ends and there are still unfulfilled requests, the connection is forcibly closed; otherwise, the connection is gracefully closed and removed from the thread-local pool or the global shared pool.

8. The Netty adaptive connection method based on EventLoop affinity and PID control according to claim 1, characterized in that, After performing expansion or contraction operations on the thread-local pool according to preset rules based on the capacity change, and reclaiming excess connections generated during contraction or idle periods to the global shared pool, the method further includes: By monitoring Netty's I / O exception events, the read / write exception rate of the connection can be statistically analyzed. By actively sending heartbeat detection packets and calculating the heartbeat RTT and its jitter, the RTT index and RTT jitter index are determined. Based on the RTT metric and the RTT jitter metric, when the RTT standard deviation of a connection within a preset period increases and reaches a preset jitter threshold, the connection is downgraded from the thread-local pool to the global shared pool or enters an isolation observation state.

9. A Netty adaptive connection system based on EventLoop affinity and PID control, characterized in that, The system includes: a data acquisition module and electronic equipment; The data acquisition module is used to acquire the business thread request queue and determine the EventLoop thread of the current business. The electronic device is used to determine available connection resources in a hierarchical connection pool based on the EventLoop thread of the current service, according to the principle of local priority. The hierarchical connection pool is a hierarchical connection pool based on EventLoop affinity, including a thread-local pool and a global shared pool. Based on the multi-dimensional indicators of the service request queue, a pressure value representing the system load status is calculated periodically. The multi-dimensional indicators refer to the backlog rate of the service request queue, TCPRTT latency, and task backlog of the EventLoop thread within a preset period. Based on the pressure value, a feedback control algorithm is used to determine the resource capacity change. Based on the capacity change, the thread-local pool is expanded or shrunk according to preset rules, and the excess connections generated by shrinking or idle connections are recycled to the global shared pool.

10. The Netty adaptive connection system based on EventLoop affinity and PID control according to claim 9, characterized in that, The electronic device is also used for: The target capacity of the thread-local pool is determined based on the aforementioned capacity change. When the number of existing connections in the thread-local pool is lower than the target capacity, new connections are created and added to the thread-local pool according to a preset increment. When the number of existing connections in the thread-local pool exceeds the target capacity, the excess connections are removed from the thread-local pool and recycled to the global shared pool.

Images