Load balancing methods, devices, equipment, and media based on adaptive concurrency control
By using an adaptive concurrency control method to dynamically adjust the channel concurrency threshold and request allocation, the problems of excessive load on the main channel and high latency on the backup channel are solved, thereby improving the system's resource utilization and stability.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- CSC FINANCIAL CO LTD
- Filing Date
- 2026-03-09
- Publication Date
- 2026-06-02
AI Technical Summary
Existing load balancing strategies in multi-channel business scenarios suffer from problems such as excessive load or crashes on the main channel and high latency on the backup channel, resulting in high overall response latency.
An adaptive concurrency control method is adopted, which calculates the average latency of channel requests in real time, adaptively adjusts the concurrency threshold, and uses an adaptive scheduling mechanism to dynamically allocate request traffic, thereby achieving intelligent channel allocation and precise load control.
It improves the resource utilization and availability of distributed systems, reduces operation and maintenance costs, and ensures system stability and response speed.
Smart Images
Figure CN122137841A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of distributed system technology, and in particular to load balancing methods, apparatus, devices and media based on adaptive concurrency control. Background Technology
[0002] In a distributed system architecture, load balancing technology is a core component responsible for rationally distributing massive requests initiated by clients to multiple backend service instances in order to maximize resource utilization, improve system response speed, and ensure service availability.
[0003] Currently, the mainstream load balancing strategies in the industry mainly include round-robin, random, weighted round-robin, and least connections. These strategies can achieve basic request distribution in normal scenarios, but they treat all backend service instances or channels as nodes with equal priority, resulting in poor scheduling flexibility and an inability to perform differentiated scheduling based on the actual situation of the service channels.
[0004] In multi-channel business scenarios, the primary channel uses high-cost hardware resources and features low latency, but its performance is limited. Backup channels use low-cost hardware resources, resulting in higher latency but lower cost. If all requests are distributed to the primary channel using traditional strategies, the primary channel will become overloaded or even crash when the concurrency exceeds its limits. However, if strategies such as round-robin, random, or least-connection allocation are used to distribute requests to both the primary and backup channels, the overall response latency will be higher when the load does not exceed the primary channel's concurrency limit, due to the higher latency of the backup channels. Summary of the Invention
[0005] In view of this, this application provides a load balancing method, apparatus, device and medium based on adaptive concurrency control. The main purpose is to solve the technical problems of existing load balancing strategies and technologies, such as excessive load or even collapse of the main channel, or high latency of the backup channel causing high overall response latency.
[0006] According to one aspect of this application, a load balancing method based on adaptive concurrency control is provided, the method comprising: Based on the request information within the concurrency threshold adjustment period obtained in real time, calculate the average latency of channel requests and determine the number of concurrent requests being processed by the channel. The current concurrency threshold of the current channel is adaptively adjusted based on the average latency of the channel requests. The adaptive scheduling mechanism is used to traverse the channels in the channel set, determine the target channel, and send the current request information to the target channel; The adaptive scheduling mechanism is set based on the number of concurrent requests being processed and the current concurrency threshold.
[0007] According to another aspect of this application, a load balancing device based on adaptive concurrency control is provided, the device comprising: The processing module is used to adjust the request information within the period based on the real-time acquired concurrency threshold, calculate the average latency of channel requests, and determine the number of concurrent requests being processed by the channel. The adjustment module is used to adaptively adjust the current concurrency threshold of the current channel based on the average latency of the channel requests; The sending module is used to traverse the channels in the channel set using an adaptive scheduling mechanism, determine the target channel, and send the current request information to the target channel; The adaptive scheduling mechanism is set based on the number of concurrent requests being processed and the current concurrency threshold.
[0008] According to another aspect of this application, a computer storage medium is provided, on which a computer program is stored, which, when executed by a processor, implements the above-described load balancing method based on adaptive concurrency control.
[0009] According to another aspect of this application, a computer device is provided, including a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, wherein the processor executes the program to implement the above-described load balancing method based on adaptive concurrency control.
[0010] By employing the above technical solutions, the load balancing method, apparatus, device, and medium based on adaptive concurrency control provided in this application, compared with existing load balancing strategies such as round-robin, random, weighted round-robin, and least-connection load balancing, can, for multi-service channel business scenarios, adjust request information within a period based on real-time acquired concurrency thresholds, calculate the average latency of channel requests and determine the concurrent number of request information being processed by a channel, adaptively adjust the current concurrency threshold of the current channel based on the average latency of channel requests, traverse the channels in the channel set using an adaptive scheduling mechanism, determine the target channel, and send the current request information to the target channel; wherein, the adaptive scheduling mechanism is set based on the concurrent number of request information being processed by the channel and the current concurrency threshold. It is evident that by introducing an adaptive scheduling mechanism, intelligent allocation of request traffic, adaptive concurrency control, and precise management of channel load are achieved, as well as further adaptive channel capacity scaling, improving the resource utilization, availability, and stability of the distributed system.
[0011] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this application more obvious and understandable, the following are specific embodiments of this application. Attached Figure Description
[0012] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings: Figure 1 A flowchart illustrating a load balancing method based on adaptive concurrency control provided in an embodiment of this application is shown. Figure 2 A flowchart illustrating another load balancing method based on adaptive concurrency control provided in an embodiment of this application is shown. Figure 3 This illustration shows a structural schematic diagram of a load balancing device based on adaptive concurrency control provided in an embodiment of this application; Figure 4 This illustration shows a schematic diagram of another load balancing device based on adaptive concurrency control provided in an embodiment of this application. Detailed Implementation
[0013] The present application will be described in detail below with reference to the accompanying drawings and embodiments. It should be noted that, unless otherwise specified, the embodiments and features described in the embodiments of the present application can be combined with each other.
[0014] To address the technical challenges in existing multi-channel business scenarios, such as excessive load or even crashes on the primary channel, or high latency on backup channels leading to overall high response times, this embodiment provides a load balancing method based on adaptive concurrency control. By designing an adaptive scheduling mechanism, it differentiates itself from traditional round-robin / fixed-weight scheduling, prioritizing traffic allocation to the optimal channel to improve user experience. Furthermore, unlike fixed-threshold control, it adjusts concurrency thresholds under adaptive scheduling, automatically adapting to channel scaling, improving resource utilization, and reducing operational costs. Figure 1 As shown, this method is illustrated using an example of its application to computer devices such as servers. The server can be a standalone server or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms. The method includes the following steps: Step 101: Adjust the request information within the period based on the real-time obtained concurrency threshold, calculate the average latency of channel requests, and determine the number of concurrent requests being processed by the channel.
[0015] In this embodiment, a counter is configured for each service channel (which could also be a server node, processing unit, etc.) to accurately record the number of requests currently being processed but not yet completed. Combined with request latency, this allows for real-time monitoring of the instantaneous load and processing efficiency of each channel, providing a quantitative basis for subsequent request information distribution decisions. Before receiving request information within the real-time concurrency threshold adjustment period, backend service channel attributes are configured, specifically including service channel identifier, initial concurrency threshold, concurrency threshold adjustment period, health check period, service channel priority, and adaptive adjustment step size for the concurrency threshold.
[0016] Step 102: Adaptively adjust the current concurrency threshold of the current channel based on the average latency of the channel requests.
[0017] In this embodiment, based on the average latency of channel requests, an adaptive adjustment mechanism for the concurrency threshold is used to adaptively adjust the current concurrency threshold of the current channel. This adaptive adjustment mechanism is set within the concurrency threshold adjustment period based on the total average latency of all request information in the channel set and the average latency of the current channel. Therefore, by periodically comparing the average latency of each service channel's requests with the global average latency, the concurrency processing limit of each service channel is adaptively adjusted to achieve a closed-loop control strategy of dynamic load balancing and overload protection, ensuring optimal overall performance.
[0018] Depending on the needs of the actual application scenario, in addition to considering the global average latency, indicators such as node CPU utilization, memory usage, and error rate can also be considered or combined to construct a comprehensive score, which can serve as a reference for adjusting the concurrency threshold. Alternatively, by predicting the future latency change trend of channel requests based on time series, a forward-looking concurrency threshold adjustment can be achieved to cope with traffic peaks in advance. Here, no specific restrictions are placed on the specific strategies involved in the adaptive adjustment mechanism of the concurrency threshold.
[0019] Step 103: Use the adaptive scheduling mechanism to traverse the channels in the channel set, determine the target channel, and send the current request information to the target channel.
[0020] In this embodiment, the adaptive scheduling mechanism is set based on a channel concurrency threshold mechanism, which is based on the number of concurrent requests being processed by the channel and the current concurrency threshold. The adaptive scheduling mechanism also includes a channel priority mechanism. By pre-marking the channels in the channel set with priorities and traversing the channel set based on these priorities, a hierarchical adaptive scheduling mechanism is achieved. The first level implements a static load balancing mechanism based on the priority markings of the channels in the channel set. Depending on the actual application scenario, priorities can be marked based on attributes such as service channel performance latency, interface cost, reliability, and hardware configuration. The second level implements a dynamic load balancing mechanism based on the number of concurrent requests being processed by the channel and the current concurrency threshold. This ensures that high-priority channels receive traffic first while preventing high-priority channels from crashing due to overload during channel traversal.
[0021] In this embodiment, following the above scheme, the average latency of channel requests is calculated based on the request information within the concurrency threshold adjustment period obtained in real time, and the number of concurrent requests being processed by the channel is determined. The current concurrency threshold of the current channel is adaptively adjusted based on the average latency of channel requests. An adaptive scheduling mechanism is used to traverse the channels in the channel set, determine the target channel, and send the current request information to the target channel. Compared with existing load balancing strategies such as round-robin, random, weighted round-robin, and least-connections, this embodiment can dynamically adjust the concurrency threshold based on global traffic fluctuations by introducing an adaptive scheduling mechanism. When channel pressure is high, the threshold is adaptively lowered to reserve a buffer; when channel pressure is low, the threshold is adaptively raised to fully utilize resources, thereby enabling the scheduling strategy to cope with scenarios of sudden traffic surges and elastic resource scaling.
[0022] Furthermore, as a refinement and extension of the specific implementation methods of the above embodiments, to fully illustrate the specific implementation process of this embodiment, another load balancing method based on adaptive concurrency control is provided. This embodiment can be applied to traffic requests, such as in OCR multi-channel scheduling scenarios. By prioritizing core channels (static load balancing mechanism) + overload intelligent traffic splitting mechanism (dynamic load balancing mechanism), it ensures optimal user experience while preventing core channel crashes, significantly improving service quality in key areas such as account opening and business processing. Simultaneously, it can also achieve adaptive scaling, health checks, and smooth fault recovery capabilities, greatly reducing manual maintenance costs and enhancing service availability and stability. This embodiment has strong reusability and can be applied to various multi-channel scheduling scenarios. Figure 2 As shown, the method includes: Step 201: Adjust the request information within the period based on the real-time obtained concurrency threshold, calculate the average latency of channel requests, and determine the number of concurrent requests being processed by the channel.
[0023] To illustrate the specific implementation of step 201, as a preferred embodiment, the concurrency threshold adjustment period includes multiple health detection periods. Within the current concurrency threshold adjustment period, the step of detecting the channel health status includes: when the current channel health status was faulty in the previous concurrency threshold adjustment period, if the heartbeat detection for the first consecutive health detection period responds normally within the current concurrency threshold adjustment period, then the current channel health status is determined to be normal; when the current channel health status was normal in the previous concurrency threshold adjustment period, if the heartbeat detection for more than the second health detection period fails to respond cumulatively within the current concurrency threshold adjustment period, then the current channel health status is determined to be faulty.
[0024] During implementation, attribute information is configured for the backend service channels. Assume there are N service channels in total: For the k-th channel, ; Within the m-th concurrency threshold adjustment period The concurrency threshold, where, express The initial concurrency threshold; Within the m-th concurrency threshold adjustment period The health status is represented by 1 for healthy and 0 for faulty. For health check-up cycles; The step size is adaptively adjusted to accommodate the concurrency threshold. The concurrency threshold adjustment period is... , The number of health check cycles included in the concurrency threshold adjustment period. .
[0025] Configure parameter information for the backend service channel algorithm, and mark the channel priority identifier, in the following order: The initial concurrency threshold for each channel is ; , Health check-up cycle For example, 1000ms; the concurrent control parameters adaptively adjust the step size. ,like This indicates a 10% adjustment range, representing the health check cycle parameter. ,like =10 indicates that the concurrent threshold adjustment cycle includes 10 health check cycles.
[0026] In implementation, based on the service channel attribute information configured above, the specific steps for channel health detection include: If If, within the m-th concurrency threshold adjustment period, continuously If all heartbeat checks are normal, then ;like If, within the m-th concurrency threshold adjustment period, the number of concurrent connections exceeds... If both heartbeat checks fail to respond normally, then .
[0027] To illustrate the specific implementation of step 201, as a preferred embodiment, the step of determining the number of concurrent request messages being processed by the channel based on the request information within the adjustment period obtained in real time includes: obtaining the number of request messages in the current channel using a channel counter; when a request message distributed to the current channel is received, incrementing the number of request messages in the channel counter by one; continuing to monitor the response status of the request message, and if the monitored response status is normal or timed out, decrementing the number of request messages in the channel counter by one, and using the adjusted number of request messages as the number of concurrent request messages being processed by the current channel.
[0028] In implementation, a counter is configured for each channel, and the counter is set... For channels The actual concurrent number is initially set to 0; if a request is detected being distributed to a channel... ,but If the channel is detected If a response is received or the response times out, then This allows for dynamic sensing of the real-time load of each channel, providing accurate input for traffic scheduling.
[0029] Step 202: Adaptively adjust the current concurrency threshold of the current channel based on the average latency of the channel requests.
[0030] To illustrate the specific implementation of step 202, as a preferred embodiment, the total average latency of all request information in the channel set is calculated within the concurrency threshold adjustment period. Step 202 includes: if the average latency of channel requests is 0, then the current concurrency threshold of the current channel is adjusted to the initial concurrency threshold; if the average latency of channel requests is greater than the total average latency, then the current concurrency threshold of the current channel is adaptively reduced; if the average latency of channel requests is less than the total average latency, then the current concurrency threshold of the current channel is adaptively increased; if the average latency of channel requests is equal to the total average latency, then the current concurrency threshold of the current channel remains unchanged.
[0031] In implementation, assuming that the average latency of all requests is calculated within the m-th concurrency threshold adjustment period, the following parameters are used: ,channel The average delays are respectively ,like If the value is 0, it means that there are no requests in the m-th concurrency threshold adjustment period, and the current concurrency threshold is reset to the initial concurrency threshold. ;like Then, based on the concurrency threshold, the adaptive adjustment step size is used to adaptively reduce the current concurrency threshold of the current channel. For example, ;like Then, based on the concurrency threshold, the adaptive adjustment step size is increased, and the current concurrency threshold of the current channel is increased adaptively. For example, ;like If so, the current concurrency threshold of the current channel remains unchanged. .
[0032] Step 203: Mark the priority of each channel in the channel set, and traverse the channel set based on the priority.
[0033] Step 204: If the number of concurrent requests being processed by the current channel is less than the current concurrency threshold, then the current channel will be designated as the target channel.
[0034] Step 205: If the number of concurrent requests being processed by the current channel is greater than the current concurrency threshold, then continue to traverse the channels in the channel set based on priority until the number of concurrent requests being processed by the current channel is less than the current concurrency threshold.
[0035] Step 206: If the current channel's health status is faulty, continue traversing the channels in the channel set based on priority until the traversed channel's health status is normal and the number of concurrent requests being processed is less than the current concurrency threshold.
[0036] During implementation, within the m-th concurrency threshold adjustment period, the channel set is traversed sequentially according to priority. ,remember The current channel being traversed; if If so, skip the current channel. Continue traversing ;like and The current channel will be designated as the target channel, and the request information will be distributed to the target channel. ;like and Then continue iterating. If k=m, then return all busy-related prompts for all channels or re-traverse the channel set to ensure the overall load balance and service continuity of the system.
[0037] Step 207: If the current channel health status was faulty in the previous concurrency threshold adjustment period, and the current channel health status is normal in the current concurrency threshold adjustment period, then the current channel status is determined to have returned to normal.
[0038] Step 208: For the current channel, use the adaptive recovery mechanism to set the concurrency threshold for each health detection cycle in the next concurrency threshold adjustment cycle.
[0039] During implementation, if and 1 indicates the channel. If the heartbeat detection passes during the m-th concurrency threshold adjustment cycle, normal operation is restored. Within the m+1 concurrency threshold adjustment cycles, data can be sent to the channel. Distribute traffic requests. Within m+1 concurrency threshold periods, if the channel health status of the current health check period is normal, the concurrency threshold for the next health check period is adaptively increased; if the channel health status of the current health check period is faulty, the current concurrency threshold remains unchanged.
[0040] The adaptive recovery mechanism involves adaptively adjusting the concurrency threshold for the current health monitoring cycle based on the channel's health status. That is, to avoid channel... In the event of a transient overload after recovery, a gradual fault recovery mechanism will be adopted, specifically including: recording... For channels The result of the i-th heartbeat check within the m-th concurrent threshold adjustment period. This indicates that no heartbeat response was received. This indicates that a heartbeat response has been received. For channels The concurrency threshold after the i-th heartbeat check within the m-th concurrency threshold adjustment period. ;like ,but ;like 0, then .
[0041] By applying the technical solution of this embodiment, and by providing a load balancing method, apparatus, device, and medium based on adaptive concurrency control, and by introducing a hierarchical scheduling mechanism, adaptive concurrency control, and progressive fault recovery strategy, intelligent allocation of request traffic, precise management and control of channel load, and smooth handling of fault recovery are achieved. Furthermore, adaptive scaling of channel capabilities is enabled, improving the resource utilization, availability, and stability of the distributed system. Compared with existing load balancing strategies such as round-robin, random, weighted round-robin, and least-connection load balancing, this embodiment differs from traditional round-robin / fixed-weight scheduling by designing a hierarchical scheduling mechanism to prioritize traffic allocation to the optimal channel, improving user experience. Unlike fixed-threshold control, it achieves adaptive adjustment of concurrency thresholds under hierarchical scheduling, enabling differentiated scheduling based on service channel performance latency, interface cost, reliability, and other attributes, automatically adapting to channel scaling, improving resource utilization, and reducing operation and maintenance costs. The progressive fault recovery strategy avoids instantaneous overload after recovery, achieving a smooth transition.
[0042] Furthermore, as Figure 1 In terms of specific implementation, this application provides a load balancing device based on adaptive concurrency control, such as... Figure 3 As shown, the device includes: a processing module 31, an adjustment module 32, and a sending module 33.
[0043] The processing module 31 is used to adjust the request information within the period based on the real-time acquired concurrency threshold, calculate the average latency of channel requests, and determine the number of concurrent request information being processed by the channel.
[0044] The adjustment module 32 is used to adaptively adjust the current concurrency threshold of the current channel based on the average latency of the channel requests.
[0045] The sending module 33 is used to traverse the channels in the channel set using an adaptive scheduling mechanism, determine the target channel, and send the current request information to the target channel; wherein, the adaptive scheduling mechanism is set according to the number of concurrent request information being processed by the channel and the current concurrency threshold.
[0046] In specific application scenarios, such as Figure 4 As shown, the concurrent threshold adjustment period includes multiple health detection periods, and the sending module 33 includes a health detection unit 331.
[0047] The health detection unit 331 is used to detect the channel health status within the current concurrency threshold adjustment period. Specifically, it is used to: when the current channel health status is faulty in the previous concurrency threshold adjustment period, if the heartbeat detection for the first consecutive health detection period is normal within the current concurrency threshold adjustment period, then the current channel health status is determined to be normal; when the current channel health status is normal in the previous concurrency threshold adjustment period, if the heartbeat detection fails for more than the second health detection period within the current concurrency threshold adjustment period, then the current channel health status is determined to be faulty.
[0048] In a specific application scenario, the processing module 31 includes: a determination unit 311.
[0049] The determining unit 311 is used to determine the concurrent number of request information being processed by the channel based on the request information within the adjustment period obtained in real time and the concurrent threshold. Specifically, it is used to: obtain the number of request information in the current channel using a channel counter; when a request information distributed to the current channel is received, the number of request information in the channel counter is incremented by one; continue to monitor the response status of the request information, and if the response status is detected as normal or timed out, the number of request information in the channel counter is decremented by one, and the adjusted number of request information is used as the concurrent number of request information being processed by the current channel.
[0050] In a specific application scenario, the adjustment module 32 includes a calculation unit 321.
[0051] The calculation unit 321 is used to calculate the total average latency of all request information in the channel set within the concurrency threshold adjustment period. If the average latency of channel requests is 0, the current concurrency threshold of the current channel is adjusted to the initial concurrency threshold. If the average latency of channel requests is greater than the total average latency, the current concurrency threshold of the current channel is adaptively reduced. If the average latency of channel requests is less than the total average latency, the current concurrency threshold of the current channel is adaptively increased. If the average latency of channel requests is equal to the total average latency, the current concurrency threshold of the current channel remains unchanged.
[0052] In specific application scenarios, the sending module 33 includes: a first channel determination unit 332 and a second channel determination unit 333.
[0053] The first channel determination unit 332 is used to mark the channels in the channel set with priorities respectively, and traverse the channel set based on the priorities; if the number of concurrent request information being processed by the current channel is less than the current concurrency threshold, then the current channel is used as the target channel; if the number of concurrent request information being processed by the current channel is greater than the current concurrency threshold, then the channels in the channel set are traversed based on the priorities until the number of concurrent request information being processed by the current channel is less than the current concurrency threshold.
[0054] The second channel determination unit 333 is used to continue traversing the channels in the channel set based on priority if the current channel's health status is faulty, until the traversed channel's health status is normal and the number of concurrent request messages being processed is less than the current concurrency threshold.
[0055] In specific application scenarios, the device also includes a recovery module 34.
[0056] The recovery module 34 is used to determine that the current channel status has been restored to normal if the current channel health status in the previous concurrency threshold adjustment cycle was faulty and the current channel health status in the current concurrency threshold adjustment cycle is normal; for the current channel, the concurrency threshold of each health detection cycle in the next concurrency threshold adjustment cycle is set using an adaptive recovery mechanism; wherein, the adaptive recovery mechanism is to adaptively adjust the concurrency threshold of the current health detection cycle by determining the channel health status in the current health detection cycle.
[0057] It should be noted that other corresponding descriptions of the functional units involved in the load balancing device based on adaptive concurrency control provided in this application embodiment can be found in the following references. Figure 1 and Figure 2 The corresponding descriptions in [the document] will not be repeated here.
[0058] Based on the above, Figure 1 and Figure 2 Accordingly, this application also provides a computer storage medium storing a computer program that, when executed by a processor, implements the above-described method. Figure 1 and Figure 2 The load balancing method based on adaptive concurrency control is shown.
[0059] Based on this understanding, the technical solution of this application can be embodied in the form of a software product, which can be stored in a storage medium (such as a CD-ROM, USB flash drive, or portable hard drive) and includes several instructions to cause a computer device (such as a personal computer, server, or network device) to execute the methods described in the various implementation scenarios of this application.
[0060] Based on the above, Figure 1 , Figure 2 The method shown, and Figure 3 To achieve the above objectives, the present application also provides a computer device, specifically a personal computer, server, network device, etc., as shown in the virtual device embodiment. This physical device includes a storage medium and a processor; the storage medium stores a computer program; the processor executes the computer program to achieve the above-described objectives. Figure 1 and Figure 2 The load balancing method based on adaptive concurrency control is shown.
[0061] Optionally, the computer device may also include a user interface, a network interface, a camera, radio frequency (RF) circuitry, sensors, audio circuitry, a Wi-Fi module, etc. The user interface may include a display screen, input units such as a keyboard, etc., and optional user interfaces may also include USB ports, card reader ports, etc. The network interface may optionally include standard wired interfaces, wireless interfaces (such as Bluetooth interfaces, Wi-Fi interfaces), etc.
[0062] Those skilled in the art will understand that the computer device structure provided in this embodiment does not constitute a limitation on the physical device, and may include more or fewer components, or combine certain components, or have different component arrangements.
[0063] The storage medium may also include an operating system and a network communication module. The operating system is a program that manages the hardware and software resources of a computer device, supporting the operation of information processing programs and other software and / or programs. The network communication module is used to enable communication between the various components within the storage medium, as well as communication with other hardware and software within the physical device.
[0064] Through the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented using software plus necessary general-purpose hardware platforms, or it can be implemented in hardware. By applying the technical solution of this application, compared with existing load balancing strategies such as round-robin, random, weighted round-robin, and least-connection strategies, this embodiment can achieve intelligent allocation of request traffic, precise control of channel load, and smooth handling of fault recovery by introducing hierarchical scheduling mechanisms, adaptive concurrency control, and progressive fault recovery strategies. It can also achieve adaptive scaling of channel capabilities, improving the resource utilization, availability, and stability of the distributed system.
[0065] Those skilled in the art will understand that the accompanying drawings are merely schematic diagrams of a preferred embodiment, and the modules or processes shown in the drawings are not necessarily essential for implementing this application. Those skilled in the art will understand that the modules in the apparatus of the embodiment can be distributed within the apparatus of the embodiment as described, or they can be located in one or more apparatuses different from this embodiment, with corresponding changes. The modules of the above-described embodiment can be combined into one module, or further divided into multiple sub-modules.
[0066] The serial numbers in this application are for descriptive purposes only and do not represent the superiority or inferiority of any particular implementation scenario. The above disclosures are merely a few specific implementation scenarios of this application; however, this application is not limited thereto, and any variations conceived by those skilled in the art should fall within the protection scope of this application.
Claims
1. A load balancing method based on adaptive concurrency control, characterized in that, include: Based on the request information within the concurrency threshold adjustment period obtained in real time, calculate the average latency of channel requests and determine the number of concurrent requests being processed by the channel. The current concurrency threshold of the current channel is adaptively adjusted based on the average latency of the channel requests. The adaptive scheduling mechanism is used to traverse the channels in the channel set, determine the target channel, and send the current request information to the target channel; The adaptive scheduling mechanism is set based on the number of concurrent requests being processed by the channel and the current concurrency threshold.
2. The method according to claim 1, characterized in that, The concurrency threshold adjustment cycle includes multiple health detection cycles. Within the current concurrency threshold adjustment cycle, the steps for detecting the channel health status include: If the current channel health status is faulty in the previous concurrency threshold adjustment period, and the heartbeat detection of the first consecutive health detection period is normal within the current concurrency threshold adjustment period, then the current channel health status is determined to be normal. If the current channel health status is normal during the previous concurrency threshold adjustment period, and if the heartbeat detection fails to respond for more than the second health detection period within the current concurrency threshold adjustment period, then the current channel health status is determined to be faulty.
3. The method according to claim 1 or 2, characterized in that, The step of determining the number of concurrent requests being processed by the channel based on the request information within the adjustment period obtained in real time includes: Use a channel counter to obtain the number of request messages for the current channel; When a request message is received to be distributed to the current channel, the number of request messages in the channel counter is incremented by one; Continue to monitor the response status of the request information. If the response status is detected as normal or timed out, the number of request information in the channel counter is reduced by one, and the adjusted number of request information is used as the number of concurrent request information being processed by the current channel.
4. The method according to claim 1, characterized in that, The step of adaptively adjusting the current concurrency threshold of the current channel based on the average latency of all requests in the channel set within the concurrency threshold adjustment period includes: If the average latency of channel requests is 0, then the current concurrency threshold of the current channel will be adjusted to the initial concurrency threshold. If the average latency of channel requests is greater than the total average latency, the current concurrency threshold of the current channel will be adaptively reduced. If the average latency of channel requests is less than the total average latency, the current concurrency threshold of the current channel will be increased adaptively. If the average latency of channel requests equals the total average latency, then the current concurrency threshold of the current channel remains unchanged.
5. The method according to claim 1, characterized in that, The step of using an adaptive scheduling mechanism to traverse the channels in the channel set and determine the target channel includes: Each channel in the channel set is assigned a priority, and the channel set is traversed based on the priority. If the number of concurrent requests being processed by the current channel is less than the current concurrency threshold, then the current channel will be used as the target channel. If the number of concurrent requests being processed by the current channel is greater than the current concurrency threshold, then the channels in the channel set will continue to be traversed based on priority until the number of concurrent requests being processed by the current channel is less than the current concurrency threshold.
6. The method according to claim 5, characterized in that, The step of using an adaptive scheduling mechanism to traverse the channels in the channel set and determine the target channel further includes: If the current channel's health status is faulty, then continue traversing the channels in the channel set based on priority until the traversed channel's health status is normal and the number of concurrent requests being processed is less than the current concurrency threshold.
7. The method according to claim 2, characterized in that, The method further includes: If the current channel health status was faulty in the previous concurrency threshold adjustment period, and the current channel health status is normal in the current concurrency threshold adjustment period, then the current channel status is determined to have returned to normal. For the current channels, use an adaptive recovery mechanism to set the concurrency threshold for each health check cycle in the next concurrency threshold adjustment cycle; The adaptive recovery mechanism involves adaptively adjusting the concurrency threshold of the current health detection cycle by determining the channel health status of the current health detection cycle.
8. A load balancing device based on adaptive concurrency control, characterized in that, include: The processing module is used to adjust the request information within the period based on the real-time acquired concurrency threshold, calculate the average latency of channel requests, and determine the number of concurrent requests being processed by the channel. The adjustment module is used to adaptively adjust the current concurrency threshold of the current channel based on the average latency of the channel requests; The sending module is used to traverse the channels in the channel set using an adaptive scheduling mechanism, determine the target channel, and send the current request information to the target channel; The adaptive scheduling mechanism is set based on the number of concurrent requests being processed by the channel and the current concurrency threshold.
9. A computer storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the load balancing method based on adaptive concurrency control as described in any one of claims 1 to 7.
10. A computer device, comprising a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, characterized in that, When the processor executes the program, it implements the load balancing method based on adaptive concurrency control as described in any one of claims 1 to 7.