Resource regulation system and method, electronic device and storage medium, program product

Abstract

This application discloses a resource control system and method, electronic device and storage medium, and program product, relating to the field of server technology. It includes: a target switch, used to divide multiple received resource usage requests into multiple workload groups, assign a correspondence between multiple servers and multiple workload groups, obtain the proportion of real-time resources provided by each server in the overall server resources, and obtain multiple resource ratios; based on the multiple resource ratios and correspondences, a target scheduling strategy is determined for balancing the real-time resource usage configuration within the multiple servers; and a server-side proxy, connected to the target switch, used to control the target number of resource usage requests sent by multiple workload groups to multiple servers using the target scheduling strategy, thereby regulating the consumption progress of real-time resources; this solves the problem that in rack-level computing environments, server load balancing cannot be performed based on differences in requested resource requirements.

Description

Technical Field

[0001] This application relates to the field of server technology, and more specifically, to a resource control system and method, electronic device and storage medium, and program product. Background Technology

[0002] As data centers evolve towards low-latency, high-throughput microsecond-level service scenarios, traditional single-server-based resource scheduling architectures are no longer sufficient to meet the stringent requirements of applications for resource elasticity and scheduling efficiency. To overcome the bottleneck of single machines, the industry generally adopts a rack-level collaborative scheduling architecture that combines programmable top-of-rack switches and local server scheduling, aiming to achieve cross-server load balancing through the network layer. However, the aforementioned technologies have significant drawbacks: First, the algorithms used are application-independent scheduling, distributing requests solely based on server queue length. This fails to recognize the fundamental differences in CPU resource consumption among different request types, leading to a mix of high-load and low-load requests deployed on the same server, causing severe head-blocking issues, significantly increasing tail latency, and compromising the stability of service level agreements (SLAs). Second, resource allocation in these technologies is mostly static or based on coarse-grained load metrics, lacking the ability to model request execution time distribution. This prevents the allocation of CPU (Central Processing Unit) resources according to actual computing needs, resulting in some servers being overloaded and others idle. Third, when workloads change dynamically, they rely heavily on manual intervention or periodic global rescheduling, lacking lightweight, low-jitter online adaptation mechanisms. Policy updates can easily cause performance fluctuations and request backlogs. Therefore, the lack of homogenization shaping of mixed workloads at the ToR switch level can easily lead to unbalanced loads between servers and head-blocking within servers, reducing resource utilization and scheduling efficiency.

[0003] There is currently no effective solution to the problem that server load balancing cannot be performed based on differences in requested resource requirements in rack-level computing environments. Summary of the Invention

[0004] This application provides a resource regulation system and method, electronic device and storage medium, and program product to at least solve the problem in related technologies that server load balancing cannot be performed based on differences in requested resource requirements in rack-level computing environments.

[0005] According to one embodiment of this application, a resource regulation system is provided, comprising: a target switch, configured to divide received multiple resource usage requests into multiple workload groups and assign multiple servers to the multiple workload groups, wherein each workload group contains at least one resource usage request, and the multiple servers are connected to the target switch; obtaining the proportion of real-time resources provided by each of the multiple servers in the total resources of the server, thereby obtaining multiple resource ratios; determining a target scheduling strategy for balancing the real-time resource usage configuration within the multiple servers based on the multiple resource ratios and the correspondence; and a server-side proxy, connected to the target switch, configured to use the target scheduling strategy to control the target number of resource usage requests sent by the multiple workload groups to the multiple servers, thereby regulating the consumption progress of the real-time resources.

[0006] According to another embodiment of this application, a resource regulation method is provided, comprising: dividing multiple received resource usage requests into multiple workload groups, and assigning multiple servers to the multiple workload groups, wherein each workload group contains at least one resource usage request, and the multiple servers are connected to a target switch; obtaining the proportion of real-time resources provided by each of the multiple servers in the total resources of the server, thereby obtaining multiple resource ratios; and determining a target scheduling strategy for balancing the real-time resource usage configuration within the multiple servers based on the multiple resource ratios and the correspondence, wherein the target scheduling strategy is used to regulate the consumption progress of real-time resources on the multiple servers and control the target number of resource usage requests sent by the multiple workload groups to the multiple servers.

[0007] According to yet another embodiment of this application, a computer-readable storage medium is also provided, wherein a computer program is stored therein, and the computer program is configured to perform the steps in any of the above method embodiments when it is run.

[0008] According to yet another embodiment of this application, an electronic device is also provided, including a memory and a processor, wherein the memory stores a computer program and the processor is configured to run the computer program to perform the steps in any of the above method embodiments.

[0009] According to yet another embodiment of this application, a computer program product is also provided, including a computer program that, when executed by a processor, implements the steps in any of the above method embodiments.

[0010] This application utilizes a target switch to collect historical request execution time and frequency information. Cluster analysis is then used to group requests with similar computational load characteristics into several workload groups, ensuring that requests within each group consume CPU and other resources in a consistent manner. Subsequently, the target switch monitors in real time the proportion of currently available resources to total resources on each server. Combining the allocation relationship between each workload group and the server, the load weight of each server is dynamically calculated, and a group-level target scheduling policy is generated accordingly. This policy not only specifies which servers each workload group should be assigned to but also precisely constrains the upper limit of the number of requests each server should receive per unit time, thereby avoiding localized overload caused by uncontrolled request influx. The server-side proxy, acting as the policy execution unit, controls the number of resource usage requests received according to the target scheduling policy, allowing requests to be accepted and processed only within the quota, ensuring that server resource consumption remains in a steady-state and controllable equilibrium. This technical solution solves the problem in related technologies where server load balancing based on differences in request resource requirements is impossible in rack-level computing environments. Furthermore, intelligent workload shaping and precise control of request quantity based on real-time resource ratio are achieved at the network layer, significantly reducing tail latency and resource contention in rack-level scheduling, and realizing load balancing, head blocking elimination, and controllable resource consumption progress. Attached Figure Description

[0011] To more clearly illustrate the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 This is a hardware structure block diagram of a switching device according to a resource regulation method according to an embodiment of this application;

[0013] Figure 2 This is a flowchart of a resource regulation method according to an embodiment of this application;

[0014] Figure 3 This is a flowchart illustrating a network load balancing method for rack-level computing resource scheduling according to an embodiment of this application;

[0015] Figure 4 This is a schematic diagram of the framework of a resource regulation system according to an embodiment of this application;

[0016] Figure 5 This is a schematic diagram of a data packet format according to an embodiment of this application;

[0017] Figure 6 This is a structural block diagram of a resource regulation system according to an embodiment of this application. Detailed Implementation

[0018] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the protection scope of this application.

[0019] It should be noted that, in the description of this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. The terms "first," "second," etc., in this application are used to distinguish similar objects and are not used to describe a specific order or sequence.

[0020] To enable those skilled in the art to better understand the present application, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0021] As an optional implementation, the method embodiments provided in this application can be executed in a switching device or a similar computing device. Taking operation on a switching device as an example, Figure 1 This is a hardware structure block diagram of a switching device according to a resource regulation method based on an embodiment of this application. Figure 1 As shown, a switch device may include one or more ( Figure 1 Only one is shown in the diagram. A processor 102 (which may include, but is not limited to, a microprocessor MPU or a programmable logic device FPGA, etc.) and a memory 104 for storing data are also shown. The aforementioned switch device may further include a transmission device 106 for communication functions and an input / output device 108. Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the aforementioned switching equipment. For example, the switching equipment may also include components that are more... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown.

[0022] The memory 104 can be used to store computer programs, such as application software programs and modules, like the computer program corresponding to the resource regulation method in this embodiment. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, thus implementing the aforementioned method. The memory 104 may include high-speed random access memory and non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to a switching device via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0023] The transmission device 106 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by a communication provider for the switching device. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 may be a Radio Frequency (RF) module used for wireless communication with the Internet.

[0024] This embodiment provides a resource regulation method applied to the aforementioned switch device. Figure 2 This is a flowchart of a resource regulation method according to an embodiment of this application, such as... Figure 2 As shown, the process includes the following steps:

[0025] Step S202: Divide the received multiple resource usage requests into multiple workload groups and assign multiple servers to multiple workload groups. Each workload group contains at least one resource usage request, and multiple servers have a connection relationship with the target switch.

[0026] Step S204: Obtain the proportion of real-time resources provided by each of the multiple servers in the total resources of the server, and obtain multiple resource proportions;

[0027] Step S206: Based on multiple resource ratios and corresponding relationships, determine the target scheduling strategy for balancing the real-time resource usage configurations within multiple servers. The target scheduling strategy is used to regulate the consumption progress of real-time resources on multiple servers and control the target number of resource usage requests sent by multiple workload groups to multiple servers.

[0028] The above method utilizes the target switch to collect historical request execution time and frequency information. Cluster analysis is then used to group requests with similar computational load characteristics into several workload groups, ensuring that requests within each group consume CPU and other resources in a consistent manner. Subsequently, the target switch monitors in real time the proportion of currently available resources to total resources on each server. Combining the allocation relationship between each workload group and the server, the load weight of each server is dynamically calculated, and a group-level target scheduling policy is generated accordingly. This policy not only specifies which servers each workload group should be allocated to but also precisely constrains the upper limit of the number of requests each server should receive per unit time, thereby avoiding localized overload caused by uncontrolled request influx. The server-side proxy, acting as the policy execution unit, controls the number of resource usage requests received according to the target scheduling policy, allowing requests to be accepted and processed only within the quota, ensuring that server resource consumption remains in a steady-state, controllable, and balanced range. This technical solution solves the problem in related technologies where server load balancing based on differences in request resource requirements is impossible in rack-level computing environments. Furthermore, intelligent workload shaping and precise control of request quantity based on real-time resource ratio are achieved at the network layer, significantly reducing tail latency and resource contention in rack-level scheduling, and realizing load balancing, head blocking elimination, and controllable resource consumption progress.

[0029] In an exemplary embodiment, multiple received resource usage requests are divided into multiple workload groups, including: collecting the execution time and execution quantity of different types of requests executed by multiple servers within a preset time period; determining the average execution time of each type of request and the proportion of the real-time request quantity of each type of request in the total number of requests based on the execution time and execution quantity; constructing a clustering analysis dataset using the average execution time and request proportion corresponding to each type of request as a feature vector; clustering requests with similar characteristics in the dataset into one category using a preset clustering algorithm to obtain multiple clustering results; and determining multiple workload groups based on the multiple clustering results.

[0030] Optionally, to achieve intelligent workload grouping, the target switch continuously collects the execution time and number of resource usage requests processed by each server connected to it for a preset period of time, such as 10 seconds to 1 minute. Through statistical analysis, the average execution time of each request type and its relative proportion in the total request flow are obtained, thereby constructing a two-dimensional feature vector reflecting the resource consumption characteristics of the requests, namely, average execution time × request proportion. Subsequently, the feature vectors of all request types are used to form a clustering input dataset, and unsupervised learning is performed on it using clustering algorithms such as K-Means. Request types with similar computational load characteristics are automatically grouped into the same cluster, such as high-time-consuming and low-frequency requests, low-time-consuming and high-frequency requests, etc. Each cluster corresponds to a workload group with highly homogeneous resource requirements. This process does not rely on manual rules, but dynamically generates the optimal grouping scheme based on real load data, eliminating the risk of resource contention and head blocking caused by the coexistence of mixed requests within the server from the source.

[0031] In an exemplary embodiment, before assigning the correspondence between multiple servers and multiple workload groups, the method further includes: determining a first CPU requirement for a single request type according to a first formula, and determining a second CPU requirement for each workload group according to a second formula, and configuring CPU resources for processing requests for each workload group based on the first CPU requirement, the second CPU requirement, and the total CPU resources corresponding to the multiple servers.

[0032] Optionally, before allocating workload groups to servers, the actual CPU resource requirements of each request type are quantified based on their resource consumption characteristics. Specifically, the CPU requirement for a single request type is calculated as follows: for each request type r, its CPU requirement D... r The first formula for calculating D is: r= (E r × r ) / (∑ i E i × i ), where E r The average execution time for request type r. r The percentage of request type r in the total requests, with E representing the total number of request types. i × i The sum is used to normalize the CPU requirements. CPU requirement calculation for workload group g: For workload group g, its CPU requirement D... g The sum of the CPU requirements for all request types within this group, i.e.: Formula 2 D g=∑ r∈g D r Server resource allocation: Allocate appropriate server CPU resources to each workload group based on their CPU demand ratio. Let the total CPU resources be R. total If the set of all workload groups is G, then the central processing unit resources R allocated to workload group g are... g For the third formula: R g =(D g / (∑ g∈ G D g ))×R total The above method achieves a mapping from request characteristics to resource requirements and then to physical resource allocation, providing a quantifiable basis for resource configuration for subsequent intra-group load balancing and server-side request quantity control, and avoiding scheduling imbalances caused by resource redundancy or insufficiency.

[0033] In an exemplary embodiment, before assigning the correspondence between multiple servers and multiple workload groups, the method further includes: calculating the weight value of each server in the corresponding workload group based on the number of CPU cores allocated to each workload group by each server; maintaining a counter for each workload group; and, when the target switch receives a new resource usage request, allocating the new resource usage request to the target server that supports the request based on the target value recorded by the counter and the weight value corresponding to each server.

[0034] Optionally, to achieve efficient cross-server load balancing within a workload group, before assigning the correspondence between servers and workload groups, the scheduling weight of each server within that workload group is calculated based on the number of CPU cores allocated to it. This means the server's weight is proportional to the number of cores it is allocated to; for example, a server with 4 cores has a weight of 4, and a server with 2 cores has a weight of 2, reflecting the relative differences in their carrying capacity. Subsequently, a circular counter is maintained in the target switch for each workload group, with an initial value of 0. When the target switch receives a new resource usage request from a client, it first identifies the workload group to which the request belongs, reads the current value of the corresponding counter, and then, according to a preset weighted round-robin rule, cyclically maps the counter values ​​according to the server weight sequence. For example, for two servers with weights [2,3], when the counter value is 0 or 1, the request is assigned to the server with weight 2; when the counter value is 2, 3, or 4, it is assigned to the server with weight 3. The counter increments by 1 after each allocation and automatically resets to 0 when it exceeds the sum of the weights, ensuring that requests are evenly distributed according to the resource capacity ratio of each server. Then, by using a lightweight counter and hardware-supported arithmetic logic, the accuracy and scalability of load balancing within the group are significantly improved.

[0035] In an exemplary embodiment, after determining the target scheduling strategy for balancing the real-time resource usage configurations within multiple servers based on multiple resource ratios and correspondences, the method further includes: after the server completes the processing of the resource usage request, extracting the actual execution time of the resource usage request from the response data packet fed back by the server to the target switch, and statistically analyzing the arrival rate and average execution time change trends of each workload group based on a preset time window.

[0036] Optionally, after generating the target scheduling policy based on resource ratios and correspondences, a feedback loop is used to continuously sense dynamic load changes: when a server completes processing any resource usage request, it embeds the actual execution time of the request, such as a Time field, into a dedicated extended field in the response data packet and sends the response back to the target switch; the target switch collects all response data packets in the control plane, based on a preset time window, such as 10ms or 100ms. Aggregate analysis is performed on the requests for each workload group, and the arrival rate of the requests within the window is statistically analyzed, i.e., the changing trend of the number of requests per unit time and the average execution time, thereby capturing the drift characteristics of the workload in real time, such as sudden increases or increases in execution time; without relying on active server reporting or independent monitoring, this ensures that the scheduling policy always closely matches the actual load state.

[0037] In an exemplary embodiment, after determining the target scheduling strategy for balancing the real-time resource usage configurations across multiple servers based on multiple resource ratios and correspondences, the method further includes: within a preset time window, counting the number of requests arriving per unit time for each workload group as the request arrival rate; and determining the dynamic processing threshold set by the operation and maintenance object for each workload group in the target switch, wherein the dynamic processing threshold is calculated based on the number of CPU cores currently allocated to the workload group and the average execution time; if the request arrival rate of a certain workload group is greater than the dynamic processing threshold, determining that a certain workload group has a newly added short-term instantaneous burst load, and generating a load alarm prompt based on the load of the short-term instantaneous burst load.

[0038] In other words, after the target scheduling policy takes effect, the data plane of the target switch continuously monitors the request arrival rate (i.e., the number of requests received per unit time) of each workload group and compares it in real time with the preset dynamic processing threshold of the group. The dynamic processing threshold is calculated by the number of CPU cores currently allocated to the workload group and the historical average execution time of the group. The formula is: Dynamic processing threshold = number of allocated cores × (1 / average execution time), which represents the maximum number of requests that the group can stably process per unit time under the current resource configuration. When the real-time request arrival rate of a workload group exceeds the threshold continuously, it is determined that it has encountered a short-term instantaneous burst of load, and the load alarm mechanism is triggered. A load alarm prompt containing the group identifier, the over-limit range, the duration and the suggested response action (such as starting request cloning) is generated and pushed to the workload adapter module to provide a trigger signal for subsequent adaptive response (such as request cloning and traffic splitting).

[0039] In an exemplary embodiment, after determining the target scheduling strategy for balancing the real-time resource usage configurations across multiple servers based on multiple resource ratios and correspondences, the method further includes: in the event of a short-term instantaneous burst of load, calculating the excess quantity corresponding to resource usage requests exceeding the dynamic processing threshold, and generating cloned resource usage requests equal to the excess quantity; and allocating the cloned resource usage requests to servers associated with the other workload group with the lowest current load among multiple workload groups for parallel processing.

[0040] Optionally, if the request arrival rate of a workload group exceeds its dynamic processing threshold, the excess number of requests exceeding the threshold is immediately calculated, i.e., request arrival rate – dynamic processing threshold. Based on this excess number, the data plane of the target switch automatically generates an equal number of "cloned resource usage requests". Each cloned resource usage request copies the business content and metadata of the original request, but adds a unique identifier Flag=2 to distinguish it from the original request Flag=1. Subsequently, in all other workload groups, the current load status of the associated servers of each group is evaluated in real time, such as queue length, CPU utilization, or number of requests processed per unit time. Several target servers with the lowest load are selected, and cloned resource usage requests are dynamically distributed to these servers for parallel processing using a weighted round-robin or minimum load priority method. This quickly diverts burst traffic without increasing the pressure on the original overloaded group. The original request is still routed to the original target server of its group according to the normal strategy. During the response phase, by identifying the Flag value, only the first response returned is adopted, which is usually the fast response of the cloned resource usage request. The redundant responses that arrive later are discarded to ensure that the client obtains the best service experience with low latency.

[0041] In an exemplary embodiment, after determining the target scheduling policy for balancing the real-time resource usage configuration within multiple servers based on multiple resource ratios and correspondences, the method further includes: calculating the cumulative change in CPU demand for multiple workload groups; and, if the cumulative change in CPU demand exceeds a preset change threshold, determining that multiple servers connected to the target switch have experienced long-term workload changes, triggering a policy regeneration process for the target scheduling policy.

[0042] Optionally, after the target scheduling strategy is deployed and running, the workload monitoring component continuously collects request execution time and arrival rate data for each workload group, and calculates the current CPU demand for each workload group in real time according to the first and second formulas. Then, the demand change of all workload groups is accumulated, which is the sum of the difference between the current total demand and the baseline demand at the time of the last strategy generation. When the accumulated CPU demand change exceeds a preset threshold, such as 10%, it is determined that a continuous, structural long-term workload change has occurred, rather than a short-term fluctuation. Then, the target scheduling strategy regeneration process is automatically triggered. Specifically, this includes re-executing K-Means clustering to identify new homogeneous groups, building a micro-benchmark environment to evaluate the tail latency and resource utilization of each candidate grouping scheme, and recalculating the CPU quota and server binding relationship of each group based on the resource ratio.

[0043] In summary, by setting a threshold for the amount of change, noise and instantaneous jitter are effectively filtered out, ensuring that policy adjustments are only initiated when the actual load structure changes, thus avoiding the oscillations caused by frequent reconfigurations. After the policy is regenerated, the new configuration is issued through incremental updates, and a request bounce mechanism is linked to smoothly transition, thereby achieving long-term dynamic alignment between the scheduling policy and the actual business load.

[0044] In an exemplary embodiment, after determining the target scheduling strategy for balancing the real-time resource usage configuration within multiple servers based on multiple resource ratios and correspondences, the method further includes: receiving response data packets from multiple servers, identifying request identifiers in the response data packets, retaining the target response data packet returned first by each server, and discarding other data packets except for the target response data packet.

[0045] Optionally, during the execution of the target scheduling strategy, when a request cloning mechanism is triggered due to a short-term burst of load, the same logical request may be sent to both the original server and the clone target server simultaneously, generating multiple response data packets. To ensure that the client receives the lowest latency response experience, after receiving response data packets from multiple servers, the target switch first extracts the unique request identifier from each response and groups multiple responses belonging to the same request. Subsequently, it retains only the target response data packet that arrives first in the request group, i.e., the one with the shortest response time, and immediately forwards the response to the client, while actively discarding the remaining redundant response packets that arrive late, regardless of whether they originate from the original request or the clone resource usage request. In this way, adaptive response selection is achieved based on the first-to-arrive principle, without relying on the preset priority of server performance or network paths, effectively eliminating the duplicate processing and client confusion caused by multi-path responses, significantly improving the consistency of user experience and the predictability of responses. Moreover, this discarding behavior is completed in a hardware-level low-latency manner in the switch data plane, without affecting the normal scheduling of other requests, thus improving request processing efficiency.

[0046] To facilitate understanding of the implementation methods of this application, relevant scenarios are explained below, but these explanations do not limit the scope of this application.

[0047] To better understand this application, the relevant terms are explained below.

[0048] Rack-level computing resource scheduling: A technology for scheduling central processing unit resources within a rack, which can break through the limitations of a single server and realize unified management and scheduling of central processing unit resources of multiple servers within a rack, so as to meet the requirements of microsecond-level services for low latency and high throughput.

[0049] ToR (Top-of-Rack Switch): A top-of-rack switch is a key device for network communication within a rack. In this invention, it undertakes important functions such as workload shaping and cross-server load balancing.

[0050] JSQ (Join-the-Shortest-Queue Algorithm): The shortest queue first algorithm is a traditional load balancing algorithm that achieves load balancing by distributing requests to servers with the shortest queue length. However, this algorithm is application-independent and cannot take into account the actual load impact of requests on the server.

[0051] HoL (Head-of-Line Blocking) refers to a problem where long requests at the front of the queue block the processing of shorter requests, causing a significant increase in response latency for shorter requests and impacting overall service performance.

[0052] CFCFS (Centralized First Come First Serve): A centralized first-come, first-served scheduling algorithm, a simple and efficient scheduling algorithm that processes requests in the order they arrive, and can achieve better performance when handling uniform workloads of the same type.

[0053] DARC (Dynamic Application-aware Reserved Cores): This algorithm reserves a dedicated CPU core for each workload group, and requests from different workload groups are processed on their respective reserved cores, thus achieving resource isolation between workload groups.

[0054] Weighted Round Robin Algorithm: A load balancing algorithm that assigns different weights to servers based on their processing capacity, and distributes requests to each server according to the weight ratio to achieve a more reasonable load distribution.

[0055] Exponentially weighted moving average algorithm: a method for calculating the average value of a sequence. By assigning higher weights to recent data, it can more quickly reflect the trend of data changes. In this invention, it is used to track the execution time of requests.

[0056] Request cloning: This invention is a technique for handling short-term, transient bursts of workload by cloning overloaded requests to servers with lower loads for processing, thereby avoiding performance degradation caused by sudden traffic surges.

[0057] Request bounce: The mechanism in this invention for dealing with long-term workload changes involves bouncing unprocessed long requests in the server queue back to the switch for rescheduling during the scheduling policy adjustment process, prioritizing the processing of newly arriving short requests, and avoiding head blocking.

[0058] SLA (Service Level Agreement): A service level agreement is a service quality standard agreed upon between a service provider and a user, including indicators such as response time, throughput, and availability.

[0059] RTT (Round-Trip Time): Round-trip time refers to the total time required for data to travel from the sender to the receiver and then from the receiver to the sender, and is an important time delay indicator in network communication and load balancing.

[0060] As an optional implementation, this application proposes a network load balancing method for rack-level computing resource scheduling. At the network layer, the ToR switch actively converts mixed workloads into multiple uniform workload groups, where requests within each workload group are highly homogeneous in terms of computing resources, such as CPU requirements. This workload shaping effectively solves the load imbalance and HoL blocking problems in existing technologies, thereby achieving simple and near-optimal cross-server load balancing and intra-server request scheduling. Subsequently, by shaping the workload at the network layer through the ToR switch, combined with cross-server load balancing and intra-server scheduling, a three-level scheduling architecture is formed.

[0061] Optionally, Figure 3 This is a flowchart illustrating a network load balancing method for rack-level computing resource scheduling according to an embodiment of this application;

[0062] It mainly includes the following parts:

[0063] Part 1: Workload Shaping. This section implements workload shaping on the ToR switch. Based on the estimated execution time of requests, it divides mixed workloads into multiple highly homogeneous workload groups. Requests within each workload group have similar computing resource requirements, thus ensuring that HoL (House of Level) blocking issues are avoided during subsequent load balancing. The core of workload shaping lies in accurately estimating the execution time of requests and grouping them accordingly. For example, requests can be divided into Group 1 (short requests), Group 2 (medium requests), and Group 3 (long requests).

[0064] Part Two: Intra-group Load Balancing. Also on the ToR switch, a weighted round-robin algorithm is used to achieve cross-server load balancing within each workload group. Based on the proportion of computing resources allocated to each server for that workload group, a server weight is determined, and requests are distributed to each server according to their weight proportions, ensuring load balancing among servers and fully utilizing server resources.

[0065] Part Three: In-Server Scheduling. On each server, a simple and efficient centralized first-come, first-served scheduling algorithm is used for scheduling of the received uniform workload of a single type. For the few servers that need to handle multiple workload groups, a dynamic application-aware reserved resource scheduling algorithm is used to ensure resource isolation and scheduling performance between different workload groups.

[0066] Part Four: Dynamic Adaptation Mechanisms. To cope with dynamic changes in workload, two dynamic adaptation mechanisms are designed: 1. Request Rebound Mechanism: When changes in long-term workload require adjustments to the scheduling strategy, unprocessed long requests in the server queue are bounced back to the switch for rescheduling, prioritizing newly arriving short requests. This avoids HoL blocking during strategy adjustments and ensures service performance stability. 2. Request Cloning Mechanism: When a short-term, transient surge in workload is detected, the overloaded requests are cloned to servers in other workload groups with lower loads for processing. This quickly alleviates the pressure on overloaded servers and avoids increased tail latency caused by sudden traffic surges.

[0067] As an optional implementation, the aforementioned network load balancing method for rack-level computing resource scheduling can be implemented based on a programmable ToR switch and a server-side agent. The ToR switch is responsible for workload shaping, intra-group load balancing, and request cloning and policy distribution in the dynamic adaptation mechanism. Each server is equipped with a server-side agent component, which is responsible for receiving scheduling policies, executing intra-server scheduling, and implementing the request bounce mechanism. In addition, the system also includes a workload monitoring component for real-time collection and analysis of workload data to provide a basis for dynamic adjustments.

[0068] Optionally, Figure 4 This is a schematic diagram of a resource regulation system according to an embodiment of this application. It includes: a rack-top switch and a server cluster. The server cluster comprises multiple individual servers. The rack-top switch includes: a workload shaper, an intra-group load balancer, a workload detector, a workload adapter, and a burst processor. Each individual server includes: an intra-cluster scheduler, a server-side proxy component, and a central processing unit for computing resources.

[0069] The workload shaper is used to generate workload shaping strategies, and its specific components include:

[0070] The first step involves using a customized K-Means clustering algorithm to generate multiple workload grouping candidate schemes based on the monitored request execution time and request ratio. The specific steps are as follows: First, data collection: using a workload monitoring component, the execution time and number of different request types are collected over a period of time, and the average execution time and proportion of each request type in the total requests are calculated. Second, feature extraction: the average execution time and request ratio of each request type are used as feature vectors to construct a dataset for clustering analysis. Third, K-Means clustering: the value of K is set to range from 1 to the total number of request types. For each K value, the K-Means clustering algorithm is run to group request types with similar characteristics into one class, forming a workload group. In this way, multiple grouping candidate schemes corresponding to different K values ​​are generated.

[0071] Step 2: Optimal candidate selection, including: setting up a micro-benchmark environment, building a test environment similar to the actual production environment, including the same configuration of ToR switches, servers, and the same network topology; performance testing, simulating request processing under different load intensities in the test environment for each group of candidate schemes, collecting and recording key performance indicators such as 99% tail latency, throughput, and server resource utilization; scheme evaluation and selection, selecting the scheme that can achieve high server resource utilization while ensuring low tail latency as the final workload shaping strategy.

[0072] The third step is resource allocation, which includes calculating the CPU resource requirements of each workload group based on the final determined workload shaping strategy and allocating corresponding server resources to them. The specific calculation process is as follows:

[0073] CPU requirement calculation for a single request type: For each request type r, its CPU requirement D r The first formula for calculating D is: r= (E r × r ) / (∑ i E i × i ), where E r The average execution time for request type r. r The percentage of request type r in the total requests, with E representing the total number of request types. i × i The sum is used to normalize CPU requirements. CPU requirement calculation for workload group g: For workload group g, its CPU requirement D... g The sum of CPU requirements for all request types within this group, i.e., Formula 2 D g =∑ r∈g D r Server resource allocation: Allocate appropriate server CPU resources to each workload group based on their CPU demand ratio. Let the total CPU resources be R. total If the set of all workload groups is G, then the CPU resources R allocated to workload group g are... g For the third formula: R g =(D g / (∑ g∈G D g ))×R total .

[0074] The above method realizes the mapping from request characteristics to resource requirements and then to physical resource allocation, providing a quantifiable basis for resource configuration for subsequent intra-group load balancing and server-side request quantity control, and avoiding scheduling imbalance caused by resource redundancy or insufficiency.

[0075] An intra-group load balancer is used on a ToR switch to achieve cross-server load balancing within a workload group based on a weighted round-robin algorithm. Specifically, it includes:

[0076] Weighting: The weight of a server is determined based on the number of CPU cores allocated to each workload group. For example, if server A is allocated 2 cores to workload group g and server B is allocated 3 cores, then the weights of servers A and B are 2 and 3, respectively.

[0077] Round-Robin Scheduling: A counter is maintained for each workload group, initially set to 0. When a request arrives at that workload group, the counter value increments by 1 sequentially. When the counter value exceeds the sum of the weights of all servers in the workload group, the counter is reset to 0. The server to which the request is assigned is determined based on the current counter value. For example, in workload group g, server A has a weight of 2, server B has a weight of 3, and the sum of their weights is 5. When the counter value is 1 or 2, the request is assigned to server A; when the counter value is 3, 4, or 5, the request is assigned to server B. This method ensures that requests are distributed according to the server's weight, achieving load balancing.

[0078] Counter Implementation: In the P4 programming language, registers are used to store the counter value for each workload group, and the increment and reset operations of the counter are implemented through the arithmetic logic unit. When a request arrives, the workload group to which the request belongs is first found, then the counter value corresponding to that workload group is read, the target server is determined based on the counter value and the server weight, and finally the counter value is updated (incremented by 1, and reset to 0 if it exceeds the sum of the weights).

[0079] Optionally, the above-mentioned intra-cluster scheduler is used for intra-server scheduling implementation, including:

[0080] For servers handling only a single workload group, a centralized First-Come, First-Served (FCFS) scheduling algorithm is used. This algorithm maintains a request queue within the server. Requests enter the queue in the order they arrive, and the scheduler sequentially retrieves requests from the head of the queue and allocates them to CPU cores for processing. Because requests within the workload group have similar execution times, the centralized FCFS scheduling algorithm avoids the HoL (Holiday-of-Arrival) blocking problem and achieves near-optimal tail latency. In its implementation, a centralized scheduler manages the request allocation across all CPU cores, ensuring that requests are processed fairly according to their arrival order.

[0081] For servers that need to handle multiple workload groups, a dynamic application-aware resource reservation scheduling algorithm is employed. This algorithm reserves dedicated CPU cores for each workload group, and requests from different workload groups are processed on their respective reserved cores, achieving resource isolation between workload groups. For example, if the server reserves 2 cores for workload group g1 and 3 cores for workload group g2, then requests for g1 will only be processed on these 2 cores, and requests for g2 will only be processed on these 3 cores, avoiding interference between different workload groups. In the DARC scheduling algorithm, a core reservation table records the core number corresponding to each workload group. When a request arrives, the core reservation table is consulted based on the workload group to which the request belongs, and the request is assigned to the corresponding core for processing.

[0082] Optionally, a workload detector, deployed in the control plane of the ToR switch, enables real-time monitoring of workload by analyzing request execution time information in the response data packet. After the server completes request processing, it records the actual execution time of the request in the Time field of the response data packet and sends the response data packet back to the ToR switch. Figure 5 This is a schematic diagram of a data packet format according to an embodiment of this application. The data packet includes: Ethernet frame header (ETH bytes), Internet Protocol header (IP bytes), User Datagram Protocol header (UDP bytes), new packet header bytes, and payload data. The new packet header bytes include: an 8-bit type byte, an 8-bit flag byte, a 32-bit index byte, and a 32-bit time byte.

[0083] Optionally, a workload adapter is used to implement a dynamic adaptive mechanism, which includes: (1) Long-term workload change adaptation (request bounce): The workload adapter periodically (e.g., every 10ms) receives statistical data reported by the workload monitoring component and calculates the difference between the current CPU demand and allocated resources for each workload group. If the cumulative demand difference of all workload groups exceeds a preset threshold (set to 10% by default in this invention), it is determined that the long-term workload has changed significantly and the scheduling strategy needs to be adjusted. The workload adapter re-executes the workload shaping strategy generation process (including group candidate generation, optimal candidate selection and resource allocation) to generate a new scheduling strategy. After the server agent detects the new scheduling strategy, it compares the currently processed workload group with the workload group allocated in the new strategy. If the new strategy requires processing short request workload groups, and there are still unprocessed long requests in the current queue, these long requests are bounced back to the ToR switch and rescheduled by the switch to the server of the corresponding long request workload group for processing. At the same time, the server prioritizes processing newly arrived short requests to avoid HoL blocking and ensure the stability of service performance during the strategy adjustment process. (2) Short-term transient burst adaptation (request cloning): In the packet processing path of the ToR switch, a request counter is maintained for each workload group to count the number of requests arriving within each fixed time interval. Optionally, the default setting is 20μs. Simultaneously, based on the CPU resources allocated to the workload group, a threshold for the maximum number of requests that can be processed within that time interval is calculated. If this threshold is exceeded, it is determined that the workload group has experienced a short-term transient burst, and the number of cloned requests is the number exceeding the threshold, ensuring that overloaded requests can be diverted in a timely manner. Then, the burst processor of the ToR switch generates a cloned resource usage request and sets different Flag values ​​for the original request and the cloned resource usage request. For example, the Flag for the original request is 1, and the Flag for the cloned resource usage request is 2. At the same time, the cloned resource usage request is assigned to another workload group with the lowest load. The original request is scheduled to the server of its respective workload group according to the normal process, and the cloned resource usage request is scheduled to the server of the assigned target workload group.

[0084] It's important to note that workload shaping divides mixed workloads into multiple evenly distributed workload groups, with each server handling only a single type of request, fundamentally avoiding HoL blocking. Simultaneously, a precise intra-group load balancing algorithm ensures load balance among servers, preventing overload and increased tail latency caused by load imbalance. By avoiding HoL blocking and load imbalance, server resources are utilized more efficiently. Furthermore, the simple and efficient CFCFS scheduling algorithm reduces intra-server scheduling overhead, further enhancing server processing capacity. Request bounce mechanisms and incremental policy updates allow for rapid adjustments to scheduling strategies when long-term workloads change, avoiding performance fluctuations during policy adjustments. Request cloning mechanisms quickly distribute overloaded requests, alleviating pressure on overloaded servers and preventing a sharp increase in tail latency due to short-term traffic surges. Precise intra-group load balancing and demand-based resource allocation ensure that CPU resources are allocated rationally according to the actual needs of the workload, avoiding resource waste. It can provide higher throughput and lower tail latency with the same hardware resources, thus reducing the number of hardware devices required and lowering hardware procurement costs while meeting the same service requirements.

[0085] In summary, the above embodiments demonstrate that the ToR switch's workload shaping technique, by analyzing the CPU execution time characteristics of requests, divides mixed workloads into multiple highly homogeneous workload groups. This technology overcomes the limitations of existing solutions that address scheduling issues on the server side, preventing HoL blocking problems at the source and laying the foundation for subsequent load balancing and intra-server scheduling. Simultaneously, a three-level collaborative scheduling architecture is constructed, encompassing workload shaping, intra-group load balancing, and intra-server scheduling. The scheduling components at each level work closely together to optimize overall scheduling performance. Furthermore, request bounce and request cloning mechanisms are designed for long-term workload changes and short-term bursts, respectively, enabling rapid adaptation to dynamic workload changes and ensuring service performance stability.

[0086] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the related technology, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0087] This embodiment also provides a resource regulation system for implementing the above embodiments and preferred embodiments; details already described will not be repeated. As used below, the term "module" can refer to a combination of software and / or hardware that performs a predetermined function. Although the apparatus described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0088] Figure 6 This is a structural block diagram of a resource regulation system according to an embodiment of this application, such as... Figure 6 As shown, the device includes:

[0089] The target switch 602 is used to divide multiple received resource usage requests into multiple workload groups and assign multiple servers to the multiple workload groups. Each workload group contains at least one resource usage request, and the multiple servers are connected to the target switch. The switch obtains the proportion of real-time resources provided by each of the multiple servers in the total server resources, resulting in multiple resource ratios. Based on the multiple resource ratios and the correspondences, a target scheduling strategy is determined to balance the real-time resource usage configuration within the multiple servers.

[0090] Server-side proxy 604 is connected to the target switch and is used to control the target number of resource usage requests sent by the multiple workload groups to the multiple servers using the target scheduling policy, so as to regulate the consumption progress of the real-time resources.

[0091] The system utilizes target switches to collect historical request execution time and frequency information. Cluster analysis is used to group requests with similar computational load characteristics into several workload groups, ensuring that requests within each group consume CPU and other resources in a consistent manner. Subsequently, the target switches monitor in real-time the proportion of available resources to total resources on each server. Combining the allocation relationship between workload groups and servers, the system dynamically calculates the load weight of each server and generates a group-level target scheduling policy. This policy not only specifies which servers each workload group should be assigned to but also precisely constrains the maximum number of requests each server should receive per unit time, thus preventing localized overload caused by uncontrolled request influx. The server-side proxy, acting as the policy execution unit, controls the number of resource usage requests received according to the target scheduling policy, allowing requests to be accepted and processed only within the quota, ensuring that server resource consumption remains in a steady, controllable equilibrium. This technical solution solves the problem in related technologies where server load balancing based on differences in request resource requirements is impossible in rack-level computing environments. Furthermore, intelligent workload shaping and precise control of request quantity based on real-time resource ratio are achieved at the network layer, significantly reducing tail latency and resource contention in rack-level scheduling, and realizing load balancing, head blocking elimination, and controllable resource consumption progress.

[0092] In an exemplary embodiment, the target switch further includes: a load shaping module and a resource ratio calculation module, respectively, for collecting the execution time and execution quantity of multiple servers executing different types of requests within a preset time period; determining the average execution time of each type of request and the proportion of the real-time request quantity of each type of request in the total number of requests based on the execution time and execution quantity; constructing a clustering analysis dataset by using the average execution time and request ratio corresponding to each type of request as a feature vector; clustering requests with similar characteristics in the dataset into one category by using a preset clustering algorithm to obtain multiple clustering results; and determining multiple workload groups based on the multiple clustering results.

[0093] Optionally, to achieve intelligent workload grouping, the target switch continuously collects the execution time and number of resource usage requests processed by each server connected to it for a preset period of time, such as 10 seconds to 1 minute. Through statistical analysis, the average execution time of each request type and its relative proportion in the total request flow are obtained, thereby constructing a two-dimensional feature vector reflecting the resource consumption characteristics of the requests, namely, average execution time × request proportion. Subsequently, the feature vectors of all request types are used to form a clustering input dataset, and unsupervised learning is performed on it using clustering algorithms such as K-Means. Request types with similar computational load characteristics are automatically grouped into the same cluster, such as high-time-consuming and low-frequency requests, low-time-consuming and high-frequency requests, etc. Each cluster corresponds to a workload group with highly homogeneous resource requirements. This process does not rely on manual rules, but dynamically generates the optimal grouping scheme based on real load data, eliminating the risk of resource contention and head blocking caused by the coexistence of mixed requests within the server from the source.

[0094] In summary, by automatically constructing a two-dimensional feature vector of average execution time and request ratio based on real load data on the target switch, and combining it with unsupervised clustering algorithms such as K-Means, intelligent, data-driven grouping of mixed workloads is achieved. This realizes semantic alignment between request type and computing resource requirements from the source, automatically clustering the originally heterogeneous and mixed request streams into multiple workload groups with highly consistent computing characteristics. This completely avoids the server head-blocking (HoL) and resource contention problems caused by the mixture of long and short requests in traditional solutions. Through long-term, continuous load sampling and adaptive clustering, the inherent structure of the business load is dynamically captured, generating a grouping strategy that accurately matches the actual resource consumption pattern. This not only significantly improves the efficiency of subsequent intra-group load balancing and CFCFS scheduling, but also achieves systematic optimization of end-to-end latency with lightweight network layer operations without modifying the server kernel or adding additional hardware.

[0095] In an exemplary embodiment, the target switch further includes: a load configuration module, configured to determine a first CPU requirement for a single request type according to a first formula, and a second CPU requirement for each workload group according to a second formula, and configure CPU resources for processing requests for each workload group based on the first CPU requirement, the second CPU requirement, and the total CPU resources corresponding to the plurality of servers.

[0096] Optionally, before allocating workload groups to servers, the actual CPU resource requirements of each request type are quantified based on their resource consumption characteristics. Specifically, the CPU requirement for a single request type is calculated as follows: for each request type r, its CPU requirement D... r The first formula for calculating D is: r= (Er × r ) / (∑ i E i × i ), where E r The average execution time for request type r. r The percentage of request type r in the total requests, with E representing the total number of request types. i × i The sum is used to normalize the CPU requirements. CPU requirement calculation for workload group g: For workload group g, its CPU requirement D... g The sum of the CPU requirements for all request types within this group, i.e.: Formula 2 D g =∑ r∈g D r Server resource allocation: Allocate appropriate server CPU resources to each workload group based on their CPU demand ratio. Let the total CPU resources be R. total If the set of all workload groups is G, then the central processing unit resources R allocated to workload group g are... g For the third formula: R g =(D g / (∑ g∈ G D g ))×R total The above method achieves a mapping from request characteristics to resource requirements and then to physical resource allocation, providing a quantifiable basis for resource configuration for subsequent intra-group load balancing and server-side request quantity control, and avoiding scheduling imbalances caused by resource redundancy or insufficiency.

[0097] In summary, the above methods significantly improve CPU utilization, avoid resource waste caused by server overload or idleness, and provide a scientific and calculable weight basis for subsequent weighted round-robin load balancing, ensuring the fairness and efficiency of cross-server scheduling.

[0098] In an exemplary embodiment, the target switch further includes: a resource ratio calculation module, configured to calculate the weight value of each server in the corresponding workload group based on the number of CPU cores allocated by each server to each workload group; and a round-robin scheduling module, connected to the load shaping module and the resource ratio calculation module respectively, configured to maintain a counter for each workload group, and, when the target switch receives a new resource usage request, allocate the new resource usage request to the target server that supports the request based on the target value recorded by the counter and the weight value corresponding to each server.

[0099] Optionally, to achieve efficient cross-server load balancing within a workload group, before assigning the correspondence between servers and workload groups, the scheduling weight of each server within that workload group is calculated based on the number of CPU cores allocated to it. This means the server's weight is proportional to the number of cores it is allocated to; for example, a server with 4 cores has a weight of 4, and a server with 2 cores has a weight of 2, reflecting the relative differences in their carrying capacity. Subsequently, a circular counter is maintained in the target switch for each workload group, with an initial value of 0. When the target switch receives a new resource usage request from a client, it first identifies the workload group to which the request belongs, reads the current value of the corresponding counter, and then, according to a preset weighted round-robin rule, cyclically maps the counter values ​​according to the server weight sequence. For example, for two servers with weights [2,3], when the counter value is 0 or 1, the request is assigned to the server with weight 2; when the counter value is 2, 3, or 4, it is assigned to the server with weight 3. The counter increments by 1 after each allocation and automatically resets to 0 when it exceeds the sum of the weights, ensuring that requests are evenly distributed according to the resource capacity ratio of each server. Then, by using a lightweight counter and hardware-supported arithmetic logic, the accuracy and scalability of load balancing within the group are significantly improved.

[0100] In an exemplary embodiment, the target switch further includes a workload monitoring module, which is used to extract the actual execution time of the resource usage request from the response data packet fed back by the server to the target switch after the server completes the processing of the resource usage request, and to statistically analyze the arrival rate and average execution time change trend of each workload group based on a preset time window.

[0101] Optionally, after generating the target scheduling policy based on resource ratios and correspondences, a feedback loop is used to continuously sense dynamic load changes: when a server completes processing any resource usage request, it embeds the actual execution time of the request, such as a Time field, into a dedicated extended field in the response data packet and sends the response back to the target switch; the target switch collects all response data packets in the control plane, based on a preset time window, such as 10ms or 100ms. Aggregate analysis is performed on the requests for each workload group, and the arrival rate of the requests within the window is statistically analyzed, i.e., the changing trend of the number of requests per unit time and the average execution time, thereby capturing the drift characteristics of the workload in real time, such as sudden increases or increases in execution time; without relying on active server reporting or independent monitoring, this ensures that the scheduling policy always closely matches the actual load state.

[0102] In an exemplary embodiment, the workload monitoring module further includes an alarm unit; the alarm unit is configured to count the number of requests arriving per unit time for each workload group within a preset time window, as the request arrival rate; and determine the dynamic processing threshold set by the maintenance object for each workload group in the target switch, wherein the dynamic processing threshold is calculated based on the number of central processing unit cores currently allocated to the workload group and the average execution time; if the request arrival rate of a certain workload group is greater than the dynamic processing threshold, determine that the certain workload group has a new short-term instantaneous burst load, and generate a load alarm prompt based on the load amount of the short-term instantaneous burst load.

[0103] In other words, after the target scheduling policy takes effect, the data plane of the target switch continuously monitors the request arrival rate (i.e., the number of requests received per unit time) of each workload group and compares it in real time with the preset dynamic processing threshold of the group. The dynamic processing threshold is calculated by the number of CPU cores currently allocated to the workload group and the historical average execution time of the group. The formula is: Dynamic processing threshold = number of allocated cores × (1 / average execution time), which represents the maximum number of requests that the group can stably process per unit time under the current resource configuration. When the real-time request arrival rate of a workload group exceeds the threshold continuously, it is determined that it has encountered a short-term instantaneous burst of load, and the load alarm mechanism is triggered. A load alarm prompt containing the group identifier, the over-limit range, the duration and the suggested response action (such as starting request cloning) is generated and pushed to the workload adapter module to provide a trigger signal for subsequent adaptive response (such as request cloning and traffic splitting).

[0104] In summary, by dynamically calculating and comparing the "request arrival rate" and "dynamic processing threshold" in real time for each workload group on the target switch data plane, millisecond-level adaptive perception and accurate identification of short-term burst loads are achieved. The dynamic processing threshold is calculated in real time based on the number of CPU cores allocated to the group and the historical average execution time, allowing the threshold to automatically adjust with resource configuration and load characteristics, avoiding misjudgments or sluggish responses caused by fixed thresholds. When the real-time arrival rate continuously exceeds the dynamic threshold, the system determines it as a real burst load rather than occasional jitter, automatically triggering a load alarm mechanism. This generates structured alarm information containing group identifier, exceedance range, duration, and recommended actions (such as request cloning), and pushes it to the adapter module in the control plane. This improves the adaptability to different load characteristics and resource allocation scenarios, significantly enhances the elastic response to instantaneous traffic surges, and effectively prevents tail latency spikes and service degradation caused by burst loads.

[0105] In an exemplary embodiment, the target switch further includes: a request cloning module; the request cloning module is connected to the workload monitoring module, and is used to calculate the excess quantity corresponding to the resource usage request that exceeds the dynamic processing threshold when a short-term instantaneous burst load is detected, and generate a cloned resource usage request equal to the excess quantity; and allocate the cloned resource usage request to the server associated with the other workload group with the lowest current load among the plurality of workload groups for parallel processing.

[0106] Optionally, if the request arrival rate of a workload group exceeds its dynamic processing threshold, the excess number of requests exceeding the threshold is immediately calculated, i.e., request arrival rate – dynamic processing threshold. Based on this excess number, the data plane of the target switch automatically generates an equal number of "cloned resource usage requests". Each cloned resource usage request copies the business content and metadata of the original request, but adds a unique identifier Flag=2 to distinguish it from the original request Flag=1. Subsequently, in all other workload groups, the current load status of the associated servers of each group is evaluated in real time, such as queue length, CPU utilization, or number of requests processed per unit time. Several target servers with the lowest load are selected, and cloned resource usage requests are dynamically distributed to these servers for parallel processing using a weighted round-robin or minimum load priority method. This quickly diverts burst traffic without increasing the pressure on the original overloaded group. The original request is still routed to the original target server of its group according to the normal strategy. During the response phase, by identifying the Flag value, only the first response returned is adopted, which is usually the fast response of the cloned resource usage request. The redundant responses that arrive later are discarded to ensure that the client obtains the best service experience with low latency.

[0107] In an exemplary embodiment, the target switch further includes: an adaptive judgment module, configured to calculate the cumulative change in CPU demand of the plurality of workload groups; and, if the cumulative change in CPU demand is greater than a preset change threshold, determine that the plurality of servers connected to the target switch have experienced long-term workload changes, and trigger the policy regeneration process of the target scheduling policy.

[0108] Optionally, after the target scheduling strategy is deployed and running, the workload monitoring component continuously collects request execution time and arrival rate data for each workload group, and calculates the current CPU demand for each workload group in real time according to the first and second formulas. Then, the demand change of all workload groups is accumulated, which is the sum of the difference between the current total demand and the baseline demand at the time of the last strategy generation. When the accumulated CPU demand change exceeds a preset threshold, such as 10%, it is determined that a continuous, structural long-term workload change has occurred, rather than a short-term fluctuation. Then, the target scheduling strategy regeneration process is automatically triggered. Specifically, this includes re-executing K-Means clustering to identify new homogeneous groups, building a micro-benchmark environment to evaluate the tail latency and resource utilization of each candidate grouping scheme, and recalculating the CPU quota and server binding relationship of each group based on the resource ratio.

[0109] In summary, by setting a threshold for the amount of change, noise and instantaneous jitter are effectively filtered out, ensuring that policy adjustments are only initiated when the actual load structure changes, thus avoiding the oscillations caused by frequent reconfigurations. After the policy is regenerated, the new configuration is issued through incremental updates, and a request bounce mechanism is linked to smoothly transition, thereby achieving long-term dynamic alignment between the scheduling policy and the actual business load.

[0110] In one exemplary embodiment, the server-side proxy further includes: a scheduling execution module, configured to, when the server is handling only a single workload group, use a centralized first-come, first-served scheduling algorithm to allocate the resource usage requests to the server's central processing unit (CPU) cores for processing according to their arrival order; and when the server needs to handle multiple workload groups, use a dynamic application-aware reserved resource scheduling algorithm to allocate a dedicated CPU core to each workload group, so that the resource usage requests corresponding to different workload groups are executed independently on their respective reserved cores.

[0111] Optionally, the server-side proxy's scheduling and execution module intelligently identifies the number of workload groups carried by the server and dynamically switches between two efficient and low-overhead kernel scheduling strategies: When the server is only processing a single workload group, because the internal requests have highly homogeneous execution characteristics (such as similar average execution times), the scheduling and execution module adopts a centralized first-come, first-served (CFCFS) algorithm, which allocates requests to any available CPU core in the order of arrival, achieving extremely simple, contention-free serialized processing, minimizing scheduling overhead and eliminating the risk of head blocking (HoL); when the server needs to process multiple workload groups simultaneously, the scheduling and execution module switches to a dynamic application-aware reserved resource scheduling algorithm (DARC), which allocates resources according to the strategy from the ToR switch, statically reserving a dedicated CPU core for each workload group, ensuring that requests from different groups are executed independently on completely isolated hardware resources, completely eliminating cross-group resource contention and scheduling interference, thereby maintaining the deterministic and low-latency characteristics of scheduling within each group while ensuring service-level isolation.

[0112] In one exemplary embodiment, the server-side proxy further includes a request bounce module; the request bounce module is connected to an adaptive judgment module on the target switch, and is used to check the server request queue on each server when the policy regeneration process of the target scheduling policy has been triggered, to determine whether there are unprocessed resource usage requests belonging to the workload group removed in the target scheduling policy in the server request queue; if there are unprocessed resource usage requests belonging to the workload group removed in the target scheduling policy, the unprocessed resource usage requests are resent as bounce requests to the target switch, instructing the target switch to reallocate the unprocessed resource usage requests carried by the bounce requests to other servers according to the updated target scheduling policy.

[0113] In other words, the server-side proxy, through its built-in request bounce module, works in conjunction with the adaptive judgment module on the ToR switch to achieve a smooth transition and zero performance fluctuation during the dynamic update of the scheduling policy: When the system triggers the regeneration of the scheduling policy due to long-term load changes, the bounce module actively scans the local request queue to identify incomplete requests that are still pending and belong to workload groups that have been removed by the new policy; once such "legacy requests" are found, they are marked as "bounce requests" and actively sent back to the ToR switch with the original request identifier and execution context; after receiving the request, the switch reroutes these requests to other target servers that are still carrying the workload group for processing according to the latest generated scheduling policy, instead of having the original server continue to block execution.

[0114] In one exemplary embodiment, the target switch further includes a response filtering module, configured to receive response data packets from multiple servers, identify request identifiers in the response data packets, retain the target response data packet returned first by each server, and discard other data packets besides the target response data packet.

[0115] Optionally, during the execution of the target scheduling strategy, when a request cloning mechanism is triggered due to a short-term burst of load, the same logical request may be sent to both the original server and the clone target server simultaneously, generating multiple response data packets. To ensure that the client receives the lowest latency response experience, after receiving response data packets from multiple servers, the target switch first extracts the unique request identifier from each response and groups multiple responses belonging to the same request. Subsequently, it retains only the target response data packet that arrives first in the request group, i.e., the one with the shortest response time, and immediately forwards the response to the client, while actively discarding the remaining redundant response packets that arrive late, regardless of whether they originate from the original request or the clone resource usage request. In this way, adaptive response selection is achieved based on the first-to-arrive principle, without relying on the preset priority of server performance or network paths, effectively eliminating the duplicate processing and client confusion caused by multi-path responses, significantly improving the consistency of user experience and the predictability of responses. Moreover, this discarding behavior is completed in a hardware-level low-latency manner in the switch data plane, without affecting the normal scheduling of other requests, thus improving request processing efficiency.

[0116] Embodiments of this application also provide a computer-readable storage medium storing a computer program, wherein the computer program is configured to execute the steps in any of the above method embodiments when run.

[0117] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard disk, magnetic disk, or optical disk.

[0118] Embodiments of this application also provide an electronic device, including a memory and a processor, wherein the memory stores a computer program and the processor is configured to run the computer program to perform the steps in any of the above method embodiments.

[0119] In one exemplary embodiment, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor and the input / output device is connected to the processor.

[0120] Embodiments of this application also provide a computer program product, which includes a computer program that, when executed by a processor, implements the steps in any of the above method embodiments.

[0121] Embodiments of this application also provide another computer program product, including a non-volatile computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps in any of the above method embodiments.

[0122] Embodiments of this application also provide a computer program that includes computer instructions stored in a computer-readable storage medium; a processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the steps in any of the above method embodiments.

[0123] Specific examples in this embodiment can be found in the examples described in the above embodiments and exemplary implementations, and will not be repeated here.

[0124] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0125] Obviously, those skilled in the art should understand that the modules or steps of this application described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. They can be implemented using computer-executable program code, and thus can be stored in a storage device for execution by a computing device. In some cases, the steps shown or described can be performed in a different order than those presented here, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, this application is not limited to any particular combination of hardware and software.

[0126] The resource regulation system and method, electronic device and storage medium, and program product provided in this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are only for the purpose of helping to understand the method and core ideas of this application. It should be noted that those skilled in the art can make several improvements and modifications to this application without departing from the principles of this application, and these improvements and modifications also fall within the protection scope of the claims of this application.

Claims

1. A resource regulation system, characterized in that, include: A target switch is used to divide multiple received resource usage requests into multiple workload groups and assign multiple servers to the multiple workload groups. Each workload group contains at least one resource usage request, and the multiple servers are connected to the target switch. The switch obtains the proportion of real-time resources provided by each of the multiple servers in the total resources of the server, resulting in multiple resource ratios. Based on the multiple resource ratios and the correspondences, a target scheduling strategy is determined to balance the real-time resource usage configuration within the multiple servers. A server-side proxy, connected to the target switch, is used to control the target number of resource usage requests sent by the multiple workload groups to the multiple servers using the target scheduling policy, so as to regulate the consumption progress of the real-time resources.

2. The resource regulation system according to claim 1, characterized in that, The target switch also includes: The load shaping module is used to collect the execution time and number of different types of requests executed by multiple servers within a preset time period; based on the execution time and number of requests, it determines the average execution time of each type of request and the proportion of the real-time request number of each type of request in the total number of requests; it uses the average execution time and request proportion corresponding to each type of request as a feature vector to construct a cluster analysis dataset; it uses a preset clustering algorithm to cluster requests with similar characteristics in the dataset into one class to obtain multiple clustering results; and it determines multiple workload groups based on the multiple clustering results.

3. The resource regulation system according to claim 1, characterized in that, The target switch also includes: The load configuration module is used to determine the first CPU requirement for a single request type and the second CPU requirement for each workload group, and to configure the CPU resources for processing requests for each workload group based on the first CPU requirement, the second CPU requirement, and the total CPU resources corresponding to the plurality of servers.

4. The resource regulation system according to claim 1, characterized in that, The target switch also includes: The resource ratio calculation module is used to calculate the weight value of each server in the corresponding workload group based on the number of central processing unit cores allocated to each workload group by each server. The polling scheduling module, connected to the load shaping module and the resource ratio calculation module respectively, is used to maintain a counter for each workload group. When the target switch receives a new resource usage request, it allocates the new resource usage request to the target server that supports the request based on the target value recorded by the counter and the weight value corresponding to each server.

5. The resource regulation system according to claim 1, characterized in that, The target switch also includes: The workload monitoring module is used to extract the actual execution time of the resource usage request from the response data packet sent by the server to the target switch after the server completes the processing of the resource usage request, and to statistically analyze the arrival rate and average execution time trend of each workload group based on a preset time window.

6. The resource regulation system according to claim 5, characterized in that, The workload monitoring module also includes: an alarm unit; The alarm unit is used to count the number of requests arriving per unit time for each workload group within a preset time window, as the request arrival rate; and to determine the dynamic processing threshold set by the operation and maintenance object for each workload group in the target switch, wherein the dynamic processing threshold is calculated based on the number of central processing unit cores currently allocated to the workload group and the average execution time; if the request arrival rate of a certain workload group is greater than the dynamic processing threshold, it is determined that a certain workload group has added a short-term instantaneous burst load, and a load alarm prompt is generated based on the load of the short-term instantaneous burst load.

7. The resource regulation system according to claim 5, characterized in that, The target switch further includes: a request cloning module; The request cloning module is connected to the workload monitoring module and is used to calculate the excess quantity corresponding to resource usage requests that exceed the dynamic processing threshold when a short-term instantaneous burst of load is detected, and generate cloned resource usage requests equal to the excess quantity; the cloned resource usage requests are then allocated to the servers associated with the other workload groups with the lowest current load among the multiple workload groups for parallel processing.

8. The resource regulation system according to claim 1, characterized in that, The target switch also includes: An adaptive judgment module is used to calculate the cumulative change in CPU demand of the multiple workload groups; if the cumulative change in CPU demand is greater than a preset change threshold, it determines that multiple servers connected to the target switch have experienced long-term workload changes, and triggers the policy regeneration process of the target scheduling policy.

9. The resource regulation system according to claim 1, characterized in that, The server-side proxy also includes: The scheduling and execution module is used to allocate resource usage requests to the server's central processing unit (CPU) cores for processing in the order of arrival when the server is handling only a single workload group, using a centralized first-come, first-served scheduling algorithm; when the server needs to handle multiple workload groups, it uses a dynamic application-aware reserved resource scheduling algorithm to allocate a dedicated CPU core to each workload group, so that resource usage requests corresponding to different workload groups can be executed independently on their respective reserved cores.

10. The resource regulation system according to claim 1, characterized in that, The server-side proxy also includes: a request reverse module; The request bounce module is connected to the adaptive judgment module on the target switch. It is used to check the server request queue on each server and determine whether there are any unprocessed resource usage requests belonging to the workload group removed in the target scheduling policy when the policy regeneration process of the target scheduling policy has been triggered. If there are unprocessed resource usage requests belonging to workload groups removed from the target scheduling policy, the unprocessed resource usage requests are resent as bounce requests to the target switch, instructing the target switch to reallocate the unprocessed resource usage requests carried in the bounce requests to other servers according to the updated target scheduling policy.

11. The resource regulation system according to claim 1, characterized in that, The target switch also includes: The response filtering module is used to receive response data packets from multiple servers, identify the request identifier in the response data packets, retain the target response data packet returned first by each server, and discard other data packets except for the target response data packet.

12. A resource regulation method, characterized in that, include: The received resource usage requests are divided into multiple workload groups, and multiple servers are assigned to the multiple workload groups. Each workload group contains at least one resource usage request, and the multiple servers are connected to the target switch. Obtain the proportion of real-time resources provided by each of the multiple servers in the total server resources, and obtain multiple resource ratios; Based on the multiple resource ratios and the corresponding relationships, a target scheduling strategy is determined for balancing the real-time resource usage configuration within the multiple servers. The target scheduling strategy is used to regulate the consumption progress of real-time resources on the multiple servers and control the target number of resource usage requests sent by the multiple workload groups to the multiple servers.

13. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor, configured to implement the steps of the resource regulation method as described in claim 12 when executing the computer program.

14. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, wherein the computer program, when executed by a processor, implements the steps of the resource regulation method as described in claim 12.

15. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the resource regulation method as described in claim 12.

Images