A method and device for regulating cloud service cold start traffic flood, equipment and medium

Abstract

The application discloses a regulation method and device for relieving cloud service cold start traffic flood, equipment and medium, the method comprises: constructing a service dependency graph for a target micro-service cluster; determining a future several-step predicted load based on the service dependency graph using a time-series-graph attention prediction model; obtaining an observed state quantity of a to-be-regulated object in the target micro-service cluster, and constructing a current state vector based on the observed state quantity and the future several-step predicted load; determining a candidate admission ratio based on the current state vector through a reinforcement learning control strategy; dynamically updating a regulation strategy based on the candidate admission ratio to regulate the request traffic entering the to-be-regulated object; re-executing the step of constructing the service dependency graph for the target micro-service cluster until the cloud service cold start traffic flood is relieved. The application forms a closed loop among load prediction, dynamic throttling and safety constraint, no longer depends on a static preheating curve or a pure reactive feedback control, can give a prospective admission decision before the cold start flood arrives, reduces the overload risk in the initial recovery stage from the source, and improves the stability of the cloud service in response to traffic fluctuations.

Description

Technical Field

[0001] This application relates to the field of traffic control technology, and in particular to a control method, apparatus, equipment and medium for mitigating traffic surges during the cold start of cloud services. Background Technology

[0002] Modern cloud services commonly employ microservice architecture, CI / CD continuous delivery, and automatic scaling mechanisms. When service instances undergo fault recovery, rolling version releases, or launch new instances to handle sudden traffic surges, they typically experience a cold start process from unready to ready. During the cold start phase, the service's external processing capacity has not fully recovered, but upstream gateways, call chains, and clients have accumulated a large number of pending requests, and client timeout retries further amplify the short-term load.

[0003] Therefore, the cold start phase is prone to traffic surges. This manifests as follows: the request arrival rate is significantly higher than the steady-state load level in the initial stage of service recovery. The first few service nodes to recover bear a disproportionate traffic surge, leading to request queuing backlog, worsening tail latency, and CPU and memory resource saturation. In severe cases, it can create a vicious cycle of "recovery followed by overload," even triggering a wider cascading failure. For interactive microservice systems, this problem directly impacts service availability, response quality, and fault recovery efficiency.

[0004] To address the traffic surge issue during the cold start phase, existing technologies generally employ static rate limiting, token bucket, or warm-up curve schemes. These schemes achieve traffic control by pre-setting fixed traffic limits or linear / segmented warm-up curves. While these schemes are logically simple, they cannot adaptively adjust to changes in request backlog, retry intensity, and current service capacity based on different cold start events. This can easily lead to problems such as overly strict rate limiting resulting in slow recovery or overly lenient rate limiting causing renewed overload.

[0005] Therefore, existing technologies still need to be improved and enhanced. Summary of the Invention

[0006] The technical problem to be solved by this application is to provide a method, apparatus, equipment and medium for mitigating traffic surges during the cold start of cloud services, in order to address the shortcomings of the existing technology.

[0007] To address the aforementioned technical problems, the first aspect of this application provides a method for mitigating traffic surges during the cold start of cloud services, wherein the method specifically includes: A service dependency graph is constructed for the target microservice cluster, wherein the graph nodes represent service nodes, the edges represent the call relationships between service nodes, and each graph node carries a historical monitoring sequence. Based on the service dependency graph, a time-series graph attention prediction model is used to determine the predicted load for several future steps. Obtain the observed state values ​​of the objects to be controlled in the target microservice cluster, and construct the current state vector based on the observed state values ​​and the predicted load for the next few steps. Based on the current state vector, the candidate admission ratio is determined through a reinforcement learning control strategy. The control strategy is dynamically updated based on the candidate admission ratio to control the request traffic entering the object to be controlled. Re-execute the steps to build the service dependency graph for the target microservice cluster until the surge in traffic during the cold start of the cloud services is alleviated.

[0008] The aforementioned method for mitigating traffic surges during cloud service cold starts includes a historical monitoring sequence comprising several observable indicators from historical sampling times, wherein the observable indicators include one or more of request traffic, queue length, tail latency, CPU utilization, and memory utilization.

[0009] The aforementioned control method for mitigating traffic surges during the cold start of cloud services includes, wherein the observed state quantities include one or more of the following: current observed load, backlog queue, tail latency, current available capacity, and safety margin.

[0010] The aforementioned method for mitigating traffic surges during cloud service cold starts, wherein determining the predicted load for several future steps based on the service dependency graph using a time-series-graph attention prediction model specifically includes: The historical monitoring sequences carried by the graph nodes in the service dependency graph are feature-encoded to obtain the local temporal features corresponding to the graph nodes. The neighbor set of the graph node is determined based on the service dependency graph, and the local temporal features corresponding to the neighbor nodes in the neighbor set are aggregated through a multi-head attention mechanism to obtain the target node representation of the graph node. Determine the global representation corresponding to the service dependency graph, and predict the future load of the microservice based on the global representation and the target node representation of the graph nodes.

[0011] The aforementioned method for mitigating traffic surges during cloud service cold starts, wherein the step of aggregating the local temporal features corresponding to neighbor nodes in the neighbor set through a multi-head attention mechanism to obtain the target node representation of the graph node specifically includes: A query vector is constructed using the local temporal features corresponding to the graph nodes, and a value vector and a key vector are constructed using the local temporal features corresponding to the neighbor nodes in the neighbor set. Obtain the relative structural distance between the graph node and its neighboring nodes, and determine the relative structural encoding between the graph node and its neighboring nodes based on the relative structural distance; An initial attention score is determined based on the query vector and the key vector, and the initial attention score is corrected using the relative structure encoding to obtain the final attention score. The value vectors corresponding to neighboring nodes are fused using the attention scores to obtain the target node representation of the graph node.

[0012] The aforementioned method for mitigating traffic surges during cloud service cold starts, wherein the dynamic updating of the control strategy based on the candidate admission ratio specifically includes: Construct a capacity safety constraint, wherein the capacity safety constraint is that the product of the currently observed loads is less than or equal to the difference between the available capacity and the safety margin; The candidate admission ratio is projected and corrected under the capacity safety constraint to obtain the safe admission ratio. The control strategy is dynamically updated based on the safety access ratio.

[0013] The aforementioned method for mitigating traffic surges during cold starts of cloud services includes a reinforcement learning control strategy where the reward is the negative of a multi-objective cost function. This multi-objective cost function uses the admission ratio as the control decision quantity, integrates tail latency SLO default, cluster idle capacity, and recovery time as continuous penalty terms in the time domain, and introduces a queue clearing terminal incentive term.

[0014] A second aspect of this application provides a control device for mitigating traffic surges during the cold start of cloud services, wherein the control device specifically includes: The load prediction module is used to construct a service dependency graph for the target microservice cluster. Based on the service dependency graph, a time-series-graph attention prediction model is used to determine the predicted load for several future steps. In this module, graph nodes represent service nodes, edges represent the call relationships between service nodes, and each graph node carries a historical monitoring sequence. The control module is used to acquire the observed state quantity of the object to be controlled in the target microservice cluster, construct the current state vector based on the observed state quantity and the predicted load for several future steps, and determine the candidate admission ratio based on the current state vector through a reinforcement learning control strategy. The control module is used to dynamically update the control strategy based on the candidate admission ratio to control the request traffic entering the object to be controlled, and to re-execute the step of building a service dependency graph for the target microservice cluster until the cold start traffic surge of the cloud service is alleviated.

[0015] A third aspect of this application provides a computer-readable storage medium storing one or more programs that can be executed by one or more processors to implement the steps in the control method for mitigating traffic surges during the cold start of cloud services as described above.

[0016] A fourth aspect of this application provides a terminal device, which includes: a processor and a memory; The memory stores a computer-readable program that can be executed by the processor; When the processor executes the computer-readable program, it implements the steps in any of the above-described methods for mitigating traffic surges during the cold start of cloud services.

[0017] Beneficial effects: 1. This application forms a closed loop between load forecasting, dynamic throttling, and safety constraints, no longer relying on static preheating curves or pure reactive feedback control. It can provide forward-looking access decisions before the cold start peak arrives, reducing the risk of overload in the early recovery stage from the source and improving the stability of cloud services in response to traffic fluctuations.

[0018] 2. This application constructs the target microservice cluster as a service dependency graph and configures historical monitoring sequences for the microservices that serve as graph nodes to treat the request arrival sequence as a time signal on the graph node. Then, it uses a time-series-graph attention prediction model to jointly model topological dependencies and time dynamics to determine the predicted load for several future steps. This can better capture the load superposition effect caused by dependency propagation between microservices, improve the accuracy of load prediction, provide a more reliable basis for subsequent traffic control decisions, and avoid the problem of insufficient load estimation due to uncaptured dependency propagation, which in turn leads to traffic control failure.

[0019] 3. This application combines reinforcement learning to generate candidate admission ratios and introduces capacity safety constraints to project and correct the candidate admission ratios. While ensuring adaptive optimization of the control strategy, it can limit traffic to the current service capacity through safety constraints, avoiding capacity penetration caused by controller exploration or prediction errors. It takes into account both the adaptability of decision-making and the safety of operation. It can dynamically adapt and adjust the traffic admission scale during the gradual recovery of service capacity, balancing recovery speed and operational stability, and avoiding the over-control or under-control problems that are easy to occur under traditional static solutions.

[0020] 4. This application generates candidate admission ratios through a joint temporal-graph attention prediction model and a reinforcement learning control strategy for control. Under the premise of ensuring service operation security, it improves the timeliness of feedback control to meet the real-time requirements of rapid feedback control during cold start recovery.

[0021] 5. Tests show that this application can maintain 0 safety violations under 30% over-predictive stress, with only a slight increase in peak P99 delay; in contrast, the existing MPC-Greedy (a predictive open-loop control scheme) is more prone to increased safety violations and performance degradation.

[0022] 6. Evaluation using FloodBench (i.e., microservice traffic flooding test) shows that this application outperforms static rate limiting, warm-up curves, Netflix-CL (a reactive concurrency limiting scheme based on latency and other hysteresis metrics), and MPC-Greedy in the overall FloodBench evaluation. Under a representative peak / steady-state load ratio of 50, this application achieves a recovery step count of 9.2±0.6, which is better than MPC-Greedy's 22.2±2.0, and has zero security violations.

[0023] 7. This application does not require reading the business request payload; it only needs to rely on observability metrics to run. Therefore, it is easy to integrate with existing cloud-native infrastructure and has strong engineering feasibility and promotional value. Attached Figure Description

[0024] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0025] Figure 1 A flowchart illustrating a method for mitigating traffic surges during the cold start of cloud services, provided in an embodiment of this application.

[0026] Figure 2 A closed-loop control flowchart of the regulation method for mitigating traffic surges during the cold start of cloud services provided in this application embodiment.

[0027] Figure 3 This is a network architecture diagram of a control method for mitigating traffic surges during the cold start of cloud services, provided in an embodiment of this application.

[0028] Figure 4 A schematic diagram of the control system for mitigating traffic surges during the cold start of cloud services, provided in an embodiment of this application.

[0029] Figure 5 A schematic block diagram of the terminal device provided in the embodiments of this application. Detailed Implementation

[0030] This application provides a method, apparatus, device, and medium for mitigating traffic surges during the cold start of cloud services. To make the objectives, technical solutions, and effects of this application clearer and more explicit, the following detailed description is provided with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and are not intended to limit the scope of this application.

[0031] Those skilled in the art will understand that, unless specifically stated otherwise, the singular forms “a,” “an,” “the,” and “the” used herein may also include the plural forms. It should be further understood that the term “comprising” as used in this application means the presence of the stated features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. It should be understood that when we say an element is “connected” or “coupled” to another element, it can be directly connected or coupled to the other element, or there may be intermediate elements. Furthermore, “connected” or “coupled” as used herein can include wireless connections or wireless coupling. The term “and / or” as used herein includes all or any units and all combinations of one or more associated listed items.

[0032] It will be understood by those skilled in the art that, unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains. It should also be understood that terms such as those defined in general dictionaries should be understood to have the same meaning as in the context of the prior art, and should not be interpreted in an idealized or overly formal sense unless specifically defined as herein.

[0033] It should be understood that the sequence number and size of each step in this embodiment do not imply the order of execution. The execution order of each process is determined by its function and internal logic, and should not constitute any limitation on the implementation process of this application embodiment.

[0034] Research has revealed that modern cloud services commonly employ microservice architecture, CI / CD continuous delivery, and automatic scaling mechanisms. When service instances undergo fault recovery, rolling version releases, or launch new instances to handle sudden traffic surges, they typically experience a cold start process from unready to ready. During the cold start phase, the service's external processing capacity has not fully recovered, but upstream gateways, call chains, and clients have accumulated a large number of pending requests, and client timeout retries further amplify the short-term load.

[0035] Therefore, the cold start phase is prone to traffic surges. This manifests as follows: the request arrival rate is significantly higher than the steady-state load level in the initial stage of service recovery. The first few service nodes to recover bear a disproportionate traffic surge, leading to request queuing backlog, worsening tail latency, and CPU and memory resource saturation. In severe cases, it can create a vicious cycle of "recovery followed by overload," even triggering a wider cascading failure. For interactive microservice systems, this problem directly impacts service availability, response quality, and fault recovery efficiency.

[0036] To address the traffic surge issue during the cold start phase, existing technologies generally employ static rate limiting, token bucket, or warm-up curve schemes. These schemes achieve traffic control by pre-setting fixed traffic limits or linear / segmented warm-up curves. While these schemes are logically simple, they cannot adaptively adjust to changes in request backlog, retry intensity, and current service capacity based on different cold start events. This can easily lead to problems such as overly strict rate limiting resulting in slow recovery or overly lenient rate limiting causing renewed overload.

[0037] In addition, existing technologies include reactive concurrency limiting schemes based on lag indicators such as latency (e.g., Netflix-CL), prediction-based open-loop control schemes (e.g., MPC-Greedy), and traditional time series forecasting schemes (e.g., ARIMA, LSTM, DeepAR). Among these, reactive concurrency limiting schemes based on lag indicators such as latency only trigger rate limiting actions after indicators like latency, rejection rate, or queue size deteriorate, resulting in inherent feedback lag. In scenarios with extremely rapid traffic surges, such as cold start traffic spikes, there is insufficient time to implement suppression measures before the peak arrives. While prediction-based open-loop control schemes introduce foresight, they often lack the ability to correct prediction errors in real time. If the prediction result is too high or too low, the control strategy is prone to continuously deviating from the true optimal state, leading to a decrease in recovery speed or breach of the safety margin. While traditional time series forecasting schemes can handle certain time series changes, they ignore the dependency topology between microservices and have difficulty depicting the propagation and amplification mechanism of requests in the call chain. On the other hand, when using only ordinary graph models such as GCN / GAT / STGCN, it may not be able to meet the requirements of low-latency inference and adaptation to irregular microservice dependency graphs.

[0038] To address the aforementioned issues, this application embodiment constructs a service dependency graph for the target microservice cluster. In this graph, nodes represent service nodes, edges represent call relationships between service nodes, and each node carries a historical monitoring sequence. Based on the service dependency graph, a time-series graph attention prediction model is used to determine the predicted load for several future steps. The observed state variables of the objects to be regulated in the target microservice cluster are obtained, and a current state vector is constructed based on these observed state variables and the predicted load for several future steps. A candidate admission ratio is determined based on the current state vector using a reinforcement learning control strategy. The regulation strategy is dynamically updated based on the candidate admission ratio to regulate the request traffic entering the objects to be regulated. The steps of constructing the service dependency graph for the target microservice cluster are re-executed until the cold start traffic surge of the cloud service is alleviated. This application forms a closed loop between load prediction, dynamic throttling, and security constraints, eliminating reliance on static warm-up curves or purely reactive feedback control. It can provide forward-looking admission decisions before the cold start surge arrives, reducing the overload risk in the initial recovery phase from the source and improving the stability of cloud services in responding to traffic fluctuations. Meanwhile, this application constructs the target microservice cluster as a service dependency graph and configures historical monitoring sequences for the microservices that serve as graph nodes to treat the request arrival sequence as a time signal on the graph node. Then, by jointly modeling topological dependencies and time dynamics through a time-series-graph attention prediction model, the load prediction for several future steps can be determined. This can better capture the load superposition effect caused by dependency propagation between microservices, improve the accuracy of load prediction, provide a more reliable basis for subsequent traffic control decisions, and avoid the problem of insufficient load estimation due to uncaptured dependency propagation, which in turn leads to traffic control failure.

[0039] The application content will be further explained below with reference to the accompanying drawings and the description of the embodiments.

[0040] This embodiment provides a method for mitigating traffic surges during the cold start of cloud services, such as... Figure 1 and Figure 2 As shown, the control method for mitigating traffic surges during cloud service cold starts specifically includes: S10. Build a service dependency graph for the target microservice cluster.

[0041] Specifically, the target microservice cluster includes multiple service nodes, each of which can be an independent microservice instance or a group of microservices within the target microservice cluster. A service dependency graph is used to reflect the topological dependencies of the target microservice cluster. The graph nodes represent service nodes, and historical monitoring sequences are used as node attributes. The edges of the service dependency graph represent the call relationships between service nodes, which can be call dependencies and / or traffic propagation relationships, etc. The historical monitoring sequences include several observable metrics at historical sampling times. These observable metrics include one or more of the following: request traffic, queue length, tail latency, CPU utilization, and memory utilization. Tail latency can be, for example, P99 tail latency.

[0042] In one embodiment, the process of constructing a service dependency graph can be as follows: Collect the call relationships between service nodes in the target microservice cluster, as well as the observable monitoring indicators of the service nodes at each sampling moment within a sliding time window of a set length; Each service node is set as a graph node in the service dependency graph. If there is a call relationship between two service nodes, a directed graph edge is added between the two corresponding graph nodes. The direction of the directed graph edge is consistent with the call direction. and the collected The observable monitoring indicators at each sampling time are arranged chronologically and used as the historical monitoring sequence carried by the corresponding graph nodes, thus completing the construction of the service dependency graph. This represents the number of sampling moments included in the sliding time window.

[0043] It's important to note that in practical applications, the service dependency graph can be reconstructed at each sliding time window, or it can be constructed in the first sliding time window. Then, for each subsequent sliding time window, the service dependency graph corresponding to the previous sliding time window is updated to obtain the service dependency graph for the current sliding time window, reducing the computational overhead of repeated construction. The update operation can be applied only to the historical monitoring sequence carried by each graph node; the topology of the service dependency graph will not change frequently. Edges between corresponding graph nodes will only be adjusted synchronously when the call relationships between service nodes change. This ensures that the service dependency graph always matches the current load observation state while keeping the additional computational overhead from graph structure updates to a low level, meeting the low-latency requirements of real-time control.

[0044] S20. Based on the service dependency graph, use a time-series-graph attention prediction model to determine the predicted load for several future steps.

[0045] Specifically, the temporal-graph attention prediction model is a trained neural network model used to predict the load for one or more future control cycles (i.e., time steps). The input to the temporal-graph attention prediction model is a service dependency graph, and the output is the predicted load for several future steps of the service nodes in the target microservice cluster. This predicted load for several future steps may include the predicted load for a single future time step. For example, the predicted load for a single future time step may include... The predicted load for a given time step; it can also include predicted load for multiple future time steps. For example, predicted load for several future time steps includes... Time step, Time step, Time step and Predicted load for a time step. This predicted load can be the request arrival rate for a time step (i.e., the control period).

[0046] The temporal-graph attention prediction model can adopt the TGAT model, or other prediction models that can utilize graph structure and temporal information simultaneously, such as Transformer with graph encoder, spatiotemporal graph convolutional network, graph-enhanced recurrent network or its distilled lightweight version.

[0047] In one embodiment, such as Figure 3 As shown, the step of determining the predicted load for several future steps based on the service dependency graph using a time-series graph attention prediction model specifically includes: S21. Encode the historical monitoring sequences carried by the graph nodes in the service dependency graph to obtain the local time features corresponding to the graph nodes. S22. Determine the neighbor set of the graph node based on the service dependency graph, and aggregate the local temporal features corresponding to the neighbor nodes in the neighbor set through a multi-head attention mechanism to obtain the target node representation of the graph node; S23. Determine the global representation corresponding to the service dependency graph, and predict the future load of the microservice based on the global representation and the target node representation of the graph nodes.

[0048] In step S21, the local temporal features are obtained by feature encoding the historical monitoring sequences of graph nodes, reflecting the load change trend of the service node within a continuous time window. The feature encoding can utilize the temporal feature extractor in the temporal-graph attention prediction model, which extracts the local temporal features corresponding to the graph nodes through operations such as one-dimensional convolution, nonlinear activation, and positional encoding. For example, ... Figure 3 As shown, generated according to the nodes in the diagram. Historical feature matrix This represents the vector dimension corresponding to the observability index. Then, Conv1d+GELU is used to process the historical feature matrix to extract the local temporal features of the graph nodes.

[0049] In step S22, the neighbor set includes graph nodes in the service dependency graph that have an incoming edge, outgoing edge, or preset hop count reachability relationship with the graph node. That is, each neighbor node in the neighbor set has an incoming edge, outgoing edge, or preset hop count reachability relationship with the graph node. The target node is represented by fusing the local temporal features of the graph node with the local temporal features of all neighbor nodes in its corresponding neighbor set using multi-head attention. This results in the target node representation containing both the temporal features of the current node itself and the feature information of adjacent nodes in the dependency topology, thereby characterizing the propagation effect of the request in the call chain.

[0050] For example, the step of aggregating the local temporal features corresponding to neighbor nodes in the neighbor set through a multi-head attention mechanism to obtain the target node representation of the graph node specifically includes: A query vector is constructed using the local temporal features corresponding to the graph nodes, and a value vector and a key vector are constructed using the local temporal features corresponding to the neighbor nodes in the neighbor set. Obtain the relative structural distance between the graph node and its neighboring nodes, and determine the relative structural encoding between the graph node and its neighboring nodes based on the relative structural distance; An initial attention score is determined based on the query vector and the key vector, and the initial attention score is corrected using the relative structure encoding to obtain the final attention score. The value vectors corresponding to neighboring nodes are fused using the attention scores to obtain the target node representation of the graph node.

[0051] Specifically, the query vector is constructed based on the local temporal features corresponding to the graph node, while the value vector and key vector are constructed based on the local temporal features corresponding to the neighboring nodes. This allows the graph node to query key adjacency information that affects its own load changes from the neighboring nodes, achieving targeted feature fusion and avoiding irrelevant topological information from interfering with the prediction results.

[0052] Furthermore, relative structure encoding does not rely on precise request-level timestamps, but is determined using the relative structure distance between service nodes. This relative structure distance can be the topological distance or index distance between service nodes, etc. The encoding method for relative structure encoding can be: , in, Represents graph nodes Its neighboring nodes The relative structure encoding between them This represents the encoding dimension of the relative structure encoding. Represents graph nodes Its neighboring nodes The relative structural distance between them Indicates the encoded dimension index. This represents the scaling constant for sine and cosine coding.

[0053] Furthermore, after obtaining the relative structure encoding, it is integrated into the process of determining the initial attention score based on the query vector and key vector. This refines the initial attention score, enabling the model to better perceive the positional relationship between two nodes in the dependency topology, distinguishing between directly dependent nodes and distant upstream nodes. This further improves the rationality of aggregation and accurately captures the differences in the impact of different dependency levels on load propagation. Specifically, when refining the initial attention score using the relative structure encoding, a linear layer can be used to map the relative structure encoding to the refined attention score. Then, the refined attention score and the initial attention score are fused to obtain the final attention score. For example, this can be done by summing the refined and initial attention scores, or by calculating their average.

[0054] Furthermore, after obtaining the attention score, the attention score can be normalized using the softmax function. Then, the normalized attention score is weighted and summed with the value vectors of the corresponding neighboring nodes to obtain the target node representation of the graph node. This target node representation includes the load time-series characteristics of the current service node itself, as well as the feature information of adjacent nodes at different positions in the service dependency topology. It can accurately characterize the propagation characteristics of request traffic in the entire microservice call chain, providing sufficient feature support for subsequent load prediction.

[0055] The embodiments of this application first extract the local temporal features of each graph node, and then aggregate the neighbor topology information through the graph attention mechanism. This can take into account both the load changes in the time dimension and the dependency propagation in the spatial dimension, avoiding the problem of traditional single-node prediction ignoring the topological superposition effect. At the same time, the lightweight graph attention structure can also meet the requirements of low-latency inference.

[0056] In step S23, after obtaining the target node representation of the graph node, the global representation corresponding to the service dependency graph is obtained, for example, by performing global pooling on the service dependency graph. Then, the target node representation and the global representation are concatenated to obtain the concatenated feature. Finally, the concatenated feature is input into the prediction head, and the prediction head outputs the predicted load of the graph node for several future steps to obtain the predicted load of the service node corresponding to the graph node for several future steps.

[0057] It should be noted that, when using the temporal-graph attention prediction model for prediction, this application uses the relative structural distance between graph nodes and their neighboring nodes to correct the attention score, enabling the model to distinguish between directly dependent nodes and distant upstream nodes, thereby improving prediction accuracy. However, in other embodiments, the attention score may not be corrected using relative structural distance, depending on actual needs.

[0058] S30. Obtain the observed state quantity of the object to be controlled in the target microservice cluster, and construct the current state vector based on the observed state quantity and the predicted load for the next few steps.

[0059] Specifically, the object to be regulated can be a service node in the target microservice cluster, or a service group in the target microservice cluster. The target service group can be multiple microservices on the same business call chain, a group of instances within the same cluster, or a key service subgraph selected according to call frequency.

[0060] The observed state variables are the actual operational status indicators collected at the current time step of the object to be regulated. Specifically, they may include one or more of the following: current observed load (i.e., the actual load at the previous time step), backlog queue, tail delay, available capacity, and safety margin. After obtaining the observed state variables, they are concatenated and integrated with the predicted load of the object to be regulated for several future steps to obtain the current state vector used for subsequent regulation decisions. This current state vector integrates the current actual operating status of the object to be regulated and the upcoming load change trend, providing a complete state basis for subsequent scaling up and down regulation decisions.

[0061] For example, a service dependency graph includes gateway A, service B, and service C, and the length of the sliding time window... The sampling period is 1 second. For service B, its current state vector is: , in, Indicates time The state vector, Indicates time The observation load, Indicates time The backlog queue This indicates that the cluster has partially recovered at time [time]. Available capacity Indicates time Tail delay; Indicates time Safety margin; These represent the predicted load for the next four time steps (i.e., control periods).

[0062] S40. Based on the current state vector, determine the candidate admission ratio through a reinforcement learning control strategy.

[0063] Specifically, the reinforcement learning control strategy is a pre-trained deep neural network strategy. It uses the current state vector as the environment state of the reinforcement learning control strategy, and determines the action corresponding to this environment state to obtain candidate admission ratios. These candidate admission ratios are used to determine the share of traffic allowed to enter the controlled object through the load balancer in the current control cycle. This reinforcement learning controller can employ DDPG, TD3, SAC, or other continuous action space policy networks. During training, entropy regularization terms can be added to enhance strategy exploration capabilities and generalization ability to different flood peak patterns. During runtime, a deterministic reinforcement learning control strategy is used for fast inference. Before inputting the current state vector into the reinforcement learning control strategy, the current state vector can be normalized.

[0064] In one embodiment, the reward of the reinforcement learning control policy is the negative of a multi-objective cost function, which uses the admission ratio as the control decision quantity, integrates tail latency SLO default, cluster idle capacity, and recovery time as continuous penalty terms in the time domain, and introduces a queue clearing terminal incentive term.

[0065] For example, the multi-objective cost function can be: , , in, Indicates time The admission ratio Describe the objective function. Indicates the recovery time when the backlog queue is cleared. Indicates time The backlog queue This indicates an indicator function that takes the value 1 when the condition is true and 0 when the condition is false. The SLO threshold representing the tail delay. Indicates time Tail delay, This indicates that the cluster has partially recovered at time [time]. Available capacity Indicates time The observation load, Indicates the weight of SLO default duration; Indicates the penalty weight per unit recovery time; Indicates the weight of free capacity; This indicates the termination reward weight after the queue is emptied.

[0066] It should be noted that in practical applications, MPC, PPO, A3C, and combinations of rule controllers and learning controllers can also be used to output the candidate admission ratio.

[0067] S50. Dynamically update the control strategy based on the candidate admission ratio to control the request traffic entering the object to be controlled.

[0068] Specifically, dynamic updates to control strategies can be implemented through control modules, such as gateways or load balancers. When dealing with large-scale service graphs, a sliding window can be used to count call frequencies and select the k most frequently accessed services to form a fixed-dimensional subgraph, balancing inference latency and critical dependency coverage. For ultra-large-scale clusters, a hierarchical control architecture can be adopted, with local controllers and upper-layer coordinators working together.

[0069] Furthermore, when dynamically updating the control strategy based on the candidate admission ratio, a capacity safety constraint can be constructed using a control barrier function, and this capacity safety constraint can be used to adjust the candidate admission ratio. Therefore, the dynamic update control strategy based on the candidate admission ratio specifically includes: Construct capacity safety constraints; The candidate admission ratio is projected and corrected under the capacity safety constraint to obtain the safe admission ratio. The control strategy is dynamically updated based on the safety access ratio.

[0070] Specifically, the capacity safety constraint is used to ensure that the current inbound traffic of the object to be regulated does not exceed its available processing capacity. The capacity safety constraint is defined as the product of the current predicted load and the safe inbound ratio being less than or equal to the difference between the effective processing capacity and the safety margin. In other words, the capacity safety constraint is: , in, This represents the predicted load at the current moment. Indicates the percentage of people who are granted safe access. This indicates that the cluster has partially recovered at time [time]. Available capacity Indicates time Safety margin.

[0071] Then, under the capacity safety constraint, the candidate admission ratio is projected and corrected, wherein the projection correction can be a closed projection, which can be expressed as: , in, Indicates the candidate admission ratio. This represents the function that takes the minimum value.

[0072] It should be noted that in practical applications, in addition to closed projection, control obstacle functions, quadratic programming projection, or equivalent closed projection can also be used to provide progressive safety constraints on any candidate admission ratio without changing the overall structure of the control target.

[0073] Furthermore, after obtaining the security access ratio, the security access ratio can be distributed to the API gateway, service mesh, or load balancer control plane to dynamically update the control policy to regulate the request traffic entering the target microservice cluster. The control measurement can include adjusting rate limits, concurrency limits, or load distribution rules.

[0074] S60. Re-execute the steps to build the service dependency graph for the target microservice cluster until the surge in traffic during the cold start of the cloud service is alleviated.

[0075] Specifically, after updating the control strategy, it is possible to detect whether the backlog queue has fallen below the target threshold, whether the tail latency has recovered to the SLO target range, or whether the cluster has recovered to steady-state capacity, in order to determine whether the cold start traffic surge has been alleviated. If it has not been alleviated, the step of building the service dependency graph for the target microservice cluster is re-executed; if it has been alleviated, the iterative control is completed.

[0076] It should be noted that the method for mitigating traffic surges during cloud service cold starts provided in this application can be used not only for microservice cold start recovery, but also for scenarios with "load surges in the initial recovery phase," such as function compute / Serverless cold starts, database replica recovery, cache cluster reconstruction, batch task recovery, edge node reconnection, and rapid scaling.

[0077] To further illustrate the method for mitigating traffic surges during the cold start of cloud services provided in this application, several application scenario embodiments are given below.

[0078] Application Scenario Example 1: Fault Recovery Scenarios When a microservice cluster goes offline, either entirely or partially, due to node failure, data center instability, or network partitioning, a backlog of requests awaiting forwarding will accumulate in the upstream load balancer, and additional retry flows will form on the client side due to timeout retries. When the cluster begins to recover, this embodiment first uses the monitoring bus to collect the request volume per second, queue length, CPU utilization, and P99 tail latency for each service, and then constructs a service dependency graph based on the service call chain.

[0079] The observability metrics for the most recent T time steps are read. These metrics include the number of requests, queue length, P99 tail latency, and CPU utilization. Each service node in the service dependency graph forms a 10×4 node-level historical monitoring sequence. First, Conv1d and GELU activation are performed on the historical monitoring sequences of each service node to extract local temporal features such as short-term rises, falls, and retry fluctuations, and positional encoding is added. Second, one-hop or multi-hop neighbor nodes are determined based on the service call relationship, the relative structure encoding between the graph node and its neighbor nodes is calculated, and neighbor node aggregation is completed through multi-head graph attention. Then, two layers of TGAT are stacked to obtain the target node representation. Finally, the target node representation is concatenated with the average pooling result of the target node representations of all graph nodes in the service dependency graph, and the predicted load for the next one or more control cycles is output through a fully connected prediction header.

[0080] Receive predicted load, backlog queue, current available capacity, and tail latency metrics for the same target service or target service group, and output candidate admission ratios using the trained DDPG continuous action policy network. .when When capacity safety constraints are not met, quadratic programming using control barrier functions can be used to resolve the issue. Projection is to meet safety constraints. For a single linear capacity constraint, closed projection is used. It is then distributed to Envoy or other traffic proxies in real time.

[0081] In the application scenario implementation, the system cyclically performs tasks such as collecting metrics, predicting load, generating candidate admission ratios, security projection, and issuing rate limiting parameters at a fixed control cycle. As more service instances recover, available capacity... As the system continues to improve, the controller can increase the admission ratio accordingly, accelerating the clearing of the backlog queue until the queue recedes and latency returns to normal. This application scenario is suitable for situations where the initial surge in service demand is sharp and service needs to be restored as quickly as possible without further breaking down capacity.

[0082] Application Scenario Example 2: For Rolling Upgrade Scenarios During CI / CD rolling upgrades, older version instances are gradually decommissioned, and new version instances are gradually made ready. Since the processing capacity of new instances is limited in the initial startup phase, if traffic is still handled according to the conventional load balancing weights, the first batch of new instances is prone to concentrated overload. This embodiment treats the services under upgrade as protected objects, continuously collecting request volume, initialization completion status, queue length, and latency data during the startup phase, and generating a dynamic subgraph based on upstream and downstream service dependencies.

[0083] In each upgrade time slice, the system predicts the short-term traffic that the new instance can withstand and outputs an admission ratio or weight adjustment value adapted to the current upgrade progress. A security layer ensures that the traffic allocated to the new instance does not exceed its phased available capacity. Compared to a fixed warm-up curve, this embodiment can adjust the diversion rate in real time based on actual retry pressure, upstream and downstream linkage strength, and instance recovery speed, reducing error rate and latency peaks within the upgrade window.

[0084] Application Scenario Example 3: For Rapid Capacity Expansion Scenarios When a business encounters a sudden surge in traffic driven by a sudden event, horizontal scaling containers create new instances. However, there can be a delay of seconds or even minutes between the creation and readiness of a new instance. In this case, the first few instances to recover will absorb most of the peak traffic. This embodiment continuously predicts the short-term arrival rate and the capacity ramp-up curve after scaling is triggered, and dynamically adjusts the admission ratio of new and old instances or the gateway rate limiting threshold based on the prediction results.

[0085] In the early stages of expansion, conservative admission can be maintained and new instances should be prioritized for protection. As capacity is gradually released, the admission ratio can be increased to quickly recover backlogged traffic. Introducing security projection can prevent new instances from being downed again in a short period of time due to misjudging expansion capacity.

[0086] Based on the above-described control method for mitigating traffic surges during cloud service cold starts, this embodiment provides a control device for mitigating traffic surges during cloud service cold starts, such as... Figure 4 As shown, the control device specifically includes: The load prediction module is used to construct a service dependency graph for the target microservice cluster. Based on the service dependency graph, a time-series-graph attention prediction model is used to determine the predicted load for several future steps. In this module, graph nodes represent service nodes, edges represent the call relationships between service nodes, and each graph node carries a historical monitoring sequence. The control module is used to acquire the observed state quantity of the object to be controlled in the target microservice cluster, construct the current state vector based on the observed state quantity and the predicted load for several future steps, and determine the candidate admission ratio based on the current state vector through a reinforcement learning control strategy. The control module is used to dynamically update the control strategy based on the candidate admission ratio to control the request traffic entering the object to be controlled, and to re-execute the step of building a service dependency graph for the target microservice cluster until the cold start traffic surge of the cloud service is alleviated.

[0087] The system is deployed on the Kubernetes control plane. The load prediction module and the control module run as independent services. Metrics are obtained through OpenTelemetry, Prometheus or service mesh telemetry, and execution policies are pushed through Envoy xDS, gateway API or rate limiting interface.

[0088] Based on the above-described method for mitigating traffic surges during the cold start of cloud services, this embodiment provides a computer-readable storage medium storing one or more programs that can be executed by one or more processors to implement the steps in the method for mitigating traffic surges during the cold start of cloud services as described in the above embodiment.

[0089] Based on the above-mentioned control methods for mitigating traffic surges during the cold start of cloud services, this application also provides a terminal device, such as... Figure 5 As shown, it includes at least one processor 20; a display screen 21; and a memory 22, and may also include a communications interface 23 and a bus 24. The processor 20, display screen 21, memory 22, and communications interface 23 can communicate with each other via the bus 24. The display screen 21 is configured to display a preset user guide interface in the initial setup mode. The communications interface 23 can transmit information. The processor 20 can invoke logical instructions in the memory 22 to execute the methods described in the above embodiments.

[0090] Furthermore, the logical instructions in the aforementioned memory 22 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium.

[0091] The memory 22, as a computer-readable storage medium, can be configured to store software programs, computer-executable programs, such as program instructions or modules corresponding to the methods in the embodiments of this disclosure. The processor 20 executes functional applications and data processing by running the software programs, instructions, or modules stored in the memory 22, thereby implementing the methods in the above embodiments.

[0092] The memory 22 may include a program storage area and a data storage area. The program storage area may store the operating system and application programs required for at least one function; the data storage area may store data created based on the use of the terminal device. Furthermore, the memory 22 may include high-speed random access memory (RAM) and non-volatile memory. Examples include various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks, as well as transient storage media.

[0093] Furthermore, the specific process of loading and executing multiple instruction processors in the aforementioned storage medium and terminal device has been described in detail in the above method, and will not be repeated here.

[0094] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A method for mitigating traffic surges during the cold start of cloud services, characterized in that, The aforementioned regulation methods specifically include: A service dependency graph is constructed for the target microservice cluster, wherein the graph nodes represent service nodes, the edges represent the call relationships between service nodes, and each graph node carries a historical monitoring sequence. Based on the service dependency graph, a time-series graph attention prediction model is used to determine the predicted load for several future steps. Obtain the observed state values ​​of the objects to be controlled in the target microservice cluster, and construct the current state vector based on the observed state values ​​and the predicted load for the next few steps. Based on the current state vector, the candidate admission ratio is determined through a reinforcement learning control strategy. The control strategy is dynamically updated based on the candidate admission ratio to control the request traffic entering the object to be controlled. Re-execute the steps to build the service dependency graph for the target microservice cluster until the surge in traffic during the cold start of the cloud services is alleviated.

2. The control method for mitigating traffic surges during the cold start of cloud services according to claim 1, characterized in that, The historical monitoring sequence includes several observability metrics for historical sampling times, including one or more of the following: request traffic, queue length, tail latency, CPU utilization, and memory utilization.

3. The control method for mitigating traffic surges during the cold start of cloud services according to claim 1, characterized in that, The observation status variables include one or more of the following: current observation load, backlog queue, tail delay, current available capacity, and safety margin.

4. The control method for mitigating traffic surges during the cold start of cloud services according to claim 1 or 2, characterized in that, The step of determining the predicted load for several future steps based on the service dependency graph using a time-series graph attention prediction model specifically includes: The historical monitoring sequences carried by the graph nodes in the service dependency graph are feature-encoded to obtain the local temporal features corresponding to the graph nodes. The neighbor set of the graph node is determined based on the service dependency graph, and the local temporal features corresponding to the neighbor nodes in the neighbor set are aggregated through a multi-head attention mechanism to obtain the target node representation of the graph node. Determine the global representation corresponding to the service dependency graph, and predict the future load of the microservice based on the global representation and the target node representation of the graph nodes.

5. The control method for mitigating traffic surges during the cold start of cloud services according to claim 4, characterized in that, The step of aggregating the local temporal features corresponding to neighbor nodes in the neighbor set through a multi-head attention mechanism to obtain the target node representation of the graph node specifically includes: A query vector is constructed using the local temporal features corresponding to the graph nodes, and a value vector and a key vector are constructed using the local temporal features corresponding to the neighbor nodes in the neighbor set. Obtain the relative structural distance between the graph node and its neighboring nodes, and determine the relative structural encoding between the graph node and its neighboring nodes based on the relative structural distance; An initial attention score is determined based on the query vector and the key vector, and the initial attention score is corrected using the relative structure encoding to obtain the final attention score. The value vectors corresponding to neighboring nodes are fused using the attention scores to obtain the target node representation of the graph node.

6. The control method for mitigating traffic surges during the cold start of cloud services according to claim 1, characterized in that, The dynamic update and adjustment strategy based on the candidate admission ratio specifically includes: Construct a capacity safety constraint, wherein the capacity safety constraint is that the product of the currently observed loads is less than or equal to the difference between the available capacity and the safety margin; The candidate admission ratio is projected and corrected under the capacity safety constraint to obtain the safe admission ratio. The control strategy is dynamically updated based on the safety access ratio.

7. The method for mitigating traffic surges during the cold start of cloud services according to claim 1, characterized in that, The reward of the reinforcement learning control strategy is the negative of the multi-objective cost function, which uses the admission ratio as the control decision quantity, integrates the tail delay SLO default, cluster idle capacity and recovery time as continuous penalty terms in the time domain, and introduces a queue clearing terminal incentive term.

8. A control device for mitigating traffic surges during the cold start of cloud services, characterized in that, The aforementioned control device specifically includes: The load prediction module is used to build a service dependency graph for the target microservice cluster. Based on the service dependency graph, a time-series graph attention prediction model is used to determine the predicted load for several future steps. The graph nodes of the service dependency graph represent service nodes, the edges represent the call relationship between service nodes, and each graph node carries a historical monitoring sequence. The control module is used to acquire the observed state quantity of the object to be controlled in the target microservice cluster, construct the current state vector based on the observed state quantity and the predicted load for several future steps, and determine the candidate admission ratio based on the current state vector through a reinforcement learning control strategy. The control module is used to dynamically update the control strategy based on the candidate admission ratio to control the request traffic entering the object to be controlled, and to re-execute the step of building a service dependency graph for the target microservice cluster until the cold start traffic surge of the cloud service is alleviated.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores one or more programs, which can be executed by one or more processors to implement the steps in the control method for mitigating traffic surges during cold starts of cloud services as described in any one of claims 1-7.

10. A terminal device, characterized in that, include: Processor and memory; The memory stores a computer-readable program that can be executed by the processor; When the processor executes the computer-readable program, it implements the steps of the control method for mitigating traffic surges during the cold start of cloud services as described in any one of claims 1-7.

Images