A flow regulation method, device, equipment, storage medium and program product

Abstract

This application discloses a flow control method, apparatus, device, storage medium, and program product. The method includes: acquiring a sequence of target performance indicators and determining the load type of each target performance indicator in the sequence; selecting a target flow prediction model from multiple flow prediction models based on the load type of each target performance indicator; wherein the load type of the target flow prediction model is the same as the load type of the corresponding target performance indicator; performing flow prediction on each target performance indicator based on the target flow prediction model to obtain a sequence of predicted performance indicator values; determining a garbage collection flow limit value sequence based on the predicted performance indicator value sequence and a preset performance loss value; and dynamically controlling the garbage collection task based on the garbage collection flow limit value sequence and a preset response mechanism.

Description

Technical Field

[0001] This application relates to the fields of cloud computing and artificial intelligence, and in particular to a traffic control method, apparatus, device, storage medium, and program product. Background Technology

[0002] With the rapid development of big data and cloud computing technologies, distributed storage systems are widely used in scenarios such as log processing, real-time analysis, and distributed databases. Among these, append-write storage architecture has become a core application technology due to its high-throughput sequential write characteristics. These systems rely on background garbage collection tasks to periodically reclaim old / expired data from the underlying storage medium to free up storage space and maintain the storage efficiency of the append-write system. However, related technologies still face problems such as control lag and limitations of control models when implementing dynamic garbage collection traffic control in append-write storage systems, and lack elastic response and efficient recovery mechanisms for control failures. Summary of the Invention

[0003] To address the aforementioned technical problems, embodiments of this application provide a flow control method, apparatus, device, storage medium, and program product.

[0004] The flow control method provided in this application includes: Obtain the target performance index sequence and determine the load type of each target performance index in the target performance index sequence; Based on the load type of each target performance indicator, a target traffic prediction model is selected sequentially from multiple traffic prediction models for multiple load types; wherein, the load type of the target traffic prediction model is the same as the load type of the corresponding target performance indicator. Based on the target traffic prediction model, traffic prediction is performed on each target performance indicator to obtain a sequence of predicted performance indicator values. Based on the predicted value sequence of the performance indicators and the preset performance loss value, a sequence of waste recycling flow restriction values ​​is determined; The waste recycling task is dynamically regulated based on the waste recycling flow limit value sequence and the preset response mechanism.

[0005] The flow control device provided in this application includes: The acquisition unit is used to acquire the target performance index sequence. A determining unit is used to determine the load type of each target performance indicator in the target performance indicator sequence; The selection unit is used to select a target traffic prediction model from multiple traffic prediction models for multiple load types based on the load type of each target performance indicator; wherein the load type of the target traffic prediction model is the same as the load type of the corresponding target performance indicator. The prediction unit is used to predict the traffic flow of each target performance indicator based on the target traffic flow prediction model, and obtain a sequence of predicted performance indicator values. The determining unit is further configured to determine a sequence of waste recycling flow restriction values ​​based on the predicted performance index value sequence and the preset performance loss value; The control unit is used to dynamically control the waste recycling task based on the waste recycling flow limit value sequence and the preset response mechanism.

[0006] The processing device provided in this application includes a processor and a memory. The memory is used to store computer programs, and the processor is used to call and run the computer programs stored in the memory to execute any of the above-described flow control methods.

[0007] The computer-readable storage medium provided in this application embodiment is used to store a computer program that causes a computer to execute any of the above-described flow control methods.

[0008] The computer program product provided in this application includes computer program instructions that cause a computer to execute any of the above-described flow control methods.

[0009] In the technical solution of this application embodiment, a sequence of target performance indicators is obtained, and the load type of each target performance indicator in the sequence is determined; based on the load type of each target performance indicator, a target traffic prediction model is selected sequentially from multiple traffic prediction models for multiple load types; wherein, the load type of the target traffic prediction model is the same as the load type of the corresponding target performance indicator; traffic prediction is performed on each target performance indicator based on the target traffic prediction model to obtain a sequence of predicted performance indicator values; based on the sequence of predicted performance indicator values ​​and a preset performance loss value, a sequence of garbage collection flow limit values ​​is determined; and the garbage collection task is dynamically regulated based on the sequence of garbage collection flow limit values ​​and a preset response mechanism. Thus, by employing dedicated traffic prediction models for different load types, and selecting a target traffic prediction model with the same load type as each target performance indicator to predict traffic for each target performance indicator, it is possible to leverage the different impacts of garbage collection on different load types (sequential read, sequential write, random read, random write) to achieve high-precision short-term traffic prediction for different load types. This solves the problem of inaccurate prediction caused by traditional models ignoring I / O mode differences. At the same time, it dynamically outputs garbage collection rate limits based on the predicted performance indicator values ​​and the user-defined acceptable preset performance loss values, and introduces a preset response mechanism to dynamically regulate garbage collection tasks. This enables elastic response and safe recovery in the event of regulation failure, and enhances the lack of layered self-healing capabilities when regulation fails, significantly improving the stability and reliability of the system, thereby providing strong support for the efficient operation of large-scale distributed systems. Attached Figure Description

[0010] Figure 1 This is a schematic flowchart of a flow control method provided in an embodiment of this application; Figure 2 This is a schematic diagram of an append write storage system based on traffic prediction and dynamic control provided in an embodiment of this application; Figure 3 This is a flowchart illustrating a load type-based prediction method provided in an embodiment of this application; Figure 4 This is a flowchart illustrating the online fine-tuning and optimization mechanism provided in the embodiments of this application; Figure 5 This is a flowchart illustrating the dynamic garbage collection control method based on performance loss provided in an embodiment of this application. Figure 6 This is a flowchart illustrating the three-level emergency response mechanism provided in the embodiments of this application; Figure 7 This is a schematic diagram of the structure of a flow control device provided in an embodiment of this application; Figure 8 This is a schematic diagram of the processing device provided in the embodiments of this application. Detailed Implementation

[0011] The technical solutions of the embodiments of this application will now be described with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0012] To facilitate understanding of the technical solutions of the embodiments of this application, the relevant technologies of the embodiments of this application are described below. The following relevant technologies are optional solutions and can be combined with the technical solutions of the embodiments of this application in any way, and they all fall within the protection scope of the embodiments of this application.

[0013] With the rapid development of big data and cloud computing technologies, distributed storage systems are widely used in scenarios such as log processing, real-time analysis, and distributed databases. Among these, append-write storage architecture has become a core application technology due to its high-throughput sequential write capability. However, such systems rely on background garbage collection tasks to periodically reclaim old / invalid data from the underlying storage medium to free up storage space and maintain the storage efficiency of the append-write system. During this process, garbage collection needs to read valid data from the storage medium and append it to a new storage location, and reclaim storage areas without valid data, thus generating a large amount of background read / write traffic. Garbage collection queue congestion leads to performance degradation, and background traffic and client request contention can also cause blocking issues. Both of these problems contribute to performance degradation, so it is necessary to further design the garbage collection mechanism to improve garbage collection efficiency while ensuring the quality of front-end business processes.

[0014] Waste recycling mechanisms in related technologies can be divided into three categories, specifically including: (1) Current storage systems typically use reactive garbage collection triggering mechanisms, which monitor the ratio or quantity of valid and invalid data in the backend hard drives and initiate garbage collection tasks when the amount of garbage data exceeds a preset threshold. This approach has a single monitoring metric and does not comprehensively consider more comprehensive performance parameters such as front-end business request latency and memory performance metrics, making it difficult to trigger garbage collection tasks at the appropriate time. Secondly, traditional garbage collection tasks use static traffic control technology, meaning that once a garbage collection task is triggered, it collects data at a preset fixed traffic threshold. This approach has defects in resource allocation and task scheduling. For example, when front-end user traffic is high, it will preempt system resources, leading to a serious decline in user service capabilities, while when front-end traffic is low, it cannot fully utilize idle system resources to improve data collection efficiency. Therefore, the traditional static garbage collection traffic control method has become a bottleneck in improving the performance and storage efficiency of append-write storage systems.

[0015] (2) Current dynamic traffic control technology mainly targets front-end business traffic. It analyzes the monitoring data of each node and, based on preset rules, dynamically determines at the ingress gateway whether it is necessary to adjust the front-end business traffic scheduling ratio, thereby achieving a balanced allocation of cloud resources. However, this method is not suitable for scheduling scenarios where front-end / back-end traffic coexists. First, the ingress gateway is not responsible for monitoring back-end garbage collection traffic, making it difficult to perform system-level overall traffic control. Second, front-end business traffic and garbage collection traffic exhibit different characteristics: front-end business traffic usually shows predictable periodic fluctuations, while garbage collection traffic has non-uniform bursts. For example, front-end traffic has peak user access periods, while garbage collection traffic may suddenly burst due to rapid data filling or the proportion of invalid data exceeding a threshold.

[0016] (3) Deep learning-based traffic scheduling methods possess powerful pattern recognition and prediction capabilities, enabling them to learn complex system states and traffic patterns from historical and real-time data. Through learning, the model can more accurately predict the fluctuation cycle of front-end business traffic and the sudden timing of garbage collection demands, and dynamically and collaboratively adjust the weight allocation and rate limits of front-end business traffic and back-end garbage collection traffic accordingly. Compared to static strategies with fixed weights, deep learning models can adaptively increase the priority and resource quota of garbage collection tasks when the system is relatively idle or facing a surge in space pressure, based on real-time system load and predicted future states, thus avoiding the risk of long-term suppression leading to space exhaustion. However, the effectiveness of deep learning-based traffic scheduling methods is highly dependent on the quality of the dataset and the level of feature engineering. Therefore, obtaining more accurate data and designing more effective features are key to improving the performance of such methods.

[0017] In summary, the relevant technologies still face the following key technical shortcomings when implementing dynamic garbage collection traffic control in append-write storage systems: (1) Lag in regulation. Related technologies (whether static regulation or dynamic weight adjustment based on real-time monitoring) are essentially a reactive regulation mechanism. They rely on monitoring the current system status (such as CPU, memory, network, disk I / O utilization or front-end traffic pressure) and only perform flow limiting adjustments after resource contention or performance degradation has already occurred.

[0018] When user traffic surges suddenly, the lag in control measures forces the system to experience a brief but significant drop in service quality (such as increased latency) before the traffic from background garbage collection tasks can be effectively reduced, thus mitigating the impact on front-end services. Conversely, when user traffic drops sharply and system resources become idle, garbage collection traffic cannot be increased in a timely and sufficient manner to accelerate space reclamation, missing the window of opportunity to improve system efficiency. Overall, the current reactive control methods struggle to achieve optimal dynamic resource balance under rapidly changing load environments.

[0019] (2) Limitations of deep learning-based traffic control models. Although related methods for traffic control using deep learning attempt to introduce intelligent prediction, their model design has significant shortcomings. Current methods mainly focus on system resource indicators (such as overall CPU, bandwidth utilization, or background garbage collection traffic usage), but neglect the impact of garbage collection on front-end traffic under different I / O modes.

[0020] Specifically, the data read / write behavior involved in garbage collection tasks is not homogeneous. Existing research indicates that garbage collection has different impacts on front-end loads under different I / O modes. However, related deep learning models have failed to incorporate the I / O mode (sequential / random read, sequential / random write) of front-end traffic as a key feature input, resulting in their predicted traffic control strategies failing to accurately match the actual resource requirements and performance impact characteristics of garbage collection tasks. This lack of feature dimension makes the model's output control recommendations potentially too aggressive (insufficiently restricting random read / write operations, exacerbating system jitter) or too conservative (overly restricting sequential read / write operations, wasting resources), making it difficult to achieve fine-grained, optimal control based on the inherent characteristics of the task.

[0021] (3) Lack of flexible response and efficient recovery mechanism for control failure. Related technologies lack effective anomaly handling mechanisms when preset control strategies (such as rule-based adjustments or model-predicted flow limits) fail to alleviate resource contention or system performance deterioration in a timely manner. They typically only maintain the original strategy or implement simple, potentially overly aggressive, global suppression (such as completely suspending garbage collection). This not only affects space reclamation efficiency but may also lead to system unavailability due to space exhaustion in extreme cases. Furthermore, when the system recovers from an abnormal state (such as a sudden high load), related technologies lack efficient and smooth recovery mechanisms. They often require a long period of gradual, tentative relaxation of restrictions or rely on fixed recovery strategies, failing to quickly restore garbage collection traffic to the optimal level matching the current idle resources, thus delaying the recovery of system storage efficiency.

[0022] To address the aforementioned technical issues, this application proposes a traffic control method, specifically a dynamic garbage collection traffic control method for append-only storage systems. This method introduces an effective prediction mechanism to anticipate user traffic trends and changes in system resource demands. It incorporates the different impacts of garbage collection tasks on various front-end I / O modes (sequential read / write, random read / write) as core features into the prediction model to construct prediction models specific to different loads. Combined with a time-decay weight fine-tuning mechanism triggered by a dynamic error threshold, this enables high-precision short-term traffic prediction, solving the problem of inaccurate predictions caused by traditional models ignoring I / O mode differences and preventing long-term prediction accuracy degradation. Furthermore, by pre-sampling Input / Output Operations Per Second (IOPS) data for four load types (sequential / random read / write), a performance loss model is constructed. The garbage collection rate limit is dynamically set based on real-time load predictions and acceptable loss. Simultaneously, a three-tiered emergency response mechanism is introduced: Tier 1 rapidly reduces flow control and releases resources in response to sudden failures (surge in business latency / garbage collection blockage); Tier 2 migrates load balancing cluster pressure when Tier 1 fails; Tier 3 gradually restores flow control values ​​after system stabilization and avoids performance regression through a rollback mechanism, ultimately forming a closed-loop control system of "predictive flow limiting - emergency response - gradual recovery". This system enables elastic response and secure recovery in the event of control failure, enhances the lack of layered self-healing capabilities when control fails, significantly improves system stability and reliability, and provides strong support for the efficient operation of large-scale distributed systems.

[0023] To facilitate understanding of the technical solutions of the embodiments of this application, the technical solutions of this application are described in detail below through specific embodiments. The above-mentioned related technologies are optional solutions and can be arbitrarily combined with the technical solutions of the embodiments of this application, all of which fall within the protection scope of the embodiments of this application. The embodiments of this application include at least some of the following contents.

[0024] This application proposes a traffic control method for an append-only write storage system, which can be used for traffic prediction and dynamic control of garbage collection tasks. Figure 1 This is a flowchart illustrating a flow control method provided in an embodiment of this application, as shown below. Figure 1 As shown, the method includes the following steps: Step 101: Obtain the target performance metric sequence and determine the load type of each target performance metric in the target performance metric sequence.

[0025] In this embodiment of the application, on a node equipped with a monitoring agent, key performance indicator data for the current time period are collected in real time to obtain a sequence of target performance indicators, including the target performance indicators at each moment in the current time period. Then, the I / O operation characteristics of IOPS in each target performance indicator are analyzed, including read / write ratio, random / sequential access mode, I / O size, and queue depth, to determine the load type of each target performance indicator.

[0026] The target performance metrics at each time point (each target performance metric) include, but are not limited to, CPU utilization, memory utilization, average latency, IOPS data, and throughput. The load type for each target performance metric is categorized as sequential write load, sequential read load, random write load, and random read load.

[0027] Step 102: Based on the load type of each target performance metric, select the target traffic prediction model sequentially from the traffic prediction models for multiple load types.

[0028] The load type of the target traffic prediction model is the same as the load type of the corresponding target performance index.

[0029] In this embodiment, the characteristics of different load types being affected differently by garbage collection can be utilized to construct multiple load type traffic prediction models. Based on the load type of each target performance indicator, a target traffic prediction model that is the same as the load type of the corresponding target performance indicator can be selected from the multiple load type traffic prediction models to achieve traffic prediction for different load types.

[0030] Among them, the traffic characteristics of different load types are significantly different. The traffic characteristics of sequential write load are usually continuous and stable or periodically fluctuating; the traffic characteristics of sequential read load are usually high throughput and low IOPS; the traffic characteristics of random read load and random write load are usually low throughput and high IOPS. In addition, the traffic characteristics of random read load depend on time locality and have strong traffic bursts, while the traffic characteristics of random write load are prone to fragmented writes, which leads to more frequent triggering of background garbage collection, thus resulting in write method effect.

[0031] In some embodiments, the above method further includes: Obtain historical performance index sequences and perform statistical analysis on the characteristics of these sequences to obtain their statistical features. Based on the statistical characteristics of historical performance index sequences, the historical performance index sequences are classified to obtain multiple load types; Based on historical performance index sequences and multiple load types, models were constructed to obtain traffic prediction models for multiple load types.

[0032] Here, based on the characteristics of different load types such as sequential write, sequential read, random write, and random read, the historical performance index data can be divided into multiple load types according to the statistical characteristics of historical performance index data collected in historical time periods, and traffic prediction models for multiple load types can be constructed respectively based on the historical performance index data of multiple load types.

[0033] Specifically, the process begins by acquiring a sequence of historical performance metrics collected over a historical time period, including the historical performance metrics at each moment within that time period. Outlier handling is then performed on each historical performance metric (each historical performance metric) to identify and remove anomalies that may interfere with model learning. Next, based on the load characteristics of each historical performance metric, statistical analysis is performed on each historical performance metric in the sequence to obtain its statistical features, including the read / write ratio of IOPS data, random / sequential access patterns, I / O size, and queue depth. Then, based on the statistical features of each historical performance metric, all historical performance metrics in the sequence are classified to obtain multiple load types, each corresponding to one or more historical performance metrics. Finally, based on each load type, and for each load type's corresponding one or more historical performance metrics, along with their corresponding statistical features, traffic prediction models for the respective load types are constructed using these historical performance metrics and their corresponding statistical features as model input data, thus obtaining traffic prediction models for multiple load types.

[0034] Furthermore, to capture the potential impact of system-level events on performance, virtual machine scheduling event creation, migration, and destruction can be introduced as external context features, labeled with three-dimensional Boolean features for their event types, serving as additional environmental information for building the traffic prediction model. Then, for each load type, the nonlinear correlation between one or more historical performance indicators corresponding to that load type, the statistical characteristics of each historical performance indicator, and the virtual machine scheduling event characteristics of each historical performance indicator is evaluated using mutual information quantification, resulting in one or more mutual information values. A preset number of historical performance indicators, the statistical characteristics of each historical performance indicator, and the virtual machine scheduling event characteristics of each historical performance indicator with the highest mutual information values ​​are retained and used as a preset number of construction features. These preset number of construction features are then used as model input data to construct the traffic prediction model for that load type. In this way, traffic prediction models for multiple load types can be constructed based on the above process.

[0035] Different traffic prediction models for different load types have different construction processes, for example: (1) For the flow prediction model of sequential read load, a model with strong periodicity and high stability can be used to construct it, such as the seasonal autoregressive integrated moving average (SARIMA) model / additivity time series decomposition model (Prophet) and other time series models to predict the IOPS data of sequential read load.

[0036] (2) For the traffic prediction model of sequential write load, a Long Short-Term Memory (LSTM) network model combined with covariates can be used to predict the IOPS data of sequential write load. The model construction steps include: a) Obtain a sequence of performance metrics for sequential write loads (such as IOPS data sequences); b) Obtain external characteristic variables (external covariates) related to the prediction target (IOPS data), including time-based covariates, event-based covariates, and system-based covariates; c) Based on the sequential write load performance index sequence and external feature variables, perform model prediction training to complete model construction.

[0037] (3) For the traffic prediction model of random read load, a lightweight LSTM model can be used to predict the IOPS data of random read load. In the process of model construction, the hidden layer dimension can be compressed to 1 / 4 of the standard LSTM instance, the network depth can be reduced to 1 / 2 of the original model, the output gate structure of the standard LSTM model can be removed, and residual connections can be added to enhance the adaptability to the burst characteristics of random read load traffic.

[0038] (4) For traffic prediction models of random write load, the extreme gradient boosting (XGBoost) model can be used in combination with system state characteristics to predict the IOPS data of random write load. The model construction steps include: a) Extract real-time system characteristics, including solid state drive (SSD) wear and data fragmentation index, from the Flash Translation Layer (FTL) logs and the Self-Monitoring, Analysis, and Reporting Technology (SMART) interface; b) Constructing cross factors: Calculate the risk product of SSD wear and data fragmentation index, and input it as an enhanced feature into the XGBoost model to improve the model's ability to model complex relationships; c) Configure regularized XGBoost models: Limit the maximum depth of a single tree to 7 layers (max_depth=7), and control the model complexity by adjusting the regularization coefficients (such as L1 regularization term reg_alpha and L2 regularization term reg_lambda), effectively suppressing the risk of overfitting.

[0039] In some embodiments, the above method further includes: Every preset prediction period, obtain the number of data points within the preset prediction period and the actual value of the performance index corresponding to each moment; A traffic prediction model based on multiple load types is used to predict the traffic at each time point, and the corresponding performance index prediction value is obtained at each time point. Based on the amount of data within the preset prediction period, the actual value of the performance index at each time point, and the predicted value of the performance index at each time point, the error is calculated to obtain the prediction error index corresponding to the preset prediction period. The traffic prediction models for multiple load types are updated based on the prediction error index corresponding to the preset prediction period.

[0040] Specifically, firstly, according to a preset prediction period, the actual value of the performance indicator at each time point and the number of data points for the performance indicator within each preset prediction period are obtained. Then, based on the load type of the actual value of the performance indicator at each time point, the traffic prediction model for the corresponding load type is used to predict the traffic at each time point online, thus obtaining the predicted value of the performance indicator at each time point. Next, the Mean Absolute Percentage Error (MAPE) can be used as the prediction error indicator. The error is calculated based on the actual value of the performance indicator at each time point within each preset prediction period, the predicted value of the performance indicator at each time point, and the number of data points for the performance indicator, thus obtaining the prediction error indicator for each preset prediction period. Finally, based on the prediction error indicator for each preset prediction period, it is determined whether the traffic prediction model for multiple load types needs to be updated.

[0041] In some embodiments, updating traffic prediction models for multiple load types based on prediction error metrics corresponding to a preset prediction period includes: Obtain the prediction error index corresponding to the first number of consecutive preset prediction periods; For the prediction error index corresponding to the first number of preset prediction periods, the arithmetic mean and standard deviation are calculated respectively to obtain the arithmetic mean and standard deviation values. The dynamic threshold is determined based on the arithmetic mean and standard deviation. If the prediction error index corresponding to a second consecutive number of preset detection cycles is found to be greater than the dynamic threshold, then the first performance index sequence is obtained, and the traffic prediction model for multiple load types is updated based on the first performance index sequence; wherein, the second number is less than the first number.

[0042] Specifically, firstly, the prediction error index corresponding to the latest time for a first number of consecutive preset prediction periods is obtained from each preset prediction period. The arithmetic mean and standard deviation of the prediction error index corresponding to the first number of preset prediction periods are calculated, and a dynamic threshold is generated based on the arithmetic mean and standard deviation. Then, in the first number of consecutive preset prediction periods, starting from the latest preset prediction period, it is determined whether the prediction error index corresponding to the second number of consecutive preset prediction periods is greater than the dynamic threshold, and the second number is less than the first number. If the prediction error index corresponding to the second number of consecutive preset prediction periods is greater than the dynamic threshold, a model update instruction is triggered. At this time, multiple consecutive sample data (first performance index sequence) of the latest time are obtained from the monitoring database, and the traffic prediction models for multiple load types are updated based on these sample data.

[0043] In some embodiments, updating the traffic prediction model for multiple load types based on a first performance metric sequence includes: The attenuation weights are assigned to each first performance index in the first performance index sequence according to the collection time order, thus obtaining the first weighted performance index sequence. Determine the load type for each first-weighted performance metric in the first-weighted performance metric sequence; The traffic prediction model with the same load type as each first-weight performance metric is updated based on that first-weight performance metric.

[0044] Specifically, firstly, based on the acquisition time of each first performance indicator in the first performance indicator sequence, starting from the latest acquisition time, the first first performance indicator is assigned the maximum weight (e.g., 1.0). Then, the weights are uniformly decreased in chronological order, and decaying weights are assigned to each first performance indicator in turn, thus obtaining the first weighted performance indicator sequence. Subsequently, the load type of each first weighted performance indicator in the first weighted performance indicator sequence is determined, and the traffic prediction model with the same load type as the first weighted performance indicator is updated according to each first weighted performance indicator. After updating the traffic prediction models for multiple load types, the updated traffic prediction models for multiple load types are deployed to the additional write storage system, and performance indicators are continuously monitored to form an adaptive optimization closed loop.

[0045] Step 103: Based on the target traffic prediction model, predict the traffic for each target performance indicator to obtain a sequence of predicted performance indicator values.

[0046] In this embodiment, based on a target traffic prediction model with the same load type as each target performance indicator, traffic prediction can be performed for each target performance indicator to obtain the performance indicator prediction value corresponding to each target performance indicator, forming a performance indicator prediction value sequence. The performance indicator prediction value sequence includes the performance indicator prediction value for each time period within a future time period after the current time period.

[0047] Step 104: Determine the waste recycling flow limit value sequence based on the performance index prediction value sequence and the preset performance loss value.

[0048] In this embodiment, the acceptable performance loss values ​​of the traffic prediction model for multiple load types set by the user are first obtained. Based on the load type of each performance index prediction value in the performance index prediction value sequence, the acceptable performance loss value of the traffic prediction model with the same load type as each performance index prediction value can be determined. Then, based on each performance index prediction value and the acceptable performance loss value of the traffic prediction model with the same load type as that performance index prediction value, the corresponding garbage collection rate limit value is calculated, thereby obtaining the garbage collection rate limit value sequence.

[0049] The acceptable performance loss values ​​for traffic prediction models of different load types can be the same or different.

[0050] In some embodiments, the above method further includes: When the system is not performing garbage collection, obtain a sequence of second performance metrics for multiple load types; Obtain a sequence of third performance metrics for multiple load types at multiple garbage collection rate limits; Based on a second performance index sequence for multiple load types and a third performance index sequence for multiple load types at multiple garbage collection current limits, the performance loss value for multiple load types at multiple garbage collection current limits is determined.

[0051] Here, the system has not performed (disabled) garbage collection, which usually means that there are unreachable objects (garbage) in the system, but the garbage collection mechanism has not yet been triggered to perform the cleanup operation.

[0052] Specifically, the process begins by designating the periods during which the system does not perform garbage collection as the pre-collection period. During this period, second performance indicators are collected at each moment, and the load type of each second performance indicator is determined, resulting in a sequence of second performance indicators for multiple load types. Next, the performance indicators for each load type at different garbage collection current limits are obtained, resulting in a sequence of third performance indicators for multiple load types at multiple garbage collection current limits. Subsequently, a performance loss assessment model is constructed based on the second performance indicator sequences and the third performance indicator sequences for multiple load types at multiple garbage collection current limits. The performance loss value for each load type at different garbage collection current limits is then calculated based on the performance loss assessment model, resulting in the performance loss value for multiple load types at multiple garbage collection current limits.

[0053] In some embodiments, step 104 includes: Based on the second performance index sequence of multiple load types and the performance loss values ​​of multiple load types at multiple garbage collection current limits, a performance loss table for multiple load types is determined. For each performance metric prediction value in the performance metric prediction value sequence, a target performance loss table is selected from multiple performance loss tables for multiple load types based on the load type of that performance metric prediction value; wherein, the load type of the target performance loss table is the same as the load type of the performance metric prediction value. Determine the waste recycling flow limit value corresponding to the predicted value and the preset performance loss value of the performance index in the target performance loss table.

[0054] Specifically, firstly, based on the second performance index sequence of multiple load types and the performance loss values ​​of multiple load types at multiple garbage collection rate limits, the second performance index of each load type, the performance loss value of each load type at different garbage collection rate limits, and the multiple garbage collection rate limits are stored separately in a performance loss table for each load type. In each load type's performance loss table, the horizontal axis represents the second performance index of the corresponding load type at each time point, the vertical axis represents different garbage collection rate limits, and each value in the table represents the performance loss value of the corresponding load type at different garbage collection rate limits.

[0055] Next, based on the load type of each performance indicator prediction value in the performance indicator prediction value sequence, a target performance loss table with the same load type is determined. The target performance loss table is then checked to see if there is a garbage collection rate limit value corresponding to the performance indicator prediction value and the preset performance loss value. If there is, the garbage collection rate limit value is used as the garbage collection suggested rate limit value, and the current garbage collection rate limit value is adjusted according to the garbage collection suggested rate limit value.

[0056] In some embodiments, step 104 further includes: Regression analysis was performed on the performance loss values ​​of multiple load types at multiple garbage collection current limits and the multiple garbage collection current limits to obtain regression models for multiple load types; For each performance indicator prediction value in the performance indicator prediction value sequence, a target regression model is selected from multiple regression models based on the load type of that performance indicator prediction value; wherein, the load type of the target regression model is the same as the load type of the performance indicator prediction value. The preset performance loss value is calculated based on the target regression model to obtain the waste recycling flow limit value corresponding to the predicted value of the performance index.

[0057] Specifically, firstly, based on the load type of each performance indicator predicted value in the performance indicator prediction value sequence, a target performance loss table with the same load type is determined. The target performance loss table is then checked to see if a garbage collection rate limit value corresponding to the predicted performance indicator value and the preset performance loss value exists. If not, a regression analysis algorithm (such as linear regression, logistic regression, etc.) is used to perform regression analysis on the performance loss value of each load type at different garbage collection rate limits and multiple garbage collection rate limits, obtaining a regression model for each load type, thus forming multiple load type regression models. Subsequently, based on the load type of each performance indicator predicted value, a target regression model with the same load type as the predicted performance indicator value is selected from the multiple load type regression models. The preset performance loss value is then regressed based on the target regression model to obtain the garbage collection rate limit value corresponding to the predicted performance indicator value. This garbage collection rate limit value is used as the suggested garbage collection rate limit value, and the current garbage collection rate limit value is adjusted based on the suggested garbage collection rate limit value.

[0058] Step 105: Dynamically regulate waste recycling tasks based on the waste recycling flow limit value sequence and preset response mechanism.

[0059] In this embodiment of the application, after determining the waste recycling flow limit value sequence, in order to verify the rationality of the waste recycling flow limit value sequence and the stability of the monitoring system performance, a three-level emergency preset response mechanism can be introduced to adjust the waste recycling flow limit value in real time, so as to dynamically adjust the waste recycling task and cope with the front-end business situation under different scenarios.

[0060] The three-level emergency response mechanism includes three types of emergency strategies: Level 1 quickly reduces flow control and releases resources in the event of sudden failures (surge in business latency / garbage collection blockage); Level 2 migrates the load balancing cluster pressure when Level 1 fails; and Level 3 gradually restores the flow control value after the system stabilizes, and avoids performance regression through a rollback mechanism, ultimately forming a closed-loop control system of "predictive flow limiting - emergency response - gradual recovery".

[0061] In some embodiments, the preset response mechanism includes a first process; step 105 includes: Get business request latency and garbage collection task completion rate at every moment; For each garbage collection rate limit in the garbage collection rate limit sequence, if at least two of the following preset conditions are met, the first process is initiated to reduce the garbage collection rate limit to obtain the reduced garbage collection rate limit: the business request latency reaches the latency threshold; the garbage collection task completion rate is less than the completion rate threshold; the system fluctuation index is greater than the fluctuation threshold. The waste recycling task is dynamically adjusted based on the reduced waste recycling limit value.

[0062] Here, the first process is the Level 1 response strategy, which is used to determine the fault type based on fault judgment, and then selects rate limiting and degradation actions according to preset degradation rules to ensure front-end traffic.

[0063] Specifically, the system monitors and collects performance metrics data at each moment in real time, including business request latency and garbage collection task completion rate. For each garbage collection rate limit value in the garbage collection rate limit value sequence, if it is detected that at least two of the following conditions are met: business request latency reaches the latency threshold, garbage collection task completion rate is less than the completion rate threshold, and system fluctuation index is greater than the fluctuation threshold, the system immediately triggers a level-one response strategy. This includes initiating a dynamic rate limit switching strategy, determining the corresponding fault type based on the fault judgment criteria, and selecting a rate limit degradation action to reduce the garbage collection rate limit value according to preset degradation rules. The reduced garbage collection rate limit value is then obtained, and the garbage collection task is dynamically adjusted based on the reduced garbage collection rate limit value to ensure the normal operation of the front-end business.

[0064] The system volatility index can be calculated using the standard deviation or mean of delay over a historical preset period.

[0065] Here, the preset degradation rules for the dynamic rate limiting switching strategy include: (1) When a sudden increase in business request latency is detected (reaching the latency threshold, such as an increase of 70% within 30 seconds), while the garbage collection task completion rate is normal (greater than or equal to the completion rate threshold), it is determined to be latency-driven, the garbage collection rate limit is reduced by the first preset ratio (such as 30%), and non-core business tasks are suspended. (2) When a sudden drop in the completion rate of garbage collection tasks is detected (less than the completion rate threshold, such as a drop of 70% within 30 seconds), it is determined to be a garbage collection blockage type. The garbage collection flow limit value is reduced by the first preset ratio and the second preset ratio (such as 50%), and idle resources are forcibly released. (3) When it is detected that the business request delay and the garbage collection task completion rate deteriorate at the same time (the business request delay reaches the delay threshold and the garbage collection task completion rate is less than the completion rate threshold), it is determined to be a mixed fault type, and the garbage collection rate limit is reduced by the first preset ratio and the third preset ratio (e.g., 40%).

[0066] In some embodiments, the preset response mechanism further includes a second process; step 105 further includes: If the duration of the first process reaches the first preset time, the number of high-load nodes reaches the preset proportion of the number of cluster nodes, and the frequency of occurrence of the preset conditions reaches the frequency threshold, then the second process is initiated to migrate part of the business load in the high-load nodes to the target load nodes.

[0067] Here, the second process is a two-level response strategy, which is used to build a node stress heatmap to identify migrateable load units and migrate some business loads from high-load nodes to low-load nodes.

[0068] Specifically, after executing the first-level response strategy, if it is detected that the duration of the first-level response strategy reaches the first preset time (e.g., 15 minutes), the high-load nodes reach the preset proportion of the number of cluster nodes (e.g., 30%), the occurrence frequency of the preset conditions in the first-level response strategy reaches the frequency threshold (e.g., 5 times per hour), and the duration of the garbage collection task completion rate being less than the completion rate threshold reaches the fifth preset time, then it is determined that the system performance has not been effectively improved after the execution of the first-level response strategy. At this time, the system immediately triggers the second-level response strategy, including starting the cross-node load migration strategy, identifying migrateable load units through the node CPU / memory (Mem) / IO usage, and migrating some business loads from high-load nodes to low-load nodes to balance the overall system load.

[0069] Examples of cross-node load migration strategies include: (1) The CPU utilization of node A is >85% for 60 seconds, and the system marks node A as a high load source node; (2) The system checks other nodes: Node B (CPU 25%, Mem 20%, IO 15%), Node C (CPU 65%, Mem 40%, IO 10%).

[0070] (3) All resources of node B are less than 30%, which meets the target conditions; CPU and Mem of node C are greater than 30%, which does not meet the target conditions.

[0071] (4) Select node B as the target node.

[0072] (5) On node A, identify a stateless container service X with moderate resource consumption (e.g., 15% CPU usage).

[0073] (6) Migrate container service X from node A to node B.

[0074] (7) After migration: CPU utilization of node A dropped to 70%, CPU utilization of node B rose to 40%, Mem and IO remained low, and the system load was more balanced.

[0075] If the target node load reaches 80% within 90 seconds after migration, the migration will stop; if two consecutive migrations fail, a manual alarm will be triggered.

[0076] In some embodiments, the preset response mechanism further includes a third process; step 105 further includes: If the duration of a business request delay being less than the delay threshold reaches the second preset time, the duration of a garbage collection task completion rate being greater than or equal to the completion rate threshold reaches the third preset time, and the duration of a system fluctuation index being less than the fluctuation threshold reaches the fourth preset time, then the third process is initiated to increase the reduced garbage collection rate limit value to obtain the increased garbage collection rate limit value. The waste recycling task is dynamically adjusted based on the increased waste recycling limit value.

[0077] Here, the third process is a three-level response strategy, which is used to gradually restore the garbage collection flow limit to the recommended garbage collection flow limit calculated in the performance loss table, and continuously monitor system performance indicators.

[0078] Specifically, after executing a Level 1 or Level 2 response strategy, if the duration of a business request delay being less than the delay threshold reaches a second preset time (e.g., 30 minutes), the duration of a garbage collection task completion rate being greater than or equal to the completion rate threshold reaches a third preset time (e.g., 6 cycles), and the duration of a system fluctuation index being less than the fluctuation threshold reaches a fourth preset time (e.g., 1 hour), then it is determined that the system fault has been initially controlled. At this time, the system immediately triggers a Level 3 response strategy, including initiating a gradual recovery strategy, increasing the reduced garbage collection rate limit value to obtain the increased garbage collection rate limit value, and dynamically adjusting the garbage collection task based on the increased garbage collection rate limit value.

[0079] The gradual recovery strategy specifically includes: (1) Each time, the reduced waste recycling limit value is increased by the fourth preset percentage (e.g., 5%) of the current value; (2) Observe the changes in business request delay and garbage collection task completion rate of the system within the preset time period. If the indicators do not deteriorate significantly, continue to increase the garbage collection rate limit until the garbage collection rate limit calculated in the performance loss table is reached. If the indicators deteriorate, roll back to the previous adjustment state and reduce the adjustment range (fourth preset ratio, such as from 5% to 3%). Continue to observe the system response to ensure that the system will not have performance problems again. Then continue to select the garbage collection rate limit according to the performance loss table for dynamic garbage collection flow control.

[0080] (3) Continuously monitor changes in business request delays and garbage collection task completion rates. If any abnormalities occur, continue to use the three-level response strategy.

[0081] It should be noted that after executing the first-level or second-level response strategy, if the duration for which the business request delay is less than the delay threshold does not reach the second preset time (e.g., 30 minutes), the duration for which the garbage collection task completion rate is greater than or equal to the completion rate threshold does not reach the third preset time (e.g., 6 cycles), and the duration for which the system fluctuation index is less than the fluctuation threshold does not reach the fourth preset time (e.g., 1 hour), then it is determined that the system fault has not been initially controlled, and users need to be notified for manual intervention.

[0082] In the technical solution of this application embodiment, a sequence of target performance indicators is obtained, and the load type of each target performance indicator in the sequence is determined; based on the load type of each target performance indicator, a target traffic prediction model is selected sequentially from multiple traffic prediction models for multiple load types; wherein, the load type of the target traffic prediction model is the same as the load type of the corresponding target performance indicator; traffic prediction is performed on each target performance indicator based on the target traffic prediction model to obtain a sequence of predicted performance indicator values; based on the sequence of predicted performance indicator values ​​and a preset performance loss value, a sequence of garbage collection flow limit values ​​is determined; and the garbage collection task is dynamically regulated based on the sequence of garbage collection flow limit values ​​and a preset response mechanism. Thus, by employing dedicated traffic prediction models for different load types, and selecting a target traffic prediction model with the same load type as each target performance indicator to predict traffic for each target performance indicator, it is possible to leverage the different impacts of garbage collection on different load types (sequential read, sequential write, random read, random write) to achieve high-precision short-term traffic prediction for different load types. This solves the problem of inaccurate prediction caused by traditional models ignoring I / O mode differences. At the same time, it dynamically outputs garbage collection rate limits based on the predicted performance indicator values ​​and the user-defined acceptable preset performance loss values, and introduces a preset response mechanism to dynamically regulate garbage collection tasks. This enables elastic response and safe recovery in the event of regulation failure, and enhances the lack of layered self-healing capabilities when regulation fails, significantly improving the stability and reliability of the system, thereby providing strong support for the efficient operation of large-scale distributed systems.

[0083] This application also proposes an append-write storage system based on traffic prediction and dynamic adjustment, such as... Figure 2 The diagram shown is a schematic of an append write storage system based on traffic prediction and dynamic control provided in an embodiment of this application. The system is implemented by two methods: a prediction method based on load type and a dynamic garbage collection control method based on performance loss.

[0084] Figure 3 This is a flowchart illustrating a load type-based prediction method provided in an embodiment of this application, as shown below. Figure 3 As shown, the method includes the following steps: Step 301: Collect performance index data and construct feature engineering.

[0085] First, on nodes equipped with monitoring agents, key performance metrics data are collected in real time, including but not limited to CPU utilization, memory utilization, average latency, IOPS, and throughput.

[0086] Secondly, to capture the potential impact of system-level events on performance, virtual machine scheduling event creation, migration, and destruction are introduced as important external context features, and three-dimensional Boolean features are used to label their event types as additional environmental information for the model.

[0087] Next, outlier handling algorithms (including but not limited to the isolated forest algorithm) are used to process the collected performance index data to identify and remove outliers that may interfere with model learning.

[0088] Then, the nonlinear correlation between each constructed feature (including performance index data, statistical features, and virtual machine scheduling event features) and the target variable (IOPS) is evaluated by mutual information quantization, and the top-K features with the highest mutual information values ​​are retained to reduce model complexity, reduce the risk of overfitting, and improve the model's generalization performance.

[0089] Step 302: Construct traffic prediction models for different load types based on the performance index data of different load types.

[0090] Storage systems primarily involve four load types: sequential read, sequential write, random read, and random write. The traffic characteristics of these different load types differ significantly: sequential read and write exhibit high throughput and low IOPS. Sequential reads, due to strong spatial locality from consecutive address accesses, have stable and predictable traffic; sequential writes, on the other hand, typically exhibit continuous stability or periodic fluctuations. Random read and write exhibit low throughput and high IOPS. Random reads rely on temporal locality, resulting in strong traffic bursts; random writes are prone to fragmented writes, leading to more frequent triggering of background garbage collection and causing write amplification.

[0091] Based on the characteristics of the four load types mentioned above, and according to the statistical features of the performance index data, the collected performance index data (such as IOPS) are divided according to the four load types to obtain the performance index data for the four load types.

[0092] For the performance metrics data of the four load types, four corresponding traffic prediction models are constructed: (1) Using the SARIMA / Prophet time series model to predict IOPS data of sequential read load has strong periodicity and high stability.

[0093] (2) The IOPS data of sequential write load is predicted using an LSTM+covariate model. The model construction steps are as follows: a) Obtain the sequential write load IOPS data sequence; b) Obtain external characteristic variables related to the prediction target (IOPS) through the indicator collection and monitoring module; c) Collect covariates The following is a sample of time-related covariates: hour, day of the week, and whether it is a holiday. Whether event-based covariates have migrated, and database maintenance. System-level covariates such as replica count, compression ratio, and node status are used as covariate information. ; d) The model prediction output is completed by writing the load IOPS data sequence in sequence and using external covariates. The calculation formula is as follows: (1) in, The predicted IOPS data at time t. To sequentially write the load IOPS data sequence, It is a covariate.

[0094] (3) A lightweight LSTM model is used to predict IOPS data for random read loads. The model construction steps are as follows: a) The hidden layer dimension is compressed to 1 / 4 of that of a standard LSTM instance; b) The network depth is reduced to half of the original model; c) Remove the output gate structure of the standard LSTM model and add residual connections to the standard LSTM model; The above steps can enhance adaptability to bursty characteristics of random read loads, such as instantaneous IOPS spikes.

[0095] (4) The XGBoost+ system state characteristic model is used to predict the IOPS data of random write load. The model construction steps are as follows: a) Extract real-time system characteristics from FTL logs and SMART interface, including SSD wear and data fragmentation index; b) Constructing cross factors: Calculate the risk product of SSD wear and data fragmentation index, and input it as an enhanced feature into the XGBoost model to improve the model's ability to model complex relationships; c) Configure regularized XGBoost models: Limit the maximum depth of a single tree to 7 layers (max_depth=7), and control the model complexity by adjusting the regularization coefficients (such as L1 regularization term reg_alpha and L2 regularization term reg_lambda), effectively suppressing the risk of overfitting.

[0096] Step 303: Fine-tune and optimize the traffic prediction model for each load type online.

[0097] To maintain the long-term prediction accuracy of the traffic prediction models for the four load types mentioned above, an online fine-tuning optimization mechanism is introduced, such as... Figure 4 The diagram shown is a flowchart illustrating the online fine-tuning and optimization mechanism provided in this application embodiment, which specifically includes the following steps: Step 401: Perform online prediction using the traffic prediction models for the four load types mentioned above according to the prediction period, and obtain the predicted IOPS value at each time point within the prediction period.

[0098] Step 402: Record the actual IOPS value at each moment within the prediction period, and use MAPE as the prediction error index to continuously calculate the error value according to the prediction period.

[0099] The formula for calculating MAPE for a single forecast period is as follows: = (2) in, The actual IOPS value at time t. Let be the predicted IOPS value at time t. This represents the number of data points within the period.

[0100] Step 403: Extract the MAPE values ​​for the most recent X consecutive prediction periods.

[0101] Here, X is set to 30 by default.

[0102] Step 404: Calculate the arithmetic mean and standard deviation of the MAPE values ​​for X consecutive prediction periods.

[0103] Step 405: Calculate the dynamic threshold using the arithmetic mean and standard deviation of the MAPE values ​​over X consecutive prediction periods.

[0104] The dynamic threshold C is generated according to the following formula: (3) in, The arithmetic mean of the MAPE values ​​over X consecutive prediction periods. The standard deviation of MAPE values ​​over X consecutive prediction periods.

[0105] Step 406: Determine whether the condition is met for N consecutive prediction periods (default N=3). If so, proceed to step 407 and trigger the model update command.

[0106] The default value here is N=3.

[0107] Step 407: Extract the latest consecutive K sample data from the monitoring database (default K=50), sort them from earliest to latest by collection time, and form a sequence {sample1 (oldest), sample2 ... sampleK (latest)}.

[0108] Step 408: Assign a linearly increasing decay weight to each sample to obtain K weighted sample data.

[0109] The earliest sample (the 1st) has a weight of 0.5, the latest sample (the Kth) has a weight of 1.0, and the weights of intermediate samples increase uniformly in sequence, calculated using the following formula: (3) Where i represents the position of the sample in the sequence (1≤i≤K).

[0110] Step 409: Update the traffic prediction model for the corresponding load type using K weighted sample data.

[0111] The training objective function is as follows: (4) in, It is the loss function of the prediction model.

[0112] Step 410: Deploy the fine-tuned traffic model to the prediction system, continuously monitor it, and form an adaptive optimization closed loop.

[0113] Figure 5 This is a flowchart illustrating a dynamic garbage collection control method based on performance loss provided in an embodiment of this application, as shown below. Figure 5 As shown, the method includes the following steps: Step 501: Confirm the load type i corresponding to the current traffic, and obtain the IOPS prediction value corresponding to the current traffic based on the traffic prediction model corresponding to load type i.

[0114] Step 502: Receive the acceptable IOPS performance loss values ​​for the traffic prediction model for the four load types.

[0115] The acceptable IOPS performance loss value for traffic prediction models with different load models can be specified by the user.

[0116] Step 503: Determine and confirm the performance loss table based on the load type i corresponding to the current traffic. .

[0117] Step 504: Determine the performance loss table If there is a garbage collection suggested rate limit corresponding to the IOPS prediction value and the IOPS performance loss value, proceed to step 505; otherwise, proceed to step 506.

[0118] Step 505: Using the performance loss table Obtain the recommended flow limit value for waste recycling and proceed to step 507.

[0119] (1) Collect IOPS at a fixed frequency F1 (e.g., 1 second), collect business request latency and garbage collection task completion rate at a fixed frequency F2 (e.g., 10 seconds) on nodes with monitoring agents, and collect garbage collection efficiency at a fixed frequency F3 (e.g., 30 seconds).

[0120] (2) When the system is not performing garbage collection, presample the IOPS values ​​of four load types (to be used later). express).

[0121] (3) Construct an IOPS loss-performance evaluation model. The IOPS loss calculation formula is as follows: (5) in, This represents the IOPS value of type i load when garbage collection is not performed (disabled). This represents the IOPS value for load type i when the garbage collection traffic is throttled to x. This represents the IOPS performance penalty for load type i. Wherein, These correspond to four load types: sequential read, random read, sequential write, and random write.

[0122] (4) Based on the above IOPS loss-performance evaluation model, the IOPS loss values ​​for the four load types under different garbage collection current limits were calculated. The calculation results were stored in four tables according to the load type. middle, In each table, the horizontal axis represents the IOPS value of the corresponding load type at different times during the pre-sampling phase, and the vertical axis represents different garbage collection rate limits. Each value in the table represents the IOPS performance loss value of the corresponding load type at different garbage collection rate limits.

[0123] Step 506: Calculate the garbage collection recommended flow limit corresponding to the predicted IOPS and the IOPS performance loss value using the regression formula.

[0124] For each load type i, a regression model between the IOPS performance loss value and the garbage collection current limit value can be established using regression analysis based on the above calculation results.

[0125] Step 507: Adjust the current waste recycling flow limit based on the recommended flow limit.

[0126] After accepting the recommended flow limit value for waste recycling adjusted in the above steps, in order to verify the rationality of the recommended flow limit value for waste recycling and the stability of the monitoring system performance, the dynamic waste recycling control method based on performance loss provided in this application embodiment also designs a three-level emergency response mechanism to deal with the front-end business situation under different scenarios.

[0127] Figure 6 This is a flowchart illustrating the three-level emergency response mechanism provided in the embodiments of this application, such as... Figure 6 As shown, the three-level emergency response mechanism includes the following steps: Step 601: Monitor and collect performance metrics data.

[0128] Step 602: Determine whether the business latency has reached the preset ratio of the current average latency, whether the garbage collection task completion rate is lower than the benchmark value, and whether the system fluctuation index is higher than the fluctuation value. If any two of the above conditions are met, proceed to step 603.

[0129] When the detected business latency reaches 200% of the current average latency (default threshold, user-configurable), the garbage collection completion rate is lower than the baseline value of 40% (default threshold, user-configurable), and the system volatility index is higher than 0.7 (calculated as the standard deviation of latency over the past 5 minutes / mean), the first-level response strategy is immediately triggered.

[0130] Step 603: Immediately trigger the Level 1 response strategy: Determine the fault type based on the fault judgment criteria, and further select rate limiting and degradation actions according to the preset degradation rules to ensure front-end traffic.

[0131] The system initiates a dynamic rate limiting switching strategy. Based on the fault diagnosis criteria, it determines the corresponding fault type and selects rate limiting and degradation actions according to preset degradation rules, prioritizing the normal operation of front-end services. The preset degradation rules include: a) When a sudden increase in latency (70% increase within 30 seconds) is detected but garbage collection is normal, it is determined to be a latency-driven type, the rate limit is reduced by 30%, and non-core business tasks are suspended. b) When a sudden drop in the garbage collection task completion rate is detected (a drop of 70% within 30 seconds), it is determined to be a garbage collection blockage, the flow limit is reduced by 50%, and idle resources are forcibly released; c) When both delay and garbage collection are detected to worsen simultaneously, it is classified as a mixed fault type, and the current limit value is reduced by 40%.

[0132] Step 604: Determine whether it is necessary to improve the resource shortage situation of multiple nodes. If yes, proceed to step 605; otherwise, proceed to step 606.

[0133] If the Level 1 response policy lasts for 15 minutes or more, the number of violations of the Level 1 threshold reaches 5 times per hour, the number of resource-stressed nodes accounts for more than 30% of the total number of clusters (user-configurable), and the garbage collection task completion rate is consistently lower than the baseline value by 40 percentage points or more, then it is determined that the system performance has not been effectively improved after the Level 1 response, and the Level 2 response policy is immediately triggered.

[0134] Among them, the first-level threshold refers to one or more of the above-mentioned business delay threshold, the benchmark value of garbage collection task completion rate, and the fluctuation value of system fluctuation index.

[0135] Step 605: Immediately trigger the secondary response strategy: Build a node stress heatmap to identify migrateable load units and migrate some service loads from high-load nodes to low-load nodes.

[0136] The system initiates a cross-node load migration strategy, identifying migrateable load units based on node CPU / memory (Mem) / IO usage, and migrating some business loads from high-load nodes to low-load nodes to balance the overall system load. An example migration process is as follows: a) If the CPU utilization of node A is >85% for 60 seconds, the system marks node A as a high-load source node; b) The system checks other nodes: Node B (CPU 25%, Mem 20%, IO 15%), Node C (CPU 65%, Mem 40%, IO 10%).

[0137] c) All resources of node B are less than 30%, which meets the target condition; CPU and Mem of node C are greater than 30%, which does not meet the target condition.

[0138] d) Select node B as the target node.

[0139] e) On node A, identify a stateless container service X with moderate resource consumption (e.g., using 15% of CPU).

[0140] f) Migrate container service X from node A to node B.

[0141] g) After migration: Node A's CPU utilization dropped to 70%, while Node B's CPU utilization rose to 40%. Mem and IO remained low, resulting in a more balanced system load.

[0142] If the target node load reaches 80% within 90 seconds after migration, the migration will stop; if two consecutive migrations fail, a manual alarm will be triggered.

[0143] Step 606: Determine whether the system fault has been initially controlled. If yes, proceed to step 607; otherwise, proceed to step 608.

[0144] When a business delay is detected to be less than the safety threshold for 30 minutes, the garbage collection completion rate is greater than the baseline value for 6 consecutive cycles, and the system fluctuation index is less than 0.3 for 1 hour, it is determined that the system fault has been initially controlled and a level 3 response strategy is immediately triggered.

[0145] Step 607: Immediately trigger the Level 3 response strategy: Gradually restore the garbage collection rate limit to [value missing]. The table calculates the recommended rate limit value, and the system performance metrics are continuously monitored.

[0146] a) The system follows a gradual recovery strategy, increasing the garbage collection limit by 5% of the current value each time. b) Observe the changes in system latency and garbage collection completion rate over 10 minutes. If the indicators do not deteriorate significantly, continue to increase them until... The table shows the recommended rate limiting values. c) If the situation worsens, roll back to the previous adjustment state and reduce the adjustment range to 3%. Continuously monitor the system's response to ensure that performance issues do not recur, and then continue as before. The table is used to select the waste recycling flow limit value for dynamic waste recycling flow control. The above indicators are continuously monitored, and if any abnormality occurs, the three-level response strategy will continue to be used.

[0147] Step 608: Perform manual intervention.

[0148] This application provides a traffic control and elastic response method for append-write storage systems. This method leverages the varying impacts of garbage collection on different load types (sequential read, sequential write, random read, and random write) to construct front-end traffic prediction models for each. Based on the front-end traffic prediction results, the back-end traffic for garbage collection is adjusted in advance to avoid control lag issues. Simultaneously, a three-level emergency response mechanism enhances system performance stability and elastic recovery capabilities. Specifically, it includes: (1) Dynamic prediction model based on load type differences: By constructing four load-specific predictors (sequential read SARIMA / Prophet, sequential write LSTM, random read lightweight LSTM, and random write XGBoost system state model), integrating I / O mode characteristics and real-time covariates, and combining a time decay weight fine-tuning mechanism triggered by dynamic error threshold, high-precision short-term traffic prediction is achieved. This method solves the problem of prediction inaccuracy caused by ignoring I / O mode differences in traditional models and prevents long-term prediction accuracy decay.

[0149] (2) Rate limiting decision and three-level elastic response mechanism based on IOPS loss table: A multi-dimensional loss table is constructed based on the pre-sampled baseline IOPS, and the garbage collection rate limiting value is dynamically output based on the predicted load type / value and user tolerance loss. At the same time, a three-level fault self-healing process is constructed (first level: dynamic degradation of fault type; second level: cross-node load migration; third level: gradual recovery rate limiting) to achieve elastic response and safe recovery in the event of regulation failure. This method enhances the lack of hierarchical self-healing capability when regulation fails, significantly improves the stability and reliability of the system, and provides strong support for the efficient operation of large-scale distributed systems.

[0150] This application also proposes a flow control device for use in an append-only write storage system, wherein the append-only write storage system can be used for flow prediction and dynamic control of garbage collection tasks. Figure 7 This is a schematic diagram of the structure of a flow control device provided in an embodiment of this application, as shown below. Figure 7 As shown, the device includes: Acquisition unit 701 is used to acquire the target performance index sequence.

[0151] The determination unit 702 is used to determine the load type of each target performance indicator in the target performance indicator sequence.

[0152] Selection unit 703 is used to select a target traffic prediction model from multiple traffic prediction models for multiple load types based on the load type of each target performance indicator; wherein the load type of the target traffic prediction model is the same as the load type of the corresponding target performance indicator.

[0153] The prediction unit 704 is used to predict the flow of each target performance index based on the target flow prediction model, and obtain a sequence of predicted performance index values.

[0154] The determining unit 702 is also used to determine the waste recycling flow limit value sequence based on the performance index prediction value sequence and the preset performance loss value.

[0155] The control unit 705 is used to dynamically control the waste recycling task based on the waste recycling flow limit value sequence and the preset response mechanism.

[0156] In some embodiments, the acquisition unit 701 is further configured to acquire a historical performance index sequence.

[0157] In some embodiments, the device further includes: The statistical unit is used to statistically analyze the characteristics of historical performance index sequences to obtain the statistical features of the historical performance index sequences.

[0158] The classification unit is used to classify historical performance index sequences based on statistical characteristics of the historical performance index sequences to obtain multiple load types.

[0159] The building unit is used to build models based on historical performance index sequences and multiple load types, resulting in traffic prediction models for multiple load types.

[0160] In some embodiments, the acquisition unit 701 is further configured to acquire the number of data within the preset prediction period and the actual value of the performance index corresponding to each moment at each preset prediction period.

[0161] In some embodiments, the prediction unit 704 is further configured to predict the traffic at each time moment based on a traffic prediction model for multiple load types, and obtain the predicted value of the performance index corresponding to each time moment.

[0162] In some embodiments, the device further includes: The calculation unit is used to calculate the error based on the amount of data within the preset prediction period, the actual value of the performance index at each time point, and the predicted value of the performance index at each time point, so as to obtain the prediction error index corresponding to the preset prediction period.

[0163] The update unit is used to update the traffic prediction model for multiple load types based on the prediction error index corresponding to the preset prediction period.

[0164] In some embodiments, the updating unit is specifically used for: Obtain the prediction error index corresponding to the first number of consecutive preset prediction periods; For the prediction error index corresponding to the first number of preset prediction periods, the arithmetic mean and standard deviation are calculated respectively to obtain the arithmetic mean and standard deviation values. The dynamic threshold is determined based on the arithmetic mean and standard deviation. If the prediction error index corresponding to a second consecutive number of preset detection cycles is found to be greater than the dynamic threshold, then the first performance index sequence is obtained, and the traffic prediction model for multiple load types is updated based on the first performance index sequence; wherein, the second number is less than the first number.

[0165] In some embodiments, the updating unit is further specifically used for: The attenuation weights are assigned to each first performance index in the first performance index sequence according to the collection time order, thus obtaining the first weighted performance index sequence. Determine the load type for each first-weighted performance metric in the first-weighted performance metric sequence; The traffic prediction model with the same load type as each first-weight performance metric is updated based on that first-weight performance metric.

[0166] In some embodiments, the acquisition unit 701 is further configured to acquire a second performance index sequence of multiple load types when the system is not performing garbage collection; and acquire a third performance index sequence of multiple load types at multiple garbage collection current limits.

[0167] In some embodiments, the determining unit 702 is further configured to determine the performance loss value of multiple load types at multiple garbage collection current limits based on a second performance index sequence of multiple load types and a third performance index sequence of multiple load types at multiple garbage collection current limits.

[0168] In some embodiments, the determining unit 702 is specifically used for: Based on the second performance index sequence of multiple load types and the performance loss values ​​of multiple load types at multiple garbage collection current limits, a performance loss table for multiple load types is determined. For each performance metric prediction value in the performance metric prediction value sequence, a target performance loss table is selected from multiple performance loss tables for multiple load types based on the load type of that performance metric prediction value; wherein, the load type of the target performance loss table is the same as the load type of the performance metric prediction value. Determine the waste recycling flow limit value corresponding to the predicted value and the preset performance loss value of the performance index in the target performance loss table.

[0169] In some embodiments, the determining unit 702 is further specifically used for: Regression analysis was performed on the performance loss values ​​of multiple load types at multiple garbage collection current limits and the multiple garbage collection current limits to obtain regression models for multiple load types; For each performance indicator prediction value in the performance indicator prediction value sequence, a target regression model is selected from multiple regression models based on the load type of that performance indicator prediction value; wherein, the load type of the target regression model is the same as the load type of the performance indicator prediction value. The preset performance loss value is calculated based on the target regression model to obtain the waste recycling flow limit value corresponding to the predicted value of the performance index.

[0170] In some embodiments, the preset response mechanism includes a first process; wherein, the control unit 705 is specifically used for: Get business request latency and garbage collection task completion rate at every moment; For each garbage collection rate limit in the garbage collection rate limit sequence, if at least two of the following preset conditions are met, the first process is initiated to reduce the garbage collection rate limit to obtain the reduced garbage collection rate limit: the business request latency reaches the latency threshold; the garbage collection task completion rate is less than the completion rate threshold; the system fluctuation index is greater than the fluctuation threshold. The waste recycling task is dynamically adjusted based on the reduced waste recycling limit value.

[0171] In some embodiments, the preset response mechanism further includes a second process; wherein, the control unit 705 is further specifically used for: If the duration of the first process reaches the first preset time, the number of high-load nodes reaches the preset proportion of the number of cluster nodes, and the frequency of occurrence of the preset conditions reaches the frequency threshold, then the second process is initiated to migrate part of the business load in the high-load nodes to the target load nodes.

[0172] In some embodiments, the preset response mechanism further includes a third process; wherein, the control unit 705 is further specifically used for: If the duration of a business request delay being less than the delay threshold reaches the second preset time, the duration of a garbage collection task completion rate being greater than or equal to the completion rate threshold reaches the third preset time, and the duration of a system fluctuation index being less than the fluctuation threshold reaches the fourth preset time, then the third process is initiated to increase the reduced garbage collection rate limit value to obtain the increased garbage collection rate limit value. The waste recycling task is dynamically adjusted based on the increased waste recycling limit value.

[0173] Those skilled in the art should understand that Figure 7 The functions of each unit in the communication device shown can be understood by referring to the relevant description of the aforementioned method. Figure 7 The functions of each unit in the communication device shown can be implemented by a program running on a processor or by specific logic circuits.

[0174] Figure 8 This is a schematic diagram of the processing device provided in an embodiment of this application. The processing device may be a terminal device or a network device. Figure 8 The processing device shown includes a processor 801, which can call and run computer programs from memory to implement the methods in the embodiments of this application.

[0175] Optionally, such as Figure 8As shown, the processing device may further include a memory 802. The processor 801 can retrieve and run computer programs from the memory 802 to implement the methods described in this embodiment.

[0176] The memory 802 can be a separate device independent of the processor 801, or it can be integrated into the processor 801.

[0177] Optionally, such as Figure 8 As shown, the processing device may also include a transceiver 803, which the processor 801 can control to communicate with other devices. Specifically, it can send information or data to other devices or receive information or data sent by other devices.

[0178] The transceiver 803 may include a transmitter and a receiver. The transceiver 803 may further include an antenna, and the number of antennas may be one or more.

[0179] The processing device may specifically be the flow control device of this application embodiment, and the processing device can implement the corresponding processes of the various methods implemented in this application embodiment. For the sake of brevity, it will not be described in detail here.

[0180] It should be understood that the processor in the embodiments of this application may be an integrated circuit chip with signal processing capabilities. In implementation, the steps of the above method embodiments can be completed by integrated logic circuits in the processor's hardware or by instructions in software form. The processor described above can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules can be located in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The storage medium is located in memory, and the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method.

[0181] It is understood that the memory in the embodiments of this application can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDR SDRAM), Enhanced Synchronous DRAM (ESDRAM), Synchlink DRAM (SLDRAM), and Direct Rambus RAM (DR RAM). It should be noted that the memory used in the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0182] This application also provides a computer-readable storage medium for storing a computer program. This computer-readable storage medium can be applied to the processing device in this application embodiment, and the computer program causes the computer to execute the corresponding processes implemented by the various methods in this application embodiment; for brevity, further details are omitted here.

[0183] This application also provides a computer program product, including computer program instructions. This computer program product can be applied to the processing device in this application embodiment, and the computer program instructions cause the computer to execute the corresponding processes implemented by the various methods in this application embodiment; for brevity, further details are omitted here.

[0184] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0185] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0186] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0187] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0188] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0189] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0190] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

Claims

1. A method of flow regulation, characterized by, The method includes: Obtain the target performance index sequence and determine the load type of each target performance index in the target performance index sequence; Based on the load type of each target performance indicator, a target traffic prediction model is selected sequentially from multiple traffic prediction models for different load types; wherein the load type of the target traffic prediction model is the same as the load type of the corresponding target performance indicator. Based on the target traffic prediction model, traffic prediction is performed on each target performance indicator to obtain a sequence of predicted performance indicator values. Based on the predicted value sequence of the performance indicators and the preset performance loss value, a sequence of waste recycling flow restriction values ​​is determined; The waste recycling task is dynamically regulated based on the waste recycling flow limit value sequence and the preset response mechanism.

2. The method of claim 1, wherein, The method further includes: Obtain a historical performance index sequence and statistically analyze the characteristics of the historical performance index sequence to obtain the statistical features of the historical performance index sequence. Based on the statistical characteristics of the historical performance index sequence, the historical performance index sequence is classified to obtain multiple load types; Based on the historical performance index sequence and the multiple load types, models are constructed to obtain traffic prediction models for the multiple load types.

3. The method of claim 1, wherein, The method further includes: Every preset prediction period, the number of data points within the preset prediction period and the actual value of the performance index corresponding to each moment are obtained; Based on the traffic prediction model of the multiple load types, the traffic at each time moment is predicted to obtain the predicted value of the performance index at each time moment; Based on the amount of data within the preset prediction period, the actual value of the performance index at each time point, and the predicted value of the performance index at each time point, error calculation is performed to obtain the prediction error index corresponding to the preset prediction period. The traffic prediction models for the multiple load types are updated based on the prediction error index corresponding to the preset prediction period.

4. The method of claim 3, wherein, The step of updating the traffic prediction model for the multiple load types based on the prediction error index corresponding to the preset prediction period includes: Obtain the prediction error index corresponding to the first number of consecutive preset prediction periods; The arithmetic mean and standard deviation are calculated for the prediction error index corresponding to the first number of preset prediction periods, respectively, to obtain the arithmetic mean and standard deviation values. The dynamic threshold is determined based on the arithmetic mean and the standard deviation. If the prediction error index corresponding to a second consecutive number of preset detection cycles is found to be greater than the dynamic threshold, then a first performance index sequence is obtained, and the traffic prediction model for the multiple load types is updated based on the first performance index sequence; wherein, the second number is less than the first number.

5. The method of claim 4, wherein, The update of the traffic prediction model for the multiple load types based on the first performance index sequence includes: The first weighted performance index sequence is obtained by assigning attenuation weights to each first performance index in the first performance index sequence according to the collection time order. Determine the load type of each first weighted performance index in the first weighted performance index sequence; The traffic prediction model with the same load type as each first weight performance index is updated based on the first weight performance index.

6. The method of claim 1, wherein, The method further includes: When the system is not performing garbage collection, obtain a sequence of second performance metrics for multiple load types; Obtain a sequence of third performance metrics for multiple load types at multiple garbage collection rate limits; Based on a second performance index sequence for multiple load types and a third performance index sequence for multiple load types at multiple garbage collection current limits, the performance loss value for multiple load types at multiple garbage collection current limits is determined.

7. The method of claim 6, wherein, The step of determining the waste recycling flow restriction value sequence based on the predicted performance index value sequence and the preset performance loss value includes: Based on the second performance index sequence of multiple load types and the performance loss values ​​of multiple load types at multiple garbage collection current limits, a performance loss table for multiple load types is determined. For each performance metric predicted value in the performance metric predicted value sequence, a target performance loss table is selected from the performance loss tables of the plurality of load types based on the load type of the performance metric predicted value; wherein the load type of the target performance loss table is the same as the load type of the performance metric predicted value. In the target performance loss table, determine the waste recycling flow limit value corresponding to the predicted value of the performance index and the preset performance loss value.

8. The method of claim 6, wherein, The step of determining the waste recycling flow restriction value sequence based on the predicted performance index value sequence and the preset performance loss value includes: Regression analysis was performed on the performance loss values ​​of the multiple load types at multiple garbage collection current limits and the multiple garbage collection current limits to obtain regression models for multiple load types; For each predicted performance index value in the predicted performance index value sequence, a target regression model is selected from the regression models of the plurality of load types based on the load type of the predicted performance index value; wherein the load type of the target regression model is the same as the load type of the predicted performance index value. The preset performance loss value is calculated based on the target regression model to obtain the waste recycling flow limit value corresponding to the predicted performance index value.

9. The method according to claim 7 or 8, characterized in that, The preset response mechanism includes a first process; the dynamic control of waste collection tasks based on the waste collection flow limit value sequence and the preset response mechanism includes: Get business request latency and garbage collection task completion rate at every moment; For each garbage collection rate limit in the garbage collection rate limit sequence, if at least two of the following preset conditions are met, the first process is initiated to reduce the garbage collection rate limit to obtain the reduced garbage collection rate limit: the service request delay reaches the delay threshold; the garbage collection task completion rate is less than the completion rate threshold; the system fluctuation index is greater than the fluctuation threshold. The waste recycling task is dynamically adjusted based on the reduced waste recycling flow limit value.

10. The method of claim 9, wherein, The preset response mechanism further includes a second process; the method further includes: If the duration of the first process reaches a first preset time, the number of high-load nodes reaches a preset proportion of the number of cluster nodes, and the frequency of occurrence of the preset conditions reaches a frequency threshold, then the second process is initiated to migrate part of the business load in the high-load nodes to the target load node.

11. The method of claim 9, wherein, The preset response mechanism also includes a third process; the method further includes: If the duration of the business request delay being less than the delay threshold reaches a second preset time, the duration of the garbage collection task completion rate being greater than or equal to the completion rate threshold reaches a third preset time, and the duration of the system fluctuation index being less than the fluctuation threshold reaches a fourth preset time, then the third process is initiated to increase the reduced garbage collection rate limit value to obtain the increased garbage collection rate limit value. The waste recycling task is dynamically adjusted based on the increased waste recycling flow limit value.

12. A flow regulating device, characterized by The device includes: The acquisition unit is used to acquire the target performance index sequence. A determining unit is used to determine the load type of each target performance indicator in the target performance indicator sequence; The selection unit is used to select a target traffic prediction model from multiple traffic prediction models for multiple load types based on the load type of each target performance indicator; wherein the load type of the target traffic prediction model is the same as the load type of the corresponding target performance indicator. The prediction unit is used to predict the traffic flow of each target performance indicator based on the target traffic flow prediction model, and obtain a sequence of predicted performance indicator values. The determining unit is further configured to determine a sequence of waste recycling flow restriction values ​​based on the predicted performance index value sequence and the preset performance loss value; The control unit is used to dynamically control the waste recycling task based on the waste recycling flow limit value sequence and the preset response mechanism.

13. A processing device, characterized by include: A processor and a memory for storing a computer program, the processor for calling and running the computer program stored in the memory to perform the method as described in any one of claims 1 to 11.

14. A computer-readable storage medium, characterized in that, Used to store a computer program that causes a computer to perform the method as described in any one of claims 1 to 11.

15. A computer program product, characterised in that, It includes computer program instructions that cause a computer to perform the method as described in any one of claims 1 to 11.

Images