Method and system for optimizing cloud platform resource allocation based on lstm model

Abstract

The application discloses a method and system for optimizing cloud platform resource allocation based on an LSTM model, comprising: collecting and preprocessing multi-dimensional heterogeneous time series data of a cloud platform; inputting a time series feature vector constructed according to the time series data based on a pre-trained long short-term memory (LSTM) model to predict resource load in a future predetermined time window; receiving and analyzing security events from a security system in real time; generating resource scheduling instructions based on the prediction results and the analyzed security events in combination with a preset collaborative strategy library; and executing the resource scheduling instructions to elastically expand or dynamically recover the cloud platform resources. The dynamic recovery adopts a parabolic smoothing recovery strategy. The application introduces machine learning prediction, security event linkage and innovative smoothing recovery algorithms to solve the problems of extensive resource allocation and delayed response in the prior art, and effectively improves resource utilization, business stability and system security.

Description

Technical Field

[0001] This invention belongs to the field of cloud computing technology, specifically relating to a method and system for optimizing cloud platform resource allocation based on the LSTM model. Background Technology

[0002] With the deepening of digital transformation, various industries are increasingly reliant on cloud computing, and enterprises and organizations are migrating their core businesses to cloud platforms on a large scale. Cloud platforms, with their on-demand allocation and elastic scaling characteristics, provide users with great convenience. However, how to efficiently, intelligently, and securely manage and allocate massive amounts of cloud resources has become a key technological challenge facing the cloud computing field.

[0003] Existing cloud platform resource management methods mostly rely on relatively simple automation strategies. For example, a common autoscaling mechanism triggers resource increases or decreases by setting static resource utilization thresholds. It monitors single metrics such as CPU utilization of cloud hosts, triggering expansion when the metric consistently exceeds a preset upper limit (e.g., 80%) or shrinking when it consistently falls below a lower limit (e.g., 20%). However, such technical solutions have revealed several shortcomings in practice. First, their strategies are too rigid and cannot adapt to complex and ever-changing business load patterns. For businesses with obvious periodicity (e.g., load differences between weekdays and weekends), tidal patterns (e.g., evening traffic peaks for video websites), or sudden events (e.g., e-commerce promotional activities), fixed thresholds often lead to response delays. Expansion operations often occur only after the business load has already surged and user experience has been affected; this "response delay window" is unacceptable for businesses with high real-time requirements. Conversely, after a business peak, a threshold-based scaling-down mechanism may be too aggressive, using a step-like or linear approach to quickly reclaim resources. Such abrupt resource changes can easily lead to performance fluctuations, or even interrupt ongoing slow transactions or long connections, affecting the stability of the business.

[0004] Secondly, existing methods generally lack foresight and predictability. They are essentially reactive, post-hoc response mechanisms that cannot anticipate upcoming load changes. As a result, the system can only passively follow load fluctuations and cannot achieve ideal resource and operational status, which presents an inherent weakness in ensuring the continuity of critical business operations and the quality of service (SLA).

[0005] Furthermore, existing technical solutions typically build and operate resource management and security protection as two separate systems, lacking an effective linkage mechanism. When a cloud platform faces resource-intensive network attacks, such as distributed denial-of-service (DDoS) attacks or application-layer CC (Challenge Collapsar) attacks, the attack traffic disguises itself as normal user requests, causing a sharp increase in the utilization of resources such as CPU and network bandwidth. At this time, traditional auto-scaling systems may mistakenly identify this as business growth, triggering expansion. This is tantamount to providing attackers with more attack resources, not only failing to alleviate the problem but also accelerating the depletion of normal business resources and generating high resource costs, which can, in severe cases, paralyze the entire platform service.

[0006] Therefore, how to overcome the shortcomings of existing technologies, such as extensive resource allocation, delayed response, lack of predictive ability, and disconnect between security and resource management, and to achieve a more refined, forward-looking, and highly secure cloud platform resource allocation method, is a technical problem that urgently needs to be solved in this field. Summary of the Invention

[0007] One aspect of this invention is to provide a method for optimizing cloud platform resource allocation based on an LSTM model, which aims to solve the technical problems of rigid resource allocation strategies, passive and delayed response, and the separation of security and resource management in the prior art.

[0008] Another aspect of the present invention is to provide a system for optimizing cloud platform resource allocation based on an LSTM model, which achieves systematic support for the above-mentioned methods through modular design.

[0009] To achieve the above objectives, this invention provides a method for optimizing cloud platform resource allocation based on an LSTM model. The method includes the following steps: First, collecting and preprocessing multi-dimensional heterogeneous time-series data from the cloud platform. Then, based on a pre-trained machine learning model, inputting a time-series feature vector constructed from the time-series data to predict resource load within a predetermined future time window. Simultaneously, the method also includes real-time reception and parsing of security events from the cloud platform's security system. Next, based on the predicted resource load and the parsed security events, and in conjunction with a pre-defined collaborative strategy library, generating resource scheduling instructions. Finally, executing the resource scheduling instructions to elastically expand or dynamically reclaim resources on the cloud platform.

[0010] In a preferred embodiment, the multi-dimensional heterogeneous time-series data includes at least three layers: first, infrastructure layer data, such as at least one of the following: CPU utilization of the central processing unit of a cloud host, memory utilization, network input / output I / O, and disk read / write operations per second (IOPS). This data directly reflects the physical consumption of underlying resources. Second, platform service layer data, such as at least one of the following: queries per second (QPS) of database services, cache hit rate, request volume of object storage services, and concurrent connections of load balancing services. This data reflects the operating status of key components in the PaaS layer. Third, business and security layer data, such as at least one of the following: API response latency, API call volume, and security alarm events. This data is closer to the end-user's business experience and the system's security posture.

[0011] In another preferred embodiment, to achieve accurate prediction, the machine learning model can be a Long Short-Term Memory (LSTM) network model. The steps for prediction based on the pre-trained LSTM model specifically include: First, applying a sliding window to preprocessed historical multi-dimensional time-series data to generate multiple training samples. The feature data of the training samples also includes security event markers and attack traffic features from historical periods. Then, the LSTM model is trained using the multiple training samples until the model converges, thereby obtaining the pre-trained machine learning model. In actual operation, the same preprocessing is performed on the multi-dimensional heterogeneous time-series data collected in real time by the cloud platform to construct the time-series feature vector. Finally, the time-series feature vector is input into the pre-trained machine learning model, which can then output a predicted curve of resource load within the predetermined future time window, providing a basis for forward-looking scheduling.

[0012] In a further preferred embodiment, to improve the smoothness of the resource recovery process and business stability, the dynamic recovery can be a parabolic smooth recovery. Specifically, this recovery step may include: after the business peak subsides, determining a dynamic resource retention level Watermark(t) that varies with time t based on a preset total recovery period T and a decay factor α. The key is to set the decay factor α to a value greater than 1. The specific calculation formula for the dynamic resource retention level Watermark(t) can be expressed as: Watermark(t) = Base_Level + (Peak_Level - Base_Level) * (1 - (t / T)) αIn this formula, t is the time variable starting from the start of recycling, Peak_Level is the peak resource level at the start of recycling, and Base_Level is the target baseline resource level to be reached at the end of recycling. Finally, the system uses the dynamic resource retention level calculated by this formula to gradually and non-linearly reduce the resource instances of the cloud platform, thereby achieving a smooth transition.

[0013] In another preferred embodiment, to achieve deep integration of security and resources, the step of receiving and parsing security events in real time may specifically include: subscribing to security event messages published by the security system through a message queue, such as Kafka or RabbitMQ, to achieve asynchronous decoupling and real-time communication between systems. Correspondingly, the step of generating resource scheduling instructions is also specially designed: when a security event message is received, the system immediately matches a predefined resource operation corresponding to the security event type from the coordination policy library, and generates a high-priority resource scheduling instruction accordingly. This resource operation is coordinated and may simultaneously include resource operations on security facilities (such as Web Application Firewalls (WAFs) and coordinated resource operations on affected business services (such as isolating or supplementing healthy instances).

[0014] To achieve the above objectives, this invention also provides a system for optimizing cloud platform resource allocation based on an LSTM model. The system includes: a data acquisition and preprocessing module, a resource load prediction module, a security and resource coordination module, a decision module, and an instruction executor. The data acquisition and preprocessing module is used to acquire and preprocess multi-dimensional heterogeneous time-series data from the cloud platform. The resource load prediction module is used to predict the resource load within a predetermined future time window based on a pre-trained machine learning model and a time-series feature vector constructed from the time-series data. The security and resource coordination module is used to receive and parse security events from the cloud platform's security system in real time. The decision module, coupled to the resource load prediction module and the security and resource coordination module, has the core function of making decisions, i.e., generating resource scheduling instructions based on the predicted resource load and the parsed security events, combined with a pre-defined coordination strategy library. Finally, the instruction executor is used to execute the resource scheduling instructions generated by the decision module, completing the actual operation of elastic scaling or dynamic reclamation of resources by calling the cloud platform's API interface.

[0015] In a preferred system implementation, the data dimensions collected by the data acquisition and preprocessing module correspond to the data dimensions described in the foregoing method embodiments, and similarly cover the infrastructure layer, platform service layer, and business and security layer.

[0016] In another preferred system implementation, the resource load prediction module is specifically configured to use a Long Short-Term Memory (LSTM) network model as its core machine learning model. The module's internal functional logic includes complete model lifecycle management: generating training samples by applying a sliding window to historical data, training the LSTM model using these samples, and finally inputting the feature vector constructed from real-time data into the trained model to obtain the prediction curve.

[0017] In a further preferred system implementation, the decision module may further include a dynamic resource reduction and recovery module, which is specifically configured to execute the aforementioned parabolic smooth recovery method. Specifically, after a business peak subsides, it calculates a dynamic resource retention level (Watermark(t)) based on a preset period T and a decay factor α greater than 1, and generates corresponding instructions for gradually reducing cloud platform resource instances accordingly.

[0018] In another preferred system implementation, the security and resource coordination module is specifically configured to subscribe to security event messages via a message queue. Upon receiving a message, the core task of this module is to match the corresponding policy from the coordination policy library, generate a high-priority resource scheduling instruction, and then send this instruction to the decision-making module for unified instruction distribution. The resource operations here are also coordinated, ensuring the linkage between security and business operations.

[0019] This invention proposes a closed-loop cloud platform resource optimization and allocation method and system integrating multi-dimensional data perception, machine learning prediction, dynamic strategy decision-making, and security event linkage. First, the technical solution of this invention provides a comprehensive and multi-dimensional data foundation for accurate decision-making by collecting data covering multiple dimensions such as infrastructure, platform services, business, and security. Second, it introduces advanced machine learning models such as Long Short-Term Memory (LSTM) networks to achieve forward-looking prediction of future resource load trends, shifting from passive response to proactive prediction and solving the response lag problem of traditional strategies. Third, this invention designs a parabolic smoothing recycling algorithm, which ensures a smooth and seamless resource scaling-down process after business peaks through non-linear resource recycling rate control, effectively guaranteeing business stability. Most importantly, this invention, by constructing an event-driven architecture based on message queues, achieves for the first time deep and automated linkage between the security system and the resource management system, transforming passive security alarms into proactive, intelligent instructions that drive resource collaborative responses, solving the technical challenge of balancing security and performance in attack scenarios.

[0020] Compared with the prior art, the beneficial effects of the present invention are mainly reflected in the following aspects: First, it significantly improves resource utilization efficiency. Through accurate prediction and refined scheduling, this invention enables the cloud platform's resource supply curve to closely match the actual business load curve, avoiding significant resource idleness and waste caused by static redundant configuration and extensive recycling. After applying this invention, the average CPU utilization rate of the cloud platform can be significantly increased from the low level of the traditional model to over 85%, thereby directly translating into significant operating cost savings.

[0021] Second, it comprehensively enhances business continuity and service stability. A forward-looking expansion mechanism ensures that resources are ready before peak business periods, achieving an ideal state where resources are readily available for business operations, fundamentally avoiding the risk of service delays or interruptions due to insufficient resources. Parabolic smooth resource reclamation avoids the impact of resource reclamation on business operations, further guaranteeing system stability.

[0022] Third, it deeply integrates security and resource scheduling to build a highly available proactive defense system. This invention, through the synergistic linkage of security and resources, enables the cloud platform to intelligently and collaboratively expand security facilities and ensure normal business resources when facing resource-intensive network attacks. This achieves the dual goals of security protection and business stability, greatly enhancing the platform's business resilience.

[0023] Fourth, it enables intelligent and automated operation and maintenance management, significantly improving operational efficiency. The closed-loop intelligent system of this invention automates complex resource planning and emergency response tasks, greatly reducing the need for manual intervention, minimizing the possibility of human error, and freeing the operation and maintenance team from tedious and repetitive tasks, allowing them to focus on higher-value business innovation. Attached Figure Description

[0024] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0025] Appendix Figure 1 This is a system architecture diagram for optimizing cloud platform resource allocation disclosed according to an embodiment of the present invention; Appendix Figure 2 This is a flowchart of a method for optimizing cloud platform resource allocation according to an embodiment of the present invention; Appendix Figure 3 This is a flowchart of the training and prediction process of a resource load prediction module disclosed in an embodiment of the present invention. Appendix Figure 4This is a schematic diagram comparing the parabolic recovery strategy disclosed in an embodiment of the present invention with existing recovery strategies; Appendix Figure 5 This is a flowchart of the security and resource collaboration module disclosed in an embodiment of the present invention; Appendix Figure 6 This is a simplified architecture and collaborative scenario diagram of the provincial-level intelligent transportation service cloud platform according to the third specific embodiment of the present invention. Appendix Figure 7 This is a schematic diagram of a security protection device cluster deployment disclosed in one embodiment of the present invention; Appendix Figure 8 This is a graph showing the convergence curve of the training loss of the LSTM model in this invention. Appendix Figure 9 This is a heatmap showing the correlation between multidimensional heterogeneous data acquisition in this invention.

[0026] Appendix Figure 10 This is a time series curve of the resource load prediction accuracy of the present invention. Detailed Implementation

[0027] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0028] It should be noted that, in the description of this invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0029] Example 1 This embodiment provides a method for optimizing cloud platform resource allocation. The subject of this method can be a software system or service deployed at the cloud platform's management layer. (See attached diagram.) Figure 2 The method may include the following steps: Step S201: Collect and preprocess multi-dimensional heterogeneous time series data.

[0030] In this step, the system needs to establish a comprehensive and accurate data foundation for subsequent intelligent decision-making. Data collection is multi-dimensional, aiming to depict the operational status of the cloud platform from different perspectives. Specifically, the collected data can be divided into three levels: The first layer is the infrastructure layer data. This layer directly reflects the consumption of underlying physical or virtual resources. For example, you can collect metrics such as average CPU utilization, CPU load, memory usage, network inbound and outbound bandwidth, and disk read / write IOPS (Input / Output Operations Per Second) for each instance by deploying a lightweight agent within the Elastic Compute Service (ECS) instance or by calling the monitoring API provided by the cloud platform. The collection frequency of this data can be set to a high level, such as once every 10 seconds or once every minute.

[0031] The second layer is the platform service layer data. Deep monitoring is also needed for the core components of the PaaS (Platform as a Service) layer. For example, for relational database services (RDS), the system needs to collect metrics such as QPS (Queries Per Second), TPS (Transactions Per Second), active connections, and slow query count. For distributed caching services (such as Redis), it's necessary to monitor key hit rate, memory fragmentation rate, and eviction frequency. For object storage services (OSS), the upload / download request rate and bandwidth are important. For load balancing services (SLB), concurrent connections and new connection rate are core monitoring metrics. This data reflects the pressure exerted on the platform services by upper-layer applications.

[0032] The third layer is the business and security layer data. This layer of data is closest to the end-user experience and system security posture. Application Performance Monitoring (APM) tools can collect business metrics such as P95 / P99 response latency, error rate, and calls per minute (RPM) for key business interfaces (APIs). Simultaneously, the system needs to access security event logs published by cloud security services (such as Cloud Shield and WAF) through subscription or API polling. These logs include, but are not limited to, DDoS attack alerts, web application attack interception records, host intrusion detection alerts, and real-time updated lists of malicious IP addresses.

[0033] After collecting the raw data, preprocessing is required. Preprocessing steps mainly include: data cleaning, which handles missing values ​​caused by network jitter (using linear interpolation, mean imputation, or nearest-neighbor prediction imputation), and identifying and removing obvious outliers or spikes (using the 3-Sigma criterion or box plot method for anomaly detection). Time series alignment: Since data collection periods from different sources may differ (e.g., CPU data every 10 seconds, RDS connection count every 1 minute), all data needs to be aligned to a standard time granularity, such as uniformly aggregating to a 1-minute average, maximum, or sum. Data normalization: To eliminate large differences in units between different metrics (e.g., CPU utilization between 0-100, while network bandwidth may be as high as Gbps), min-max scaling or Z-score normalization methods are used to map all time series data to similar numerical ranges (e.g., [-1, 1]), which is crucial for the stable training of subsequent machine learning models.

[0034] Furthermore, for non-numerical features contained in the collected data, such as security alarm event types in the business and security layer data (e.g., DDoS_Attack_Start, Web_Scan, Malicious_IP_Blocked), numerical encoding is required before they can be used as input to the machine learning model. In an optional implementation, the present invention may employ one or more of the following encoding techniques: 1. One-Hot Encoding: For features with a limited number of categories and no ordinal relationship between them, one-hot encoding can be used. For example, assuming there are 3 alarm event types, DDoS_Attack_Start can be encoded as a vector [1, 0, 0], Web_Scan can be encoded as [0, 1, 0], and Malicious_IP_Blocked can be encoded as [0, 0, 1]. These vectors can be concatenated with other numerical features to form the input of the LSTM model at a certain time step.

[0035] 2. Word embedding: In a preferred embodiment, word embedding techniques can be employed to capture potential semantic relationships between different alert event types and handle situations with a large number of categories. Specifically, each unique alert event type is first mapped to a unique integer index (e.g., DDoS_Attack_Start -> 1, Web_Scan -> 2). Then, an embedding layer is constructed before the input layer of the LSTM model. This embedding layer is a trainable weight matrix that takes integer indices as input and outputs a low-dimensional, dense vector of floating-point numbers (i.e., embedding vectors). For example, each alert event type can be mapped to an 8-dimensional or 16-dimensional embedding vector. During model training, the weights of the embedding layer are optimized along with other model parameters, so that semantically similar event types (e.g., DDoS_Attack_Start and CC_Attack_Detected) have closer embedding vectors in the vector space. In this way, the model can gain a deeper understanding of the potential impact of different security events on resource load. Finally, at each time step, the embedding vector is concatenated with other numerical features to form a complete feature vector, which is then input into the core network of the LSTM.

[0036] By adopting the above encoding method, the present invention can effectively integrate multi-dimensional heterogeneous time series data, including numerical and non-numerical features, into the prediction model, thereby improving the prediction accuracy and generalization ability of the model.

[0037] like Figure 9 As shown, this invention demonstrates the correlation analysis results between multidimensional heterogeneous time-series data on a cloud platform. The heatmap is divided into three data layers: the infrastructure layer (CPU utilization, memory utilization, network I / O, disk IOPS), the platform service layer (RDS-QPS, cache hit rate, OSS request volume, SLB concurrency), and the business and security layer (API latency, API call volume, security alarms). The color intensity in the graph represents the correlation coefficient between each indicator, ranging from -1 to 1. The heatmap shows that indicators within the same layer typically exhibit high positive correlations; for example, the correlation coefficient between CPU utilization and memory utilization is 0.85, and the correlation coefficient between RDS-QPS and SLB concurrent connections is 0.78. Meanwhile, API response latency is negatively correlated with API call volume (-0.72), indicating that increased call volume leads to increased latency. By analyzing these cross-layer correlation characteristics, this invention can more accurately construct time-series feature vectors and improve the accuracy of the LSTM prediction model.

[0038] Step S202: Construct temporal feature vectors.

[0039] In this step, the preprocessed data needs to be structured into a format suitable for input to a machine learning model. For time series prediction problems, a sliding window approach is typically used to construct features and labels. Specifically, the system sets a time step, for example, T=120, representing the use of data from the past 120 minutes. Then, the system moves this window along the time axis, moving one time unit (1 minute) at a time. At each window position, the monitoring data (after preprocessing) from all N dimensions at T time points within the window together constitute a T x N feature matrix. This feature matrix is ​​a time series feature sample. The label corresponding to this sample can be the actual value of a key resource indicator at the next or future time points after the window ends (e.g., the average CPU utilization over the next 30 minutes). By continuously sliding the window across the entire historical dataset, a large number of (feature, label) sample pairs can be generated, forming a dataset for model training and prediction.

[0040] Step S203: Predict future resource load based on the pre-trained LSTM model.

[0041] This step is the core of achieving forward-looking resource scheduling. This embodiment preferably uses a Long Short-Term Memory (LSTM) network as the prediction model. Cloud platform resource loads typically exhibit complex nonlinear characteristics, interspersed with obvious periodicity (e.g., daily or weekly cycles) and trends. Traditional statistical models (such as ARIMA) struggle to capture these complex long-term dependencies. LSTM, as a special type of recurrent neural network (RNN), with its unique gating structure (forget gate, input gate, output gate), can effectively learn and remember long-term dependency patterns in time-series data, making it very suitable for this scenario.

[0042] See attached document Figure 3 This step can be divided into two stages: training and prediction.

[0043] During the offline training phase, the network architecture of the LSTM model needs to be defined first. A typical architecture may include: an input layer whose shape matches the feature matrix constructed in step S202; one or more stacked LSTM hidden layers, such as two layers, each containing 64 or 128 LSTM units; dropout layers can be added between the LSTM layers to prevent overfitting; and finally, one or more fully connected layers to map the output of the LSTM layers to the final prediction target. The number of neurons in the output layer depends on the prediction task; for example, to predict CPU demand over the next 60 minutes, there could be 60 output neurons.

[0044] Then, the model is trained using sample pairs (features, labels) generated from historical data. During training, Mean Squared Error (MSE) or Mean Absolute Error (MAE) is used as the loss function. An efficient optimizer, such as Adam (Adaptive Moment Estimation), is used to iteratively update the model's weight parameters through backpropagation until the loss function converges to a sufficiently small value. After training, the resulting model weights are saved, forming a pre-trained model. This training process can be performed periodically (e.g., daily or weekly) to achieve incremental learning of the model and adapt to the evolution of business models.

[0045] During the online prediction phase, the system collects and preprocesses data from the latest time window (the past 120 minutes) in real time, constructing an input feature vector. This vector is then input into the pre-trained model. The model performs a single forward propagation calculation, outputting a sequence of predicted values ​​for one or more key resource indicators for a predetermined future time window (e.g., the next 60 minutes). This sequence constitutes the prediction curve for future resource load, which is then passed to subsequent decision-making modules.

[0046] In a preferred embodiment, the LSTM model includes two hidden layers, each containing 128 neurons. To prevent overfitting, a dropout layer with a dropout rate of 0.2 is configured between the two LSTM layers. The optimizer is Adam, with an initial learning rate set to 0.001 and a batch size of 64.

[0047] like Figure 10 As shown, this invention demonstrates the prediction performance of the LSTM resource load prediction model within a typical 24-hour period. The horizontal axis represents time (hours), ranging from 0 to 24 hours; the vertical axis represents resource load (percentage), ranging from 0 to 100%. The solid dotted curve represents the actual resource load, exhibiting a clear daily periodic variation, with peak business hours from 6 AM to 6 PM, where the load can reach 70-80%. The dashed dotted curve represents the predicted load of the LSTM model, closely matching the actual load curve; the gray filled area represents the 95% confidence interval. The horizontal dashed line marks the 70% capacity expansion level, and the double arrow indicates a prediction lead time of 20 minutes. Statistics in the upper left corner show a mean absolute percentage error (MAPE) of approximately 5%, indicating that the model has high prediction accuracy and can effectively support forward-looking resource scheduling decisions.

[0048] Step S204: Receive and parse security events in real time.

[0049] Alongside load forecasting, the system also needs to constantly monitor the platform's security posture. To achieve low-latency and highly reliable communication, this embodiment preferably uses a message queue based on a publish / subscribe model (such as Apache Kafka or RabbitMQ) as the communication bus between the security system and this resource scheduling system.

[0050] Specifically, various security services on the cloud platform (such as WAF, DDoS protection, and host security) act as producers. When a specific security event is detected, the platform immediately encapsulates the event into a structured message and publishes it to a designated topic in the message queue. This message is typically in JSON format and may include: the event's unique ID, the time the event occurred, the event type (e.g., DDoS_Attack_Start, Web_Scan, Malicious_IP_Blocked), the severity level (e.g., high, medium, low), the attack target (e.g., the IP address or domain name of a service), the attack source information (e.g., source IP, geographical distribution), and the attack's traffic characteristics (e.g., protocol distribution, request rate).

[0051] The security and resource collaboration module in this system acts as a consumer, continuously subscribing to these security-related topics. Once a new message is published, the module receives it within milliseconds, immediately parses it, and extracts all key fields for subsequent decision-making.

[0052] Step S205: Determine if a high-priority security event exists.

[0053] After parsing a security event, the system needs to determine whether immediate resource coordination intervention is required based on the event's type and severity. Not all security events require resource scheduling. Therefore, the system maintains a list of high-priority events. For example, resource-intensive attacks such as DDoS_Attack_Start and CC_Attack_Detected, or all "high-risk" events during a specific period (such as during an attack and defense exercise), can be defined as high-priority events. If the currently parsed event belongs to this list, the process proceeds to step S206; otherwise, the event may only be recorded or trigger other non-resource-related responses, and the resource scheduling process continues based on the prediction results, i.e., proceeding to step S207.

[0054] Step S206: Generate security-driven resource scheduling instructions based on the collaborative strategy library.

[0055] When a high-priority security event is identified, the system needs to respond immediately. The response logic and specific operations are predefined in a coordination policy library. This policy library is a configurable set of rules that maps different security event types to one or more sets of resource scheduling operations.

[0056] For example, the following coordination policy might be defined in the policy library for the DDoS_Attack_Start event: Strategy 1 (Security Infrastructure Enhancement): Immediately perform emergency scaling operations on the load balancer (SLB) instance and web application firewall (WAF) cluster at the front end of the attacked target, and upgrade their specifications (such as maximum concurrent connections and QPS processing capacity) to a preset high level to ensure sufficient capacity to absorb and clean up attack traffic.

[0057] like Figure 7 As shown, this invention demonstrates a multi-layered cluster deployment architecture for cloud platform security protection devices. From top to bottom, it consists of four layers: the top layer is the external traffic entry point, where all requests from the internet enter; the first protection layer is a DDoS traffic scrubbing layer with a processing capacity of 100Gbps, containing three parallel scrubbing nodes responsible for identifying and filtering distributed denial-of-service attack traffic; the second protection layer is a Web Application Firewall (WAF) cluster containing five WAF nodes responsible for detecting and blocking application-layer attacks such as SQL injection and cross-site scripting (XSS); the third layer is a load balancing (SLB) layer containing three SLB devices, which evenly distributes the securely scrubbed legitimate traffic to the backend business service cluster; the bottom layer is the backend ECS business service cluster, divided into three service groups: Service A, Service B, and Service C. The security and resource coordination module marked on the right is connected to each protection layer via bidirectional arrows, indicating that resource configurations at each layer can be adjusted in conjunction when an attack is detected.

[0058] Strategy 2 (Business Resource Assurance): Identify the core business service under attack and immediately supplement the resource group (or cluster) where the service is located with a certain number of healthy instances, which are dedicated to handling legitimate user traffic after being cleaned by security facilities, to prevent the exhaustion of normal service resources due to the attack.

[0059] Strategy 3 (Access Control Linkage): The malicious source IP addresses identified by the security system are synchronized in real time to the network access control lists (ACLs) or security group rules of relevant resources (such as ECS instances and RDS databases) to block the attack source at the network level.

[0060] Furthermore, the collaborative strategy library also includes refined resource mapping rules for different types of security threats to achieve multi-dimensional protection: Strategy 4 (Malicious Crawler Control): When the security system detects high-frequency malicious crawler events targeting a specific interface, the generated resource scheduling instruction does not expand capacity, but instead triggers a resource limiting operation. Specifically, it calls the API gateway or SLB interface to rate limit the bandwidth of the channel where the request originates from the specific fingerprint, and simultaneously sends an instruction to the backend service to downgrade it to 'static page mode' (only returning cached data and not consuming database resources), thereby protecting core data without increasing costs.

[0061] Strategy 5 (Mining / Ransomware Isolation): When the host security component detects an abnormal spike in CPU utilization on a cloud host accompanied by known mining protocol outbound behavior, the coordination module should not identify it as normal business load and scale it up. Instead, it should apply the 'isolation policy'. The instruction executor will immediately modify the security group rules of the cloud host, cutting off its outbound traffic, isolating its VLAN, and freezing the instance's Auto Scaling Group (ASG) permissions to prevent the abnormal instance from being automatically replicated.

[0062] Strategy Six (Authentication Brute-Force Attack): When a brute-force attack targeting the login interface is detected, in addition to the usual IP blocking, the policy library will trigger temporary resource adjustments to the Authentication Service. Considering that the attack may cause a surge in CPU-intensive hash calculations, the instruction executor will automatically increase the computing specifications of the authentication microservice and simultaneously enable resource preloading for the CAPTCHA challenge page on the front-end load balancer to balance user experience and security.

[0063] Finally, based on the matched strategy, the system generates one or more specific, high-priority resource scheduling instructions. Simultaneously, the traffic characteristics of this attack (such as request rate and protocol distribution) can be fed back as a special event feature to the resource load prediction module in step S203. This allows the model to learn this type of attack pattern during future training, potentially predicting similar attack traffic in the future, thus achieving an upgrade from passive coordination to proactive prediction.

[0064] Step S207: Generate regular resource scheduling instructions based on the prediction results and preset strategies.

[0065] In the absence of high-priority security events, or after security events have been resolved, the system will make routine resource scheduling decisions based on the resource load prediction curve obtained in step S203. This mainly includes two operations: elastic expansion and dynamic resource reclamation.

[0066] For elastic scaling, the system employs a dual-mode strategy, primarily prediction-driven and secondarily event-driven. The primary mode is prediction-driven: the system analyzes the prediction curve, and if it detects that the load is about to exceed the "proactive scaling threshold" (e.g., 70% resource utilization) within a preset time window (e.g., 15 minutes in advance), it triggers the scaling process in advance, smoothly increasing resources. This ensures that resources are ready before the business load arrives. As a supplement and safeguard, the system retains an event-driven scaling mechanism based on real-time performance metrics (e.g., CPU utilization instantaneously exceeding 90%) to handle sudden traffic surges that the model cannot predict.

[0067] In one specific implementation, prediction-driven expansion instructions have a regular priority. However, when an event-driven mechanism (e.g., detecting a sudden spike in P99 latency exceeding a threshold) is triggered, a higher-priority emergency expansion instruction is generated. The decision module or instruction executor prioritizes executing higher-priority instructions when processing the instruction queue. Furthermore, an emergency expansion event itself can be considered a feature, input into the LSTM model to help the model learn and identify such unpredictable burst traffic patterns, potentially transforming such events into predictable ones in the future.

[0068] In a preferred embodiment, to achieve adaptive adjustment of the resource reclamation rate, the system first needs to quantitatively assess the overall health status of the cloud platform, which is defined in this invention as the Business Security Level (BSL). BSL is a comprehensive health score; a higher score indicates a healthier and more stable system. In this embodiment, the system uses a weighted normalization model to calculate BSL in real time, as shown in the following formula: BSL = W_app × Norm(S_app) + W_res × Norm(S_res) + W_sec × (1 -Norm(S_sec)) in: S_app (Application Stability Metric): This is the success rate (SuccessRate) of the core business API over the past 5 minutes. For example, if the success rate is 99.99%, then Norm(S_app) is 99.99.

[0069] S_res (Resource Health Index): This is the inverse of the smoothness of the utilization rate of key resources (such as CPU and memory). Specifically, it can be measured by calculating the first-order difference variance of resource utilization over the past 5 minutes. The smaller the variance (the smaller the fluctuation), the higher the score, indicating a more stable resource status.

[0070] S_sec (Security Threat Index): This index is input by the security system and is defined as: ∑(Event Level Weight × Number of Events). For example, a high-risk event has a weight of 10, and a medium-risk event has a weight of 5. The system calculates the total score of currently unprocessed security alarms and normalizes it to the interval [0, 100]. The formula uses (1-Norm) because the greater the security threat, the lower the threshold should be.

[0071] W_app, W_res, and W_sec are the weighting coefficients for application, resources, and security, respectively. The default values ​​are 0.5, 0.3, and 0.2, respectively, and they can be dynamically adjusted according to business preferences.

[0072] This formula allows the system to abstract heterogeneous operational states into a score between 0 and 100. When the system is attacked (S_sec increases) or the business error rate fluctuates (S_app decreases), the BSL value will automatically decrease, thereby slowing down the resource reclamation rate through the adjustment function f(BSL) to achieve risk awareness.

[0073] For dynamic recycling, the system will initiate the recycling process when the forecast curve indicates that the load will continue to decline over a future period. (See attached image) Figure 4 As shown, this embodiment employs an innovative parabolic smooth recovery algorithm. The core of this algorithm is to calculate a dynamically changing resource retention level based on a formula containing a decay factor α>1. This level decreases slowly in the initial recovery phase, providing the system with a sufficient buffer period, and then the recovery rate gradually increases. This avoids the performance fluctuations caused by traditional step or linear recovery methods.

[0074] See attached document Figure 4 This graph visually compares three different resource reclamation strategies. The horizontal axis represents the time (in minutes) after the end of a business peak, and the vertical axis represents the resource retention (using vCPU cores as an example). In the graph, the step reclamation (existing technology) causes the resource quantity to drop instantly from 100 cores to 20 cores after t=0, a sudden change that can easily cause system performance problems. Linear reclamation (existing technology) causes the resource quantity to decrease evenly over time. The dashed line marked with a triangle represents the parabolic reclamation (α=2) of this invention. It can be seen that in the method of this invention, the resource retention decreases very slowly in the initial stage of reclamation (e.g., 0-20 minutes), providing the system with sufficient buffer time; while in the later stage of reclamation (e.g., 30-60 minutes), the rate of decrease accelerates significantly to achieve cost-effectiveness. This slow-then-fast curve effectively balances business stability and resource reclamation efficiency.

[0075] Based on the above strategy, the system will generate specific expansion (e.g., "add 2 ECS instances with 8 vCPUs to cluster A") or recycling (e.g., "remove 1 instance from cluster B") instructions.

[0076] In one specific implementation, the parabolic smoothing recycling algorithm ensures that the resource recycling rate is slow in the initial stage, providing sufficient buffer time for potentially long-lived connections and slow tasks in the system. Then, as time goes on, the recycling rate gradually increases, eventually adjusting the resources to the baseline level of the stable period. This dynamic process of slow at first and then fast later can be described by the right half of an upward-opening parabolic function.

[0077] In the specific implementation, a dynamic resource retention water level, Watermark(t), is defined that varies with time t, and its calculation formula is as follows: Watermark(t)=Base_Level+(Peak_Level-Base_Level)*(1-(t / T) α ) in: t: The time variable starting from the start of recycling, 0 ≤ t ≤ T.

[0078] Watermark(t): The target number of resources (e.g., vCPU cores) that the system should retain at time t. The reclamation module will ensure that the actual amount of resources does not fall below this watermark.

[0079] Peak_Level: The amount of resources at the start of reclamation (t=0), i.e., the peak resource amount.

[0080] Base_Level: The target resource amount at the end of the recycling process (t=T), which is the baseline resource amount during the business stability period. This value is predicted by the LSTM model.

[0081] T: The preset total recycling period, for example, 30 minutes. This period can be configured according to business characteristics.

[0082] α: Decay factor (or curvature factor), is a key parameter of this algorithm, set to α>1 (e.g., α = 2). When α=1, the recycling is linear; when α>1, the function curve is parabolic, achieving a decay effect that is slow at first and then fast.

[0083] Preferably, the total recovery period T is chosen in relation to the average lifespan of the business session. For businesses with a large number of long-lived connections (such as online collaborative documents), T can be set to a longer value, such as 60 minutes, to ensure a smooth session termination. For stateless, short-connection businesses, T can be set to a shorter value, such as 15 minutes, to accelerate cost recovery. The choice of the attenuation factor α is related to the business's sensitivity to performance fluctuations. For sensitive businesses such as financial transactions, α can be set to a larger value (such as 2.5 or 3.0) to provide a longer initial buffer; for non-sensitive businesses such as offline batch processing, α can be set to a smaller value close to 1 (such as 1.5) to balance smoothness and recovery efficiency.

[0084] Step S208: Execute resource scheduling instructions (expansion or reclamation).

[0085] Whether it's a high-priority instruction from step S206 or a regular instruction from step S207, both will ultimately be sent to an instruction executor module. This module is responsible for translating logical scheduling instructions into actual calls to the underlying cloud platform APIs. For example, it calls APIs provided by the cloud vendor, such as CreateInstance, DeleteInstance, and ModifyInstanceAttribute, and processes the return results of these API calls to ensure successful instruction execution. After execution, the entire process forms a closed loop, and the system returns to step S201 for the next round of monitoring, prediction, and decision-making.

[0086] Example 2 This embodiment provides a system for optimizing cloud platform resource allocation, which can be deployed on one or more servers. The servers can be general-purpose physical servers or virtual machines, and their hardware configuration includes, but is not limited to, processors (CPU), memory (RAM), storage devices (such as SSDs), and network interfaces. (See attached diagram.) Figure 1 The system 100 may include the following modules: a data acquisition and preprocessing module 110, a resource load prediction module 120, a dynamic downsizing and recovery module 130, an elastic expansion decision module 140, a security and resource coordination module 150, and an instruction executor 160. Among them, the dynamic downsizing and recovery module 130 and the elastic expansion decision module 140 can be integrated or regarded as a generalized decision module.

[0087] The data acquisition and preprocessing module 110 is mainly responsible for serving as the system's data source. It is responsible for comprehensively and in real-time collecting multi-dimensional and multi-level operational data from the underlying cloud platform infrastructure and service layer, and for standardizing these raw data for use by other modules.

[0088] In its implementation, module 110 can be a distributed data collection framework. It includes a set of collection agents deployed within the target cloud host. These agents use lightweight system calls to acquire infrastructure layer data such as CPU, memory, disk, and network. For platform services like RDS and SLB, this module can pull monitoring data by sending periodic requests to the API endpoints of cloud platform monitoring services (such as Alibaba Cloud's CloudMonitor and AWS's CloudWatch). For business and security layer data, this module can provide a data receiving endpoint, allowing APM and security systems to proactively send data via HTTP push or writing to a message queue. After data aggregation, the module's internal data processing pipeline can consist of a series of data processing services, such as using stream processing engines like Apache Flink or Spark Streaming to perform preprocessing tasks like data cleaning, time window alignment, and normalization. The cleaned data can be stored in a time-series database (such as InfluxDB or Prometheus) for other modules to query.

[0089] The core function of the resource load prediction module 120 is to use machine learning technology to accurately predict resource demand for a period of time in the future, providing a core basis for the system's forward-looking decision-making.

[0090] In its implementation, module 120 can be an AI service built using the Python language and deep learning frameworks (such as TensorFlow or PyTorch). (See attached image.) Figure 3 As shown, the implementation of this module includes two stages: "offline training" and "online prediction." The offline training stage is responsible for periodically pulling long-term historical data from the time-series database, performing the feature engineering, model definition, training, and evaluation steps detailed in Example 1, and serializing the trained model (including network structure and weight parameters) and storing it in a model registry. The online prediction stage is a persistent service that periodically retrieves the latest time-series data window from the data acquisition module, loads the latest production model from the model registry, performs model inference, and publishes the output prediction curve results to the system's internal message bus or a high-speed cache (such as Redis) for the decision module to subscribe to and use.

[0091] See attached document Figure 3The detailed process of resource load prediction in this invention is divided into two stages. In the offline training stage, the system first obtains historical multidimensional monitoring data from the historical database, and then sends this data to the data preprocessing unit for the aforementioned cleaning, alignment, and normalization operations. Next, in the temporal feature engineering unit, the preprocessed data is constructed into a large number of (feature, label) sample pairs using a sliding window approach. These samples are used to define the LSTM model architecture, which may include hidden layers and Dropout layers. Subsequently, in the model training unit, the model is trained using the backpropagation algorithm and an optimizer (such as Adam) until the loss function converges. The final pre-trained model is saved for online use.

[0092] During the online prediction phase, the system acquires real-time multidimensional monitoring data from the monitoring source and performs the same real-time data preprocessing as in the offline phase. The preprocessed data is constructed into an input feature vector, which, along with the loaded pre-trained model, is fed into the model inference / prediction unit. This unit performs one forward propagation and ultimately outputs a future resource load prediction curve, providing a basis for subsequent scheduling decisions.

[0093] In this embodiment, the decision-making module is jointly operated by the dynamic downsizing and recovery module 130 and the elastic expansion decision-making module 140. This module is responsible for formulating specific resource scheduling strategies based on prediction results and security events.

[0094] In its implementation, the elastic scaling decision module 140 can be a service combining a rule engine and an algorithm. It subscribes to the prediction curves published by the resource load prediction module 120. Its internal prediction-driven logic continuously analyzes the slope and future values ​​of the curves. Once it predicts that the load will exceed the preset scaling threshold, it calculates the required resource increment and generates scaling instructions. Its event-driven logic subscribes to real-time monitoring data; when it detects a sudden and abnormal spike in certain key indicators (such as P99 latency), it immediately triggers an emergency scaling procedure.

[0095] In its implementation, the dynamic resource reduction and reclamation module 130 is a dedicated algorithm service. When it learns from the prediction module 120 that the load will enter a low period in the future, it will initiate a parabolic smooth reclamation algorithm. This service internally maintains a state machine that records parameters such as the start time t, peak resource Peak_Level, and base resource Base_Level for each reclamation task. It periodically calculates the amount of resources that should be retained at the current moment according to the Watermark(t) formula, and then compares it with the actual resource amount to generate refined, progressively decreasing reclamation instructions.

[0096] After peak business periods subside, resource reclamation should be a soft landing, not a precipitous drop. This algorithm ensures a slow initial rate of resource reclamation, providing ample buffer time for potentially long-running connections and slow tasks. Then, over time, the reclamation rate gradually increases, eventually adjusting resources to a baseline level suitable for a stable period. This dynamic process, initially slow and then accelerating, can be described by the right half of an upward-opening parabolic function.

[0097] In the specific implementation, a dynamic resource retention water level, Watermark(t), is defined that varies with time t, and its calculation formula is as follows: Watermark(t)=Base_Level+(Peak_Level-Base_Level)*(1-(t / T) α ) in: t: The time variable starting from the start of recycling, 0 ≤ t ≤ T.

[0098] Watermark(t): The target number of resources (e.g., vCPU cores) that the system should retain at time t. The reclamation module will ensure that the actual amount of resources does not fall below this watermark.

[0099] Peak_Level: The amount of resources at the start of reclamation (t=0), i.e., the peak resource amount.

[0100] Base_Level: The target resource amount at the end of the recycling process (t=T), which is the baseline resource amount during the business stability period. This value is predicted by the LSTM model.

[0101] T: The preset total recycling period, for example, 30 minutes. This period can be configured according to business characteristics.

[0102] α: Decay factor (or curvature factor), is a key parameter of this algorithm, set to α>1 (e.g., α = 2). When α=1, the recycling is linear; when α>1, the function curve is parabolic, achieving a decay effect that is slow at first and then fast.

[0103] The Security and Resource Collaboration Module 150 functions as a bridge between the security system and the resource management system, enabling intelligent and collaborative responses to security incidents.

[0104] As attached Figure 5As shown, module 150, in its specific implementation, can be a service based on a message-driven architecture. As a Kafka consumer, it continuously listens for event topics from security systems such as Cloud Shield. Internally, the module contains a hot-reloadable collaborative policy library, which can be defined using YAML or JSON files, allowing operations personnel to flexibly add, delete, modify, and query policies without restarting the service. Upon receiving a message, the module parses the event content and performs pattern matching in the policy library. Once a match is successful, it generates one or more high-priority resource scheduling instructions based on the policy definition. These instructions are assigned a special tag (e.g., priority: critical, source: security) and then published to the instruction queue for consumption by the decision-making module. Simultaneously, it can also format event characteristic information (such as the rate of attack traffic) and send it to the resource load prediction module 120 via API calls for model optimization.

[0105] In a specific implementation, refer to the appendix. Figure 5 The workflow of the security and resource coordination module in this invention is as follows: The process begins with the Cloud Shield security system. When it detects a security event (such as a DDoS attack), it immediately publishes the event message to the message queue (Kafka). The security and resource coordination module 150, acting as a consumer, subscribes to / consumes the message and parses the event. Subsequently, the module queries matching strategies based on the parsed event type. These strategies are pre-stored in the coordination strategy library. Once a matching strategy is found, the module generates a coordination scheduling instruction, such as a high-priority scaling instruction. This instruction is sent to the elastic scaling decision module 140 for execution, and finally, the instruction executor 160 completes the actual operation on the underlying resources. Simultaneously, an important feedback loop is formed: the event feature feedback extracted from the event (such as the rate and protocol distribution of attack traffic) is sent to the resource load prediction module 120 to optimize the model, enabling it to learn such attack patterns in future training.

[0106] The instruction executor 160 is responsible for converting the abstract instructions generated by the upper-level decision-making module into specific operations on the underlying cloud platform.

[0107] In its implementation, module 160 can be an adaptation layer service deeply integrated with cloud vendor APIs. It receives scheduling instructions from the decision module (e.g., {action: scale_out, target_group: web_servers, instance_type: ecs.g6.xlarge, count: 2}). Internally, the module selects the appropriate API client (SDK) based on the cloud vendor information and resource type in the instruction. It handles authentication, request construction, exception handling, and retry logic during the API call process. For example, it might call Alibaba Cloud's ECSOpenAPI to create or release instances. This module also records the execution status of each instruction (success, failure, in progress) and provides a query interface, thereby providing auditing and tracing capabilities for the entire system's operation, forming a complete closed loop.

[0108] In addition, such as Figure 1 As shown, the system may also include a cloud platform infrastructure and service layer, which includes basic resources such as computing (ECS), storage (OSS), network (SLB), and database (RDS), as well as security services such as "Cloud Shield".

[0109] Example 3 This embodiment will use a specific application scenario—the operation and maintenance management of a "provincial-level intelligent transportation travel service cloud platform" during statutory holidays—to illustrate in detail the application process, parameter settings, and technical effects of the method and system of this invention.

[0110] Technical requirements and hardware selection for the scenario: This transportation cloud platform supports real-time traffic information and route planning services for tens of millions of users, placing extremely high demands on business continuity (requiring 99.99% availability), request response time (core query P95 latency <300ms), and data processing capabilities. Its workload exhibits extreme tidal effects, with concurrent requests during peak holiday periods reaching 5 to 10 times the normal level. To support this platform, its underlying deployment is in a private cloud environment, and the hardware selection includes: Compute Nodes: A computing cluster consisting of 200 high-performance physical servers, virtualized to provide ECS resources. Servers are configured with dual Intel Xeon Platinum 8362 CPUs (32 cores) and 512GB of DDR4 memory.

[0111] Database: A highly available PostgreSQL cluster is deployed and runs on a high-IOPS physical machine equipped with NVMe SSDs.

[0112] Security facilities: A hardware WAF and DDoS mitigation equipment cluster with a processing capacity of 100Gbps has been deployed.

[0113] The system of this invention is deployed in an independent management cluster, consisting of 3 servers configured with 16 CPU cores and 64GB of memory.

[0114] Model building and parameter setting: Data collection: The data collection cycle is uniformly set to 1 minute.

[0115] LSTM model training: Training dataset: Complete monitoring data from the past 365 days using the platform, covering all holidays and special weather events.

[0116] Feature selection: A total of 52 features were selected, including ECS ​​cluster total CPU utilization, RDS cluster QPS, path planning API call volume, weather information (encoded as numerical features), and holiday markers (one-hot encoding).

[0117] Model architecture: A stacked LSTM network with two hidden LSTM layers and 128 neurons per layer was constructed. Dropout=0.2 was used between layers to prevent overfitting.

[0118] Training parameters: The Adam optimizer was used, with an initial learning rate of 0.001, a batch size of 64, and 100 training epochs.

[0119] Decision strategy parameters: Active expansion water level: Set to 75% of the average CPU utilization of the resource pool.

[0120] Forecast lead time: set to 20 minutes.

[0121] Parabolic algorithm recovery parameters: The total recovery period T is set to 60 minutes based on the business type, and the decay factor α is set to 2.5 to provide a longer initial buffer.

[0122] Coordination Strategy: A coordination strategy against CC attacks is configured, which is triggered when the WAF detects that the QPS of a single interface increases by more than 20 times within 1 minute. The strategy action is: the WAF cluster is expanded by 50%, and 30% redundant instances are added to the attacked backend service.

[0123] Control and optimize the processing flow (see attached document) Figure 6 ): The complete process in this scenario, taking the combination of holiday evening rush hour and CC attacks as an example: 1. Predictive Scaling: On holiday afternoons, the resource load prediction module 120 accurately predicts, based on historical data, that the evening peak will begin at 19:00 and reach its peak at 20:30. The elastic scaling decision module 140 begins executing prediction-driven scaling at 18:40 (20 minutes in advance), smoothly increasing the resources of the path planning service cluster (path_planning) and the RDS database cluster (rds_cluster) to a level capable of handling the predicted peak.

[0124] 2. Attack Trigger: At 20:15, during peak business hours, a large influx of CC attack traffic (attackers) flooded the "Real-time Traffic Query Service" (traffic_query) interface. The front-end SLB&WAF cluster (slb_waf) detected the abnormal traffic, and the security system immediately sent the attack event to the security and resource coordination module 150 via a message queue.

[0125] 3. Collaborative Response: Upon receiving an event, the security and resource collaboration module 150 immediately matches the preset CC attack collaboration strategy. It generates two high-priority instructions: (1) The instruction executor 160 performs emergency expansion of the SLB&WAF cluster itself to enhance its cleaning capabilities; (2) The instruction elastic expansion decision module 140 bypasses the conventional prediction logic and immediately supplements the resource pool where the "real-time traffic query service" is located with 30% of the ECS instances.

[0126] 4. Business Assurance: Due to the timely and coordinated enhancement of security facilities and business resources, malicious attack traffic is effectively blocked, while requests from normal users (normal_user) can still be processed in a timely manner by newly added healthy instances, ensuring the SLA of core businesses.

[0127] 5. Smooth Recovery: After the evening peak and the attack subside, the load drops. Prediction module 120 predicts the low load period in the latter half of the night. Dynamic decommissioning and recovery module 130 initiates a "parabolic" recovery, smoothly recovering all temporarily added resources (including those added to cope with peaks and attacks) over the next 60 minutes in a "slow first, fast later" manner, restoring the system to its normal low-cost operating state.

[0128] Example 4 The technical solution of this invention is also applied to the "Provincial Smart Transportation Travel Service Cloud Platform", and the applicable scenario time is from the day before the May Day holiday to the end of the holiday.

[0129] Phase 1: The Eve of the Peak – Forward-Looking Capacity Expansion Based on Forecasts Time: 18:00 on April 30 (the day before the holiday).

[0130] Event: The resource load prediction module started rolling predictions of resource load for the next 24 hours. Based on holiday patterns learned from historical data, the LSTM model output a clear prediction curve: starting from 21:00 that evening, various resource indicators of the platform (especially the number of path planning API calls and RDS query pressure) will begin to climb significantly, reaching their peak between 9:00 and 11:00 the next day (May 1st), with peak resource demand approximately three times that of a normal day.

[0131] Decision-making and execution: The elastic scaling decision-making module receives this prediction result. After analyzing the prediction curve, the system determines that the load will exceed the preset 70% "active scaling threshold" in about 3 hours. The system does not wait, but immediately initiates the prediction-driven scaling strategy.

[0132] Operation Process: Starting at 18:30, the command executor smoothly added a small number of instances to ECS cluster A every 15 minutes, gradually increasing the IOPS specification of RDS. The entire expansion process was gradual and smooth, avoiding the pressure on the underlying cloud platform caused by creating a large number of instances instantaneously.

[0133] Result: At 21:00 that evening, the evening rush hour for public travel arrived as scheduled, and the number of users accessing the platform began to surge. However, at this time, the platform's computing and data service resources were already prepared in advance and accurately. The response time for all users' path planning requests remained at an excellent level of less than 200ms, achieving the ideal effect for resources and other business operations, and completely avoiding the service delay or lag problems of the traditional passive expansion model.

[0134] Phase Two: Peak Periods – Security and Resources Collaborate to Defend Against Application Layer Attacks Time: 10:15 AM, May 1st, when the workload was at its peak.

[0135] Incident: The platform's Cloud Shield WAF system suddenly detected a 50-fold surge in API calls to the "Real-time Traffic Query" interface within just one minute, with the request source IPs being highly dispersed. The security system determined this to be a typical Challenge Collapsar (CC) attack designed to exhaust application resources. While performing traffic scrubbing and IP blocking, the Cloud Shield system immediately published a structured message, "Security Incident - Application Layer DDoS Attack - High Risk," to its Kafka message queue.

[0136] Decision and Execution: The security and resource coordination module consumes the message within milliseconds and immediately matches and triggers the preset "DDoS attack coordination defense strategy".

[0137] Operation process: 1. Enhanced Security Resources: The system immediately issues the highest priority instruction to the instruction executor to urgently expand the front-end WAF cluster and SLB instance, instantly doubling their processing capacity to ensure sufficient capacity to clean up massive amounts of malicious requests.

[0138] 2. Business Resource Isolation and Supplementation: Simultaneously, the collaborative module's command scheduling system dynamically allocates the ECS instance hosting the attacked "Real-time Traffic Query" service to an independent resource group with enhanced access control policies. More importantly, the system also adds an additional 50% of healthy ECS instances specifically for handling legitimate user requests cleaned by the WAF.

[0139] Results: The entire collaborative response process was completed automatically within 2 minutes. Malicious attack traffic was efficiently blocked by WAF and SLB (reference data: a total of over 498,000 malicious attacks were blocked during the period). Because backend business resources were replenished in a timely and intelligent manner, the traffic query service for normal users was almost unaffected, and the business stability compliance rate remained at 100%. This invention successfully transformed a high-risk security incident that could have paralyzed the platform service into a seamless, automated collaborative defense process.

[0140] Phase Three: Peak Retreat – Parabolic Smooth Recovery Time: May 3, 23:00. The peak of the holiday return travel has passed, and the overall load of the platform has begun to decline continuously.

[0141] Event: The prediction module has confirmed that the platform will remain in a low-load night mode for the next 6 hours. The dynamic resource reduction and recycling module has been activated, preparing to perform resource reclamation operations.

[0142] Decision-making and execution: Instead of the traditional one-size-fits-all strategy of immediately shutting down instances once CPU usage drops below 20%, the system has launched a parabolic smooth recycling algorithm.

[0143] Operation process: Parameter settings: Set the total recovery period T = 60 minutes and the decay factor α = 2. Peak_Level is the current resource inventory, and Base_Level is the predicted nighttime baseline resource inventory.

[0144] Recycling execution: Within 0-15 minutes, the resource reservation level decreased slowly, and the system only shut down a small number of the least active instances. This ensured that long-lived connections that were still processing data synchronization across midnight or had a small number of users traveling at night could terminate normally.

[0145] Within 15-45 minutes, the recycling rate gradually increases, and most redundant resources are recycled in an orderly manner during this stage.

[0146] Within 45-60 minutes, the recycling rate reaches its fastest, accurately converging the resource volume to the Base_Level.

[0147] Results: The entire recycling process was smooth and seamless. Backend monitoring showed no fluctuations in the API error rate during the one-hour recycling window. Compared to traditional simulated strategies, this method avoided approximately 5% of long-term connection interruption errors caused by resource mutations. Ultimately, through intelligent prediction, collaborative defense, and smooth recycling throughout the holiday period, the system achieved significant benefits: a cumulative saving of 7662 vCPU core hours, with an estimated annual IT cost saving of over 40 million yuan for the platform. Furthermore, during peak business periods, the resource usage of core services was precisely controlled within the optimal range of 72%-80%.

[0148] like Figure 8 As shown in the figure, this invention demonstrates the convergence process of the training loss of the LSTM resource load prediction model. The horizontal axis represents the number of training epochs, ranging from 1 to 100 epochs; the vertical axis represents the mean squared error loss (MSE). The figure presents two curves: the solid dotted curve represents the training loss, and the dashed dotted curve represents the validation loss. From the curve trends, it can be observed that the loss value is approximately 0.8 in the early stages of training, and decreases rapidly with an exponential decay as the number of training epochs increases. Around the 60th epoch, the model tends to converge, with the training loss stabilizing at around 0.05 and the validation loss stabilizing at around 0.08. The slightly higher validation loss compared to the training loss indicates that the model has good generalization ability and does not exhibit overfitting. This training process uses the Adam optimizer, with an initial learning rate of 0.001 and a batch size of 64. After 100 training epochs, the model weights are saved for online prediction.

[0149] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer programs. When the computer program is loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this disclosure are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer program can be stored in a computer-readable storage medium or transferred from one computer-readable storage medium to another. For example, the computer program can be transferred from one website, computer, server, or data center to another via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer or a data storage device such as a server or data center that integrates one or more available media. The available media may be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., high-density digital video discs (DVDs)), or semiconductor media (e.g., solid-state disks (SSDs)).

[0150] Those skilled in the art will understand that the various numerical designations such as "first," "second," etc., used in this disclosure are merely for the convenience of description and are not intended to limit the scope of the embodiments of this disclosure, nor do they indicate the order of events.

[0151] At least one of the features described in this disclosure can also be described as one or more, and multiple features can be two, three, four or more, and this disclosure does not impose any limitations. In the embodiments of this disclosure, for a technical feature, the technical features in that technical feature are distinguished by "first", "second", "third", "A", "B", "C" and "D", etc., and there is no sequential order or size order among the technical features described by "first", "second", "third", "A", "B", "C" and "D".

[0152] The correspondences shown in the tables of this disclosure can be configured or predefined. The values ​​of the information in each table are merely examples and can be configured to other values; this disclosure is not limiting. When configuring the correspondences between information and parameters, it is not necessarily required to configure all the correspondences shown in each table. For example, the correspondences shown in some rows of the tables in this disclosure may not be configured. Furthermore, appropriate modifications and adjustments can be made based on the above tables, such as splitting, merging, etc. The names of the parameters shown in the headers of the above tables can also use other names that the communication device can understand, and the values ​​or representations of the parameters can also use other values ​​or representations that the communication device can understand. In the implementation of the above tables, other data structures can also be used, such as arrays, queues, containers, stacks, linear lists, pointers, linked lists, trees, graphs, structures, classes, heaps, hash tables, or hash tables, etc.

[0153] The predefined terms in this disclosure can be understood as defined, pre-defined, stored, pre-stored, pre-negotiated, pre-configured, solidified, or pre-burned. Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this disclosure.

[0154] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0155] The above description is merely a specific embodiment of this disclosure, but the scope of protection of this disclosure is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this disclosure should be included within the scope of protection of this disclosure. Therefore, the scope of protection of this disclosure should be determined by the scope of the claims.

Claims

1. A method for optimizing cloud platform resource allocation based on an LSTM model, characterized in that, Includes the following steps: Collect and preprocess multi-dimensional heterogeneous time-series data from the cloud platform; Based on a pre-trained machine learning model, a time-series feature vector constructed from the time-series data is input to predict the resource load within a predetermined time window in the future; the machine learning model is a Long Short-Term Memory (LSTM) network model. Real-time reception and analysis of security events from the security system of the cloud platform; Based on the predicted resource load and the parsed security events, and combined with a preset collaborative strategy library, resource scheduling instructions are generated. The resource scheduling instructions are executed to elastically expand or dynamically reclaim the resources of the cloud platform.

2. The method according to claim 1, characterized in that, The multi-dimensional heterogeneous time-series data includes at least: Infrastructure layer data is selected from at least one of the following: CPU utilization of the central processing unit of the cloud host, memory utilization, network input / output I / O, and disk read / write operations per second (IOPS). Platform service layer data is selected from at least one of the following: queries per second (QPS) of database service, hit rate of cache service, request volume of object storage service, and concurrent connection count of load balancing service. In addition, business and security layer data, selected from at least one of the following: API response latency, API call volume, and security alarm events.

3. The method according to claim 1, characterized in that, The steps for making predictions based on the pre-trained machine learning model include: By applying a sliding window to the preprocessed historical multidimensional time-series data, multiple training samples are generated. The feature data of the training samples also includes security event markers and attack traffic characteristics from historical periods. The LSTM model is trained using the multiple training samples to obtain the pre-trained machine learning model; The multi-dimensional heterogeneous time-series data collected in real time by the cloud platform is preprocessed and the time-series feature vector is constructed. The time-series feature vector is input into the pre-trained machine learning model to obtain the predicted curve of resource load within the predetermined future time window.

4. The method according to claim 1, characterized in that, The dynamic recycling is a parabolic smooth recycling; The dynamic recycling steps include: After the business peak subsides, the dynamic resource retention level Watermark(t) that changes with time t is determined based on the preset total recovery period T and decay factor α, where α is greater than 1; The formula for calculating the dynamic resource retention water level Watermark(t) is as follows: Watermark(t)=Base_Level+(Peak_Level-Base_Level)*(1-(t / T) α ); Where t is the time variable from the start of recycling, Peak_Level is the peak resource amount at the start of recycling, and Base_Level is the target baseline resource amount at the end of recycling; Based on the dynamic resource retention level, the number of resource instances on the cloud platform is gradually reduced.

5. The method according to claim 1, characterized in that, The steps for receiving and parsing security events in real time include: Subscribe to security event messages published by the security system via a message queue; The steps for generating the resource scheduling instruction include: When a security event message is received, the resource operation corresponding to the security event type is matched from the collaborative policy library to generate a high-priority resource scheduling instruction. The resource operations include resource operations on security facilities and resource coordination operations on affected business services.

6. A system for optimizing cloud platform resource allocation based on an LSTM model, characterized in that, include: The data acquisition and preprocessing module is used to acquire and preprocess multi-dimensional heterogeneous time-series data from the cloud platform. The resource load prediction module is used to predict the resource load within a predetermined time window by taking a time-series feature vector constructed based on the time-series data as input to a pre-trained machine learning model; the machine learning model is a Long Short-Term Memory (LSTM) network model. The security and resource collaboration module is used to receive and parse security events from the security system of the cloud platform in real time. The decision module, coupled to the resource load prediction module and the security and resource coordination module, is used to generate resource scheduling instructions based on the predicted resource load and the parsed security events, combined with a preset coordination strategy library. And an instruction executor, used to execute the resource scheduling instructions to elastically expand or dynamically reclaim the resources of the cloud platform.

7. The system according to claim 6, characterized in that, The multi-dimensional heterogeneous time-series data includes at least: Infrastructure layer data is selected from at least one of the following: CPU utilization, memory utilization, network I / O, and disk IOPS of the cloud host. Platform service layer data is selected from at least one of the following: QPS of database service, hit rate of cache service, request volume of object storage service, and concurrent connection number of load balancing service. In addition, business and security layer data, selected from at least one of the following: API response latency, API call volume, and security alarm events.

8. The system according to claim 6, characterized in that, The resource load prediction module is configured as follows: By applying a sliding window to the preprocessed historical multidimensional time-series data, multiple training samples are generated. The feature data of the training samples also includes security event markers and attack traffic characteristics from historical periods. The LSTM model is trained using the multiple training samples to obtain the pre-trained machine learning model; The time-series feature vector, constructed based on real-time collected data, is input into the pre-trained machine learning model to obtain the predicted curve of resource load within the predetermined future time window.

9. The system according to claim 6, characterized in that, The decision-making module includes a dynamic downgrade and recycling module, which is used to perform parabolic smooth recycling and is configured as follows: After the business peak subsides, the dynamic resource retention level Watermark(t) that changes with time t is determined based on the preset total recovery period T and the decay factor α greater than 1. The formula for calculating the dynamic resource retention water level is as follows: Watermark(t)=Base_Level+(Peak_Level-Base_Level)*(1-(t / T) α ); Where t is the time variable from the start of recycling, Peak_Level is the peak resource amount at the start of recycling, and Base_Level is the target baseline resource amount at the end of recycling; It also generates instructions for gradually reducing the number of cloud platform resource instances.

10. The system according to claim 6, characterized in that, The security and resource collaboration module is configured as follows: Subscribe to security event messages published by the security system via a message queue; When a security event message is received, the resource operation corresponding to the security event type is matched from the collaborative policy library to generate a high-priority resource scheduling instruction, and the instruction is sent to the decision module. The resource operations include resource operations on security facilities and resource coordination operations on affected business services.

Images