Host machine high-load period prediction method, device and system

Abstract

The invention provides a host machine high-load period prediction method, device and system, and the method comprises the steps: S101, building a long-term LSTM integrated network model and a short-term LSTM integrated network model based on the performance index data of a previous load of a host machine, and carrying out the weighting processing, and obtaining an optimized integrated LSTM network model; s102, acquiring performance index data related to the current host machine and the load thereof, and preprocessing the performance index data; s103, on the basis of the current time, inputting the preprocessed performance index data into the optimized integrated LSTM network model for prediction to obtain a future load prediction value of the current host machine; s104, acquiring a high load threshold value of the host machine, and judging whether the load prediction value of the current host machine is greater than the high load threshold value or not; if yes, predicting that the current host machine is in a high-load period in the future; according to the scheme, prediction, resource scheduling and migration can be carried out on the host machine and the computing power resource load borne by the host machine in the future time period.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a method, apparatus and system for predicting high load periods of a host machine. Background Technology

[0002] In the field of cloud computing monitoring, high-load monitoring of computing resource hosts refers to the ability to monitor and analyze computing resource hosts in a cloud computing environment in real time. Specifically, it is a system design and implementation method aimed at improving the visibility and understanding of the performance and behavior of computing resources during periods of high load. In a cloud computing environment, high load on computing resource hosts (such as cloud servers) can lead to performance degradation and increased response latency, thereby affecting the availability of cloud services and user experience. Therefore, the goal of high-load monitoring of cloud servers is to promptly capture host resource usage, identify the risk of high load, and provide corresponding optimization and maintenance strategies.

[0003] Currently, high-load monitoring of the host machine uses Prometheus as the core of the monitoring system, combined with its associated Exporter. This solution mainly uses open-source components and has the following disadvantages: (1) Complex data processing: Prometheus and its associated Exporter have certain complexity in data acquisition and processing, requiring the writing and configuration of a large number of rules and query statements to obtain and process monitoring data; (2) Storage limitations: Prometheus uses local disk to store monitoring data by default. For long-term storage and large-scale data processing, local storage may face capacity limitations and performance bottlenecks; (3) System complexity: Using Prometheus and its associated Exporter requires the deployment and configuration of multiple components, including Prometheus server, various Exporters, etc., which increases the complexity and maintenance cost of the system; (4) Limited predictive ability: Prometheus is mainly used for real-time monitoring and alarms, and its predictive ability for future high-load periods is limited. Prometheus mainly focuses on real-time monitoring and alarms, and its ability for long-term trend analysis and pattern recognition is weak. It may not be powerful enough in some scenarios that require in-depth analysis of historical data. Summary of the Invention

[0004] This application provides a method, apparatus, and system for predicting high-load periods, which can predict, schedule, and migrate the load of the host machine and its computing resources in a future time period.

[0005] According to one aspect of this application, a method for predicting high-load periods is provided, the method comprising: S101. Based on the performance index data of the previous load of the host machine, establish a long-term LSTM integrated network model and a short-term LSTM integrated network model. Then, perform weighted processing on the long-term LSTM integrated network model and the short-term LSTM integrated network model to obtain an optimized integrated LSTM network model. S102, acquire performance index data related to the current host machine and its load, and preprocess the performance index data; S103, based on the current time, input the preprocessed performance index data into the optimized integrated LSTM network model for prediction, and obtain the predicted load value of the current host in the future. S104, obtain the host machine high load threshold and determine whether the current host machine load prediction value is greater than the high load threshold; If so, it is predicted that the current host machine will be in a period of high load in the future; If not, the current host machine is operating normally.

[0006] One aspect of this application provides a method for predicting high-load periods, wherein step S101 includes: Obtain performance metrics data of the host machine's previous load and preprocess them; The preprocessed performance metric data is converted into historical time series data to obtain the training dataset and the test dataset; The first predicted load value for the long period of time is obtained by capturing historical time series data within the training dataset using a long-term LSTM ensemble network model; the second predicted load value is obtained by capturing historical time series data within the short period of time of time in the training dataset using a short-term LSTM ensemble network model. By inputting historical time series data over a long period in the test dataset into a long-term LSTM ensemble network model, the third predicted load value for that long period is obtained; by inputting historical time series data over a short period in the test dataset into a short-term LSTM ensemble network model, the fourth predicted load value for that short period is obtained. The optimized LSTM network model is obtained by comprehensively evaluating and allocating weights based on the first and third load predictions of the long-term LSTM ensemble network model and the third and fourth load predictions of the short-term LSTM ensemble network model.

[0007] One aspect of this application provides a method for predicting high-load periods, wherein step S102 further includes: The performance of the ensemble LSTM network model is evaluated based on the first and second load predictions of the long-term LSTM ensemble network model and the third and fourth load predictions or cross-validation results of the short-term LSTM ensemble network model. The root mean square error of the first, second, third, and fourth load predictions is used to verify whether the integrated LSTM network model meets the prediction accuracy requirements. If so, the accuracy of the predictions of the integrated LSTM network model is evaluated. If not, the parameters will be automatically adjusted, and the integrated LSTM network model will be returned. The weight allocation of long-term and short-term LSTM ensemble network models is comprehensively evaluated based on the performance and prediction accuracy of the ensemble LSTM network model. The prediction results of the long-term LSTM ensemble network model and the short-term LSTM ensemble network model are weighted and averaged according to the assigned weights to calculate the comprehensive prediction result of the host load.

[0008] One aspect of this application provides a high-load period prediction method, in which a high-load threshold for the host machine is set based on the comprehensive prediction results of the host machine load during the high-load period during the optimization of the integrated LSTM network model. When the predicted host load value of the test dataset is greater than the high load threshold, it is predicted that the host will be in a high load period in the future.

[0009] One aspect of this application provides a method for predicting high-load periods, wherein the load performance index data includes the load performance index of the host machine and the computing resources it supports, and external time series data related to the load; the external time series data includes performance index data of at least one external factor related to the load, such as weather, holidays, indoor activities, video entertainment, business activity arrangements, and media events. The steps for preprocessing the performance index data include: data parsing, formatting, and filtering of the performance index data.

[0010] One aspect of this application provides a high-load period prediction method, wherein the step of acquiring performance index data related to the current host machine and its load includes: collecting load performance indicators of the host machine and the computing resources it carries based on an integrated acquisition component; The acquisition component exposes the acquired performance indicator data as an accessible interface, provides the performance indicator data to the data storage stack and other monitoring components, and is compatible with the monitoring data format alarms of the monitoring components.

[0011] One aspect of this application provides a method for predicting high-load periods, wherein the working process of the host load performance metric stack includes: Logstash is used as the middleware for data collection and processing. In the Logstash configuration file, the input plugin is used to receive performance metric data from the collection component; the filter plugin is used to process and transform the received performance metric data; according to the data format of the collection component, the corresponding filter plugin is used to parse, format, and filter the data; the output plugin is used to send the processed data to Elasticsearch; by configuring the Elasticsearch address, index name, and other relevant parameters, the data is written to the specified Elasticsearch index. By connecting to Elasticsearch using Kibana, you can visualize and analyze imported data by creating index schemas and dashboards; By connecting load performance metrics data to Prometheus and using it as a data source for Grafana, Grafana can be used to create charts and display various load performance metrics data.

[0012] One aspect of this application provides a method for predicting high-load periods, wherein the method further includes the following step between step S102 and step S103: The preprocessed and performance metrics data are transformed into time series data, which is configured as a sliding window, including a longer time window and a shorter time window. Step S104 further includes that, after calculating the load prediction value, the current host machine and its load-related performance index data are configured as the previous load performance index data of the host machine, and then the process re-enters step S101.

[0013] According to another aspect of this application, a high-load period prediction device is provided, comprising: An integrated LSTM network module is used to build long-term and short-term LSTM integrated network models based on the performance index data of the previous load of the host machine. The long-term and short-term LSTM integrated network models are weighted to obtain an optimized integrated LSTM network model. The integrated LSTM network model includes a gating unit, which controls the flow of performance index data through forget gates, input gates and output gates to realize the processing of long sequence data and the learning of long-term dependencies. The performance metrics data module is used to acquire performance metrics data related to the current host machine and its load, and to preprocess the performance metrics data. The load prediction module is used to input preprocessed performance index data into an optimized integrated LSTM network model based on the current time to predict the future load of the current host. The comparison module is used to obtain the host machine's high load threshold and determine whether the current host machine's predicted load value is greater than the high load threshold. If so, it predicts that the current host machine will be in a high load period in the future; if not, the current host machine is operating normally.

[0014] According to another aspect of this application, a high-load period prediction system is provided, comprising: a cloud host and the load computing resources it carries, wherein the computing resources include containers, bare metal servers and virtual networks; The cloud host includes several host machines deployed in a cluster. Each host machine predicts the load trend according to the steps of the host machine high load period prediction method according to any one of claims 1 to 8, and automatically triggers the redistribution and migration strategy of host machine resource scheduling; by migrating some computing resources from high-load host machines to idle or low-load host machines, host machine load balancing and resource optimization are achieved.

[0015] According to an embodiment of this application, a method, apparatus, and system for predicting high-load periods for a host machine comprises the following steps: Step S101: Establishing a long-term LSTM ensemble network model and a short-term LSTM ensemble network model based on the performance index data of the host machine's previous load; weighting the long-term and short-term LSTM ensemble network models to obtain an optimized ensemble LSTM network model, which can learn the historical patterns and trends of load indicators, thereby weighted prediction of the probability of future high-load periods; Step S102: Obtaining performance index data related to the current host machine and its load, and preprocessing the performance index data; converting the preprocessed load data into a format suitable for training the ensemble LSTM network; Step S103: Based on the current time, inputting the preprocessed performance index data into the optimized ensemble LSTM network model for prediction, obtaining the predicted load value of the current host machine in the future; Step S104: Determining whether the predicted load value falls within a high-load period. The algorithm implements strategies such as scheduling and migrating host machine resources, which helps host machine managers make adjustments and optimizations in advance during high-load periods to ensure system stability and performance.

[0016] It should be understood that both the foregoing general description and the following detailed description are exemplary and intended to provide further illustration of the claimed technology. Attached Figure Description

[0017] The above and other objects, features, and advantages of the present invention will become more apparent from the more detailed description of the embodiments of the invention in conjunction with the accompanying drawings. The drawings are provided to further illustrate the embodiments of the invention and form part of the specification. They are used together with the embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings, the same reference numerals generally represent the same parts or steps.

[0018] Figure 1 This is a flowchart illustrating a host high-load period prediction method according to an embodiment of the present invention; Figure 2 This is a flowchart illustrating a prediction method applied according to an embodiment of the present invention; Figure 3 This is a flowchart illustrating another application of the prediction method according to an embodiment of the present invention; Figure 4 This is another flowchart illustrating a prediction method according to an embodiment of the present invention; Figure 5 This is a schematic diagram of the host machine high load period prediction device according to an embodiment of the present invention; Figure 6 This is a schematic diagram of the host high load period prediction system according to an embodiment of the present invention; Figure 7 This is a schematic diagram of the structure of a computer device according to an embodiment of the present invention. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of the present invention more apparent, exemplary embodiments according to the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of the present invention, and not all embodiments of the present invention. It should be understood that the present invention is not limited to the exemplary embodiments described herein.

[0020] In the field of cloud computing monitoring, high-load monitoring of computing resource hosts refers to the ability to monitor and analyze computing resource hosts in a cloud computing environment in real time. Specifically, it is a system design and implementation method aimed at improving the visibility and understanding of the performance and behavior of computing resources during periods of high load. In a cloud computing environment, high load on computing resource hosts (such as cloud servers) can lead to performance degradation and increased response latency, thereby affecting the availability of cloud services and user experience. Therefore, the goal of high-load monitoring of cloud servers is to promptly capture host resource usage, identify the risk of high load, and provide corresponding optimization and maintenance strategies.

[0021] Currently, high-load monitoring of the host machine uses Prometheus as the core of the monitoring system, combined with its associated Exporter. This solution mainly uses open-source components and has the following disadvantages: (1) Complex data processing: Prometheus and its associated Exporter have certain complexity in data acquisition and processing, requiring the writing and configuration of a large number of rules and query statements to obtain and process monitoring data; (2) Storage limitations: Prometheus uses local disk to store monitoring data by default. For long-term storage and large-scale data processing, local storage may face capacity limitations and performance bottlenecks; (3) System complexity: Using Prometheus and its associated Exporter requires the deployment and configuration of multiple components, including Prometheus server, various Exporters, etc., which increases the complexity and maintenance cost of the system; (4) Limited predictive ability: Prometheus is mainly used for real-time monitoring and alarms, and its predictive ability for future high-load periods is limited. Prometheus mainly focuses on real-time monitoring and alarms, and its ability for long-term trend analysis and pattern recognition is weak. It may not be powerful enough in some scenarios that require in-depth analysis of historical data.

[0022] The above description, with reference to the accompanying drawings, illustrates a method, apparatus, and system for predicting high-load periods for a host machine according to an embodiment of the present invention. Step S101 involves establishing a long-term LSTM ensemble network model and a short-term LSTM ensemble network model based on previous host machine load performance data. The long-term and short-term LSTM ensemble network models are then weighted to obtain an optimized ensemble LSTM network model, which can learn historical patterns and trends in load indicators, thereby weighted prediction of the likelihood of future high-load periods. Step S102 involves acquiring performance indicator data related to the current host machine and its load, and preprocessing the performance indicator data. The preprocessed load data can be converted into a format suitable for training the ensemble LSTM network. Step S103 involves inputting the preprocessed performance indicator data into the optimized ensemble LSTM network model based on the current time for prediction, obtaining the predicted load value for the current host machine in the future. Step S104 involves determining whether the predicted load value falls within a high-load period. This algorithm implements strategies such as host machine resource scheduling and migration, which helps host machine administrators make adjustments and optimizations in advance during high-load periods to ensure system stability and performance.

[0023] On the other hand, in the method for predicting high load periods of the host machine, the step of obtaining performance index data related to the current host machine and its load includes: collecting load performance indicators of the host machine and the computing resources it carries based on the integrated acquisition components. The acquisition component exposes the acquired performance metrics data as an accessible interface, providing performance metrics data to the data storage stack and other monitoring components, and is compatible with the monitoring data format alarms of monitoring components such as Prometheus; it also integrates LSTM network algorithms to achieve high load monitoring and alarms on the host machine.

[0024] To facilitate understanding of this embodiment, a detailed description of the host machine high-load period prediction method, apparatus, and system disclosed in this invention embodiment is provided first. The execution entity of the host machine high-load period prediction method provided in this invention embodiment is generally a computer device with a certain computing capability. This computer device may include, for example, a terminal device, a server, or other processing devices. The terminal device may be a user equipment (UE), mobile device, user terminal, terminal, cellular phone, cordless phone, personal digital assistant (PDA), handheld device, computing device, vehicle-mounted device, wearable device, etc. In some possible implementations, the host machine high-load period prediction method can be implemented by the processor calling computer-readable instructions stored in memory.

[0025] like Figure 1 The diagram shows a flowchart of a host machine high load period prediction method provided in an embodiment of the present invention. The method includes steps S101-S104: S101. Based on the performance index data of the previous load of the host machine, establish a long-term LSTM integrated network model and a short-term LSTM integrated network model. Then, perform weighted processing on the long-term LSTM integrated network model and the short-term LSTM integrated network model to obtain an optimized integrated LSTM network model. Specifically, the previous load of the host machine and the current dynamic load of the host machine are relative concepts; the performance index data of the previous load of the host machine is based on historical data before the current point in time.

[0026] In this embodiment, the Long Short-Term Memory (LSTM) network model integrating LSTM is a special type of recurrent neural network (RNN) used to address the long-term dependency problem in traditional RNNs. The integrated LSTM model includes a long-term LSTM ensemble network model for capturing long-term dependencies and a short-term LSTM ensemble network model for capturing short-term dependencies.

[0027] Specifically, the host machine's previous load performance metrics data are input into the ELK training set to train a Short-Term LSTM ensemble network model specifically designed to capture short-term dependencies. When inputting data, a shorter time window is selected, such as load performance metrics data from the past few hours to several days. These load performance metrics data sequences are then fed into the Short-Term LSTM ensemble network model to train the model to predict future load values.

[0028] Using data from the ELK training set, train a long-term LSTM ensemble network model specifically designed to capture long-term dependencies. When inputting data, select a longer time window, such as load data from the past few weeks. Feed these load data sequences into the long-term LSTM ensemble network model to train it to predict future load values.

[0029] It is understandable that long-term LSTM ensemble network models and short-term LSTM ensemble network models have different structures and parameter settings to adapt to load changes at different time scales. Assigning different weights to the two models improves the accuracy of predicting future load values, further optimizing the ensemble LSTM network model.

[0030] S102, acquire performance metrics data related to the current host machine and its load, and preprocess the performance metrics data; To construct a profile of computing resource utilization and predict periods of high load, machine learning and time series analysis techniques can be used. Load data obtained from ELK needs to be preprocessed to meet the input requirements of the integrated LSTM network.

[0031] In this embodiment, preprocessing includes data cleaning, missing value handling, and feature engineering. Appropriate preprocessing is performed on the data according to the specific data type and the requirements of the integrated LSTM network. Finally, the preprocessed payload data is converted into a format suitable for training the integrated LSTM network.

[0032] Specifically, between steps S102 and S103, the process further includes converting the preprocessed and performance index data into time series data, whereby the time series data is configured as a sliding window, which includes a longer time window and a shorter time window.

[0033] In this embodiment, the time series data is configured with a longer time window format and a shorter time window format; the longer time window format is input into the long-term LSTM ensemble network model, and the shorter time window format is input into the short-term LSTM ensemble network model.

[0034] S103, based on the current time, input the preprocessed performance index data into the optimized integrated LSTM network model for prediction, and obtain the predicted load value of the current host in the future. It should be noted that the preprocessed performance index data consists of both long-term and short-term time-series data. These data are input into the long-term LSTM ensemble network model and the short-term LSTM ensemble network model respectively for prediction, and then weighted to obtain the predicted load value of the current host machine in the future.

[0035] S104, obtain the host machine high load threshold and determine whether the current host machine load prediction value is greater than the high load threshold; If so, it is predicted that the current host machine will be in a period of high load in the future; If not, the current host machine is operating normally.

[0036] To improve prediction accuracy, a high load threshold for the host machine is set based on the comprehensive prediction results of the host machine load during high load periods. The load prediction value obtained in step S103 is compared with the high load threshold to predict the future load pattern and trend of the host machine. If the host machine is in a high load trend in the future, it will help the administrator to achieve more accurate and effective resource scheduling and migration.

[0037] Steps S101-104: Based on the integrated LSTM network model, the historical patterns and trends of load indicators can be learned, thereby weighted prediction of the probability of future high load periods. The algorithm can be used to implement strategies such as scheduling and migration of host resources, which helps host administrators to make adjustments and optimizations in advance during high load periods to ensure the stability and performance of the system.

[0038] In a preferred embodiment, step S104 further includes configuring the current host machine and its load-related performance index data, after calculating the load prediction value, as the previous load performance index data of the host machine, and then re-entering step S101. At this time, the current host machine and load-related performance index data are iteratively converted into historical data and input into the integrated LSTM network model for retraining. The training and prediction process of the integrated LSTM network is integrated with the real-time load data stream, and this process is repeated to update the model when new load data arrives, thereby improving prediction accuracy in advance.

[0039] like Figure 2 The diagram shows a flowchart of a host high-load period prediction method provided in an embodiment of the present invention. Step S101 includes: S1. Obtain the performance metrics data of the host machine's previous load and perform preprocessing; S2. Convert the preprocessed performance index data into historical time series data to obtain the training dataset and the test dataset; S3. Based on the long-term LSTM ensemble network model, capture historical time series data within a long time period in the training dataset to obtain the first predicted load value within the long time period; based on the short-term LSTM ensemble network model, capture historical time series data within a short time period in the training dataset to obtain the second predicted load value. S4. Input the historical time series data within a long period of time in the test dataset into the long-term LSTM ensemble network model to obtain the third predicted load value within that long period of time; input the historical time series data within a short period of time in the test dataset into the short-term LSTM ensemble network model to obtain the fourth predicted load value within that short period of time. S5. Based on the first and third load predictions of the long-term LSTM ensemble network model, and the third and fourth load predictions of the short-term LSTM ensemble network model, the weights are comprehensively evaluated and allocated to obtain the optimized ensemble LSTM network model.

[0040] Steps S1-S5 describe in detail the optimization process of the integrated LSTM network model. Specifically, the collected historical load performance data undergoes preprocessing such as cleaning, missing value handling, and feature engineering. Finally, the preprocessed load data is converted into long-term series data and short-term series data formats.

[0041] The long-term LSTM ensemble network model is trained by capturing long-term series data in the training dataset or long-term series data in the test dataset, and the first and third predicted load values ​​are obtained in sequence. The short-term LSTM ensemble network model is trained by capturing short-time series data in the training dataset or short-time series data in the test dataset, and then obtains the second and fourth predicted load values ​​in sequence. The final load prediction value is obtained by assigning weights (weighting coefficients) to the four prediction values ​​based on the comprehensive evaluation, and the integrated LSTM network model is further optimized.

[0042] like Figure 3 The diagram shows another flowchart of the host high load period prediction method provided in this embodiment of the invention. Step S102 further includes: S6. Evaluate the performance of the ensemble LSTM network model based on the first and second load predictions of the long-term LSTM ensemble network model and the third and fourth load predictions or cross-validation results of the short-term LSTM ensemble network model. S7. Verify whether the integrated LSTM network model meets the prediction accuracy requirements based on the root mean square error (RMSE) of the first, second, third, and fourth predicted load values. If so, the accuracy of the predictions of the integrated LSTM network model is evaluated. If not, the parameters will be automatically adjusted, and the integrated LSTM network model will be returned. S8. Evaluate the weight allocation of long-term LSTM ensemble network models and short-term LSTM ensemble network models based on their performance and prediction accuracy. S9. Calculate the comprehensive prediction result of the host load by weighting the prediction results of the long-term LSTM ensemble network model and the short-term LSTM ensemble network model according to the assigned weights.

[0043] Steps S6-S7 describe the evaluation method for the performance and prediction accuracy of the ensemble LSTM network model. Steps S8-S9 comprehensively evaluate the weight allocation and load prediction values ​​of the long-term LSTM ensemble network model and the short-term LSTM ensemble network model.

[0044] This maintains the accuracy of the integrated LSTM network and allows for timely responses to changes in load patterns and external factors. Long-term and short-term LSTM integrated network models have different structures and parameter settings to adapt to load variations across different time scales. An automated model parameter adjustment mechanism is introduced, automatically adjusting the model's hyperparameters and the weights of the long-term and short-term LSTM integrated networks by monitoring the performance and prediction accuracy of the integrated LSTM network. This improves prediction accuracy and stability, thereby helping administrators achieve more precise and effective resource scheduling.

[0045] As can be seen from the above embodiments, in the process of optimizing the integrated LSTM network model, a high load threshold for the host machine is set based on the comprehensive prediction results of the host machine load during the high load period. When the predicted host load value of the test dataset is greater than the high load threshold, it is predicted that the host will be in a high load period in the future.

[0046] Setting a high host load threshold based on the comprehensive prediction results can improve the prediction accuracy of the integrated LSTM network model.

[0047] In this embodiment, taking into account the possible special circumstances, the load performance index data includes the load performance index of the host machine and the computing resources it supports, as well as external time series data related to the load; For example, external time series data includes performance metrics data of at least one load-related external factor, such as weather, holidays, indoor activities, video entertainment, business activity schedules, and media events. The steps for preprocessing performance indicator data include: data parsing, formatting, and filtering.

[0048] Integrating LSTM networks typically requires input load data with time-series characteristics. This data can be converted to time-series data, such as using a sliding window method. It's crucial to ensure the input data format meets the requirements of the LSTM network. Furthermore, considering fluctuations in computing resource utilization due to external factors—such as increased user activity during inclement weather or holidays, leading to higher access and load on computing resources—or increased access to related applications or websites due to specific business activities or events, can also increase computing resource load. Therefore, factors that may cause fluctuations in computing resource utilization, such as weather, holidays, indoor activities, video entertainment, business activity schedules, and media events, can be converted into time-series data as additional input data. Specifically, external factors can be one or more of the aforementioned factors.

[0049] In this preferred embodiment, the step of obtaining performance index data related to the current host machine and its load in the host machine high load period prediction method includes: collecting the load performance index of the host machine and the computing resources it carries based on the integrated acquisition component; The acquisition component exposes the acquired performance metrics data as an accessible interface, providing the performance metrics data to the data storage stack and other monitoring components, and is compatible with the monitoring data format alarms of monitoring components such as Prometheus.

[0050] Specifically, to achieve high availability and stability of the cloud computing platform (including multiple host machines), it is necessary to monitor the load of each host machine in real time in order to make timely resource scheduling decisions. At the same time, in order to statistically analyze and further predict historical host machine performance indicators, the implementation of this solution requires the collection of host machine load-related performance indicator data.

[0051] The highly integrated load metric data acquisition component, implemented in Go, primarily collects and reports performance metrics data related to the host machine's load (such as CPU utilization and memory utilization). This information is crucial for assessing the host machine's load status. In addition to host machine load information, this component can also collect information about the computing resources hosted on the host machine, such as resource specifications and resource utilization. By collecting the specifications of computing resources, host machine administrators can understand parameters such as the number of CPU cores, memory capacity, storage space, and network bandwidth of each resource, enabling a better understanding of resource requirements and performance demands. By collecting resource utilization data, host machine administrators can understand the actual resource usage of each resource. This provides host machine administrators with more multi-dimensional information for decision-making and analysis. The acquisition component exposes the acquired performance metric information as an accessible interface, providing data to the data stack and other monitoring components, and is compatible with the monitoring data format and alarms of monitoring components such as Prometheus.

[0052] This proposal employs a highly integrated data acquisition component to collect and report multiple computing resource load-related indicators, compatible with the Prometheus monitoring data format; it utilizes ELK for log recording and analysis, and combines the stored computing resources with the load performance indicators of the cloud hosts they support, along with multiple external time-series data that may affect the overall usage of the cloud hosts, to input into an integrated LSTM neural network for model training, and further predicts computing resource load for future time periods.

[0053] By training long-term and short-term LSTM ensemble network models using the latest data in the load performance metric data storage stack, historical patterns and trends of load metrics can be learned. This allows for weighted prediction of the likelihood of future high-load periods. Furthermore, algorithms can be used to implement strategies such as scheduling and migrating host resources. This helps host administrators make adjustments and optimizations in advance during high-load periods to ensure system stability and performance.

[0054] like Figure 4 The diagram shown illustrates another flowchart of the host machine high load period prediction method provided in this embodiment of the invention. The working process of the host machine load performance index stack in this embodiment of the invention includes: Logstash is used as the middleware for data collection and processing. In the Logstash configuration file, the input plugin is used to receive performance metric data from the collection component; the filter plugin is used to process and transform the received performance metric data; according to the data format of the collection component, the corresponding filter plugin is used to parse, format, and filter the data; the output plugin is used to send the processed data to Elasticsearch; by configuring the Elasticsearch address, index name, and other relevant parameters, the data is written to the specified Elasticsearch index. By connecting to Elasticsearch using Kibana, you can visualize and analyze imported data by creating index schemas and dashboards; By connecting load performance metrics data to Prometheus and using it as a data source for Grafana, Grafana can be used to create charts and display various load performance metrics data.

[0055] According to another aspect of the present invention, a host high-load period prediction device is provided, such as... Figure 5 As shown, the device includes: An integrated LSTM network module 201 is used to establish a long-term LSTM integrated network model and a short-term LSTM integrated network model based on the performance index data of the previous load of the host machine. The long-term LSTM integrated network model and the short-term LSTM integrated network model are weighted to obtain an optimized integrated LSTM network model. The integrated LSTM network model includes a gating unit, which controls the flow of performance index data through forget gate, input gate and output gate to realize the processing of long sequence data and the learning of long-term dependencies. The performance metric data module 202 is used to acquire performance metric data related to the current host machine and its load, and to preprocess the performance metric data. The load prediction module 203 is used to input preprocessed performance index data into an optimized integrated LSTM network model based on the current time to predict the load of the current host in the future. The comparison module 204 is used to obtain the host machine's high load threshold and determine whether the current host machine's predicted load value is greater than the high load threshold. If so, it predicts that the current host machine will be in a high load period in the future; if not, the current host machine will operate normally. In one or more embodiments, the integrated LSTM network includes multiple gating units, primarily three types: forget gate, input gate, and output gate. These gating units control the flow of performance metrics data for the workload, thereby more effectively capturing and remembering important information in long sequences. By controlling the flow of workload data through the forget gate, input gate, and output gate, the processing of long-term series data and the learning of long-term dependencies are achieved. The forget gate determines which information to retain, the input gate determines which information to update, and the output gate determines how much information to output. This gating mechanism effectively processes long-term series data, enabling the integrated LSTM network to perform exceptionally well in prediction tasks.

[0056] The host high load period prediction device and the host high load period prediction method provided in the embodiments of the present invention are based on the same inventive concept and have the same beneficial effects as the methods they adopt, operate or implement.

[0057] According to another aspect of the present invention, a host high-load period prediction system is provided, such as... Figure 6 As shown, the system includes: The cloud host 205 and the computing power resources 206 it carries, the computing power resources 206 including containers, bare metal servers and virtual networks; The cloud host 205 includes several host machines 207 deployed in a cluster. Each host machine 207 predicts the load trend according to the steps of the host machine high load period prediction method described above, and automatically triggers the redistribution and migration strategy of host machine resource scheduling. By migrating some computing resources from high-load host machines to idle or low-load host machines, host machine load balancing and resource optimization are achieved.

[0058] Specifically, a cloud server is a virtualization technology implementation in a cloud computing environment that allows the resources of a physical server (host machine) to be virtualized and divided into multiple independent virtual machine instances. These virtual machine instances, i.e., cloud servers, share the host machine's computing, storage, and network resources, but each runs independently with different configurations and specifications to adapt to various applications and user needs.

[0059] Besides cloud servers, cloud computing offers other types of computing resources, including containers, bare metal servers, and virtual networks, to meet computing needs at different levels and scales. Containers are a lightweight virtualization technology that allows developers to package applications and their dependencies in isolated environments for rapid deployment and scaling. Containers share the host machine's kernel but have their own file system, network configuration, and runtime environment, providing greater flexibility and portability. Virtual networks, on the other hand, are resources in cloud computing used to simulate traditional network environments. They allow users to create and manage virtual networks in the cloud, enabling network isolation, routing, load balancing, and other functions to meet complex network requirements.

[0060] In practical deployment models, host machines are deployed in a cluster, with unified management and scheduling achieved through control nodes. Control nodes are a special type of node in the host machine cluster, responsible for coordinating and managing host machine resources throughout the cluster. Control nodes manage cloud hosts, containers, and virtual networks on the host machines through cluster management software. Control nodes can automatically allocate and migrate resources based on load and performance requirements, achieving optimal resource utilization and fault recovery, ensuring the efficient operation and reliability of the cloud computing environment.

[0061] The host high load period prediction system and the host high load period prediction method provided in this embodiment of the invention are based on the same inventive concept and have the same beneficial effects as the methods they adopt, run or implement.

[0062] This invention also provides a computer device for executing the above-described method for predicting high-load periods on a host machine. Please refer to... Figure 7 It illustrates a schematic diagram of a computer device provided by some embodiments of the present invention. For example... Figure 7 As shown, the computer device 8 includes a processor 800, a memory 801, a bus 802, and a communication interface 803. The processor 800, the communication interface 803, and the memory 801 are connected via the bus 802. The memory 801 stores a computer program that can run on the processor 800. When the processor 800 runs the computer program, it executes the host high-load period prediction method provided in any of the foregoing embodiments of this disclosure. The memory 801 may include high-speed random access memory (RAM) or non-volatile memory, such as at least one disk drive. Communication between the device network element and at least one other network element is achieved through at least one communication interface 803 (which can be wired or wireless), such as the Internet, wide area network, local area network, metropolitan area network, etc.

[0063] Bus 802 can be an ISA bus, PCI bus, or EISA bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. The memory 801 is used to store programs. After receiving an execution instruction, the processor 800 executes the program. The host high-load period prediction method disclosed in any of the foregoing embodiments of the present invention can be applied to the processor 800, or implemented by the processor 800.

[0064] The processor 800 may be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method can be completed by the integrated logic circuitry in the hardware of the processor 800 or by instructions in software form. The processor 800 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), an off-the-shelf programmable gate array (FPTA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this invention. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this invention can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules may reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The storage medium is located in memory 801. Processor 800 reads the information in memory 801 and, in conjunction with its hardware, completes the steps of the above method.

[0065] The computer device provided in this embodiment of the invention and the host high load period prediction method provided in this embodiment of the invention are based on the same inventive concept and have the same beneficial effects as the methods they adopt, run or implement.

[0066] The present invention also provides a computer-readable storage medium corresponding to the host high load period prediction method provided in the foregoing embodiments. The computer-readable storage medium is an optical disc, on which a computer program (i.e., a computer program product) is stored. When the computer program is run by a processor, it executes the host high load period prediction method provided in any of the foregoing embodiments.

[0067] It should be noted that examples of the computer-readable storage medium may also include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other optical and magnetic storage media, which will not be elaborated here.

[0068] The basic principles of the present invention have been described above with reference to specific embodiments. However, it should be noted that the advantages, benefits, and effects mentioned in the present invention are merely examples and not limitations, and should not be considered as essential features of the various embodiments of this disclosure. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the present invention to the necessity of employing the specific details described above. The block diagrams of devices, apparatuses, equipment, and systems involved in the present invention are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, equipment, and systems can be connected, arranged, and configured in any manner. Words such as "comprising," "including," "having," etc., are open-ended terms meaning "including but not limited to," and are used interchangeably with them. The terms "or" and "and" as used herein refer to the terms "and / or," and are used interchangeably with them unless the context explicitly indicates otherwise. The term "such as" as used herein refers to the phrase "such as but not limited to," and is used interchangeably with it.

[0069] Additionally, as used herein, the "or" used in a list of items beginning with "at least one" indicates a separate list, such that a list of, for example, "at least one of A, B, or C" means A or B or C, or AB or AC or BC, or ABC (i.e., A and B and C). Furthermore, the word "exemplary" does not imply that the described example is preferred or better than other examples. It should also be noted that in the systems and methods of the present invention, the components or steps can be decomposed and / or recombined. Such decomposition and / or recombination should be considered equivalent to the present invention.

[0070] Various changes, substitutions, and modifications can be made to the technology described herein without departing from the teachings defined by the appended claims. Furthermore, the scope of the claims is not limited to the specific aspects of the processes, machines, manufacturing processes, events, means, methods, and actions described above. Currently existing or later-developed processes, machines, manufacturing processes, events, means, methods, or actions that perform substantially the same function or achieve substantially the same result as the corresponding aspects described herein can be utilized. Therefore, the appended claims include such processes, machines, manufacturing processes, events, means, methods, or actions within their scope.

[0071] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use the invention. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein can be applied to other aspects without departing from the scope of the invention. Therefore, the invention is not intended to be limited to the aspects shown herein, but rather to be carried out within the widest scope consistent with the principles and novel features disclosed herein.

[0072] The above description has been given for purposes of illustration and description. Furthermore, this description is not intended to limit the embodiments of the invention to the forms disclosed herein. Although numerous exemplary aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations therein.

Claims

1. A method for predicting high-load periods on a host machine, characterized in that, include: S101. Based on the performance index data of the previous load of the host machine, establish a long-term LSTM integrated network model and a short-term LSTM integrated network model. Then, perform weighted processing on the long-term LSTM integrated network model and the short-term LSTM integrated network model to obtain an optimized integrated LSTM network model. S102, acquire performance index data related to the current host machine and its load, and preprocess the performance index data; S103, based on the current time, input the preprocessed performance index data into the optimized integrated LSTM network model for prediction, and obtain the predicted load value of the current host in the future. S104, obtain the host machine high load threshold and determine whether the current host machine load prediction value is greater than the high load threshold; If so, it is predicted that the current host machine will be in a period of high load in the future; If not, the current host machine is operating normally.

2. The method as described in claim 1, characterized in that, Step S101 includes: Obtain performance metrics data of the host machine's previous load and preprocess them; The preprocessed performance metric data is converted into historical time series data to obtain the training dataset and the test dataset; The first predicted load value for the long period of time is obtained by capturing historical time series data within the training dataset using a long-term LSTM ensemble network model; the second predicted load value is obtained by capturing historical time series data within the short period of time of time in the training dataset using a short-term LSTM ensemble network model. By inputting historical time series data over a long period in the test dataset into a long-term LSTM ensemble network model, the third predicted load value for that long period is obtained; by inputting historical time series data over a short period in the test dataset into a short-term LSTM ensemble network model, the fourth predicted load value for that short period is obtained. The optimized LSTM network model is obtained by comprehensively evaluating and allocating weights based on the first and third load predictions of the long-term LSTM ensemble network model and the third and fourth load predictions of the short-term LSTM ensemble network model.

3. The method as described in claim 2, characterized in that, Step S102 also includes: The performance of the ensemble LSTM network model is evaluated based on the first and second load predictions of the long-term LSTM ensemble network model and the third and fourth load predictions or cross-validation results of the short-term LSTM ensemble network model. The root mean square error of the first, second, third, and fourth load predictions is used to verify whether the integrated LSTM network model meets the prediction accuracy requirements. If so, the accuracy of the predictions of the integrated LSTM network model is evaluated. If not, the parameters will be automatically adjusted, and the integrated LSTM network model will be returned. The weight allocation of long-term and short-term LSTM ensemble network models is comprehensively evaluated based on the performance and prediction accuracy of the ensemble LSTM network model. The prediction results of the long-term LSTM ensemble network model and the short-term LSTM ensemble network model are weighted and averaged according to the assigned weights to calculate the comprehensive prediction result of the host load.

4. The method as described in claim 3, characterized in that, During the optimization of the integrated LSTM network model, a high load threshold for the host machine is set based on the comprehensive prediction results of the host machine load during high load periods. When the predicted host load value of the test dataset is greater than the high load threshold, it is predicted that the host will be in a high load period in the future.

5. The method according to any one of claims 1 to 4, characterized in that, The load performance metrics data include the load performance metrics of the host machine and the computing resources it supports, and external time series data related to the load; the external time series data includes performance metrics data of at least one external factor related to the load, such as weather, holidays, indoor activities, video entertainment, business activity arrangements, and media events. The steps for preprocessing the performance index data include: data parsing, formatting, and filtering of the performance index data.

6. The method according to any one of claims 1 to 4, characterized in that, The step of acquiring performance index data related to the current host machine and its load includes: collecting load performance indexes of the host machine and the computing resources it carries based on the integrated acquisition components; The acquisition component exposes the acquired performance indicator data as an accessible interface, provides the performance indicator data to the data storage stack and other monitoring components, and is compatible with the monitoring data format alarms of the monitoring components.

7. The method as described in claim 6, characterized in that, The working process of the host load performance metric stack includes: Logstash is used as the middleware for data collection and processing. In the Logstash configuration file, the input plugin is used to receive performance metric data from the collection component; the filter plugin is used to process and transform the received performance metric data; according to the data format of the collection component, the corresponding filter plugin is used to parse, format, and filter the data; the output plugin is used to send the processed data to Elasticsearch; by configuring the Elasticsearch address, index name, and other relevant parameters, the data is written to the specified Elasticsearch index. By connecting to Elasticsearch using Kibana, you can visualize and analyze imported data by creating index schemas and dashboards; By connecting load performance metrics data to Prometheus and using it as a data source for Grafana, Grafana can be used to create charts and display various load performance metrics data.

8. The method as described in claim 1, characterized in that, The process between steps S102 and S103 also includes: The preprocessed and performance metrics data are transformed into time series data, which is configured as a sliding window, including a longer time window and a shorter time window. Step S104 further includes that, after calculating the load prediction value, the current host machine and its load-related performance index data are configured as the previous load performance index data of the host machine, and then the process re-enters step S101.

9. A device for predicting high-load periods of a host machine, characterized in that, include An integrated LSTM network module is used to build long-term and short-term LSTM integrated network models based on the performance index data of the previous load of the host machine. The long-term and short-term LSTM integrated network models are weighted to obtain an optimized integrated LSTM network model. The integrated LSTM network model includes a gating unit, which controls the flow of performance index data through forget gates, input gates and output gates to realize the processing of long sequence data and the learning of long-term dependencies. The performance metrics data module is used to acquire performance metrics data related to the current host machine and its load, and to preprocess the performance metrics data. The load prediction module is used to input preprocessed performance index data into an optimized integrated LSTM network model based on the current time to predict the future load of the current host. The comparison module is used to obtain the host machine's high load threshold and determine whether the current host machine's predicted load value is greater than the high load threshold. If so, it is predicted that the current host machine will be in a period of high load in the future; otherwise, the current host machine is operating normally.

10. A host machine high-load period prediction system, characterized in that, This includes cloud hosts and the computing power resources they support, including containers, bare metal servers, and virtual networks; The cloud host includes several host machines deployed in a cluster. Each host machine predicts the load trend according to the steps of the host machine high load period prediction method according to any one of claims 1 to 8, and automatically triggers the reallocation and migration strategy of host machine resource scheduling. By migrating some computing resources from high-load host machines to idle or low-load host machines, host machine load balancing and resource optimization are achieved.

Images