Method and system for collecting cloud host monitoring data

Abstract

The application relates to the computer technical field, in particular to a cloud host monitoring data collection method and system thereof. The cloud host monitoring data collection method comprises the following steps: obtaining historical data of a target cloud host; calculating a fitting function based on the historical data; determining a target data set in the historical data, calculating a linear deviation average value of the target data set, and calculating a historical average value of the target data set based on the fitting function; calculating a linear deviation rate of the target cloud host based on the linear deviation average value and the historical average value; calculating a monitoring collection period of the target cloud host based on the linear deviation rate; and collecting monitoring data of the target cloud host based on the monitoring collection period. The linear deviation rate of the target cloud host is calculated through the historical data of the target cloud host, the linear deviation rate of the target cloud host can reflect the proportion of the data fluctuation of the target cloud host relative to the overall trend, and the dynamic adjustment of the monitoring data collection frequency of the target cloud host is realized.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a method and system for collecting cloud host monitoring data. Background Technology

[0002] In cloud computing environments, resource monitoring is fundamental to ensuring service quality, optimizing resource utilization, and promptly detecting system failures. Resource monitoring not only helps maintain service level agreements (SLAs) but is also crucial for cost control and system health.

[0003] However, traditional monitoring systems often use a fixed sampling frequency (periodic sampling method). This static approach cannot effectively cope with dynamic changes in resource demand, leading to resource waste or untimely monitoring. For example, during peak load periods, a fixed frequency may not provide enough data to make timely resource adjustment decisions, while during low load periods, it may generate a large amount of unnecessary data collection, resulting in resource waste.

[0004] Therefore, developing a cloud host monitoring data collection method that can adaptively adjust monitoring strategies has become an urgent need. Summary of the Invention

[0005] Therefore, it is necessary to provide a method and system for collecting cloud host monitoring data that can improve sampling efficiency in response to the above-mentioned technical problems.

[0006] Firstly, this application provides a method for collecting cloud server monitoring data, including:

[0007] Obtain historical data from the target cloud host;

[0008] Based on the historical data, calculate the fitting function;

[0009] The target dataset is determined from the historical data, the average linear deviation of the target dataset is calculated, and the historical average of the target dataset is calculated based on the fitting function.

[0010] The linear deviation rate of the target cloud host is calculated based on the average linear deviation and the historical average.

[0011] Based on the linear deviation rate, the monitoring and data collection cycle of the target cloud host is calculated;

[0012] Based on the aforementioned monitoring collection period, monitoring data of the target cloud host is collected.

[0013] In one embodiment, calculating the monitoring and data collection cycle of the target cloud host based on the linear deviation rate includes:

[0014] Determine the target sensitivity;

[0015] Based on the target sensitivity, calculate the sampling data window value and / or jitter data window value of the target cloud host.

[0016] In one embodiment, calculating the monitoring and data collection cycle of the target cloud host based on the linear deviation rate includes:

[0017] Based on the linear deviation rate and the threshold, the monitoring and collection cycle of the target cloud host is determined.

[0018] In one embodiment, if the linear deviation rate is not less than a threshold, the maximum collection time is determined from the historical data of the target cloud host;

[0019] The monitoring and collection cycle of the target cloud host is determined based on the collection interval time corresponding to the maximum collection time.

[0020] In one embodiment, if the linear deviation rate is less than a threshold, the maximum and minimum collection times are determined from the historical data of the target cloud host.

[0021] Based on the maximum acquisition time, the minimum acquisition time, and the linear deviation rate, the monitoring acquisition cycle of the target cloud host is determined.

[0022] In one embodiment, after collecting monitoring data of the target cloud host based on the monitoring collection period, the process includes:

[0023] The monitoring data collected from the target cloud host is stored.

[0024] Secondly, this application also provides a cloud server monitoring data collection system, including:

[0025] The cloud center component is used to manage cloud servers;

[0026] An edge cluster component connects to the cloud center component and performs the cloud host monitoring data collection method as described in any of the preceding items.

[0027] In one embodiment, the edge cluster component includes a data acquisition parameter configuration module, which configures a data acquisition list in response to a configuration instruction sent by the cloud center component.

[0028] In one embodiment, the edge cluster component includes a monitoring data acquisition module that connects to multiple cloud hosts and acquires historical data from the multiple cloud hosts.

[0029] In one embodiment, the edge cluster component includes a sampling frequency adjustment module and / or a monitoring data storage module, wherein the sampling frequency adjustment module is used to adjust the sampling frequency of the target cloud host, and the monitoring data storage module is used to store the monitoring data of the target cloud host.

[0030] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0031] Obtain historical data from the target cloud host;

[0032] Based on the historical data, calculate the fitting function;

[0033] The target dataset is determined from the historical data, the average linear deviation of the target dataset is calculated, and the historical average of the target dataset is calculated based on the fitting function.

[0034] The linear deviation rate of the target cloud host is calculated based on the average linear deviation and the historical average.

[0035] Based on the linear deviation rate, the monitoring and data collection cycle of the target cloud host is calculated;

[0036] Based on the aforementioned monitoring collection period, monitoring data of the target cloud host is collected.

[0037] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, performs the following steps:

[0038] Obtain historical data from the target cloud host;

[0039] Based on the historical data, calculate the fitting function;

[0040] The target dataset is determined from the historical data, the average linear deviation of the target dataset is calculated, and the historical average of the target dataset is calculated based on the fitting function.

[0041] The linear deviation rate of the target cloud host is calculated based on the average linear deviation and the historical average.

[0042] Based on the linear deviation rate, the monitoring and data collection cycle of the target cloud host is calculated;

[0043] Based on the aforementioned monitoring collection period, monitoring data of the target cloud host is collected.

[0044] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, performs the following steps:

[0045] Obtain historical data from the target cloud host;

[0046] Based on the historical data, calculate the fitting function;

[0047] The target dataset is determined from the historical data, the average linear deviation of the target dataset is calculated, and the historical average of the target dataset is calculated based on the fitting function.

[0048] The linear deviation rate of the target cloud host is calculated based on the average linear deviation and the historical average.

[0049] Based on the linear deviation rate, the monitoring and data collection cycle of the target cloud host is calculated;

[0050] Based on the aforementioned monitoring collection period, monitoring data of the target cloud host is collected.

[0051] The aforementioned cloud server monitoring data collection method and system calculate the linear deviation rate of the target cloud server using historical data. This linear deviation rate reflects the proportion of data fluctuations of the target cloud server relative to the overall trend, thereby predicting the future trend of key performance indicators. This allows for dynamic adjustment of the monitoring data collection frequency, achieving maximum information acquisition efficiency with minimal data collection, thus enhancing the information value of the monitoring data. Specifically, this application can reduce data collection when resource usage is relatively stable, thereby alleviating the burden on cloud resources. Conversely, when resource usage changes drastically, it can increase the collection frequency to promptly capture key performance changes and reduce data distortion caused by insufficient sampling. Attached Figure Description

[0052] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0053] Figure 1 This is a flowchart illustrating the method for collecting cloud host monitoring data in the first embodiment;

[0054] Figure 2 This is a flowchart illustrating the method for collecting cloud host monitoring data in the second embodiment;

[0055] Figure 3 This is a flowchart illustrating the method for collecting cloud host monitoring data in the third embodiment;

[0056] Figure 4 This is a flowchart illustrating the method for collecting cloud host monitoring data in the fourth embodiment;

[0057] Figure 5 This is a flowchart illustrating the method for collecting cloud host monitoring data in the fifth embodiment;

[0058] Figure 6 This is a flowchart illustrating the method for collecting cloud host monitoring data in the sixth embodiment;

[0059] Figure 7 This is a structural block diagram of a cloud host monitoring data acquisition system in one embodiment;

[0060] Figure 8 This is a flowchart illustrating the acquisition frequency adjustment module in one embodiment;

[0061] Figure 9 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0062] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0063] As mentioned in the background section, traditional monitoring systems often employ a fixed sampling frequency (periodic sampling method). This static approach cannot effectively address dynamic changes in resource demand, leading to resource waste or untimely monitoring. In periodic sampling methods, a fixed sampling time interval is typically pre-set. When the set sampling time point is reached, the monitoring software collects and stores relevant monitoring metrics from the cloud host. Furthermore, traditional technologies also include threshold-based event-driven methods. In threshold-based event-driven methods, various thresholds for resource usage are preset. Once the actual resource usage reaches or exceeds these thresholds, the sampling frequency is increased or decreased.

[0064] However, the inventors discovered that both periodic sampling methods and threshold-based event-driven methods have drawbacks. Periodic sampling methods are limited in their adaptability when dealing with non-linear or rapidly changing data. A fixed sampling interval may fail to capture key details during periods of drastic data fluctuation, resulting in the omission of important information at critical moments. Furthermore, a fixed sampling frequency leads to insufficient information density in the monitoring data, failing to adequately cover all key change points, especially when resource usage spikes or abrupt changes. This results in smoothed analysis results, failing to accurately reflect sudden performance issues, thus affecting the accuracy of fault diagnosis and performance optimization. Moreover, due to the constant sampling frequency, periodic sampling methods may fail to provide timely updates when data changes rapidly, thus failing to accurately reflect the system's real-time status. This is a significant drawback for cloud environments requiring rapid response. In threshold-based event-driven methods, when resource usage changes are relatively gradual, threshold-based event-driven methods may trigger excessively frequent sampling, generating a large amount of redundant data. This not only increases the storage and processing burden on the monitoring system but may also put additional resource pressure on cloud hosts, especially when the cloud hosts are already under sustained high load, where the impact of this additional pressure is more significant. Furthermore… When resource metrics change rapidly, but the magnitude of the change has not yet reached a preset threshold, threshold-based event-driven methods may fail to respond to this change in a timely manner, causing the sampling frequency to lag behind the actual rate of resource change. This delay may result in insufficient sampling, making it impossible to accurately capture the real-time trend of resource usage changes.

[0065] Based on this, several embodiments of this application provide a method and system for collecting cloud host monitoring data, which dynamically and efficiently monitors the resource usage of cloud hosts through an adaptive algorithm.

[0066] The cloud host monitoring data collection method provided in this application can be applied to environments where terminals and servers interact. Terminals can be, but are not limited to, various personal computers, laptops, smartphones, tablets, and IoT devices. Servers can be independent physical servers, server clusters or distributed systems composed of multiple physical servers, or cloud servers providing cloud computing services.

[0067] In one embodiment, such as Figure 1 As shown, a method for collecting monitoring data from a cloud server is provided. This embodiment illustrates the method by applying it to a terminal. It is understood that this method can also be applied to a server, and to a system including both a terminal and a server, and is implemented through interaction between the terminal and the server. In this embodiment, the method includes the following steps: wherein:

[0068] Step S100: Obtain historical data of the target cloud host.

[0069] Step S200: Calculate the fitting function based on historical data.

[0070] Step S300: Determine the target dataset from historical data, calculate the average linear deviation of the target dataset, and calculate the historical average of the target dataset based on the fitting function.

[0071] Step S400: Calculate the linear deviation rate of the target cloud host based on the average linear deviation and the historical average.

[0072] Step S500: Calculate the monitoring and data collection cycle of the target cloud host based on the linear deviation rate.

[0073] Step S600: Collect monitoring data of the target cloud host based on the monitoring collection cycle.

[0074] In step S100, performance data of the target cloud host over a past period of time is obtained through the cloud platform's API interface, or through a local database or log file. For example, key indicators such as CPU utilization, memory usage, disk I / O, and network traffic of the target cloud host can be obtained.

[0075] In step S200, data fitting can be performed using a univariate linear method to obtain a fitting function that can describe the data trend of the target cloud host. This embodiment does not limit the specific form of the fitting function.

[0076] In step S300, as an example, the target dataset can be the first M data points sampled most recently. Then, the average linear deviation between the target dataset and the fitted function can be calculated to assess the data volatility. Additionally, the historical average of the target dataset based on the fitted function can be calculated to understand the overall trend of the data.

[0077] In step S400, a ratio reflecting data fluctuations relative to the overall trend, namely the linear deviation rate, can be calculated by comparing the average linear deviation with the historical average. As an example, the linear deviation rate can be equal to the ratio of the average linear deviation to the historical average.

[0078] In step S500, the data acquisition cycle of the monitoring data can be dynamically adjusted according to the degree of data fluctuation. For example, a threshold or rule can be set, such as shortening the acquisition cycle when the linear deviation rate exceeds a certain value, and extending the acquisition cycle when the linear deviation rate is below a certain value.

[0079] In step S600, the performance data of the target cloud host can be collected periodically according to the calculated monitoring and collection cycle, using a timer or scheduler.

[0080] In this embodiment, the linear deviation rate of the target cloud host is calculated using historical data. This linear deviation rate reflects the proportion of the target cloud host's data fluctuations relative to the overall trend, thereby allowing for the prediction of future trends in key performance indicators. This enables dynamic adjustment of the monitoring data collection frequency for the target cloud host, achieving maximum information acquisition efficiency with minimal data collection, thus enhancing the information value of the monitoring data. Specifically, this embodiment can reduce data collection when resource usage is relatively stable, thereby alleviating the burden on cloud resources. Conversely, when resource usage changes drastically, the collection frequency can be increased to promptly capture key performance changes and reduce data distortion caused by insufficient sampling.

[0081] In one embodiment, see Figure 2 Step S500 includes:

[0082] Step S510: Determine the target sensitivity.

[0083] Step S511: Based on the target sensitivity, calculate the sampling data window value and / or jitter data window value of the target cloud host.

[0084] In step S510, sensitivity can be used to determine whether changes in cloud server performance data are significant. The target sensitivity can be a fixed value, or a percentage or standard deviation dynamically calculated based on historical data.

[0085] In step S511, two key parameters for monitoring data acquisition can be determined based on the target sensitivity: the sampling data window value and the jitter data window value. The sampling data window value can define the time range or number of data points considered for each data acquisition. A larger window value can smooth out data fluctuations but may mask significant changes over a short period. A smaller window value can more sensitively capture data changes but may increase noise. The jitter data window value can be used to determine whether the data has changed significantly (i.e., jitter) over a short period.

[0086] In this embodiment, the sampling data window value and jitter data window value can be set according to different monitoring indicators and the required sensitivity, thereby reducing the impact of sampling on the operating conditions. Specifically, increasing the window width can increase the number of sampling points for jitter ratio calculation, thereby reducing the impact of individual data mutations on the jitter ratio, but it will also reduce sensitivity. Similarly, decreasing the window width will increase sensitivity, but it will lead to a high false positive rate when encountering data with mutations, increasing unnecessary sampling frequency.

[0087] In one embodiment, see Figure 3 Step S500 includes:

[0088] Step S520: Determine the monitoring and collection cycle of the target cloud host based on the linear deviation rate and the threshold.

[0089] The linear deviation rate can be used to assess the volatility of target cloud server performance data, and a preset threshold can be used to determine whether the monitoring and data collection cycle needs to be adjusted. For example, if the linear deviation rate exceeds the threshold, it indicates significant data volatility, requiring more frequent data collection. Conversely, the collection frequency can be appropriately reduced if the linear deviation rate is below the threshold.

[0090] For details, please refer to Figure 4 In one possible example, step S520 may include:

[0091] Step S521: If the linear deviation rate is not less than the threshold, determine the maximum collection time from the historical data of the target cloud host.

[0092] Step S522: Determine the monitoring and collection cycle of the target cloud host based on the collection interval time corresponding to the maximum collection time.

[0093] In steps S521 and S522, a linear deviation rate not less than a threshold indicates significant data fluctuation. In this case, the maximum acquisition time can be found in historical data. Then, within the maximum acquisition time, the average time interval between adjacent data points is calculated, which serves as the new monitoring acquisition cycle.

[0094] In another possible example, see Figure 5 Step S520 may include:

[0095] Step S523: If the linear deviation rate is less than the threshold, determine the maximum and minimum collection times from the historical data of the target cloud host.

[0096] Step S524: Determine the monitoring and collection cycle of the target cloud host based on the maximum collection time, minimum collection time, and linear deviation rate.

[0097] In steps S523 and S524, a linear deviation rate less than the threshold indicates relatively small data fluctuations. At this point, historical data can be analyzed to identify the maximum and minimum acquisition times. Then, by combining the maximum and minimum acquisition times, and the linear deviation rate, a more reasonable monitoring acquisition cycle can be calculated. As an example, the threshold can be 1.

[0098] In this embodiment, the monitoring and data collection cycle is dynamically adjusted based on the fluctuations in the target cloud host's performance data, thereby reducing collection costs and storage requirements while ensuring data accuracy. This helps optimize the cloud host's performance monitoring strategy, improving operational efficiency and system stability.

[0099] In one embodiment, see Figure 6Step S600 includes:

[0100] Step S700: Store the collected monitoring data of the target cloud host.

[0101] First, this ensures that the collected monitoring data is saved completely and accurately. Then, the monitoring data collected from the target cloud host can serve as a data source for subsequent data analysis, performance evaluation, and troubleshooting. Specifically, relational databases (such as MySQL and PostgreSQL) can be used to store the monitoring data. Alternatively, the monitoring data collected from the target cloud host can be stored as files on a local file system or a distributed file system, or it can be stored using the storage services provided by a cloud service provider.

[0102] In this embodiment, by effectively storing the collected monitoring data of the target cloud host, reliable data support is provided for subsequent data analysis and performance monitoring to adapt to ever-changing business needs.

[0103] The following example illustrates the cloud server monitoring data collection method provided in this application. First, for a certain cloud server E that needs to have its monitoring data collected... i To obtain the following configuration: Maximum acquisition time T max Minimum acquisition time T min The system requires a list of collected metrics F, a sampling data window value N, and a jitter data window value M. The sampling data window value N and the jitter data window value M can be set according to different monitoring metrics and the required sensitivity. Then, the system can read the most recent N historical data sets D of the required metrics F for this cloud host from the monitoring data storage module. For historical data D1 to D2... n Given N sampled values, a univariate linear regression method is used to fit the data. The univariate linear regression model is as follows:

[0104] y = β0 + β1x

[0105] D = {(x1,y1),(x2,y2),...,(x n ,y n )} can represent the sample set, x1 <x2<…<x n This represents the sampling time point, corresponding to y. n This represents the actual data of the monitoring indicators. The least squares method is used to solve for β0 and β1, resulting in the following system of equations:

[0106]

[0107] Further, it is necessary to calculate the linear deviation rate. At this point, select the first M data points from the most recent sampling and calculate the average linear deviation:

[0108]

[0109] Historical average of the first M data points:

[0110]

[0111] The linear deviation rate can be expressed as:

[0112]

[0113] The linear deviation rate reflects the linear deviation of the indicators collected by the monitoring system. The larger the value, the more drastic the non-linear change in the data, and the shorter the data collection interval should be; conversely, the shorter the value, the shorter the data collection interval should be. Simultaneously, the linear deviation rate is a ratio of equal magnitudes, which can accommodate differences in the magnitude of different monitoring indicators. Finally, the collection time can be updated. The data collection interval of the monitoring system should be adjusted in real time according to the linear deviation rate. When... If the value is greater than 1, the monitoring indicator is considered to be changing drastically in a non-linear manner, and the minimum data collection interval should be used; otherwise, according to... Adjust the data collection interval based on the value.

[0114]

[0115] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0116] Based on the same inventive concept, please refer to Figure 7 and Figure 8 In one embodiment, a cloud server monitoring data acquisition system is provided. This system includes a cloud center component and edge cluster components. Specifically, the cloud center component can connect to multiple edge cluster components.

[0117] The cloud center has a built-in cloud server management module, which is responsible for the management of cloud servers throughout their entire lifecycle and ensures that monitoring configurations are synchronized to the data collection parameter configuration module in real time. The edge cluster deploys a cloud server monitoring service, which is responsible for the comprehensive monitoring of multiple cloud servers, including key indicators such as CPU utilization, memory usage, disk I / O, and network traffic, and supports the flexible configuration needs of cloud servers.

[0118] The edge cluster component can implement the cloud host monitoring data collection method provided in one or more of the aforementioned embodiments. Specifically, the edge cluster component includes a collection frequency adjustment module, a monitoring data collection module, a monitoring data storage module, and a collection parameter configuration module.

[0119] Specifically, the parameter configuration acquisition module is used to read parameter configurations and refine configuration management, including the list of cloud hosts to be monitored, E = {E1, E2, ..., E...}. n The list of collected metrics for the cloud server is F = {F1, F2, ..., F}. n This ensures personalized deployment of monitoring strategies. For example, specifically, before step S100, the cloud center selects an edge cluster, starts a cloud host, and immediately pushes the monitoring configuration to the collection configuration module of the monitoring service. Under the guidance of the cloud host management module, the cloud host is activated, and its monitoring configuration instructions are passed to the collection configuration module of the cloud host monitoring service. The configuration specifies the monitoring metrics (including CPU utilization F...). cpu and memory usage F mem ), and their respective maximum and minimum acquisition periods T max T min The sampling data window value is N, and the jitter data window value is M. At this time, the monitoring configuration items of each cloud host (such as the collection indicators, window size, etc.) can be flexibly adjusted according to business needs and can be updated in real time through the cloud host configuration channel.

[0120] The data acquisition frequency adjustment module can read historical data, perform linear regression prediction, calculate the linear deviation rate, and update the upcoming acquisition period T accordingly. The monitoring data acquisition module communicates with the cloud host's agent according to the set acquisition period T to collect monitoring data. Afterward, the monitoring data acquisition module writes the collected data to the monitoring data storage module, achieving persistent data storage and subsequent querying.

[0121] The data acquisition frequency adjustment module can be responsible for monitoring data model regression and dynamically adjusting the acquisition frequency. As an example, a data acquisition module named `node_exporter` can be deployed to collect key performance indicators, including CPU utilization (F). cpu Memory usage F mem and disk I / O activity F IOThis module captures data through proxy communication with the cloud server, and its monitoring metrics can be flexibly selected according to the configuration file, ensuring a high degree of customization. The data collection frequency adjustment module can be activated before monitoring data collection begins. (The CPU utilization F...) cpu For example, after the last data collection was completed, the collection frequency adjustment module adjusted the frequency according to the historical window size N. cpu Data is extracted from the monitoring data storage module. If the data in the storage module is less than the sampling data window size, the acquisition time is updated. Otherwise, based on the set window size, the data acquisition frequency adjustment module collects data and performs linear regression analysis to derive the model y = β0 + β1x, where y1, y2, ..., y n , where is the regression value for each sampling point, based on the jitter window size M. cpu Calculate the linear mean deviation of the nearest M data points and obtain the average value of these M data points. Based on this, calculate the linear deviation ratio. It reflects the drastic degree of data change. Furthermore, the sampling frequency is adjusted based on the linear jitter ratio. If the linear deviation ratio... A value greater than 1 indicates significant data fluctuations; therefore, the next data collection interval should be adjusted to the minimum value. When linear jitter ratio If the jitter is considered to be gentle, then it is considered to have gentle fluctuations, and the sampling interval is adjusted to... The data acquisition frequency module initiates the next round of data capture tasks. The process for collecting memory usage is similar to that for CPU usage, following the same steps described above to ensure consistency and comprehensiveness in resource monitoring.

[0122] The monitoring data acquisition module can communicate with the cloud server agent to collect monitoring metrics and store the collected data in the monitoring data storage module. The monitoring data storage module is used to centrally store monitoring data from various cloud servers, supporting subsequent analysis and querying.

[0123] The modules in the aforementioned cloud server monitoring data acquisition system can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of a computer device in hardware form or independent of it, or stored in the memory of the computer device in software form, so that the processor can call and execute the corresponding operations of each module.

[0124] In one exemplary embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 9As shown, the computer device includes a processor, memory, input / output interfaces (I / O), and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operating system and computer programs stored in the non-volatile storage media. The database stores collected cloud host monitoring data. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network connection. When the computer program is executed by the processor, it implements a method for collecting cloud host monitoring data.

[0125] Those skilled in the art will understand that Figure 9 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0126] In one exemplary embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0127] Step S100: Obtain historical data of the target cloud host.

[0128] Step S200: Calculate the fitting function based on historical data.

[0129] Step S300: Determine the target dataset from historical data, calculate the average linear deviation of the target dataset, and calculate the historical average of the target dataset based on the fitting function.

[0130] Step S400: Calculate the linear deviation rate of the target cloud host based on the average linear deviation and the historical average.

[0131] Step S500: Calculate the monitoring and data collection cycle of the target cloud host based on the linear deviation rate.

[0132] Step S600: Collect monitoring data of the target cloud host based on the monitoring collection cycle.

[0133] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0134] Step S510: Determine the target sensitivity.

[0135] Step S511: Based on the target sensitivity, calculate the sampling data window value and / or jitter data window value of the target cloud host.

[0136] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0137] Step S520: Determine the monitoring and collection cycle of the target cloud host based on the linear deviation rate and the threshold.

[0138] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0139] Step S521: If the linear deviation rate is not less than the threshold, determine the maximum collection time from the historical data of the target cloud host.

[0140] Step S522: Determine the monitoring and collection cycle of the target cloud host based on the collection interval time corresponding to the maximum collection time.

[0141] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0142] Step S523: If the linear deviation rate is less than the threshold, determine the maximum and minimum collection times from the historical data of the target cloud host.

[0143] Step S524: Determine the monitoring and collection cycle of the target cloud host based on the maximum collection time, minimum collection time, and linear deviation rate.

[0144] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0145] Step S700: Store the collected monitoring data of the target cloud host.

[0146] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program performing the following steps when executed by a processor:

[0147] Step S100: Obtain historical data of the target cloud host.

[0148] Step S200: Calculate the fitting function based on historical data.

[0149] Step S300: Determine the target dataset from historical data, calculate the average linear deviation of the target dataset, and calculate the historical average of the target dataset based on the fitting function.

[0150] Step S400: Calculate the linear deviation rate of the target cloud host based on the average linear deviation and the historical average.

[0151] Step S500: Calculate the monitoring and data collection cycle of the target cloud host based on the linear deviation rate.

[0152] Step S600: Collect monitoring data of the target cloud host based on the monitoring collection cycle.

[0153] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0154] Step S510: Determine the target sensitivity.

[0155] Step S511: Based on the target sensitivity, calculate the sampling data window value and / or jitter data window value of the target cloud host.

[0156] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0157] Step S520: Determine the monitoring and collection cycle of the target cloud host based on the linear deviation rate and the threshold.

[0158] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0159] Step S521: If the linear deviation rate is not less than the threshold, determine the maximum collection time from the historical data of the target cloud host.

[0160] Step S522: Determine the monitoring and collection cycle of the target cloud host based on the collection interval time corresponding to the maximum collection time.

[0161] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0162] Step S523: If the linear deviation rate is less than the threshold, determine the maximum and minimum collection times from the historical data of the target cloud host.

[0163] Step S524: Determine the monitoring and collection cycle of the target cloud host based on the maximum collection time, minimum collection time, and linear deviation rate.

[0164] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0165] Step S700: Store the collected monitoring data of the target cloud host.

[0166] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, performs the following steps:

[0167] Step S100: Obtain historical data of the target cloud host.

[0168] Step S200: Calculate the fitting function based on historical data.

[0169] Step S300: Determine the target dataset from historical data, calculate the average linear deviation of the target dataset, and calculate the historical average of the target dataset based on the fitting function.

[0170] Step S400: Calculate the linear deviation rate of the target cloud host based on the average linear deviation and the historical average.

[0171] Step S500: Calculate the monitoring and data collection cycle of the target cloud host based on the linear deviation rate.

[0172] Step S600: Collect monitoring data of the target cloud host based on the monitoring collection cycle.

[0173] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0174] Step S510: Determine the target sensitivity.

[0175] Step S511: Based on the target sensitivity, calculate the sampling data window value and / or jitter data window value of the target cloud host.

[0176] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0177] Step S520: Determine the monitoring and collection cycle of the target cloud host based on the linear deviation rate and the threshold.

[0178] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0179] Step S521: If the linear deviation rate is not less than the threshold, determine the maximum collection time from the historical data of the target cloud host.

[0180] Step S522: Determine the monitoring and collection cycle of the target cloud host based on the collection interval time corresponding to the maximum collection time.

[0181] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0182] Step S523: If the linear deviation rate is less than the threshold, determine the maximum and minimum collection times from the historical data of the target cloud host.

[0183] Step S524: Determine the monitoring and collection cycle of the target cloud host based on the maximum collection time, minimum collection time, and linear deviation rate.

[0184] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0185] Step S700: Store the collected monitoring data of the target cloud host.

[0186] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0187] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0188] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for collecting monitoring data from a cloud server, characterized in that, The method includes: Obtain historical data from the target cloud host; Based on the historical data, calculate the fitting function; The target dataset is determined from the historical data, the average linear deviation of the target dataset is calculated, and the historical average of the target dataset is calculated based on the fitting function. The linear deviation rate of the target cloud host is calculated based on the average linear deviation and the historical average. The target sensitivity is determined, and the monitoring and acquisition cycle of the target cloud host is calculated based on the linear deviation rate and the threshold. Specifically, if the linear deviation rate is not less than the threshold, the maximum acquisition time is determined from the historical data of the target cloud host, and the monitoring and acquisition cycle of the target cloud host is determined based on the acquisition interval corresponding to the maximum acquisition time. If the linear deviation rate is less than the threshold, the maximum and minimum acquisition times are determined from the historical data of the target cloud host, and the monitoring and acquisition cycle of the target cloud host is determined based on the maximum acquisition time, the minimum acquisition time, and the linear deviation rate. Based on the aforementioned monitoring collection period, monitoring data of the target cloud host is collected.

2. The method for collecting cloud host monitoring data according to claim 1, characterized in that, The calculation of the fitting function includes: performing data fitting using a univariate linear method to obtain a fitting function that can describe the data trend of the target cloud host.

3. The method for collecting cloud host monitoring data according to claim 1, characterized in that, The linear deviation rate is equal to the ratio of the average linear deviation to the historical average.

4. The method for collecting cloud host monitoring data according to claim 1, characterized in that, Based on the linear deviation rate and the threshold, the monitoring and data collection cycle for the target cloud host is calculated as follows: When the linear deviation rate exceeds the set threshold, the acquisition cycle is shortened; when the linear deviation rate is below the set threshold, the acquisition cycle is extended.

5. The method for collecting cloud host monitoring data according to claim 2, characterized in that, The data fitting using a univariate linear method includes: Read the required metrics from the cloud host in the monitoring data storage module The recent A set of historical data Historical data arrive of For each sampled value, a univariate linear regression method is used to fit the data. The univariate linear regression model is as follows: y=β0+β1x Let x1 represent the sample set. <x2<…<x n Indicates the sampling time point, corresponding to This represents the actual data for the monitoring indicators.

6. The method for collecting cloud host monitoring data according to claim 1, characterized in that, After collecting monitoring data from the target cloud host based on the monitoring collection period, the process includes: The monitoring data collected from the target cloud host is stored.

7. A cloud server monitoring data acquisition system, characterized in that, include: The cloud center component is used to manage cloud servers; An edge cluster component connects to the cloud center component and performs the cloud host monitoring data collection method as described in any one of claims 1-6.

8. The cloud host monitoring data acquisition system according to claim 7, characterized in that, The edge cluster component includes a data acquisition parameter configuration module, which configures the data acquisition list in response to configuration instructions sent by the cloud center component.

9. The cloud host monitoring data acquisition system according to claim 7, characterized in that, The edge cluster component includes a monitoring data acquisition module, which connects to multiple cloud hosts and acquires historical data from these cloud hosts.

10. The cloud host monitoring data acquisition system according to claim 7, characterized in that, The edge cluster component includes a data acquisition frequency adjustment module and / or a monitoring data storage module. The data acquisition frequency adjustment module is used to adjust the data acquisition frequency of the target cloud host, and the monitoring data storage module is used to store the monitoring data of the target cloud host.

Images