Data monitoring method and server

Abstract

The embodiment of the invention discloses a data monitoring method and a server, and the method comprises the steps: determining monitoring indexes corresponding to one or more first virtual computing instances, the one or more first virtual computing instances running in a virtual machine, a data collection tool being deployed in the virtual machine, and the virtual machine running on the server; generating index data of a monitoring index corresponding to each first virtual computing instance through each first virtual computing instance; through the data acquisition tool, acquiring index data corresponding to each first virtual computing instance; and determining a monitoring result according to the index data acquired by the data acquisition tool. By implementing the embodiment of the invention, the convenience of data monitoring can be improved.

Description

Technical Field

[0001] This application relates to the technical field of server data monitoring, and particularly to a data monitoring method and a server. Background Art

[0002] During system operation and maintenance, data monitoring is an important part of ensuring system stability and reliability. Through data monitoring, operation and maintenance personnel can collect, analyze, and monitor various operation indicators of the system in real time, thereby improving operation and maintenance efficiency and the accuracy of decision-making. Currently, data collection for different system services usually relies on third-party data collection tools. For example, the nginx service needs to collect data through nginx_exporter. When the number of monitored system services increases, different data collection tool interfaces need to be called separately during data collection, resulting in a more complex and cumbersome data monitoring process. Summary of the Invention

[0003] Embodiments of this application disclose a data monitoring method and a server, which can improve the convenience of data collection, thereby improving the convenience of data monitoring.

[0004] Embodiments of this application disclose a data monitoring method, including:

[0005] Determine monitoring indicators corresponding to one or more first virtual computing instances respectively, where the one or more first virtual computing instances run in a virtual machine, a data collection tool is deployed in the virtual machine, and the virtual machine runs on a server;

[0006] Generate indicator data of the monitoring indicators corresponding to each of the first virtual computing instances through each of the first virtual computing instances;

[0007] Collect the indicator data corresponding to each of the first virtual computing instances through the data collection tool;

[0008] Determine a monitoring result according to the indicator data collected by the data collection tool.

[0009] In an embodiment of the present application, one or more first virtual computing instances are running in a virtual machine. A data collection tool is deployed in the virtual machine. It is possible to determine the monitoring metrics corresponding to each of the one or more first virtual computing instances, and through each first virtual computing instance, generate the metric data of the monitoring metrics corresponding to each first virtual computing instance. Then, through the data collection tool, it is possible to collect the metric data corresponding to each first virtual computing instance, and thus determine the monitoring result according to the metric data collected by the data collection tool. Implementing this embodiment does not require setting up multiple data collection tools to separately collect the metric data of the monitoring metrics in each first virtual computing instance. Instead, it is possible to generate the metric data of the monitoring metrics corresponding to each first virtual computing instance through each first virtual computing instance, and then collect the metric data generated by each first virtual computing instance through a single data collection tool in the virtual machine, achieving the effect that a single data collection tool can collect the metric data of multiple monitoring objects in a virtualized environment. Only by invoking a single data collection tool can the data collection process for one or more first virtual computing instances be completed, thereby reducing the complexity of data collection and improving the convenience of data monitoring.

[0010] In one embodiment, the virtual machine includes a script program; the generating, through each of the first virtual computing instances, the metric data of the monitoring metrics corresponding to each of the first virtual computing instances includes:

[0011] Running the script program through the virtual machine to access a target first virtual computing instance, and generating the metric data of the monitoring metrics corresponding to the target first virtual computing instance; the target first virtual computing instance is any one of the first virtual computing instances.

[0012] Implementing this embodiment, the virtual machine runs the script program, which can automatically generate the metric data of the monitoring metrics corresponding to each first virtual computing instance, simplify the generation process of the metric data of the monitoring metrics, and ensure the automation, continuity, and accuracy of data monitoring.

[0013] In one embodiment, the script program is a custom script program for generating the metric data corresponding to custom monitoring metrics.

[0014] Implementing this embodiment, through the expansion and coordination of the script program, it is possible to perform data collection and monitoring for specific monitoring metrics in the first virtual computing instance, achieving lightweight data collection.

[0015] In one embodiment, the running the script program through the virtual machine to access a target first virtual computing instance and generating the metric data of the monitoring metrics corresponding to the target first virtual computing instance includes:

[0016] Run the script program according to the target period through the virtual machine to access the target first virtual computing instance, and generate metric data of the monitoring metrics corresponding to the target first virtual computing instance.

[0017] Implementing this embodiment to generate metric data based on the target period can achieve automated data collection and real-time data monitoring.

[0018] In one embodiment, after generating the metric data of the monitoring metrics corresponding to each of the first virtual computing instances through each of the first virtual computing instances, the method further includes:

[0019] Convert the metric data corresponding to each of the first virtual computing instances according to the target data format corresponding to the data collection tool, and save the converted metric data in the monitoring metric file corresponding to the data collection tool;

[0020] The step of collecting the metric data corresponding to each of the first virtual computing instances through the data collection tool includes:

[0021] Collect the metric data corresponding to each of the first virtual computing instances from the monitoring metric file through the data collection tool.

[0022] Implementing this embodiment to convert the metric data can ensure that all metric data meet the requirements of the data collection tool, avoid compatibility issues during the data processing process, and ensure that the metric data of different monitoring metrics can be processed and stored in a consistent manner.

[0023] In one embodiment, determining the monitoring result according to the metric data collected by the data collection tool includes:

[0024] Call the monitoring metric interface corresponding to the data collection tool according to the target period to obtain the metric data collected by the data collection tool;

[0025] Determine the monitoring result according to the obtained metric data.

[0026] Implementing this embodiment to obtain the metric data collected by the data collection tool according to the target period can enable the steps of metric data generation, collection, and monitoring to be coordinated, avoiding data omission.

[0027] In one embodiment, determining the monitoring result according to the obtained metric data includes:

[0028] Send the obtained metric data to the data analysis tool;

[0029] Analyze the obtained metric data through the data analysis tool to obtain the monitoring result;

[0030] The method further includes:

[0031] By means of the data analysis tool, in case that the monitoring result characterizes an abnormal state, an alarm message is output.

[0032] Implementing this embodiment and using a dedicated data analysis tool to determine the monitoring result can improve the accuracy of the monitoring result, and the data analysis tool can also output an alarm message to prompt the operation and maintenance personnel, improving the functionality of data monitoring.

[0033] In one embodiment, determining the monitoring result according to the metric data collected by the data collection tool includes:

[0034] According to the target label, obtain the metric value corresponding to the target label from the metric data respectively collected by the data collection tools deployed in one or more virtual machines; the metric data includes a metric value and a label;

[0035] Determine the monitoring result according to the metric value corresponding to the target label.

[0036] Implementing this embodiment and obtaining the corresponding metric value according to the target label can ensure the accuracy of the acquired data, thereby improving the accuracy of the monitoring result.

[0037] In one embodiment, the multiple virtual machines form a cluster, the label includes a cluster label or a service label, there are multiple metric values corresponding to one cluster label, and there is one metric value corresponding to one service label;

[0038] The determining the monitoring result according to the metric value corresponding to the target label includes:

[0039] If the target label is a cluster label, determine a first monitoring result according to the multiple metric values corresponding to one cluster label, and the first monitoring result is used to characterize the cluster state of the multiple virtual machines in one cluster corresponding to the multiple metric values;

[0040] If the target label is a service label, determine a second monitoring result according to the one metric value corresponding to one service label, and the second monitoring result is used to characterize the service state of the first virtual computing instance of one virtual machine corresponding to the one metric value.

[0041] Implementing this embodiment, by classifying the cluster label and the service label to monitor the cluster state and the service state respectively, the comprehensiveness of data monitoring and the accuracy of data acquisition are realized.

[0042] An embodiment of the present application discloses a data monitoring device, including:

[0043] An index determination module determines monitoring indices respectively corresponding to one or more first virtual computing instances, where the one or more first virtual computing instances run in a virtual machine, a data collection tool is deployed in the virtual machine, and the virtual machine runs on a server;

[0044] A data generation module generates index data of the monitoring indices respectively corresponding to the first virtual computing instances through each of the first virtual computing instances;

[0045] A data collection module is configured to collect the index data respectively corresponding to the first virtual computing instances through the data collection tool;

[0046] A data monitoring module is configured to determine a monitoring result according to the index data collected by the data collection tool.

[0047] An embodiment of the present application discloses a server, including:

[0048] A memory storing executable program code;

[0049] A processor coupled to the memory;

[0050] The processor calls the executable program code stored in the memory and executes the method according to any of the above embodiments.

[0051] An embodiment of the present application discloses a computer-readable storage medium storing a computer program, where when the computer program is executed by a processor, the processor is caused to execute the method according to any of the above embodiments. Description of the Drawings

[0052] To more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for the embodiments. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0053] Figure 1A is a schematic diagram of an application scenario of a data monitoring method disclosed in an embodiment of the present application;

[0054] Figure 1B is a schematic diagram of another application scenario of a data monitoring method disclosed in an embodiment of the present application;

[0055] Figure 1C is a schematic diagram of still another application scenario of a data monitoring method disclosed in an embodiment of the present application;

[0056] Figure 2 It is a schematic flowchart of a data monitoring method disclosed in an embodiment of the present application;

[0057] Figure 3 It is a schematic flowchart of another data monitoring method disclosed in an embodiment of the present application;

[0058] Figure 4 It is a schematic flowchart of yet another data monitoring method disclosed in an embodiment of the present application;

[0059] Figure 5 It is an architecture diagram of a data monitoring system disclosed in an embodiment of the present application;

[0060] Figure 6 It is a schematic flowchart of a method for determining a monitoring result disclosed in an embodiment of the present application;

[0061] Figure 7 It is a modular schematic diagram of a data monitoring device disclosed in an embodiment of the present application;

[0062] Figure 8 It is a structural block diagram of a server disclosed in an embodiment of the present application. Detailed implementation manners

[0063] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

[0064] It should be noted that the terms "including" and "having" in the embodiments of the present application and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or device that includes a series of steps or units does not necessarily limit to those clearly listed steps or units, but may include other steps or units that are not clearly listed or are inherent to these processes, methods, products, or devices.

[0065] It can be understood that the terms "first", "second", etc. used in the present application can be used to describe various elements in this document, but these elements are not limited by these terms. These terms are only used to distinguish the first element from another element. For example, without departing from the scope of the present application, the first virtual computing instance can be called a virtual machine, and similarly, the virtual machine can be called the first virtual computing instance. Both the first virtual computing instance and the virtual machine are virtual computing instances, but they are not the same virtual computing instance.

[0066] Embodiments of the present application disclose a data monitoring method and a server, which can improve the convenience of data collection, thereby improving the convenience of data monitoring.

[0067] The following will be described in detail with reference to the accompanying drawings.

[0068] As Figure 1A shown, Figure 1A FIG. 1 is a schematic diagram of an application scenario of a data monitoring method disclosed in an embodiment of the present application. The application scenario may include a server 100, and the server 100 may include, but is not limited to, a server, a PC (Personal Computer), a laptop, a tablet computer, etc.

[0069] A virtual machine 120 runs in the server 100, and one or more first virtual computing instances 110 run in the virtual machine 120. It should be understood that Figure 1A the two first virtual computing instances in FIG. 1 are only for illustration. Among them, the first virtual computing instance refers to a virtualized computing unit running on a physical server or a virtualized environment, and the virtual computing instance may include containers, applications, etc. Specifically, the virtual machine 120 is a virtual machine running in an entity device (server 110), and the first virtual computing instance 110 is a container or an application running in a virtualized environment (virtual machine 120).

[0070] In a more specific application scenario, as Figure 1B shown, Figure 1B FIG. 2 is a schematic diagram of another application scenario of a data monitoring method disclosed in an embodiment of the present application. Among them, the server 100 may be deployed with a k8s (Kubernetes) platform, the virtual machine 120 may be a k8s node, and the first virtual computing instance 110 may be an application instance running on the virtual machine 120 through service containerization technology, that is, k8s Pod, nginx, etc. in FIG. 1. The virtual machine 120 also includes a data collection tool, and through the data collection tool, the metric data corresponding to each first virtual computing instance 110 can be collected, that is, the metric data corresponding to k8s Pod, nginx, etc. can be collected.

[0071] Among them, containerization refers to the technology of packaging an application program or service into a container so that it can run in any environment that supports containers and is not affected by differences in the underlying operating system or platform. And the k8s Pod refers to a deployment unit when the virtual machine 120 is a k8s node. Among them, after setting up the k8s platform in the virtual machine 120, the virtual machine 120 can be used as a k8s node, and then one or more containers can be generated based on the k8s platform. The one or more containers can be used as a k8s Pod, and all containers in a k8s Pod share the network, storage space, and are jointly managed.

[0072] It should be understood that k8s nodes usually do not exist alone, but a k8s cluster is deployed. The k8s cluster can include multiple k8s nodes. For the sake of simplicity in this application, only a virtual machine 120 is taken as an example, that is Figure 1B the virtual machine 120 shown in Figure 1C is only one of the k8s nodes, as Figure 1C shown.

[0073] Nginx is an HTTP and reverse proxy web server, used for reverse proxy, load balancer, and HTTP caching. Nginx can run in the container of the virtual machine 120 to provide reverse proxy services.

[0074] One or more first virtual computing instances 110 can run in the virtual machine 120. Since the first virtual computing instances 110 are usually isolated from each other, in order to monitor the monitoring metrics corresponding to each first virtual computing instance 110, it is necessary to collect the data of each first virtual computing instance 110 by corresponding data collection tools respectively. For example, the data collection tool for k8s pod is cAdvisor, and the data collection tool for Nginx is nginx_exporter. The server 100 needs to call the interfaces of multiple data collection tools to collect data, making the data collection process complex and inconvenient.

[0075] Therefore, in one embodiment, the server 100 can determine the monitoring metrics corresponding to one or more first virtual computing instances 110 respectively, generate the metric data of the monitoring metrics corresponding to each first virtual computing instance 110 through each first virtual computing instance 110, and then collect the metric data corresponding to each first virtual computing instance 110 through the data collection tool deployed in the virtual machine 120, so as to determine the monitoring result according to the metric data collected by the data collection tool.

[0076] As Figure 2 shown, Figure 2 is a flowchart of a data monitoring method disclosed in an embodiment of the present application. This data monitoring method can be applied to the server in the above embodiment. This data monitoring method can include the following steps:

[0077] Step 210: Determine the monitoring metrics corresponding to one or more first virtual computing instances. The one or more first virtual computing instances run in a virtual machine, in which a data collection tool is deployed, and the virtual machine runs on a server.

[0078] Among them, the monitoring metrics corresponding to a first virtual computing instance can refer to the metrics that need to be monitored for this first virtual computing instance, such as CPU occupancy rate, network occupancy rate, etc. Optionally, a first virtual computing instance can correspond to one monitoring metric or multiple monitoring metrics, and there is no restriction on this. For example, the monitoring metrics that a first virtual computing instance can correspond to can include CPU occupancy rate and / or network occupancy rate.

[0079] Optionally, the monitoring metrics can be preset in the program. The virtual machine can obtain the monitoring metrics corresponding to each first virtual computing instance based on the programs corresponding to the respective first virtual computing instances. Moreover, the data collection tool in the virtual machine can also obtain the monitoring metrics corresponding to each first virtual computing instance based on the program of the data collection tool.

[0080] Step 220: Generate the metric data of the monitoring metrics corresponding to each first virtual computing instance through each first virtual computing instance.

[0081] Step 230: Collect the metric data corresponding to each first virtual computing instance through the data collection tool.

[0082] Compared with calling the interfaces of multiple data collection tools to collect data, in this application, the server only deploys a data collection tool in the virtual machine, that is, the metric data corresponding to each first virtual computing instance is collected through the data collection tool in the virtual machine. Specifically, the server can generate the metric data of the monitoring metrics corresponding to each first virtual computing instance through each first virtual computing instance, and then collect the generated metric data of the monitoring metrics corresponding to each first virtual computing instance through the data collection tool in the virtual machine.

[0083] It should be understood that this data collection tool is the data collection tool of the virtual machine and also has the data collection ability for the virtual machine. That is, when the virtual machine also corresponds to monitoring metrics, the metric data corresponding to the virtual machine can also be collected through the data collection tool, such as the node status when the virtual machine is a k8s node. Thus, in the k8s cluster, each k8s node, that is, each virtual machine, only needs to deploy one data collection tool, achieving the unity of data collection in the entire k8s cluster and improving the data collection efficiency.

[0084] In one embodiment, the server may implement step 220 and step 230 based on a script program in a virtual machine. Deploy the script program in the virtual machine, and the script program can generate metric data of monitoring metrics corresponding to each first virtual computing instance by calling system commands corresponding to each first virtual computing instance, and store the metric data in a monitoring file. The data collection tool of the virtual machine can access the monitoring file generated in the first virtual computing instance to obtain the metric data of the monitoring metrics corresponding to the first virtual computing instance.

[0085] As an example, if the virtual machine is a Linux system, the cron task in the Linux system can be deployed as the script program, and the script program can run regularly and call system commands corresponding to each first virtual computing instance to obtain the metric data of the monitoring metrics corresponding to each first virtual computing instance. Among them, the system commands of the Linux system can include top, free, iostat, netstat, etc.

[0086] In one embodiment, the server may implement step 220 and step 230 based on a log file. The first virtual computing instance may run an application program (such as a Web server, a database, etc.), and the metric data of the monitoring metrics corresponding to the first virtual computing instance may refer to the running state of the application program, and the application program may save the running state to the log file regularly. The running state may include, but is not limited to, the number of processed requests, response time, latency, etc. The data collection tool in the virtual machine can be configured to regularly read the log file or obtain the metric data of the monitoring metrics corresponding to the first virtual computing instance by calling the API (Application Programming Interface) of the application program running on the first virtual computing instance.

[0087] Step 240, determine the monitoring result according to the metric data collected by the data collection tool.

[0088] The server can obtain the metric data collected by the data collection tool and can determine the monitoring result according to the metric data collected by the data collection tool. Among them, the monitoring result can be used to characterize whether each monitoring metric is in an abnormal state.

[0089] Optionally, the metric data collected by the data collection tool can be analyzed, for example, compared with a metric threshold. When the metric data exceeds or is lower than the metric threshold, corresponding alarm signals can be generated to prompt the operation and maintenance personnel to take corresponding maintenance measures.

[0090] Optionally, trend analysis can also be performed based on the metric data within a preset time period to determine the change trend of the monitored metric, so as to judge whether the monitored metric is in an abnormal state in the current time period and / or future time period. Among them, the methods of trend analysis can include, but are not limited to, time series analysis algorithms, etc. Through this implementation method, not only can the monitored metric be monitored in real time, but also predictions can be made based on the change trend of the monitored metric, thereby improving the comprehensiveness of data monitoring.

[0091] In the embodiment of the present application, one or more first virtual computing instances are running in a virtual machine. A data collection tool is deployed in the virtual machine. The monitored metrics corresponding to the one or more first virtual computing instances can be determined, and through each first virtual computing instance, the metric data of the monitored metrics corresponding to each first virtual computing instance is generated. Then, through the data collection tool, the metric data corresponding to each first virtual computing instance can be collected, and thus, based on the metric data collected by the data collection tool, the monitoring result can be determined. Implementing this embodiment does not require setting multiple data collection tools to separately collect the metric data of the monitored metrics in each first virtual computing instance. Instead, the metric data of the monitored metrics corresponding to each first virtual computing instance can be generated by each first virtual computing instance respectively, and then the metric data generated by each first virtual computing instance can be collected through a data collection tool in the virtual machine, achieving the effect that one data collection tool can collect the metric data of multiple monitoring objects in a virtualized environment. Only one data collection tool needs to be called to complete the data collection process for one or more first virtual computing instances, thereby reducing the complexity of data collection and improving the convenience of data monitoring.

[0092] As Figure 3 shown, Figure 3 is a schematic flowchart of another data monitoring method disclosed in the embodiment of the present application. This data monitoring method can be applied to the server in the above embodiment. This data monitoring method can include the following steps:

[0093] Step 310, determine the monitored metrics corresponding to one or more first virtual computing instances respectively. One or more first virtual computing instances are running in a virtual machine. A data collection tool is deployed in the virtual machine, and the virtual machine is running on the server.

[0094] Step 320, run a script program through the virtual machine to access the target first virtual computing instance and generate the metric data of the monitored metrics corresponding to the target first virtual computing instance; the target first virtual computing instance is any one of the first virtual computing instances.

[0095] Among them, the script program can be preset in the virtual machine and is used to call the interface corresponding to the target first virtual computing instance, and automatically generate the metric data of the monitoring metrics corresponding to the target first virtual computing instance. The script program can include predefined commands and logics to extract various performance and resource usage information from the target first virtual computing instance. For example, the script program can collect information such as the CPU, memory, and network traffic of the target first virtual computing instance, and organize the collected information into the metric data of the monitoring metrics.

[0096] It should be understood that compared with using multiple data collection tools to collect the metric data of the monitoring metrics corresponding to each first virtual computing instance, deploying the script program is to simplify the collection process of the metric data of the monitoring metrics and ensure the automation, continuity, and accuracy of data monitoring. Since the number of first virtual computing instances may be large and the status of each first virtual computing instance may change at any time, it will be too complex and consume a lot of resources to monitor each first virtual computing instance in real time by the virtual machine calling the interfaces of various data collection tools. By deploying the script program in the virtual machine, the script program can run automatically at regular intervals or based on events, without the need for the virtual machine to continuously run multiple data collection tools.

[0097] Optionally, the script program of the virtual machine can run automatically in the virtual machine, such as at regular intervals or triggered by specific events, to generate the metric data of the monitoring metrics corresponding to the target first virtual computing instance. In one embodiment, the virtual machine runs the script program according to the target period to access the target first virtual computing instance, and generates the metric data of the monitoring metrics corresponding to the target first virtual computing instance. Through this embodiment, the automation of data collection and real-time data monitoring can be achieved.

[0098] Among them, the script program can obtain the instance information of the target first virtual computing instance by calling specific system commands, and determine the metric data of the monitoring metrics corresponding to the target first virtual computing instance according to the instance information. As an example, the script program obtains the container information of the k8s Pod by using the commands of the k8s platform, and generates the metric information of the monitoring metrics according to the container information of the k8s Pod, such as the CPU usage time and memory usage of the k8s Pod.

[0099] In one embodiment, one or more first virtual computing instances running in a virtual machine may mean that one or more first virtual computing instances are managed by the virtual machine and run accordingly. However, the virtual machine may also need to manage other virtual machines. For example, in a Kubernetes (k8s) cluster including multiple k8s nodes, when a k8s platform is deployed in the virtual machine, the virtual machine can be a computing node of k8s, and the virtual machine can be a management node of k8s. The management node also needs to manage other computing nodes. Therefore, when the virtual machine is a management node of k8s, a corresponding script program can also be included in the virtual machine to collect the metric data of the monitoring metrics corresponding to each computing node. For example, the script program in the management node can call the kubectl command to collect the metric data of the node status corresponding to each computing node. Implementing this embodiment can improve the data collection scope and the functional diversity of the script program.

[0100] Step 330: Convert the metric data corresponding to each first virtual computing instance according to the target data format corresponding to the data collection tool, and save the converted metric data in the monitoring metric file corresponding to the data collection tool.

[0101] A data collection tool is deployed in the virtual machine. This data collection tool is used to obtain and aggregate the metric data of the monitoring metrics corresponding to each first virtual computing instance running in the virtual machine. For example, the data collection tool can include node_exporter of Prometheus, cAdvisor, etc. It should be understood that the data collection tool can monitor metrics such as the resource usage and running status of the virtual machine in real time. However, in this application, by expanding this data collection tool, the data collection tool can also collect the metric data of the monitoring metrics corresponding to one or more first virtual computing instances.

[0102] The target data format refers to the standardized format required by the data collection tool so that the data collection tool can process and parse the metric data of the monitoring metrics corresponding to different first virtual computing instances. Taking node_exporter of Prometheus as an example, the target data format is the Prometheus metric format. The metric data in the Prometheus metric format can include the name of the monitoring metric, labels, and the metric value. For example, a metric data can be k8s_cluster_status{cluster_ip="xxx"}0, where k8s_cluster_status is the name of the monitoring metric, cluster_ip="xxx" is used to represent that the label of the monitoring metric is xxx, and 0 is the metric value of the monitoring metric.

[0103] It should be understood that since each first virtual computing instance may use different platforms, etc., the data formats of the generated metric data may be inconsistent, resulting in the inability of the data collection tool to directly process it. Therefore, by converting the format of the metric data to ensure that all metric data meets the requirements of the data collection tool, compatibility issues during the data processing process can be avoided, and the metric data of different monitoring metrics can be processed and stored in a consistent manner.

[0104] The monitoring metric file is used to store the metric data after format conversion. The monitoring metric file can be a text file, a table file, etc., and there is no restriction on this. This monitoring metric file is the data source of the data collection tool, and the data collection tool can scan and read this monitoring metric file to obtain the metric data of the monitoring metrics corresponding to each first virtual computing instance. For ease of understanding, it can be considered that the monitoring metric file is equivalent to a temporary data buffer to centrally store the metric data from different first virtual computing instances for subsequent processing by the data collection tool. Optionally, this monitoring metric file can be located in the file directory under the data collection tool to facilitate data collection by the data collection tool.

[0105] By converting the metric data corresponding to each first virtual computing instance into the target data format required by the data collection tool and saving it in the monitoring metric file corresponding to the data collection tool, it is ensured that the data collection tool can efficiently and accurately obtain and process the monitoring data of each virtual computing instance. This not only improves the compatibility and consistency of the metric data but also enables the stable acquisition of metric data from the monitoring metric file corresponding to the data collection tool, ensuring the stability of the data monitoring process.

[0106] Step 340, through the data collection tool, collect the metric data corresponding to each first virtual computing instance from the monitoring metric file.

[0107] Step 350, determine the monitoring result according to the metric data collected by the data collection tool.

[0108] In one embodiment, when it is detected that data is written to the monitoring metric file, through the data collection tool, the metric data corresponding to each first virtual computing instance can be collected from the monitoring metric file.

[0109] Optionally, the server can determine that data is written to the monitoring metric file when it detects a data write instruction for the monitoring metric file.

[0110] Optionally, the server can obtain the most recent modification time of the monitoring metric file and compare this most recent modification time with the historical modification time. When the most recent modification time is different from the historical modification time, it is determined that data is written to the monitoring metric file.

[0111] To more clearly explain the above steps 310 - 340, as an example, the target first virtual computing instance can be a container corresponding to nginx. The data collection tool deployed on the virtual machine can include node_exporter. The script program corresponding to the target first virtual computing instance can generate metric data for the monitoring metrics corresponding to the target first virtual computing instance. After formatting the metric data, it can be saved in the custom.prom file under node_exporter. Thus, the textfile collector in node_exporter can collect the metric data in the custom.prom file, and then provide the data interface of the metric data to Prometheus. The monitoring metrics can be monitored through Prometheus, or OpenTelemetry can pull the metric data from Prometheus and send it to the data analysis tool to monitor the monitoring metrics.

[0112] This application can generate metric data through a script program and save it in the corresponding variable in the custom.prom file, thus avoiding calling the interface of the data collection tool corresponding to nginx and enabling unified data collection through the data collection tool node_exporter deployed on the virtual machine.

[0113] In one embodiment, the script program is a custom script program for generating metric data corresponding to custom monitoring metrics. The data collection tool deployed on the virtual machine can also include a custom collection module to collect the metric data corresponding to the custom monitoring metrics generated by the script program. Through the expansion and coordination of the script program and the data collection tool, data collection and monitoring can be performed for specific monitoring metrics in the first virtual computing instance, achieving lightweight data collection and improving the efficiency of data monitoring.

[0114] As Figure 4 shown, Figure 4 is a schematic flowchart of another data monitoring method disclosed in the embodiment of this application. This data monitoring method can be applied to the server in the above embodiment. This data monitoring method can include the following steps:

[0115] Step 410, determine the monitoring metrics corresponding to one or more first virtual computing instances respectively. One or more first virtual computing instances run in a virtual machine, and a data collection tool is deployed in the virtual machine, and the virtual machine runs on the server.

[0116] Step 420, through each first virtual computing instance, generate metric data for the monitoring metrics corresponding to each first virtual computing instance according to the target period.

[0117] Step 430: Use a data collection tool to collect the metric data corresponding to each first virtual computing instance according to the target period.

[0118] Step 440: Call the monitoring metric interface corresponding to the data collection tool according to the target period to obtain the metric data collected by the data collection tool.

[0119] It can be understood that, in order to coordinate the steps of generating, collecting, and monitoring metric data, the script program of the virtual machine can generate the metric data of the monitoring metrics corresponding to each first virtual computing instance according to the target period, the data collection tool can collect the metric data corresponding to each first virtual computing instance according to the target period, and the server can call the monitoring metric interface corresponding to the data collection tool according to the target period to obtain the metric data collected by the data collection tool. In each stage of generating, collecting, and monitoring metric data, a variable can be used to store the metric data. If the operation periods of generating, collecting, and monitoring metric data are different, it may cause the variable to be replaced by new metric data before the generated metric data is collected by the data collection tool or obtained by the server.

[0120] Step 450: Determine the monitoring result based on the obtained metric data.

[0121] For the metric data corresponding to each first virtual computing instance collected by the data collection tool, the server can obtain all the metric data to determine the monitoring result based on all the obtained metric data, or the server can also obtain partial metric data to determine the monitoring result based on the obtained partial metric data.

[0122] Optionally, the monitoring result can be used to indicate whether each monitoring metric is in an abnormal state. For example, the monitoring result can include one or more monitoring values, and the one or more monitoring values can correspond to one or more monitoring metrics one by one, and each monitoring value can indicate whether the corresponding monitoring metric is in an abnormal state.

[0123] Optionally, the monitoring result can also be used to indicate whether the comprehensive metric corresponding to one or more monitoring metrics is in an abnormal state. For example, the monitoring result can only include one monitoring value, and this monitoring value is jointly determined according to the metric data corresponding to one or more monitoring metrics respectively.

[0124] Among them, a data analysis tool can be deployed in the virtual machine, and the data analysis tool can be used to analyze the obtained metric data to obtain a monitoring result. In one embodiment, the virtual machine sends the obtained metric data to the data analysis tool, and through the data analysis tool, the obtained metric data is analyzed to obtain a monitoring result, and through the data analysis tool, when it is detected that the monitoring result represents an abnormal state, an alarm message is output. Implementing this embodiment and using a dedicated data analysis tool to determine the monitoring result can improve the accuracy of the monitoring result, and the data analysis tool can also output an alarm message to prompt the operation and maintenance personnel, improving the comprehensiveness of data monitoring.

[0125] The data analysis tool can be used to store and analyze the obtained metric data and output an alarm message. For example, when the metric data exceeds or is lower than a preset metric threshold, the data analysis tool can determine that the monitoring result represents that the corresponding monitoring metric is in an abnormal state.

[0126] Optionally, a data push tool can also be deployed in the virtual machine, which is used to call the monitoring metric interface corresponding to the data collection tool to push the metric data obtained from the monitoring metric interface to the data analysis tool.

[0127] As an example, as Figure 5 shown, Figure 5 is an architecture diagram of a data monitoring system disclosed in an embodiment of the present application. The first virtual computing instance can include a k8s Pod, a container corresponding to nginx, etc. The data push tool can be OpenTelemetry, and the data analysis tool can be SkyWalking. Among them, the script program can generate metric data of the k8s Pod status and metric data of the nginx service status. The data collection tool can collect metric data of the k8s Pod status and metric data of the nginx service status. OpenTelemetry can call the monitoring metric interface corresponding to the data collection tool in the virtual machine to push the metric data of the k8s Pod status and the metric data of the nginx service status to SkyWalking.

[0128] Through the coordinated cooperation of the script program, the data collection tool, the data push tool, and the data analysis tool, the stability and consistency of data monitoring are achieved.

[0129] To more clearly explain the process of data analysis, as Figure 6 shown, Figure 6 is a flowchart of a method for determining a monitoring result disclosed in an embodiment of the present application. The method for determining the monitoring result can include the following steps:

[0130] Step 610: Obtain the metric value corresponding to the target label from the metric data collected by the data collection tools deployed in one or more virtual machines respectively; the metric data includes metric values and labels.

[0131] Optionally, the server may include multiple analysis modules, each analysis module is respectively used to determine different monitoring results. For example, the first analysis module is used to determine whether the CPU occupancy rate is abnormal, and the second analysis module is used to determine whether the node status is abnormal. Each analysis module may correspond to a label of a type of metric data. The server can provide the metric value corresponding to the label to the analysis module according to the label corresponding to the analysis module to determine the monitoring result.

[0132] Among them, the target label refers to the label corresponding to the target analysis module, that is, the target label can refer to the label corresponding to the current analysis process. The server can determine the target metric data containing the target label from the metric data collected by the data collection tools deployed in one or more virtual machines respectively, and obtain the metric value contained in the target metric data to obtain the metric value corresponding to the target label.

[0133] Step 620: Determine the monitoring result according to the metric value corresponding to the target label.

[0134] The target analysis module can obtain the monitoring result corresponding to the target analysis module according to the obtained metric value corresponding to the target label.

[0135] Optionally, each label may also correspond to a configuration file. The virtual machine can obtain the configuration file corresponding to the target label to execute the program corresponding to the configuration file, so as to determine the monitoring result according to the metric value corresponding to the target label. Among them, the configuration file may include but is not limited to the rule name of the monitoring rule, the name of the metric data, the metric threshold, and the operator corresponding to the metric threshold (greater than, less than, etc.).

[0136] In the embodiments of the present application, by obtaining the corresponding metric value according to the target label, the accuracy of the obtained data can be ensured, thereby improving the accuracy of the monitoring result.

[0137] It should be understood that multiple virtual machines can form a cluster. Specifically, one or more virtual machines in step 610 can be used as k8s nodes respectively to form a k8s cluster. In one embodiment, the label may include a cluster label or a service label. The metric data corresponding to the cluster label is used to determine the cluster status, and the metric data corresponding to the service label is used to determine the service of the first virtual computing instance in a virtual machine.

[0138] The analysis module corresponding to the cluster label can determine the monitoring results corresponding to the monitoring metrics of one or more virtual machines based on the metric data of the monitoring metrics corresponding to the cluster label. For example, based on the metric data of the node status corresponding to the cluster label, the node status of each k8s node in the k8s cluster is monitored to obtain the status of the k8s cluster, which is used as the monitoring result corresponding to the monitoring metrics of one or more virtual machines. The analysis module corresponding to the service label can determine the monitoring result corresponding to the monitoring metrics of the first virtual machine computing instance in a virtual machine based on the metric data of the monitoring metrics corresponding to the service label. For example, the status of the CPU occupancy rate of the nginx service in a virtual machine.

[0139] Among them, the metric values corresponding to a cluster label include multiple ones, and the multiple metric values can be respectively corresponding to multiple virtual machines. For example, the multiple node statuses respectively corresponding to multiple virtual machines when the multiple virtual machines are k8s nodes. The metric value corresponding to the service label includes one, and one metric value can correspond to one virtual machine. For example, the CPU occupancy rate of the nginx service or the CPU occupancy rate of the k8sPod in a virtual machine, etc.

[0140] If the target label is a cluster label, then based on the multiple metric values corresponding to a cluster label, a first monitoring result is determined. The first monitoring result is used to characterize the cluster status of multiple virtual machines in a cluster corresponding to the multiple metric values, where the multiple virtual machines have the same first virtual computing instance. The cluster status is used to characterize the statistical characteristics of the monitoring metrics corresponding to the same first virtual machine computing instance of the multiple virtual machines. For example, if multiple virtual machines all have containers corresponding to the nginx service, the cluster status can characterize whether the CPU occupancy rates of the nginx service in the multiple virtual machines are all normal. It should be understood that in the scenario of a k8s cluster, the k8s cluster includes management nodes and computing nodes. Usually, the management node determines the cluster status of the k8s cluster. That is, in the case where the virtual machine is a management node, the virtual machine can obtain the metric value with the target label being a cluster label and determine the first monitoring result.

[0141] If the target label is a service label, then based on the one metric value corresponding to a service label, a second monitoring result is determined. The second monitoring result is used to characterize the service status of the first virtual machine computing instance in a virtual machine corresponding to the metric value. The service status can be used to characterize whether the monitoring metrics corresponding to the first virtual machine computing instance are normal. For example, when the first virtual machine computing instance is a container corresponding to the nginx service, the service status can characterize whether the CPU occupancy rate of the nginx service is normal.

[0142] The virtual machine can calculate the sum of multiple metric values corresponding to a cluster label and determine a first monitoring result based on the sum value. Optionally, the cluster label can correspond to a first metric threshold, and the service label can correspond to a second metric threshold. If the target label is a cluster label, the sum value can be compared with the first metric threshold to obtain a first comparison result, and then the first monitoring result can be determined based on the first comparison result. If the target label is a service label, a metric value corresponding to the service label can be compared with the first metric threshold to obtain a second comparison result, and then the second monitoring result can be determined based on the second comparison result.

[0143] Implementing this embodiment monitors the cluster status and service status separately through the classification of cluster labels and service labels, achieving the comprehensiveness of data monitoring and the accuracy of data acquisition.

[0144] As Figure 7 shown, Figure 7 is a modular schematic diagram of a data monitoring device disclosed in an embodiment of the present application. The data monitoring device 700 may include a metric determination module 710, a data generation module 720, a data collection module 730, and a data monitoring module 740, where:

[0145] The metric determination module 710 is configured to determine monitoring metrics corresponding to one or more first virtual computing instances respectively. The one or more first virtual computing instances run in a virtual machine, a data collection tool is deployed in the virtual machine, and the virtual machine runs on a server.

[0146] The data generation module 720 is configured to generate metric data of monitoring metrics corresponding to each of the first virtual computing instances through each of the first virtual computing instances.

[0147] The data collection module 730 is configured to collect the metric data corresponding to each of the first virtual computing instances through the data collection tool.

[0148] The data monitoring module 740 is configured to determine a monitoring result based on the metric data collected by the data collection tool.

[0149] In one embodiment, the virtual machine includes a script program; the data generation module 720 is further configured to run the script program through the virtual machine to access a target first virtual computing instance and generate metric data of monitoring metrics corresponding to the target first virtual computing instance. The target first virtual computing instance is any one of the first virtual computing instances.

[0150] In one embodiment, the script program is a custom script program for generating metric data corresponding to custom monitoring metrics.

[0151] In one embodiment, the data generation module 720 is further configured to run the script program according to a target period through a virtual machine to access the target first virtual computing instance, and generate metric data of monitoring metrics corresponding to the target first virtual computing instance.

[0152] In one embodiment, the data generation module 720 is further configured to convert the metric data corresponding to each of the first virtual computing instances according to the target data format corresponding to the data collection tool, and save the converted metric data in the monitoring metric file corresponding to the data collection tool; the data collection module 730 is further configured to collect, through the data collection tool, the metric data corresponding to each of the first virtual computing instances from the monitoring metric file.

[0153] In one embodiment, the data monitoring module 740 is further configured to call the monitoring metric interface corresponding to the data collection tool according to a target period to obtain the metric data collected by the data collection tool; and determine a monitoring result according to the obtained metric data.

[0154] In one embodiment, the data monitoring module 740 is further configured to send the obtained metric data to a data analysis tool; analyze the obtained metric data through the data analysis tool to obtain a monitoring result; and output an alarm message through the data analysis tool when it is detected that the monitoring result represents an abnormal state.

[0155] In one embodiment, the data monitoring module 740 is further configured to obtain, according to a target label, a metric value corresponding to the target label from the metric data respectively collected by the data collection tools deployed in one or more virtual machines; the metric data includes a metric value and a label; and determine a monitoring result according to the metric value corresponding to the target label.

[0156] In one embodiment, the multiple virtual machines form a cluster, the label includes a cluster label or a service label, there are multiple metric values corresponding to one cluster label, and there is one metric value corresponding to one service label; the data monitoring module 740 is further configured to, if the target label is a cluster label, determine a first monitoring result according to the multiple metric values corresponding to one cluster label, where the first monitoring result is used to represent the cluster state of the multiple virtual machines in one cluster corresponding to the multiple metric values; if the target label is a service label, determine a second monitoring result according to the one metric value corresponding to one service label, where the second monitoring result is used to represent the service state of the first virtual computing instance of one virtual machine corresponding to the one metric value.

[0157] In an embodiment of the present application, one or more first virtual computing instances are running in a virtual machine. A data collection tool is deployed in the virtual machine. It is possible to determine monitoring metrics corresponding to each of the one or more first virtual computing instances, and through each first virtual computing instance, generate metric data of the monitoring metrics corresponding to each first virtual computing instance. Then, through the data collection tool, it is possible to collect the metric data corresponding to each first virtual computing instance, and thus determine a monitoring result based on the metric data collected by the data collection tool. Implementing this embodiment does not require setting up multiple data collection tools to separately collect the metric data of the monitoring metrics in each first virtual computing instance. Instead, each first virtual computing instance can generate the metric data of the corresponding monitoring metrics by itself, and then a data collection tool of the virtual machine is used to collect the metric data generated by each first virtual computing instance, achieving the effect that one data collection tool can collect the metric data of multiple monitoring objects in a virtualized environment. Only by calling one data collection tool can the data collection process for one or more first virtual computing instances be completed, thereby reducing the complexity of data collection and improving the convenience of data monitoring.

[0158] As Figure 8 shown, in one embodiment, a server is provided. The server may include:

[0159] A memory 810 storing executable program code;

[0160] A processor 820 coupled to the memory 810;

[0161] The processor 820 calls the executable program code stored in the memory 810, and can implement the data monitoring method provided in the above embodiments.

[0162] The memory 810 may include a random access memory (RAM), and may also include a read-only memory (ROM). The memory 810 can be used to store instructions, programs, codes, code sets or instruction sets. The memory 810 may include a program storage area and a data storage area. Among them, the program storage area may store instructions for implementing an operating system, instructions for implementing at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the above method embodiments, etc. The data storage area may also store data created during the use of the server.

[0163] The processor 820 may include one or more processing cores. The processor 820 connects various parts within the entire server using various interfaces and circuits. By running or executing instructions, programs, code sets, or instruction sets stored in the memory 810, and by invoking the data stored in the memory 810, it performs various functions of the server and processes data. Optionally, the processor 820 may be implemented in at least one hardware form of digital signal processing (DSP), field-programmable gate array (FPGA), or programmable logic array (PLA). The processor 820 may integrate a combination of one or several of a central processing unit (CPU), a graphics processing unit (GPU), and a modem, etc. Among them, the CPU mainly processes the operating system, user interface, application programs, etc.; the GPU is responsible for rendering and drawing display content; the modem is used to process wireless communications. It can be understood that the above-mentioned modem may not be integrated into the processor 820 and may be implemented separately through a communication chip.

[0164] It can be understood that the server may include more or fewer structural elements than those shown in the above structural block diagram. For example, it may include a power module, physical buttons, a WiFi (Wireless Fidelity) module, a speaker, a Bluetooth module, sensors, etc., and no further limitation will be made here.

[0165] An embodiment of the present application discloses a computer-readable storage medium that stores a computer program, where the computer program causes a computer to execute the methods described in the above various embodiments.

[0166] In addition, an embodiment of the present application further discloses a computer program product. When the computer program product runs on a computer, it enables the computer to execute all or part of the steps in any one of the data monitoring methods described in the above embodiments.

[0167] Those of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing relevant hardware through a program. This program can be stored in a computer-readable storage medium, which includes read-only memory (ROM), random access memory (RAM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), one-time programmable read-only memory (OTPROM), electrically-erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc memories, magnetic disc memories, tape memories, or any other computer-readable medium capable of carrying or storing data.

[0168] The above has introduced in detail a data monitoring method and a server disclosed in the embodiments of the present application. Specific examples are used herein to elaborate on the principle and implementation manner of the present application. The description of the above embodiments is only used to help understand the method and its core idea of the present application; at the same time, for those of ordinary skill in the art, according to the idea of the present application, there will be changes in the specific implementation manner and application scope. In summary, the content of this specification should not be construed as a limitation to the present application.

Claims

1. A data monitoring method, characterized in that, Including: Determine monitoring metrics corresponding to one or more first virtual computing instances respectively, where the one or more first virtual computing instances run in a virtual machine, a data collection tool is deployed in the virtual machine, and the virtual machine runs on a server; Generate metric data of the monitoring metrics corresponding to each of the first virtual computing instances through each of the first virtual computing instances; Collect the metric data corresponding to each of the first virtual computing instances through the data collection tool; Determine a monitoring result according to the metric data collected by the data collection tool.

2. The method according to claim 1, wherein The virtual machine includes a script program; the generating, through each of the first virtual computing instances, metric data of the monitoring metrics corresponding to each of the first virtual computing instances includes: Run the script program through the virtual machine to access a target first virtual computing instance, and generate metric data of the monitoring metrics corresponding to the target first virtual computing instance; the target first virtual computing instance is any one of the first virtual computing instances.

3. The method according to claim 2, wherein The script program is a customized script program for generating metric data corresponding to customized monitoring metrics.

4. The method according to claim 2, characterized in that The running, through the virtual machine, the script program to access a target first virtual computing instance and generate metric data of the monitoring metrics corresponding to the target first virtual computing instance includes: Run the script program through the virtual machine according to a target period to access the target first virtual computing instance, and generate metric data of the monitoring metrics corresponding to the target first virtual computing instance.

5. The method according to any one of claims 1 to 4, characterized in that After the generating, through each of the first virtual computing instances, metric data of the monitoring metrics corresponding to each of the first virtual computing instances, the method further includes: Convert the format of the metric data corresponding to each of the first virtual computing instances according to the target data format corresponding to the data collection tool, and save the converted metric data in a monitoring metric file corresponding to the data collection tool; The collecting, through the data collection tool, the metric data corresponding to each of the first virtual computing instances includes: Collect the metric data corresponding to each of the first virtual computing instances from the monitoring metric file through the data collection tool.

6. The method according to claim 1, characterized in that, The determining a monitoring result according to the metric data collected by the data collection tool includes: Call a monitoring metric interface corresponding to the data collection tool according to a target period to obtain the metric data collected by the data collection tool; Determine a monitoring result according to the obtained metric data.

7. The method according to claim 6, characterized in that, The determining a monitoring result according to the obtained metric data includes: Send the obtained metric data to a data analysis tool; Analyze the obtained metric data through the data analysis tool to obtain a monitoring result; The method further includes: Output an alarm message through the data analysis tool when it detects that the monitoring result represents an abnormal state.

8. The method according to claim 1, wherein The determining a monitoring result according to the metric data collected by the data collection tool includes: Obtain a metric value corresponding to the target label from the metric data respectively collected by data collection tools deployed in one or more virtual machines according to the target label; the metric data includes a metric value and a label. Determine a monitoring result according to an index value corresponding to the target tag.

9. The method according to claim 8, wherein The multiple virtual machines form a cluster, the tags include cluster tags or service tags, there are multiple index values corresponding to one cluster tag, and there is one index value corresponding to one service tag; The determining the monitoring result according to the index value corresponding to the target tag includes: If the target tag is a cluster tag, determine a first monitoring result according to multiple index values corresponding to one cluster tag, and the first monitoring result is used to characterize the cluster state of the multiple virtual machines in one cluster corresponding to the multiple index values; If the target tag is a service tag, determine a second monitoring result according to one index value corresponding to one service tag, and the second monitoring result is used to characterize the service state of the first virtual computing instance of one virtual machine corresponding to the one index value.

10. A server, characterized in that, including: A memory storing executable program code; A processor coupled to the memory; The processor calls the executable program code stored in the memory and executes the method according to any one of claims 1 to 9.

Images