Historical performance monitoring data management methods, devices, equipment and storage media

Abstract

This application relates to a method, apparatus, device, and storage medium for managing historical performance monitoring data, and pertains to the field of server technology. This application involves setting a performance monitoring configuration file to define metrics as monitoring items, summarizing these monitoring items to form a monitoring item list, selectively setting different monitoring item lists for different users to differentiate the monitoring of performance monitoring data from different vendors, and setting up a task script to periodically collect monitoring data. This script periodically traverses the monitoring item list to obtain corresponding performance monitoring data, and uploads the performance monitoring data of the metrics matching the monitoring items in the list to a cloud server for storage. This supports long-term storage of historical performance monitoring data and allows for quick querying of cross-cluster backup performance monitoring data, which is then displayed on the interface.

Description

Technical Field

[0001] This application relates to the field of server technology, and in particular to a method, apparatus, computer equipment, and storage medium for managing historical performance monitoring data. Background Technology

[0002] Performance monitoring involves monitoring and recording the performance metrics of software, hardware, or systems during operation in order to analyze and optimize system performance. By collecting and analyzing performance data, system bottlenecks can be identified, resource allocation can be optimized, and system reliability and stability can be improved.

[0003] However, with the development of digitalization, large data centers require the operation, maintenance, and monitoring of an increasing number of storage nodes, making it increasingly complex to query the performance monitoring data of a specific cluster node in a timely manner. For a single cluster, performance monitoring records hundreds of performance metrics across more than ten types. Data centers contain thousands of storage nodes, and different vendors use varying recording and display methods, further increasing the difficulty of operation and maintenance. Historical performance monitoring data cannot be stored long-term. In large-scale storage clusters, the volume of performance monitoring data is enormous, necessitating reduced data retention time to alleviate database pressure. Regular deletion of historical performance monitoring data is also required to ensure the cluster's normal operation. Therefore, how to preserve the massive amounts of monitoring data accumulated over time and support rapid querying and display of this data has become a pressing technical problem to be solved. Summary of the Invention

[0004] Based on this, a method, apparatus, computer equipment, and storage medium for managing historical performance monitoring data are provided to solve the technical problem of how to preserve massive amounts of monitoring data accumulated over a long period of time and support rapid querying and display of monitoring data.

[0005] On the one hand, a method for managing historical performance monitoring data is provided, the method comprising:

[0006] Configure a performance monitoring configuration file to set metrics as monitoring items, and summarize the monitoring items to form a monitoring item list.

[0007] Set up a task script to collect monitoring data on a regular basis. The task script includes a list of database paths corresponding to the monitoring items, a transfer period, the last time the data was transferred, and the path of the remote server where the monitoring data is stored.

[0008] In response to the startup of the task script, the latest performance monitoring configuration file is obtained from the remote server path where the monitoring data is stored, and the list of monitoring items and the list of database paths corresponding to the metric items are updated.

[0009] Based on the last data transfer time and the transfer period, it is determined whether the transfer time has been reached. If so, the performance monitoring data transfer is triggered. The performance monitoring data of the indicator items collected by the task script is matched with the monitoring item list. The performance monitoring data of the indicator items that match the monitoring items in the monitoring item list is uploaded to the cloud server for storage.

[0010] In response to the display of monitoring data for the target cluster, the system retrieves the performance monitoring data of the collected metrics for the target cluster, generates monitoring charts, and displays them.

[0011] In one embodiment, the step of setting a performance monitoring configuration file to set metrics as monitoring items and summarizing the monitoring items to form a monitoring item list includes:

[0012] Obtain the metrics for performance monitoring of a single storage cluster;

[0013] Based on the monitoring needs of each user, the indicators to be monitored are selected as monitoring items, and the monitoring items are summarized to form a monitoring item list for each user.

[0014] In one embodiment, the step of setting the task script for periodically collecting monitoring data includes:

[0015] A storage folder is set up for each user's monitoring item list, and the remote server path for storing monitoring data is formed based on the address of the storage folder;

[0016] Set the daily start and stop times for the task script.

[0017] In one embodiment, the step of determining whether the transfer time has been reached based on the last transferred data time and the transfer period, and triggering the performance monitoring data transfer if so, and matching the performance monitoring data of the indicator items collected by the task script with the monitoring item list includes:

[0018] Detect whether the current time is later than the sum of the last data transfer time and the transfer period;

[0019] If so, the performance monitoring data will be transferred, and a list of monitoring items for each user will be obtained.

[0020] Obtain all the metrics collected by the task script, and iterate through the monitoring item list of each user to determine whether each of the metrics collected by the task script exists.

[0021] When a monitoring item with the same target metric item exists in the monitoring item list, a transfer directory is generated based on the monitoring level and transfer time of the monitoring item, and the performance monitoring data is exported to the cluster directory.

[0022] When no monitoring item is found in the monitoring item list that is the same as the target indicator item, the monitoring items in the next user's monitoring item list are traversed to determine whether the target indicator item exists.

[0023] When no monitoring item matching the target metric item exists in the monitoring item list of all users, the next metric item collected by the task script is obtained, and the monitoring items are traversed one by one in the monitoring item list of each user to determine whether there is a next metric item.

[0024] Determine whether all metrics collected by the task script have been matched and exported, and the performance monitoring data has been uploaded to the cloud server for storage. If yes, the process ends; otherwise, obtain the next metric collected by the task script and iterate through the monitoring item list of each user to determine whether the next metric exists.

[0025] In one embodiment, the step of uploading the performance monitoring data of the indicator items that match the monitoring items in the monitoring item list to the cloud server for storage includes:

[0026] Obtain the performance monitoring data of the indicator items that match the monitoring items in the monitoring item list for each user, and obtain the remote server path of the storage folder set for the monitoring item list for each user;

[0027] Obtain the performance monitoring level of the monitoring items in the monitoring item list, and store them in the storage folder corresponding to the remote server path according to the performance monitoring level of the monitoring items.

[0028] In one embodiment, the step of retrieving the performance monitoring data of the target cluster's collected metrics, generating monitoring charts, and displaying them in response to the display of monitoring data for the target cluster includes:

[0029] Query the remote server, update the list of monitoring items and the cluster network interconnection protocol of the collected monitoring data, and return the cluster network interconnection protocol of the collected performance monitoring data to the interface.

[0030] Based on the cluster network interconnection protocol selected by the user, create a corresponding monitoring chart. Based on the collected monitoring data, use the first-level directory under the cluster monitoring data as the monitoring object type and the second-level directory as a monitoring instance. Set one monitoring object type to match multiple monitoring items.

[0031] The remote server calculates and returns the time period of the backed-up monitoring data according to the performance monitoring level classification storage directory, and provides the user with the option to select the time period.

[0032] Import the monitoring data corresponding to the time period selected by the user into the database, and query the monitoring indicators in the monitoring data corresponding to the time period selected by the user for the user to query.

[0033] Based on the monitoring metrics queried by the user, the saved monitoring data is used to generate monitoring charts and returned to the interface to display the query results.

[0034] In one embodiment, the historical performance monitoring data management method further includes:

[0035] Export and back up the performance monitoring data corresponding to the monitoring items queried by the user. The storage directory for the exported backup is IP address / monitoring object type / first-level monitoring / second-level monitoring item / time / monitoring item name;

[0036] Upon receiving metadata of the target monitoring item, a metadata monitoring chart is created to obtain the monitoring object type, monitoring item name, and time corresponding to the target monitoring item.

[0037] The user obtains the remote path where the monitoring data of the target monitoring item is stored based on the monitoring object type, monitoring item name and time corresponding to the target monitoring item.

[0038] If the path exists and the sub-path is time-based, then the metadata of the target monitoring item within that time period is retrieved and imported into the database, and displayed to the user; if the sub-path is not time-based, then all sub-paths are returned to the interface, and after the user selects one of the sub-paths, the queried target monitoring item metadata is returned to the interface for display.

[0039] On the other hand, a historical performance monitoring data management device is provided, the device comprising:

[0040] The performance monitoring configuration file module is used to set up performance monitoring configuration files, set metrics as monitoring items, and summarize the monitoring items to form a monitoring item list.

[0041] The task script setting module is used to set up task scripts for periodically collecting monitoring data. The task scripts include a list of database paths corresponding to the monitoring items, a transfer cycle, the last data transfer time, and the path of the remote server where the monitoring data is stored.

[0042] The monitoring information update module is used to respond to the startup of the task script by obtaining the latest performance monitoring configuration file from the remote server path where the monitoring data is stored, and updating the list of monitoring items and the list of database paths corresponding to the indicator items.

[0043] The monitoring data backup module is used to determine whether the transfer time has been reached based on the last data transfer time and the transfer period. If so, it triggers the performance monitoring data transfer, matches the performance monitoring data of the indicator items collected by the task script with the monitoring item list, and uploads the performance monitoring data of the indicator items that match the monitoring items in the monitoring item list to the cloud server for storage.

[0044] The import and display module is used to retrieve the performance monitoring data of the target cluster's collected metrics when displaying monitoring data of the target cluster's monitoring items, generate monitoring charts, and display them.

[0045] In another aspect, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to perform the following steps:

[0046] Configure a performance monitoring configuration file to set metrics as monitoring items, and summarize the monitoring items to form a monitoring item list.

[0047] Set up a task script to collect monitoring data on a regular basis. The task script includes a list of database paths corresponding to the monitoring items, a transfer period, the last time the data was transferred, and the path of the remote server where the monitoring data is stored.

[0048] In response to the startup of the task script, the latest performance monitoring configuration file is obtained from the remote server path where the monitoring data is stored, and the list of monitoring items and the list of database paths corresponding to the metric items are updated.

[0049] Based on the last data transfer time and the transfer period, it is determined whether the transfer time has been reached. If so, the performance monitoring data transfer is triggered. The performance monitoring data of the indicator items collected by the task script is matched with the monitoring item list. The performance monitoring data of the indicator items that match the monitoring items in the monitoring item list is uploaded to the cloud server for storage.

[0050] In response to the display of monitoring data for the target cluster, the system retrieves the performance monitoring data of the collected metrics for the target cluster, generates monitoring charts, and displays them.

[0051] In another aspect, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, performs the following steps:

[0052] Configure a performance monitoring configuration file to set metrics as monitoring items, and summarize the monitoring items to form a monitoring item list.

[0053] Set up a task script to collect monitoring data on a regular basis. The task script includes a list of database paths corresponding to the monitoring items, a transfer period, the last time the data was transferred, and the path of the remote server where the monitoring data is stored.

[0054] In response to the startup of the task script, the latest performance monitoring configuration file is obtained from the remote server path where the monitoring data is stored, and the list of monitoring items and the list of database paths corresponding to the metric items are updated.

[0055] Based on the last data transfer time and the transfer period, it is determined whether the transfer time has been reached. If so, the performance monitoring data transfer is triggered. The performance monitoring data of the indicator items collected by the task script is matched with the monitoring item list. The performance monitoring data of the indicator items that match the monitoring items in the monitoring item list is uploaded to the cloud server for storage.

[0056] In response to the display of monitoring data for the target cluster, the system retrieves the performance monitoring data of the collected metrics for the target cluster, generates monitoring charts, and displays them.

[0057] The aforementioned historical performance monitoring data management method, device, computer equipment, and storage medium, by setting a performance monitoring configuration file to set indicator items as monitoring items, summarizing the monitoring items to form a monitoring item list, selectively setting different monitoring item lists for different users to differentiate the monitoring of performance monitoring data from different vendors, and setting a task script to periodically collect monitoring data to periodically traverse the monitoring item list to obtain the corresponding performance monitoring data, and uploading the indicator item performance monitoring data that matches the monitoring item in the monitoring item list to a cloud server for storage, supports long-term storage of historical performance monitoring data, supports quick querying of cross-cluster backup performance monitoring data and displays it on the interface. Attached Figure Description

[0058] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0059] Figure 1 This is a diagram illustrating the application environment of a historical performance monitoring data management method in one embodiment of this application.

[0060] Figure 2 This is a flowchart illustrating a historical performance monitoring data management method in one embodiment of this application;

[0061] Figure 3This is a flowchart illustrating the steps of setting a performance monitoring configuration file in one embodiment of the present application, which is used to set indicator items as monitoring items and summarize the monitoring items to form a monitoring item list.

[0062] Figure 4 This is a flowchart illustrating the steps of determining whether the transfer time has been reached based on the last transferred data time and the transfer period in one embodiment of this application, and triggering the transfer of performance monitoring data, and matching the performance monitoring data of the indicator items collected by the task script with the monitoring item list.

[0063] Figure 5 This is a flowchart illustrating the steps of collecting performance monitoring data for indicator items using a performance monitoring configuration file and task script in one embodiment of this application.

[0064] Figure 6 This is a flowchart illustrating the steps of retrieving the performance monitoring data of the target cluster's collected metrics and generating and displaying the monitoring chart in response to the display of monitoring data of the target cluster's monitoring items in one embodiment of this application.

[0065] Figure 7 This is a flowchart illustrating the steps of querying historical performance monitoring data, generating monitoring charts, and displaying query results in one embodiment of this application.

[0066] Figure 8 This is a structural block diagram of a historical performance monitoring data management device in one embodiment of this application;

[0067] Figure 9 This is an internal structural diagram of a computer device in one embodiment of this application. Detailed Implementation

[0068] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0069] As described in the background section, performance monitoring involves monitoring and recording the performance metrics of software, hardware, or systems during operation to analyze and optimize system performance. By collecting and analyzing performance data, system bottlenecks can be identified, resource allocation optimized, and system reliability and stability improved. The objects monitored in performance monitoring include clusters, nodes, network interface cards (NICs), and hard drives, indicating the type that needs to be monitored. Monitoring items are the specific items corresponding to the monitored object type, and their monitoring metrics include, for example, cluster bandwidth and IOPS, and NIC bandwidth and packet loss rate.

[0070] However, with the development of digitalization, large data centers require the operation, maintenance, and monitoring of an increasing number of storage nodes, making it increasingly complex to query the performance monitoring data of a specific cluster node in a timely manner. For a single cluster, performance monitoring records hundreds of performance metrics across more than ten types (such as various metrics for cluster, node, disk, network port, file, object, and block services). Data centers contain thousands of storage nodes, and different vendors use varying recording and display methods, further increasing the difficulty of operation and maintenance. How to preserve the massive amounts of monitoring data accumulated over a long period and support rapid querying and display of this data has become a problem worthy of discussion.

[0071] In existing storage management platforms, the system monitors and statistically analyzes the cluster's operational status based on the deployed storage services, storing the monitoring data in a database and supporting user interface display. When querying historical performance data for a monitored item, the management interface sends query conditions. The server then queries the database based on these conditions and returns the data to the interface, which renders the data as charts for user viewing. Existing performance monitoring systems periodically collect monitoring data and record it in the database. In large-scale storage clusters with numerous storage nodes and many monitoring items to support, long-term storage of performance monitoring data puts significant pressure on the cluster database. Therefore, a periodic deletion mechanism is typically implemented to remove historical performance monitoring data. Existing performance monitoring systems support data export, allowing users to save data in text format for long-term storage of historical performance monitoring data. However, historical performance monitoring data cannot be stored indefinitely. In large-scale storage clusters, the volume of performance monitoring data is large, necessitating reduced data retention time to alleviate database pressure and ensure normal cluster operation. Furthermore, in data centers, different vendors' management platforms implement different methods for saving, exporting, and displaying performance monitoring data, resulting in inconsistent data management and inconvenience for operation and maintenance. Cluster-exported data is in text format, making it difficult for operations personnel and users to easily compare cluster operational status when querying long-term stored historical performance statistics. Data centers generate massive amounts of performance monitoring data daily, and existing methods cannot quickly retrieve the required historical performance data.

[0072] To address the aforementioned issues, this invention creatively proposes a historical performance monitoring data management method. This method backs up and imports historical performance monitoring data for display, is compatible with regular backups of performance monitoring data from different management systems, and supports quick querying and importing of historical backup data for display in massive data backups, thus solving the problem of long-term preservation of historical performance monitoring data.

[0073] The historical performance monitoring data management method provided in this application can be applied to, for example... Figure 1In the application environment shown, terminal 102, remote server 103, and cloud server 104 communicate via a network. Terminal 102 can be, but is not limited to, various personal computers, laptops, smartphones, tablets, and portable wearable devices. Remote server 103 and cloud server 104 can be implemented using independent servers or a server cluster composed of multiple servers. Terminal 102 is operated by the user to obtain performance monitoring data of metrics from remote server 103 to understand the performance of remote server 103. Remote server 103 has performance monitoring configuration files and timed data collection task scripts to obtain performance monitoring data corresponding to the user-defined metrics. Remote server 103 stores the corresponding performance monitoring data. Terminal 102 can download and view the performance monitoring data corresponding to the monitored items and transfer historical performance monitoring data to cloud server 104 for easy retrieval and viewing by the user through terminal 102.

[0074] In one embodiment, such as Figure 2 As shown, a method for managing historical performance monitoring data is provided, which can be applied to... Figure 1 Taking remote server 103 as an example, the following steps are included:

[0075] Step S1: Set up a performance monitoring configuration file to set metrics as monitoring items, and summarize the monitoring items to form a monitoring item list;

[0076] Step S2: Set up a task script for periodically collecting monitoring data. The task script includes a list of database paths corresponding to the monitoring items, a transfer period, the last time the data was transferred, and the path of the remote server where the monitoring data is stored.

[0077] Step S3: In response to starting the task script, obtain the latest performance monitoring configuration file from the remote server path where the monitoring data is stored, and update the list of monitoring items and the list of database paths corresponding to the indicator items.

[0078] Step S4: Determine whether the transfer time has been reached based on the last data transfer time and the transfer period. If so, trigger the performance monitoring data transfer, match the performance monitoring data of the indicator items collected by the task script with the monitoring item list, and upload the performance monitoring data of the indicator items that match the monitoring items in the monitoring item list to the cloud server for storage.

[0079] Step S5: In response to the display of monitoring data for the target cluster, retrieve the performance monitoring data of the collected metrics for the target cluster, generate monitoring charts, and display them.

[0080] Specifically, by setting a performance monitoring configuration file, metrics are set as monitoring items. These monitoring items are then aggregated into a monitoring item list. Different monitoring item lists can be selectively set for different users to differentiate the monitoring of performance monitoring data from different vendors. In addition, a task script is set to collect monitoring data on a regular basis, which periodically traverses the monitoring item list to obtain the corresponding performance monitoring data. The performance monitoring data of the metrics that match the monitoring items in the monitoring item list is uploaded to the cloud server for storage. This supports long-term storage of historical performance monitoring data and allows for quick querying of backup performance monitoring data across clusters, which is then displayed on the interface.

[0081] like Figure 3 As shown, in this embodiment, the step of setting a performance monitoring configuration file to set metric items as monitoring items and summarizing the monitoring items to form a monitoring item list includes:

[0082] Step S11: Obtain the performance monitoring metrics for a single storage cluster;

[0083] Step S12: Select the indicator items to be monitored as monitoring items according to the monitoring needs of each user, and form a monitoring item list for each user by summarizing the monitoring items.

[0084] Specifically, to meet the performance monitoring data requirements of different vendors, the storage cluster can select the metrics to be monitored as monitoring items according to the monitoring needs of each user, and then summarize the monitoring items to form a monitoring item list for each user, thus satisfying the performance monitoring data needs of different users. The monitoring item lists with different needs are stored in the performance monitoring configuration file, and the corresponding performance monitoring data can be obtained by periodically traversing the monitoring item list.

[0085] It is understood that the performance monitoring configuration file includes:

[0086] Get the access address for each user and set the username for each user's access address;

[0087] If the selected indicator item exists in the access address corresponding to each user name, then the selected indicator item is used as a monitoring item, and a performance monitoring level is set for the monitoring item.

[0088] The user name is associated with the monitoring item, and the monitoring items are aggregated to form a monitoring item list corresponding to the user name.

[0089] This allows setting usernames for different users' access addresses and a list of monitoring items corresponding to each username partition. This enables the selection of specific metrics from the storage cluster based on each user's monitoring needs, and the aggregation of these metrics to form a monitoring item list for each user. This satisfies the performance monitoring data requirements of different users, and the monitoring item lists for different needs are stored in the performance monitoring configuration file.

[0090] In this embodiment, the step of setting the task script for periodically collecting monitoring data includes:

[0091] A storage folder is set up for each user's monitoring item list, and the remote server path for storing monitoring data is formed based on the address of the storage folder;

[0092] Set the daily start and stop times for the task script.

[0093] The task script can be set to start and stop daily, enabling it to run daily. The task script includes a list of database paths corresponding to the monitored items, a transfer period, the last data transfer time, and the path to the remote server storing the monitoring data. For each user, a storage folder is set up on remote server 103 and / or cloud server 104 to store the corresponding performance monitoring data. Performance monitoring data on remote server 103 can be periodically cleared, while cloud server 104 permanently stores historical performance monitoring data.

[0094] like Figure 4 As shown, in this embodiment, the step of determining whether the transfer time has been reached based on the last transferred data time and the transfer period, and triggering the performance monitoring data transfer if so, and matching the performance monitoring data of the indicator items collected by the task script with the monitoring item list, includes:

[0095] Step S41: Detect whether the current time is later than the sum of the last data transfer time and the transfer period;

[0096] Step S42: If so, trigger the performance monitoring data transfer and obtain the monitoring item list for each user;

[0097] Step S43: Obtain all the indicator items collected by the task script, and iterate through the monitoring item list of each user to determine whether the indicator items collected by the task script exist.

[0098] Step S44: In response to the existence of a monitoring item that is the same as the target indicator item in the monitoring item list, a transfer directory is generated according to the monitoring level and transfer data time of the monitoring item, and the performance monitoring data is exported to the cluster directory.

[0099] Step S45: In response to the absence of a monitoring item identical to the target indicator item in the monitoring item list, the monitoring items in the monitoring item list of the next user are traversed to determine whether the target indicator item exists.

[0100] Step S46: In response to the absence of a monitoring item identical to the target metric item in the monitoring item list of all users, obtain the next metric item collected by the task script, and iterate through the monitoring items in the monitoring item list of each user to determine whether there is a next metric item.

[0101] Step S47: Determine whether all the metrics collected by the task script have been matched and exported, and the performance monitoring data has been uploaded to the cloud server for storage. If yes, the process ends. Otherwise, obtain the next metric collected by the task script and iterate through the monitoring item list of each user to determine whether there is a next metric.

[0102] To ensure the long-term storage of massive amounts of logs generated by the data center and enable rapid retrieval of backup data, historical performance monitoring data is transferred to multiple levels of directories based on performance items, including cluster IP, metric, node, and time. This data is regularly backed up and synchronized to prevent data loss. During data transfer, performance monitoring data from different vendors is compatiblely stored.

[0103] Specifically, such as Figure 5 As shown, a performance monitoring configuration file, perf_save.conf, and a script, perf_collect.py, are required for periodically collecting monitoring data in the storage cluster. The configuration file records the collected monitoring items, forming a monitoring item list, perf_level. Each metric item corresponds to a list of database paths (indi_path), a save period (period_t), a save time (last_t), and a remote server path (ip_path) for storing the monitoring data.

[0104] Here, `i` represents different data types for monitoring performance, and `j` represents the number of database query commands. For example, if `i` represents a cluster, and data from multiple vendors needs to be transferred, with each vendor exporting different data from the database, the database path `indi_path[i]` will be traversed, and each database export command will be executed. `len` is a built-in function in Python used to calculate the number of elements in a sequence or set.

[0105] For data transfer, a performance monitoring script is run daily at set times.

[0106] (1) Pull the latest configuration file from the remote server path ip_path where the monitoring data is stored, and update the monitoring item list of the metric perf_level, the database path list of the corresponding metric item indi_path, and the transfer period period_t.

[0107] (2) When detecting whether the current time current_t is later than the sum of the last update time and the dump period, that is, when current_t < last_t + period_t, trigger the performance monitoring data dump, start the timed monitoring data collection script perf_collect.py, traverse the list of database paths indi_path corresponding to the metric items at the corresponding positions of the monitoring item indicator to check whether the database tables exist. If not, it means that the management system does not record the monitoring item indicator and there is no need to dump, then query the next monitoring item; if it exists, the monitoring item indicator needs to be exported.

[0108] In this embodiment, the step of uploading the performance monitoring data of the metric items matching the monitoring items in the monitoring item list to the cloud server for storage includes:

[0109] Obtain the performance monitoring data of the metric items matching the monitoring items in the monitoring item list for each user, and obtain the remote server path of the storage folder set for the monitoring item list corresponding to each user.

[0110] Obtain the performance monitoring level of the monitoring items in the monitoring item list, and classify and store them in the storage folder corresponding to the remote server path according to the performance monitoring level of the monitoring items.

[0111] For the performance metrics with records in the cluster, query the monitoring item list perf_level. If it is a cluster-level performance metric, the data storage path is: ip / cluster / last_t - current_t / cluster.csv,

[0112] If the query is for the cluster-node level of the perf_level monitoring item, the data storage path is: ip / node / node_ip / last_t - current_t / node_ip.csv, where node_ip is the ip address after de-duplication of the node column recorded in the monitoring item database. For example, in the node.sql table, if the data of two nodes ip1 and ip2 are recorded, and the column name of the node is the node column, it will be recorded at the corresponding position of perf_level. The script will first query and generate the node ip, and then save and export the data: ip / node / ip1 / last_t - current_t / ip1.csv, ip / node / ip2 / last_t - current_t / ip2.csv;

[0113] If other more detailed monitoring data is required, the data will be saved according to the above rules. That is, for data of cluster-level_1-…-level_n, it will be recorded as: ip / level_1 / .. / level_n / last_t-current_t / level_n.csv, where level_1 represents different monitoring types and level_n is the monitoring instance.

[0114] As mentioned above, when transferring data, the performance monitoring data is stored in the remote server's path IP folder, and the data is saved according to different performance monitoring levels.

[0115] After the data transfer is complete, compress the transferred data IP folder and transfer it to the cloud or a dedicated server for storing monitoring data for decompression and storage.

[0116] When a data center connects to a new vendor's management platform, compatibility in saving performance monitoring data from different vendors can be achieved simply by adding the following configuration items to the remote server's `ip_path` field: the `perf_level` list of monitoring items and the `indi_path` list of database paths corresponding to the metric items. Alternatively, the `period_t` period can be updated to adjust the save period.

[0117] After the collected monitoring data is uploaded to the server, the management platform can perform cross-cluster queries by querying the collected monitoring data on the server.

[0118] like Figure 6 As shown, in this embodiment, the step of retrieving the performance monitoring data of the target cluster's collected metrics, generating monitoring charts, and displaying them in response to the display of monitoring data for the target cluster includes:

[0119] Step S51: Query the remote server, update the list of monitoring items and the cluster network interconnection protocol of the collected monitoring data, and return the cluster network interconnection protocol of the collected performance monitoring data to the interface.

[0120] Step S52: Based on the cluster network interconnection protocol selected by the user, create a corresponding monitoring chart. Based on the collected monitoring data, use the first-level directory under the cluster monitoring data as the monitoring object type and the second-level directory as a monitoring instance. Set one monitoring object type to match multiple monitoring items.

[0121] Step S53: The remote server calculates and returns the time period of the backed-up monitoring data according to the performance monitoring level classification storage directory, and provides the user with the selection;

[0122] Step S54: Import the monitoring data corresponding to the time period selected by the user into the database, and query and obtain the monitoring indicators in the monitoring data corresponding to the time period selected by the user for the user to query.

[0123] Step S55: Based on the monitoring metrics queried by the user, generate a monitoring chart from the saved monitoring data and return it to the interface to display the query results.

[0124] In performance monitoring, there are various monitoring items at different granularities, such as cluster and node levels, or monitoring of node network ports, mounted clients, node hard drives, etc. For data of different granularities, this invention provides a query process. Users can create performance monitoring charts on the management platform, and query historical performance monitoring data of any cluster in the data center. The query process is as follows: Figure 7 .

[0125] like Figure 7 As shown, the query process is as follows:

[0126] (1) First, query the remote server, update the monitoring item list perf_level and the cluster IP of the collected monitoring data, and return the cluster IP of the collected performance monitoring data to the interface.

[0127] (2). Users select cluster IP, create corresponding monitoring charts, and based on the collected information, the first-level directory under the IP is used as the monitoring object type, and the second-level directory up to the parent directory of / last_t-current_t is used as a monitoring instance. Multiple monitoring items can be selected for one monitoring object type.

[0128] (3) The remote server calculates and returns the data for the backed-up time period based on the last_t-current_t level directory and provides the user with the option to select.

[0129] (4) Import the data saved for the time period selected by the user into the database, query what monitoring indicators are saved, and provide them to the user for query. Based on the monitoring indicators queried by the user, return the saved data to the interface to complete the query and display process.

[0130] For cluster monitoring object types, for monitoring items, the corresponding monitoring metrics, and other steps are the same.

[0131] In this embodiment, the historical performance monitoring data management method further includes:

[0132] Export and back up the performance monitoring data corresponding to the monitoring items queried by the user. The storage directory for the exported backup is IP address / monitoring object type / first-level monitoring / second-level monitoring item / time / monitoring item name;

[0133] Upon receiving metadata of the target monitoring item, a metadata monitoring chart is created to obtain the monitoring object type, monitoring item name, and time corresponding to the target monitoring item.

[0134] The user obtains the remote path where the monitoring data of the target monitoring item is stored based on the monitoring object type, monitoring item name and time corresponding to the target monitoring item.

[0135] If the path exists and the sub-path is time-based, then the metadata of the target monitoring item within that time period is retrieved and imported into the database, and displayed to the user; if the sub-path is not time-based, then all sub-paths are returned to the interface, and after the user selects one of the sub-paths, the queried target monitoring item metadata is returned to the interface for display.

[0136] In large data centers with thousands of storage clusters, it is difficult to quickly select cluster IPs and monitoring items using conventional search methods. In order to quickly select a monitoring data item, this invention designs a module for importing and displaying data based on metadata.

[0137] When exporting backups from performance monitoring, the perf_level list of monitoring items already records the monitoring object types of each vendor, and the directory rule is ip / monitoring object type / level 1 monitoring / ... / level N monitoring items / time / monitoring item.csv. Based on the above rules, a metadata retrieval and import method is implemented.

[0138] First, the user creates a metadata monitoring chart, where IP address, monitored object type, monitored item, and time are required inputs. After the chart is sent, the IP address, monitored object type, and monitored item are concatenated to obtain the remote path. If the path exists and the sub-path is a time period, the data within the user's data time period is retrieved, imported into the database, and displayed to the user. If the sub-path is not a time period, all sub-paths are returned to the interface. After the user selects one of the sub-paths, the queried data is returned to the interface for display.

[0139] The above solutions enable the long-term storage of massive amounts of logs generated by the data center, compatible storage of performance monitoring data from different vendors, and support for quick querying of backup data. The data is also displayed on the interface and supports the long-term storage of historical performance monitoring data.

[0140] The remote server 103 includes a monitoring data backup, import and display module and a metadata import and display module. An embodiment of the method for backing up and importing historical performance monitoring data implemented by the remote server 103 is shown below.

[0141] I. Monitoring Data Backup Module

[0142] In the monitoring data backup module, it is assumed that there are two storage management clusters from different vendors.

[0143] IP: 100.100.100.1, Nodes: 100.100.100.2, 100.100.100.3, each node has one network interface card (NIC1);

[0144] IP address: 100.100.101.1, Nodes: 100.100.101.2, 100.100.101.3, Each node has two disks: disk1 and disk2.

[0145] The configuration file is shown in the table below:

[0146] ip_path:100.100.100.100: / var / perf_data

[0147] perf_level:["cluster","cluster-node","cluster-node-nic","cluster-node-disk"]

[0148] indi_path:[

[0149] ["sqlite3 data.db"SELECT*FROM cluster","sqlite3 data2.db"SELECT*FROMcluster"],

[0150] ["sqlite3 data.db"SELECT*FROM node","sqlite3 data2.db"SELECT*FROMnode"],

[0151] ["sqlite3 data.db"SELECT*FROM nic"],

[0152] ["sqlite3 data2.db"SELECT*FROM disk"]]

[0153] last_t:2024-01-0100:00:00

[0154] period_t:7.

[0155] The cluster A management platform records data in the data.db database, while the cluster B management platform records data in the data2.db database. Cluster A records monitoring data of the cluster, nodes, and network cards, while B records data of the cluster, nodes, and hard drives.

[0156] The data transfer period is 7 days. The data transfer path is to the / var / perf_data path of the remote server 100.100.100.100. The cluster records the last transfer time last_t = 2024-01-01 00:00:00.

[0157] Clusters A and B run once a day to check if data needs to be transferred. The next transfer time is reached on 2024-01-08 00:00:00, triggering the data transfer.

[0158] For cluster A, for cluster monitoring items, query the data in the cluster table of the data.db database from 2024-01-01 00:00:00 to 2024-01-08 00:00:00, export and save the data.

[0159] The file 100.100.100.1 / cluster / 2024-01-01_2024-01-08 / cluster.csv;

[0160] For node monitoring data, first query the `node` table in the `data.db` database. The `perf_level` record is `cluster-node`. Then, query the `node` column in the `node` table to check for the nodes 100.100.100.2 and 100.100.100.3 in the cluster, and then transfer the data for both nodes.

[0161] 100.100.100.1 / node / 100.100.100.2 / 2024-01-01_2024-01-08 / node.csv

[0162] 100.100.100.1 / node / 100.100.100.3 / 2024-01-01_2024-01-08 / node.csv

[0163] For network interface card (NIC) data, the `node` column of the transfer query contains two nodes, representing the NICs that these nodes represent, and the data is saved as follows:

[0164] 100.100.100.1 / nic / 100.100.100.2 / nic1 / 2024-01-01_2024-01-08 / nic.csv 100.100.100.1 / nic / 100.100.100.3 / nic1 / 2024-01-01_2024-01-08 / nic.csv

[0165] Regarding hard disk data, if cluster A is found to have no data2.db database, it indicates that the A management platform does not collect hard disk metrics, so there is no need to transfer hard disk data.

[0166] Complete the performance data transfer for cluster A, and upload all data from the "100.100.100.1 / " directory to / var / perf_data at 100.100.100.100. Modify the local transfer time to: last_t = 2024-01-08

[0167] 00:00:00, waiting for the next transfer.

[0168] The data transfer process for cluster B is similar to that for cluster A, with the following differences: When querying cluster B, there is no data.db database. When transferring cluster and node data, the second SQL statement in indi_path will be used to query the data2.db database. Network interface monitoring items will not be transferred, but network interface monitoring object types will be transferred.

[0169] II. Import and Display Module

[0170] Historical performance data can be viewed, taking the data of cluster A from January 5, 2024 to January 10, 2024 as an example, with three monitoring object types: cluster, node, and hard disk.

[0171] When a user needs to query the performance monitoring data of cluster A from January 5, 2024 to January 10, 2024.

[0172] (1). Create a monitoring chart for cluster A node in the management platform interface, obtain the structure under the 100.100.100.1 directory, and query the monitoring object types according to the 100.100.100.1 directory structure, which include cluster, node, and nic.

[0173] (2). If you select the cluster monitoring object type,

[0174] 100.100.100.1 / cluster / 2024-01-01_2024-01-08 / cluster.csv

[0175] The data from 100.100.100.1 / cluster / 2024-01-08_2024-01-15 / cluster.csv is exported to the database, and the monitoring metrics recorded in the database, such as IOPS and cluster bandwidth, are returned to the interface for users to query.

[0176] (3) If the node monitoring object type is selected, the two node data, 100.100.100.2 and 100.100.100.3, exist in the "100.100.100.1 / cluster" directory. The user selects which node is needed, that is, selects the indicator item. After selection, the node.csv data is imported for query. The interface returns monitoring indicators, such as IOPS and node bandwidth, for the user to query.

[0177] (4) If the nic monitoring object type is selected, according to the "100.100.100.1 / nic" directory, select the indicator items as 100.100.100.2-nic1, 100.100.100.3-nic. After the user selects, import nic.csv. After selection, import node.csv data for query. The interface returns monitoring indicators, such as IOPS and network interface bandwidth, for the user to query.

[0178] III. Metadata Import and Display Module

[0179] Metadata import demonstration using the creation of cluster A node and cluster B network interface card as an example:

[0180] Cluster A node: Create a metadata retrieval monitoring chart. Input query IP: 100.100.100.1, node: 100.100.100.2, time: 2024-01-01_2024-01-08, monitoring item: node. Concatenate the directory to the remote server and find that the subdirectory 100.100.100.1 / 100.100.100.2 contains the time.

[0181] Import and display the data from the node.csv file in the directory 100.100.100.1 / node / 100.100.100.2 / 2024-01-01_2024-01-08.

[0182] Cluster B network interface: Create a metadata retrieval monitoring chart. Enter the query IP: 100.100.101.1, time: 2024-01-01_2024-01-10, monitoring item: nic. At this point, the user enters no node value for the metadata item. The query shows that the 100.100.101.1 / nic directory had two sibling directories, 100.100.101.2 and 100.100.101.3, before the specified time. After the user selects 100.100.101.2, the query will...

[0183] 100.100.101.1 / nic / 100.100.101.2 / 2024-01-01_2024-01-08 / nic.csv,

[0184] Import the data from 100.100.101.1 / nic / 100.100.101.2 / 2024-01-08_2024-01-15 / nic.csv into the database and return the data for 2024-01-01 to 2024-01-10 to the front-end interface.

[0185] This enables long-term storage of massive amounts of logs generated by the data center, and provides a unified backup query and interface display that is compatible with different vendor platforms and monitors different data.

[0186] This application supports the long-term storage of historical performance monitoring data, implementing a method for long-term storage of performance monitoring data. It is compatible with data storage management platforms from different vendors, storing performance data from various platforms. It implements a method for querying massive performance monitoring data, allowing the management platform to query performance monitoring data from different clusters and supporting queries at different granularities. Finally, it proposes a metadata retrieval method for quickly querying stored performance monitoring data.

[0187] In the above-described historical performance monitoring data management method, a performance monitoring configuration file is set to set metrics as monitoring items. The monitoring items are then aggregated to form a monitoring item list. Different monitoring item lists are selectively set for different users to differentiate the monitoring of performance monitoring data from different vendors. In addition, a task script is set to periodically collect monitoring data to periodically traverse the monitoring items in the monitoring item list to obtain the corresponding performance monitoring data. The performance monitoring data of the metrics that match the monitoring items in the monitoring item list is uploaded to the cloud server for storage. This method supports long-term storage of historical performance monitoring data and allows for quick querying of backup performance monitoring data across clusters, which is then displayed on the interface.

[0188] It should be understood that, although Figures 2-7 The steps in the flowchart are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order in which these steps are executed, and they can be performed in other orders. Figures 2-7 At least some of the steps in the process may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least some of the sub-steps or stages of other steps.

[0189] In one embodiment, such as Figure 8 As shown, a historical performance monitoring data management device 10 is provided, including: a performance monitoring configuration file setting module 1, a task script setting module 2, a monitoring information update module 3, a monitoring data backup module 4, and an import and display module 5.

[0190] The performance monitoring configuration file module 1 is used to set the performance monitoring configuration file, set the indicator items as monitoring items, and summarize the monitoring items to form a monitoring item list.

[0191] The task script setting module 2 is used to set a task script for periodically collecting monitoring data. The task script includes a list of database paths corresponding to the monitoring items, a transfer period, the last time the data was transferred, and the path of the remote server where the monitoring data is stored.

[0192] The monitoring information update module 3 is used to respond to the startup of the task script by obtaining the latest performance monitoring configuration file from the remote server path where the monitoring data is stored, and updating the monitoring item list and the database path list corresponding to the indicator items.

[0193] The monitoring data backup module 4 is used to determine whether the transfer time has been reached based on the last transferred data time and the transfer period. If so, it triggers the performance monitoring data transfer, matches the performance monitoring data of the indicator items collected by the task script with the monitoring item list, and uploads the performance monitoring data of the indicator items that match the monitoring items in the monitoring item list to the cloud server for storage.

[0194] The import and display module 5 is used to retrieve the performance monitoring data of the target cluster's collected indicators, generate monitoring charts, and display them in response to the display of monitoring data of the target cluster's monitoring items.

[0195] In this embodiment, the step of setting a performance monitoring configuration file to set metrics as monitoring items and summarizing the monitoring items to form a monitoring item list includes:

[0196] Obtain the metrics for performance monitoring of a single storage cluster;

[0197] Based on the monitoring needs of each user, the indicators to be monitored are selected as monitoring items, and the monitoring items are summarized to form a monitoring item list for each user.

[0198] In this embodiment, the step of setting the task script for periodically collecting monitoring data includes:

[0199] A storage folder is set up for each user's monitoring item list, and the remote server path for storing monitoring data is formed based on the address of the storage folder;

[0200] Set the daily start and stop times for the task script.

[0201] In this embodiment, the step of determining whether the transfer time has been reached based on the last transferred data time and the transfer period, and triggering the performance monitoring data transfer if so, and matching the performance monitoring data of the indicator items collected by the task script with the monitoring item list, includes:

[0202] Detect whether the current time is later than the sum of the last data transfer time and the transfer period;

[0203] If so, the performance monitoring data will be transferred, and a list of monitoring items for each user will be obtained.

[0204] Obtain all the metrics collected by the task script, and iterate through the monitoring item list of each user to determine whether each of the metrics collected by the task script exists.

[0205] When a monitoring item with the same target metric item exists in the monitoring item list, a transfer directory is generated based on the monitoring level and transfer time of the monitoring item, and the performance monitoring data is exported to the cluster directory.

[0206] When no monitoring item is found in the monitoring item list that is the same as the target indicator item, the monitoring items in the next user's monitoring item list are traversed to determine whether the target indicator item exists.

[0207] When no monitoring item matching the target metric item exists in the monitoring item list of all users, the next metric item collected by the task script is obtained, and the monitoring items are traversed one by one in the monitoring item list of each user to determine whether there is a next metric item.

[0208] Determine whether all metrics collected by the task script have been matched and exported, and the performance monitoring data has been uploaded to the cloud server for storage. If yes, the process ends; otherwise, obtain the next metric collected by the task script and iterate through the monitoring item list of each user to determine whether the next metric exists.

[0209] In this embodiment, the step of uploading the performance monitoring data of the indicator items that match the monitoring items in the monitoring item list to the cloud server for storage includes:

[0210] Obtain the performance monitoring data of the indicator items that match the monitoring items in the monitoring item list for each user, and obtain the remote server path of the storage folder set for the monitoring item list for each user;

[0211] Obtain the performance monitoring level of the monitoring items in the monitoring item list, and store them in the storage folder corresponding to the remote server path according to the performance monitoring level of the monitoring items.

[0212] In this embodiment, the step of retrieving the performance monitoring data of the target cluster's collected metrics, generating monitoring charts, and displaying them in response to the display of monitoring data for the target cluster includes:

[0213] Query the remote server, update the list of monitoring items and the cluster network interconnection protocol of the collected monitoring data, and return the cluster network interconnection protocol of the collected performance monitoring data to the interface.

[0214] Based on the cluster network interconnection protocol selected by the user, create a corresponding monitoring chart. Based on the collected monitoring data, use the first-level directory under the cluster monitoring data as the monitoring object type and the second-level directory as a monitoring instance. Set one monitoring object type to match multiple monitoring items.

[0215] The remote server calculates and returns the time period of the backed-up monitoring data according to the performance monitoring level classification storage directory, and provides the user with the option to select the time period.

[0216] Import the monitoring data corresponding to the time period selected by the user into the database, and query the monitoring indicators in the monitoring data corresponding to the time period selected by the user for the user to query.

[0217] Based on the monitoring metrics queried by the user, the saved monitoring data is used to generate monitoring charts and returned to the interface to display the query results.

[0218] In this embodiment, as Figure 8 As shown, the historical performance monitoring data management device 10 also includes a metadata import and display module 6.

[0219] The metadata import and display module 6 is used to: export and back up the performance monitoring data corresponding to the monitoring item queried by the user, with the storage directory of the exported backup being IP address / monitoring object type / first-level monitoring / second-level monitoring item / time / monitoring item name; in response to receiving the metadata of the target monitoring item, create a metadata monitoring chart and obtain the monitoring object type, monitoring item name and time corresponding to the target monitoring item; the user obtains the remote path for storing the monitoring data of the monitoring item according to the monitoring object type, monitoring item name and time corresponding to the target monitoring item; if the path exists and the sub-path is time, then the metadata of the target monitoring item within the time period is retrieved and imported into the database and displayed to the user; if the sub-path is not time, then all sub-paths are returned to the interface, and after the user selects one of the sub-paths, the metadata of the queried target monitoring item is returned to the interface for display.

[0220] In the aforementioned historical performance monitoring data management device, a performance monitoring configuration file is set to set indicator items as monitoring items. The monitoring items are then aggregated to form a monitoring item list. Different monitoring item lists are selectively set for different users to differentiate the monitoring of performance monitoring data from different vendors. In addition, a task script is set to collect monitoring data on a regular basis, which periodically traverses the monitoring items in the monitoring item list to obtain the corresponding performance monitoring data. The performance monitoring data of the indicator items that match the monitoring items in the monitoring item list is uploaded to the cloud server for storage. This supports long-term storage of historical performance monitoring data and supports quick querying of backup performance monitoring data across clusters, which is then displayed on the interface.

[0221] Specific limitations regarding the historical performance monitoring data management device can be found in the limitations of the historical performance monitoring data management method described above, and will not be repeated here. Each module in the aforementioned historical performance monitoring data management device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the corresponding operations of each module.

[0222] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, performs the following steps:

[0223] Configure a performance monitoring configuration file to set metrics as monitoring items, and summarize the monitoring items to form a monitoring item list.

[0224] Set up a task script to collect monitoring data on a regular basis. The task script includes a list of database paths corresponding to the monitoring items, a transfer period, the last time the data was transferred, and the path of the remote server where the monitoring data is stored.

[0225] In response to the startup of the task script, the latest performance monitoring configuration file is obtained from the remote server path where the monitoring data is stored, and the list of monitoring items and the list of database paths corresponding to the metric items are updated.

[0226] Based on the last data transfer time and the transfer period, it is determined whether the transfer time has been reached. If so, the performance monitoring data transfer is triggered. The performance monitoring data of the indicator items collected by the task script is matched with the monitoring item list. The performance monitoring data of the indicator items that match the monitoring items in the monitoring item list is uploaded to the cloud server for storage.

[0227] In response to the display of monitoring data for the target cluster, the system retrieves the performance monitoring data of the collected metrics for the target cluster, generates monitoring charts, and displays them.

[0228] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0229] The steps of setting up the performance monitoring configuration file, which is used to set metrics as monitoring items, and summarizing the monitoring items to form a monitoring item list include:

[0230] Obtain the metrics for performance monitoring of a single storage cluster;

[0231] Based on the monitoring needs of each user, the indicators to be monitored are selected as monitoring items, and the monitoring items are summarized to form a monitoring item list for each user.

[0232] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0233] The steps for setting up a task script to collect monitoring data periodically include:

[0234] A storage folder is set up for each user's monitoring item list, and the remote server path for storing monitoring data is formed based on the address of the storage folder;

[0235] Set the daily start and stop times for the task script.

[0236] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0237] The step of determining whether the transfer time has been reached based on the last transferred data time and the transfer period, and triggering the performance monitoring data transfer if so, and matching the performance monitoring data of the indicator items collected by the task script with the monitoring item list includes:

[0238] Detect whether the current time is later than the sum of the last data transfer time and the transfer period;

[0239] If so, the performance monitoring data will be transferred, and a list of monitoring items for each user will be obtained.

[0240] Obtain all the metrics collected by the task script, and iterate through the monitoring item list of each user to determine whether each of the metrics collected by the task script exists.

[0241] When a monitoring item with the same target metric item exists in the monitoring item list, a transfer directory is generated based on the monitoring level and transfer time of the monitoring item, and the performance monitoring data is exported to the cluster directory.

[0242] When no monitoring item is found in the monitoring item list that is the same as the target indicator item, the monitoring items in the next user's monitoring item list are traversed to determine whether the target indicator item exists.

[0243] When no monitoring item matching the target metric item exists in the monitoring item list of all users, the next metric item collected by the task script is obtained, and the monitoring items are traversed one by one in the monitoring item list of each user to determine whether there is a next metric item.

[0244] Determine whether all metrics collected by the task script have been matched and exported, and the performance monitoring data has been uploaded to the cloud server for storage. If yes, the process ends; otherwise, obtain the next metric collected by the task script and iterate through the monitoring item list of each user to determine whether the next metric exists.

[0245] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0246] The step of uploading the performance monitoring data of the indicator items that match the monitoring items in the monitoring item list to the cloud server for storage includes:

[0247] Obtain the performance monitoring data of the indicator items that match the monitoring items in the monitoring item list for each user, and obtain the remote server path of the storage folder set for the monitoring item list for each user;

[0248] Obtain the performance monitoring level of the monitoring items in the monitoring item list, and store them in the storage folder corresponding to the remote server path according to the performance monitoring level of the monitoring items.

[0249] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0250] The steps for responding to the display of monitoring data for the target cluster by retrieving the performance monitoring data of the collected metrics of the target cluster, generating monitoring charts, and displaying them include:

[0251] Query the remote server, update the list of monitoring items and the cluster network interconnection protocol of the collected monitoring data, and return the cluster network interconnection protocol of the collected performance monitoring data to the interface.

[0252] Based on the cluster network interconnection protocol selected by the user, create a corresponding monitoring chart. Based on the collected monitoring data, use the first-level directory under the cluster monitoring data as the monitoring object type and the second-level directory as a monitoring instance. Set one monitoring object type to match multiple monitoring items.

[0253] The remote server calculates and returns the time period of the backed-up monitoring data according to the performance monitoring level classification storage directory, and provides the user with the option to select the time period.

[0254] Import the monitoring data corresponding to the time period selected by the user into the database, and query the monitoring indicators in the monitoring data corresponding to the time period selected by the user for the user to query.

[0255] Based on the monitoring metrics queried by the user, the saved monitoring data is used to generate monitoring charts and returned to the interface to display the query results.

[0256] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0257] Export and back up the performance monitoring data corresponding to the monitoring items queried by the user. The storage directory for the exported backup is IP address / monitoring object type / first-level monitoring / second-level monitoring item / time / monitoring item name;

[0258] Upon receiving metadata of the target monitoring item, a metadata monitoring chart is created to obtain the monitoring object type, monitoring item name, and time corresponding to the target monitoring item.

[0259] The user obtains the remote path where the monitoring data of the target monitoring item is stored based on the monitoring object type, monitoring item name and time corresponding to the target monitoring item.

[0260] If the path exists and the sub-path is time-based, then the metadata of the target monitoring item within that time period is retrieved and imported into the database, and displayed to the user; if the sub-path is not time-based, then all sub-paths are returned to the interface, and after the user selects one of the sub-paths, the queried target monitoring item metadata is returned to the interface for display.

[0261] For specific limitations on the steps implemented when a computer program is executed by a processor, please refer to the limitations on the methods for managing historical performance monitoring data mentioned above, which will not be repeated here.

[0262] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 9 As shown, the computer device includes a processor, memory, network interface, and database connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and the database. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores historical performance monitoring data and management data. The network interface communicates with external terminals via a network connection. When the computer program is executed by the processor, it implements a historical performance monitoring data management method.

[0263] Those skilled in the art will understand that Figure 9The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0264] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to perform the following steps:

[0265] Configure a performance monitoring configuration file to set metrics as monitoring items, and summarize the monitoring items to form a monitoring item list.

[0266] Set up a task script to collect monitoring data on a regular basis. The task script includes a list of database paths corresponding to the monitoring items, a transfer period, the last time the data was transferred, and the path of the remote server where the monitoring data is stored.

[0267] In response to the startup of the task script, the latest performance monitoring configuration file is obtained from the remote server path where the monitoring data is stored, and the list of monitoring items and the list of database paths corresponding to the metric items are updated.

[0268] Based on the last data transfer time and the transfer period, it is determined whether the transfer time has been reached. If so, the performance monitoring data transfer is triggered. The performance monitoring data of the indicator items collected by the task script is matched with the monitoring item list. The performance monitoring data of the indicator items that match the monitoring items in the monitoring item list is uploaded to the cloud server for storage.

[0269] In response to the display of monitoring data for the target cluster, the system retrieves the performance monitoring data of the collected metrics for the target cluster, generates monitoring charts, and displays them.

[0270] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0271] The steps of setting up the performance monitoring configuration file, which is used to set metrics as monitoring items, and summarizing the monitoring items to form a monitoring item list include:

[0272] Obtain the metrics for performance monitoring of a single storage cluster;

[0273] Based on the monitoring needs of each user, the indicators to be monitored are selected as monitoring items, and the monitoring items are summarized to form a monitoring item list for each user.

[0274] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0275] The steps for setting up a task script to collect monitoring data periodically include:

[0276] A storage folder is set up for each user's monitoring item list, and the remote server path for storing monitoring data is formed based on the address of the storage folder;

[0277] Set the daily start and stop times for the task script.

[0278] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0279] The step of determining whether the transfer time has been reached based on the last transferred data time and the transfer period, and triggering the performance monitoring data transfer if so, and matching the performance monitoring data of the indicator items collected by the task script with the monitoring item list includes:

[0280] Detect whether the current time is later than the sum of the last data transfer time and the transfer period;

[0281] If so, the performance monitoring data will be transferred, and a list of monitoring items for each user will be obtained.

[0282] Obtain all the metrics collected by the task script, and iterate through the monitoring item list of each user to determine whether each of the metrics collected by the task script exists.

[0283] When a monitoring item with the same target metric item exists in the monitoring item list, a transfer directory is generated based on the monitoring level and transfer time of the monitoring item, and the performance monitoring data is exported to the cluster directory.

[0284] When no monitoring item is found in the monitoring item list that is the same as the target indicator item, the monitoring items in the next user's monitoring item list are traversed to determine whether the target indicator item exists.

[0285] When no monitoring item matching the target metric item exists in the monitoring item list of all users, the next metric item collected by the task script is obtained, and the monitoring items are traversed one by one in the monitoring item list of each user to determine whether there is a next metric item.

[0286] Determine whether all metrics collected by the task script have been matched and exported, and the performance monitoring data has been uploaded to the cloud server for storage. If yes, the process ends; otherwise, obtain the next metric collected by the task script and iterate through the monitoring item list of each user to determine whether the next metric exists.

[0287] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0288] The step of uploading the performance monitoring data of the indicator items that match the monitoring items in the monitoring item list to the cloud server for storage includes:

[0289] Obtain the performance monitoring data of the indicator items that match the monitoring items in the monitoring item list for each user, and obtain the remote server path of the storage folder set for the monitoring item list for each user;

[0290] Obtain the performance monitoring level of the monitoring items in the monitoring item list, and store them in the storage folder corresponding to the remote server path according to the performance monitoring level of the monitoring items.

[0291] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0292] The steps for responding to the display of monitoring data for the target cluster by retrieving the performance monitoring data of the collected metrics of the target cluster, generating monitoring charts, and displaying them include:

[0293] Query the remote server, update the list of monitoring items and the cluster network interconnection protocol of the collected monitoring data, and return the cluster network interconnection protocol of the collected performance monitoring data to the interface.

[0294] Based on the cluster network interconnection protocol selected by the user, create a corresponding monitoring chart. Based on the collected monitoring data, use the first-level directory under the cluster monitoring data as the monitoring object type and the second-level directory as a monitoring instance. Set one monitoring object type to match multiple monitoring items.

[0295] The remote server calculates and returns the time period of the backed-up monitoring data according to the performance monitoring level classification storage directory, and provides the user with the option to select the time period.

[0296] Import the monitoring data corresponding to the time period selected by the user into the database, and query the monitoring indicators in the monitoring data corresponding to the time period selected by the user for the user to query.

[0297] Based on the monitoring metrics queried by the user, the saved monitoring data is used to generate monitoring charts and returned to the interface to display the query results.

[0298] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0299] Export and back up the performance monitoring data corresponding to the monitoring items queried by the user. The storage directory for the exported backup is IP address / monitoring object type / first-level monitoring / second-level monitoring item / time / monitoring item name;

[0300] Upon receiving metadata of the target monitoring item, a metadata monitoring chart is created to obtain the monitoring object type, monitoring item name, and time corresponding to the target monitoring item.

[0301] The user obtains the remote path where the monitoring data of the target monitoring item is stored based on the monitoring object type, monitoring item name and time corresponding to the target monitoring item.

[0302] If the path exists and the sub-path is time-based, then the metadata of the target monitoring item within that time period is retrieved and imported into the database, and displayed to the user; if the sub-path is not time-based, then all sub-paths are returned to the interface, and after the user selects one of the sub-paths, the queried target monitoring item metadata is returned to the interface for display.

[0303] For specific limitations on the steps a processor takes when executing a computer program, please refer to the limitations on the methods for managing historical performance monitoring data mentioned above, which will not be repeated here.

[0304] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program performing the following steps when executed by a processor:

[0305] Configure a performance monitoring configuration file to set metrics as monitoring items, and summarize the monitoring items to form a monitoring item list.

[0306] Set up a task script to collect monitoring data on a regular basis. The task script includes a list of database paths corresponding to the monitoring items, a transfer period, the last time the data was transferred, and the path of the remote server where the monitoring data is stored.

[0307] In response to the startup of the task script, the latest performance monitoring configuration file is obtained from the remote server path where the monitoring data is stored, and the list of monitoring items and the list of database paths corresponding to the metric items are updated.

[0308] Based on the last data transfer time and the transfer period, it is determined whether the transfer time has been reached. If so, the performance monitoring data transfer is triggered. The performance monitoring data of the indicator items collected by the task script is matched with the monitoring item list. The performance monitoring data of the indicator items that match the monitoring items in the monitoring item list is uploaded to the cloud server for storage.

[0309] In response to the display of monitoring data for the target cluster, the system retrieves the performance monitoring data of the collected metrics for the target cluster, generates monitoring charts, and displays them.

[0310] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0311] The steps of setting up the performance monitoring configuration file, which is used to set metrics as monitoring items, and summarizing the monitoring items to form a monitoring item list include:

[0312] Obtain the metrics for performance monitoring of a single storage cluster;

[0313] Based on the monitoring needs of each user, the indicators to be monitored are selected as monitoring items, and the monitoring items are summarized to form a monitoring item list for each user.

[0314] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0315] The steps for setting up a task script to collect monitoring data periodically include:

[0316] A storage folder is set up for each user's monitoring item list, and the remote server path for storing monitoring data is formed based on the address of the storage folder;

[0317] Set the daily start and stop times for the task script.

[0318] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0319] The step of determining whether the transfer time has been reached based on the last transferred data time and the transfer period, and triggering the performance monitoring data transfer if so, and matching the performance monitoring data of the indicator items collected by the task script with the monitoring item list includes:

[0320] Detect whether the current time is later than the sum of the last data transfer time and the transfer period;

[0321] If so, the performance monitoring data will be transferred, and a list of monitoring items for each user will be obtained.

[0322] Obtain all the metrics collected by the task script, and iterate through the monitoring item list of each user to determine whether each of the metrics collected by the task script exists.

[0323] When a monitoring item with the same target metric item exists in the monitoring item list, a transfer directory is generated based on the monitoring level and transfer time of the monitoring item, and the performance monitoring data is exported to the cluster directory.

[0324] When no monitoring item is found in the monitoring item list that is the same as the target indicator item, the monitoring items in the next user's monitoring item list are traversed to determine whether the target indicator item exists.

[0325] When no monitoring item matching the target metric item exists in the monitoring item list of all users, the next metric item collected by the task script is obtained, and the monitoring items are traversed one by one in the monitoring item list of each user to determine whether there is a next metric item.

[0326] Determine whether all metrics collected by the task script have been matched and exported, and the performance monitoring data has been uploaded to the cloud server for storage. If yes, the process ends; otherwise, obtain the next metric collected by the task script and iterate through the monitoring item list of each user to determine whether the next metric exists.

[0327] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0328] The step of uploading the performance monitoring data of the indicator items that match the monitoring items in the monitoring item list to the cloud server for storage includes:

[0329] Obtain the performance monitoring data of the indicator items that match the monitoring items in the monitoring item list for each user, and obtain the remote server path of the storage folder set for the monitoring item list for each user;

[0330] Obtain the performance monitoring level of the monitoring items in the monitoring item list, and store them in the storage folder corresponding to the remote server path according to the performance monitoring level of the monitoring items.

[0331] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0332] The steps for responding to the display of monitoring data for the target cluster by retrieving the performance monitoring data of the collected metrics of the target cluster, generating monitoring charts, and displaying them include:

[0333] Query the remote server, update the list of monitoring items and the cluster network interconnection protocol of the collected monitoring data, and return the cluster network interconnection protocol of the collected performance monitoring data to the interface.

[0334] Based on the cluster network interconnection protocol selected by the user, create a corresponding monitoring chart. Based on the collected monitoring data, use the first-level directory under the cluster monitoring data as the monitoring object type and the second-level directory as a monitoring instance. Set one monitoring object type to match multiple monitoring items.

[0335] The remote server calculates and returns the time period of the backed-up monitoring data according to the performance monitoring level classification storage directory, and provides the user with the option to select the time period.

[0336] Import the monitoring data corresponding to the time period selected by the user into the database, and query the monitoring indicators in the monitoring data corresponding to the time period selected by the user for the user to query.

[0337] Based on the monitoring metrics queried by the user, the saved monitoring data is used to generate monitoring charts and returned to the interface to display the query results.

[0338] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0339] Export and back up the performance monitoring data corresponding to the monitoring items queried by the user. The storage directory for the exported backup is IP address / monitoring object type / first-level monitoring / second-level monitoring item / time / monitoring item name;

[0340] Upon receiving metadata of the target monitoring item, a metadata monitoring chart is created to obtain the monitoring object type, monitoring item name, and time corresponding to the target monitoring item.

[0341] The user obtains the remote path where the monitoring data of the target monitoring item is stored based on the monitoring object type, monitoring item name and time corresponding to the target monitoring item.

[0342] If the path exists and the sub-path is time-based, then the metadata of the target monitoring item within that time period is retrieved and imported into the database, and displayed to the user; if the sub-path is not time-based, then all sub-paths are returned to the interface, and after the user selects one of the sub-paths, the queried target monitoring item metadata is returned to the interface for display.

[0343] For specific limitations on the steps implemented when a computer program is executed by a processor, please refer to the limitations on the methods for managing historical performance monitoring data mentioned above, which will not be repeated here.

[0344] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0345] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0346] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A method for managing historical performance monitoring data, characterized in that, include: Configure a performance monitoring configuration file to set metrics as monitoring items, and summarize the monitoring items to form a monitoring item list. Set up a task script to collect monitoring data on a regular basis. The task script includes a list of database paths corresponding to the monitoring items, a transfer period, the last time the data was transferred, and the path of the remote server where the monitoring data is stored. In response to the startup of the task script, the latest performance monitoring configuration file is obtained from the remote server path where the monitoring data is stored, and the list of monitoring items and the list of database paths corresponding to the metric items are updated. Based on the last data transfer time and the transfer period, it is determined whether the transfer time has been reached. If so, the performance monitoring data transfer is triggered. The performance monitoring data of the indicator items collected by the task script is matched with the monitoring item list. The performance monitoring data of the indicator items that match the monitoring items in the monitoring item list is uploaded to the cloud server for storage. In response to the display of monitoring data for the target cluster, the system retrieves the performance monitoring data of the collected metrics for the target cluster, generates monitoring charts, and displays them. The step of determining whether the transfer time has been reached based on the last transferred data time and the transfer period, and triggering the performance monitoring data transfer if so, and matching the performance monitoring data of the indicator items collected by the task script with the monitoring item list includes: Detect whether the current time is later than the sum of the last data transfer time and the transfer period; If so, the performance monitoring data will be transferred, and a list of monitoring items for each user will be obtained. Obtain all the metrics collected by the task script, and iterate through the monitoring item list of each user to determine whether each of the metrics collected by the task script exists. When a monitoring item with the same target metric item exists in the monitoring item list, a transfer directory is generated based on the monitoring level and transfer time of the monitoring item, and the performance monitoring data is exported to the cluster directory. When no monitoring item is found in the monitoring item list that is the same as the target indicator item, the monitoring items in the next user's monitoring item list are traversed to determine whether the target indicator item exists. When no monitoring item matching the target metric item exists in the monitoring item list of all users, the next metric item collected by the task script is obtained, and the monitoring items are traversed one by one in the monitoring item list of each user to determine whether there is a next metric item. Determine whether all the metrics collected by the task script have been matched and exported, and uploaded to the cloud server for storage. If yes, the process ends; otherwise, obtain the next metric collected by the task script and iterate through the monitoring items in each user's monitoring item list to determine whether there is a next metric. The step of uploading the performance monitoring data of the indicator items that match the monitoring items in the monitoring item list to the cloud server for storage includes: Obtain the performance monitoring data of the indicator items that match the monitoring items in the monitoring item list for each user, and obtain the remote server path of the storage folder set for the monitoring item list for each user; Obtain the performance monitoring level of the monitoring items in the monitoring item list, and store them in the storage folder corresponding to the remote server path according to the performance monitoring level of the monitoring items. The step of retrieving the performance monitoring data of the target cluster's collected metrics, generating monitoring charts, and displaying them in response to the display of monitoring data for the target cluster includes: Query the remote server, update the list of monitoring items and the cluster network interconnection protocol of the collected monitoring data, and return the cluster network interconnection protocol of the collected performance monitoring data to the interface. Based on the cluster network interconnection protocol selected by the user, create a corresponding monitoring chart. Based on the collected monitoring data, use the first-level directory under the cluster monitoring data as the monitoring object type and the second-level directory as a monitoring instance. Set one monitoring object type to match multiple monitoring items. The remote server calculates and returns the time period of the backed-up monitoring data according to the performance monitoring level classification storage directory, and provides the user with the option to select the time period. Import the monitoring data corresponding to the time period selected by the user into the database, and query the monitoring items in the monitoring data corresponding to the time period selected by the user for the user to query. Based on the monitoring items queried by the user, the saved monitoring data is used to generate monitoring charts and returned to the interface to display the query results; The method further includes: Export and back up the performance monitoring data corresponding to the monitoring items queried by the user. The storage directory for the exported backup is IP address / monitoring object type / first-level monitoring / second-level monitoring item / time / monitoring item name; Upon receiving metadata of the target monitoring item, a metadata monitoring chart is created to obtain the monitoring object type, monitoring item name, and time corresponding to the target monitoring item. The user obtains the remote path where the monitoring data of the target monitoring item is stored based on the monitoring object type, monitoring item name and time corresponding to the target monitoring item. If the path exists and the sub-path is time-based, then the metadata of the target monitoring item within that time period is retrieved and imported into the database, and displayed to the user; if the sub-path is not time-based, then all sub-paths are returned to the interface, and after the user selects one of the sub-paths, the queried target monitoring item metadata is returned to the interface for display.

2. The historical performance monitoring data management method according to claim 1, characterized in that, The steps of setting up the performance monitoring configuration file, which is used to set metrics as monitoring items, and summarizing the monitoring items to form a monitoring item list include: Obtain the metrics for performance monitoring of a single storage cluster; Based on the monitoring needs of each user, the indicators to be monitored are selected as monitoring items, and the monitoring items are summarized to form a monitoring item list for each user.

3. The historical performance monitoring data management method according to claim 2, characterized in that, The steps for setting up a task script to collect monitoring data periodically include: A storage folder is set up for each user's monitoring item list, and the remote server path for storing monitoring data is formed based on the address of the storage folder; Set the daily start and stop times for the task script.

4. A historical performance monitoring data management device, characterized in that, The apparatus for implementing the historical performance monitoring data management method according to any one of claims 1 to 3, the apparatus comprising: The performance monitoring configuration file module is used to set up performance monitoring configuration files, set metrics as monitoring items, and summarize the monitoring items to form a monitoring item list. The task script setting module is used to set up task scripts for periodically collecting monitoring data. The task scripts include a list of database paths corresponding to the monitoring items, a transfer cycle, the last data transfer time, and the path of the remote server where the monitoring data is stored. The monitoring information update module is used to respond to the startup of the task script by obtaining the latest performance monitoring configuration file from the remote server path where the monitoring data is stored, and updating the list of monitoring items and the list of database paths corresponding to the indicator items. The monitoring data backup module is used to determine whether the transfer time has been reached based on the last data transfer time and the transfer period. If so, it triggers the performance monitoring data transfer, matches the performance monitoring data of the indicator items collected by the task script with the monitoring item list, and uploads the performance monitoring data of the indicator items that match the monitoring items in the monitoring item list to the cloud server for storage. The import and display module is used to retrieve the performance monitoring data of the target cluster's collected metrics when displaying monitoring data of the target cluster's monitoring items, generate monitoring charts, and display them.

5. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 3.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 3.

Images