Server cluster monitoring method and device and server cluster

Abstract

The invention provides a server cluster monitoring method and device and a server cluster, and relates to the technical field of computers, and the method comprises the steps: collecting resource use data of each calculation task in the server cluster, generating first data, and storing the first data in a relational database; collecting resource use data at each monitoring moment in the server cluster to generate second data, and storing the second data in a time sequence database; scanning the first data and the second data, and respectively generating a first resource use matrix and a second resource use matrix; the resource use matrix is used for representing resource use data of each calculation task in the server cluster at each monitoring moment; and performing data correction on the relational database or the time sequence database based on a difference value between the first resource use matrix and the second resource use matrix. According to the method and the device provided by the invention, the data correction efficiency of server cluster monitoring is improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a server cluster monitoring method, device and server cluster. Background Art

[0002] A server cluster is a system consisting of multiple servers that are connected through a network and work together to achieve higher computing power, availability, and reliability. For example, it can run large neural network models and provide artificial intelligence computing services. In order to ensure that the server cluster can provide better services, it is necessary to monitor the resource usage data in the server cluster.

[0003] Related technologies use different types of databases to record resource usage data in server clusters. When data inconsistencies occur, manual investigation is usually used to compare data in different databases, which makes the data correction efficiency of server cluster monitoring low and cannot be used to accurately analyze resource usage data in server clusters.

[0004] Therefore, how to improve the correction efficiency of server cluster monitoring data has become a technical problem that needs to be solved urgently in the industry. Summary of the invention

[0005] The present application provides a server cluster monitoring method, device and server cluster, which are used to solve the technical problem of how to improve the correction efficiency of server cluster monitoring data.

[0006] The present application provides a server cluster monitoring method, comprising: Collect resource usage data of each computing task in the server cluster to generate first data, and store the first data in a relational database; Collect resource usage data at each monitoring moment in the server cluster, generate second data, and store the second data in a time series database; Scanning the first data and the second data to generate a first resource usage matrix and a second resource usage matrix respectively; the resource usage matrix is ​​used to represent resource usage data of each computing task in the server cluster at each monitoring time; Based on the difference between the first resource usage matrix and the second resource usage matrix, data correction is performed on the relational database or the time series database.

[0007] In some embodiments, scanning the first data and the second data to generate a first resource usage matrix and a second resource usage matrix respectively includes: determining a target time period and a type of resource usage to be monitored during the target time period; Scanning the first data corresponding to the target time period in the relational database to generate a first resource usage matrix corresponding to each resource usage type; Scan the second data corresponding to the target time period in the time series database to generate a second resource usage matrix corresponding to each resource usage type; The rows in the first resource usage matrix and the second resource usage matrix are determined based on computing tasks, the columns are determined based on monitoring moments, and the element values ​​are determined based on resource usage data of any computing task at any monitoring moment.

[0008] In some embodiments, the element values ​​in the first resource usage matrix and the second resource usage matrix are determined based on the following steps: Determine the resource usage type corresponding to the element value; Based on the time series database, obtaining resource usage data of each computing task for the resource usage type at each monitoring moment; Determining a resource usage baseline value for each computing task at each monitoring moment based on resource usage data of the resource usage type for each computing task at each monitoring moment; In the case where the resource usage data of any computing task for the resource usage type at any monitoring time is greater than the resource usage reference value, determining the element value corresponding to the any computing task at the any monitoring time as a first preset value; When the resource usage data of any computing task for the resource usage type at any monitoring moment is less than the resource usage reference value, the element value corresponding to the any computing task at the any monitoring moment is determined as a second preset value.

[0009] In some embodiments, the performing data correction on the relational database or the time series database based on the difference between the first resource usage matrix and the second resource usage matrix includes: Obtaining the system load of the server cluster; When the system load is less than a preset load, data correction is performed on the relational database or the time series database based on a difference between the first resource usage matrix and the second resource usage matrix.

[0010] In some embodiments, the performing data correction on the relational database or the time series database based on the difference between the first resource usage matrix and the second resource usage matrix includes: performing a difference operation on an element value at any position in the first resource usage matrix and an element value at a corresponding position in the first resource usage matrix; When the difference is zero, determining that there is no deviation between the first data and the second data; When the difference is not zero, it is determined that there is a deviation between the first data and the second data, the credibility of the first data and the second data is determined, and data correction is performed on the relational database or the time series database based on the data with higher credibility.

[0011] In some embodiments, after performing data correction on the relational database or the time series database based on the difference between the first resource usage matrix and the second resource usage matrix, the method further includes: Generate a report based on the data correction status of the relational database or the time series database; The report is sent to the monitoring platform of the server cluster.

[0012] The present application provides a server cluster monitoring device, comprising: A first collection module, used to collect resource usage data of each computing task in the server cluster, generate first data, and store the first data in a relational database; A second collection module, used to collect resource usage data at each monitoring moment in the server cluster, generate second data, and store the second data in a time series database; A data scanning module, used to scan the first data and the second data to generate a first resource usage matrix and a second resource usage matrix respectively; the resource usage matrix is ​​used to represent resource usage data of each computing task in the server cluster at each monitoring time; A data correction module is used to correct data on the relational database or the time series database based on the difference between the first resource usage matrix and the second resource usage matrix.

[0013] The present application provides a server cluster, including a server cluster monitoring device and a plurality of server nodes; The server cluster monitoring device is connected to the server node and is used to execute the server cluster monitoring method.

[0014] The present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the server cluster monitoring method is implemented when the processor executes the computer program.

[0015] The present application provides a non-transitory computer-readable storage medium having a computer program stored thereon, and the computer program implements the server cluster monitoring method when executed by a processor.

[0016] The server cluster monitoring method, device and server cluster provided in the present application collect resource usage data of each computing task in the server cluster, generate first data, and store the first data in a relational database; collect resource usage data at each monitoring moment in the server cluster, generate second data, and store the second data in a time series database; scan the first data and the second data to generate a first resource usage matrix and a second resource usage matrix respectively; based on the difference between the first resource usage matrix and the second resource usage matrix, perform data correction on the relational database or the time series database; because the resource usage data of each computing task in the server cluster at each monitoring moment is represented by a resource usage matrix, matrix operations can be used to quickly and accurately determine the data deviation between the two databases and the location of the data with the deviation, without the need for manual investigation, thereby improving the data correction efficiency of server cluster monitoring and improving the accuracy of resource usage data analysis in the server cluster. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0018] In order to more clearly illustrate the technical solutions in the present application or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0019] Figure 1 This is one of the flow charts of the server cluster monitoring method provided by this application.

[0020] Figure 2 This is the second flow chart of the server cluster monitoring method provided by this application.

[0021] Figure 3 It is a structural diagram of the server cluster monitoring device provided in this application.

[0022] Figure 4 It is a schematic diagram of the structure of the server cluster provided in this application.

[0023] Figure 5 It is a structural schematic diagram of the electronic device provided by this application. DETAILED DESCRIPTION

[0024] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0025] It should be noted that the terms "first", "second" etc. in the present application are used to distinguish similar objects, and need not be used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchangeable in appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, the process, method, system, product or equipment comprising a series of steps or units or modules need not be limited to those steps or units or modules clearly listed, but may include other steps or units or modules that are not clearly listed or inherent to these processes, methods, products or equipment.

[0026] In the monitoring of server clusters, databases play a vital role, and different types of databases are usually used to store monitoring data. Monitoring data helps users (administrators) better understand cluster problems, and monitoring statistics need to provide users with more detailed and accurate data reports. As time goes by and the amount of data surges, the data stored in the database may be inconsistent, resulting in inaccurate monitoring results, affecting the stability and reliability of the system, and increasing the pressure on cluster maintenance personnel.

[0027] In order to solve the shortcomings of related technologies, Figure 1 This is one of the flow charts of the server cluster monitoring method provided by this application, such as Figure 1 As shown, the method includes step 110 , step 120 , step 130 and step 140 .

[0028] Step 110: Collect resource usage data of each computing task in the server cluster, generate first data, and store the first data in a relational database.

[0029] Specifically, the server cluster monitoring method provided in the embodiment of the present application is implemented by a server cluster monitoring device. The device can be implemented by software, such as a server cluster monitoring program running in a storage system; or it can be a device that executes the server cluster monitoring method, such as a desktop computer or server in a storage system.

[0030] Computing tasks refer to data computing tasks performed by the server cluster, such as neural network model training and reasoning tasks. Resource usage data refers to the usage of various hardware resources and software resources in the server cluster, including system load, network traffic usage, processor usage, memory space usage, and disk space usage.

[0031] Resource usage data of each computing task can be collected to generate first data. The first data can be resource usage data recorded in units of computing tasks, and can be stored in a relational database. A relational database organizes data in the form of tables, which are composed of rows and columns. Data tables can be connected and associated through relationships such as foreign keys. Relational databases support complex queries, data integrity, and transaction processing, and are suitable for the storage and management of structured data. Common relational database systems include MySQL and the like.

[0032] Step 120: Collect resource usage data at each monitoring moment in the server cluster, generate second data, and store the second data in a time series database.

[0033] Specifically, relational databases may encounter performance bottlenecks in processing large-scale time series data, and time series databases can be used to store resource usage data.

[0034] The main difference between time series databases and relational databases lies in their design concepts and data storage methods. Time series databases are specifically designed to process time series data, which is usually a series of records generated in chronological order. In the server cluster monitoring business, time series databases can efficiently store and query monitoring data that changes over time, such as system load, network traffic, storage usage, etc.

[0035] The resource usage data of each computing task in the server cluster can be collected in the time sequence of each monitoring moment to form second data, and stored in a time series database.

[0036] Relational databases and time series databases store the monitoring data of server clusters synchronously. Relational databases use computing tasks as units, support complex queries and transaction processing, and have high data consistency, but have performance bottlenecks when processing large-scale data and high-concurrency scenarios. Time series databases use monitoring moments as units, optimize the storage and query performance of time series data, and can efficiently handle high-frequency data writing and querying, but are not suitable for complex relational queries and transaction management. The combination of the two can effectively store server monitoring data.

[0037] Step 130: Scan the first data and the second data to generate a first resource usage matrix and a second resource usage matrix respectively; the resource usage matrix is ​​used to represent resource usage data of each computing task in the server cluster at each monitoring time.

[0038] Specifically, as time goes by and the amount of data increases, data in relational databases and time series databases may deviate. Related technologies use manual investigation to extract all business data within a certain period of time from the relational database and then compare them one by one with the data in the time series database.

[0039] In an embodiment of the present application, the first data in the relational database can be scanned, and a first resource usage matrix can be generated for the resource usage data in the server cluster at each monitoring time according to each computing task; the second data in the time series database can be scanned, and a second resource usage matrix can be generated for the resource usage data in the server cluster at each monitoring time according to each computing task.

[0040] Step 140: Based on the difference between the first resource usage matrix and the second resource usage matrix, perform data correction on the relational database or the time series database.

[0041] Specifically, the first resource usage matrix and the second resource usage matrix can be subtracted, and the data deviations in the two databases and the positions corresponding to the data deviations can be quickly determined based on the difference, so as to correct the data in the relational database or the time series database. The comparison of resource usage matrices can also reduce the interference with a large amount of normal data and accurately locate abnormal data.

[0042] For example, if the difference in any element value is zero, it means that the resource usage data of the same computing task represented by the element value in the two databases at the same monitoring time are consistent, and no correction is needed; if the difference is not zero, it means that the resource usage data of the same computing task represented by the element value in the two databases at the same monitoring time are inconsistent, and correction is needed. According to the position of the element value in the matrix, the corresponding computing task and monitoring time can be quickly located, and the data of the other database can be corrected according to the database with higher credibility.

[0043] The server cluster monitoring method provided in the embodiment of the present application collects resource usage data of each computing task in the server cluster to generate first data, and stores the first data in a relational database; collects resource usage data at each monitoring moment in the server cluster to generate second data, and stores the second data in a time series database; scans the first data and the second data to generate a first resource usage matrix and a second resource usage matrix, respectively; and performs data correction on the relational database or the time series database based on the difference between the first resource usage matrix and the second resource usage matrix; because the resource usage data of each computing task in the server cluster at each monitoring moment is represented by a resource usage matrix, matrix operations can be used to quickly and accurately determine the data deviation between the two databases and the location of the data with the deviation, without the need for manual investigation, thereby improving the data correction efficiency of server cluster monitoring and improving the accuracy of resource usage data analysis in the server cluster.

[0044] It should be noted that each implementation method of the present application can be freely combined, the order can be changed, or it can be executed separately, and does not need to rely on or depend on a fixed execution order.

[0045] In some embodiments, scanning the first data and the second data to generate a first resource usage matrix and a second resource usage matrix respectively includes: Determine the target time period and the type of resource usage to be monitored during the target time period; Scanning the first data corresponding to the target time period in the relational database to generate a first resource usage matrix corresponding to each resource usage type; Scan the second data corresponding to the target time period in the time series database to generate a second resource usage matrix corresponding to each resource usage type; The rows in the first resource usage matrix and the second resource usage matrix are determined based on computing tasks, the columns are determined based on monitoring moments, and the element values ​​are determined based on resource usage data of any computing task at any monitoring moment.

[0046] Specifically, since the amount of data monitored by the server cluster is large, a specified time period and a specified resource usage type may be compared. When scanning the first data and the second data, a target time period and a resource usage type to be monitored within the target time period may be determined. Resource usage types may include processors (including central processing units and graphics processing units, etc.), memory, network traffic, and disks, etc.

[0047] Scan the first data corresponding to the target time period in the relational database. Each time a piece of data is scanned, the first resource usage matrix is ​​determined according to the data entry rules or data recording rules of the relational database, with computing tasks as rows, monitoring times as columns, and resource usage data of any computing task at any monitoring time as element values. For example, the target time period is from 11 a.m. to 12 p.m., and monitoring is performed every 15 minutes. A total of 4 computing tasks are monitored. The resource usage types are central processing unit (CPU) usage data and graphics processing unit (GPU) usage data. The first resource usage matrix corresponding to the central processing unit and the first resource usage matrix corresponding to the graphics processing unit can be obtained, which can be expressed as:

[0048] In the formula, an element value of 0 indicates that the corresponding resource is not used, and an element value of 1 indicates that the corresponding resource is used.

[0049] In the first resource usage matrix corresponding to the CPU, there are 4 rows, corresponding to 4 computing tasks respectively; there are 5 columns, corresponding to 5 monitoring times (11:00, 11:15, 11:30, 11:45 and 12:00). The meanings of the rows and columns in other resource usage matrices are the same as those in this matrix, and will not be repeated here.

[0050] The second data corresponding to the target time period is scanned in the time series database. Each time a piece of data is scanned, the second resource usage matrix is ​​determined according to the data entry rules or data recording rules of the time series database, with computing tasks as rows, monitoring time as columns, and resource usage data of any computing task at any monitoring time as element values. For example, the second resource usage matrix corresponding to the central processing unit and the second resource usage matrix corresponding to the graphics processing unit can be obtained, which can be expressed as:

[0051] In the formula, an element value of 0 indicates that the corresponding resource is not used, and an element value of 1 indicates that the corresponding resource is used.

[0052] The server cluster monitoring method provided in the embodiment of the present application scans the first data and the second data to generate a first resource usage matrix and a second resource usage matrix respectively, and uses the resource usage matrix to represent the resource usage data in the server cluster for each computing task at each monitoring moment, which is beneficial to using matrix operations to quickly and accurately determine the data deviation between the two databases.

[0053] In some embodiments, the element values ​​in the first resource usage matrix and the second resource usage matrix are determined based on the following steps: Determine the resource usage type corresponding to the element value; Based on the time series database, obtain resource usage data of each computing task for resource usage types at each monitoring time; Determine a resource usage benchmark value for each computing task at each monitoring moment based on resource usage data for each computing task at each monitoring moment; When the resource usage data of the resource usage type of any computing task at any monitoring time is greater than the resource usage reference value, the element value corresponding to any computing task at any monitoring time is determined as a first preset value; When the resource usage data of the resource usage type of any computing task at any monitoring time is less than the resource usage reference value, the element value corresponding to any computing task at any monitoring time is determined as the second preset value.

[0054] Specifically, in order to facilitate comparison, the resource usage data can be effectively parameterized, that is, the resource usage data can be represented by appropriate effective parameters, which is conducive to realizing fast matrix operations.

[0055] It is understandable that the parameterization methods for different types of resources are different. For discrete resource usage data, two preset values ​​can be used to represent it, for example, 1 represents the use of resources and 0 represents the unused resources. For continuous resource usage data, a benchmark value can be determined based on the relevant records in the database, the average, median, etc. of the data, and the resource usage data of each computing task at each monitoring time can be compared with the benchmark value to achieve parameterization and represent it through a small number of effective parameters.

[0056] Specifically, first determine the resource usage type corresponding to the element value. Then obtain the resource usage data of each computing task at each monitoring time for the resource usage type from the time series database. The time series database collects various resource usage data according to the monitoring time, can store a large amount of time series data, and the data storage format is suitable for fast retrieval and analysis of time series data.

[0057] According to the resource usage data of each computing task for the resource usage type at each monitoring moment, the resource usage benchmark value of each computing task at each monitoring moment is determined by using methods such as finding the lowest common divisor, average or median.

[0058] When the resource usage data of any computing task for the resource usage type at any monitoring time is greater than the resource usage benchmark value, the element value corresponding to any computing task at any monitoring time is determined as a first preset value, for example, represented by 1.

[0059] When the resource usage data of any computing task for the resource usage type at any monitoring time is less than the resource usage benchmark value, the element value corresponding to any computing task at any monitoring time is determined as a second preset value, for example, represented by 0.

[0060] The server cluster monitoring method provided in the embodiment of the present application parameterizes the resource usage data and determines the element values ​​in the first resource usage matrix and the second resource usage matrix, which is beneficial to using matrix operations to quickly and accurately determine the data deviations between the two databases.

[0061] In some embodiments, based on the difference between the first resource usage matrix and the second resource usage matrix, data correction is performed on the relational database or the time series database, including: Get the system load of the server cluster; When the system load is less than the preset load, data correction is performed on the relational database or the time series database based on the difference between the first resource usage matrix and the second resource usage matrix.

[0062] Specifically, business computing tasks, such as artificial intelligence computing tasks, are running in the server cluster. In order to avoid resource competition and conflict between the server cluster and the business computing tasks when performing data correction, data correction can be performed during system idle time. A scheduled task with a configurable running interval can be set up to scan the data of the relational database and the time series database within a certain period of time for comparison. This task can be triggered manually and a time period needs to be specified. For example, during system idle time such as early morning, the above method is scheduled to perform data correction. It can also be determined based on the system load.

[0063] The resource configuration data of all server nodes in the server cluster can be counted, including the number of processor settings, processor computing power, memory settings, memory space size, network traffic data, and the number of server nodes. The maximum system load that the server cluster can bear is determined based on the resource configuration data. The preset load is determined based on the maximum system load and the preset proportional coefficient. The preset proportional coefficient can be a coefficient less than 1 and can be set according to actual conditions.

[0064] Obtain the system load of the server cluster and compare the system load with the preset load. If the system load is greater than or equal to the preset load, it means that the system load is large, the business computing tasks are heavy, and the resources are not sufficient; if the system load is less than the preset load, it means that the system load is small, the business computing tasks are light, and the resources are sufficient, and data correction can be performed. At this time, data correction can be performed on the relational database or the time series database based on the difference between the first resource usage matrix and the second resource usage matrix.

[0065] The server cluster monitoring method provided in the embodiment of the present application determines the timing of data correction according to the system load of the server cluster, which can achieve user-imperceptible execution and avoid the server cluster competing for resources for data correction during peak user usage periods, thereby improving the operating stability of the server cluster and the user experience.

[0066] In some embodiments, based on the difference between the first resource usage matrix and the second resource usage matrix, data correction is performed on the relational database or the time series database, including: Performing a difference operation on an element value at any position in the first resource usage matrix and an element value at a corresponding position in the first resource usage matrix; When the difference is zero, it is determined that there is no deviation between the first data and the second data; When the difference is not zero, it is determined that there is a deviation between the first data and the second data, the credibility of the first data and the second data is determined, and data correction is performed on the relational database or the time series database based on the data with higher credibility.

[0067] Specifically, when performing data correction, the element value at any position in the first resource usage matrix can be subtracted from the element value at the corresponding position in the first resource usage matrix. The corresponding position refers to the same row and column of the element value, representing the resource usage data of the same computing task at the same monitoring time.

[0068] If the difference is zero, it means that there is no deviation in the resource usage data of the computing task represented by the position at the monitoring time between the first data and the second data. If the difference is not zero, it means that there is a deviation in the resource usage data of the computing task represented by the position at the monitoring time between the first data and the second data, and the specific computing task and the specific monitoring time can be determined.

[0069] At this time, it is necessary to determine the credibility of the first data and the second data. If the credibility of the first data is high, it means that the data in the relational database is relatively accurate and can be used to correct the data in the time series database; if the credibility of the second data is high, it means that the data in the time series database is relatively accurate and can be used to correct the data in the relational database.

[0070] The number of data revisions in relational databases and time series databases can be counted. The fewer the number of data revisions, the higher the credibility of the corresponding database.

[0071] The server cluster monitoring method provided in the embodiment of the present application can quickly and accurately determine the data deviation between two databases based on matrix operations, and correct the data of the relational database or the time series database based on the data with higher credibility, thereby improving the data correction efficiency of server cluster monitoring and improving the accuracy of resource usage data analysis in the server cluster.

[0072] In some embodiments, after correcting data in the relational database or the time series database based on the difference between the first resource usage matrix and the second resource usage matrix, the method further includes: Generate reports based on data revisions in relational databases or time series databases; Send reports to the monitoring platform of the server cluster.

[0073] Specifically, after the data is corrected, the data correction status of the relational database or time series database can be collected and summarized, a report can be generated, and sent to the monitoring platform of the server cluster so that the user (administrator) can check the data correction status.

[0074] The server cluster monitoring method provided in the embodiment of the present application generates a report based on the data correction status of the relational database or the time series database. By dividing the work between the relational database and the time series database, the summary statistics of different databases are processed by the monitoring platform and displayed to the user, thereby helping the user to accurately obtain cluster information and improving the user's experience of using the server cluster.

[0075] Figure 2 This is the second flow chart of the server cluster monitoring method provided by this application, such as Figure 2 As shown, the method includes: Step 1: Subdivide the types of monitored resources, generally divided into all cluster resources, resources occupied by the cluster's own built-in services, resources requested by users, and resources being used by users. Monitor the server cluster and store the monitoring data in the relational database and time series database respectively.

[0076] Step 2: Scan the relational database and the time series database respectively to generate a resource usage matrix corresponding to the relational database and a resource usage matrix corresponding to the time series database.

[0077] Step 3: If the time series database data is controlled, the data is processed according to the data in the relational database and then updated or inserted; If the time series database data is not empty, the difference between the resource usage matrix corresponding to the relational database and the resource usage matrix corresponding to the time series database is determined according to the time statistical interval. If the difference between the two is 0, no correction is made. If it is not zero, the data that needs to be corrected is determined by the element value in the matrix. The data is based on the relational database (higher credibility). Based on the relational database, it ensures that the time series data statistics are closer to the user experience.

[0078] Step 4: After the data comparison is completed, a corresponding report is generated and sent to the administrator, who can verify the correction of the data difference. The administrator can download and view the original data and replacement data of the modified data, and approve it after verification. If the verification fails, the scheduled task will be manually executed.

[0079] If there are other types of monitoring data anomalies, it is necessary to construct matrix parameters specifically, set the number of matrix columns and rows according to the time series database capture parameters, and fill in the values ​​with actual business data as real data.

[0080] The server cluster monitoring method provided in the embodiment of the present application converts resource usage data into a resource usage matrix, and locates abnormal data very quickly and corrects it by means of matrix subtraction. The matrix generated by the business data can eliminate other unnecessary information and more accurately judge the business parameters of inconsistent data. All information of the corrected data is packaged and notified to the administrator, who analyzes whether the corrected data is correct. When it is abnormal, the administrator can correct it again based on the corrected report. The scope of inconsistent data is located in the fastest way, and the data is corrected with the least operation on the database, thereby improving overall efficiency, especially when there is a large amount of data and the time period when the problem occurs is not standardized, the effect is more obvious.

[0081] The device provided in the embodiment of the present application is described below. The device described below and the method described above can be referenced to each other.

[0082] Figure 3 It is a structural diagram of the server cluster monitoring device provided by this application, such as Figure 3 As shown, the device comprises: A first collection module 310 is used to collect resource usage data of each computing task in the server cluster, generate first data, and store the first data in a relational database; The second collection module 320 is used to collect resource usage data at each monitoring moment in the server cluster, generate second data, and store the second data in a time series database; A data scanning module 330 is used to scan the first data and the second data to generate a first resource usage matrix and a second resource usage matrix respectively; the resource usage matrix is ​​used to represent resource usage data of each computing task in the server cluster at each monitoring time; The data correction module 340 is used to correct the data of the relational database or the time series database based on the difference between the first resource usage matrix and the second resource usage matrix.

[0083] The server cluster monitoring device provided in the embodiment of the present application collects resource usage data of each computing task in the server cluster, generates first data, and stores the first data in a relational database; collects resource usage data at each monitoring moment in the server cluster, generates second data, and stores the second data in a time series database; scans the first data and the second data to generate a first resource usage matrix and a second resource usage matrix, respectively; and performs data correction on the relational database or the time series database based on the difference between the first resource usage matrix and the second resource usage matrix; since the resource usage data of each computing task in the server cluster at each monitoring moment is represented by a resource usage matrix, matrix operations can be used to quickly and accurately determine the data deviation between the two databases and the location of the data with the deviation, without the need for manual investigation, thereby improving the data correction efficiency of server cluster monitoring and improving the accuracy of resource usage data analysis in the server cluster.

[0084] In some embodiments, the data scanning module is used to: Determine the target time period and the type of resource usage to be monitored during the target time period; Scanning the first data corresponding to the target time period in the relational database to generate a first resource usage matrix corresponding to each resource usage type; Scan the second data corresponding to the target time period in the time series database to generate a second resource usage matrix corresponding to each resource usage type; The rows in the first resource usage matrix and the second resource usage matrix are determined based on computing tasks, the columns are determined based on monitoring moments, and the element values ​​are determined based on resource usage data of any computing task at any monitoring moment.

[0085] In some embodiments, the data scanning module is further configured to: Determine the resource usage type corresponding to the element value; Based on the time series database, obtain resource usage data of each computing task for resource usage types at each monitoring time; Determine a resource usage benchmark value for each computing task at each monitoring moment based on resource usage data for each computing task at each monitoring moment; When the resource usage data of the resource usage type of any computing task at any monitoring time is greater than the resource usage reference value, the element value corresponding to any computing task at any monitoring time is determined as a first preset value; When the resource usage data of the resource usage type of any computing task at any monitoring time is less than the resource usage reference value, the element value corresponding to any computing task at any monitoring time is determined as the second preset value.

[0086] In some embodiments, the data correction module is used to: Get the system load of the server cluster; When the system load is less than the preset load, data correction is performed on the relational database or the time series database based on the difference between the first resource usage matrix and the second resource usage matrix.

[0087] In some embodiments, the data correction module is used to: Performing a difference operation on an element value at any position in the first resource usage matrix and an element value at a corresponding position in the first resource usage matrix; When the difference is zero, it is determined that there is no deviation between the first data and the second data; When the difference is not zero, it is determined that there is a deviation between the first data and the second data, the credibility of the first data and the second data is determined, and data correction is performed on the relational database or the time series database based on the data with higher credibility.

[0088] In some embodiments, the data correction module is further configured to: Generate reports based on data revisions in relational databases or time series databases; Send reports to the monitoring platform of the server cluster.

[0089] Figure 4 It is a schematic diagram of the structure of the server cluster provided by this application, such as Figure 4 As shown, the server cluster 400 includes a server cluster monitoring device 410 and a plurality of server nodes 420; The server cluster monitoring device is connected to the server node and is used to execute the server cluster monitoring method in the above embodiment.

[0090] The server cluster provided in the embodiment of the present application, since the server cluster monitoring device executes the server cluster monitoring method, since the resource usage data of each computing task in the server cluster at each monitoring time is represented by a resource usage matrix, matrix operations can be used to quickly and accurately determine the data deviation between the two databases and the location of the data with the deviation, without the need for manual investigation, thereby improving the data correction efficiency of the server cluster monitoring and improving the accuracy of the resource usage data analysis in the server cluster.

[0091] Figure 5 is a schematic diagram of the structure of the electronic device provided by this application, such as Figure 5 As shown, the electronic device may include: a processor (Processor) 510, a communication interface (Communications Interface) 520, a memory (Memory) 530 and a communication bus (Communications Bus) 540, wherein the processor 510, the communication interface 520, and the memory 530 communicate with each other through the communication bus 540. The processor 510 may call the logic command in the memory 530 to execute the method described in the above embodiment, for example: The resource usage data of each computing task in the server cluster is collected to generate first data, and the first data is stored in a relational database; the resource usage data at each monitoring moment in the server cluster is collected to generate second data, and the second data is stored in a time series database; the first data and the second data are scanned to generate a first resource usage matrix and a second resource usage matrix respectively; the resource usage matrix is ​​used to represent the resource usage data of each computing task in the server cluster at each monitoring moment; based on the difference between the first resource usage matrix and the second resource usage matrix, data correction is performed on the relational database or the time series database.

[0092] In addition, the logical commands in the above-mentioned memory can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several commands to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., which can store program code.

[0093] The processor in the electronic device provided in the embodiment of the present application can call the logic instructions in the memory to implement the above method. Its specific implementation method is consistent with the implementation method of the aforementioned method and can achieve the same beneficial effects, which will not be repeated here.

[0094] An embodiment of the present application also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the methods provided in the above embodiments are implemented.

[0095] Its specific implementation is consistent with the aforementioned method implementation and can achieve the same beneficial effects, so it will not be repeated here.

[0096] An embodiment of the present application provides a computer program product, including a computer program, which implements the above method when executed by a processor.

[0097] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0098] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0099] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A server cluster monitoring method, characterized in that: include: Collect resource usage data of each computing task in the server cluster to generate first data, and store the first data in a relational database; Collect resource usage data at each monitoring moment in the server cluster, generate second data, and store the second data in a time series database; Scanning the first data and the second data to generate a first resource usage matrix and a second resource usage matrix respectively; the resource usage matrix is ​​used to represent resource usage data of each computing task in the server cluster at each monitoring time; Based on the difference between the first resource usage matrix and the second resource usage matrix, data correction is performed on the relational database or the time series database.

2. The server cluster monitoring method according to claim 1, characterized in that: The scanning of the first data and the second data to generate a first resource usage matrix and a second resource usage matrix respectively includes: determining a target time period and a type of resource usage to be monitored during the target time period; Scanning the first data corresponding to the target time period in the relational database to generate a first resource usage matrix corresponding to each resource usage type; Scan the second data corresponding to the target time period in the time series database to generate a second resource usage matrix corresponding to each resource usage type; The rows in the first resource usage matrix and the second resource usage matrix are determined based on computing tasks, the columns are determined based on monitoring moments, and the element values ​​are determined based on resource usage data of any computing task at any monitoring moment.

3. The server cluster monitoring method according to claim 2, characterized in that: The element values ​​in the first resource usage matrix and the second resource usage matrix are determined based on the following steps: Determine the resource usage type corresponding to the element value; Based on the time series database, obtaining resource usage data of each computing task for the resource usage type at each monitoring moment; Determining a resource usage baseline value for each computing task at each monitoring moment based on resource usage data of the resource usage type for each computing task at each monitoring moment; In the case where the resource usage data of any computing task for the resource usage type at any monitoring time is greater than the resource usage reference value, determining the element value corresponding to the any computing task at the any monitoring time as a first preset value; When the resource usage data of any computing task for the resource usage type at any monitoring moment is less than the resource usage reference value, the element value corresponding to the any computing task at the any monitoring moment is determined as a second preset value.

4. The server cluster monitoring method according to claim 1, characterized in that: The performing data correction on the relational database or the time series database based on the difference between the first resource usage matrix and the second resource usage matrix includes: Obtaining the system load of the server cluster; When the system load is less than a preset load, data correction is performed on the relational database or the time series database based on a difference between the first resource usage matrix and the second resource usage matrix.

5. The server cluster monitoring method according to claim 1, characterized in that: The performing data correction on the relational database or the time series database based on the difference between the first resource usage matrix and the second resource usage matrix includes: performing a difference operation on an element value at any position in the first resource usage matrix and an element value at a corresponding position in the first resource usage matrix; When the difference is zero, determining that there is no deviation between the first data and the second data; When the difference is not zero, it is determined that there is a deviation between the first data and the second data, the credibility of the first data and the second data is determined, and data correction is performed on the relational database or the time series database based on the data with higher credibility.

6. The server cluster monitoring method according to any one of claims 1 to 5, characterized in that: After performing data correction on the relational database or the time series database based on the difference between the first resource usage matrix and the second resource usage matrix, the method further includes: Generate a report based on the data correction status of the relational database or the time series database; The report is sent to the monitoring platform of the server cluster.

7. A server cluster monitoring device, characterized in that: include: A first collection module, used to collect resource usage data of each computing task in the server cluster, generate first data, and store the first data in a relational database; A second collection module, used to collect resource usage data at each monitoring moment in the server cluster, generate second data, and store the second data in a time series database; A data scanning module, used to scan the first data and the second data to generate a first resource usage matrix and a second resource usage matrix respectively; the resource usage matrix is ​​used to represent resource usage data of each computing task in the server cluster at each monitoring time; A data correction module is used to correct data on the relational database or the time series database based on the difference between the first resource usage matrix and the second resource usage matrix.

8. A server cluster, characterized in that: It includes a server cluster monitoring device and multiple server nodes; The server cluster monitoring device is connected to the server node and is used to execute the server cluster monitoring method according to any one of claims 1 to 6.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the server cluster monitoring method according to any one of claims 1 to 6 is implemented.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the server cluster monitoring method according to any one of claims 1 to 6 is implemented.

Images