Server resource data processing method and device, equipment and storage medium

Abstract

The invention provides a server resource data processing method, apparatus and device, and a storage medium. The method comprises the steps of obtaining a resource data acquisition request corresponding to a server cluster; the resource data acquisition request comprises acquisition time and an acquisition data category; obtaining original resource index data corresponding to the collection data category from each server through a preset monitoring component based on the collection time; performing data integration on the original resource index data according to a preset data integration strategy to generate composite index data corresponding to each server; and performing corresponding display processing on the composite index data. According to the server resource data processing method, the overall resource condition of the server can be displayed, and the efficiency and accuracy of subsequent data analysis are improved.

Description

Technical Field

[0001] This application relates to the technical field of data processing, and in particular, to a method, device, equipment and storage medium for processing server resource data. Background Art

[0002] HPC (High-Performance Computing) is a computing method that uses large-scale computing resources to solve scientific, engineering, and commercial problems that require a large amount of computing. Slurm (Simple Linux Utility for Resource Management), which is translated into Chinese as a simple Linux utility for resource management, is an open-source software for managing and controlling HPC cluster systems. It can help users efficiently manage computing resources, schedule tasks, allocate computing nodes, and monitor system status, etc. Server nodes in an HPC cluster need to continuously monitor resources in order to understand the performance data of the nodes in real time, optimize task scheduling, and improve system utilization.

[0003] Currently, the Prometheus system can be used to monitor the resource situation of servers. However, the current processing method of the Prometheus system is to monitor and display individual metrics of servers, and it is unable to monitor and display the overall resource situation of servers, resulting in low efficiency and accuracy of data analysis. Summary of the Invention

[0004] This application provides a method, device, equipment and storage medium for processing server resource data to solve the problem that the current monitoring method of server resource data cannot monitor and display the overall resource situation of servers, resulting in low efficiency and accuracy of data analysis.

[0005] In the first aspect of this application, a method for processing server resource data is provided. The server cluster includes multiple servers, and the method includes:

[0006] Obtain a resource data collection request corresponding to the server cluster; the resource data collection request includes: collection time and collection data category;

[0007] Based on the collection time, obtain the original resource metric data corresponding to the collection data category from each server through a preset monitoring component;

[0008] Integrate the original resource metric data according to a preset data integration strategy to generate composite metric data corresponding to each server;

[0009] Perform corresponding display processing on the composite metric data.

[0010] Further, in the method described above, the preset monitoring component is the Prometheus component; a collection component corresponding to the Prometheus component is deployed in each of the servers;

[0011] The obtaining of the original resource metric data corresponding to the collection data category from each of the servers by the preset monitoring component based on the collection time includes:

[0012] Sending a data acquisition request to the collection component in each of the servers by the Prometheus component according to the collection time;

[0013] Receiving the original resource metric data corresponding to the collection data category sent by the collection component; the original resource metric data is collected and generated by the collection component.

[0014] Further, in the method described above, the data integration of the original resource metric data according to the preset data integration strategy to generate composite metric data corresponding to each server includes:

[0015] Performing data preprocessing on the original resource metric data to generate preprocessed original resource metric data;

[0016] Performing aggregation processing on the preprocessed original resource metric data according to the server dimension to generate intermediate resource metric data corresponding to each server;

[0017] Performing data classification and assembly integration on the intermediate resource metric data according to the preset data analysis category and preset data format to generate the composite metric data corresponding to each server.

[0018] Further, in the method described above, the corresponding display processing of the composite metric data includes:

[0019] Sending the composite metric data to the message queue middleware so that the message queue middleware forwards the composite metric data to a display device for corresponding display.

[0020] Further, in the method described above, before sending the composite metric data to the message queue middleware, it further includes:

[0021] Judging whether all the data corresponding to the preset data analysis category in the composite metric data is complete;

[0022] If it is determined that all the data corresponding to the preset data analysis category is complete, then execute the step of sending the composite metric data to the message queue middleware;

[0023] If it is determined that all the data corresponding to the preset data analysis category is incomplete, then sending the composite index data to the message queue middleware includes:

[0024] Sending the composite index data and a notification message indicating that the composite index data is abnormal data to the message queue middleware.

[0025] Further, in the method as described above, the corresponding display processing of the composite index data includes:

[0026] Performing corresponding data extraction on the composite index data to generate index-related data; the index-related data includes a resource index name, a resource index value, a time, and a resource index unit;

[0027] Using a preset drawing component to draw an icon for the index-related data to generate a corresponding index display chart;

[0028] Displaying the index display chart in a preset display area.

[0029] Further, in the method as described above, before obtaining the original resource index data corresponding to the collection data category from each of the servers through a preset monitoring component based on the collection time, it further includes:

[0030] Judging whether it is currently in a task busy state; the task busy state is a state where the current data processing tasks are greater than the maximum load capacity;

[0031] If it is determined that it is not in a task busy state, then execute the step of obtaining the original resource index data corresponding to the collection data category from each of the servers through a preset monitoring component based on the collection time;

[0032] If it is determined that it is in a task busy state, then re-judge whether it is currently in a data collection state after a preset interval time.

[0033] The second aspect of the present application provides a server resource data processing device. The server cluster includes multiple servers, and it includes:

[0034] A first acquisition module, configured to acquire a resource data collection request corresponding to the server cluster; the resource data collection request includes: a collection time and a collection data category;

[0035] A second acquisition module, configured to obtain the original resource index data corresponding to the collection data category from each of the servers through a preset monitoring component based on the collection time;

[0036] An integration module, configured to perform data integration on the original resource index data according to a preset data integration strategy to generate composite index data corresponding to each server;

[0037] A display module for performing corresponding display processing on the composite index data.

[0038] Further, in the device as described above, the preset monitoring component is a Prometheus component; a collection component corresponding to the Prometheus component is deployed in each of the servers;

[0039] The second acquisition module is specifically used for:

[0040] Sending a data acquisition request to the collection component in each of the servers by using the Prometheus component according to the acquisition time; receiving the original resource index data corresponding to the acquired data category sent by the collection component; the original resource index data is generated by the collection component.

[0041] Further, in the device as described above, the integration module is specifically used for:

[0042] Performing data preprocessing on the original resource index data to generate preprocessed original resource index data; performing aggregation processing on the preprocessed original resource index data according to the server dimension to generate intermediate resource index data corresponding to each server; performing data classification and assembly integration on the intermediate resource index data according to a preset data analysis category and a preset data format to generate the composite index data corresponding to each server.

[0043] Further, in the device as described above, the display module is specifically used for:

[0044] Sending the composite index data to a message queue middleware, so that the message queue middleware forwards the composite index data to a display device for corresponding display.

[0045] Further, in the device as described above, the display module is further used for:

[0046] Judging whether all the data corresponding to the preset data analysis category in the composite index data is complete; if it is determined that all the data corresponding to the preset data analysis category is complete, then executing the step of sending the composite index data to the message queue middleware; if it is determined that all the data corresponding to the preset data analysis category is incomplete, then the step of sending the composite index data to the message queue middleware includes: sending the composite index data and a notification message indicating that the composite index data is abnormal data to the message queue middleware.

[0047] Further, in the device as described above, the display module is specifically used for:

[0048] Extract corresponding data from the composite index data to generate index-related data; the index-related data includes resource index names, resource index values, time, and resource index units; use a preset drawing component to draw an icon for the index-related data to generate a corresponding index display chart; display the index display chart in a preset display area.

[0049] Further, for the device described above, the device further includes:

[0050] A judgment module, configured to judge whether it is currently in a task busy state; the task busy state is a state where the current data processing task is greater than the maximum load capacity; if it is determined that it is not in a task busy state, then execute the step of obtaining the original resource index data corresponding to the collected data category from each of the servers through a preset monitoring component based on the collection time; if it is determined that it is in a task busy state, then re-judge whether it is currently in a data collection state after a preset interval.

[0051] A third aspect of the present application provides an electronic device, including: a memory and a processor;

[0052] The memory stores computer execution instructions;

[0053] The processor executes the computer execution instructions stored in the memory to implement the server resource data processing method according to any one of the first aspects.

[0054] A fourth aspect of the present application provides a computer-readable storage medium, in which computer execution instructions are stored, and when the computer execution instructions are executed by a processor, they are used to implement the server resource data processing method according to any one of the first aspects.

[0055] A fifth aspect of the present application provides a computer program product, including a computer program, and when the computer program is executed by a processor, it implements the server resource data processing method according to any one of the first aspects.

[0056] A method, apparatus, device, and storage medium for processing server resource data provided by the present application. The method includes: obtaining a resource data collection request corresponding to the server cluster; the resource data collection request includes: a collection time and a collection data category; based on the collection time, obtaining original resource index data corresponding to the collection data category from each of the servers through a preset monitoring component; performing data integration on the original resource index data according to a preset data integration strategy to generate composite index data corresponding to each server; and performing corresponding display processing on the composite index data. The server resource data processing method of the present application obtains the original resource index data corresponding to the collection data category from each of the servers through a preset monitoring component based on the collection time. At the same time, data integration is performed on the original resource index data according to a preset data integration strategy to generate composite index data corresponding to each server, and corresponding display processing is performed on the composite index data, so as to display the overall resource situation of the server, and improve the efficiency and accuracy of subsequent data analysis. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] The accompanying drawings herein are incorporated into and constitute a part of this specification, showing embodiments consistent with the present application, and together with the specification are used to explain the principles of the present application.

[0058] Figure 1 It is a scenario diagram for implementing the server resource data processing method of the embodiments of the present application;

[0059] Figure 2 It is a flowchart of the server resource data processing method provided by the present application Figure 1 ;

[0060] Figure 3 It is a flowchart of the server resource data processing method provided by the present application Figure 2 ;

[0061] Figure 4 It is an overall flowchart of the server resource data processing method provided by the present application;

[0062] Figure 5 It is a structural diagram of the server resource data processing apparatus provided by the present application;

[0063] Figure 6 It is a structural diagram of the electronic device provided by the present application.

[0064] Through the above accompanying drawings, specific embodiments of the present application have been shown, and there will be more detailed descriptions hereinafter. These drawings and textual descriptions are not intended to limit the scope of the concept of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0065] Exemplary embodiments will be described in detail herein, and examples thereof are shown in the accompanying drawings. When the following description refers to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. On the contrary, they are merely examples of apparatuses and methods consistent with some aspects of the present application as detailed in the appended claims.

[0066] In the technical solutions of the embodiments of the present application, the collection, storage, use, processing, transmission, provision, and disclosure of user personal information and other processing all comply with the provisions of relevant laws and regulations and do not violate public order and good customs.

[0067] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data authorized by the user or fully authorized by all parties. Moreover, the collection, use, and processing of relevant data need to comply with the relevant laws, regulations, and standards of relevant countries and regions, and corresponding operation entrances are provided for users to choose to authorize or reject.

[0068] The technical solutions of the present application will be described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below with reference to the accompanying drawings.

[0069] To clearly understand the technical solutions of the present application, the solutions of the prior art will be introduced in detail first. Currently, the server nodes in the HPC cluster need to continuously monitor resources in order to understand the performance data of the nodes in real time, optimize task scheduling, and improve system utilization.

[0070] In terms of monitoring the server host resources, Prometheus and the collection component node-exporter can be used to monitor the resource status of the server. The specific implementation method is to deploy node-exporter on each server node to obtain the resource metric status of the affiliated server, and Prometheus is used to obtain the server host resource metric data collected by each node-exporter. However, the current processing method of the Prometheus system is to monitor and display individual server metrics, and it is unable to monitor and display the overall resource status of the server, resulting in the efficiency and accuracy of data analysis.

[0071] Therefore, aiming at the problem that the monitoring method of server resource data in the prior art cannot monitor and display the overall server resource situation, resulting in low efficiency and accuracy of data analysis, the inventor found in the research that the original resource index data corresponding to each server can be obtained through a monitoring component. At the same time, the original resource index data is integrated to generate composite index data, and the composite index data is subjected to corresponding display processing, so as to realize the monitoring and display of the overall server resource situation and improve the efficiency and accuracy of data analysis.

[0072] Specifically, obtain the resource data collection request corresponding to the server cluster. The resource data collection request includes: collection time and collection data category. Based on the collection time, obtain the original resource index data corresponding to the collection data category from each server through a preset monitoring component. Integrate the original resource index data according to a preset data integration strategy to generate the composite index data corresponding to each server. Perform corresponding display processing on the composite index data.

[0073] The server resource data processing method of the present application obtains the original resource index data corresponding to the collection data category from each server through a preset monitoring component based on the collection time. At the same time, the original resource index data is integrated according to a preset data integration strategy to generate the composite index data corresponding to each server, and corresponding display processing is performed on the composite index data, so as to display the overall server resource situation and improve the efficiency and accuracy of subsequent data analysis.

[0074] Based on the above creative discovery, the inventor proposed the technical solution of the present application.

[0075] Next, the application scenario of the server resource data processing method provided by the embodiments of the present application will be introduced. As Figure 1 shown, where 1 is an electronic device, 2 is the first server, and 3 is the second server. The network architecture of the application scenario corresponding to the server resource data processing method provided by the embodiments of the present application includes: an electronic device 1, a first server 2, and a second server 3. Among them, the first server 2 and the second server 3 belong to the servers in the server cluster. At the same time, in other application scenarios, there may be 3 or more servers. In this embodiment, only two servers are used as an example for illustration.

[0076] Exemplarily, when server resource data monitoring, display, etc. need to be performed, the electronic device 1 performs the following processing:

[0077] ① Obtain the resource data collection request corresponding to the server cluster (such as the first server 2 and the second server 3). The resource data collection request includes: collection time and collection data category.

[0078] ②Based on the collection time, obtain the original resource metric data corresponding to the collection data category from the first server 2 and the second server 3 through a preset monitoring component.

[0079] ③Integrate the original resource metric data according to a preset data integration strategy to generate composite metric data corresponding to the first server 2 and the second server 3.

[0080] ④Perform corresponding display processing on the composite metric data.

[0081] The display processing can be directly displayed in the electronic device 1 or displayed in other terminal devices.

[0082] Users can view the middleware operation status according to the monitoring view, or analyze the monitoring view through other applications to determine the specific problems of the middleware.

[0083] Next, the embodiments of the present application will be introduced in conjunction with the accompanying drawings of the specification.

[0084] Figure 2 It is a flowchart of the server resource data processing method provided by the present application Figure 1 as Figure 2 shown. In this embodiment, the execution subject of the embodiment of the present application is a server resource data processing device, and this server resource data processing device can be integrated in an electronic device. Then, the server resource data processing method provided in this embodiment includes the following steps:

[0085] Step S101, obtain a resource data collection request corresponding to the server cluster. The resource data collection request includes: collection time and collection data category.

[0086] In this embodiment, the collection time can be a collection cycle time, a one-time collection time, etc. The collection data category can be divided according to hardware, such as processor data, memory data, etc. The acquisition method can be generated based on the user's collection request operation, or a resource data collection request can be received from other devices. This embodiment does not make any limitations in this regard.

[0087] Step S102, based on the collection time, obtain the original resource metric data corresponding to the collection data category from each server through a preset monitoring component.

[0088] In this embodiment, the preset monitoring component can be Prometheus, and the preset monitoring component can collect data through a collection component preset in the server.

[0089] Step S103, integrate the original resource metric data according to a preset data integration strategy to generate composite metric data corresponding to each server.

[0090] In this embodiment, the preset data integration strategy can be data integration based on the server dimension, or multiple data integrations according to the server and monitoring targets, etc. The composite index data is the data that is convenient for subsequent display after the original resource index data is processed.

[0091] Step S104: Perform corresponding display processing on the composite index data.

[0092] The display method can be to display on a preset display page, or to send it to the user terminal for display.

[0093] A server resource data processing method provided by an embodiment of the present application includes: obtaining a resource data collection request corresponding to a server cluster. The resource data collection request includes: collection time and collection data category. Based on the collection time, obtain the original resource index data corresponding to the collection data category from each server through a preset monitoring component. Integrate the original resource index data according to the preset data integration strategy to generate composite index data corresponding to each server. Perform corresponding display processing on the composite index data.

[0094] In the server resource data processing method of the present application, based on the collection time, the original resource index data corresponding to the collection data category is obtained from each server through a preset monitoring component. At the same time, the original resource index data is integrated according to the preset data integration strategy to generate composite index data corresponding to each server, and corresponding display processing is performed on the composite index data, so that the overall resource situation of the server can be displayed, and the efficiency and accuracy of subsequent data analysis can be improved.

[0095] Figure 3 It is a flow diagram of the server resource data processing method provided by the present application Figure 2 , as Figure 3 shown, the server resource data processing method provided in this embodiment is a further refinement based on the server resource data processing method provided in the previous embodiment of the present application. In this embodiment, the preset monitoring component is the Prometheus component, and a collection component corresponding to the Prometheus component is deployed in each server. Then the server resource data processing method provided in this embodiment includes the following steps.

[0096] Step S201: Obtain a resource data collection request corresponding to a server cluster.

[0097] In this embodiment, the implementation manner of S201 is similar to that of S101, and will not be described in detail here.

[0098] Step S202: Send a data acquisition request to the collection components in each server using the Prometheus component according to the collection time.

[0099] In this embodiment, the collection component can adopt node-exporter. The role of the exporter is to expose the endpoints for collecting monitoring data to the Prometheus service in the form of an HTTP (Hypertext Transfer Protocol) service. Then, the Prometheus service can obtain the monitoring data to be collected by accessing the endpoints provided by the Exporter.

[0100] Optionally, in this embodiment, before S202, it is also possible to further determine whether it is currently in a task-busy state, specifically as follows:

[0101] Determine whether it is currently in a task-busy state. The task-busy state is a state where the current data processing tasks are greater than the maximum load capacity.

[0102] If it is determined that it is not in a task-busy state, then execute S202.

[0103] If it is determined that it is in a task-busy state, then re-determine whether it is currently in a data collection state after a preset interval.

[0104] The preset interval can be set according to actual application requirements. For example, it can be set to 1 minute, 5 minutes, 30 seconds, etc. In this embodiment, by pre-determining whether it is in a task-busy state, it is possible to avoid continuing to execute data processing tasks when the load capacity of data processing tasks is too large, thereby improving the overall data processing efficiency.

[0105] Step S203: Receive the original resource metric data corresponding to the data collection category sent by the collection component. The original resource metric data is generated by the collection component.

[0106] In this embodiment, when the collection component collects data, it can collect data at a preset collection time interval. For example, it can collect data at an interval of 30 seconds. This collection interval is different from the collection time in the resource data collection request, and the collection time can include the time of multiple collection intervals.

[0107] Step S204: Perform data preprocessing on the original resource metric data to generate preprocessed original resource metric data.

[0108] In this embodiment, data preprocessing includes data cleaning, data filtering, etc. The original resource metric data is unprocessed data with a complex data structure and some dirty data and invalid values. Through preprocessing, the data format can be standardized, the code can be simplified, and the code level can be made more simple and maintainable.

[0109] Step S205: Aggregate the preprocessed original resource metric data according to the server dimension to generate intermediate resource metric data corresponding to each server.

[0110] In this embodiment, the server dimension refers to aggregating according to the server to which the original resource metric data belongs. Exemplarily, if the servers include server a and server b, then all the original resource metric data is aggregated according to server a and server b to generate intermediate resource metric data corresponding to server a and server b.

[0111] Step S206: Classify and assemble the intermediate resource metric data according to the preset data analysis categories and preset data formats to generate composite metric data corresponding to each server.

[0112] In this embodiment, the preset data analysis categories may be different from or the same as the data collection categories. The preset data analysis categories may be sub-categories in the data collection categories. For example, if the data collection categories are divided into processor data and memory data, the preset data analysis categories may be processor occupancy, memory occupancy, etc.

[0113] The preset data formats are used to assemble and integrate the intermediate resource metric data. For example, when analyzing processor data, the total number of processor cores, processor occupancy, etc. can be analyzed. At this time, the processor occupancy and the total number of processor cores at different times need to be spliced according to the preset data formats to facilitate subsequent display processing.

[0114] Step S207: Send the composite metric data to the message queue middleware so that the message queue middleware forwards the composite metric data to the display device for corresponding display.

[0115] By sending the composite metric data to the message queue middleware, the electronic device and the display device can be decoupled. At the same time, the asynchronous communication function is realized, and the processing and display efficiency of the composite metric data is improved.

[0116] Optionally, in this embodiment, before S207, it is also possible to determine whether the data is complete, specifically as follows:

[0117] Determine whether all the data corresponding to the preset data analysis categories in the composite metric data is complete.

[0118] If it is determined that all the data corresponding to the preset data analysis categories is complete, then execute S207.

[0119] If it is determined that all the data corresponding to the preset data analysis categories is incomplete, then send the composite metric data and a notification message indicating that the composite metric data is abnormal data to the message queue middleware.

[0120] In this embodiment, by determining whether the data in the composite index data is complete, it is possible to determine whether there is an anomaly in the composite index data. For example, if the memory occupancy data for a certain time is missing in the composite index data, it means that the composite index data is abnormal. The composite index data and a notification message indicating that the composite index data is abnormal data are sent to the message queue middleware. Subsequently, the server can be checked based on the composite index data to determine the specific anomaly problem.

[0121] Optionally, in this embodiment, the method of performing corresponding display processing on the composite index data can also be to display it directly through the electronic device, specifically as follows:

[0122] Extract corresponding data from the composite index data to generate index-related data. The index-related data includes the resource index name, resource index value, time, and resource index unit.

[0123] Use a preset drawing component to draw an icon for the index-related data to generate a corresponding index display chart.

[0124] Display the index display chart in a preset display area.

[0125] The preset drawing component can use common drawing components and generate a corresponding index display chart based on the resource index name, resource index value, time, and resource index unit, so as to display the data indicators of the server through the index display chart.

[0126] To further illustrate the server resource data processing method of this embodiment in detail, the following will be further described in conjunction with the accompanying drawings. The overall architecture is as Figure 4 shown. The cluster server includes a management node (which can also be called the master node) and computing nodes (which can also be called slave nodes). Each node is equipped with a Node-exporter collection component. The monitoring component uses Prometheus, and the data processing device is the aforementioned electronic device, which adopts the method of a main working copy and a slave standby copy. The data analysis and display device is used for data analysis and corresponding display.

[0127] The overall process of this embodiment is as follows:

[0128] 1. Deploy the node-exporter collection component on each management node and computing node of the HPC cluster respectively to collect the original resource index data of the affiliated server (including memory occupancy, total memory, total number of processor cores, processor occupancy, etc.).

[0129] 2. Use the Prometheus monitoring component to collect the original resource index data of each node's node-exporter collection component.

[0130] 3. The data processing device deployed in the multi-copy master-slave cluster mode periodically requests Prometheus to obtain the original resource monitoring metrics of the HPC cluster server.

[0131] 4. The data processing device re-integrates the original resource metric data in server dimension according to the configured preset data integration strategy to generate composite metric data.

[0132] 5. The data processing device pushes the integrated composite metric data to the message queue.

[0133] 6. The data analysis and display device obtains the composite metric data from the message queue for display and analysis.

[0134] Among them, the data processing device is deployed with an odd number of copies (3 copies here). When starting up, it obtains the preset data integration strategy, collection frequency period, etc. of the server original resource metric data from the configuration and loads them into the memory. After starting up, the copies communicate with each other, and a main working copy and multiple slave standby copies are selected. The main working copy will execute the subsequent data integration and pushing tasks. The slave standby copies will continuously detect the status of the main working copy. When the main working copy service is abnormal, a new main working copy will be re-selected from the slave standby copies to execute the tasks.

[0135] In this embodiment, through the multi-copy master-slave deployment mode, the continuous stability and high availability of the data integration technical solution corresponding to the data processing device are ensured.

[0136] At the same time, this technical solution decouples and splits the specific business functions, improving the scalability and stability of the system. Through this solution, the implementation process of monitoring the HPC cluster server nodes can be simplified, facilitating users' analysis and comparison of the overall metric data, thereby improving the efficiency of data analysis and decision-making.

[0137] Figure 5 It is a schematic structural diagram of the server resource data processing device provided by this application. As Figure 5 shown, in this embodiment, the server resource data processing device 300 can be set in an electronic device. The server cluster includes multiple servers. The server resource data processing device 300 includes:

[0138] The first acquisition module 301 is used to acquire the resource data acquisition request corresponding to the server cluster. The resource data acquisition request includes: acquisition time and acquisition data category.

[0139] The second acquisition module 302 is used to acquire the original resource metric data corresponding to the acquisition data category from each server based on the acquisition time through a preset monitoring component.

[0140] An integration module 303 for integrating the original resource metric data according to a preset data integration strategy to generate composite metric data corresponding to each server.

[0141] A display module 304 for performing corresponding display processing on the composite metric data.

[0142] The server resource data processing device provided in this embodiment can execute Figure 2 the technical solution of the method embodiment shown, and its implementation principle and technical effect are similar to Figure 2 the method embodiment shown, and will not be elaborated here one by one.

[0143] Based on the server resource data processing device provided in the previous embodiment, the server resource data processing device provided in this application further refines the server resource data processing device. The server resource data processing device 300 includes:

[0144] Optionally, in this embodiment, the preset monitoring component is the Prometheus component. A collection component corresponding to the Prometheus component is deployed in each server.

[0145] The second acquisition module 302 is specifically used for:

[0146] Sending a data acquisition request to the collection components in each server using the Prometheus component according to the acquisition time. Receiving the original resource metric data corresponding to the acquired data category sent by the collection component. The original resource metric data is generated by the collection component.

[0147] Optionally, in this embodiment, the integration module 303 is specifically used for:

[0148] Performing data preprocessing on the original resource metric data to generate preprocessed original resource metric data. Aggregating the preprocessed original resource metric data according to the server dimension to generate intermediate resource metric data corresponding to each server. Classifying and assembling and integrating the intermediate resource metric data according to a preset data analysis category and a preset data format to generate composite metric data corresponding to each server.

[0149] Optionally, in this embodiment, the display module 304 is specifically used for:

[0150] Sending the composite metric data to the message queue middleware so that the message queue middleware forwards the composite metric data to the display device for corresponding display.

[0151] Optionally, in this embodiment, the display module 304 is further used for:

[0152] Determine whether all the data corresponding to the preset data analysis category in the composite index data is complete. If it is determined that all the data corresponding to the preset data analysis category is complete, then perform the step of sending the composite index data to the message queue middleware. If it is determined that all the data corresponding to the preset data analysis category is incomplete, then send the composite index data to the message queue middleware, including: sending the composite index data and the notification information that the composite index data is abnormal data to the message queue middleware.

[0153] Optionally, in this embodiment, the display module 304 is specifically configured to:

[0154] Extract corresponding data from the composite index data to generate index-related data. The index-related data includes the resource index name, resource index value, time, and resource index unit. Use a preset drawing component to draw an icon for the index-related data to generate a corresponding index display graph. Display the index display graph in a preset display area.

[0155] Optionally, in this embodiment, the server resource data processing device 300 further includes:

[0156] A judgment module, configured to judge whether it is currently in a task busy state. The task busy state is a state where the current data processing tasks being processed are greater than the maximum load capacity. If it is determined that it is not in a task busy state, then perform the step of obtaining the original resource index data corresponding to the data collection category from each server based on the collection time through a preset monitoring component. If it is determined that it is in a task busy state, then re-judge whether it is currently in a data collection state after a preset interval time.

[0157] The server resource data processing device provided in this embodiment can execute Figure 2 - Figure 3 the technical solution of the method embodiment shown, and its implementation principle and technical effects are similar to those of Figure 2 - Figure 3 the method embodiment shown, and will not be elaborated here one by one.

[0158] According to the embodiments of the present application, the present application further provides an electronic device, a computer-readable storage medium, and a computer program product.

[0159] As Figure 6 shown, Figure 6 is a schematic structural diagram of the electronic device provided by the present application. The electronic device is intended to be various forms of digital computers, such as, a laptop computer, a desktop computer, a workbench, a personal digital assistant, a blade server, a mainframe computer, and other suitable computers. The components shown herein, their connections and relationships, and their functions are only examples and are not intended to limit the implementation of the present application described herein and / or required.

[0160] As Figure 6As shown, the electronic device includes: a processor 401 and a memory 402. Each component is interconnected using different buses and can be installed on a common motherboard or in other ways as needed. The processor can process instructions executed within the electronic device.

[0161] The memory 402 is the non-transitory computer-readable storage medium provided by this application. Among them, the memory stores instructions executable by at least one processor, so that at least one processor executes the server resource data processing method provided by this application. The non-transitory computer-readable storage medium of this application stores computer instructions, and these computer instructions are used to cause a computer to execute the server resource data processing method provided by this application.

[0162] As a non-transitory computer-readable storage medium, the memory 402 can be used to store non-transitory software programs, non-transitory computer-executable programs, and modules, such as the program instructions / modules corresponding to the server resource data processing method in the embodiments of this application (for example, the first acquisition module 301, the second acquisition module 302, the integration module 303, and the display module 304 shown in the appendix). Figure 5 By running the non-transitory software programs, instructions, and modules stored in the memory 402, the processor 401 thus executes various functional applications of the electronic device and server resource data processing, that is, implements the server resource data processing method in the above method embodiments.

[0163] At the same time, this embodiment also provides a computer product. When the instructions in this computer product are executed by the processor of the electronic device, the electronic device can execute the server resource data processing method in the above embodiments.

[0164] Those skilled in the art will readily think of other implementation schemes of the embodiments of this application after considering the specification and practicing the invention disclosed herein. This application aims to cover any variations, uses, or adaptive changes of the embodiments of this application. These variations, uses, or adaptive changes follow the general principles of the embodiments of this application and include the common general knowledge or conventional technical means in the technical field not disclosed in the embodiments of this application.

[0165] It should be understood that the embodiments of this application are not limited to the exact structures already described and shown in the drawings, and various modifications and changes can be made without departing from its scope. The scope of the embodiments of this application is only limited by the appended claims.

Claims

1. A method for processing server resource data, where the server cluster includes multiple servers, Characterized in that, It includes: Obtain a resource data collection request corresponding to the server cluster; The resource data collection request includes: collection time and collection data category; Based on the collection time, obtain the original resource index data corresponding to the collection data category from each server through a preset monitoring component; Integrate the original resource index data according to a preset data integration strategy to generate composite index data corresponding to each server; Perform corresponding display processing on the composite index data.

2. The method according to claim 1, Characterized in that, The preset monitoring component is the Prometheus component; a collection component corresponding to the Prometheus component is deployed in each server; The step of obtaining the original resource index data corresponding to the collection data category from each server through a preset monitoring component based on the collection time includes: According to the collection time, use the Prometheus component to send a data acquisition request to the collection component in each server; Receive the original resource index data corresponding to the collection data category sent by the collection component; the original resource index data is collected and generated by the collection component.

3. The method according to claim 1, Characterized in that, The step of integrating the original resource index data according to a preset data integration strategy to generate composite index data corresponding to each server includes: Perform data preprocessing on the original resource index data to generate preprocessed original resource index data; Aggregate the preprocessed original resource index data according to the server dimension to generate intermediate resource index data corresponding to each server; Classify and assemble and integrate the intermediate resource index data according to a preset data analysis category and a preset data format to generate the composite index data corresponding to each server.

4. The method according to claim 3, Characterized in that, The step of performing corresponding display processing on the composite index data includes: Send the composite index data to a message queue middleware, so that the message queue middleware forwards the composite index data to a display device for corresponding display.

5. The method according to claim 4, Characterized in that, Before sending the composite index data to the message queue middleware, it further includes: Judge whether all the data corresponding to the preset data analysis category in the composite index data is complete; If it is determined that all the data corresponding to the preset data analysis category is complete, then execute the step of sending the composite index data to the message queue middleware; If it is determined that all the data corresponding to the preset data analysis category is incomplete, then the step of sending the composite index data to the message queue middleware includes: Send the composite index data and a notification message indicating that the composite index data is abnormal data to the message queue middleware.

6. The method according to claim 3, Characterized in that, The step of performing corresponding display processing on the composite index data includes: Extract corresponding data from the composite index data to generate index-related data; the index-related data includes resource index name, resource index value, time, and resource index unit; Use a preset plotting component to draw an icon for the index-related data to generate a corresponding index display chart; Display the index display chart in a preset display area.

7. The method according to any one of claims 1 to 6, characterized in that, before obtaining the original resource index data corresponding to the collection data category from each of the servers based on the collection time through a preset monitoring component, further comprising: judging whether it is currently in a task busy state; the task busy state is a state where the current data processing task is greater than the maximum load capacity; if it is determined that it is not in a task busy state, then execute the step of obtaining the original resource index data corresponding to the collection data category from each of the servers based on the collection time through a preset monitoring component; if it is determined that it is in a task busy state, then re-judge whether it is currently in a data collection state after a preset interval time.

8. A server resource data processing device, the server cluster includes multiple servers, characterized in that, comprising: a first obtaining module, configured to obtain a resource data collection request corresponding to the server cluster; the resource data collection request includes: collection time and collection data category; a second obtaining module, configured to obtain the original resource index data corresponding to the collection data category from each of the servers based on the collection time through a preset monitoring component; an integration module, configured to integrate the original resource index data according to a preset data integration strategy to generate composite index data corresponding to each server; a display module, configured to perform corresponding display processing on the composite index data.

9. An electronic device, characterized in that, comprising: a memory and a processor; the memory stores computer execution instructions; the processor executes the computer execution instructions stored in the memory to implement the server resource data processing method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, the computer-readable storage medium stores computer execution instructions, and when the computer execution instructions are executed by a processor, they are used to implement the server resource data processing method according to any one of claims 1 to 7.

Images