Method for monitoring a baseboard management controller process, electronic device

Abstract

The application discloses a kind of monitoring method of substrate management controller process, electronic equipment, it is related to server technical field, including: by monitoring the file handle quantity of multiple sampling time analysis file handle growth trend, and the correlation intensity between file handle growth trend and central processing unit use rate, according to correlation intensity, the file handle quantity of multiple sampling time and central processing unit use rate judges substrate management controller process exception, can improve the accuracy of process exception judgment, also can avoid in a certain extent because sampling interval is too long and misses short time inside large amplitude or sudden increase handle leak problem;In the case where substrate management controller process exception is determined, according to the context information of substrate management controller process, the specific abnormal code path is located, reduces the time of investigation, and does not need to stop or restart service, does not affect business continuity.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of servers, and particularly relates to a monitoring method for a baseboard management controller process and an electronic device. BACKGROUND

[0002] In a baseboard management controller (BMC) system, part of the background service processes may continuously grow in file descriptors due to resource leakage (such as handle leakage) after long-time running, finally exhaust system resources, and cause service unavailability or crash. However, the related art can only take a timing snapshot of the file descriptor directory of the current baseboard management controller process, and cannot timely find the handle growth anomaly, nor can it perform code-level leakage positioning. SUMMARY

[0003] The present application provides a monitoring method for a baseboard management controller process and an electronic device, to at least solve the problem that the related art cannot timely find the handle growth anomaly, nor can it perform code-level leakage positioning.

[0004] The present application provides a monitoring method for a baseboard management controller process, applied to a baseboard management controller, comprising: acquiring central processor usage and the number of file handles in a process file descriptor directory at multiple collection moments; determining file handle growth information at the multiple collection moments based on the number of file handles in the process file descriptor directory at the multiple collection moments; determining the correlation strength between the file handle growth information and the central processor usage based on the file handle growth information at the multiple collection moments and the central processor usage; in the case of determining that the baseboard management controller process is abnormal according to the correlation strength, the file handle growth information at the multiple collection moments and the central processor usage, determining an abnormal code path based on context information of the baseboard management controller process, to monitor the baseboard management controller process.

[0005] The present application also provides an electronic device, comprising a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the foregoing monitoring method for a baseboard management controller process when executing the computer program.

[0006] The application analyzes the growth trend of file handles by monitoring the number of file handles at multiple sampling time points, and analyzes the correlation strength between the growth trend of file handles and the central processor usage rate. According to the correlation strength, the number of file handles at multiple sampling time points and the central processor usage rate, the abnormality of the baseboard management controller process is judged, which can improve the accuracy of process abnormality judgment, and can also avoid missing the handle leakage problem of large amplitude or sudden increase in a short time due to too long sampling interval; in the case of determining the abnormality of the baseboard management controller process according to the file handle growth information, the specific abnormal code path is located according to the context information of the baseboard management controller process, the troubleshooting time is reduced, and the service does not need to be stopped or restarted, which does not affect the business continuity. BRIEF DESCRIPTION OF DRAWINGS

[0007] In order to more clearly illustrate the embodiments of the present application, the drawings needed in the embodiments will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0008] Figure 1 A schematic diagram of a monitoring method of a baseboard management controller process according to some embodiments of the present application;

[0009] Figure 2 A flowchart of a monitoring method of a baseboard management controller process according to some embodiments of the present application;

[0010] Figure 3 A block schematic diagram of a monitoring system of a baseboard management controller process according to some embodiments of the present application;

[0011] Figure 4 A block schematic diagram of a monitoring system of a baseboard management controller process according to some embodiments of the present application;

[0012] Figure 5 A block schematic diagram of an electronic device according to some embodiments of the present application. DETAILED DESCRIPTION

[0013] The technical solutions in the embodiments of the present application will be described clearly and completely in combination with the drawings in the embodiments of the present application. Obviously, the described embodiments are only some embodiments of the present application, not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the protection scope of the present application.

[0014] It should be noted that in the description of the present application, the terms "comprising", "containing" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such a process, method, article or device. The terms "first", "second", and the like in the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. In order for those skilled in the art to better understand the present application, the following will further describe the present application in detail in combination with the drawings and specific embodiments. In combination with the specific application environment architecture or specific hardware architecture on which the execution of the monitoring method of the substrate management controller process depends, the specific application environment architecture or specific hardware architecture is described here.

[0015] The monitoring method of the substrate management controller process, the electronic device of the embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0016] With the continuous expansion of the scale of mass servers and edge devices, system reliability and availability have become the core requirements in data center and industrial control scenarios. As the "doorkeeper" of the server platform, BMC undertakes key functions such as hardware monitoring, firmware management, remote operation and maintenance. In recent years, the open source firmware project OpenBMC based on Linux (an operating system), D-Bus (Desktop Bus) and Redfish (an open standard developed by Distributed Management Task Force) standards has developed rapidly and been widely used, providing an open and extensible underlying platform for centralized management and intelligent operation and maintenance of heterogeneous hardware. The original BMC is mostly based on IPMI (Intelligent Platform Management Interface), which can only monitor a small number of hardware indicators such as temperature, voltage and fan rate of the server, and then the industry promotes the Redfish standard, which uses RESTful API+JSON (Representational State Transfer Application Programming Interface+JavaScript Object Notation) format to replace IPMI, expands the function and is easy to integrate with the upper layer cloud platform. With the popularity of containerization and microservices, time series databases and visualization platforms have emerged, realizing multi-dimensional and customizable index collection and display. However, process-level resource leaks (especially the continuous growth of file handles) still rely on general monitoring Agent (a kind of monitoring agent) for periodic sampling.

[0017] For example, in the related art, an external script or a lightweight monitoring Agent, such as a collectd (an open-source system performance data collection tool) plug-in, is usually deployed to take a timed snapshot of the file descriptor directory of the baseboard management controller process and write the result to a log or send it to a monitoring system. Referring to Figure 1 , the method comprises the following steps:

[0018] First, a sampling script, such as a cron (a kind of timed task scheduling tool) of an operating system or a lightweight monitoring Agent, takes a timed discrete snapshot (for example, a discrete snapshot every 5 minutes) of the file descriptor directory (for example, / proc / [pid] / fd) of the baseboard management controller process; then, the number of entries in the / proc / [pid] / fd directory, that is, the total number of file descriptors opened by the current process, is read; then, the total number of file descriptors opened by the current process is written to a local log or sent to a time series database through an HTTP Push (client push) or a Grafana Agent (a kind of monitoring proxy) interface; finally, the total number of file descriptors opened by the current process is compared with a preset threshold value, and if the total number of file descriptors exceeds the preset threshold value, an email is sent or a notification is made through an SNMP Trap (Simple Network Management Protocol Trap), an Alertmanager (alert manager).

[0019] However, the sampling interval of the number of file handles is fixed and long, which makes it difficult to capture a large number of handle leaks in a short time or a sudden increase in a short time, and the manually set threshold value is difficult to adapt to different modules or different running stages, which is likely to cause false positives or false negatives, and thus cannot timely discover abnormal trends of handle growth and cannot locate the code-level leak.

[0020] Based on this, the application analyzes the growth trend of the number of file handles at multiple sampling times and analyzes the correlation strength between the growth trend of the number of file handles and the central processor usage rate, determines the abnormality of the baseboard management controller process according to the correlation strength, the number of file handles at multiple sampling times and the central processor usage rate, which can improve the accuracy of process abnormality determination and also avoid missing a large number of handle leaks in a short time or a sudden increase in a short time due to a long sampling interval; in the case of determining the abnormality of the baseboard management controller process, the specific abnormal code path is located according to the context information of the baseboard management controller process, which reduces the troubleshooting time and does not need to stop or restart the service, without affecting business continuity.

[0021] Figure 2 A flowchart of a monitoring method for a baseboard management controller process according to some embodiments of the application. Referring to Figure 2The monitoring method of the substrate management controller process can include the following steps:

[0022] S110, acquiring the central processor usage and the number of file handles in the process file descriptor directory at multiple collection time points.

[0023] Specifically, a timer, such as an Asio timer, can be used to periodically perform Linux file I / O to collect the number of file handles in the process file descriptor directory at multiple collection time points. For example, a user can select a substrate management controller process identifier, such as a process name or a PID (Process Identifier), in a BMC Web UI (Baseboard Management Controller Web User Interface) and configure a sampling period (such as 1 second) for the selected substrate management controller process. After the configuration is successful, the timer is started, and the number of file handles in the process file descriptor directory is read every 1 second, and the timestamp is recorded, and the number of file handles and the timestamp are stored in a preset time series database, such as InfluxDB (Influx Database) or lightweight SQLite (a type of database). Correspondingly, a system command, such as top, can be used to poll the central processor usage at the corresponding collection time point.

[0024] In this way, the configurable second-level sampling period can monitor the number of file handles in the process file descriptor directory in real time, so that the sudden increase in leakage can also be accurately captured, and the risk of missing detection of the snapshot can be avoided to some extent, thereby effectively avoiding system call failures and process crashes caused by rapid exceedance of the handle number limit.

[0025] S120, determining file handle growth information at multiple collection time points based on the number of file handles in the process file descriptor directory at the multiple collection time points.

[0026] Specifically, after the number of file handles count_i in the process file descriptor directory at the current collection time point is stored in the database, the number of file handles in the process file descriptor directory at the current collection time point and the number of file handles in the process file descriptor directory at the previous collection time point can be used to determine the file handle growth information at the collection time point. For example, the number of file handles in the process file descriptor directory at the current collection time point and the number of file handles in the process file descriptor directory at the previous collection time point can be input into a preset formula to output the file handle growth rate at the current collection time point, and the file handle growth information at multiple collection time points can be determined.

[0027] S130, determine the correlation strength between the file handle growth information and the central processor usage rate based on the file handle growth information and the central processor usage rate of the plurality of collection time points.

[0028] Specifically, to determine the correlation strength between the file handle growth information and the central processor usage rate, after a preset number (e.g., 5) of file handle growth information and central processor usage rates are collected, the correlation strength between the file handle growth information and the central processor usage rate is determined based on the preset number of file handle growth information and central processor usage rates. For example, the file handle growth information and the central processor usage rate of 5 collection time points can be input into a preset formula to output the correlation strength between the file handle growth information and the central processor usage rate.

[0029] S140, in the case where it is determined that the baseboard management controller process is abnormal according to the correlation strength, the file handle growth information and the central processor usage rate of the plurality of collection time points, determine the abnormal code path based on the context information of the baseboard management controller process to monitor the baseboard management controller process.

[0030] Specifically, after the correlation strength, the file handle growth information and the central processor usage rate of the plurality of collection time points are determined, it can be determined whether the baseboard management controller process is abnormal according to the correlation strength, the file handle growth information and the central processor usage rate of the plurality of collection time points, for example, the average of the file handle growth information of the plurality of collection time points, the average of the central processor usage rate of the plurality of collection time points, in the case where a plurality of the correlation strength, the average of the file handle growth information of the plurality of collection time points and the average of the central processor usage rate of the plurality of collection time points are greater than the corresponding threshold value, it is determined that the baseboard management controller process is abnormal.

[0031] In the case where it is determined that the baseboard management controller process is abnormal, the snapshot collection interface is called to collect the context information of the baseboard management controller process, for example, the call stack table and the file descriptor list of the baseboard management controller process, and the abnormal code path is determined based on the call stack table and the file descriptor list of the baseboard management controller process. For example, based on the call stack table, it is determined that the function being executed is performing a file path opening operation, and based on the file descriptor list, it is determined that the number of opening times of a certain file path is continuously increasing, so the function chain in the call stack table can be determined as the abnormal code path.

[0032] The application analyzes the growth trend of file handles by monitoring the number of file handles at multiple sampling time points, analyzes the correlation strength between the growth trend of file handles and the central processor usage rate, determines the abnormality of the baseboard management controller process according to the correlation strength, the number of file handles at multiple sampling time points and the central processor usage rate, can improve the accuracy of process abnormality determination, and can also avoid missing the handle leakage problem of large amplitude or sudden increase in a short time due to too long sampling interval; in the case of determining the abnormality of the baseboard management controller process, the specific abnormal code path is located according to the context information of the baseboard management controller process, the troubleshooting time is reduced, and the service does not need to be stopped or restarted, which does not affect the business continuity. In some embodiments, the file handle growth information includes a file handle growth rate, the correlation strength between the file handle growth information and the central processor usage rate is determined based on the file handle growth information and the central processor usage rate at multiple collection time points, including: calculating the mean value of the file handle growth rate and the mean value of the central processor usage rate respectively; based on the difference between the file handle growth rate at multiple collection time points and the mean value of the file handle growth rate, the deviation of the file handle growth rate at multiple collection time points is determined; based on the difference between the central processor usage rate at multiple collection time points and the mean value of the central processor usage rate, the deviation of the central processor usage rate at multiple collection time points is determined; based on the deviation of the file handle growth rate at multiple collection time points and the deviation of the central processor usage rate at multiple collection time points, the correlation strength between the file handle growth information and the central processor usage rate is determined.

[0033] In some embodiments, based on the deviation of the file handle growth rate at multiple collection time points and the deviation of the central processor usage rate at multiple collection time points, the correlation strength between the file handle growth information and the central processor usage rate is determined, including: determining the product between the square root of the sum of squares of the deviation of the file handle growth rate at multiple collection time points and the square root of the sum of squares of the deviation of the central processor usage rate at multiple collection time points as a first product; determining the sum of products between the deviation of the file handle growth rate at multiple collection time points and the deviation of the central processor usage rate at the corresponding collection time as a first sum; based on the ratio between the first sum and the first product, the correlation strength between the file handle growth information and the central processor usage rate is determined.

[0034] In the embodiments of the application, the correlation strength between the file handle growth information and the central processor usage rate is determined based on the file handle growth information and the central processor usage rate at five collection time points, but this is not a limitation of the application, that is, the correlation strength between the file handle growth information and the central processor usage rate is calculated once for every five file handle growth information and central processor usage rate, and then the process is determined to be abnormal or not based on the correlation strength.

[0035] For example, after obtaining the file handle growth rate and the central processor usage rate at the first to fifth collection time points, the obtained data is preprocessed, such as checking the integrity and consistency of the data, and removing obviously erroneous or unreasonable data points, to remove noise and outliers in the collected data, and ensure the data quality. Assuming that the preprocessed file handle growth rate at the first to fifth collection time points is 12%, 15%, 18%, 20%, and 22% respectively, and the corresponding central processor usage rate at the first to fifth collection time points is 60%, 65%, 70%, 75%, and 80% respectively.

[0036] The average of the file handle growth rates at the five data collection time points is 17.4%, and the average of the central processor usage rates at the five data collection time points is 70%.

[0037] The deviations between the file handle growth rates at the multiple collection time points and the average of the file handle growth rates are -5.4%, -2.4%, 0.6%, 2.6%, and 4.6% respectively, and the deviations between the central processor usage rates at the multiple collection time points and the average of the central processor usage rates are -10%, -5%, 0%, 5%, and 10% respectively.

[0038] The sum of the products between the deviations of the file handle growth rates at the five collection time points and the deviations of the central processor usage rates at the corresponding collection time points is 0.125, that is, the first sum value is determined to be 12.5%.

[0039] The square root of the sum of the squares of the deviations of the file handle growth rates at the five collection time points is =0.0795.

[0040] The square root of the sum of the squares of the deviations of the central processor usage rates at the five collection time points is =0.025.

[0041] The product between the square root of the sum of the squares of the deviations of the file handle growth rates at the five collection time points and the square root of the sum of the squares of the deviations of the central processor usage rates at the five collection time points is 0.0795×0.025=0.01256.

[0042] The ratio between the first sum value and the first product is 9.95, and thus the correlation strength between the file handle growth information and the central processor usage rate is determined to be 9.95.

[0043] In this way, by determining the correlation strength between the file handle growth information and the central processor usage rate, the accuracy of the process anomaly monitoring can be improved.

[0044] In some embodiments, the file handle growth information comprises a file handle growth rate, and the file handle growth information of the plurality of collection time points is determined based on the number of file handles in the process file descriptor directory of the plurality of collection time points, comprising: determining the number of file handle growth of the plurality of collection time points based on the difference between the number of file handles in the process file descriptor directory of the plurality of collection time points and the number of file handles in the process file descriptor directory of the corresponding previous collection time point; and determining the file handle growth rate of the plurality of collection time points based on the ratio between the number of file handle growth of the plurality of collection time points and the number of file handles in the process file descriptor directory of the corresponding previous collection time point.

[0045] For example, the number of file handle growth of the current collection time point can be calculated based on the difference between the number of file handles in the process file descriptor directory of the current collection time point and the number of file handles in the process file descriptor directory of the previous collection time point. Further, the file handle growth rate of the current collection time point can be calculated based on the ratio between the number of file handle growth of the current collection time point and the number of file handles in the process file descriptor directory of the previous collection time point. Similarly, the file handle growth rates of the plurality of collection time points can be calculated.

[0046] In this way, by calculating the file handle growth rate to analyze the growth trend of the file handle, it can be more accurate to determine whether the BMC process is abnormal.

[0047] In some embodiments, the file handle growth information comprises a file handle growth rate, and the BMC process is determined to be abnormal according to the correlation strength, the file handle growth information of the plurality of collection time points and the central processor usage rate, comprising: calculating the mean of the file handle growth rate and the mean of the central processor usage rate; determining a process abnormality coefficient according to the mean of the file handle growth rate, the mean of the central processor usage rate and the correlation strength; and determining the BMC process to be abnormal in the case that the process abnormality coefficient is greater than a preset process abnormality coefficient threshold. The preset process abnormality coefficient threshold can be calibrated according to actual conditions, and the user can configure the above weight coefficient in the BMC Web UI, which is not specifically limited here.

[0048] Specifically, after calculating the correlation strength, the mean of the file handle growth rate and the mean of the central processor usage rate, the correlation strength, the mean of the file handle growth rate and the mean of the central processor usage rate can be input into a preset formula or a preset model to output a process abnormality coefficient; further, the process abnormality coefficient is compared with the preset process abnormality coefficient threshold to determine whether the BMC process is abnormal. For example, if the process abnormality coefficient is greater than the preset process abnormality coefficient threshold, it is determined that the BMC process is abnormal; if the process abnormality coefficient is less than or equal to the preset process abnormality coefficient threshold, it is determined that the BMC process is not abnormal.

[0049] In some embodiments, the process anomaly coefficient is determined according to the mean of the file handle growth rate, the mean of the central processor usage rate, and the correlation strength, including: determining the process anomaly coefficient based on the sum of the product of the mean of the file handle growth rate, the mean of the central processor usage rate, and the correlation strength and the corresponding weight coefficients. Wherein the corresponding weight coefficients of the mean of the file handle growth rate, the mean of the central processor usage rate, and the correlation strength can be calibrated according to actual conditions, for example, the first weight coefficient corresponding to the mean of the file handle growth rate is 0.4, the second weight coefficient corresponding to the mean of the central processor usage rate is 0.3, and the third weight coefficient corresponding to the correlation strength is 0.3. The user can configure the above weight coefficients in the BMC Web UI, which is not specifically limited here.

[0050] For example, the mean of the file handle growth rate, the mean of the central processor usage rate, and the correlation strength can be input into the following formula to calculate the process anomaly coefficient:

[0051] Process anomaly coefficient = (mean of file handle growth rate x first weight coefficient) + (mean of central processor usage rate x second weight coefficient) + (correlation strength x third weight coefficient).

[0052] In this way, by comprehensively considering the file handle growth rate, the central processor usage rate, and the correlation strength, and calculating the process anomaly coefficient accordingly, the abnormality of the BMC process can be effectively and accurately identified.

[0053] In some embodiments, the file handle growth information includes the file handle growth rate, and the BMC process anomaly is determined according to the correlation strength, the file handle growth information at multiple collection time points, and the central processor usage rate, including: calculating the mean of the file handle growth rate and the mean of the central processor usage rate; in the case that multiple of the correlation strength, the mean of the file handle growth rate, and the mean of the central processor usage rate are greater than the corresponding preset threshold, determining that the BMC process is abnormal.

[0054] Specifically, after calculating the correlation strength, the mean of the file handle growth rate, and the mean of the central processor usage rate, the correlation strength, the mean of the file handle growth rate, and the mean of the central processor usage rate can also be compared with the corresponding threshold to determine whether the BMC process is abnormal. For example, if two or three of the correlation strength, the mean of the file handle growth rate, and the mean of the central processor usage rate are greater than the corresponding threshold, it is determined that the BMC process is abnormal; if only one of the correlation strength, the mean of the file handle growth rate, and the mean of the central processor usage rate is greater than the corresponding threshold, or if the correlation strength, the mean of the file handle growth rate, and the mean of the central processor usage rate are all less than or equal to the corresponding threshold, it is determined that the BMC process is not abnormal.

[0055] Exemplarily, the correlation strength can be compared with a preset correlation strength threshold, the average of the file handle growth rate can be compared with a preset file handle growth rate threshold, and the average of the central processor usage rate can be compared with a preset central processor usage rate threshold. If the correlation strength is greater than the preset correlation strength threshold, the average of the file handle growth rate is greater than the preset file handle growth rate threshold, and the average of the central processor usage rate is greater than the preset central processor usage rate threshold, it is determined that the baseboard management controller process is abnormal. The preset correlation strength threshold, the preset file handle growth rate threshold, and the central processor usage rate threshold can be calibrated according to actual conditions, and the user can configure the above thresholds in the BMC Web UI, which is not specifically limited here.

[0056] In this way, by comprehensively analyzing the file handle growth rate and the central processor usage rate at multiple collection moments, calculating the average of each, and combining the correlation strength, it can be quickly and accurately determined whether the baseboard management controller process is abnormal. In the case where multiple key indicators simultaneously exceed the preset threshold, the process abnormality is quickly determined, avoiding the problem of single indicator misjudgment. Not only does this improve the accuracy of abnormal detection and reduce the possibility of false positives and false negatives, but it also significantly improves the detection speed.

[0057] In some embodiments, the number of file handles in the process file descriptor directory, the process memory usage, the central processor usage rate, the number of threads, and the number of context switches at multiple collection moments can also be obtained. The file handle growth information at multiple collection moments is determined based on the number of file handles in the process file descriptor directory at multiple collection moments. The correlation strength between the file handle growth information and the process memory usage, the central processor usage rate, the number of threads, and the number of context switches is determined based on the file handle growth information, the process memory usage, the central processor usage rate, the number of threads, and the number of context switches at multiple collection moments. The baseboard management controller process is determined to be abnormal according to the correlation strength, the process memory usage, the central processor usage rate, the number of threads, and the number of context switches at multiple collection moments.

[0058] Specifically, the process memory usage, the central processor usage rate, the number of threads, and the number of context switches at multiple collection moments can be obtained by polling using system commands such as top. The file handle growth information can be the file handle growth rate, and the confirmation method of the correlation strength between the file handle growth rate and the process memory usage, the number of threads, and the number of context switches is the same as the confirmation method of the correlation strength between the file handle growth information and the central processor usage rate, which is not described here.

[0059] The mean value of the file handle growth rate, the mean value of the process memory usage, the mean value of the central processor usage rate, the mean value of the thread number, and the mean value of the context switch number are calculated respectively. The process abnormality coefficient is determined based on the sum of the mean value of the file handle growth rate, the mean value of the process memory usage, the mean value of the central processor usage rate, the mean value of the thread number, the mean value of the context switch number, and the product of the correlation strength between the file handle growth information and the process memory usage, the thread number, and the context switch number and the corresponding weight coefficient. In the case that the process abnormality coefficient is greater than a preset process abnormality coefficient threshold, it is determined that the baseboard management controller process is abnormal; otherwise, it is determined that the baseboard management controller process is not abnormal.

[0060] In this way, by collecting multiple key system indicators (including the number of file handles, process memory usage, central processor usage rate, thread number, and context switch number) and calculating their mean values, while determining the correlation strength between the file handle growth information and each indicator, the state of the baseboard management controller process can be more comprehensively and accurately evaluated. By introducing a weight coefficient and comprehensively calculating a process abnormality coefficient, not only the accuracy of abnormality detection is improved, avoiding the problem of single indicator misjudgment, but also the flexibility and adaptability of detection are enhanced.

[0061] In some embodiments, the file handle growth information further includes the file handle growth number and the file handle growth rate, and the baseboard management controller process abnormality can also be determined according to the file handle growth information, including: in the case that the file handle growth number is greater than a preset file handle increment threshold, and / or the file handle growth rate is greater than a preset file handle growth rate threshold, it is determined that the baseboard management controller process is abnormal. The preset file handle increment threshold and the preset file handle growth rate threshold can be calibrated according to actual conditions, which is not specifically limited here, and the user can configure the preset file handle increment threshold and the preset file handle growth rate threshold of the process in the BMC Web UI.

[0062] Specifically, the file handle growth number can be compared with the preset file handle increment threshold to determine whether the baseboard management controller process is abnormal. For example, if the file handle growth number is greater than the preset file handle increment threshold, it means that the number of file handles at the current collection time has grown too much compared to the number of file handles at the previous collection time, and it is determined that the baseboard management controller process is abnormal; if the file handle growth number is less than or equal to the preset file handle increment threshold, it is determined that the baseboard management controller process is not abnormal.

[0063] Alternatively, the file handle growth rate can be compared with a preset file handle growth rate threshold to determine whether the baseboard management controller process is abnormal. For example, if the file handle growth rate is greater than the preset file handle growth rate threshold, it indicates that the number of file handles at the current collection time grows too fast, and it is determined that the baseboard management controller process is abnormal; if the file handle growth rate is less than or equal to the preset file handle growth rate threshold, it is determined that the baseboard management controller process is not abnormal.

[0064] Alternatively, the file handle growth quantity can also be compared with a preset file handle growth quantity threshold, and the file handle growth rate can be compared with a preset file handle growth rate threshold to determine whether the baseboard management controller process is abnormal. For example, if the file handle growth quantity is greater than the preset file handle growth quantity threshold, and the file handle growth rate is greater than the preset file handle growth rate threshold, it indicates that the number of file handles at the current collection time grows too much compared to the number of file handles at the previous collection time, and the number of file handles at the current collection time grows too fast, and it is determined that the baseboard management controller process is abnormal; if one or both of the file handle growth quantity and the file handle growth rate is less than or equal to the corresponding threshold, it is determined that the baseboard management controller process is not abnormal.

[0065] In this way, by monitoring the growth quantity and growth rate of file handles, and combining the preset file handle growth quantity threshold and the file handle growth rate threshold for abnormality judgment, the file handle abnormality in the baseboard management controller process can be quickly and accurately detected, thereby better adapting to the characteristics of different running stages and modules, significantly improving the accuracy and adaptability of abnormality detection, and providing more reliable technical support for server operation and maintenance.

[0066] In addition, a first average of the file handle growth quantities at multiple times and a second average of the file handle growth rates at multiple times can also be determined, and in the case that the first average is greater than the preset file handle growth quantity threshold, and / or the second average is greater than the preset file handle growth rate threshold, it is determined that the baseboard management controller process is abnormal. In this way, the accuracy of abnormality judgment of the baseboard management controller process can be further improved.

[0067] In some embodiments, the context information of the baseboard management controller process includes a call stack table and a file descriptor list of the baseboard management controller process, and determining the abnormal code path according to the context information of the baseboard management controller process includes: determining a collection time of the context information of the baseboard management controller process, and obtaining a log table corresponding to the collection time; in the case that a current execution function in the call stack table is executing a file path opening operation, the number of opening times of the file path in the file descriptor list is continuously increasing, and the log table records a repeated opening error of the file path, determining a function chain corresponding to the current execution function in the call stack table as the abnormal code path.

[0068] Specifically, the context information of the BMC process includes a call stack table of the BMC process and a file descriptor list of the BMC process, the call stack table of the BMC process includes an operation that a current execution function is performing, the file descriptor list includes a file descriptor number (i.e., a file handle number), a file path corresponding to each file descriptor number, for example, each file descriptor under the / proc / [pid] / fd directory is linked to an actual file path, and a file open mode flag (e.g., read-only, write-only, read-write) of each file descriptor. In order to further improve the accuracy of the determination of the abnormal code path, a collection time of the context information of the BMC process can be determined, and the collection time or a log table corresponding to a time close to the collection time is read, the log table includes an error entry of the collection time or error entries of a time close to the collection time, for example, a file path open error. According to the call stack table of the BMC process, the file descriptor list, and the error entry of the collection time or the error entries of a time close to the collection time in the log table, the abnormal code path is determined.

[0069] For example, in the call stack table, the current execution function is performing a file path open operation, and the number of openings of a file path in the file descriptor list continues to increase, and when the error entry in the log table shows that the file path is repeatedly opened, it can be analyzed that the function chain corresponding to the current execution function in the call stack table of the BMC process is the abnormal code path.

[0070] In this way, in the case of determining that the BMC process is abnormal, the call stack table and the file descriptor list of the process are automatically triggered to be snapped, without stopping or restarting the service, without affecting business continuity; at the same time, combined with the associated analysis of the system log table, the code module and the function call path causing the handle leakage are accurately locked, and the problem positioning and repair time is greatly shortened.

[0071] In some embodiments, the above method further includes: in the case of determining that the BMC process is abnormal, generating alarm information and sending the alarm information to an external terminal device.

[0072] Specifically, in the case of determining that the BMC process is abnormal, the alarm information is generated, for example, "the current BMC process is abnormal" is generated, and combined with Redfish Event (Redfish event), mail or SNMPTrap, etc. Multiple output channels are uniformly sent to the terminal device through D-Bus, for example, sent to the computer or mobile phone of the maintenance personnel.

[0073] In addition, an alarm frequency limit can be set, and for the same type of alarm, only one alarm is sent in a certain time interval (e.g., every 5 minutes), so as to avoid repeated sending of the same alarm in a short time, or a plurality of alarms of the same type are combined into one alarm information, so as to reduce the number of alarms.

[0074] In some embodiments, the method further comprises generating a file handle change curve according to the file handle growth number and the file handle growth rate at the plurality of collection time points, and storing and displaying the file handle change curve.

[0075] Specifically, the file handle growth number and the file handle growth rate at each collection time point are continuously collected and stored in a time series database, and a file handle change curve is dynamically drawn by using a sliding window algorithm. An operation and maintenance personnel can view the file handle change curve through a Dashboard in a Web management interface of the BMC, and the file handle change curve is stored in a preset storage location in a PNG (Portable Network Graphics), SVG (Scalable Vector Graphics), or JSON (JavaScript Object Notation) format. In this way, the change of the file handle number with the collection time point can be intuitively reflected.

[0076] In some embodiments, the method further comprises displaying the abnormal code path and the alarm information.

[0077] Specifically, the operation and maintenance personnel can view the abnormal code path and the alarm information through the Dashboard in the Web management interface of the BMC.

[0078] In addition, a "reload configuration", a "restart process", or a "manual recovery" button is provided in the Web management interface of the BMC, and the operation and maintenance personnel can quickly intervene.

[0079] In this way, the file handle change curve, the abnormal code path, and the alarm information are uniformly presented on the Dashboard in the Web management interface of the BMC, and a "one-key restart process" and a "trigger resource recovery" button are provided, so that the operation and maintenance personnel can directly complete the intervention on the interface without logging in to a command line or relying on external tools. The cooling strategy is combined with a multi-channel alarm channel (Redfish Event, email, and SNMP Trap), so as to avoid an alarm storm to a certain extent, and ensure that abnormal information is timely delivered.

[0080] In summary, the application can intervene in the OpenBMC daemon before resource exhaustion, significantly reduce the number of service interruptions caused by file handle leakage, accelerate fault repair according to abnormal code paths, reduce the average recovery time of system downtime, and improve the overall availability of data centers or edge devices.

[0081] In some embodiments, referring to Figure 3 , the monitoring system of the baseboard management controller process includes a collection module, an analysis module, and a root cause tracing module.

[0082] The collection module 210 is configured to obtain the central processor usage and the number of file handles in the process file descriptor directory at multiple collection time points. The analysis module 220 is configured to determine file handle growth information at the multiple collection time points based on the number of file handles in the process file descriptor directory at the multiple collection time points, determine the correlation strength between the file handle growth information and the central processor usage based on the file handle growth information at the multiple collection time points and the central processor usage, and determine the baseboard management controller process anomaly according to the correlation strength, the file handle growth information at the multiple collection time points, and the central processor usage. The root cause tracing module 230 is configured to determine an abnormal code path based on the context information of the baseboard management controller process in the case of determining the baseboard management controller process anomaly according to the correlation strength, the file handle growth information at the multiple collection time points, and the central processor usage, to monitor the baseboard management controller process.

[0083] According to an embodiment of the application, the file handle growth information includes a file handle growth rate, and the analysis module 220 is specifically configured to calculate the mean of the file handle growth rate and the mean of the central processor usage, determine the file handle growth rate deviation at the multiple collection time points based on the difference between the file handle growth rate at the multiple collection time points and the mean of the file handle growth rate, determine the central processor usage deviation at the multiple collection time points based on the difference between the central processor usage at the multiple collection time points and the mean of the central processor usage, and determine the correlation strength between the file handle growth information and the central processor usage based on the file handle growth rate deviation at the multiple collection time points and the central processor usage deviation at the multiple collection time points.

[0084] According to an embodiment of the application, the analysis module 220 is specifically configured to determine the product between the square root of the sum of the squares of the file handle growth rate deviations at the multiple collection time points and the square root of the sum of the squares of the central processor usage deviations at the multiple collection time points as a first product, determine the sum of the products between the file handle growth rate deviations at the multiple collection time points and the central processor usage deviations at the corresponding collection time points as a first sum, and determine the correlation strength between the file handle growth information and the central processor usage based on the ratio between the first sum and the first product.

[0085] According to an embodiment of the present application, the analysis module 220 is specifically configured to calculate the mean of the file handle growth rate and the mean of the central processor usage rate; determine the process anomaly coefficient according to the mean of the file handle growth rate, the mean of the central processor usage rate and the correlation strength; and determine that the baseboard management controller process is abnormal in the case that the process anomaly coefficient is greater than a preset process anomaly coefficient threshold.

[0086] According to an embodiment of the present application, the analysis module 220 is specifically configured to determine the process anomaly coefficient based on the sum of the product of the mean of the file handle growth rate, the mean of the central processor usage rate and the corresponding weight coefficient.

[0087] According to an embodiment of the present application, the analysis module 220 is further configured to determine the file handle growth quantity at the plurality of collection time instants based on the difference between the file handle quantity in the process file descriptor directory at the plurality of collection time instants and the file handle quantity in the process file descriptor directory at the corresponding previous collection time instant; and determine the file handle growth rate at the plurality of collection time instants based on the ratio between the file handle growth quantity at the plurality of collection time instants and the file handle quantity in the process file descriptor directory at the corresponding previous collection time instant.

[0088] According to an embodiment of the present application, the analysis module 220 is further configured to generate a file handle change curve diagram according to the file handle growth quantity and the file handle growth rate at the plurality of collection time instants.

[0089] According to an embodiment of the present application, the context information of the baseboard management controller process includes a call stack table and a file descriptor list of the baseboard management controller process, and the root cause tracing module 230 is specifically configured to determine the collection time of the context information of the baseboard management controller process, and acquire a log table corresponding to the collection time; in the case that the current execution function in the call stack table is performing a file path opening operation, the number of opening times of the file path in the file descriptor list is continuously increasing, and the log table records a repeated opening error of the file path, determine that a function chain corresponding to the current execution function in the call stack table is an abnormal code path.

[0090] As a specific example, with reference to Figure 4 The monitoring system of the baseboard management controller process of the present application further includes a configuration module 240, a storage module 250, an alarm module 260 and a display module 270.

[0091] The configuration module 240 is configured to provide a RESTful interface, allowing the client to interact with the baseboard management controller through an HTTP request. An operation and maintenance personnel can configure the baseboard management controller process to be monitored, the sampling period of the baseboard management controller process, the preset process exception coefficient threshold, the preset file handle increment threshold, the preset file handle growth rate threshold, the first weight coefficient corresponding to the mean value of the file handle growth rate, the second weight coefficient corresponding to the mean value of the central processing unit usage rate, and the third weight coefficient corresponding to the correlation strength through the BMC Web UI.

[0092] The storage module 250 is configured to store the number of file handles in the process file descriptor directory and the time stamp at each collection time. The storage module 250 can be a time series database, such as InfluxDB or lightweight SQLite.

[0093] The alarm module 260 is configured to generate alarm information and send it to an external terminal device when it is determined that the baseboard management controller process is abnormal. For example, when it is determined that the baseboard management controller process is abnormal, alarm information is generated, such as "current baseboard management controller process is abnormal", and is sent to a terminal device, such as a computer or a mobile phone of a maintenance personnel, through a D-Bus, in combination with a Redfish Event, a mail, an SNMP Trap, and other output channels.

[0094] The display module 270 is integrated in the Web management interface of the BMC and is configured to display the abnormal code path, the alarm information, and a file handle change curve diagram. Meanwhile, "reload configuration", "restart process", or "manual recovery" buttons are provided to support an operation and maintenance personnel to quickly intervene.

[0095] It should be noted that the above explanations and beneficial effects of the embodiments of the baseboard management controller process monitoring method also apply to the baseboard management controller process monitoring system of the embodiments of the present application. To avoid redundancy, they will not be described in detail here.

[0096] Corresponding to the above embodiments, the present application also provides an electronic device.

[0097] Referring to FIG. 3, Figure 5 The electronic device 300 includes a memory 310, a processor 320, and a computer program stored in the memory 310 and executable on the processor 320. When the processor executes the computer program, the baseboard management controller process monitoring method described above is implemented.

[0098] It should be noted that the above explanations and beneficial effects of the embodiments of the baseboard management controller process monitoring method also apply to the electronic device of the embodiments of the present application. To avoid redundancy, they will not be described in detail here.

[0099] Through the above description of the embodiments, those skilled in the art can clearly understand that the method according to the above-mentioned embodiments can be realized by means of software on a general hardware platform as necessary, and of course, can also be realized by hardware, but in many cases, the former is a better embodiment.

[0100] In an example embodiment, the computer readable storage medium described above can include, but is not limited to, a U disk, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and various media that can store computer programs.

[0101] The skilled person can further realize that the units and algorithm steps of the examples described in conjunction with the embodiments disclosed herein can be realized by electronic hardware, computer software or a combination of both. In order to clearly illustrate the interchangeability of hardware and software, the components and steps of the examples have been described in the above description in general terms. Whether the functions are realized in hardware or software depends on the specific application and design constraints of the technical solution. The skilled person can use different methods to realize the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0102] The above provides a detailed description of the method, system, electronic device and medium of the substrate management controller process monitoring method provided by the present application. The principles and embodiments of the present application are described in this paper using specific examples. The above example is only used to help understand the method and its core idea of the present application. It should be pointed out that for those skilled in the art, without departing from the principles of the present application, some improvements and modifications can be made to the present application, and these improvements and modifications also fall within the scope of protection of the claims of the present application.

Claims

1. A method for monitoring the process of a baseboard management controller, characterized in that, Applied to a baseboard management controller, the method includes: Obtain the CPU utilization and the number of file handles in the process file descriptor directory at multiple acquisition times; The file handle growth information for multiple acquisition times is determined based on the number of file handles in the process file descriptor directory at multiple acquisition times; Based on file handle growth information and CPU utilization at multiple acquisition times, the correlation strength between the file handle growth information and CPU utilization is determined. If the board management controller process is determined to be abnormal based on the correlation strength, file handle growth information at multiple acquisition times, and CPU utilization, the abnormal code path is determined based on the context information of the board management controller process in order to monitor the board management controller process.

2. The method for monitoring the process of the baseboard management controller according to claim 1, characterized in that, The file handle growth information includes a file handle growth rate. Based on the file handle growth information and CPU utilization at multiple acquisition times, the correlation strength between the file handle growth information and the CPU utilization is determined, including: Calculate the average growth rate of the file handles and the average utilization rate of the central processing unit; The file handle growth rate deviation at multiple acquisition times is determined based on the difference between the file handle growth rate at multiple acquisition times and the mean of the file handle growth rate. The CPU utilization deviation at multiple acquisition times is determined based on the difference between the CPU utilization at multiple acquisition times and the mean of the CPU utilization. Based on the file handle growth rate deviation and the CPU utilization deviation at multiple acquisition times, the correlation strength between the file handle growth information and the CPU utilization is determined.

3. The method for monitoring the process of the baseboard management controller according to claim 2, characterized in that, Based on the file handle growth rate deviation and the CPU utilization deviation at multiple acquisition times, the correlation strength between the file handle growth information and the CPU utilization is determined, including: The product of the square root of the sum of squares of the deviations in file handle growth rates at multiple acquisition times and the square root of the sum of squares of the deviations in CPU utilization at multiple acquisition times is determined as the first product. The sum of the products of the file handle growth rate deviations at multiple acquisition times and the corresponding CPU utilization deviations at the acquisition times is determined as the first sum value; The strength of the correlation between the file handle growth information and the CPU utilization rate is determined based on the ratio between the first sum and the first product.

4. The method for monitoring the process of the baseboard management controller according to claim 1, characterized in that, The file handle growth information includes a file handle growth rate, which is determined based on the number of file handles in the process file descriptor directory at multiple acquisition times, including: The number of file handles increased at each of the multiple acquisition times is determined based on the difference between the number of file handles in the file descriptor directory of the process at the multiple acquisition times and the number of file handles in the file descriptor directory of the corresponding previous acquisition time. The file handle growth rate for the multiple acquisition times is determined based on the ratio between the number of file handles increasing at multiple acquisition times and the number of file handles in the corresponding process file descriptor directory at the previous acquisition time.

5. The method for monitoring the process of the baseboard management controller according to claim 1, characterized in that, The file handle growth information includes the file handle growth rate. Based on the correlation strength, file handle growth information at multiple acquisition times, and CPU utilization, the abnormality of the baseboard management controller process is determined, including: Calculate the average growth rate of the file handles and the average utilization rate of the central processing unit; The process anomaly coefficient is determined based on the average file handle growth rate, the average CPU utilization, and the correlation strength. If the process anomaly coefficient is greater than a preset process anomaly coefficient threshold, the process of the baseboard management controller is determined to be abnormal.

6. The method for monitoring the process of the baseboard management controller according to claim 5, characterized in that, The process anomaly coefficient is determined based on the average file handle growth rate, the average CPU utilization, and the correlation strength, including: The process anomaly coefficient is determined based on the average file handle growth rate, the average CPU utilization, and the sum of the products of the correlation strength and the corresponding weight coefficients.

7. The method for monitoring the process of the baseboard management controller according to claim 1, characterized in that, The file handle growth information includes the file handle growth rate. Based on the correlation strength, file handle growth information at multiple acquisition times, and CPU utilization, the abnormality of the baseboard management controller process is determined, including: Calculate the average growth rate of the file handles and the average utilization rate of the central processing unit; If multiple values ​​of the association strength, the average value of the file handle growth rate, and the average value of the central processing unit utilization exceed the corresponding preset thresholds, the baseboard management controller process is determined to be abnormal.

8. The method for monitoring the process of the baseboard management controller according to claim 1, characterized in that, The context information of the baseboard management controller process includes the call stack table and file descriptor list of the baseboard management controller process. The exception code path is determined based on the context information of the baseboard management controller process, including: Determine the acquisition time of the context information of the baseboard management controller process, and obtain the log table corresponding to the acquisition time; If the currently executing function in the call stack is performing a file path opening operation, the number of times the file path in the file descriptor list is opened continues to increase, and the log table records that the file path has a duplicate opening error, then the function chain corresponding to the currently executing function in the call stack is determined to be the abnormal code path.

9. The method for monitoring the process of the baseboard management controller according to claim 4, characterized in that, The method further includes: A file handle change curve is generated based on the number of file handle increases and the file handle growth rate at multiple acquisition times, and the file handle change curve is stored and displayed.

10. An electronic device, characterized in that, The system includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements a method for monitoring the process of a substrate management controller according to any one of claims 1-9.

Images