Fault processing system, method, equipment and medium

Abstract

The embodiment of the invention provides a fault processing system and method, equipment and a medium, and relates to the field of system fault problem analysis. The system comprises a data mining processing unit and a monitoring unit, the monitoring unit is used for monitoring environment information of the server; and the data mining processing unit determines the fault occurrence time of the external device with the fault, obtains the environment information of the server through the monitoring unit, screens out the environment information of the server in a target time period corresponding to the fault occurrence time, and records and stores the environment information. Through the mode, fault analysis personnel can comprehensively analyze the information of the fault occurrence moment, accurately obtain the working state of the server within a period of time before and after the fault mode moment, analyze the fault problem more conveniently and clearly, and record the information of the fault moment, so that the working state of the server can be accurately analyzed. The method can avoid the situation that the fault phenomenon cannot be reproduced due to the timeliness problem, and the judgment of the fault reason is affected, and greatly improves the fault problem processing efficiency.

Description

Technical Field

[0001] The present application relates to the field of system fault problem analysis, and in particular to a fault handling system, method, device and medium. Background Art

[0002] In recent years, with the rapid development and widespread application of artificial intelligence technology, as a key infrastructure supporting the operation of AI (artificial intelligence), the AI ​​server market has shown explosive growth, and a large number of diverse server products have been put on the market. Generally speaking, general-purpose servers mainly adopt a serial architecture dominated by CPU (central processing unit), which is better at logical operations; while AI servers mainly adopt a heterogeneous form dominated by accelerator cards, which is better at parallel computing with high throughput. According to the number of CPUs, general-purpose servers can be divided into two-way, four-way and eight-way. Although AI servers are generally equipped with only 1-2 CPUs, the number of GPUs (graphics processing units) is significantly superior. According to the number of GPUs, AI servers can be divided into four-way, eight-way and sixteen-way servers, among which eight-way AI servers equipped with 8 GPUs are the most common.

[0003] Since AI servers need to be equipped with different numbers of GPU modules, in order to support the implementation of GPU functions, the server also needs to support the same number of network cards and NVME (NVM Express, non-volatile memory express, non-volatile memory host controller interface specification) disks at the same time. The network card is used to interconnect other GPUs in the node or GPUs in other nodes, and the NVME disk is used to store and read large data. This means that AI servers will require a large number of PCIe (PCI-Express, peripheral component interconnect express, high-speed serial computer expansion bus standard) resources, which also introduces the use of PCIe Switch (PCIe interface chip). Due to the high PCIe signal rate and the large number and variety of PCIe devices on the board, the failure rate is also relatively high. Therefore, how to quickly and accurately locate the problem and formulate solutions is the top priority of server R&D work.

[0004] With the demand for small space and modularization, there is an urgent need to enable server maintenance personnel to promptly identify the health status of the server as a whole and important chips when analyzing problems, so as to quickly locate problems and develop solutions. Summary of the invention

[0005] Based on the above technical problems, the embodiments of the present application provide a fault handling system, method, device and medium, aiming at how to quickly and comprehensively collect fault information and locate faults.

[0006] A first aspect of an embodiment of the present application provides a fault handling system, the system comprising:

[0007] Data mining processing unit and monitoring unit;

[0008] The monitoring unit is used to monitor the environment information of the server;

[0009] The data mining processing unit determines the time of occurrence of a fault of the external device that has a fault;

[0010] The data mining processing unit is connected to the monitoring unit via a first communication link, obtains the environmental information of the server from the monitoring unit, and filters out the environmental information of the server within a target time period corresponding to the time when the fault occurs, wherein the target time period includes a first time period before the time when the fault occurs and / or a second time period after the time when the fault occurs.

[0011] Optionally, the system further comprises:

[0012] A first processor is used to connect to an external device, and the first processor is used to collect a fault log of the connected external device;

[0013] The data mining processing unit is connected to the first processor via a second communication link, obtains the fault log of the external device via the first processor, and analyzes the fault log to determine the fault occurrence time of the external device.

[0014] Optionally, the system further comprises:

[0015] A second processor is used to monitor the voltage abnormality record of the external device;

[0016] The data mining processing unit is connected to the second processor via a third communication link, obtains the voltage anomaly records of the external device from the second processor, and screens out the voltage anomaly records of the external device within the target time period.

[0017] Optionally, the system further comprises:

[0018] The second processor is further used to monitor the voltage abnormality record of the first processor;

[0019] The data mining processing unit is further configured to obtain voltage anomaly records of the first processor from the second processor, and filter out voltage anomaly records of the first processor within the target time period.

[0020] Optionally, the system further comprises:

[0021] Complex programmable logic devices for monitoring timing signals and alarm signals;

[0022] The data mining processing unit is connected to the complex programmable logic device via a fourth communication link, obtains the timing signal and the alarm signal from the complex programmable logic device, filters out the timing signal and the alarm signal within the target time period, and marks timestamps for the filtered timing signal and the alarm signal.

[0023] Optionally, the system further comprises:

[0024] A third processor;

[0025] The data mining processing unit is connected to the third processor via the fifth communication link, and obtains the error record generated by the third processor, and analyzes the error record to determine the time when the error occurs, and filters out the environmental information of the server within the error time period corresponding to the time when the error occurs, and the error time period includes a first time period before the time when the error occurs and / or a second time period after the time when the error occurs.

[0026] Optionally, the data mining processing unit in the system includes a storage card for storing at least one of the following:

[0027] Environmental information of the server within a target time period corresponding to the time when the fault occurs;

[0028] Environmental information of the server within a target time period corresponding to the time when the error occurs;

[0029] abnormal voltage record of the first processor within the target time period;

[0030] The voltage abnormality record of the external device within the target time period;

[0031] The timing signal and the alarm signal within the target time period.

[0032] Optionally, the system further comprises:

[0033] Management controller;

[0034] The data mining processing unit is connected to the management controller via a sixth communication link, and the management controller receives the environment information of the server in the target time period corresponding to the time when the fault occurs, which is sent by the data mining processing unit.

[0035] Optionally, the system further comprises:

[0036] A second processor is used to monitor the voltage abnormality record of the external device and monitor the voltage abnormality record of the first processor;

[0037] The data mining processing unit is connected to the second processor via the third communication link, obtains the voltage abnormality record of the external device from the second processor, and filters out the voltage abnormality record of the external device within the target time period, and obtains the voltage abnormality record of the first processor from the second processor, and filters out the voltage abnormality record of the first processor within the target time period;

[0038] The management controller is connected to the monitoring unit via the first communication link, and is connected to the second processor via a seventh communication link, and the management controller is used for at least one of the following:

[0039] Acquire the environment information of the server from the monitoring unit and display it through the external window of the management controller;

[0040] Acquire the voltage abnormality record of the external device from the second processor and display it through the external window of the management controller;

[0041] Acquire the voltage abnormality record of the first processor from the second processor and display it through the external window of the management controller;

[0042] The information stored in the memory card of the data mining processing unit is read and displayed through the external window of the management controller.

[0043] A second aspect of an embodiment of the present application provides a fault handling method, the method comprising:

[0044] Monitor the server's environmental information;

[0045] The environment information of the server is obtained, and the environment information of the server in a target time period corresponding to the time when the fault occurs is filtered out, wherein the target time period includes a first time period before the time when the fault occurs and / or a second time period after the time when the fault occurs.

[0046] Optionally, the method further comprises:

[0047] Collecting fault logs of the connected external device;

[0048] Obtain a fault log of the external device, and analyze the fault log to determine the fault occurrence time of the external device that has a fault.

[0049] Optionally, the method further comprises:

[0050] The voltage anomaly records of the external device are acquired from the second processor, and the voltage anomaly records of the external device within the target time period are screened out.

[0051] Optionally, the method further comprises:

[0052] The voltage abnormality record of the first processor is obtained from the second processor, and the voltage abnormality record of the first processor within the target time period is filtered out.

[0053] Optionally, the method further comprises:

[0054] The timing signal and the alarm signal are acquired from the complex programmable logic device, the timing signal and the alarm signal within the target time period are screened out, and timestamps are marked for the screened timing signal and the alarm signal.

[0055] Optionally, the method further comprises:

[0056] Obtain the error record generated by the third processor, analyze the error record, determine the time when the error occurred, and filter out the environmental information of the server within the error time period corresponding to the time when the error occurred, the error time period including a first time period before the time when the error occurred and / or a second time period after the time when the error occurred.

[0057] Optionally, the method further comprises:

[0058] Store at least one of the following:

[0059] Environmental information of the server within a target time period corresponding to the time when the fault occurs;

[0060] Environmental information of the server within a target time period corresponding to the time when the error occurs;

[0061] abnormal voltage record of the first processor within the target time period;

[0062] The voltage abnormality record of the external device within the target time period;

[0063] The timing signal and the alarm signal within the target time period.

[0064] Optionally, the method further comprises:

[0065] Receive the environment information of the server in the target time period corresponding to the time when the fault occurs, which is sent by the data mining processing unit.

[0066] Optionally, the method further comprises:

[0067] Acquire the environment information of the server from the monitoring unit and display it through the external window of the management controller;

[0068] Acquire the voltage abnormality record of the external device from the second processor and display it through the external window of the management controller;

[0069] Acquire the voltage abnormality record of the first processor from the second processor and display it through the external window of the management controller;

[0070] The information stored in the memory card of the data mining processing unit is read and displayed through the external window of the management controller.

[0071] A third aspect of an embodiment of the present application provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the computer program is executed by the processor, a fault handling method as in the second aspect of an embodiment of the present application is implemented.

[0072] A fourth aspect of an embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the fault handling method of the second aspect of the embodiment of the present application is implemented.

[0073] A fifth aspect of an embodiment of the present application provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the fault handling method of the second aspect of an embodiment of the present application.

[0074] Through the fault handling system of the embodiment of the present application, a data mining processing unit is connected to the first processor, the second processor, the monitoring unit, the complex programmable logic device and the third processor to collect fault logs and error records, and collect server environment information, voltage anomaly records, timing signals and alarm signals within the target time period corresponding to the time when the fault and / or error occurred, and organize the above information and store it in the storage card contained in the data mining processing unit, and display the above collected information to the user when the user needs it.

[0075] In this application, the existing fault analysis method is improved by collecting and recording a series of information at the time of the fault, so that the fault analyst can conduct a comprehensive analysis of the information at the time of the fault, accurately obtain the working status of the server at the time of the fault, and analyze the fault problem more conveniently and clearly to find the problem point. Moreover, by recording the information at the time of the fault, it can be avoided that the fault phenomenon cannot be reproduced due to timeliness issues, which affects the judgment of the cause of the fault. Through the recorded information, the fault analyst can intuitively understand the information of various key monitoring items when the fault problem occurs, and can control the overall status of the server without reproducing the fault, and the analysis of the fault problem is more comprehensive. Moreover, through the fault handling system proposed in this application, remote analysis of fault problems can be realized. For customer problems that are not convenient to go to the site for processing at the first time, the relevant information when the fault problem occurs can be understood through the fault handling system, and the fault problem analysis can be carried out at the first time, which greatly improves the efficiency of fault problem handling. BRIEF DESCRIPTION OF THE DRAWINGS

[0076] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments of the present application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0077] Figure 1 is a schematic diagram of a monitoring unit proposed in an embodiment of the present application;

[0078] Figure 2 is a schematic diagram of a fault handling system proposed in an embodiment of the present application;

[0079] Figure 3 This is a schematic diagram of a management controller connection relationship proposed in an embodiment of the present application;

[0080] Figure 4 It is a schematic diagram of an electronic device shown in an embodiment of the present application. DETAILED DESCRIPTION

[0081] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0082] In the drawings, the size of the constituent elements, the thickness of the layer or the area may be exaggerated for the sake of clarity. Therefore, any implementation of the present disclosure is not necessarily limited to the size shown in the drawings, and the shapes and sizes of the components in the drawings do not reflect the true proportions. In addition, the drawings schematically show ideal examples, and any implementation of the present disclosure is not limited to the shapes or values ​​shown in the drawings.

[0083] In the related technology, although the processing function of the server is greatly improved due to the large number of PCIe devices used in the AI ​​server, it will also bring various possible problems. Among them, due to the use of a large number of PCIe devices, the failure rate of PCIe devices will also increase. In the past, when performing fault analysis on various PCIe device problems that occurred in server systems, two solutions were usually adopted: the first method is to access the register status of the PCIe device under the OS (operating system), and compare the actual value read by access with the standard value of the register. If the two are different and the difference is large, it means that there is a problem with the register. However, this method often requires a large amount of register data comparison, and may only find the abnormality of the register but cannot accurately locate the core of the problem; the second method is to perform fault analysis through the fault manual provided by the PCIe device manufacturer. The manual often lists some common problems that may occur in the device. According to the debug work provided in the manual, corresponding troubleshooting is performed. By reading the registers inside the PCIe Switch chip and comparing them with the standards in the manual, it is determined whether the problems listed in the manual have occurred. Obviously, this method is more inclined to targeted troubleshooting of known common problems, but in actual use, various problems may occur, which are obviously not covered by the fault manual.

[0084] The troubleshooting methods used in the prior art usually require operators to have a large amount of relevant knowledge reserves and troubleshooting experience, so that they can quickly locate the device where the problem may occur based on the phenomenon that occurs, and then perform corresponding troubleshooting work. However, even experienced operators still need to compare the specific definition of each register with the standard document one by one, which greatly increases the time cost; and some problems may not be caused by a certain device itself, but are affected by environmental factors, or are chain reactions caused by problems in other devices. In the prior art, only a certain PCIe device or its register itself is often checked. If it is necessary to check the server operating environment or other devices, it is necessary to use other channels, which not only increases the difficulty of troubleshooting, but also greatly reduces the efficiency of fault handling.

[0085] Based on the problems existing in the prior art, the present application proposes a fault handling system, including:

[0086] Data mining processing unit and monitoring unit;

[0087] The monitoring unit is used to monitor the environment information of the server;

[0088] The data mining processing unit determines the time of occurrence of a fault of the external device that has a fault;

[0089] The data mining processing unit is connected to the monitoring unit via a first communication link, obtains the environmental information of the server from the monitoring unit, and filters out the environmental information of the server within a target time period corresponding to the time when the fault occurs, wherein the target time period includes a first time period before the time when the fault occurs and / or a second time period after the time when the fault occurs.

[0090] In an embodiment of the present application, in order to solve the problems existing in the prior art, a fault handling system is proposed, which mainly includes a data mining processing unit and a monitoring unit, wherein the monitoring unit is used to monitor the environmental information of the server, and through its built-in sensors, monitor the temperature, humidity, air pressure, vibration and other information of the environment in which the server is located. The above environmental information is only an example given in the embodiment of the present application. During actual use, monitoring of other environmental information can be added according to the user's usage needs.

[0091] In addition, the monitoring unit and the data mining processing unit are equipped with a built-in clock module and a power supply module to ensure that they are always monitoring and recording according to time, and will be calibrated with the system time when the system is powered on to keep it consistent, so as to ensure that when a fault occurs, all information at the time of the fault is consistent with the time in the fault log. Among them, since the occurrence of a fault may last for a certain period of time, when collecting information, information should be collected and recorded for a period of time before and after the fault time to avoid missing information affecting the analysis of the fault problem.

[0092] In combination with the above embodiments, in one implementation, the present application further provides a fault handling system, which further includes:

[0093] A first processor is used to connect to an external device, and the first processor is used to collect a fault log of the connected external device;

[0094] The data mining processing unit is connected to the first processor via a second communication link, obtains the fault log of the external device via the first processor, and analyzes the fault log to determine the fault occurrence time of the external device.

[0095] In an embodiment of the present application, the first processor is used to connect to external devices. When an external device fails, the first processor can generate fault logs for each connected external device and transmit the fault logs to a data mining processing unit to facilitate the data mining processing unit to accurately determine the time when the fault occurred.

[0096] In an optional embodiment, the monitoring unit proposed in the present application may be an integrated TSOM module, such as Figure 1 As shown, the module is internally integrated with a microcontroller unit and multiple sensors, and the current environmental information is monitored through the information of each sensor, and can be sent to the data mining processing unit through I2C communication, and a power supply module is integrated to power the TSOM module to ensure continuous monitoring of the environmental conditions during the power-off process of the server; in an optional embodiment, the first processor proposed in the present application may be a PCIe Switch chip, and the connected external devices may be multiple PCIe devices, and the external devices include but are not limited to graphics cards, network cards, sound cards, storage controllers, etc. When a PCIe device connected thereto fails, the fault information can be sent to the data mining processing unit through UART communication; in an optional embodiment, the data mining processing unit proposed in the present application may be a DMPU module, which can integrate and process the various information collected by it, and when the fault information is received, the various information at the time of the fault is integrated to facilitate the troubleshooting personnel to view.

[0097] In combination with the above embodiments, in one implementation, the present application further provides a fault handling system, which further includes:

[0098] A second processor is used to monitor the voltage abnormality record of the external device;

[0099] The data mining processing unit is connected to the second processor via a third communication link, obtains the voltage anomaly records of the external device from the second processor, and screens out the voltage anomaly records of the external device within the target time period.

[0100] In the embodiment of the present application, the server also includes a second processor, which can monitor the voltage conditions of various external devices. When a voltage value is found to be different from the standard, the abnormal voltage value and the voltage monitoring conditions in the target time period before and after the abnormality are sent to the data mining processing unit to ensure that possible abnormal phenomena are recorded in real time for reference by subsequent troubleshooting personnel. In addition, when the above-mentioned fault phenomenon occurs, the voltage monitoring conditions in the target time period will also be sent to the data mining processing unit for reference by subsequent troubleshooting personnel.

[0101] In an optional embodiment, the second processor proposed in the present application can be an ADC module, which is used to monitor the power supply status of the PCIe device. When a power supply abnormality is detected, the voltage status 5 seconds before and after the abnormal time point can be recorded and sent to the DMPU module through the SPI bus for reference in subsequent troubleshooting.

[0102] In combination with the above embodiments, in one implementation, the present application further provides a fault handling system, which further includes:

[0103] The second processor is further used to monitor the voltage abnormality record of the first processor;

[0104] The data mining processing unit is further configured to obtain voltage anomaly records of the first processor from the second processor, and filter out voltage anomaly records of the first processor within the target time period.

[0105] In an embodiment of the present application, the second processor is not only used to monitor the voltage condition of the external device, but also can monitor the voltage condition of the first processor. Once there is an abnormal voltage condition in the first processor, the second processor will also send the voltage records in the target time period before and after the abnormal moment to the data mining processing unit for reference by subsequent troubleshooting personnel.

[0106] In combination with the above embodiments, in one implementation, the present application further provides a fault handling system, which further includes:

[0107] Complex programmable logic devices for monitoring timing signals and alarm signals;

[0108] The data mining processing unit is connected to the complex programmable logic device via a fourth communication link, obtains the timing signal and the alarm signal from the complex programmable logic device, filters out the timing signal and the alarm signal within the target time period, and marks timestamps for the filtered timing signal and the alarm signal.

[0109] In the embodiment of the present application, the complex programmable logic device is used to monitor the timing signal and the alarm signal. When a fault occurs, the complex programmable logic device will also send the monitored timing signal and the alarm signal within the target time period to the data mining processing unit, and attach the timestamp of the corresponding moment to ensure the consistency of the information in the timing. Among them, when the level of the received alarm signal changes, the complex programmable logic device will also send it to the data mining processing unit to ensure that the alarm information is recorded in the data mining processing unit for reference in subsequent troubleshooting.

[0110] In an optional embodiment, the complex programmable logic device proposed in the present application can be a mainboard CPLD, which is used to monitor the timing signals and alarm signals of other devices in the board, such as the monitoring of the VR chip in the board. In some fault problems, the faults may be caused by other devices and manifested in the PCIe device. Therefore, by obtaining and recording the monitoring information of the mainboard CPLD, it is ensured that the troubleshooting personnel can take all aspects of the fault problem into consideration.

[0111] In combination with the above embodiments, in one implementation, the present application further provides a fault handling system, which further includes:

[0112] A third processor;

[0113] The data mining processing unit is connected to the third processor via the fifth communication link, and obtains the error record generated by the third processor, and analyzes the error record to determine the time when the error occurs, and filters out the environmental information of the server within the error time period corresponding to the time when the error occurs, and the error time period includes a first time period before the time when the error occurs and / or a second time period after the time when the error occurs.

[0114] In the embodiment of the present application, the third processor serves as the calculation and control core of the system and is the final execution unit for information processing and program running. In the event of certain faults, some error records may be generated in the third processor. The data mining processing unit also needs to collect the error records, analyze the time when the error occurred, and collect the information recorded by each module in the server in the target time period when the error occurred, including but not limited to the environmental information and power supply information mentioned above. The data mining processing unit collects and integrates various information in the target time period for reference by subsequent troubleshooting personnel.

[0115] In an optional embodiment, the third processor proposed in the present application may be a CPU, which may generate error records of some devices. The CPU sends the error records to the DMPU module to record various information when the error occurs in the DMPU module.

[0116] In combination with the above embodiments, in one implementation, the present application further provides a fault handling system, which further includes:

[0117] The data mining processing unit includes a memory card for storing at least one of the following:

[0118] Environmental information of the server within a target time period corresponding to the time when the fault occurs;

[0119] Environmental information of the server within a target time period corresponding to the time when the error occurs;

[0120] abnormal voltage record of the first processor within the target time period;

[0121] The voltage abnormality record of the external device within the target time period;

[0122] The timing signal and the alarm signal within the target time period.

[0123] In an embodiment of the present application, there is at least one storage card in the data mining processing unit, which is used to integrate and store the information sent to the data mining processing unit from the various modules mentioned above, including but not limited to: environmental information, voltage anomaly records, timing signals and alarm signals, so that subsequent troubleshooting personnel can view them. This is also the part that the present application has improved compared with previous methods. Since there are records of various information in the target time period corresponding to the fault time or error reporting time, the troubleshooting personnel can understand the environment in which the server is located, the status of each device, and whether there are any abnormal conditions other than the fault phenomenon when viewing the information. This kind of fault integration information that can be traced back greatly improves the work efficiency of the troubleshooting personnel and avoids the problem of inaccurate fault location and untimely fault handling due to the inability to understand the actual situation at the scene when the fault occurs after the fault occurs in the previous methods.

[0124] For example, Figure 2 As shown, the DMPU module receives and records: environmental information in the TSOM module, voltage anomaly records in the ADC module, timing signals and alarm signals in the CPLD, error records in the CPU, and fault information issued by the PCIe Switch. After collecting and integrating various information, it is sent to the BMC module for display through an external window.

[0125] In combination with the above embodiments, in one implementation, the present application further provides a fault handling system, which further includes:

[0126] Management controller;

[0127] The data mining processing unit is connected to the management controller via a sixth communication link, and the management controller receives the environment information of the server in the target time period corresponding to the time when the fault occurs, which is sent by the data mining processing unit.

[0128] In an embodiment of the present application, the management controller is connected to the data mining processing unit. Since the data mining processing unit only collects, integrates and stores information but does not display it, the management controller can receive and display various server information within the target time period recorded by the data mining processing unit.

[0129] In an optional embodiment, the management controller proposed in the present application may be a BMC, such as Figure 3 As shown, as out-of-band management of the server, in the previous technology, the board temperature, ADC voltage status, etc. are also monitored. In the embodiment of the present application, it is also connected to the monitoring unit, the second processor and the mining processing unit. The environmental information monitored by the monitoring unit is sent to the mining processing unit and the management controller through the multiplexer, but the management controller only performs simple recording and display, and does not collect and integrate various information according to the target time period.

[0130] In combination with the above embodiments, in one implementation, the present application further provides a fault handling system, which further includes:

[0131] A second processor is used to monitor the voltage abnormality record of the external device and monitor the voltage abnormality record of the first processor;

[0132] The data mining processing unit is connected to the second processor via the third communication link, obtains the voltage abnormality record of the external device from the second processor, and filters out the voltage abnormality record of the external device within the target time period, and obtains the voltage abnormality record of the first processor from the second processor, and filters out the voltage abnormality record of the first processor within the target time period;

[0133] The management controller is connected to the monitoring unit via the first communication link, and is connected to the second processor via a seventh communication link, and the management controller is used for at least one of the following:

[0134] Acquire the environment information of the server from the monitoring unit and display it through the external window of the management controller;

[0135] Acquire the voltage abnormality record of the external device from the second processor and display it through the external window of the management controller;

[0136] Acquire the voltage abnormality record of the first processor from the second processor and display it through the external window of the management controller;

[0137] The information stored in the memory card of the data mining processing unit is read and displayed through the external window of the management controller.

[0138] In the embodiment of the present application, the management controller also receives information related to the monitoring unit and the second processor, but as described above, the information is not integrated and processed. However, the management controller can display the acquired information through an external display window, including but not limited to: environmental information monitored by the monitoring unit, voltage records acquired by the second processor, integrated fault information stored by the data mining processing unit, etc. The external display of the management controller can help troubleshooting personnel to view various information more intuitively.

[0139] In the embodiment of the present application, the information recorded in the data mining processing unit includes but is not limited to the situation described above in the present application, that is, when a fault log or an error record is received, various information within the target time period before and after it is recorded. According to the different needs of users, the recording method can be adjusted. For example, various information can be periodically recorded, integrated and stored according to a certain specified time, or continuous monitoring and recording can be performed for some time periods with a higher probability of failure, such as power-on and power-off processes. Similarly, for the various monitoring contents described above, corresponding adjustments can be made according to the actual use environment. For example, for storage servers and computing servers, the DMPU module can be biased towards storage modules and computing modules for corresponding targeted supervision. For those of ordinary skill in the art, according to the ideas of this application, there may be changes in the specific recording methods and monitoring contents.

[0140] In the embodiments of the present application, the contents recorded by the data mining processing unit can be viewed not only through a display window external to the management controller as described above, but also by reading the internal information of the data mining processing unit through a data interface, or by removing the internal storage card of the data mining processing unit and directly reading the contents stored in the storage card. According to the concept of the present application, there may be changes in the specific information viewing method.

[0141] Based on the same design concept, another embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps in the fault handling method described in any of the above embodiments of the present application are implemented.

[0142] Based on the same design concept, another embodiment of the present application provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the steps in the fault handling method described in any of the above embodiments of the present application.

[0143] Based on the same design concept, another embodiment of the present application provides an electronic device, such as Figure 4 shown. Figure 41 is a schematic diagram of an electronic device shown in an embodiment of the present application. The electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the steps of the fault handling method described in any of the above embodiments of the present application when executing the computer program.

[0144] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0145] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, devices, or computer program products. Therefore, the embodiments of the present application may adopt the form of complete hardware embodiments, complete software embodiments, or embodiments in combination with software and hardware. Moreover, the embodiments of the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0146] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0147] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0148] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0149] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present application.

[0150] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0151] The above is a detailed introduction to a fault handling system, method, device and medium provided by the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for general technical personnel in this field, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A fault handling system, comprising: Data mining processing unit and monitoring unit; The monitoring unit is used to monitor the environment information of the server; The data mining processing unit determines the time of occurrence of a fault of the external device that has a fault; The data mining processing unit is connected to the monitoring unit via a first communication link, obtains the environmental information of the server from the monitoring unit, and filters out the environmental information of the server within a target time period corresponding to the time when the fault occurs, wherein the target time period includes a first time period before the time when the fault occurs and / or a second time period after the time when the fault occurs.

2. The fault handling system according to claim 1, characterized in that: Also includes: A first processor is used to connect to an external device, and the first processor is used to collect a fault log of the connected external device; The data mining processing unit is connected to the first processor via a second communication link, obtains the fault log of the external device via the first processor, and analyzes the fault log to determine the fault occurrence time of the external device.

3. The fault handling system according to claim 1, characterized in that: Also includes: A second processor is used to monitor the voltage abnormality record of the external device; The data mining processing unit is connected to the second processor via a third communication link, obtains the voltage anomaly records of the external device from the second processor, and screens out the voltage anomaly records of the external device within the target time period.

4. The fault handling system according to claim 3, characterized in that: Also includes: The second processor is further used to monitor the voltage abnormality record of the first processor; The data mining processing unit is further configured to obtain voltage anomaly records of the first processor from the second processor, and filter out voltage anomaly records of the first processor within the target time period.

5. The fault handling system according to claim 1, characterized in that: Also includes: Complex programmable logic devices for monitoring timing signals and alarm signals; The data mining processing unit is connected to the complex programmable logic device via a fourth communication link, obtains the timing signal and the alarm signal from the complex programmable logic device, filters out the timing signal and the alarm signal within the target time period, and marks timestamps for the filtered timing signal and the alarm signal.

6. The fault handling system according to claim 1, characterized in that: Also includes: A third processor; The data mining processing unit is connected to the third processor via the fifth communication link, and obtains the error record generated by the third processor, and analyzes the error record to determine the time when the error occurs, and filters out the environmental information of the server within the error time period corresponding to the time when the error occurs, and the error time period includes a first time period before the time when the error occurs and / or a second time period after the time when the error occurs.

7. The fault handling system according to claim 6, characterized in that: The data mining processing unit includes a memory card for storing at least one of the following: Environmental information of the server within a target time period corresponding to the time when the fault occurs; Environmental information of the server within a target time period corresponding to the time when the error occurs; abnormal voltage record of the first processor within the target time period; The voltage abnormality record of the external device within the target time period; The timing signal and the alarm signal within the target time period.

8. The fault handling system according to claim 1, characterized in that: Also includes: Management controller; The data mining processing unit is connected to the management controller via a sixth communication link, and the management controller receives the environment information of the server in the target time period corresponding to the time when the fault occurs, which is sent by the data mining processing unit.

9. The fault handling system according to claim 8, characterized in that: Also includes: A second processor is used to monitor the voltage abnormality record of the external device and monitor the voltage abnormality record of the first processor; The data mining processing unit is connected to the second processor via the third communication link, obtains the voltage abnormality record of the external device from the second processor, and filters out the voltage abnormality record of the external device within the target time period, and obtains the voltage abnormality record of the first processor from the second processor, and filters out the voltage abnormality record of the first processor within the target time period; The management controller is connected to the monitoring unit via the first communication link, and is connected to the second processor via a seventh communication link, and the management controller is used for at least one of the following: Acquire the environment information of the server from the monitoring unit and display it through the external window of the management controller; Acquire the voltage abnormality record of the external device from the second processor and display it through the external window of the management controller; Acquire the voltage abnormality record of the first processor from the second processor and display it through the external window of the management controller; The information stored in the memory card of the data mining processing unit is read and displayed through the external window of the management controller.

10. A fault handling method, characterized in that: The method comprises: Monitor the server's environmental information; The environment information of the server is obtained, and the environment information of the server in a target time period corresponding to the time when the fault occurs is filtered out, wherein the target time period includes a first time period before the time when the fault occurs and / or a second time period after the time when the fault occurs.

11. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the computer program is executed by the processor, the fault handling method according to claim 10 is implemented.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the fault handling method according to claim 10 is implemented.

Images