Server abnormality processing method, program product, device and storage medium
By performing noise reduction processing and self-recovery operations on the server electrical signals, the problems of mislocking the motherboard and complex fault location in the existing technology are solved, and more efficient exception handling and self-recovery are achieved, reducing the risk of business interruption.
Patent Information
- Application Number
- CN202411104729.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-13
- Publication Date
- 2024-11-08
- Estimated Expiration
- 2044-08-13
AI Technical Summary
The existing server exception handling technology is complex, the probability of accidentally locking the motherboard is high, and the failure cannot be quickly located, resulting in business interruption, and the self-recovery mechanism is insufficient, which cannot effectively reduce the occurrence of no-warning shutdown.
By monitoring the server's electrical signals, performing noise reduction processing, determining abnormal conditions, and performing self-recovery operations based on preset self-recovery rules. If the abnormal electrical signals return to normal, continue monitoring, otherwise shutdown will be locked and logged for subsequent analysis.
It reduces the probability of accidental locking motherboards, improves the convenience of abnormal handling, reduces the risk of shutdown without warning, and improves the efficiency of fault location and processing.
Smart Images

Figure CN118642881B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of servers, and in particular to a method, program product, device and storage medium for processing server exceptions. Background Art
[0002] CPLD (Complex Programmable Logic Device) is an integrated circuit that can be programmed to implement specific logic functions. The CPLD of some server motherboards can be divided into two parts during the motherboard design process. The first part is the peripheral function module, which includes functions such as monitoring CPU (Central Processing Unit) serious errors, CPU overheating, fan control, etc., and plays an auxiliary role in the normal operation of the server. The second part is to control the power-on and power-off timing of the motherboard.
[0003] When the server is running normally, the CPLD will monitor various electrical signals of the server. Any abnormality will cause the CPLD to lock the mainboard, causing the server to shut down abnormally. The power button on the front panel will automatically change from green to yellow to indicate shutdown, and pressing the power button again will not restart the server. At this time, the server must be reconnected to the AC power plug, that is, the AC must be disconnected and reconnected. The CPLD will re-detect whether all electrical signals are normal before restarting.
[0004] At present, the above situation indicates that the electrical signal on the server motherboard is abnormal, causing the CPLD to lock the motherboard. In some designs, the CPLD will display the abnormal information of the motherboard electrical signal through the debug LED (fault light) through the arrangement and combination of lights. Engineers need to determine the name of the abnormal electrical signal based on the debug LED on the motherboard and the LED display mapping table of the corresponding model. The operation is complicated in actual application, especially some motherboards are equipped with large devices such as GPU (Graphics Processing Unit, graphics processor), which will cause a certain degree of obstruction to the debug LED. In some other solutions, the CPLD will send the abnormal electrical signal to the BMC (Board Management Controller) through I2C (Inter-Integrated Circuit, two-wire serial bus), and then the on-site engineer will use the I2C test tool under the BMC to send a command request, read the relevant abnormal electrical signal information and then process it, and the operation complexity still exists.
[0005] Moreover, in the current solution, if any electrical signal has a slight abnormality, such as the temperature of a certain memory is lower than the set value for a short period of time, whether it is a false alarm or a real low temperature, the CPLD will directly lock the motherboard, causing the server to automatically shut down and be unable to boot up, which will cause the customer's business deployed on the server to be suddenly interrupted without warning. At this time, the cause of the fault cannot be directly seen in the BMC's onekey or operating system log, and it cannot be quickly located. It relies on the collection of debug information after reproduction in the same environment or in the laboratory, which is not conducive to the rapid location and resolution of the problem.
[0006] In summary, how to effectively handle server exceptions, reduce the probability of incorrectly locking the motherboard, and improve the convenience of exception handling are technical problems that technical personnel in this field urgently need to solve. Summary of the invention
[0007] The purpose of the present invention is to provide a method, program product, device and storage medium for processing server exceptions, so as to effectively implement the processing of server exceptions, reduce the probability of incorrectly locking the motherboard, and improve the convenience of exception processing.
[0008] In order to solve the above technical problems, the present invention provides the following technical solutions:
[0009] In a first aspect, the present invention provides a method for processing a server exception, comprising:
[0010] Monitor each specified electrical signal in the server;
[0011] When any monitored electrical signal is abnormal, the abnormal electrical signal is denoised to obtain a denoising result;
[0012] Determining whether the noise reduction result is still abnormal;
[0013] If the noise reduction result is still abnormal, a self-recovery operation is performed based on the abnormal electrical signal and a preset self-recovery rule;
[0014] After the self-recovery operation is performed, it is determined whether the abnormal electrical signals have returned to normal;
[0015] If it returns to normal, returning to execute the operation of each designated electrical signal in the monitoring server;
[0016] If it does not return to normal, the server is locked and shut down.
[0017] On the other hand, when any monitored electrical signal is abnormal, the abnormal electrical signal is subjected to noise reduction to obtain a noise reduction result, including:
[0018] When any monitored electrical signal is abnormal, a sampling window and a filtering window of the electrical signal are established;
[0019] The value obtained by subtracting the length of the filtering window from the length of the sampling window is used as the filtering value n; wherein n is an even number;
[0020] Placing the monitored data sample points of the electrical signal into the sampling window according to the first-in-first-out principle;
[0021] When the sampling window is not filled with data sample points and a statistical operation is triggered, the average value of each data sample point in the current sampling window is used as the statistical value obtained by this statistical operation;
[0022] When the sampling window is filled with data sample points and a statistical operation is triggered, the average value of each data sample point in the current sampling window, excluding n / 2 maximum values and n / 2 minimum values, is used as the statistical value obtained by this statistical operation;
[0023] Each statistical value obtained when each statistical operation is triggered is used as the noise reduction result.
[0024] On the other hand, determining whether the noise reduction result is still abnormal includes:
[0025] Determine whether the total number of abnormalities in each statistical value in the noise reduction result is lower than a first threshold;
[0026] If it is lower than the first threshold, it is determined that the noise reduction result is not abnormal;
[0027] If it is not lower than the first threshold, determining whether the latest k statistical values in the noise reduction result are all normal;
[0028] If all are normal, it is determined that the noise reduction result is normal;
[0029] If not all are normal, it is determined that the noise reduction result is still abnormal.
[0030] On the other hand, based on the abnormal electrical signal and the preset self-recovery rules, a self-recovery operation is performed, including:
[0031] The information of the abnormal electrical signal is written into the corresponding register, so that the baseboard management controller performs a self-recovery operation after reading the value of the corresponding register.
[0032] On the other hand, the baseboard management controller performs a self-recovery operation after reading the corresponding register, including:
[0033] When the abnormal electrical signal belongs to the first type of electrical signal, information of the abnormal electrical signal is written into the first register, so that the baseboard management controller performs a self-recovery operation after reading the value of the first register;
[0034] When the abnormal electrical signal belongs to the second type of electrical signal, information of the abnormal electrical signal is written into the second register, so that the baseboard management controller performs a self-recovery operation after reading the value of the second register;
[0035] When the abnormal electrical signal belongs to the third type of electrical signal, information of the abnormal electrical signal is written into a third register, so that the baseboard management controller performs a self-recovery operation after reading the value of the third register;
[0036] Among them, the first type of electrical signal is an electrical signal pre-classified as a voltage drop type, the second type of electrical signal is an electrical signal pre-classified as a component power supply abnormality type, and the third type of electrical signal is an electrical signal pre-classified as a temperature abnormality type.
[0037] On the other hand, the baseboard management controller performs a self-recovery operation after reading the value of the first register, including:
[0038] When the baseboard management controller determines that the value of the first register is not a first default value, the power supply of the server is switched to a backup power supply for self-recovery.
[0039] On the other hand, the baseboard management controller performs a self-recovery operation after reading the value of the second register, including:
[0040] When the baseboard management controller determines that the value of the second register is not a second default value, determining whether an automatic restart function is enabled;
[0041] If yes, automatically restart the server to perform self-recovery;
[0042] If not, an alarm is sent, and when no instruction to automatically restart the server is received within the first time period, the server is locked and shut down.
[0043] On the other hand, the baseboard management controller performs a self-recovery operation after reading the value of the third register, including:
[0044] When the baseboard management controller determines that the value of the third register is not the third default value, the baseboard management controller adjusts the speed of the corresponding fan to perform self-recovery based on the value of the third register and in accordance with the set fan speed adjustment rule.
[0045] On the other hand, it also includes:
[0046] The baseboard management controller adjusts the opening degree of the front window and the rear window of the server based on the value of the third register and in accordance with the set front and rear window control rules, so as to perform self-recovery.
[0047] On the other hand, after determining that the abnormal electrical signals have not all returned to normal, before locking and shutting down the server, the method further includes:
[0048] Record the electrical signal information including the name, meaning and reporting time of the abnormal electrical signal in the system event log;
[0049] An independent automatic shutdown log file is created in a designated directory of the system, and associated information of the abnormal electrical signal is recorded in the automatic shutdown log file.
[0050] On the other hand, the associated information of abnormal electrical signals includes:
[0051] Serial port log of the baseboard management controller, fan speed and control parameter information, and log information of components related to abnormal electrical signals.
[0052] On the other hand, after determining that the abnormal electrical signals have returned to normal and returning to execute the operations of each designated electrical signal in the monitoring server, it also includes:
[0053] Record the electrical signal information including the name, meaning and reporting time of the abnormal electrical signal in the system event log;
[0054] An independent self-recovery log file is created in a designated directory of the system, and self-recovery related information including abnormal recovery time, self-recovery measure name, and register value corresponding to the abnormal electrical signal is recorded in the self-recovery log file.
[0055] In a second aspect, the present invention provides a computer program product, including a computer program / instruction, which implements the steps of the above-mentioned method for handling server exceptions when executed by a processor.
[0056] In a third aspect, the present invention provides a server exception processing device, comprising:
[0057] Memory for storing computer programs;
[0058] A processor is used to execute the computer program to implement the steps of the method for handling server exceptions as described above.
[0059] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of the method for handling server exceptions as described above are implemented.
[0060] By applying the technical solution provided by the embodiment of the present invention, after monitoring each designated electrical signal in the server, considering the existence of false alarms, when any monitored electrical signal is abnormal, the abnormal electrical signal will be subjected to noise reduction to obtain a noise reduction result, which is also conducive to reducing the probability of the server motherboard being locked by mistake in the present application solution. If the noise reduction result is still abnormal after noise reduction, the present application solution does not lock and shut down the server, but performs a self-recovery operation based on the abnormal electrical signal and the preset self-recovery rules. If it recovers to normal, it means that it is not a serious fault and has been processed, so it can return to the operation of monitoring each designated electrical signal in the server. Therefore, it can be seen that the present application solution improves the convenience of exception handling through the self-recovery design. Of course, if it does not recover to normal, it is necessary to lock and shut down the server.
[0061] In summary, the solution of the present application can effectively implement the handling of server exceptions, reduce the probability of incorrectly locking the motherboard, and improve the convenience of exception handling. BRIEF DESCRIPTION OF THE DRAWINGS
[0062] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0063] Figure 1 A flowchart of a method for handling server exceptions provided in a specific embodiment of the present invention;
[0064] Figure 2 A schematic diagram of the principle of obtaining a noise reduction result in a specific implementation manner of the present invention;
[0065] Figure 3 A schematic diagram of the structure of a server abnormality processing device provided in a specific embodiment of the present invention;
[0066] Figure 4 The present invention is a schematic diagram of the structure of a computer-readable storage medium. DETAILED DESCRIPTION
[0067] The core of the present invention is to provide a method, program product, device and storage medium for processing server exceptions, which can effectively implement the processing of server exceptions, reduce the probability of incorrectly locking the motherboard, and improve the convenience of exception processing.
[0068] In order to enable those skilled in the art to better understand the scheme of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific implementation methods. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.
[0069] Please refer to Figure 1 , Figure 1 The present invention provides a flowchart of a method for handling server exceptions in accordance with a specific embodiment of the present invention. The method for handling server exceptions may include the following steps:
[0070] Step S101: monitor each designated electrical signal in the server.
[0071] Specifically, CPLD is an integrated circuit that can be programmed to implement specific logic functions, and current servers are usually equipped with CPLDs for monitoring electrical signals. Therefore, the solution of the present application can usually be implemented directly based on CPLD. Of course, in other specific implementations, devices such as FPGA (Field Programmable Gate Array) can be used as needed to implement the solution of the present application, which does not affect the implementation of the present application.
[0072] When monitoring each designated electrical signal in the server, it is usually monitored continuously or at a certain period. Of course, for different electrical signals, the respective monitoring periods can be set and adjusted accordingly according to actual needs, which does not affect the implementation of the present invention. In addition, the number of specific electrical signals monitored can also be set and adjusted according to actual needs, for example, it can include the Power Good electrical signal on the motherboard, the Normal Power Fail electrical signal of each component (CPU, memory and other components power supply failure electrical signal), the electrical signal related to temperature, the 12V voltage failure electrical signal, etc.
[0073] Step S102: when any monitored electrical signal is abnormal, the abnormal electrical signal is subjected to noise reduction to obtain a noise reduction result.
[0074] It should be noted that, considering the cost and space requirements of hardware circuit filtering for the circuit, and the frequency of false alarms of electrical signals will not be too high, the solution of the present application is to reduce the noise of the abnormal electrical signal after detecting any abnormal electrical signal to obtain a noise reduction result. In other words, the solution of the present application adopts a software filtering solution, which is also called digital filtering, and weakens the influence of noise through a certain algorithm. Of course, when reducing the noise of abnormal electrical signals, the specific algorithm used can be set according to actual needs.
[0075] For example, in a specific implementation of the present invention, step S102 may specifically include:
[0076] When any monitored electrical signal is abnormal, a sampling window and a filtering window of the electrical signal are established;
[0077] The value obtained by subtracting the length of the filter window from the length of the sampling window is used as the filter value n; wherein n is an even number;
[0078] The data sample points of the monitored electrical signal are placed into the sampling window according to the first-in-first-out principle;
[0079] When the sampling window is not filled with data sample points and a statistical operation is triggered, the average value of each data sample point in the current sampling window is used as the statistical value obtained by this statistical operation;
[0080] When the sampling window is filled with data sample points and the statistical operation is triggered, the average value of each data sample point in the current sampling window, excluding the n / 2 maximum values and n / 2 minimum values, is used as the statistical value obtained by this statistical operation;
[0081] Each statistical value obtained when each statistical operation is triggered is used as the noise reduction result.
[0082] In this implementation, a sliding window filtering algorithm is specifically used to reduce noise on the monitored abnormal electrical signals.
[0083] First, in this implementation, when any monitored electrical signal is abnormal, it is necessary to establish a sampling window and a filtering window for the electrical signal. For easier understanding, please refer to Figure 2 ,and Figure 2 In the implementation mode of , the length of the sampling window is 7, and the length of the filtering window is 5. In addition, it should be noted that for different types of electrical signals, the lengths of the established sampling windows and filtering windows can be adaptively set. That is, if multiple electrical signal abnormalities are detected, each electrical signal can be digitally filtered according to the principle of this implementation mode to obtain a noise reduction result for the electrical signal, but the lengths of the sampling windows and filtering windows used do not have to be exactly the same.
[0084] After the sampling window and filtering window of the electrical signal are established, the length of the sampling window is subtracted from the length of the filtering window, and the resulting value is the filtering value n. Figure 2 In the example, 7-5=2, then Figure 2 In the example, the filter value n=2.
[0085] Afterwards, the data sample points of the monitored electrical signal are placed into the sampling window according to the first-in-first-out principle, for example Figure 2 In the example, data sample point ① is first placed in the sampling window, then data sample point ② is placed in the sampling window, then data sample point ③ is placed in the sampling window, and then data sample point ④ is placed in the sampling window.
[0086] When a statistical operation is triggered, it is necessary to determine whether the current sampling window is filled with data sample points. Figure 2 In the example, when the statistical operation is triggered for the first time, data sample points ① to ④ are stored in the current sampling window, and the sampling window is not filled with data sample points. At this time, the average value of data sample points ① to ④ can be used as the statistical value obtained by this statistical operation.
[0087] For example, Figure 2 In the example, when the statistical operation is triggered for the second time, data sample points ① to ⑦ are stored in the current sampling window, and the sampling window is filled with data sample points. Since n=2 in this example, one maximum value and one minimum value in the current sampling window can be removed, that is, after removing the maximum value and minimum value from data sample points ① to ⑦, the average value of the remaining 5 data sample points is used as the statistical value obtained by this statistical operation.
[0088] Afterwards, as new data sample points arrive, the earliest data sample point ① will be removed, for example, Figure 2 In the example, when the statistical operation is triggered for the third time, data sample points ② to ⑧ are stored in the current sampling window. The sampling window is filled with data sample points, and one maximum value and one minimum value in the current sampling window can be removed. That is, after removing the maximum value and the minimum value from data sample points ② to ⑧, the average value of the remaining 5 data sample points is used as the statistical value obtained by this statistical operation.
[0089] It should also be noted that the timing of triggering the statistical operation can be set as needed. For example, in one case, it is considered that during the execution of step S102, the statistical operation in this embodiment can be periodically triggered. For another example, in one case, it is considered that when the data sample point in the sampling window reaches the starting value a, a statistical operation is triggered, and then each time a data sample point is stored in the sampling window, a statistical operation is triggered. The setting of this embodiment is considered that when the amount of data in the sampling window is small, the obtained statistical value is not easy to filter out interference such as burrs, so there is no need to trigger the statistical operation at this time. After the data sample point in the sampling window reaches the starting value a, it means that there is a certain amount of data in the sampling window, and a statistical operation can be triggered. a is a preset positive integer, and it can be understood that the value of a should be lower than the length of the sampling window. After that, each time a data sample point is stored in the sampling window, a statistical operation is triggered. For example, in actual applications, when the obtained statistical value reaches a certain number, it can be determined that the noise reduction result for the abnormal electrical signal is obtained, and then subsequent operations are performed.
[0090] Step S103: Determine whether the noise reduction result is still abnormal.
[0091] It is understandable that, for any electrical signal, under normal circumstances, the electrical signal has a normal value range, which cannot be too high or too low. Accordingly, after performing noise reduction on the abnormal electrical signal and obtaining the noise reduction result, interference such as burrs should be filtered out to obtain a normal noise reduction result. For example, in a specific implementation, when all values in the noise reduction result are within the normal range, it can be determined that the noise reduction result is normal, otherwise it is determined that the noise reduction result is still abnormal.
[0092] If it is determined that the noise reduction result is still abnormal, a self-recovery operation can be performed subsequently. If it is determined that the noise reduction result is normal, the operation of step S101 can be returned to continue monitoring each designated electrical signal in the server.
[0093] In a specific implementation of the present invention, step S103 may include:
[0094] Determine whether the total number of abnormalities in each statistical value in the noise reduction result is lower than a first threshold;
[0095] If it is lower than the first threshold, it is determined that the noise reduction result is not abnormal;
[0096] If it is not lower than the first threshold, determining whether the latest k statistical values in the denoising result are all normal;
[0097] If all are normal, it is determined that the noise reduction result is normal;
[0098] If not all are normal, it is determined that the noise reduction result is still abnormal.
[0099] This implementation takes into account that after each statistical operation is triggered to obtain each statistical value, if the total number of abnormalities in each statistical value in the noise reduction result is lower than the first threshold, it means that after noise reduction, the obtained statistical values are generally normal values, with few abnormal values, so it can be considered that the noise reduction result is not abnormal. Of course, the first threshold cannot be set too high, for example, it can be set to 1 or 2.
[0100] If the total number of abnormalities in each statistical value in the noise reduction result is not less than the first threshold, in order to avoid misjudgment, this implementation will further determine whether the latest k statistical values generated in the noise reduction result are all normal. If they are all normal, it means that although the number of abnormal statistical values in the noise reduction result is not less than the first threshold, the latest k statistical values are normal, that is, the abnormal electrical signal is likely not to continue to appear but has returned to a normal value. Therefore, it can be determined that the noise reduction result is not abnormal. On the contrary, if the latest k statistical values in the noise reduction result are not all normal, this implementation can determine that the noise reduction result is still abnormal.
[0101] Step S104: If the noise reduction result is still abnormal, a self-recovery operation is performed based on the abnormal electrical signal and a preset self-recovery rule.
[0102] The present application takes into account that when the noise reduction result is still abnormal, it means that there is indeed an abnormal electrical signal, rather than a false detection caused by interference. However, for some abnormal situations, they can be handled through self-recovery operations so that the electrical signal can return to normal without having to directly lock and shut down the server, which effectively reduces the probability of unwarned server interruptions.
[0103] When performing a self-recovery operation based on an abnormal electrical signal and a preset self-recovery rule, there may be a variety of specific implementation schemes. Usually, an adaptive self-recovery operation may be set according to different types of electrical signals.
[0104] In a specific implementation of the present invention, step S104 may include:
[0105] The information of the abnormal electrical signal is written into the corresponding register, so that the baseboard management controller performs a self-recovery operation after reading the value of the corresponding register.
[0106] This implementation takes into account that BMC has many functions, and it is more convenient to use BMC for self-recovery operation. Therefore, CPLD can, for example, write the information of abnormal electrical signals into the corresponding register through the I2C channel, and BMC can perform self-recovery operation after reading the value of the corresponding register.
[0107] Further, in a specific implementation of the present invention, the baseboard management controller performs a self-recovery operation after reading the corresponding register, which may specifically include:
[0108] When the abnormal electrical signal belongs to the first type of electrical signal, information of the abnormal electrical signal is written into the first register, so that the baseboard management controller performs a self-recovery operation after reading the value of the first register;
[0109] When the abnormal electrical signal belongs to the second type of electrical signal, information of the abnormal electrical signal is written into the second register, so that the baseboard management controller performs a self-recovery operation after reading the value of the second register;
[0110] When the abnormal electrical signal belongs to the third type of electrical signal, information of the abnormal electrical signal is written into the third register, so that the baseboard management controller performs a self-recovery operation after reading the value of the third register;
[0111] Among them, the first type of electrical signal is an electrical signal pre-classified as a voltage drop type, the second type of electrical signal is an electrical signal pre-classified as a component power supply abnormality type, and the third type of electrical signal is an electrical signal pre-classified as a temperature abnormality type.
[0112] This implementation takes into account that there are many types of electrical signals. If a corresponding self-recovery method is set for each electrical signal, it will be more complicated, and for some electrical signals, the same self-recovery method can be used. Therefore, this implementation takes into account that the electrical signals can be divided into three major categories, namely the first type, the second type and the third type, so that three self-recovery methods can be set. And it can be understood that for the first type, the second type and the third type, the specific types of electrical signals included can be set according to actual needs, that is, for any one electrical signal, it can be divided into the first type or the second type or the third type according to actual needs. Of course, in other specific implementations, the electrical signals can be divided into more detailed major categories as needed.
[0113] In actual applications, the BMC can usually poll the first register, the second register, and the third register, so as to perform a self-recovery operation when the value of any register is not the corresponding default value.
[0114] As described above, the first type of electrical signal is an electrical signal that is pre-classified as a voltage drop type, i.e., a power drop type. When the abnormal electrical signal belongs to the first type of electrical signal, the information of the abnormal electrical signal is written into the first register, and the BMC can perform a self-recovery operation after reading the value of the first register. Of course, the specific self-recovery operation can be set according to actual needs. For example, in a specific embodiment of the present invention, the baseboard management controller performs a self-recovery operation after reading the value of the first register, which can specifically include:
[0115] When the baseboard management controller determines that the value of the first register is not the first default value, the power supply of the server is switched to the backup power supply for self-recovery.
[0116] This implementation takes into account that the first default value of the first register is, for example, 0x00. In the case of power drop, the first register is a non-zero value. At this time, the most likely fault direction is that the server power supply is abnormal, and a feasible solution is that the BMC sends a command to the server power supply through the I2C channel to switch the server power supply to the backup power supply. It should be noted that most server power supply designs currently support 1+1 redundancy to avoid the situation where the main power supply problem affects the server business operation.
[0117] The second type of electrical signal is an electrical signal that is pre-classified as a component power supply abnormality type, that is, a Powergood type. When the abnormal electrical signal belongs to the second type of electrical signal, the information of the abnormal electrical signal is written into the second register, and the BMC can perform a self-recovery operation after reading the value of the second register. Of course, the specific self-recovery operation can be set according to actual needs. For example, in a specific embodiment of the present invention, the baseboard management controller performs a self-recovery operation after reading the value of the second register, which can specifically include:
[0118] When the baseboard management controller determines that the value of the second register is not the second default value, determining whether the automatic restart function is enabled;
[0119] If yes, the server is automatically restarted for self-recovery;
[0120] If not, an alarm is sent, and if no instruction to automatically restart the server is received within the first period of time, the server is locked and shut down.
[0121] This implementation takes into account that the second default value of the second register is, for example, 0x00. For the case of Power Good abnormality, the second register is a non-zero value. At this time, the most likely fault direction is that the power supply of a component of the server (such as CPU, memory) is abnormal, which may be caused by a transient / permanent hardware failure. A feasible solution is to automatically restart the server to avoid further hardware damage caused by the continued operation of key components of the server under abnormal power supply conditions due to the abnormal Power Good signal.
[0122] This implementation also takes into account that when a Power Good signal abnormality occurs, the server may not be automatically restarted directly, but an alarm may be sent first, for example, by sending an alarm to the operation and maintenance personnel through SEL, SMTP, etc., so that the server operation and maintenance personnel can observe and decide whether to restart and recover. If an instruction to automatically restart the server is received within the first time period, the server can be automatically restarted for self-recovery. If an instruction to automatically restart the server is not received within the first time period, the server will be locked and shut down to avoid the expansion of the fault.
[0123] In addition, it is understandable that whether to send an alarm first or directly automatically restart the server depends on the pre-set function settings, that is, the BMC needs to determine whether the automatic restart function is enabled.
[0124] The third type of electrical signal is an electrical signal that is pre-classified as an abnormal temperature type, and is mainly a high temperature or low temperature electrical signal of components such as CPU and memory. In addition, in some implementations, other types except the power drop type and the power good type can be regarded as the third type of electrical signal.
[0125] When the abnormal electrical signal belongs to the third type of electrical signal, the CPLD will write the information of the abnormal electrical signal into the third register, and the BMC can perform a self-recovery operation after reading the value of the third register. Of course, the specific self-recovery operation can be set according to actual needs. For example, in a specific embodiment of the present invention, the baseboard management controller performs a self-recovery operation after reading the value of the third register, which can specifically include:
[0126] When the baseboard management controller determines that the value of the third register is not the third default value, the baseboard management controller adjusts the speed of the corresponding fan to perform self-recovery based on the value of the third register and according to the set fan speed adjustment rule.
[0127] The third default value of the third register is, for example, 0x00. When the value of the third register is not the third default value, the most likely fault direction is that the temperature of a component of the server exceeds or is lower than the specified operating temperature of the component due to the ambient temperature and improper heat dissipation of the component. For example, in one scenario, when the value of the third register is 0x09 or 0x19, it indicates that CPU0 is overheated, when the value of the third register is 0x0A or 0x1A, it indicates that CPU1 is overheated, when the value of the third register is 0x0B or 0x18, the 12V voltage is abnormal, when the value of the third register is 0x0C or 0x1C, it indicates that the first memory bar of CPU0 is overheated, when the value of the third register is 0x0D or 0x1D, it indicates that the second memory bar of CPU0 is overheated, when the value of the third register is 0x0E or 0x1E, it indicates that the first memory bar of CPU1 is overheated, and when the value of the third register is 0x0F or 0x1F, it indicates that the second memory bar of CPU1 is overheated.
[0128] In the case where the value of the third register is not the third default value, a feasible solution is to adjust the fan speed of the corresponding component position, that is, based on the value of the third register and in accordance with the set fan speed adjustment rule, adjust the speed of the corresponding fan for self-recovery. When an electrical signal indicating that a certain component is overheated is detected, the speed of the relevant fan can be increased for the component position; conversely, when an electrical signal indicating that a certain component is underheated is detected, the speed of the relevant fan can be reduced for the component position.
[0129] Furthermore, in a specific embodiment of the present invention, it may also include:
[0130] The baseboard management controller adjusts the opening degrees of the front window and the rear window of the server based on the value of the third register and in accordance with the set front and rear window control rules, so as to perform self-recovery.
[0131] This implementation further takes into account that for some servers, the degree of opening of the front window and the rear window can be adjusted, which can also affect the temperature. Therefore, when this implementation performs self-recovery, the BMC can adjust the degree of opening of the front window and the rear window of the server based on the value of the third register and in accordance with the set front and rear window control rules to assist in adjusting the fan speed and achieve self-recovery.
[0132] For example, in some cases, the degree of opening of the front window and the rear window of the server is adjusted, specifically by adjusting the degree of opening of the switch blades of the front window of the server and the degree of opening of the switch blades of the rear window. Especially in some high-latitude cold regions, if the value of the third register reflects the low temperature of the components, by completely closing the front window and the rear window of the server and reducing the fan speed to 0, the temperature of the internal components of the server can be effectively maintained. In addition, in some cases, when the server is idle for a long time, the front window and the rear window can also be closed to prevent dust.
[0133] Step S105: After the self-recovery operation is performed, it is determined whether the abnormal electrical signals have all returned to normal. If they have returned to normal, the operation of step S101 is returned to perform the operation of monitoring each designated electrical signal in the server. If they have not returned to normal, step S106 is performed: the server is locked and shut down.
[0134] After the self-recovery operation is performed, for some relatively minor abnormal situations, the abnormal electrical signals can return to normal. Therefore, if all abnormal electrical signals have returned to normal, the operation of step S101 can be performed, that is, continue to monitor each designated electrical signal in the server. In addition, it should be noted that in actual applications, there may be only one or more abnormal electrical signals. Therefore, when performing self-recovery, a single self-recovery method may be used, or multiple self-recovery methods may be used at the same time. If not all abnormal electrical signals have returned to normal, it is still necessary to perform the operation of step S106, that is, lock and shut down the server.
[0135] After locking and shutting down the server, you cannot directly power on the server by pressing the power button. Instead, you need to disconnect the AC and reconnect it. The CPLD will recheck whether all electrical signals are normal before you can restart the server.
[0136] In a specific implementation of the present invention, after determining that the abnormal electrical signals have not all returned to normal, before locking and shutting down the server, the following may also be included:
[0137] Record the electrical signal information including the name, meaning and reporting time of the abnormal electrical signal in the system event log;
[0138] An independent automatic shutdown log file is created in a designated directory of the system, and associated information of the abnormal electrical signal is recorded in the automatic shutdown log file.
[0139] This implementation takes into account that after determining that the abnormal electrical signals have not all returned to normal, before locking and shutting down the server, the electrical signal information including the name, meaning and reporting time of the abnormal electrical signal can be recorded in the system event log (SEL) so that engineers and customers can understand the electrical signal and its meaning that triggered the locking and shutdown of the server.
[0140] In this implementation, an independent automatic shutdown log file is created specifically in a designated directory of the system. For example, a log file independent of other logs (such as maintenance log maintenance.log, etc.) is created in the Var directory of the system, which is called the automatic shutdown log file, and the associated information of the abnormal electrical signal is recorded in the automatic shutdown log file, so that the maintenance engineer can be provided with the associated information when the CPLD electrical signal is abnormal and triggers the locked shutdown of the server, avoiding the situation where other associated information is washed away due to untimely information collection and can only rely on the next collection.
[0141] The specific content of the abnormal electrical signal associated information can be set and adjusted according to actual needs. For example, in a specific implementation of the present invention, the abnormal electrical signal associated information may include:
[0142] The serial port log of the baseboard management controller, the fan speed and control parameter information, and the log information of the associated components of the abnormal electrical signal can more comprehensively reflect the associated information of the abnormal electrical signal and effectively assist the subsequent processing work of the maintenance engineer. Of course, the specific associated components of the abnormal electrical signal need to be set in advance.
[0143] In a specific embodiment of the present invention, after determining that the abnormal electrical signals have returned to normal and returning to perform the operation of each designated electrical signal in the monitoring server, it also includes:
[0144] Record the electrical signal information including the name, meaning and reporting time of the abnormal electrical signal in the system event log;
[0145] An independent self-recovery log file is created in the specified directory of the system, and self-recovery related information including abnormal recovery time, self-recovery measure name, and register value corresponding to the abnormal electrical signal is recorded in the self-recovery log file.
[0146] This implementation takes into account that after determining that the abnormal electrical signals have returned to normal and returning to execute the operations of each specified electrical signal in the monitoring server, the electrical signal information including the name, meaning and reporting time of the abnormal electrical signal will be recorded in the system event log, so that engineers and customers can understand the relevant electrical signals and their meanings of this self-recovery operation.
[0147] In this implementation, an independent self-recovery log file is created specifically in a designated directory of the system, for example, a log file independent of other logs is created in the Var directory of the system, which is called a self-recovery log file. Self-recovery related information including the abnormal recovery time, the name of the self-recovery measure, and the register value corresponding to the abnormal electrical signal is recorded in the self-recovery log file, so that it can be used as a reference for the next occurrence of the same or related problems. In addition, in actual applications, the self-recovery log file can also be pushed to operation and maintenance personnel through the SMTP service.
[0148] By applying the technical solution provided by the embodiment of the present invention, after monitoring each designated electrical signal in the server, considering the existence of false alarms, when any monitored electrical signal is abnormal, the abnormal electrical signal will be subjected to noise reduction to obtain a noise reduction result, which is also conducive to reducing the probability of the server motherboard being locked by mistake in the present application solution. If the noise reduction result is still abnormal after noise reduction, the present application solution does not lock and shut down the server, but performs a self-recovery operation based on the abnormal electrical signal and the preset self-recovery rules. If it recovers to normal, it means that it is not a serious fault and has been processed, so it can return to the operation of monitoring each designated electrical signal in the server. Therefore, it can be seen that the present application solution improves the convenience of exception handling through the self-recovery design. Of course, if it does not recover to normal, it is necessary to lock and shut down the server.
[0149] In summary, the solution of the present application can effectively implement the handling of server exceptions, reduce the probability of incorrectly locking the motherboard, and improve the convenience of exception handling.
[0150] Corresponding to the above method embodiment, the embodiment of the present invention further provides a system for processing server exceptions, which can be referred to in correspondence with the above.
[0151] The server exception processing system may include:
[0152] A monitoring module, used to monitor various specified electrical signals in the server;
[0153] A noise reduction module is used to reduce the noise of any abnormal electrical signal when any monitored electrical signal is abnormal, and obtain a noise reduction result;
[0154] A first judging module, used to judge whether the noise reduction result is still abnormal;
[0155] A self-recovery module, configured to perform a self-recovery operation based on the abnormal electrical signal and a preset self-recovery rule if the noise reduction result is still abnormal;
[0156] The second judgment module is used to judge whether the abnormal electrical signals have returned to normal after the self-recovery operation is performed;
[0157] If it returns to normal, the monitoring module is triggered again;
[0158] If it does not return to normal, the locking and shutting down module is triggered, and the locking and shutting down module is used to perform locking and shutting down of the server.
[0159] Corresponding to the above method and system embodiments, embodiments of the present invention further provide a server exception processing device, a computer-readable storage medium, and a computer program product, which can be referred to in correspondence with the above.
[0160] See also Figure 3 As shown, the device may include:
[0161] Memory 301, used for storing computer programs;
[0162] The processor 302 is used to execute a computer program to implement the steps of the method for handling server exceptions in any of the above embodiments.
[0163] The computer program product includes a computer program / instruction, and when the computer program / instruction is executed by a processor, the steps of the method for handling server exceptions in any of the above embodiments are implemented.
[0164] See also Figure 4 The computer readable storage medium 40 stores a computer program 41, and when the computer program 41 is executed by the processor, the steps of the server abnormality processing method in any of the above embodiments are implemented. The computer readable storage medium 40 mentioned here includes RAM (Random Access Memory), memory, ROM (Read-Only Memory), EEPROM (Electrically Erasable Programmable Read Only Memory), register, hard disk, removable disk, or any other form of storage medium known in the technical field.
[0165] It should also be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.
[0166] Professionals may further realize that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to the function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention. Specific examples are used herein to illustrate the principles and implementation methods of the present invention, and the description of the above embodiments is only used to help understand the technical solution and its core idea of the present invention. It should be pointed out that for ordinary technicians in the technical field, without departing from the principle of the present invention, the present invention can also be improved and modified in a number of ways, and these improvements and modifications also fall within the scope of protection of the present invention.
Claims
1. A method for handling server exceptions. It is characterized in that include: Monitor each specified electrical signal in the server; When any monitored electrical signal is abnormal, the abnormal electrical signal is denoised to obtain a denoising result; Determining whether the noise reduction result is still abnormal; If the noise reduction result is still abnormal, a self-recovery operation is performed based on the abnormal electrical signal and a preset self-recovery rule; After the self-recovery operation is performed, it is determined whether the abnormal electrical signals have returned to normal; If it returns to normal, returning to execute the operation of each designated electrical signal in the monitoring server; If the server is not restored to normal, the server is locked and shut down; When any monitored electrical signal is abnormal, the abnormal electrical signal is denoised to obtain a denoising result, including: When any monitored electrical signal is abnormal, a sampling window and a filtering window of the electrical signal are established; The value obtained by subtracting the length of the filtering window from the length of the sampling window is used as the filtering value n; wherein n is an even number; Placing the monitored data sample points of the electrical signal into the sampling window according to the first-in-first-out principle; When the sampling window is not filled with data sample points and a statistical operation is triggered, the average value of each data sample point in the current sampling window is used as the statistical value obtained by this statistical operation; When the sampling window is filled with data sample points and a statistical operation is triggered, the average value of each data sample point in the current sampling window, excluding n / 2 maximum values and n / 2 minimum values, is used as the statistical value obtained by this statistical operation; Each statistical value obtained when each statistical operation is triggered is used as the noise reduction result; Determining whether the noise reduction result is still abnormal includes: Determine whether the total number of abnormalities in each statistical value in the noise reduction result is lower than a first threshold; If it is lower than the first threshold, it is determined that the noise reduction result is not abnormal; If it is not lower than the first threshold, determining whether the latest k statistical values in the noise reduction result are all normal; If all are normal, it is determined that the noise reduction result is normal; If they are not all normal, determining that the noise reduction result is still abnormal; Based on abnormal electrical signals and preset self-recovery rules, self-recovery operations are performed, including: Writing the information of the abnormal electrical signal into the corresponding register so that the baseboard management controller performs a self-recovery operation after reading the value of the corresponding register; The baseboard management controller performs self-recovery operations after reading the corresponding registers, including: When the abnormal electrical signal belongs to the first type of electrical signal, information of the abnormal electrical signal is written into the first register, so that the baseboard management controller performs a self-recovery operation after reading the value of the first register; When the abnormal electrical signal belongs to the second type of electrical signal, information of the abnormal electrical signal is written into the second register, so that the baseboard management controller performs a self-recovery operation after reading the value of the second register; When the abnormal electrical signal belongs to the third type of electrical signal, information of the abnormal electrical signal is written into a third register, so that the baseboard management controller performs a self-recovery operation after reading the value of the third register; Among them, the first type of electrical signal is an electrical signal pre-classified as a voltage drop type, the second type of electrical signal is an electrical signal pre-classified as a component power supply abnormality type, and the third type of electrical signal is an electrical signal pre-classified as a temperature abnormality type.
2. The method for handling server abnormality according to claim 1, It is characterized in that The baseboard management controller performs a self-recovery operation after reading the value of the first register, including: When the baseboard management controller determines that the value of the first register is not a first default value, the power supply of the server is switched to a backup power supply for self-recovery.
3. The method for handling server abnormality according to claim 1, It is characterized in that The baseboard management controller performs a self-recovery operation after reading the value of the second register, including: When the baseboard management controller determines that the value of the second register is not a second default value, determining whether an automatic restart function is enabled; If yes, automatically restart the server to perform self-recovery; If not, an alarm is sent, and when no instruction to automatically restart the server is received within the first time period, the server is locked and shut down.
4. The method for handling server abnormality according to claim 1, It is characterized in that The baseboard management controller performs a self-recovery operation after reading the value of the third register, including: When the baseboard management controller determines that the value of the third register is not the third default value, the baseboard management controller adjusts the speed of the corresponding fan to perform self-recovery based on the value of the third register and in accordance with the set fan speed adjustment rule.
5. The method for handling server abnormality according to claim 4, It is characterized in that Also includes: The baseboard management controller adjusts the opening degree of the front window and the rear window of the server based on the value of the third register and in accordance with the set front and rear window control rules, so as to perform self-recovery.
6. The method for handling server abnormality according to any one of claims 1 to 5, It is characterized in that After determining that the abnormal electrical signals have not all returned to normal, and before locking and shutting down the server, the method further includes: Record the electrical signal information including the name, meaning and reporting time of the abnormal electrical signal in the system event log; An independent automatic shutdown log file is created in a designated directory of the system, and associated information of the abnormal electrical signal is recorded in the automatic shutdown log file.
7. The method for handling server abnormality according to claim 6, It is characterized in that The associated information of abnormal electrical signals includes: Serial port log of the baseboard management controller, fan speed and control parameter information, and log information of components related to abnormal electrical signals.
8. The method for handling server abnormality according to any one of claims 1 to 5, It is characterized in that After determining that the abnormal electrical signals have returned to normal and returning to execute the operations of each designated electrical signal in the monitoring server, the method further includes: Record the electrical signal information including the name, meaning and reporting time of the abnormal electrical signal in the system event log; An independent self-recovery log file is created in a designated directory of the system, and self-recovery related information including abnormal recovery time, self-recovery measure name, and register value corresponding to the abnormal electrical signal is recorded in the self-recovery log file.
9. A computer program product comprising a computer program / instructions, It is characterized in that When the computer program / instruction is executed by a processor, the steps of the method for handling server exceptions described in any one of claims 1 to 8 are implemented.
10. A server abnormality processing device, It is characterized in that include: Memory for storing computer programs; A processor is used to execute the computer program to implement the steps of the method for handling server exceptions as described in any one of claims 1 to 8.
11. A computer-readable storage medium, It is characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method for handling server exceptions according to any one of claims 1 to 8 are implemented.
