Fault processing method and device, equipment and medium

Abstract

The invention discloses a fault processing method and device, equipment and a medium, and belongs to the technical field of servers, and the method comprises the steps: setting a security network configuration item and an error processing configuration item for a manager port, and obtaining processor register information in response to fault alarm information received through a manager, the processor register information comprises a current fault diagnosis state, and whether fault diagnosis source data are collected or not is judged according to the current fault diagnosis state; in response to the collected fault diagnosis source data, judging whether the system is subjected to soft restart or not according to the error processing configuration item; and in response to soft restart of the system according to the security network configuration item, determining a fault type according to the fault diagnosis source data, and performing automatic analysis processing on the fault according to the fault type. Through compatibility of hardware and software, after a fault occurs, the manager can automatically collect fault source data and process the fault, and the correctness of fault processing is improved.

Description

Technical Field

[0001] The present application relates to the field of server technology, and in particular to a fault handling method, device, equipment and medium. Background Art

[0002] At present, there are a variety of storage devices and computing devices on servers, which are provided and developed by different manufacturers and different R&D teams. During the operation of the CPU, unpredictable errors may occur, causing the server to fail to operate normally.

[0003] The relevant technology is to capture fault register information during the BIOS system restart process, and then pass the information to the baseboard management controller through the IPMI channel for logging. However, the register information needs to be trimmed on the BIOS side and needs to fully comply with the IPMI protocol for information transmission, which limits the capacity and integrity of the log information.

[0004] At the same time, the system has been loaded, and the processor needs to be soft-restarted to reload the operating system to obtain the error information in the register, and then pass the register information to the manager for logging and alarming, and provide diagnostic suggestions. Therefore, when the problem occurs, there is no way to retain the original error environment, conduct fault analysis in time, and accurately locate the cause of the fault. Summary of the invention

[0005] In order to overcome the above technical defects, the purpose of the present application is to provide a fault handling method, device, equipment and medium, the method comprising: setting a security network configuration item and an error handling configuration item for the manager port, wherein the security network configuration item is used to prevent the change of the downtime restart environment, and the error handling configuration item is used to determine different restart strategies according to the error type; in response to receiving fault alarm information through the manager, the processor register information is obtained, wherein the processor register information includes the current fault diagnosis state, and the fault diagnosis state includes at least one of error detection, error reporting, error recovery and fault isolation; judging whether to collect fault diagnosis source data according to the current fault diagnosis state; in response to collecting fault diagnosis source data, judging whether the system is soft restarted according to the error handling configuration item; in response to soft restarting the system according to the security network configuration item, the fault type is determined according to the fault diagnosis source data, and the fault is automatically parsed and processed according to the fault type. The present application can realize that after a fault occurs, the manager can automatically collect fault source data and process the fault through hardware and software compatibility, thereby improving the correctness of fault processing.

[0006] The specific technical solutions provided by the embodiments of this application are as follows:

[0007] In a first aspect, the present application provides a fault handling method, the method comprising:

[0008] Setting a security network configuration item and an error handling configuration item for the manager port, wherein the security network configuration item is used to prevent changes in the shutdown restart environment, and the error handling configuration item is used to determine different restart strategies according to the error type;

[0009] In response to receiving the fault warning information through the manager, acquiring processor register information, wherein the processor register information includes a current fault diagnosis state, and the fault diagnosis state includes at least one of error detection, error reporting, error recovery, and fault isolation;

[0010] Determining whether to collect fault diagnosis source data according to the current fault diagnosis state;

[0011] In response to collecting the fault diagnosis source data, judging whether the system is soft-restarted according to the safety network configuration item according to the error handling configuration item;

[0012] In response to a soft restart of the system according to the safety network configuration item, a fault type is determined according to the fault diagnosis source data, and the fault is automatically parsed and processed according to the fault type.

[0013] In one embodiment, before setting the security network configuration item and the error handling configuration item for the manager port, the method includes:

[0014] Setting a fault diagnosis function in the manager, recording status information when a fault occurs, and monitoring the fault alarm information; importing a crash dump error code in the manager, and monitoring the fault diagnosis status;

[0015] The security network configuration item is enabled, and when a number of flood attack requests appear in the system, a processor reset is triggered to clear the flood attack resources;

[0016] Enable the error handling configuration item. When the system has several fault errors, the PCIe controller is reset to protect the system.

[0017] In one embodiment, the fault diagnosis source data includes a plurality of fault data blocks, the fault data blocks include status data and Internet Protocol identifier data, and the method of determining the fault type according to the fault diagnosis source data and automatically parsing the fault according to the fault type includes:

[0018] Judging the validity of the fault data block according to the status data;

[0019] In response to the fault data block being valid, determining the fault type according to the status data, the Internet protocol identifier data and the standard document, and performing analysis and processing on the fault according to the fault type;

[0020] The analyzing and processing the fault according to the fault type includes:

[0021] parsing the fault memory location according to the fault type;

[0022] Analyze the fault severity level and fault code according to the fault type;

[0023] Determine a detailed description of the fault according to the fault type;

[0024] Record a fault log for the fault according to the fault memory location, fault severity level, fault code and fault detailed description;

[0025] Develop a fault handling solution based on the fault log.

[0026] In one embodiment, the method comprises:

[0027] In response to not soft-restarting the system, the downtime interface is retained, the downtime processor register information is collected remotely or on-site through a collection tool, the downtime processor register information is parsed, and the fault is processed.

[0028] In one embodiment, the determining the fault type according to the fault diagnosis source data and automatically analyzing the fault according to the fault type further includes:

[0029] Cleaning and formatting the fault diagnosis source data to remove data noise, errors and redundant information;

[0030] Extracting fault features from the cleaned fault diagnosis source data, wherein the fault features include: trend analysis, statistical indicators and frequency domain analysis of time series data;

[0031] Classify fault features through the fault mode training model and determine the fault type;

[0032] Obtain a fault code manual or knowledge base according to the fault type, and analyze the fault log file according to the fault code manual or knowledge base;

[0033] Compare the fault log files with historical fault cases and analyze the faults.

[0034] Verify whether the fault handling is complete by retesting;

[0035] The checking whether the fault handling is completed by retesting includes:

[0036] Reproduce the conditions when the fault occurred to confirm whether the same fault occurs;

[0037] Obtain the application's log files to confirm whether there are any abnormal records;

[0038] Use performance monitoring tools to monitor the system's CPU usage, memory usage, disk, and network traffic to confirm whether the system performance has returned to normal;

[0039] Stress test the system to confirm whether it can operate;

[0040] Verify that all system configuration files have been modified and saved.

[0041] In one embodiment, the determining of the fault type according to the fault diagnosis source data and automatically analyzing the fault according to the fault type include:

[0042] Recording fault error logs through a manager, and optimizing fault handling according to the fault error logs;

[0043] The optimizing the fault handling according to the fault error log includes:

[0044] Get the fault error log;

[0045] Unifying the log format of the fault error log and structuring it;

[0046] Aggregate and statistically analyze the structured fault error log data to determine the fault mode and pattern;

[0047] Create a fault tree based on failure modes and rules to analyze the failures and optimize the fault handling strategy.

[0048] In one embodiment, the method comprises:

[0049] When a problem occurs in the automatic processing of the fault, the system is soft-restarted, the processor register information is obtained, and the fault error type and the faulty component are determined according to the processor register information;

[0050] and creating first fault information according to the fault error type and the fault component information;

[0051] sending the first fault information to the manager;

[0052] The manager records a fault log according to the first fault information and parses the fault log;

[0053] And replace the faulty component according to the faulty component information.

[0054] In a second aspect, the present application further provides a fault handling device, the device comprising:

[0055] A setting module, used to set a security network configuration item and an error handling configuration item for the manager port, wherein the security network configuration item is used to prevent changes in the shutdown restart environment, and the error handling configuration item is used to determine different restart strategies according to the error type;

[0056] a receiving module, configured to obtain processor register information in response to receiving fault warning information through the manager, wherein the processor register information includes a current fault diagnosis state, and the fault diagnosis state includes at least one of error detection, error reporting, error recovery, and fault isolation;

[0057] A judgment module, used for judging whether to collect fault diagnosis source data according to the current fault diagnosis state;

[0058] In response to collecting the fault diagnosis source data, judging whether the system is soft-restarted according to the safety network configuration item according to the error handling configuration item;

[0059] A processing module is used for, in response to soft restarting the system according to the safety network configuration item, determining a fault type according to the fault diagnosis source data, and automatically analyzing and processing the fault according to the fault type.

[0060] In a third aspect, a fault handling device is also provided, including:

[0061] one or more processors;

[0062] A storage device for storing one or more programs;

[0063] When the one or more programs are executed by the one or more processors, the one or more processors implement the fault handling method as described in any one of the first aspects.

[0064] In a fourth aspect, the present application further provides a computer device, the device comprising:

[0065] A memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the fault handling method as described in any one of the first aspects.

[0066] In a fifth aspect, the present application further provides a computer program product, comprising a computer program, which, when executed by a processor, implements the steps of any of the fault handling methods described in the first aspect.

[0067] In a sixth aspect, the present application further provides a computer storage medium, the medium comprising:

[0068] A computer program is stored thereon, and when the computer program is executed by a processor, the steps of any fault handling method described in the first aspect are implemented.

[0069] Compared with the prior art, the method described in the technical solution provided in the embodiment of the present application includes: setting a security network configuration item and an error handling configuration item for the manager port, wherein the security network configuration item is used to prevent changes in the downtime restart environment, and the error handling configuration item is used to determine different restart strategies according to the error type; in response to receiving fault alarm information through the manager, obtaining processor register information, wherein the processor register information includes the current fault diagnosis status, and the fault diagnosis status includes at least one of error detection, error reporting, error recovery and fault isolation; judging whether to collect fault diagnosis source data according to the current fault diagnosis status; in response to collecting fault diagnosis source data, judging whether the system is soft-restarted according to the error handling configuration item; in response to soft-restarting the system according to the security network configuration item, determining the fault type according to the fault diagnosis source data, and automatically parsing and processing the fault according to the fault type. The present application can realize, through the joint debugging and compatibility of hardware and software, that after a fault occurs, the baseboard management controller can automatically collect the fault source data, integrate the parsing code in the manager, realize automatic parsing of the fault data and record detailed logs, assist in fault diagnosis and troubleshooting, and avoid the problem of inefficient technical personnel traveling to the customer site for support after a fault occurs.

[0070] The technical solution provided in the embodiment of the present application detects the operating status of the processor through a baseboard management controller, automatically collects and analyzes processor register information when the server crashes, records detailed fault logs, error content, and error causes based on the analysis results, maintains logs, and gives diagnostic suggestions, which can provide more accurate guidance for on-site operation and maintenance. BRIEF DESCRIPTION OF THE DRAWINGS

[0071] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0072] Figure 1 A first flow chart of the fault handling method provided in Embodiment 1 of the present application;

[0073] Figure 2A second flow chart of the fault handling method provided in Embodiment 2 of the present application;

[0074] Figure 3 A third flow chart of the fault handling method provided in Embodiment 2 of the present application;

[0075] Figure 4 A fourth flow chart of the fault handling method provided in Embodiment 2 of the present application;

[0076] Figure 5 A structural diagram of a fault handling device provided in Embodiment 3 of the present application;

[0077] Figure 6 The exemplary system provided for Embodiment 7 of the present application can be used to implement the various embodiments described in the present application. DETAILED DESCRIPTION

[0078] In order to make the purpose, technical solutions and advantages of this application clearer, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0079] It should be noted that, unless the context clearly requires otherwise, words such as “include”, “including” and similar words throughout the specification and claims should be interpreted as including rather than exclusive or exhaustive; that is, the meaning is “including but not limited to”.

[0080] Furthermore, in the description of the present application, unless otherwise specified, “plurality” means two or more.

[0081] First, when a fault occurs, if the machine does not restart, R&D personnel are required to collect register information remotely or on-site. At this time, the downtime environment needs to be retained, and most problems will occur in the customer's computer room, which is extremely inconvenient for the efficiency of customer production and R&D problem solving.

[0082] Secondly, when a fault occurs, if the machine is soft-restarted, the operating system BIOS will parse the fault register information during the restart process. However, during the system restart process, the re-running of the processor CPU will cause some fault register information to be refreshed and overwritten, and the downtime environment at the time cannot be fully reproduced, affecting R&D to locate the root cause. At this time, the only way to reproduce the problem is to re-perform stress testing in the R&D laboratory. Moreover, the probability of most problems is small, and it is not easy to reproduce them in the environment of one or two machines in the laboratory, which affects the location and resolution of the root cause of the problem.

[0083] Finally, the system BIOS parses the fault register information and transmits the fixed format information to the BMC through the IPMB channel to record the log. Due to the limitations of IPMI, the SEL log information transmitted by the BIOS is relatively limited and can only record the approximate error components and rough error information. Some errors do not require component replacement, but because the SEL log cannot record in detail, on-site operation and maintenance can only replace components based on the component location recorded in the log, so there is no more accurate and efficient solution.

[0084] Embodiment 1

[0085] The present application embodiment provides a fault handling method, such as Figure 1 As shown, the method includes:

[0086] Setting a security network configuration item and an error handling configuration item for the manager port, wherein the security network configuration item is used to prevent changes in the shutdown restart environment, and the error handling configuration item is used to determine different restart strategies according to the error type;

[0087] In response to receiving the fault warning information through the manager, acquiring processor register information, wherein the processor register information includes a current fault diagnosis state, and the fault diagnosis state includes at least one of error detection, error reporting, error recovery, and fault isolation;

[0088] Determining whether to collect fault diagnosis source data according to the current fault diagnosis state;

[0089] In response to collecting the fault diagnosis source data, judging whether the system is soft-restarted according to the safety network configuration item according to the error handling configuration item;

[0090] In response to a soft restart of the system according to the safety network configuration item, a fault type is determined according to the fault diagnosis source data, and the fault is automatically parsed and processed according to the fault type.

[0091] Specifically, the present application detects the operating status of the processor CPU through the baseboard management controller BMC, automatically collects and analyzes the CPU register information when it crashes, and records detailed fault logs, error content, error causes, maintenance logs and diagnostic suggestions based on the analysis results, providing more accurate guidance for on-site operation and maintenance.

[0092] It can be understood that the present application realizes that the manager BMC can automatically collect fault source data after a fault occurs through the joint debugging and compatibility of hardware and software, the necessary conditions required for the BMC to implement fault diagnosis, and the complete software and hardware operation process and fault source data analysis process after the fault diagnosis occurs.

[0093] Among them, the baseboard management controller (BMC) acts as a housekeeper in the server. It can manage the power supply, temperature and other performance of the entire system. When the system crashes, it can also act as a watchdog to restart the CPU. Here, the baseboard management controller is an independent system that does not rely on other hardware on the system (such as CPU, memory, etc.), nor does it rely on BIOS, OS, etc. However, the baseboard management controller can interact with BIOS and OS, which can play a better role in platform management. There is system management software under OS that can work with BMC to achieve better management results.

[0094] The beneficial effects of the technical solution provided by the embodiment of the present application are:

[0095] The present application can realize, through the joint debugging and compatibility of hardware and software, that after a fault occurs, the baseboard management controller can automatically collect the fault source data, integrate the parsing code in the manager, realize automatic parsing of the fault data and record detailed logs, assist in fault diagnosis and troubleshooting, and avoid the problem of inefficient technical personnel traveling to the customer site for support after a fault occurs.

[0096] The technical solution provided in the embodiment of the present application detects the operating status of the processor through a baseboard management controller, automatically collects and analyzes processor register information when it fails, records detailed fault logs, error content, and error causes based on the analysis results, maintains logs, and gives diagnostic suggestions, which can provide more accurate guidance for on-site operation and maintenance.

[0097] Embodiment 2

[0098] The present application embodiment provides a fault handling method, such as Figure 2 As shown, the method includes:

[0099] Step S01, setting a fault diagnosis function in the manager, recording state information when a fault occurs, and monitoring the fault alarm information; importing a crash dump error code in the manager, and monitoring the fault diagnosis state;

[0100] The security network configuration item is enabled, and when a number of flood attack requests appear in the system, a processor reset is triggered to clear the flood attack resources;

[0101] Enable the error handling configuration item. When the system has several fault errors, the PCIe controller is reset to protect the system.

[0102] Specifically, the fault diagnosis function Crashdump Code is integrated in the baseboard management controller to monitor APML_ALERT (advanced platform management link) warnings and collect CPU register data; wherein the fault diagnosis function generally refers to a piece of data or code automatically generated by the system when a software or hardware failure occurs, and is used to record the status information when the failure occurs;

[0103] Import the crash dump error code (Crashdump Code) in the BMC, implement the fault diagnosis status RAS (Reliability, Availability, and Serviceability) daemon, and monitor APML_ALERT in real time;

[0104] When a fatal error occurs in the CPU, the enable APML_ALERT identification code Pin is implemented. This is provided by the CPU manufacturer and varies from one CPU manufacturer to another.

[0105] Step S02, setting a safety network configuration item and an error handling configuration item for the manager port, wherein the safety network configuration item is used to prevent changes in the shutdown restart environment, and the error handling configuration item is used to determine different restart strategies according to the error type.

[0106] Specifically, a security network configuration item PcdResetCpuOnSyncFlood is set on the BMC side to prevent the BMC from resetting the SOC (system on chip) to reset or restart the management controller and core system chip in the server or computing platform during the collection of downtime register information, which may cause a change in the downtime environment and affect fault diagnosis. Among them, the security network configuration item is a configuration item or function identifier related to computer system security or network equipment. The PcdResetCpuOnSyncFlood configuration item can be used to specify whether the system should take measures to reset the CPU status or perform other protective operations to prevent the system from crashing due to resource exhaustion. It usually involves the security policy setting of network equipment or servers to improve the robustness and security of the system.

[0107] The error handling configuration item PcdAmdPcieSyncFloodOnFatal is set on the BMC side, and different restart strategies are set after different errors occur. According to different customer needs, the downtime site is retained after the downtime problem occurs; wherein, the error handling configuration item is a configuration item related to AMD PCIe (Peripheral Component Interconnect Express), which usually appears in the firmware or BIOS settings of the AMD platform. The configuration item is mainly used to handle synchronization flood attacks (Sync Flood Attack) or fatal errors (Fatal Error) on the PCIe link.

[0108] Step S03, receiving fault warning information through the manager;

[0109] Acquiring processor register information through a platform management link, wherein the processor register information includes a current fault diagnosis state, and the fault diagnosis state includes at least one of error detection, error reporting, error recovery, and fault isolation;

[0110] Whether to collect fault diagnosis source data is determined according to the current fault diagnosis state.

[0111] Specifically, a fatal error is triggered, which triggers the interruption of the microcontroller SMU (System Management Unit). After the SMU interruption is triggered, the logic element CPLD receives the interruption signal and raises the APML_ALERT alarm information. After the BMC monitors that the APML_ALERT alarm information is raised, it obtains the fault diagnosis status Rasstatus (Reliability, Availability, and Serviceability, RAS features usually include error detection, error reporting, error recovery and fault isolation functions) through the advanced platform management link APML.

[0112] Step S04, in response to collecting the fault diagnosis source data, judging whether the system is soft-restarted according to the safety network configuration item according to the error handling configuration item;

[0113] In response to a soft restart of the system according to the safety network configuration item, a fault type is determined according to the fault diagnosis source data, and the fault is automatically parsed and processed according to the fault type.

[0114] Specifically, the BMC determines whether it is necessary to collect fault source data and whether to perform a soft restart or a hard restart based on the fault diagnosis status (Ras status); for most faults, it is necessary to collect fault source data and then perform a soft restart. After the fault source data is collected, it determines whether a restart is required based on the user-defined settings.

[0115] When the user sets a restart after a downtime problem occurs, the BMC collects data and parses it according to the fault diagnosis sel log parsing specification. It analyzes the fault cause and fault severity level according to the error type, points to the faulty component and provides diagnostic suggestions.

[0116] Step S04 also includes:

[0117] Step S041, judging the validity of the fault data block according to the status data;

[0118] In response to the fault data block being valid, determining the fault type according to the status data, the Internet protocol identifier data and the standard document, and performing analysis and processing on the fault according to the fault type;

[0119] The analyzing and processing the fault according to the fault type includes:

[0120] parsing the fault memory location according to the fault type;

[0121] Analyze the fault severity level and fault code according to the fault type;

[0122] Determine a detailed description of the fault according to the fault type;

[0123] Record a fault log for the fault according to the fault memory location, fault severity level, fault code and fault detailed description;

[0124] Develop a fault handling solution based on the fault log.

[0125] Specifically, it is determined whether it is necessary to collect fault diagnosis source data according to the fault status of the current processor CPU register. When the fault diagnosis data collection is completed, the fault data is parsed. It is first determined whether the user setting is to restart. For example, if the machine restarts after a fault occurs, after the restart is completed, the fault diagnosis source data and the system BIOS transmission fault data are combined for processing. The fault data collected by BMC is mainly used, and the BIOS transmission data is deduplicated and processed together.

[0126] Here, the fault source data includes multiple fault data blocks, and the multiple data blocks are processed in sequence as follows: select key data from a single fault source data block for analysis (for example, each data block is a complete structure, including status, addr, ipid, synd and other data), and judge whether the current fault data block is valid fault data according to certain bits of the status data. If it is valid data, judge the current fault type according to certain bits of the status data and ipid data combined with the specification document provided by AMD; if the current fault data block is not valid data, the invalid data is discarded without parsing;

[0127] After the fault type is determined, the fault pointing device, device location, error severity level, fault code, EXT fault type, and fault detailed description are determined based on the fault data and analysis rules. The above parsed data is recorded in a detailed log and fault handling suggestions are given.

[0128] For example: memory Umc error requires parsing the faulty memory location, such as which processor CPU, which channel, which DIMM; parsing the fault severity level, such as CorrectError (CE), UnCorrectError (UCE), FatalError, etc.; parsing the fault code, such as 0C260600, corresponding to the location and severity level information, as log specification information; parsing the EXT fault type, such as DramEcc, and recording the fault log according to the faulty memory location, fault severity level, fault code and detailed fault description; formulating a fault handling solution based on the fault log.

[0129] Step S042, cleaning and formatting the fault diagnosis source data to remove data noise, errors and redundant information;

[0130] Extracting fault features from the cleaned fault diagnosis source data, wherein the fault features include: trend analysis, statistical indicators and frequency domain analysis of time series data;

[0131] Classify fault features through the fault mode training model and determine the fault type;

[0132] Obtain a fault code manual or knowledge base according to the fault type, and analyze the fault log file according to the fault code manual or knowledge base;

[0133] Compare the fault log files with historical fault cases and analyze the faults.

[0134] Verify whether the fault handling is complete by retesting;

[0135] The checking whether the fault handling is completed by retesting includes:

[0136] Reproduce the conditions when the fault occurred to confirm whether the same fault occurs;

[0137] Obtain the application's log files to confirm whether there are any abnormal records;

[0138] Use performance monitoring tools to monitor the system's CPU usage, memory usage, disk, and network traffic to confirm whether the system performance has returned to normal;

[0139] Stress test the system to confirm whether it can operate;

[0140] Verify that all system configuration files have been modified and saved.

[0141] Step S05, in response to not soft-restarting the system, retaining the downtime interface, remotely or on-site collecting the downtime processor register information through a collection tool, parsing the downtime processor register information, and processing the fault.

[0142] Specifically, when the user sets the system not to restart after a downtime problem occurs, the BMC does not process the fault data after collecting it, but only retains the original fault data for on-site operation and maintenance by developers.

[0143] That is, the user sets the machine not to restart after a crash, and the machine stops at the crash interface after a crash. R&D collects the crash register information remotely or on-site, and then parses the crash register information to provide professional fault diagnosis opinions and suggestions for on-site operation and maintenance personnel to solve the crash problem.

[0144] Step S06, recording a fault error log through the manager, and optimizing fault processing according to the fault error log;

[0145] The optimizing the fault handling according to the fault error log includes:

[0146] Get the fault error log;

[0147] Unifying the log format of the fault error log and structuring it;

[0148] Aggregate and statistically analyze structured fault error log data to determine fault modes and trends;

[0149] Create a fault tree based on failure modes and trends to analyze the failures and optimize the fault handling strategy.

[0150] Step S07, when a problem occurs in the automatic processing of the fault, the system is soft-restarted, the processor register information is obtained, and the fault error type and the faulty component are determined according to the processor register information;

[0151] and creating first fault information according to the fault error type and the fault component information;

[0152] sending the first fault information to the manager;

[0153] The manager records a fault log according to the first fault information and parses the fault log;

[0154] And replace the faulty component according to the faulty component information.

[0155] Specifically, the user sets the machine to restart after a crash, and the machine will perform a soft restart after a crash. The register information read out is used on the BIOS side to determine the current approximate error type and faulty components based on the fault analysis specifications provided by the CPU manufacturer. The combined information is then sent to the manager BMC through the IPMB channel. The manager BMC records the system event log (SEL), and the on-site operation and maintenance personnel replace the faulty components according to the prompts in the SEL log.

[0156] Among them, soft restart is a method of restarting the system through operating system commands, without turning off the power of the server. The main steps of soft restart include the administrator or user executing a restart command, such as reboot or shutdown-r now, or using a system management tool to perform a soft restart operation; during the soft restart process, the operating system will begin to shut down currently running processes, release resources, unmount file systems, etc., and then restart the system; the main difference between soft restart and hard restart is that the hard restart is achieved by forcibly shutting down the server and then restarting it through the terminal, and the operating system and application programs will be reloaded; while the soft restart is achieved through the operating system command, the system kernel and hardware resources will not be reinitialized, and during the soft restart process, the BIOS is reloaded, and the CPU register information is read during the loading process. The soft restart ensures that the CPU register information is retained for fault analysis.

[0157] like Figure 3 , 4As shown, the technical solution of the present application provides an automatic fault diagnosis method after a downtime problem occurs. The relevant technical solution is that after the downtime problem occurs, the BMC cannot collect fault data and can only receive BIOS transmission information during the startup process, and parse and record logs according to the BIOS transmission information. Due to the limitations of the IPMI specification, the cause of the error in the log is only roughly pointed out, and the cause of the problem and the problem component cannot be confirmed in detail; the actual situation is that after the downtime problem occurs, the log recorded by the BIOS cannot assist in analysis in many cases. Developers can only bring professional equipment, read the data on site, and then parse it into a readable file through professional tools for analysis, and give diagnostic suggestions on site; the current solution greatly simplifies the fault diagnosis and analysis process. Through the joint debugging and compatibility of hardware and software, the BMC can automatically collect the fault source data after the fault occurs, integrate the parsing code in the BMC, and realize the automatic parsing of the fault data and then record detailed logs to assist in fault diagnosis and fault resolution, so as to avoid the problem of low fault handling efficiency caused by the technical personnel traveling to the customer site for support after the fault occurs.

[0158] In addition, the setting of the security network configuration item and the error handling configuration item for the manager port includes:

[0159] Log in to the baseboard management controller management interface;

[0160] Determine the settings related to network security or attack protection in the manager interface;

[0161] Locate the security network configuration items and error handling configuration items;

[0162] Configure the security network configuration item (PcdResetCpuOnSyncFlood); Configure the error handling configuration item (PcdAmdPcieSyncFloodOnFatal);

[0163] The configuration of the security network configuration items includes:

[0164] Setting the enabling or disabling function of the security network configuration item;

[0165] Set the SYN flood attack threshold that triggers a processor reset;

[0166] The configuration error handling configuration items include:

[0167] Setting the enabling or disabling function of the error handling configuration item;

[0168] Set thresholds for triggering protective measures;

[0169] After completing the configuration, save the configuration information and verify whether the configuration items are set up properly.

[0170] Through the joint debugging and compatibility of hardware and software, after a fault occurs, the baseboard management controller can automatically collect fault source data, integrate parsing code in the manager, automatically parse the fault data and record detailed logs to assist in fault diagnosis and resolution, and avoid the inefficient problem of technical personnel traveling to the customer site for support after a fault occurs.

[0171] The fault handling method provided in the embodiment of the present application may be further improved and optimized without departing from the technical solution of the present application, and these improvements and optimizations should also be regarded as within the scope of protection of the present application.

[0172] This technical solution can also be applied to other terminal devices to diagnose faults of CPUs from other manufacturers and other PCIE devices.

[0173] The beneficial effects of the technical solution provided by the embodiment of the present application are:

[0174] The present application can realize, through the joint debugging and compatibility of hardware and software, that after a fault occurs, the baseboard management controller can automatically collect the fault source data, integrate the parsing code in the manager, realize automatic parsing of the fault data and record detailed logs, assist in fault diagnosis and troubleshooting, and avoid the problem of inefficient technical personnel traveling to the customer site for support after a fault occurs.

[0175] The technical solution provided in the embodiment of the present application detects the operating status of the processor through a baseboard management controller, automatically collects and analyzes processor register information when it fails, records detailed fault logs, error content, and error causes based on the analysis results, maintains logs, and gives diagnostic suggestions, which can provide more accurate guidance for on-site operation and maintenance.

[0176] Embodiment 3

[0177] The present application provides a fault handling device, such as Figure 5 As shown, the device includes: a setting module, a receiving module, a judgment module, a processing module, and an optimization module.

[0178] In this embodiment, the setting module is used to set a security network configuration item and an error handling configuration item for the manager port, wherein the security network configuration item is used to prevent changes in the shutdown restart environment, and the error handling configuration item is used to determine different restart strategies according to the error type;

[0179] a receiving module, configured to obtain processor register information in response to receiving fault warning information through the manager, wherein the processor register information includes a current fault diagnosis state, and the fault diagnosis state includes at least one of error detection, error reporting, error recovery, and fault isolation;

[0180] A judgment module, used for judging whether to collect fault diagnosis source data according to the current fault diagnosis state;

[0181] In response to collecting the fault diagnosis source data, judging whether the system is soft-restarted according to the safety network configuration item according to the error handling configuration item;

[0182] A processing module is used for, in response to soft restarting the system according to the safety network configuration item, determining a fault type according to the fault diagnosis source data, and automatically analyzing and processing the fault according to the fault type.

[0183] In one of the embodiments, a setting module is used to set a fault diagnosis function in the manager, record the state information when the fault occurs, and monitor the fault alarm information; import the crash dump error code in the manager, and monitor the fault diagnosis state;

[0184] The security network configuration item is enabled, and when a number of flood attack requests appear in the system, a processor reset is triggered to clear the flood attack resources;

[0185] Enable the error handling configuration item. When the system has several fault errors, the PCIe controller is reset to protect the system.

[0186] In one of the embodiments, the processing module is used to determine the validity of the fault data block according to the status data;

[0187] In response to the fault data block being valid, determining the fault type according to the status data, the Internet protocol identifier data and the standard document, and performing analysis and processing on the fault according to the fault type;

[0188] The analyzing and processing the fault according to the fault type includes:

[0189] parsing the fault memory location according to the fault type;

[0190] Analyze the fault severity level and fault code according to the fault type;

[0191] Determine a detailed description of the fault according to the fault type;

[0192] Record a fault log for the fault according to the fault memory location, fault severity level, fault code and fault detailed description;

[0193] Develop a fault handling solution based on the fault log.

[0194] In one of the embodiments, the processing module is used to retain the downtime interface in response to not soft-restarting the system, collect downtime processor register information remotely or on-site through a collection tool, parse the downtime processor register information, and process the fault.

[0195] In one of the embodiments, the processing module is used to clean and format the fault diagnosis source data to remove data noise, errors and redundant information;

[0196] Extracting fault features from the cleaned fault diagnosis source data, wherein the fault features include: trend analysis, statistical indicators and frequency domain analysis of time series data;

[0197] Classify fault features through the fault mode training model and determine the fault type;

[0198] Obtain a fault code manual or knowledge base according to the fault type, and analyze the fault log file according to the fault code manual or knowledge base;

[0199] Compare the fault log files with historical fault cases and analyze the faults.

[0200] Verify whether the fault handling is complete by retesting;

[0201] The checking whether the fault handling is completed by retesting includes:

[0202] Reproduce the conditions when the fault occurred to confirm whether the same fault occurs;

[0203] Obtain the application's log files to confirm whether there are any abnormal records;

[0204] Use performance monitoring tools to monitor the system's CPU usage, memory usage, disk, and network traffic to confirm whether the system performance has returned to normal;

[0205] Stress test the system to confirm whether it can operate;

[0206] Verify that all system configuration files have been modified and saved.

[0207] In one of the embodiments, the optimization module is used to record the fault error log through the manager and optimize the fault processing according to the fault error log;

[0208] The optimizing the fault handling according to the fault error log includes:

[0209] Get the fault error log;

[0210] Unifying the log format of the fault error log and structuring it;

[0211] Aggregate and statistically analyze the structured fault error log data to determine the fault mode and pattern;

[0212] Create a fault tree based on failure modes and rules to analyze the failures and optimize the fault handling strategy.

[0213] In one embodiment, the processing module is used to soft-restart the system when a problem occurs in the automatic processing of the fault, obtain processor register information, and determine the fault error type and the faulty component according to the processor register information;

[0214] and creating first fault information according to the fault error type and the fault component information;

[0215] sending the first fault information to the manager;

[0216] The manager records a fault log according to the first fault information and parses the fault log;

[0217] And replace the faulty component according to the faulty component information.

[0218] The beneficial effects of the technical solution provided by the embodiment of the present application are:

[0219] The present application can realize, through the joint debugging and compatibility of hardware and software, that after a fault occurs, the baseboard management controller can automatically collect the fault source data, integrate the parsing code in the manager, realize automatic parsing of the fault data and record detailed logs, assist in fault diagnosis and troubleshooting, and avoid the problem of inefficient technical personnel traveling to the customer site for support after a fault occurs.

[0220] The technical solution provided in the embodiment of the present application detects the operating status of the processor through a baseboard management controller, automatically collects and analyzes processor register information when it fails, records detailed fault logs, error content, and error causes based on the analysis results, maintains logs, and gives diagnostic suggestions, which can provide more accurate guidance for on-site operation and maintenance.

[0221] Embodiment 4

[0222] The present invention also provides a fault handling device, comprising:

[0223] one or more processors;

[0224] A storage device for storing one or more programs;

[0225] When the one or more programs are executed by the one or more processors, the one or more processors execute the following fault handling method:

[0226] Setting a security network configuration item and an error handling configuration item for the manager port, wherein the security network configuration item is used to prevent changes in the shutdown restart environment, and the error handling configuration item is used to determine different restart strategies according to the error type;

[0227] In response to receiving the fault warning information through the manager, acquiring processor register information, wherein the processor register information includes a current fault diagnosis state, and the fault diagnosis state includes at least one of error detection, error reporting, error recovery, and fault isolation;

[0228] Determining whether to collect fault diagnosis source data according to the current fault diagnosis state;

[0229] In response to collecting the fault diagnosis source data, judging whether the system is soft-restarted according to the safety network configuration item according to the error handling configuration item;

[0230] In response to a soft restart of the system according to the safety network configuration item, a fault type is determined according to the fault diagnosis source data, and the fault is automatically parsed and processed according to the fault type.

[0231] The beneficial effects of the technical solution provided by the embodiment of the present application are:

[0232] The present application can realize, through the joint debugging and compatibility of hardware and software, that after a fault occurs, the baseboard management controller can automatically collect the fault source data, integrate the parsing code in the manager, realize automatic parsing of the fault data and record detailed logs, assist in fault diagnosis and troubleshooting, and avoid the problem of inefficient technical personnel traveling to the customer site for support after a fault occurs.

[0233] The technical solution provided in the embodiment of the present application detects the operating status of the processor through a baseboard management controller, automatically collects and analyzes processor register information when it fails, records detailed fault logs, error content, and error causes based on the analysis results, maintains logs, and gives diagnostic suggestions, which can provide more accurate guidance for on-site operation and maintenance.

[0234] Embodiment 5

[0235] The present application provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the following fault handling method can be executed:

[0236] Setting a security network configuration item and an error handling configuration item for the manager port, wherein the security network configuration item is used to prevent changes in the shutdown restart environment, and the error handling configuration item is used to determine different restart strategies according to the error type;

[0237] In response to receiving the fault warning information through the manager, acquiring processor register information, wherein the processor register information includes a current fault diagnosis state, and the fault diagnosis state includes at least one of error detection, error reporting, error recovery, and fault isolation;

[0238] Determining whether to collect fault diagnosis source data according to the current fault diagnosis state;

[0239] In response to collecting the fault diagnosis source data, judging whether the system is soft-restarted according to the safety network configuration item according to the error handling configuration item;

[0240] In response to a soft restart of the system according to the safety network configuration item, a fault type is determined according to the fault diagnosis source data, and the fault is automatically parsed and processed according to the fault type.

[0241] The beneficial effects of the technical solution provided by the embodiment of the present application are:

[0242] The present application can realize, through the joint debugging and compatibility of hardware and software, that after a fault occurs, the baseboard management controller can automatically collect the fault source data, integrate the parsing code in the manager, realize automatic parsing of the fault data and record detailed logs, assist in fault diagnosis and troubleshooting, and avoid the problem of inefficient technical personnel traveling to the customer site for support after a fault occurs.

[0243] The technical solution provided in the embodiment of the present application detects the operating status of the processor through a baseboard management controller, automatically collects and analyzes processor register information when it fails, records detailed fault logs, error content, and error causes based on the analysis results, maintains logs, and gives diagnostic suggestions, which can provide more accurate guidance for on-site operation and maintenance.

[0244] Embodiment 6

[0245] The present application also provides a computer program product, including a computer program, which can implement the following fault handling method when executed by a processor:

[0246] Setting a security network configuration item and an error handling configuration item for the manager port, wherein the security network configuration item is used to prevent changes in the shutdown restart environment, and the error handling configuration item is used to determine different restart strategies according to the error type;

[0247] In response to receiving the fault warning information through the manager, acquiring processor register information, wherein the processor register information includes a current fault diagnosis state, and the fault diagnosis state includes at least one of error detection, error reporting, error recovery, and fault isolation;

[0248] Determining whether to collect fault diagnosis source data according to the current fault diagnosis state;

[0249] In response to collecting the fault diagnosis source data, judging whether the system is soft-restarted according to the safety network configuration item according to the error handling configuration item;

[0250] In response to a soft restart of the system according to the safety network configuration item, a fault type is determined according to the fault diagnosis source data, and the fault is automatically parsed and processed according to the fault type.

[0251] The beneficial effects of the technical solution provided by the embodiment of the present application are:

[0252] The present application can realize, through the joint debugging and compatibility of hardware and software, that after a fault occurs, the baseboard management controller can automatically collect the fault source data, integrate the parsing code in the manager, realize automatic parsing of the fault data and record detailed logs, assist in fault diagnosis and troubleshooting, and avoid the problem of inefficient technical personnel traveling to the customer site for support after a fault occurs.

[0253] The technical solution provided in the embodiment of the present application detects the operating status of the processor through a baseboard management controller, automatically collects and analyzes processor register information when it fails, records detailed fault logs, error content, and error causes based on the analysis results, maintains logs, and gives diagnostic suggestions, which can provide more accurate guidance for on-site operation and maintenance.

[0254] Embodiment 7

[0255] The present application provides a computer storage medium, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the following steps are implemented:

[0256] Setting a security network configuration item and an error handling configuration item for the manager port, wherein the security network configuration item is used to prevent changes in the shutdown restart environment, and the error handling configuration item is used to determine different restart strategies according to the error type;

[0257] In response to receiving the fault warning information through the manager, acquiring processor register information, wherein the processor register information includes a current fault diagnosis state, and the fault diagnosis state includes at least one of error detection, error reporting, error recovery, and fault isolation;

[0258] Determining whether to collect fault diagnosis source data according to the current fault diagnosis state;

[0259] In response to collecting the fault diagnosis source data, judging whether the system is soft-restarted according to the safety network configuration item according to the error handling configuration item;

[0260] In response to a soft restart of the system according to the safety network configuration item, a fault type is determined according to the fault diagnosis source data, and the fault is automatically parsed and processed according to the fault type.

[0261] In one embodiment, before setting the security network configuration item and the error handling configuration item for the manager port, the method includes:

[0262] Setting a fault diagnosis function in the manager, recording status information when a fault occurs, and monitoring the fault alarm information; importing a crash dump error code in the manager, and monitoring the fault diagnosis status;

[0263] The security network configuration item is enabled, and when a number of flood attack requests appear in the system, a processor reset is triggered to clear the flood attack resources;

[0264] Enable the error handling configuration item. When the system has several fault errors, the PCIe controller is reset to protect the system.

[0265] In one embodiment, the fault diagnosis source data includes a plurality of fault data blocks, the fault data blocks include status data and Internet Protocol identifier data, and the method of determining the fault type according to the fault diagnosis source data and automatically parsing the fault according to the fault type includes:

[0266] Judging the validity of the fault data block according to the status data;

[0267] In response to the fault data block being valid, determining the fault type according to the status data, the Internet protocol identifier data and the standard document, and performing analysis and processing on the fault according to the fault type;

[0268] The analyzing and processing the fault according to the fault type includes:

[0269] parsing the fault memory location according to the fault type;

[0270] Analyze the fault severity level and fault code according to the fault type;

[0271] Determine a detailed description of the fault according to the fault type;

[0272] Record a fault log for the fault according to the fault memory location, fault severity level, fault code and fault detailed description;

[0273] Develop a fault handling solution based on the fault log.

[0274] In one embodiment, the method comprises:

[0275] In response to not soft-restarting the system, the downtime interface is retained, the downtime processor register information is collected remotely or on-site through a collection tool, the downtime processor register information is parsed, and the fault is processed.

[0276] In one embodiment, the determining the fault type according to the fault diagnosis source data and automatically analyzing the fault according to the fault type further includes:

[0277] Cleaning and formatting the fault diagnosis source data to remove data noise, errors and redundant information;

[0278] Extracting fault features from the cleaned fault diagnosis source data, wherein the fault features include: trend analysis, statistical indicators and frequency domain analysis of time series data;

[0279] Classify fault features through the fault mode training model and determine the fault type;

[0280] Obtain a fault code manual or knowledge base according to the fault type, and analyze the fault log file according to the fault code manual or knowledge base;

[0281] Compare the fault log files with historical fault cases and analyze the faults.

[0282] Verify whether the fault handling is complete by retesting;

[0283] The checking whether the fault handling is completed by retesting includes:

[0284] Reproduce the conditions when the fault occurred to confirm whether the same fault occurs;

[0285] Obtain the application's log files to confirm whether there are any abnormal records;

[0286] Use performance monitoring tools to monitor the system's CPU usage, memory usage, disk, and network traffic to confirm whether the system performance has returned to normal;

[0287] Stress test the system to confirm whether it can operate;

[0288] Verify that all system configuration files have been modified and saved.

[0289] In one embodiment, the determining of the fault type according to the fault diagnosis source data and automatically analyzing the fault according to the fault type include:

[0290] Recording fault error logs through a manager, and optimizing fault handling according to the fault error logs;

[0291] The optimizing the fault handling according to the fault error log includes:

[0292] Get the fault error log;

[0293] Unifying the log format of the fault error log and structuring it;

[0294] Aggregate and statistically analyze the structured fault error log data to determine the fault mode and pattern;

[0295] Create a fault tree based on failure modes and rules to analyze the failures and optimize the fault handling strategy.

[0296] In one embodiment, the method comprises:

[0297] When a problem occurs in the automatic processing of the fault, the system is soft-restarted, the processor register information is obtained, and the fault error type and the faulty component are determined according to the processor register information;

[0298] and creating first fault information according to the fault error type and the fault component information;

[0299] sending the first fault information to the manager;

[0300] The manager records a fault log according to the first fault information and parses the fault log;

[0301] And replace the faulty component according to the faulty component information.

[0302] This application can achieve that after a fault occurs, the manager can automatically collect fault source data and handle the fault through compatibility of hardware and software, thereby improving the correctness of fault handling.

[0303] Figure 6 An exemplary system provided for Embodiment 7 of the present application that can be used to implement various embodiments described in the present application;

[0304] like Figure 6As shown, in some embodiments, the system can be used as any of the above-mentioned devices for fault handling in each of the described embodiments. In some embodiments, the system may include one or more computer-readable media (e.g., system memory or NVM / storage device) having results and one or more processors (e.g., (one or more) processors) coupled to the one or more computer-readable media and configured to execute the results to implement the module to perform the actions described in the present application.

[0305] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0306] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0307] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A fault handling method, characterized in that: The method comprises: Setting a security network configuration item and an error handling configuration item for the manager port, wherein the security network configuration item is used to prevent changes in the shutdown restart environment, and the error handling configuration item is used to determine different restart strategies according to the error type; In response to receiving the fault warning information through the manager, acquiring processor register information, wherein the processor register information includes a current fault diagnosis state, and the fault diagnosis state includes at least one of error detection, error reporting, error recovery, and fault isolation; Determining whether to collect fault diagnosis source data according to the current fault diagnosis state; In response to collecting the fault diagnosis source data, judging whether the system is soft-restarted according to the safety network configuration item according to the error handling configuration item; In response to a soft restart of the system according to the safety network configuration item, a fault type is determined according to the fault diagnosis source data, and the fault is automatically parsed and processed according to the fault type.

2. The fault handling method according to claim 1, characterized in that: Before setting the security network configuration item and the error handling configuration item for the manager port, the method includes: Setting a fault diagnosis function in the manager, recording status information when a fault occurs, and monitoring the fault alarm information; importing a crash dump error code in the manager, and monitoring the fault diagnosis status; The security network configuration item is enabled, and when a number of flood attack requests appear in the system, a processor reset is triggered to clear the flood attack resources; Enable the error handling configuration item. When the system has several fault errors, the PCIe controller is reset to protect the system.

3. The fault handling method according to claim 1, characterized in that: The fault diagnosis source data includes a plurality of fault data blocks, each of which includes status data and Internet Protocol identifier data. The method of determining the fault type according to the fault diagnosis source data and automatically analyzing the fault according to the fault type includes: Judging the validity of the fault data block according to the status data; In response to the fault data block being valid, determining the fault type according to the status data, the Internet protocol identifier data and the standard document, and performing analysis and processing on the fault according to the fault type; The analyzing and processing the fault according to the fault type includes: parsing the fault memory location according to the fault type; Analyze the fault severity level and fault code according to the fault type; Determine a detailed description of the fault according to the fault type; Record a fault log for the fault according to the fault memory location, fault severity level, fault code and fault detailed description; Develop a fault handling solution based on the fault log.

4. The fault handling method according to claim 1, characterized in that: The method comprises: In response to not soft-restarting the system, the downtime interface is retained, the downtime processor register information is collected remotely or on-site through a collection tool, the downtime processor register information is parsed, and the fault is processed.

5. The fault handling method according to claim 1, characterized in that: The determining of the fault type according to the fault diagnosis source data and automatically analyzing the fault according to the fault type also includes: Cleaning and formatting the fault diagnosis source data to remove data noise, errors and redundant information; Extracting fault features from the cleaned fault diagnosis source data, wherein the fault features include: trend analysis, statistical indicators and frequency domain analysis of time series data; Classify fault features through the fault mode training model and determine the fault type; Obtain a fault code manual or knowledge base according to the fault type, and analyze the fault log file according to the fault code manual or knowledge base; Compare the fault log files with historical fault cases and analyze the faults. Verify whether the fault handling is complete by retesting; The checking whether the fault handling is completed by retesting includes: Reproduce the conditions when the fault occurred to confirm whether the same fault occurs; Obtain the application's log files to confirm whether there are any abnormal records; Use performance monitoring tools to monitor the system's CPU usage, memory usage, disk, and network traffic to confirm whether the system performance has returned to normal; Stress test the system to confirm whether it can operate; Verify that all system configuration files have been modified and saved.

6. The fault handling method according to claim 1, characterized in that: After determining the fault type according to the fault diagnosis source data and automatically analyzing the fault according to the fault type, the method includes: Recording fault error logs through a manager, and optimizing fault handling according to the fault error logs; The optimizing the fault handling according to the fault error log includes: Get the fault error log; Unifying the log format of the fault error log and structuring it; Aggregate and statistically analyze the structured fault error log data to determine the fault mode and pattern; Create a fault tree based on failure modes and rules to analyze the failures and optimize the fault handling strategy.

7. The fault handling method according to claim 1, characterized in that: The method comprises: When a problem occurs in the automatic processing of the fault, the system is soft-restarted, the processor register information is obtained, and the fault error type and the faulty component are determined according to the processor register information; and creating first fault information according to the fault error type and the fault component information; sending the first fault information to the manager; The manager records a fault log according to the first fault information and parses the fault log; And replace the faulty component according to the faulty component information.

8. A fault handling device, characterized in that: The device comprises: A setting module, used to set a security network configuration item and an error handling configuration item for the manager port, wherein the security network configuration item is used to prevent changes in the shutdown restart environment, and the error handling configuration item is used to determine different restart strategies according to the error type; a receiving module, configured to obtain processor register information in response to receiving fault warning information through the manager, wherein the processor register information includes a current fault diagnosis state, and the fault diagnosis state includes at least one of error detection, error reporting, error recovery, and fault isolation; A judgment module, used for judging whether to collect fault diagnosis source data according to the current fault diagnosis state; In response to collecting the fault diagnosis source data, judging whether the system is soft-restarted according to the safety network configuration item according to the error handling configuration item; A processing module is used for, in response to soft restarting the system according to the safety network configuration item, determining a fault type according to the fault diagnosis source data, and automatically analyzing and processing the fault according to the fault type.

9. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the fault handling method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the fault handling method according to any one of claims 1 to 7 are implemented.

Images