A system fault diagnosis method, device, equipment and storage medium

Abstract

Embodiments of the present application relate to the technical field of computer, in particular to a system fault diagnosis method, device and equipment and storage medium, aiming at quickly diagnosing and positioning the cause of system fault. The method comprises: classifying and extracting fault data from system log data to obtain multiple different types of fault data; obtaining component state information of all fault components according to the multiple different types of fault data; respectively performing fault correlation analysis on different types of fault components through corresponding fault diagnosis modules according to the component state information of the fault components to obtain multiple fault causes; and comprehensively analyzing the multiple fault causes based on a two-line solidification rule to obtain a final fault cause.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and more specifically, to a system fault diagnosis method, apparatus, device, and storage medium. Background Technology

[0002] IPMI (Intelligent Platform Management Interface) is a management system that spans multiple operating systems, firmware, and hardware platforms, intelligently monitoring, controlling, and automatically reporting the operational status of a large number of servers. During IPMI system operation, IPMI system logs, also known as SEL (IPMI System Event Log) logs, are generated. These logs record the full and cumulative changes in components, thresholds, and states of various devices running under IPMI system management. Fault diagnosis of the IPMI system generally requires analysis of the SEL logs to pinpoint the cause of the fault. In existing technologies, fault analysis based on SEL logs typically uses keyword analysis to obtain fault data from the SEL logs, which is then handed over to maintenance personnel for analysis.

[0003] Current technologies can only retrieve fault data from SEL logs through keyword searches, which cannot quickly analyze the root cause of the fault. Summary of the Invention

[0004] This invention provides a system fault diagnosis method, apparatus, device, and storage medium, which aims to quickly diagnose and locate the cause of system faults.

[0005] The first aspect of this invention provides a system fault diagnosis method, the method comprising:

[0006] Fault data is categorized and extracted from system log data to obtain various types of fault data;

[0007] Based on the various types of fault data, the component status information of all faulty components is obtained;

[0008] Based on the component status information of the faulty component, the corresponding fault diagnosis module performs fault correlation analysis on different types of faulty components to obtain multiple fault causes.

[0009] Based on the two-line solidification rule, a comprehensive analysis of the multiple fault causes is performed to obtain the final fault cause.

[0010] Optionally, the fault data classification and extraction of the system logs yields various types of fault data, including:

[0011] Based on the pre-defined fault keywords, search for the fault data corresponding to the fault keywords in the system log data;

[0012] The fault data is classified according to the category of the fault keywords to obtain the various types of fault data.

[0013] Optionally, obtaining the component status information of all faulty components based on the various types of fault data includes:

[0014] For each type of fault data, determine the relevant fault data of the faulty component included in the fault data;

[0015] Based on the relevant fault data of the faulty component, the component status information of each faulty component is determined. The component status information includes: component name, component fault content, and component fault status.

[0016] Optionally, before performing fault correlation analysis on different types of faulty components based on the component status information of the faulty component and obtaining multiple fault causes through the corresponding fault diagnosis module, the method further includes:

[0017] When the component status information contains an unresolved component name code, the component name code is extracted to obtain the target component name code;

[0018] In a pre-set component name encoding library, the component name corresponding to the target component name code is searched to obtain the target component name;

[0019] The component status information is obtained by replacing the component name code in the fault data with the target component name.

[0020] Optionally, based on the component status information of the faulty component, the corresponding fault diagnosis module performs fault correlation analysis on different types of faulty components to obtain multiple fault causes, including:

[0021] Based on the component status information of the faulty component, determine the component type of the faulty component;

[0022] Based on the component type of the faulty component, determine the related component type of the faulty component;

[0023] By using the fault diagnosis module corresponding to the faulty component and the related type of component, a fault correlation analysis is performed on the faulty component and the related type of component to obtain the cause of the fault.

[0024] Optionally, the step of performing fault correlation analysis on the faulty component and its related types of components through the fault diagnosis module corresponding to the faulty component and the related types of components to obtain the cause of the fault includes:

[0025] Based on the preset correspondence between components and fault diagnosis modules, determine the faulty component and the fault diagnosis module corresponding to the related type of component of the faulty component;

[0026] The fault diagnosis module performs fault correlation analysis on the faulty component and related components according to preset fault diagnosis rules to obtain the cause of the fault.

[0027] Optionally, the step of comprehensively analyzing the multiple fault causes based on the two-line solidification rule to obtain the final fault cause includes:

[0028] When the relationship between the fault diagnosis modules corresponding to the multiple fault causes is parallel, the fault causes are output to obtain the final fault cause;

[0029] When the relationship between the fault diagnosis modules corresponding to the multiple fault causes is an association relationship, the multiple fault causes are analyzed according to the preset fault diagnosis rules to obtain the final fault cause.

[0030] A second aspect of the present invention provides a system fault diagnosis device, the device comprising:

[0031] The fault data extraction module is used to classify and extract fault data from system log data, resulting in various types of fault data.

[0032] The component status information determination module is used to obtain the component status information of all faulty components based on the various types of fault data.

[0033] The fault diagnosis module is used to perform fault correlation analysis on different types of faulty components based on the component status information of the faulty components, and obtain multiple fault causes through the corresponding fault diagnosis module;

[0034] The comprehensive analysis module is used to perform a comprehensive analysis of the multiple fault causes based on the two-line solidification rules to obtain the final fault cause.

[0035] Optionally, the fault data extraction module includes:

[0036] The fault data lookup submodule is used to look up the fault data corresponding to the pre-defined fault keywords in the system log data.

[0037] The fault data classification submodule is used to classify the fault data according to the category of the fault keywords to obtain the various types of fault data.

[0038] Optionally, the component status information determination module includes:

[0039] The component-related data determination submodule is used to determine the relevant fault data of the faulty component included in the fault data for each type of fault data.

[0040] The component status information determination submodule is used to determine the component status information of each faulty component based on the relevant fault data of the faulty component. The component status information includes: component name, component fault content, and component fault status.

[0041] Optionally, before performing fault correlation analysis on different types of faulty components based on the component status information of the faulty component and obtaining multiple fault causes through the corresponding fault diagnosis module, the method further includes:

[0042] When the component status information contains an unresolved component name code, the component name code is extracted to obtain the target component name code;

[0043] In a pre-set component name encoding library, the component name corresponding to the target component name code is searched to obtain the target component name;

[0044] The component status information is obtained by replacing the component name code in the fault data with the target component name.

[0045] Optionally, the fault diagnosis module includes:

[0046] The component type determination submodule is used to determine the component type of the faulty component based on the component status information of the faulty component;

[0047] The related type component determination submodule is used to determine the related type components of the faulty component based on the component type of the faulty component;

[0048] The fault cause determination submodule is used to perform fault correlation analysis on the faulty component and related type components through the fault diagnosis module corresponding to the faulty component and related type components to obtain the fault cause.

[0049] Optionally, the fault cause determination submodule includes:

[0050] The fault diagnosis module determines the sub-module, which is used to determine the faulty component and the fault diagnosis module corresponding to the related type of component of the faulty component according to the preset correspondence between the component and the fault diagnosis module;

[0051] The fault correlation analysis submodule is used to perform fault correlation analysis on the faulty component and related components of the faulty component according to preset fault diagnosis rules through the fault diagnosis module, so as to obtain the cause of the fault.

[0052] Optionally, the comprehensive analysis module includes:

[0053] The first cause determination submodule is used to output the fault cause when the relationship between the fault diagnosis modules corresponding to the multiple fault causes is parallel, so as to obtain the final fault cause.

[0054] The second cause determination submodule is used to perform correlation analysis on the multiple fault causes according to preset fault diagnosis rules when the relationship between the fault diagnosis modules corresponding to the multiple fault causes is an association relationship, so as to obtain the final fault cause.

[0055] A third aspect of the present invention provides a readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described in the first aspect of the present invention.

[0056] A fourth aspect of the present invention provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, it implements the steps of the method described in the first aspect of the present invention.

[0057] The system fault diagnosis method provided by this invention performs fault data classification and extraction on system log data to obtain multiple types of fault data; based on the multiple types of fault data, the component status information of all faulty components is obtained; based on the component status information of the faulty components, the corresponding fault diagnosis module performs fault location analysis on different types of faulty components to obtain multiple fault causes; based on the two-line solidification rule, the multiple fault causes are comprehensively analyzed to obtain the final fault cause. This method first classifies and extracts fault data from system log data, identifying various types of faults and initially identifying the faulty components. Then, based on the specific information in the fault data, the status information of the faulty components is determined, further locating and analyzing the status of each faulty component. This helps to determine the correlation between these components and analyze the causal relationship of the fault. Next, different fault diagnosis modules are used to perform fault correlation analysis on different types of faulty components, obtaining multiple fault causes. Using different diagnostic modules for fault correlation analysis on different types of faulty components allows for in-depth analysis of the causes of faults based on the correlation between components of the same type, facilitating the identification of the root cause. Finally, multiple fault causes are comprehensively analyzed based on the two-line solidification rule to obtain the final fault cause. The comprehensive analysis of the fault causes obtained from multiple diagnostic modules based on the two-line solidification rule allows for further determination of the root cause of the system fault based on the correlation between modules, thus enabling a comprehensive and rapid analysis of the root cause of the entire system fault. Attached Figure Description

[0058] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0059] Figure 1 This is a flowchart of a system fault diagnosis method proposed in an embodiment of the present invention;

[0060] Figure 2 This is a schematic diagram of a fault data extraction process proposed in an embodiment of the present invention;

[0061] Figure 3 This is a schematic diagram of a fault diagnosis process proposed in an embodiment of the present invention;

[0062] Figure 4 This is a schematic diagram of the comprehensive analysis process proposed in an embodiment of the present invention;

[0063] Figure 5This is a schematic flowchart of a system fault diagnosis method proposed in an embodiment of the present invention;

[0064] Figure 6 This is a schematic diagram of a system fault diagnosis device proposed in an embodiment of the present invention. Detailed Implementation

[0065] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0066] refer to Figure 1 , Figure 1 This is a flowchart of a system fault diagnosis method proposed in an embodiment of the present invention.

[0067] like Figure 1 As shown, the method includes the following steps:

[0068] S11: Perform fault data classification and extraction on system log data to obtain various types of fault data.

[0069] In this embodiment, system log data is information about system hardware, software, and problems that occur during system runtime, recorded in the background. It also monitors events happening within the system, allowing for the determination of error causes or the search for traces of attacks. Fault data consists of various hardware faults recorded in the system log. Fault classification and extraction involves categorizing and extracting fault data from the log based on the type of component corresponding to the fault data.

[0070] In this embodiment, the system log data is read by a query tool to find the fault data in the system log data. Then, the fault data is classified and extracted according to the type of component corresponding to the fault data to obtain various types of fault data.

[0071] In this embodiment, taking the IPMI system as an example, the IPMI system log is also called the SEL log. The SEL log records fault data in the log according to certain rules and specifications. The SEL log is read by the log query tool. According to the corresponding fault data recording rules, the fault data in the SEL log is found. Then, according to the type of the component corresponding to the fault data, the fault data is extracted according to different types. For example, the fault of the memory-related component is a memory fault, the fault of the CPU (central processing unit) component is a CPU fault, and the fault of the hard disk component is a hard disk fault.

[0072] S12: Based on the various types of fault data, obtain the component status information of all faulty components.

[0073] In this embodiment, the faulty component is a component that has appeared in the fault data. The faulty component can be a component that is currently malfunctioning or a component that has previously malfunctioned and been repaired. The component status information is the relevant status information of the component extracted and summarized from the component's fault data. It is a summary of the fault information of the same faulty component, including the name of the faulty component, the specific content of the fault, the status of the fault, etc. The fault status is whether the problem currently exists, the problem has been repaired, the problem has recurred, or the problem has occurred before.

[0074] In this embodiment, after obtaining various types of fault data, further extraction is performed on this fault data to obtain the status information of the faulty component. For example... Figure 2 As shown, Figure 2 This is a schematic diagram of a fault data extraction process proposed in an embodiment of the present invention. First, the SEL log is extracted once to identify fault data of various fault types. Simultaneously, the reference time (bmctime), i.e., the occurrence time of each fault data, is obtained from the SEL log. Then, the SEL log is extracted a second time according to rules recorded in the general rule base to obtain the component's status information, i.e., the faulty component, fault content, and fault status (current problem, recently fixed, problem reproduced, previously occurred). Next, the components are classified and summarized, and component correction is performed using a Sensor (component name encoding library), finally obtaining fault metadata, which contains the component's status information.

[0075] For example, further extraction in memory-related faults reveals that the faulty component is memory slot CPU0_C0D0, the fault content is Uncorrectable ECC (encountered an uncorrectable ECC error, ECC stands for Error Checking and Correcting), and the component's fault status is currently problematic.

[0076] S13: Based on the component status information of the faulty component, the corresponding fault diagnosis module performs fault correlation analysis on different types of faulty components to obtain multiple fault causes.

[0077] In this embodiment, the fault diagnosis module is a module for diagnosing and analyzing faulty components. A fault diagnosis module is a piece of encapsulated code. By running this code, the corresponding fault diagnosis rules can be invoked to perform correlation analysis on the faulty components and thus obtain the cause of the fault.

[0078] In this embodiment, the fault diagnosis module includes a computing diagnosis submodule, a storage diagnosis submodule, a boot diagnosis submodule, a thermal diagnosis submodule, and a power supply diagnosis submodule. The computing diagnosis submodule diagnoses faults in computing-related components of the system, such as the CPU, memory, and PCIe (Peripheral Component Interconnect Express) bus. The storage diagnosis submodule diagnoses faults in storage-related components, such as hard drives. The boot diagnosis submodule diagnoses faults in system-related components, such as drives. The thermal diagnosis submodule diagnoses faults in heat dissipation-related components, such as fans and temperature sensors. The power supply diagnosis submodule diagnoses faults in power supply-related components, such as power connectors.

[0079] In this embodiment, after obtaining the component status information of each component, the component status information is input into the corresponding fault diagnosis module according to the type of the component. The fault diagnosis module performs correlation analysis on the faulty component through the corresponding diagnosis rules. That is, based on the cause of the faulty component and the cause of the fault of the related components, the causal relationship between the causes of the fault is analyzed, the main cause of the fault of this type of component is determined, and the fault cause is assigned a corresponding weight according to the main cause of the fault. The fault cause with the largest weight is selected as the main fault cause, and the fault diagnosis module outputs the fault cause.

[0080] For example, taking the computational diagnostic submodule as an example, the computational related components are CPU, memory, and PCIe. The component status information of these three types of components is input into the computational diagnostic submodule. The computational diagnostic submodule analyzes the fault data of these three types of components according to the pre-defined CPU fault diagnosis rules, PCIe fault diagnosis rules, and memory diagnosis rules, and determines the correlation between the fault data of these three types of components. For example, if the fault data indicates that the CPU IERR (motherboard) has failed and the memory CPU0_C0DD has failed, then according to the pre-set rules, it is determined that the memory failure caused the CPU failure, and the cause of the failure is determined to be the memory CPU0_C0DD failure.

[0081] S14: Based on the two-line solidification rule, a comprehensive analysis of the multiple fault causes is performed to obtain the final fault cause.

[0082] In this embodiment, the two-line solidification rule is a classification rule for the relationships between fault diagnosis modules. The two-line solidification rule divides the relationships between fault diagnosis modules into two categories: parallel relationships and associated relationships. Parallel relationships mean that the types of components corresponding to the two calculation and diagnosis modules are not related, while associated relationships mean that the types of components corresponding to the two calculation and diagnosis modules are related. The ultimate fault cause is one or more root causes that lead to system failure.

[0083] In this embodiment, after obtaining multiple fault causes, the correlation between the components corresponding to these fault causes is analyzed based on the two-line solidification rule. Then, a comprehensive analysis is performed based on the correlation between the faulty components to obtain the final fault cause. When the component types handled by the calculation and diagnosis module that outputs the fault cause are not related, they are considered to be parallel, and both fault causes are output as the final fault cause. When the component types handled by the calculation and diagnosis module that outputs the fault cause are related, the causal relationship between the fault causes is judged to obtain the final fault cause.

[0084] For example, computation-related component types and storage-related component types are generally unrelated. Therefore, when the fault cause is memory failure and hard drive failure, both fault causes will be output as the final fault cause. If the thermal diagnostic module outputs the fault cause as air intake and CPU high temperature alarm, while the computation diagnostic module outputs the fault cause as CPU frequency lock, then the final fault cause is the CPU frequency lock problem caused by high air intake temperature.

[0085] In this embodiment, the system log data is first classified and extracted to obtain various types of fault data. Then, the fault data of different categories is further extracted to obtain the component status information of each faulty component. Through different types of diagnostic modules, the correlation analysis of faults occurring in the same type of component is performed to obtain the fault cause of each type of component. Then, the fault causes diagnosed by multiple diagnostic modules are comprehensively analyzed. Based on the correlation between the component types corresponding to each diagnostic module, the final fault cause is determined. This realizes the extraction and analysis of fault data based on system log data, and the determination of the root cause of system failure based on the correlation between fault data.

[0086] In another embodiment of the present invention, the fault data classification and extraction of the system log yields various types of fault data, including:

[0087] S21: Based on the pre-set fault keywords, search for the fault data corresponding to the fault keywords in the system log.

[0088] In this embodiment, the fault keywords are set according to specific rules and are keywords that frequently appear in fault data. If a fault keyword appears in a certain data entry in the system log data, then that data is regarded as the corresponding fault data.

[0089] In this embodiment, system log data is read by a log query tool. The log query tool searches for data containing fault keywords in all data in the system log based on pre-set fault keywords, and extracts the data containing fault keywords as the corresponding fault data.

[0090] For example, fault keywords could be "memory failure", "hard drive failure", "abnormal temperature", etc.

[0091] S22: Classify the fault data according to the category of the fault keywords to obtain the various types of fault data.

[0092] In this embodiment, fault keywords belong to different categories. Based on the categories of fault keywords, various types of fault keywords are classified to obtain various types of fault data.

[0093] For example, when the fault keyword is "hard drive failure", the fault keyword category belongs to storage faults, and when the fault keyword is "abnormal temperature", the fault keyword category belongs to heat dissipation faults.

[0094] In this embodiment, fault data is extracted from system log data using pre-defined fault keywords. The extracted fault data is then categorized according to the type of fault keywords, resulting in various types of fault data. This not only extracts fault data but also categorizes it, which is beneficial for subsequent correlation analysis of fault causes.

[0095] In another embodiment of the present invention, obtaining the component status information of all faulty components based on the various types of fault data includes:

[0096] S31: For each type of fault data, determine the relevant fault data of the faulty component included in the fault data.

[0097] In this embodiment, the faulty component is the component in the system that has failed, and the relevant data of the faulty component refers to the fault data that includes the name of the faulty component.

[0098] In this embodiment, after obtaining multiple types of fault data, all faulty components are identified in each type of fault data, and then the relevant fault data of each faulty component is searched to determine the relevant fault data of that faulty component.

[0099] For example, the faulty component is memory slot CPU0_C0D0, and the relevant fault data for this memory slot is: "Memory slot CPU0_C0D0; Error Uncorrectable ECC; January 1, 2022, 22:00". "Memory slot CPU0_C0D0; Error Uncorrectable ECC; May 6, 2022, 13:00". "Memory slot CPU0_C0D0; Replace memory module; June 12, 2022, 21:35".

[0100] S32: Based on the relevant fault data of the faulty component, determine the component status information of each faulty component, wherein the component status information includes: component name, component fault content, and component fault status.

[0101] In this embodiment, the component status information refers to the relevant status information of the faulty component, including the component name, component fault content, and component fault status. The component name is the name registered by the component in the system; the component fault content is the specific fault content recorded by the system when the component malfunctions; and the component fault status includes the current problem of the component, the component's repair records, whether the repaired problem has recurred, and problems that the component has previously experienced.

[0102] In this embodiment, after obtaining the relevant fault data of the faulty component, the content of this fault data is analyzed to obtain a record of the specific problems that have occurred in the faulty component and to obtain the component status information of the faulty component.

[0103] In this embodiment, when the fault data shows that the faulty component has a fault and there is no repair record, the status of the faulty component is determined to be currently having a problem; when the fault data shows that the faulty component has a fault and subsequent records show that it has been repaired, it is determined that a problem has occurred and has been repaired; when the fault data shows that the faulty component repeatedly has the same problem, it is determined that the problem of the faulty component has been reproduced.

[0104] In another embodiment of the present invention, before performing fault correlation analysis on different types of faulty components based on the component status information of the faulty component and obtaining multiple fault causes through the corresponding fault diagnosis module, the method further includes:

[0105] S41: When there is an unresolvable component name code in the component status information, the component name code is extracted to obtain the target component name code.

[0106] In this embodiment, the component name code refers to the code that represents a specific component. It can be a combination of numbers or letters. The target component name code is the name code of the faulty component that cannot be parsed or is parsed incorrectly.

[0107] In this embodiment, some products or software use a certain rule of encoding to replace the name of the component when recording system operation data. At this time, the component name in the IPMI system is recorded in the form of component name encoding. When there is a component name encoding that cannot be parsed or is parsed incorrectly in the component status information, the component name encoding is extracted to obtain the target component name encoding.

[0108] S42: In the pre-set component name encoding library, search for the component name corresponding to the target component name encoding to obtain the target component name.

[0109] S43: Replace the component name code in the fault data with the target component name to obtain the component status information.

[0110] In this embodiment, the component name encoding library stores the names of all devices controlled by the IPMI system and all components of the system, as well as the name codes corresponding to those component names.

[0111] In this embodiment, a component name encoding library is pre-set to confirm the names of all devices and system components in the IPMI system. When a component name has a corresponding name code in some system or software, the component name and its corresponding name code are stored in the name encoding library. The component name encoding library is also called Senor. After obtaining the target component name code through S41, the component name corresponding to the component name code is searched in the name encoding library. The found component name code is used to replace the component name code in the fault data corresponding to the faulty component. The replaced fault data does not contain the component name code and only displays the corresponding component name.

[0112] In this embodiment, to prevent SEI logs from being unparsed or parsed incorrectly due to certain products, devices, or software running under the IPMI system, the naming of components is corrected by querying the component name encoding library. This ensures that subsequent fault cause analysis is more accurate and that the fault cause is not affected by unparsed or incorrectly parsed component names.

[0113] In another embodiment of the present invention, based on the component status information of the faulty component, a fault correlation analysis is performed on different types of faulty components through a corresponding fault diagnosis module to obtain multiple fault causes, including:

[0114] S51: Determine the component type of the faulty component based on the component status information of the faulty component.

[0115] In this embodiment, the component type is obtained by classifying the components according to their specific location and function on the hardware device.

[0116] In this embodiment, after obtaining the component status information of the faulty component through S32, the component name of the faulty component can be confirmed in the component status information, and then the component type of the faulty component can be determined based on the component name of the faulty component.

[0117] For example, the arithmetic logic unit (ALU), control unit, and registers are located on the CPU and are components of the CPU that assist the CPU in performing various calculations; they belong to the CPU category. Memory modules belong to the memory category. Fans are cooling devices and belong to the cooling category.

[0118] S52: Determine the relevant type of component based on the component type of the faulty component.

[0119] In this embodiment, the related type of component is the type of component that belongs to the same functional module as the component. In system devices, components are often related to each other. A certain functional module often requires multiple related components to work together to realize the function of the module.

[0120] In this embodiment, based on the component type of the faulty component, the related type of component can be determined. Based on the component type of the faulty component, the working module to which the faulty component belongs can be determined. Components that serve or are controlled by the working module can be considered as related components of the faulty component.

[0121] For example, the motherboard belongs to the CPU category, the memory module belongs to the memory category, and the PCIe bus belongs to the PCIe category. CPU-type components, memory-type components, and PCIe-type components all belong to the computing modules of the device. These three types of components are related to each other and all belong to the computing category.

[0122] S53: By using the fault diagnosis module corresponding to the faulty component and the related type of component, perform fault correlation analysis on the faulty component and the related type of component to obtain the cause of the fault.

[0123] In this embodiment, the cause of the fault is the root cause of the faulty component and related components. Since the fault data of several related components when they fail may be due to the component's own sound fault or the fault caused by the influence of related components, it is necessary to locate the root cause of the fault.

[0124] In this embodiment, after identifying the faulty component and related components, the corresponding fault diagnosis module is determined, and the fault data of the faulty component and related components are input into the corresponding fault diagnosis module. The cause of the fault is then obtained through the corresponding fault diagnosis module. The specific steps include:

[0125] S53-1: Based on the preset correspondence between components and fault diagnosis modules, determine the faulty component and the fault diagnosis module corresponding to the related type of component of the faulty component.

[0126] In this embodiment, each component has its own fault diagnosis module. Components with faults of the same type belong to the same fault diagnosis module. The component type to which the component belongs and the fault diagnosis module corresponding to the component are all preset.

[0127] In this embodiment, the faulty component and the corresponding fault diagnosis module are determined according to the preset correspondence between components and fault diagnosis modules.

[0128] For example, the arithmetic logic unit (ALU) belongs to the CPU class of components, and the fast storage belongs to the memory class of components. Both the CPU class of components and the memory class of components belong to the computing module. Therefore, the ALU and the fast storage correspond to the computing diagnostic module.

[0129] S53-2: Through the fault diagnosis module, according to the preset fault diagnosis rules, the faulty component and related components of the faulty component are subjected to fault correlation analysis to obtain the cause of the fault.

[0130] In this embodiment, the fault diagnosis rule is a judgment rule for the cause of component failure derived from common sense rules and past diagnostic experience. The fault correlation analysis is an analysis method that finds the mutual influence between these components based on the component failure data, explores the correlation between the failures of these components, and finds the root cause of the failure.

[0131] In this embodiment, the fault data of the faulty component and related components are input into the corresponding fault diagnosis module. The fault diagnosis module calls the corresponding fault diagnosis rules to perform fault correlation analysis on the faulty component and related components to obtain the cause of the fault. The fault rules include general rules and typical rules. General rules are commonly used for analyzing system component faults, typically including common component fault scenarios such as poor contact leading to component failure, excessive load leading to component failure, etc. For interconnected components, when one component fails, the operation of another component, even if it is not faulty, will be hindered. For example, when memory is damaged, the CPU's computing frequency will be limited, leading to operational failure. When a fan fails, components such as the CPU will also experience abnormal temperature rise. These rules are summarized by maintenance personnel during daily system diagnosis and are therefore called typical rules. The rules also include rules corresponding to each type of component, such as CPU rules (fault diagnosis rules for CPU-type components), PCIE rules (fault diagnosis rules for PCIE-type components), and memory rules (fault diagnosis rules for memory-type components).

[0132] In this embodiment, a rule base is pre-defined. The rule base includes general rules and typical rules for diagnosing various modules of the system, as well as rules for extracting SEL log data. These rules are pre-collected and summarized. When data is received, the corresponding rules are directly called to analyze and process the current data and obtain the analysis results.

[0133] For example, see reference Figure 3 , Figure 3 This is a schematic diagram of a fault diagnosis process proposed in an embodiment of the present invention, as shown below. Figure 3As shown, taking the computational diagnostic submodule as an example, both CPU and memory belong to the computational category. The fault metadata (multi-CPU IRRR fault, memory CPU0_C0D0GU fault) is input into the computational diagnostic module. The computational diagnostic module combines the CPU rules, memory rules, and PCIE rules in the rule base to analyze the fault data, determine that the CPU fault is caused by the memory fault, and then output the fault cause as memory CPU0_C0D0GU fault.

[0134] In this embodiment, multiple types of fault diagnosis modules are used to analyze and process fault data of corresponding types. Based on the type of faulty component, the corresponding rules are called from the rule base to analyze the fault data, obtain the cause of the fault, analyze the causal relationship between faults occurring between related components within a working module, find the root cause of the fault, and thus achieve rapid location of the fault cause within a functional module.

[0135] In another embodiment of the present invention, the step of comprehensively analyzing the multiple fault causes based on the two-line solidification rule to obtain the final fault cause includes:

[0136] S61: When the relationship between the fault diagnosis modules corresponding to the multiple fault causes is parallel, the fault causes are output to obtain the final fault cause.

[0137] In this embodiment, the parallel rule means that the component types corresponding to the two fault diagnosis modules are not related to each other.

[0138] In this embodiment, when the component types corresponding to the two fault diagnosis modules are not related to each other, the fault causes input by the two fault diagnosis modules are both output as the final fault causes. That is, the fault causes diagnosed by the two fault diagnosis modules are important causes affecting the operation of the entire system and need to be repaired. The system can only be restored to stable operation after both faulty components are repaired.

[0139] For example, when both the computational diagnostic module and the storage diagnostic module output the cause of the fault, the components corresponding to the computational module and the storage module in the system are generally not related to each other, so the fault causes output by the two modules are the final fault causes.

[0140] S62: When the relationship between the fault diagnosis modules corresponding to the multiple fault causes is an association relationship, the multiple fault causes are analyzed according to the preset fault diagnosis rules to obtain the final fault cause.

[0141] In this embodiment, the association relationship means that the types of the components corresponding to the two calculation and diagnostic modules are mutually related.

[0142] In this embodiment, when the types of components corresponding to the two calculation and diagnosis modules are related, the fault causes input by the two fault diagnosis modules also have a causal relationship. That is, the fault of one component usually causes the other component to fail. When one of the faulty components is replaced, the affected component will return to normal and there is no need to replace the other component.

[0143] For example, the thermal diagnostic module outputs a high-temperature alarm for both the air intake and the CPU, while the computing diagnostic module outputs a CPU frequency lock. Since the components corresponding to the thermal module and the computing module in the system are generally interconnected, a correlation exists between the two types of components. Therefore, analysis based on the correlation rules reveals the final fault cause to be a CPU frequency lock issue caused by high air intake temperature. Clearing the air intake or cleaning and replacing the fan will automatically resolve the CPU high-temperature frequency lock issue.

[0144] In this embodiment, reference Figure 4 , Figure 4 This is a schematic diagram of the comprehensive analysis process proposed in an embodiment of the present invention, as shown below. Figure 4 As shown in the diagram, the conclusions of each module are input into the comprehensive analysis module. The conclusions (fault causes) of each module are analyzed through parallel rules and association rules in the rule base to obtain the final conclusion (final fault cause). Based on the second-line solidified rules, the fault causes obtained by each diagnostic module are comprehensively analyzed to obtain the final fault cause. The operation and maintenance personnel only need to solve the final fault cause obtained by the comprehensive analysis module to repair the entire system, which improves the efficiency of system problem location and repair.

[0145] refer to Figure 5 ,like Figure 5 As shown, Figure 5This is a flowchart illustrating a system fault diagnosis method proposed in an embodiment of the present invention. First, SEL log data is input into the fault data extraction module to obtain the status information of the faulty component (faulty component, fault content, fault status). Then, the extracted data is input into multiple fault diagnosis modules. Finally, the fault cause output by the fault diagnosis modules is input into the comprehensive analysis module to obtain the final fault cause. During operation, these three modules all call general and typical rules from the rule base to extract and analyze the fault data. In this embodiment, after performing a comprehensive analysis of the SEL logs from multiple perspectives, clear analysis results and suggestions are quickly provided, effectively improving the efficiency of system fault diagnosis and maintenance. Based on IPMI general rules, this can serve as a general diagnostic solution applicable to the extraction and diagnosis of SEL log data from various devices, servers, and switches. It analyzes the relationships between multiple fault causes, locates specific components, avoids the replacement of multiple components, and reduces maintenance costs. Furthermore, it can identify hidden problems such as repeated occurrences of the same type of problem and repeated repairs of the same component, and provides defensive measures to eliminate recurring problems and avoid the hidden dangers of multiple repairs.

[0146] Based on the same inventive concept, one embodiment of the present invention provides a system fault diagnosis device. (See reference...) Figure 6 , Figure 6 This is a schematic diagram of a system fault diagnosis device 600 according to an embodiment of the present invention. Figure 6 As shown, the device includes:

[0147] The fault data extraction module 601 is used to classify and extract fault data from system log data to obtain various types of fault data.

[0148] The component status information determination module 602 is used to obtain the component status information of all faulty components based on the various types of fault data.

[0149] The fault diagnosis module 603 is used to perform fault correlation analysis on different types of faulty components based on the component status information of the faulty component, and obtain multiple fault causes through the corresponding fault diagnosis module.

[0150] The comprehensive analysis module 604 is used to perform a comprehensive analysis of the multiple fault causes based on the two-line solidification rule to obtain the final fault cause.

[0151] Optionally, the fault data extraction module includes:

[0152] The fault data lookup submodule is used to look up the fault data corresponding to the pre-defined fault keywords in the system log data.

[0153] The fault data classification submodule is used to classify the fault data according to the category of the fault keywords to obtain the various types of fault data.

[0154] Optionally, the component status information determination module includes:

[0155] The component-related data determination submodule is used to determine the relevant fault data of the faulty component included in the fault data for each type of fault data.

[0156] The component status information determination submodule is used to determine the component status information of each faulty component based on the relevant fault data of the faulty component. The component status information includes: component name, component fault content, and component fault status.

[0157] Optionally, before performing fault correlation analysis on different types of faulty components based on the component status information of the faulty component and obtaining multiple fault causes through the corresponding fault diagnosis module, the method further includes:

[0158] When the component status information contains an unresolved component name code, the component name code is extracted to obtain the target component name code;

[0159] In a pre-set component name encoding library, the component name corresponding to the target component name code is searched to obtain the target component name;

[0160] The component status information is obtained by replacing the component name code in the fault data with the target component name.

[0161] Optionally, the fault diagnosis module includes:

[0162] The component type determination submodule is used to determine the component type of the faulty component based on the component status information of the faulty component;

[0163] The related type component determination submodule is used to determine the related type components of the faulty component based on the component type of the faulty component;

[0164] The fault cause determination submodule is used to perform fault correlation analysis on the faulty component and related type components through the fault diagnosis module corresponding to the faulty component and related type components to obtain the fault cause.

[0165] Optionally, the fault cause determination submodule includes:

[0166] The fault diagnosis module determines the sub-module, which is used to determine the faulty component and the fault diagnosis module corresponding to the related type of component of the faulty component according to the preset correspondence between the component and the fault diagnosis module;

[0167] The fault correlation analysis submodule is used to perform fault correlation analysis on the faulty component and related components of the faulty component according to preset fault diagnosis rules through the fault diagnosis module, so as to obtain the cause of the fault.

[0168] Optionally, the comprehensive analysis module includes:

[0169] The first cause determination submodule is used to output the fault cause when the relationship between the fault diagnosis modules corresponding to the multiple fault causes is parallel, so as to obtain the final fault cause.

[0170] The second cause determination submodule is used to perform correlation analysis on the multiple fault causes according to preset fault diagnosis rules when the relationship between the fault diagnosis modules corresponding to the multiple fault causes is an association relationship, so as to obtain the final fault cause.

[0171] Based on the same inventive concept, another embodiment of the present invention provides a readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps in the system fault diagnosis method as described in any of the above embodiments of the present invention.

[0172] Based on the same inventive concept, another embodiment of the present invention provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the steps of the system fault diagnosis method described in any of the above embodiments of the present invention.

[0173] As the device embodiment is basically similar to the method embodiment, the description is relatively simple, and relevant parts can be found in the description of the method embodiment.

[0174] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0175] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, apparatus, or computer program products. Therefore, embodiments of the present invention can take the form of entirely hardware embodiments, entirely software embodiments, or embodiments combining software and hardware aspects. Furthermore, embodiments of the present invention can take the form of computer program products implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0176] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing terminal device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal device, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0177] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing terminal device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0178] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal equipment, causing a series of operational steps to be performed on the computer or other programmable terminal equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable terminal equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0179] Although preferred embodiments of the present invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the embodiments of the present invention.

[0180] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal device that includes said element.

[0181] The above provides a detailed description of the system fault diagnosis method, apparatus, equipment, and storage medium provided by the present invention. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A system fault diagnosis method, characterized in that, The method includes: Fault data is categorized and extracted from system log data to obtain various types of fault data; Based on the various types of fault data, the component status information of all faulty components is obtained; Based on the component status information of the faulty component, the corresponding fault diagnosis module performs fault correlation analysis on different types of faulty components to obtain multiple fault causes, including: determining the component type of the faulty component based on its component status information; determining related component types based on the component type of the faulty component; determining the fault diagnosis module corresponding to the faulty component and its related component types based on a preset correspondence between components and fault diagnosis modules; and using the fault diagnosis module, performing fault correlation analysis on the faulty component and its related component types according to preset fault diagnosis rules to obtain the fault causes. The fault diagnosis module includes a calculation diagnosis submodule, a storage diagnosis submodule, a startup diagnosis submodule, and a heat dissipation diagnosis submodule or a power supply diagnosis submodule. Based on the two-line solidification rule, a comprehensive analysis is performed on the multiple fault causes to obtain the final fault cause, including: when the relationship between the fault diagnosis modules corresponding to the multiple fault causes is parallel, the fault causes are output to obtain the final fault cause; when the relationship between the fault diagnosis modules corresponding to the multiple fault causes is correlated, the multiple fault causes are correlated according to the preset fault diagnosis rule to obtain the final fault cause.

2. The method according to claim 1, characterized in that, The process of classifying and extracting fault data from system logs yields various types of fault data, including: Based on the pre-defined fault keywords, search for the fault data corresponding to the fault keywords in the system log data; The fault data is classified according to the category of the fault keywords to obtain the various types of fault data.

3. The method according to claim 1, characterized in that, The step of obtaining component status information for all faulty components based on the various types of fault data includes: For each type of fault data, determine the relevant fault data of the faulty component included in the fault data; Based on the relevant fault data of the faulty component, the component status information of each faulty component is determined. The component status information includes: component name, component fault content, and component fault status.

4. The method according to claim 1, characterized in that, Before performing fault correlation analysis on different types of faulty components based on the component status information of the faulty components and obtaining multiple fault causes through the corresponding fault diagnosis module, the method further includes: When the component status information contains an unresolved component name code, the component name code is extracted to obtain the target component name code; In a pre-set component name encoding library, the component name corresponding to the target component name code is searched to obtain the target component name; The component status information is obtained by replacing the component name code in the fault data with the target component name.

5. A system fault diagnosis device, characterized in that, The device includes: The fault data extraction module is used to classify and extract fault data from system log data, resulting in various types of fault data. The component status information determination module is used to obtain the component status information of all faulty components based on the various types of fault data. The fault diagnosis module is used to perform fault correlation analysis on different types of faulty components based on the component status information of the faulty component, and to obtain multiple fault causes. This includes: determining the component type of the faulty component based on its component status information; determining related component types based on the component type of the faulty component; determining the corresponding fault diagnosis module for the faulty component and its related component types based on a preset correspondence between components and fault diagnosis modules; and performing fault correlation analysis on the faulty component and its related component types according to preset fault diagnosis rules using the fault diagnosis module to obtain the fault causes. The fault diagnosis module includes a calculation diagnosis submodule, a storage diagnosis submodule, a startup diagnosis submodule, and a heat dissipation diagnosis submodule or a power supply diagnosis submodule. The comprehensive analysis module is used to perform comprehensive analysis on the multiple fault causes based on the two-line solidification rules to obtain the final fault cause. This includes: when the relationship between the fault diagnosis modules corresponding to the multiple fault causes is parallel, outputting the fault causes to obtain the final fault cause; when the relationship between the fault diagnosis modules corresponding to the multiple fault causes is correlated, performing correlation analysis on the multiple fault causes according to the preset fault diagnosis rules to obtain the final fault cause.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1 to 4.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method as described in any one of claims 1 to 4.

Images