Multidimensional index-based it system troubleshooting method, system, and media

Abstract

The application provides a multi-dimensional index-based IT system troubleshooting method, system and medium, belonging to the field of information technology. The method comprises: constructing a multi-dimensional emergency troubleshooting index library; listening to fault alarm information of an information technology (IT) system, identifying the fault type, influence range and fault occurrence device corresponding to the fault alarm information, matching professional operation and maintenance dimensions and troubleshooting scripts according to the fault type, influence range and fault occurrence device; executing troubleshooting instructions of the troubleshooting scripts in an automatic triggering mode and / or a manual triggering mode according to the matched troubleshooting scripts, and collecting troubleshooting index data corresponding to each professional operation and maintenance dimension; integrating and analyzing the troubleshooting index data according to abnormal judgment rules associated with the fault type to obtain a fault root cause analysis result; and generating and displaying corresponding fault troubleshooting information according to the fault root cause analysis result. The application can solve the problems of low automation, insufficient troubleshooting flexibility and difficulty in locating fault root causes in the prior art.

Description

Technical Field

[0001] This application belongs to the field of information technology, specifically relating to a method, system, and medium for troubleshooting IT systems based on multidimensional indicators. Background Technology

[0002] As enterprises deepen their digital operations, IT system architecture continues to upgrade towards distributed, microservice, and cloud-native architectures. IT systems encompass various hardware and software components, including servers, storage devices, databases, switches, and middleware, making their operation and maintenance scenarios increasingly complex. System failures can directly lead to business interruptions, service delays, and data anomalies, causing direct losses to enterprise production and operations. Therefore, troubleshooting and root cause analysis have become core requirements for IT system operation and maintenance.

[0003] Troubleshooting of existing IT systems mainly relies on conventional monitoring platforms to monitor basic operational status. The mainstream operation and maintenance model in the industry mainly revolves around single monitoring alarms, and fault handling depends on manual intervention by operation and maintenance personnel. The overall technical system is built around the "passive alarm + manual verification" approach, and a standardized and automated multi-dimensional emergency troubleshooting mechanism has not been formed.

[0004] The closest existing technology is the traditional IT centralized monitoring system, which is implemented as follows: A basic monitoring module is built into the IT system to collect common operating metrics such as CPU, memory, disk, and network bandwidth. Simple alarm thresholds are preset, and alarms are triggered via sound, light, SMS, or email when the metrics exceed these thresholds. This basic monitoring module only collects data and pushes alarms; it lacks professional metric categorization, and all monitoring data is displayed in a mixed manner. After receiving alarms, maintenance personnel must manually log in to the server, network device, and storage device backends. Furthermore, they need to manually enter troubleshooting commands one by one, check each operating parameter sequentially, and combine their own maintenance experience to determine the cause of the fault. Some advanced monitoring systems support simple scripts executed on a scheduled basis, but this is only used for basic data collection and cannot trigger targeted troubleshooting based on alarm events. It also lacks multi-dimensional metric linkage analysis capabilities, and the troubleshooting process lacks standardized specifications.

[0005] Therefore, existing IT centralized monitoring systems have low automation levels, only support passive alarms, cannot achieve automatic troubleshooting, and have no manual triggering for supplementary troubleshooting. This results in insufficient flexibility in troubleshooting in fault scenarios, difficulty in accurately locating the root cause of the fault, low troubleshooting efficiency, and a high risk of human error. Summary of the Invention

[0006] This application aims to provide a method, system, and medium for troubleshooting IT systems based on multi-dimensional indicators, which can solve the problems of low automation, insufficient troubleshooting flexibility, and difficulty in locating the root cause of failure in existing technologies.

[0007] According to a first aspect of this application, this application provides a method for troubleshooting IT systems based on multi-dimensional indicators, including: Construct a multi-dimensional emergency troubleshooting indicator library; the multi-dimensional emergency troubleshooting indicator library includes multiple professional operation and maintenance dimensions, fault troubleshooting indicators corresponding to each professional operation and maintenance dimension, and the correlation mapping relationship between fault troubleshooting indicators and fault types; Monitor fault alarm information of information technology IT systems, identify the fault type, scope of impact and device where the fault occurred, and match professional operation and maintenance dimensions and troubleshooting scripts according to the fault type, scope of impact and device where the fault occurred; Based on the matched troubleshooting script, execute the troubleshooting instructions of the troubleshooting script in automatic and / or manual triggering modes, and collect the troubleshooting indicator data corresponding to each professional operation and maintenance dimension; Based on the anomaly judgment rules associated with fault types, the troubleshooting indicator data are integrated and analyzed to obtain the root cause analysis results of the faults. Based on the root cause analysis results, the corresponding troubleshooting information is generated and displayed.

[0008] Preferably, in the above-mentioned IT system fault diagnosis methods, a multi-dimensional emergency diagnosis indicator library is constructed, including: Based on professional operation and maintenance dimensions, the monitoring data of IT systems are divided into application dimension, computing dimension and network dimension; Set up separate troubleshooting indicators for the application dimension, computing dimension, and network dimension; After setting the fault diagnosis indicators, configure the normal threshold and abnormal judgment rules for each fault diagnosis indicator respectively; Establish a mapping relationship between fault diagnosis indicators and fault types.

[0009] Preferably, in the above IT system fault diagnosis method, the fault alarm information of the information technology IT system is monitored, the fault type, scope of impact, and fault-occurring device corresponding to the fault alarm information are identified, and professional operation and maintenance dimensions and troubleshooting scripts are matched according to the fault type, scope of impact, and fault-occurring device, including: Connect to the IT system alarm platform to receive fault alarm information from various devices in the IT system that are experiencing faults. The fault alarm information is classified and analyzed according to the multi-dimensional emergency investigation indicator database to identify the fault type, scope of impact and equipment where the fault occurred. Based on the fault type, affected scope, and the device where the fault occurred, we match professional operation and maintenance dimensions and troubleshooting scripts.

[0010] Preferably, in the above IT system fault diagnosis method, the diagnosis instructions of the diagnosis script are executed in automatic and / or manual triggering modes according to the matched diagnosis script, and the diagnosis indicator data corresponding to each professional operation and maintenance dimension are collected, including: In automatic trigger mode, the corresponding troubleshooting script is automatically invoked according to the fault type; the troubleshooting instructions of the troubleshooting script are executed on the faulty device, and troubleshooting indicator data corresponding to each professional operation and maintenance dimension are collected; In manual trigger mode, based on the manual instructions of the operation and maintenance personnel, the professional operation and maintenance dimension corresponding to the fault type is selected and the custom troubleshooting script is executed. The special verification indicators corresponding to the custom troubleshooting script are supplemented, and the troubleshooting indicator data corresponding to each professional operation and maintenance dimension are collected.

[0011] Preferably, in the above IT system fault diagnosis method, the fault indicator data is integrated and analyzed according to the anomaly judgment rules associated with the fault type to obtain the fault root cause analysis results, including: Based on the preset normal threshold of the indicators, the data of the investigated indicators are compared with the historical normal data to filter out the abnormal indicator data. Based on the anomaly judgment rules associated with fault types, root cause analysis is performed on the abnormal indicator data to obtain the root cause analysis results, which include core abnormal indicators and fault location conclusions.

[0012] Preferably, in the above IT system fault diagnosis method, based on the anomaly judgment rules associated with fault types, root cause analysis is performed on the abnormal indicator data to obtain the root cause analysis results, including: Based on the scope of impact and the type of fault, the deviation between the abnormal indicator data and the pre-stored benchmark indicator data is calculated to generate a time-series deviation signal of the abnormal indicator data over time. Based on the time sequence of occurrence of abnormal indicator data and the topological dependency of the IT system, the propagation path of the abnormal indicator data is inferred to determine the propagation path signal corresponding to the abnormal indicator data. The timing deviation signal is dynamically time-warped and matched with the standard timing deviation templates corresponding to each fault type in the anomaly determination rules to obtain the timing similarity. The spatial similarity is obtained by performing directed graph matching between the propagation path signal and the standard propagation path templates corresponding to each fault type in the anomaly determination rules. We perform weighted fusion of temporal similarity and spatial similarity to obtain the comprehensive data similarity. When the overall similarity of the data is greater than or equal to a predetermined similarity threshold, the root cause of the fault corresponding to a specific fault type in the anomaly determination rule is matched.

[0013] Among them, the root causes of specific fault types in the matching anomaly determination rules include: Perform partial correlation analysis on abnormal indicator data to identify spurious correlations between indicators; A structural causal model is used to model the causal relationships between indicators and to construct a causal dependency graph; Based on the causal dependency graph, causal verification is performed on each candidate root cause in the candidate root cause set obtained by root cause matching. Candidate root causes that cannot explain the occurrence of abnormal indicators are eliminated, and the root cause corresponding to a specific fault type is determined.

[0014] Preferably, in the above IT system fault diagnosis method, based on the root cause analysis results, corresponding fault diagnosis information is generated and displayed, including: Generate standardized fault investigation reports, which include investigation indicator data, anomalies, and root cause conclusions for each professional operation and maintenance dimension; Save troubleshooting logs generated during the troubleshooting process to optimize the troubleshooting process.

[0015] According to a second aspect of this application, this application also provides an IT system fault diagnosis system based on multi-dimensional indicators, used in any of the above-mentioned IT system fault diagnosis methods; the IT system fault diagnosis system includes: The indicator construction module is used to build a multi-dimensional emergency investigation indicator library. The multi-dimensional emergency investigation indicator library includes multiple professional operation and maintenance dimensions, fault investigation indicators corresponding to each professional operation and maintenance dimension, and the association mapping relationship between fault investigation indicators and fault types. The fault identification module is used to monitor fault alarm information of information technology (IT) systems and identify the fault type, scope of impact, and equipment where the fault occurred. The script matching module is used to match professional operation and maintenance dimensions and troubleshooting scripts based on the fault type, scope of impact, and device where the fault occurred. The troubleshooting execution module is used to execute the troubleshooting instructions of the matching troubleshooting script in automatic and / or manual triggering modes, and collect the troubleshooting indicator data corresponding to each professional operation and maintenance dimension. The root cause analysis module is used to integrate and analyze the troubleshooting indicator data according to the anomaly judgment rules associated with the fault type, and obtain the root cause analysis results of the fault. The information generation module is used to generate and display corresponding troubleshooting information based on the root cause analysis results.

[0016] According to a third aspect of this application, this application also provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed, it implements the IT system fault diagnosis method provided by any of the above technical solutions.

[0017] This application provides a multi-dimensional indicator-based IT system fault diagnosis solution. By constructing a multi-dimensional emergency troubleshooting indicator library, this library includes multiple professional operation and maintenance dimensions, corresponding fault diagnosis indicators for each professional operation and maintenance dimension, and the association mapping relationship between these indicators and fault types. This enables the standardized integration of core troubleshooting indicators for computing, storage, and network. Upon detecting an IT system fault alarm, the solution identifies the fault type, affected scope, and faulty device corresponding to the alarm. Based on this information, it matches the professional operation and maintenance dimension and the troubleshooting script. The troubleshooting script is automatically executed after an alarm, achieving a fault diagnosis mechanism with both automatic and manual triggering, and collecting troubleshooting indicator data. Complex scenarios support manual supplementary troubleshooting, reducing excessive reliance on manual operation. Furthermore, by automatically executing the troubleshooting script upon alarm, precise linkage between alarm events and the troubleshooting script is achieved. Corresponding dimension investigations are triggered for different types of alarms, and fault root cause-related data is quickly filtered based on the anomaly judgment rules associated with the fault type. The above methods can establish a standardized emergency troubleshooting process, reduce human error, and improve the accuracy and efficiency of IT system fault diagnosis, thereby minimizing fault handling time. This approach addresses the problems of insufficient flexibility in fault diagnosis under current technologies, difficulty in accurately locating the root cause of faults, low efficiency, and susceptibility to human error. Attached Figure Description

[0018] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings: Figure 1 A flowchart illustrating an IT system fault diagnosis method based on multidimensional indicators provided in this application embodiment; Figure 2 A flowchart illustrating another IT system fault diagnosis method based on multidimensional indicators provided in this application embodiment; Figure 3 A schematic diagram of the structure of an IT system fault diagnosis system based on multidimensional indicators provided in this application embodiment; Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0019] To more clearly illustrate the overall concept of this application, a detailed explanation is provided below with reference to the accompanying drawings.

[0020] Many specific details are set forth in the following description to provide a thorough understanding of this application. However, this application may also be implemented in other ways different from those described herein. Therefore, the scope of protection of this application is not limited to the specific embodiments disclosed below. It should be noted that, unless otherwise specified, the embodiments of this application and the features thereof can be combined with each other.

[0021] In this application, unless otherwise expressly specified and limited, the descriptions using terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0022] The existing technology has the following technical problems: Existing centralized IT monitoring systems have low automation levels, only support passive alarms, and cannot achieve automatic troubleshooting or manual triggering of supplementary troubleshooting. This results in insufficient flexibility in troubleshooting during fault scenarios, difficulty in accurately locating the root cause of the fault, low troubleshooting efficiency, and susceptibility to human error.

[0023] To solve the above technical problems, see [link to relevant documentation]. Figure 1 , Figure 1 This is a flowchart illustrating a method for troubleshooting IT systems based on multi-dimensional indicators, provided as an embodiment of this application. Figure 1 As shown, this IT system fault diagnosis method based on multi-dimensional indicators includes: S110: Construct a multi-dimensional emergency investigation indicator database.

[0024] The multi-dimensional emergency troubleshooting indicator library includes multiple professional operation and maintenance dimensions, fault troubleshooting indicators corresponding to each professional operation and maintenance dimension, and the correlation mapping relationship between fault troubleshooting indicators and fault types.

[0025] Specifically, as a preferred embodiment, S110: constructing a multi-dimensional emergency investigation indicator library, including: S111: Based on professional operation and maintenance dimensions, the monitoring data of IT systems are divided into application dimension, computing dimension and network dimension.

[0026] Specifically, IT systems encompass various types of hardware and software, including servers, storage devices, databases, switches, and middleware. This application's embodiment, based on a conventional monitoring platform, reorganizes and classifies monitoring data according to professional operation and maintenance dimensions, dividing it into three core dimensions: application dimension, computing dimension, and network dimension. This ensures that subsequent troubleshooting can accurately focus on the core parameters of each professional field, eliminating interference from redundant monitoring data.

[0027] S112: Set up troubleshooting indicators for the application dimension, computing dimension and network dimension respectively.

[0028] Among them, the troubleshooting indicators for the application dimension include abnormal log keywords, number of process threads, number of process connections, status of key processes, and number of database connections; the troubleshooting indicators for the computing dimension include file system read-only status and number of Linux system handles; and the troubleshooting indicators for the network dimension include network connectivity, bandwidth utilization, round-trip latency, packet loss rate, and routing status.

[0029] Specifically, for each professional operation and maintenance dimension, key indicators closely related to fault diagnosis are identified and set for that dimension.

[0030] In some embodiments of this application, the fault diagnosis indicators corresponding to the application dimension include, but are not limited to: abnormal log keywords, process thread count, process connection count, critical process status, and database connection count. These indicators can reflect the operational health status of the application layer. For example, when an application service malfunctions, abnormal fluctuations in the number of process threads, unexpected termination of critical processes, or sudden increases or decreases in the number of database connections are often precursory signals of a fault.

[0031] In some embodiments of this application, the fault diagnosis indicators corresponding to the computing dimension include, but are not limited to: file system read-only status and Linux system handle count. The computing dimension focuses on the health status at the operating system and computing resource levels. File system read-only status usually indicates storage failure or file system corruption, while exhausting the system handle count may prevent new connections from being established, thereby causing business interruption.

[0032] In some embodiments of this application, the troubleshooting indicators corresponding to the network dimension include, but are not limited to: network connectivity, bandwidth utilization, round-trip latency, packet loss rate, and routing status. The network dimension focuses on the health status of the network infrastructure. Network connectivity is the basic guarantee of service reachability, while bandwidth utilization, round-trip latency, and packet loss rate directly reflect network transmission quality. Abnormal routing status may lead to incorrect traffic paths or network unreachability.

[0033] It should be noted that the troubleshooting indicators for each of the above dimensions are only illustrative examples. In practical applications, these indicators can be added, deleted, or adjusted according to the specific architecture and operational needs of the IT system to adapt to different troubleshooting scenarios.

[0034] S113: After setting the fault diagnosis indicators, configure the normal threshold and abnormal judgment rules for each fault diagnosis indicator.

[0035] Specifically, for each fault diagnosis indicator, the system configures the normal value range (i.e., the normal threshold) of the indicator and the anomaly judgment rules when the indicator exceeds the normal range, based on historical normal operation data. For example, for the "bandwidth utilization" indicator, its normal threshold can be set to below 80%, and an anomaly judgment is triggered when the bandwidth utilization continuously exceeds 80%; for the "round-trip latency" indicator, its normal threshold can be set to less than 50ms, and an anomaly judgment is triggered when the round-trip latency exceeds 50ms.

[0036] S114: Establish the correlation mapping relationship between fault diagnosis indicators and fault types.

[0037] The specific method for establishing the correlation mapping relationship is as follows: Correlate each fault diagnosis indicator with a possible fault type. For example, abnormal "network connectivity" and "routing status" indicators can be correlated with the "network link interruption" fault type; abnormal "database connection count" and "critical process status" indicators can be correlated with the "application service freeze" or "database connection pool exhaustion" fault types. By establishing this correlation mapping relationship, when a specific indicator becomes abnormal, the system can quickly identify the possible corresponding fault type, providing a basis for subsequent troubleshooting script matching and root cause analysis.

[0038] Figure 1 The technical solution provided in the illustrated embodiment, after constructing a multi-dimensional emergency investigation indicator library, further includes: S120: Monitors fault alarm information of IT systems, identifies the fault type, scope of impact and device where the fault occurred, and matches professional operation and maintenance dimensions and troubleshooting scripts according to the fault type, scope of impact and device where the fault occurred.

[0039] Specifically, in a preferred embodiment, S120 involves: monitoring fault alarm information from the information technology (IT) system, identifying the fault type, scope of impact, and fault-causing device corresponding to the fault alarm information, and matching professional operation and maintenance dimensions and troubleshooting scripts based on the fault type, scope of impact, and fault-causing device, including: S121: Connects to the IT system alarm platform to receive fault alarm information from various faulty devices in the IT system.

[0040] Specifically, the system connects to the IT system alarm platform in real time, receiving full alarm information from business systems, servers, network devices, storage devices, middleware, etc. The connection method can adopt standard interface protocols (such as RESTful API, Syslog, or SNMP Trap) to ensure that various fault alarms can be captured in real time and completely.

[0041] S122: Classify and analyze fault alarm information according to the multi-dimensional emergency investigation indicator library to identify the fault type, scope of impact and equipment where the fault occurred.

[0042] Specifically, the received fault alarm information is parsed and classified for analysis. For example, when a "database connection timeout" alarm is received, based on the pre-stored correlation mapping relationship in the multi-dimensional emergency troubleshooting indicator library, the fault type corresponding to the alarm is identified as "database connection pool exhaustion" or "excessive network latency". The scope of impact may involve all business systems that rely on the database, and the faulty device is the specific data server.

[0043] S123: Match professional operation and maintenance dimensions and troubleshooting scripts according to the fault type, scope of impact and the equipment where the fault occurred.

[0044] Specifically, after identifying the fault type, affected scope, and faulty device, the system matches the corresponding professional operation and maintenance dimensions and pre-stored troubleshooting scripts from a multi-dimensional emergency troubleshooting indicator library. For example, for the "database connection pool exhausted" fault type, the system matches the "application dimension" as the primary troubleshooting dimension and calls a pre-written troubleshooting script for checking database connection status; for the "network link interruption" fault type, the system matches the "network dimension" as the primary troubleshooting dimension and calls a troubleshooting script for checking network connectivity and routing status.

[0045] Figure 1 The technical solution provided in the illustrated embodiment, after step S120: matching professional operation and maintenance dimensions and troubleshooting scripts, further includes: S130: Based on the matched troubleshooting script, execute the troubleshooting instructions of the troubleshooting script in automatic and / or manual triggering modes, and collect the troubleshooting indicator data corresponding to each professional operation and maintenance dimension.

[0046] Specifically, in a preferred embodiment, step S130 involves: executing the troubleshooting instructions of the matching troubleshooting script in automatic and / or manual triggering modes, and collecting troubleshooting indicator data corresponding to each professional operation and maintenance dimension, including: S131: In automatic trigger mode, the corresponding troubleshooting script is automatically invoked according to the fault type; the troubleshooting instructions of the troubleshooting script are executed on the faulty device, and the troubleshooting indicator data corresponding to each professional operation and maintenance dimension are collected.

[0047] Specifically, once the fault alarm is identified and the troubleshooting script is matched, the automatic trigger mode is activated. Based on the fault type, the corresponding preset troubleshooting script is automatically invoked. Without manual intervention, the troubleshooting instructions are executed directly on the target device (the device where the fault occurred), and troubleshooting indicator data corresponding to various professional operation and maintenance dimensions are collected in real time.

[0048] For example, when a "network link interruption" fault type is identified, the network-level troubleshooting script is automatically invoked to execute troubleshooting commands such as ping connectivity test, traceroute path tracing, bandwidth utilization query, and routing table status check on the relevant network devices in sequence, and to collect data on indicators such as network connectivity, round-trip latency, packet loss rate, and routing status.

[0049] The type of troubleshooting script can be adapted to the operating system type of the target device, supporting troubleshooting commands in shell, python, bat formats, etc. for Linux / Windows system servers and mainstream network switches.

[0050] S132: In manual trigger mode, based on the manual instructions of the operation and maintenance personnel, select the professional operation and maintenance dimension corresponding to the fault type and execute the custom troubleshooting script, supplement the special verification indicators corresponding to the custom troubleshooting script, and collect the troubleshooting indicator data corresponding to each professional operation and maintenance dimension.

[0051] Specifically, for complex fault scenarios or scenarios where automatic troubleshooting fails to pinpoint the root cause, the system provides a manual trigger mode as a supplement. Operations personnel can manually select the professional operations dimension corresponding to the fault type, customize the troubleshooting script, and supplement the verification of specific indicators according to actual needs.

[0052] Figure 1 The technical solution provided in the illustrated embodiment, after step S130: executing the investigation instructions of the investigation script in automatic trigger mode and / or manual trigger mode according to the matched investigation script, and collecting the investigation indicator data corresponding to each professional operation and maintenance dimension, further includes: S140: Integrate and analyze the troubleshooting indicator data according to the anomaly judgment rules associated with the fault type to obtain the root cause analysis results of the fault.

[0053] Specifically, in a preferred embodiment, step S140 involves: integrating and analyzing the investigation indicator data according to the anomaly determination rules associated with the fault type to obtain the root cause analysis results, including: S141: Based on the preset normal threshold of the indicator, compare the investigated indicator data with the historical normal data to filter out the abnormal indicator data.

[0054] Specifically, the troubleshooting indicator data corresponding to each professional operation and maintenance dimension collected in step S130 above are compared with the preset normal thresholds for indicators in S113 and historical normal operation data. Indicator data that exceeds the normal threshold range or deviates significantly from historical normal data are marked as abnormal indicator data for subsequent targeted root cause analysis.

[0055] S142: Based on the anomaly judgment rules associated with the fault type, perform root cause analysis on the abnormal indicator data to obtain the root cause analysis results, which include core abnormal indicators and fault location conclusions.

[0056] In order to accurately locate the root cause of the fault and obtain accurate root cause analysis results, this application embodiment performs root cause detection on abnormal indicator data from two dimensions: time series and space.

[0057] Specifically, as a preferred embodiment, in the above IT system fault diagnosis method, S142: based on the anomaly judgment rules associated with fault types, perform root cause analysis on the abnormal indicator data to obtain the root cause analysis results, including: S1421: According to the scope of influence and fault type, calculate the deviation between abnormal indicator data and pre-stored benchmark indicator data, and generate a time-series deviation signal of abnormal indicator data changing over time. Specifically, for each abnormal indicator, the deviation between its measured value before and after the fault occurs and the pre-stored benchmark indicator data (i.e., the value of the indicator under historical normal conditions) is calculated to generate a deviation sequence of the indicator over time, i.e., the time-series deviation signal.

[0058] As an example, the timing deviation signal can be calculated using the following formula: ; in, As an indicator i At any moment t The measured value, and These represent the mean and standard deviation of the indicator under historical normal conditions, respectively. The time-series deviation signal is calculated using the above formula. It can clearly reflect the trajectory of how much the indicator deviates from the normal level over time, thus providing a data foundation for subsequent time-series similarity matching.

[0059] S1422: Based on the time sequence of occurrence of abnormal indicator data and the topological dependency relationship of the IT system, perform propagation path reasoning on the abnormal indicator data to determine the propagation path signal corresponding to the abnormal indicator data.

[0060] Specifically, the system constructs a directed graph of anomaly propagation based on the topological dependencies of the IT system (including service call chains, network topology, and device connection relationships) and the chronological order of occurrence of various anomaly indicator data. G The formula is as follows: , where nodes V The device, component, or indicator that has malfunctioned is represented by an edge E, which indicates the direction of malfunction propagation. The weight of the edge reflects the time delay and intensity of the propagation.

[0061] The propagation path signal can be represented as an ordered sequence of nodes from the fault source to each downstream node: ; in, to All of them are candidate root cause nodes.

[0062] For example, suppose in a distributed system, an abnormal number of connections is first detected on the database server. Then on the application server If thread blocking is detected, and finally a response timeout is detected by the load balancer, the propagation path signal can be represented as follows: .

[0063] Path=[database server, application server, load balancer].

[0064] S1423: Perform dynamic time warping matching between the timing deviation signal and the standard timing deviation template corresponding to each fault type in the anomaly judgment rule to obtain the timing similarity; Specifically, the anomaly detection rules pre-store standard timing deviation signal templates corresponding to each fault type. The system uses the Dynamic Time Warping (DTW) algorithm to calculate the timing similarity between the measured timing deviation signals and each standard timing template. The formula for timing similarity is as follows:

[0065] in, This represents the measured timing deviation signal. This represents the standard timing template corresponding to each fault type in the anomaly determination rules. Representing temporal similarity, the DTW algorithm can effectively handle the scaling and offset of two time series on the time axis. Even if there are uncertain time delays during fault propagation, it can still accurately measure the similarity of their temporal patterns, thereby avoiding the matching inaccuracy problem caused by time axis misalignment.

[0066] S1424: Perform directed graph matching between the propagation path signal and the standard propagation path templates corresponding to each fault type in the anomaly determination rules to obtain spatial similarity.

[0067] Specifically, the anomaly detection rules also pre-store standard propagation path templates for each fault type. The system uses graph matching algorithms (such as directed graph edit distance or longest common subsequence (LCS)) to calculate the spatial similarity between the measured propagation path signal and each standard propagation path template. The specific calculation formula is as follows: ; in, This represents the measured propagation path signal. This indicates the standard propagation path template corresponding to the fault type. This indicates spatial similarity. Spatial similarity reflects the degree to which the propagation trajectory of an anomaly in the actual system matches the typical propagation trajectory of a certain type of fault. For example, the standard propagation path template for a "network link interruption" fault type might exhibit a top-down propagation pattern of "core switch → aggregation switch → access switch → server"; while the standard propagation path template for an "application service freeze" fault type might exhibit a service dependency chain propagation pattern of "application server → database server → cache server".

[0068] S1425: Weighted fusion of temporal similarity and spatial similarity to obtain the comprehensive data similarity.

[0069] Specifically, the system performs a weighted fusion of the temporal similarity obtained in S1423 and the spatial similarity obtained in S1424. The weighted fusion formula is as follows: ; in, Indicates the overall similarity of the data, with weight parameters. It can be dynamically adjusted according to the actual application scenario. For fault types sensitive to time-series changes (such as sudden traffic surges), the adjustment can be increased. α The value of can be increased for fault types sensitive to topology propagation paths (such as network link interruptions). β The value of .

[0070] S1426: When the overall similarity of the data is greater than or equal to the predetermined similarity threshold, the root cause of the fault corresponding to the specific fault type in the anomaly judgment rule is matched.

[0071] Specifically, the system compares the overall data similarity calculated in S1425 with a predetermined similarity threshold. Compare them.

[0072] When the overall similarity of data When the current fault matches the fault type, the root cause corresponding to the fault type in the anomaly determination rules is taken as the preliminary root cause analysis result.

[0073] It should be noted that the predetermined similarity threshold θ The threshold can be configured according to the actual system's fault diagnosis accuracy requirements. Setting the threshold too high may lead to missed alarms (failure to match the correct fault type), while setting it too low may lead to false alarms (matching the wrong fault type). In practical applications, the optimal threshold value can be determined through statistical analysis of historical fault data.

[0074] In some embodiments of this application, to further improve the accuracy of root cause localization, a causal verification step is introduced after S1426, specifically including: (1) Perform partial correlation analysis on abnormal indicator data to identify pseudo-correlation between indicators.

[0075] Specifically, the system performs partial correlation analysis on the abnormal indicator data. After controlling for the influence of other indicator variables, it calculates the partial correlation coefficient between any two indicators. If the partial correlation coefficient between two indicators decreases significantly or is no longer significant, it indicates that the correlation between them is a spurious correlation caused by other indicators (common causes). Through partial correlation analysis, the system can effectively eliminate correlation noise caused by common root causes, avoiding misjudging accompanying symptoms as causal relationships.

[0076] (2) The structural causal model is used to model the causal relationship between indicators and construct a causal dependency graph.

[0077] Specifically, based on domain knowledge and historical fault data, the system employs a structural causal model (SCM) to model the causal relationships between various anomaly indicators, constructing a directed acyclic graph (DAG) to clarify the causal direction between each anomaly indicator. In the causal dependency graph, each node represents an indicator or component, and directed edges represent the causal direction (from cause to effect).

[0078] (3) Based on the causal dependency graph, perform causal verification on each candidate root cause in the candidate fault root cause set obtained by fault root cause matching, eliminate candidate root causes that cannot explain the occurrence of abnormal indicators, and determine the fault root cause corresponding to a specific fault type.

[0079] Specifically, the system substitutes each candidate root cause from the set of candidate fault root causes matched in S1426 into the causal dependency graph for verification. The verification method includes checking whether the candidate root cause node is in an upstream position in the causal graph and whether it can explain the occurrence of other abnormal indicators through a causal path. Only candidate root causes that can reasonably explain the occurrence of other abnormal indicators are retained, while candidate root causes that cannot explain the occurrence of other abnormal indicators are eliminated. After causal verification, the root cause corresponding to the specific fault type is finally determined.

[0080] Figure 1 The technical solution provided in the illustrated embodiment, after step S140: integrating and analyzing the investigation indicator data according to the anomaly judgment rules associated with the fault type to obtain the root cause analysis results, further includes: S150: Generate and display corresponding troubleshooting information based on the root cause analysis results.

[0081] Specifically, as a preferred embodiment, in the above IT system fault diagnosis method, S150: Based on the root cause analysis results, generate and display corresponding fault diagnosis information, including: S151: Generate a standardized fault investigation report. The standardized fault investigation includes the investigation indicator data, anomalies and root cause conclusions corresponding to each professional operation and maintenance dimension. Specifically, the system integrates the root cause analysis results obtained in step S140, along with the troubleshooting indicator data corresponding to each professional operation and maintenance dimension collected in S130, and the anomalies filtered in S141, to generate a standardized fault troubleshooting report. The report visually displays the indicator data for each dimension, anomaly annotations, root cause location conclusions, and the complete anomaly propagation chain, facilitating a quick understanding of the overall fault by operations and maintenance personnel. This standardization includes standardization of format, units, terminology, and value ranges.

[0082] S152: Save the troubleshooting logs generated during the troubleshooting process to optimize the troubleshooting process.

[0083] Specifically, the system persistently stores all data generated during each troubleshooting process (including alarm information, executed troubleshooting scripts, collected indicator data, intermediate analysis results, and final root cause conclusions) in the form of troubleshooting logs. This log data can be used for subsequent fault review, troubleshooting process optimization, and continuous improvement of anomaly judgment rules, enabling the system to have self-learning and continuous evolution capabilities.

[0084] The IT system fault diagnosis method based on multi-dimensional indicators provided in this application constructs a multi-dimensional professional fault emergency diagnosis indicator system independent of conventional monitoring systems, achieving standardized integration of diagnosis indicators across three core dimensions: application, computing, and network. By establishing a dual diagnosis mode of automatic triggering plus manual supplementation, it achieves automatic execution of specialized diagnosis scripts after an alarm, while retaining a flexible mechanism for manual supplementation in complex scenarios. Through the establishment of a precise linkage mechanism between alarm events and diagnosis scripts, it achieves rapid response to triggering corresponding dimension diagnoses for different types of alarms. In particular, through a three-layer progressive root cause analysis architecture of timing deviation signal matching, propagation path signal matching, and causal verification, it achieves precise location of the root cause of the fault from both "time" and "space" dimensions, and effectively eliminates spurious correlation interference through causal inference. Through the above methods, this application establishes a standardized emergency diagnosis process, effectively reducing human error and reliance on experience, significantly improving the accuracy and efficiency of IT system fault diagnosis, reducing the single fault diagnosis time from over 30 minutes to less than 5 minutes, and improving fault cause location efficiency by over 80%.

[0085] Additionally, see Figure 2 , Figure 2 This is a flowchart illustrating another IT system fault diagnosis method based on multi-dimensional indicators provided in an embodiment of this application. Figure 2 As shown, the troubleshooting methods for this IT system include: 1. Construct a multi-dimensional emergency investigation indicator database.

[0086] 2. Pre-configuration of handling scripts and strategies.

[0087] 3. Construct a scenario that maps alarms to handling strategies.

[0088] 4. Monitor and analyze system alarms in real time.

[0089] 5. Match the troubleshooting scripts corresponding to the alarms.

[0090] 6. Includes 6.1 executing troubleshooting scripts in automatic mode and / or 6.1 executing troubleshooting scripts in manual mode.

[0091] 7. Collect and analyze multi-dimensional indicator data.

[0092] 8. This includes 8.1 application-specific data and indicators; 8.2 computation-specific data and indicators; and 8.3 network-specific data and indicators.

[0093] 9. Causes of positioning system malfunction.

[0094] 10. Output a troubleshooting report and retain logs.

[0095] See also: Figure 3 , Figure 3This application also provides an IT system fault diagnosis system based on multi-dimensional indicators, used for the IT system fault diagnosis method provided in any of the above embodiments; the IT system fault diagnosis system includes: The indicator construction module 110 is used to construct a multi-dimensional emergency investigation indicator library; the multi-dimensional emergency investigation indicator library includes multiple professional operation and maintenance dimensions, fault investigation indicators corresponding to each professional operation and maintenance dimension, and the association mapping relationship between fault investigation indicators and fault types. The fault identification module 120 is used to monitor fault alarm information of information technology (IT) systems and identify the fault type, scope of impact, and equipment where the fault occurred. The script matching module 130 is used to match professional operation and maintenance dimensions and troubleshooting scripts according to the fault type, scope of impact and the device where the fault occurred; The troubleshooting execution module 140 is used to execute the troubleshooting instructions of the matching troubleshooting script in automatic triggering mode and / or manual triggering mode, and collect the troubleshooting indicator data corresponding to each professional operation and maintenance dimension. The root cause analysis module 150 is used to integrate and analyze the troubleshooting indicator data according to the anomaly judgment rules associated with the fault type, and obtain the root cause analysis results of the fault. The information generation module 160 is used to generate and display corresponding fault investigation information based on the root cause analysis results.

[0096] This application provides a multi-dimensional indicator-based IT system fault diagnosis system. By constructing a multi-dimensional emergency troubleshooting indicator library, this library includes multiple professional operation and maintenance dimensions, corresponding fault diagnosis indicators for each professional operation and maintenance dimension, and the association mapping relationship between these indicators and fault types. This enables the standardized integration of core troubleshooting indicators for computing, storage, and network. Upon detecting an IT system fault alarm, the system identifies the fault type, affected scope, and faulty device corresponding to the alarm. Based on this information, it matches the professional operation and maintenance dimension and the troubleshooting script. The troubleshooting script is automatically executed after an alarm, achieving a fault diagnosis mechanism with both automatic and manual triggering modes, collecting troubleshooting indicator data. Complex scenarios support manual supplementary troubleshooting, reducing excessive reliance on manual operation. Furthermore, by automatically executing the troubleshooting script upon alarm, precise linkage between alarm events and the troubleshooting script is achieved. Corresponding dimension investigations are triggered for different types of alarms, and root cause-related data is quickly filtered based on the anomaly judgment rules associated with the fault type. The above methods can establish a standardized emergency troubleshooting process, reduce human error, and improve the accuracy and efficiency of IT system fault diagnosis, thereby minimizing fault handling time. This approach addresses the problems of insufficient flexibility in fault diagnosis under current technologies, difficulty in accurately locating the root cause of faults, low efficiency, and susceptibility to human error.

[0097] See last. Figure 4 , Figure 4 An electronic device provided in this application includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the IT system fault diagnosis method based on multi-dimensional indicators provided in any of the above embodiments.

[0098] The following is for reference. Figure 4 The diagram illustrates a structural schematic of an electronic device suitable for implementing embodiments of this application. The electronic devices in the embodiments of this application may include, but are not limited to, mobile terminals and / or fixed terminals. Figure 4 The device shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0099] like Figure 4 As shown, the electronic device can include a processing unit 1001, such as a central processing unit and / or a graphics processing unit, which can perform various appropriate actions and processes according to a program stored in ROM 1002 or a program loaded from storage device 1003 into RAM 1004. RAM 1004 also stores various programs and data required for the operation of the electronic device. The processing unit 1001, ROM 1002, and RAM 1004 are interconnected via bus 1005. Input / output interface 1006 is also connected to bus 1005. Typically, the following systems can be connected to input / output interface 1006: input devices 1007, such as touchscreens, touchpads, keyboards, mice, image sensors, microphones, accelerometers, and / or gyroscopes; output devices 1008, such as liquid crystal displays (LCDs), speakers, and / or vibrators; storage devices 1003, such as magnetic tape and / or hard disks; and communication devices 1009. Communication device 1009 is capable of enabling the electronic device to exchange data with other devices wirelessly or via wired communication. Although the diagram shows a model building device with various systems, it should be understood that it is not required to implement or have all of the systems shown. It is possible to implement or have more or fewer systems alternatively.

[0100] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from ROM 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0101] This application provides a computer-readable storage medium having computer-readable program instructions stored thereon, namely the computer program described above, which is used to perform the methods in the above embodiments. The computer-readable storage medium carries one or more programs that, when executed by a model-building device, can be used to write computer program code for performing the operations of this application in one or more programming languages ​​or combinations thereof. The programming languages ​​include object-oriented programming languages—such as Java, Smalltalk, or C++—and conventional procedural programming languages—such as the "C" language or similar programming languages.

[0102] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram can represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks can actually be executed substantially in parallel, and they can sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or can be implemented using a combination of dedicated hardware and computer instructions.

[0103] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0104] The various embodiments in this specification are described in a progressive manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.

[0105] The above description is merely an embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A method for troubleshooting IT systems based on multidimensional indicators, characterized in that, include: Construct a multi-dimensional emergency troubleshooting indicator library; wherein, the multi-dimensional emergency troubleshooting indicator library includes multiple professional operation and maintenance dimensions, fault troubleshooting indicators corresponding to each professional operation and maintenance dimension, and the association mapping relationship between the fault troubleshooting indicators and fault types; Monitor fault alarm information of information technology (IT) systems, identify the fault type, scope of impact and fault occurrence device corresponding to the fault alarm information, and match the professional operation and maintenance dimension and troubleshooting script according to the fault type, scope of impact and fault occurrence device; According to the matched troubleshooting script, execute the troubleshooting instructions of the troubleshooting script in automatic triggering mode and / or manual triggering mode, and collect the troubleshooting indicator data corresponding to each professional operation and maintenance dimension; The investigation indicator data are integrated and analyzed according to the anomaly determination rules associated with the fault type to obtain the root cause analysis results, including: Based on the preset normal threshold values, the investigated indicator data is compared with historical normal data to filter out abnormal indicator data. Based on the anomaly determination rules associated with the fault type, the abnormal indicator data is subjected to root cause analysis to obtain the root cause analysis results, wherein the root cause analysis results include core abnormal indicators and fault location conclusions. According to the scope of influence and fault type, the deviation between the abnormal indicator data and the pre-stored benchmark indicator data is calculated to generate a time-series deviation signal of the abnormal indicator data over time. Based on the time sequence of occurrence of the abnormal indicator data and the topological dependency of the IT system, the propagation path of the abnormal indicator data is inferred to determine the propagation path signal corresponding to the abnormal indicator data. The timing deviation signal is dynamically time-warped and matched with the standard timing deviation templates corresponding to each fault type in the anomaly determination rule to obtain the timing similarity. The propagation path signal is matched with the standard propagation path templates corresponding to each fault type in the anomaly determination rule to obtain spatial similarity. The temporal similarity and spatial similarity are weighted and fused to obtain the comprehensive data similarity. When the overall similarity of the data is greater than or equal to a predetermined similarity threshold, partial correlation analysis is performed on the abnormal indicator data to identify pseudo-correlation between indicators; A structural causal model is used to model the causal relationships between indicators and to construct a causal dependency graph; Based on the causal dependency graph, causal verification is performed on each candidate root cause in the candidate fault root cause set obtained by fault root cause matching, candidate root causes that cannot explain the occurrence of abnormal indicators are eliminated, and the fault root cause corresponding to a specific fault type is determined. Based on the root cause analysis results, corresponding troubleshooting information is generated and displayed.

2. The method according to claim 1, characterized in that, The construction of a multi-dimensional emergency investigation indicator database includes: Based on professional operation and maintenance dimensions, the monitoring data of the IT system is divided into application dimension, computing dimension and network dimension; Set fault diagnosis indicators for the application dimension, the computing dimension, and the network dimension respectively; After setting the fault diagnosis indicators, configure the normal threshold and abnormal judgment rules for each fault diagnosis indicator respectively; Establish a mapping relationship between the fault diagnosis indicators and the fault types.

3. The method according to claim 1, characterized in that, The system monitors fault alarm information from IT systems, identifies the fault type, scope of impact, and device where the fault occurred, and matches the professional maintenance dimensions and troubleshooting scripts based on the fault type, scope of impact, and device where the fault occurred, including: Connect to the IT system alarm platform to receive fault alarm information from each faulty device in the IT system through the IT system alarm platform; The fault alarm information is classified and analyzed according to the multi-dimensional emergency investigation index library to identify the fault type, scope of impact and equipment where the fault occurred. Based on the fault type, affected scope, and faulty device, match the professional operation and maintenance dimensions and troubleshooting scripts.

4. The method according to claim 1, characterized in that, The process involves executing the troubleshooting instructions of the matched troubleshooting script in automatic and / or manual triggering modes, and collecting troubleshooting indicator data corresponding to various professional operation and maintenance dimensions, including: In the automatic triggering mode, the corresponding troubleshooting script is automatically invoked according to the fault type; the troubleshooting instructions of the troubleshooting script are executed on the faulty device, and troubleshooting indicator data corresponding to each professional operation and maintenance dimension are collected; In the manual trigger mode, according to the manual instructions of the operation and maintenance personnel, the professional operation and maintenance dimension corresponding to the fault type is selected and the custom troubleshooting script is executed. The special verification indicators corresponding to the custom troubleshooting script are supplemented, and the troubleshooting indicator data corresponding to each professional operation and maintenance dimension are collected.

5. The method according to claim 1, characterized in that, The step of generating and displaying corresponding troubleshooting information based on the root cause analysis results includes: Generate a standardized fault investigation report, which includes investigation indicator data, anomalies, and root cause conclusions corresponding to each professional operation and maintenance dimension; Save the troubleshooting logs generated during the troubleshooting process to optimize the troubleshooting process.

6. An IT system fault diagnosis system based on multi-dimensional indicators, characterized in that, The IT system fault diagnosis system is used in the IT system fault diagnosis method according to any one of claims 1 to 5; the IT system fault diagnosis system includes: The indicator construction module is used to build a multi-dimensional emergency investigation indicator library; wherein, the multi-dimensional emergency investigation indicator library includes multiple professional operation and maintenance dimensions, fault investigation indicators corresponding to each professional operation and maintenance dimension, and the association mapping relationship between the fault investigation indicators and fault types; The fault identification module is used to monitor fault alarm information of information technology (IT) systems and identify the fault type, scope of impact, and equipment where the fault occurred corresponding to the fault alarm information. The script matching module is used to match the professional operation and maintenance dimensions and troubleshooting scripts according to the fault type, scope of impact and fault-occurring device; The troubleshooting execution module is used to execute the troubleshooting instructions of the matching troubleshooting script in automatic and / or manual triggering modes, and collect the troubleshooting indicator data corresponding to each professional operation and maintenance dimension. The root cause analysis module is used to integrate and analyze the investigation indicator data according to the anomaly judgment rules associated with the fault type to obtain the root cause analysis results. The information generation module is used to generate and display corresponding fault investigation information based on the root cause analysis results.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed, it implements the IT system fault diagnosis method as described in any one of claims 1 to 5.

Images