An AI large model-based full-automatic log analysis and fault processing system

Abstract

This invention discloses a fully automated log analysis and fault handling system based on an AI large-scale model, belonging to the field of industrial data analysis technology. The system includes a data acquisition module that collects multi-source data from a preset target system and preprocesses the multi-source data to generate a standardized multi-source data stream; a data fusion module that uses an AI large-scale model to construct an initial relationship graph and adjusts the attributes of relationship nodes and the connection strength weights of relationship edges in the initial relationship graph; a path construction module that generates fault propagation paths based on the iteratively updated initial relationship graph; and a root cause localization module that analyzes the distribution pattern of the connection strength weights of all relationship edges in the fault propagation path and ranks each relationship node by its centrality in the topology to locate the root cause of the fault. This invention, through its path construction module, uses dynamic weighting to filter key edges and traces back to the source, automatically constructing logically complete fault propagation paths, thus improving the depth and reliability of root cause analysis.

Description

Technical Field

[0001] This invention relates to the field of industrial data analysis technology, and in particular to a fully automated log analysis and fault handling system based on an AI large model. Background Technology

[0002] Log analysis refers to the process of collecting, parsing, summarizing, and mining the massive amounts of log data automatically generated during the operation of a computer system, aiming to extract key information reflecting the system's status and behavior; fault handling, on the other hand, is the activity of quickly locating the root cause of a problem and implementing repair measures based on the analysis results when a system malfunctions or its performance degrades, in order to restore the continuous operation of the system.

[0003] Existing technologies typically employ rule engines or shallow machine learning models to process log events, performance metrics, and topology data independently or in simple parallel. Their working principle is to first match log keywords using predefined rules, then combine thresholds to determine whether performance metrics are abnormal, and finally perform limited association retrieval within static topology relationships.

[0004] However, existing technologies struggle to deeply integrate and perform multi-dimensional correlation analysis of discrete log events, real-time performance metrics, and dynamic system topology during data processing. This makes it difficult to accurately construct a complete propagation path depicting the fault spread process, leading to difficulties in root cause localization and superficial analytical conclusions. Ultimately, this limits the intelligence level of system fault handling and overall operational efficiency. Therefore, there is an urgent need to provide a fully automated log analysis and fault handling system based on AI large-scale models to solve these problems. Summary of the Invention

[0005] The technical problem to be solved by this invention is to overcome the shortcomings of the prior art in the process of analyzing and processing data. It is difficult to deeply integrate and multi-dimensionally correlate discrete log events, real-time changing performance indicators and dynamic system topology. This makes it impossible to accurately construct a complete propagation path that describes the fault propagation process, which in turn makes it difficult to locate the root cause and the analysis conclusions remain superficial. Ultimately, this limits the intelligent level of system fault handling and the overall operation and maintenance efficiency. The invention provides a fully automatic log analysis and fault handling system based on an AI large model.

[0006] To solve the above-mentioned technical problems, one technical solution adopted by the present invention is to provide a fully automatic log analysis and fault handling system based on an AI large model, including a data acquisition module, a data fusion module, a path construction module and a root cause localization module; The data acquisition module is used to acquire multi-source data from a preset target system in real time, and to preprocess the multi-source data to generate a standardized multi-source data stream. The data fusion module uses an AI big data model to perform real-time relationship reasoning on log events, performance indicators and topology in the standardized multi-source data stream, constructs an initial relationship graph, and iteratively updates the initial relationship graph according to the new standardized multi-source data stream, adjusting the attributes of relationship nodes and the connection strength weights of relationship edges in the initial relationship graph. The path construction module, based on the iteratively updated initial relationship graph, filters out the relationship edges whose connection strength weight values ​​exceed the preset weight threshold of the target system, marks the relationship edges as high-weight key edges, and takes the relationship node corresponding to the detected abnormal indicators or error logs as the starting point, performs reverse tracing and path splicing along the high-weight key edges until it backtracks to the point where no new starting point is generated, thus generating a fault propagation path. The root cause localization module analyzes the distribution pattern of the connection strength weights of all the relation edges in the fault propagation path, sorts the relation nodes by centrality in the topology, and compares them with a preset fault mode library to locate the root cause of the fault.

[0007] The present invention is further configured such that: the preset target system in the data acquisition module is specifically a software system containing one or more interconnected service components; The data acquisition module includes preprocessing the multi-source data by performing timestamp alignment, outlier removal, and format standardization on log events, performance metrics, and topology data from different data sources to generate a standardized multi-source data stream.

[0008] The present invention is further configured such that the method for constructing the initial relationship graph in the data fusion module is as follows: defining the record entries, performance indicators, and specific components or connections in the topology of each independent log event in the standardized multi-source data stream as independent data entities, and mining the temporal, causal, and spatial relationships between the data entities, defining the temporal, causal, and spatial relationships as relationships between entities, and constructing the initial relationship graph with the data entities as relationship nodes and the relationships between entities as relationship edges.

[0009] The present invention is further configured such that the adjustment steps for the connection strength weights in the data fusion module are as follows: S1. Based on the new input standardized multi-source data stream, identify each data entity contained therein and the relationship between the entities, and calculate the incremental strength of the newly identified relationship between the entities relative to the existing relationship between the entities in the current initial relationship graph; S2. A preset time decay factor is introduced to correct the incremental intensity. The time decay factor is determined based on the interval length between the timestamp of the newly input standardized multi-source data stream and the latest timestamp of the current record in the initial relationship graph. A first time threshold and a second time threshold are preset. When the interval length is less than or equal to the first time threshold, the time decay factor is 1; when the interval length is greater than the first time threshold and less than or equal to the second time threshold, the time decay factor is 0.5; when the interval length is greater than the second time threshold, the time decay factor is 0. The incremental intensity is multiplied by the time decay factor determined accordingly to obtain the time-corrected instantaneous relationship intensity. S3. Weighted fusion of the instantaneous relationship strength with the current connection strength weight of the corresponding relationship edge in the initial relationship graph; assign a preset dynamic influence coefficient to the instantaneous relationship strength, and assign a static compensation coefficient equal to one minus the dynamic influence coefficient to the current connection strength weight; multiply the instantaneous relationship strength by the dynamic influence coefficient, and multiply the current connection strength weight by the static compensation coefficient, add the two product results, and calculate the new candidate value of the connection strength weight of the relationship edge; S4. Compare the new candidate value of the connection strength weight with the preset weight rationality boundary. If the new candidate value of the connection strength weight is within the weight rationality boundary, it is directly updated as the connection strength weight of the relation edge. If the new candidate value of the connection strength weight exceeds the weight rationality boundary, the boundary value of the weight rationality boundary is taken as the connection strength weight, thus completing the adjustment and update of the connection strength weight of the relation edge in the initial relation graph.

[0010] The present invention is further configured such that the steps for generating the fault propagation path in the path construction module are as follows: Q1. Based on the iteratively updated initial relation graph, the relation edges whose connection strength weight exceeds the preset weight threshold of the target system are selected and marked as high-weight key edges. At the same time, all relation nodes marked as abnormal at the current time are identified as candidate path starting points, forming a key subgraph structure connected by the high-weight key edges. Q2. Based on the key subgraph structure, starting from the starting point of each candidate path marked as an abnormal state, traverse backwards along the high-weight key edge to trace all possible causal upstream relationship nodes, and record the relationship nodes and relationship edges passed through in real time during the traversal process to form multiple initial potential fault propagation chains. Q3. Evaluate all the initial potential fault propagation chains and calculate the comprehensive propagation strength index of each initial potential fault propagation chain. The comprehensive propagation strength index is calculated by geometrically averaging the connection strength weights of all the relational edges on the links of the initial potential fault propagation chain. At the same time, it is corrected by combining the centrality index of each relational node on the links of the initial potential fault propagation chain in the target system topology. Based on the corrected comprehensive propagation strength index, all the initial potential fault propagation chains are sorted in descending order, and a number of the initial potential fault propagation chains with the highest ranking according to a preset proportion are selected as the fault propagation paths. The complete sequence of relation nodes, the sequence of relation edges, and the corresponding comprehensive propagation strength index for each fault propagation path are output in a structured manner.

[0011] The present invention is further configured such that the method for forming the initial potential fault propagation chain in step Q2 is as follows: Q201. Using the candidate path starting point marked as abnormal state in the key subgraph structure as the starting node of the current traversal, initialize an empty current propagation path record, and at the same time create a set of nodes to be explored and add all the starting nodes to the set of nodes to be explored. Q202. Take a starting node from the set of nodes to be explored as the current node. Among the relational edges between the current node and other relational nodes in the key subgraph structure, filter out the relational edges pointing to the current node. If such relational edges exist, mark the source relational node connected to the relational edge pointing to the current node as a new node to be explored and add it to the set of nodes to be explored. At the same time, add the current node, the relational edge pointing to the current node, and the corresponding source relational node to the current propagation path record in the exploration order. Q203. Repeat step Q202 until the set of nodes to be explored is empty or the preset maximum backtracking depth is reached. At this time, the complete sequence of relation nodes and relation edges stored in the current propagation path record is output as an initial potential fault propagation chain. After the above process is completed for all the starting nodes, multiple initial potential fault propagation chains are obtained.

[0012] The present invention is further configured such that: the specific steps for correcting the centrality index of each relation node in the target system topology in step Q3, based on the initial potential fault propagation chain, are as follows: Q301. For each of the initial potential fault propagation chains, extract all the relational nodes contained in the initial potential fault propagation chain, calculate the basic centrality value based on the number of direct connections between each relational node and other relational nodes in the topology of the target system, and normalize the basic centrality value to obtain the node influence factor. Q302. The node influence factor is fused and corrected with the comprehensive propagation intensity index of the initial potential fault propagation chain. A preset topological weight coefficient is assigned to the node influence factor, and its complement weight coefficient is assigned to the comprehensive propagation intensity index. The product of the node influence factor and the topological weight coefficient and the product of the comprehensive propagation intensity index and the complement weight coefficient are added together to obtain the corrected comprehensive propagation intensity index. The complement weight coefficient is a difference weight coefficient obtained by subtracting the topological weight coefficient.

[0013] The present invention is further configured such that the calculation step of the basic centrality value in step Q301 is as follows: Q3011. For each relation node in the topology of the target system, count the total number of direct connection edges between the relation node and all other relation nodes to obtain the original connection degree of the relation node, and use the original connection degree as the initial input for calculating the basic centrality value. Q3012. Divide the original connectivity degree of each relation node by the normalized baseline value preset by the target system. The normalized baseline value is the historical maximum connectivity degree of all relation nodes in the topology of the target system to obtain a normalized connectivity ratio between 0 and 1. Multiply the normalized connectivity ratio by 100 to convert it into a percentage value to obtain the basic centrality value of the relation node.

[0014] The present invention is further configured such that: the method for locating the root cause of the fault in the root cause localization module is: R1. Receive the fault propagation path and its corresponding comprehensive propagation intensity index from the path construction module, extract the basic centrality value of all relation nodes in each fault propagation path, and perform weighted fusion of the basic centrality value and the comprehensive propagation intensity index according to a preset weight to calculate the root cause suspicion index of each relation node. R2. Mark the relationship nodes whose root cause suspicion index exceeds a preset dynamic threshold as candidate root cause nodes. Combine the historical fault mode features recorded in the fault mode library to calculate the matching degree value between each candidate root cause node and the historical fault mode features. Compare the matching degree value with the preset matching degree threshold, filter out all qualified candidate root cause nodes that exceed the matching degree threshold, and select the node with the largest matching degree value from the qualified candidate root cause nodes to determine it as the root cause of the fault.

[0015] The present invention is further configured such that: after locating the root cause of the fault in the root cause localization module, it also includes generating a processing suggestion and feeding the processing suggestion back to the data acquisition module.

[0016] The beneficial effects of this invention are as follows: 1. This invention utilizes a data acquisition and fusion module and an AI large model to perform real-time relationship reasoning and dynamic graph construction on multi-source heterogeneous data, achieving deep integration of logs, performance, and topology information, and providing a unified and evolving data foundation for accurate analysis; 2. This invention, through a path construction module, uses dynamic weights to filter key edges and traces them backwards, which can automatically construct a logically complete fault propagation path, thereby efficiently and accurately locating the source of the fault and greatly improving the depth and reliability of root cause analysis. 3. This invention uses a root cause localization module to compare and verify fault modes in conjunction with a fault mode library, and generates closed-loop processing suggestions after localization and feeds them back to the front end, realizing an automated process from intelligent diagnosis to decision support, and comprehensively enhancing the system's autonomous fault handling capabilities. Attached Figure Description

[0017] Figure 1 This is a system flowchart of the present invention; Figure 2 This is a flowchart illustrating the steps for generating the fault propagation path according to the present invention. Detailed Implementation

[0018] The preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings, so that the advantages and features of the present invention can be more easily understood by those skilled in the art, thereby providing a clearer and more explicit definition of the scope of protection of the present invention.

[0019] Please see Figures 1-2 A fully automated log analysis and fault handling system based on an AI big model, including a data acquisition module, a data fusion module, a path construction module and a root cause localization module; The data acquisition module is used to collect multi-source data from a preset target system in real time, and to preprocess the multi-source data to generate a standardized multi-source data stream. The data fusion module uses an AI big model to perform real-time relationship reasoning on log events, performance indicators and topology in standardized multi-source data streams, constructs an initial relationship graph, and iteratively updates the initial relationship graph based on new standardized multi-source data streams, adjusting the attributes of relationship nodes and the connection strength weights of relationship edges in the initial relationship graph. The path construction module, based on the iteratively updated initial relationship graph, filters out relationship edges whose connection strength weight values ​​exceed the preset weight threshold of the target system, marks the relationship edges as high-weight key edges, takes the relationship node corresponding to the detected abnormal indicators or error logs as the starting point, and performs reverse tracing and path splicing along the high-weight key edges until it backtracks to the point where no new starting point is generated, thus generating a fault propagation path. The root cause localization module analyzes the distribution pattern of connection strength weights of all relation edges in the fault propagation path, ranks each relation node by its centrality in the topology, and compares it with a preset fault mode library to locate the root cause of the fault.

[0020] The AI ​​large model is specifically a multimodal temporal Transformer model that has been jointly pre-trained with large-scale log sequences, system monitoring indicators, and network topology data. Its core architecture includes an encoder for parsing log text, a temporal attention module for learning performance indicator time series, and a graph neural network component for modeling topological node relationships. It achieves deep fusion and relationship reasoning of three types of heterogeneous data: log events, performance indicators, and topology structures through a cross-modal attention mechanism.

[0021] This system collects and standardizes multi-source data in real time through the data acquisition module, providing a high-quality data foundation for analysis; it uses the data fusion module to build and dynamically update the relationship graph with the help of AI large models, accurately depicting the internal relationships of the system; it uses the path construction module to filter high-weight key edges and trace them back to quickly build a complete fault propagation path; and it uses the root cause localization module to compare multi-dimensional information to efficiently locate the root cause of the fault, forming an intelligent diagnostic closed loop.

[0022] The target system in the data acquisition module is specifically a software system containing one or more interconnected service components; The data acquisition module's preprocessing steps for multi-source data include timestamp alignment, outlier removal, and format standardization of log events, performance metrics, and topology data from different data sources, generating a standardized multi-source data stream with unified time-series identification and a standardized data structure. The specific steps for multi-source data preprocessing are existing technologies and will not be elaborated upon here.

[0023] The initial relationship graph in the data fusion module is constructed as follows: each independent log event in the standardized multi-source data stream is defined as an independent data entity, including its record entries, performance indicators, and specific components or connections in the topology. The temporal, causal, and spatial relationships between these data entities are then mined. These relationships are defined as the relationships between entities. The initial relationship graph is constructed using data entities as relationship nodes and the relationships between entities as relationship edges.

[0024] The mining methods for temporal, causal, and spatial correlations are as follows: Based on standardized multi-source data streams, firstly, sliding window analysis is applied to the time series of each data entity to extract its trend, periodicity, and fluctuation characteristics as the basis for temporal correlations; then, an improved variable-delay Granger causality test method is used to analyze whether there are leading and lagging relationships between the time series of different data entities, and causal correlations are confirmed through statistical significance tests; simultaneously, based on the physical connections and logical dependencies between entities in the system topology, combined with spatial distance and network path indicators, spatial proximity and connectivity are defined to quantify spatial correlations; finally, the strength of temporal correlations, the confidence level of causal correlations, and the tightness of spatial correlations are weighted and fused to form a comprehensive inter-entity relationship metric, which is used to construct relation edges.

[0025] The steps for adjusting the connection strength weights in the data fusion module are as follows: S1. Based on the standardized multi-source data stream of the new input, identify the data entities contained therein and the relationships between entities, and calculate the incremental strength of the newly identified relationships between entities relative to the existing relationships between entities in the current initial relationship graph. The calculation steps for incremental intensity are as follows: S101. Based on the standardized multi-source data stream of the new input, identify the newly emerging inter-entity relationship and perform feature matching with all existing inter-entity relationships in the current initial relationship graph. Calculate the multi-dimensional feature similarity between the new inter-entity relationship and each existing inter-entity relationship in terms of temporal pattern, causal direction and spatial attributes. S102. Compare the calculated multidimensional feature similarity with a preset dynamic similarity threshold, filter out existing relationships whose feature similarity exceeds the dynamic similarity threshold as candidate associations, and calculate the absolute difference between the new relationship and each candidate association in terms of relationship strength index. The optimal value for the preset dynamic similarity threshold needs to be dynamically determined by combining the historical fault data characteristics and real-time operating status of the target system. Specifically, firstly, the inter-entity relationship feature data corresponding to typical fault cases that occurred in the target system in the past three months are collected. The K-means clustering algorithm is used to divide these historical relationship features into normal association pattern clusters and abnormal association pattern clusters, and the Euclidean distance between the centers of the two clusters is calculated as the basic threshold reference value. Then, the current resource utilization (CPU, memory, network bandwidth) and transaction throughput indicators of the system are monitored in real time. When the system is under high load (resource utilization exceeds 80% or throughput fluctuation is greater than 30%), the basic threshold reference value is lowered by 15% to improve the sensitivity of association identification and avoid missing potential fault associations. When the system is under low load and operating stably, the basic threshold reference value is increased by 10% to reduce the false association rate. Simultaneously, the minimum step size for threshold adjustment is set to 0.05, and the range of the dynamic similarity threshold is strictly limited to between 0.3 and 0.8 to ensure the stability and effectiveness of the dynamic similarity threshold adjustment. The dynamic similarity threshold determined by the above method can adapt to changes in the system's operating state while ensuring the accuracy of relationship matching, providing a reliable screening basis for subsequent incremental strength calculation.

[0026] S103. Calculate a weighted average of the absolute differences corresponding to all candidate relationships, with the weights determined by their respective feature similarities. Then, normalize this weighted average and finally output the incremental strength of the newly identified inter-entity relationship relative to the existing inter-entity relationships in the current initial relationship graph.

[0027] S2. A preset time decay factor is introduced to correct the incremental intensity. The time decay factor is determined based on the interval length between the timestamp of the newly input standardized multi-source data stream and the latest timestamp of the current record in the initial relationship graph. A first time threshold and a second time threshold are preset. When the interval length is less than or equal to the first time threshold, the time decay factor is 1; when the interval length is greater than the first time threshold and less than or equal to the second time threshold, the time decay factor is 0.5; when the interval length is greater than the second time threshold, the time decay factor is 0. The incremental intensity is multiplied by the time decay factor determined accordingly to obtain the time-corrected instantaneous relationship intensity. First time threshold: When determining the time decay factor, the optimal value of the first time threshold is recommended to be 30 seconds. This threshold is intended to define the time window in which the data stream can be regarded as real-time continuous, and is suitable for scenarios with high requirements for system state continuity and low noise interference. Second time threshold: The optimal value of the second time threshold is recommended to be 10 minutes. This threshold is used to define the critical point where the data flow is significantly delayed. It is suitable for scenarios where a certain delay is allowed but outdated information must be avoided from dominating the update. The setting of the first time threshold and the second time threshold together ensures the sensitivity of the relationship graph update to timeliness.

[0028] S3. Weighted fusion of the instantaneous relation strength and the current connection strength weight of the corresponding relation edge in the initial relation graph; assign a dynamic influence coefficient based on the importance of relation type to the instantaneous relation strength, and assign a static compensation coefficient equal to one minus the dynamic influence coefficient to the current connection strength weight; multiply the instantaneous relation strength by the dynamic influence coefficient, and multiply the current connection strength weight by the static compensation coefficient, add the two product results, and calculate the new candidate value of the connection strength weight of the relation edge; Dynamic Influence Coefficient: Dynamically generated through a lightweight neural network model. The input of this model includes the historical stability assessment value of the current relation edge, the importance preset weight of the association type to which the relation edge belongs (such as causal association, time-series association, etc.), and the current overall load status index of the system. The dynamic influence coefficient corresponding to each relation edge is output in real time through the calculation of this neural network model, thereby adaptively adjusting the influence of new information in weight updates.

[0029] S4. Compare the new candidate values ​​of connection strength weights with the preset weight rationality boundary. If the new candidate values ​​of connection strength weights are within the weight rationality boundary range, they are directly updated as the connection strength weights of the relation edges. If the new candidate values ​​of connection strength weights exceed the weight rationality boundary, the boundary value of the weight rationality boundary is taken as the connection strength weight, thus completing the adjustment and update of the connection strength weights of the relation edges in the initial relation graph.

[0030] The preset weight rationality boundary is a closed interval. The optimal value for its lower bound is suggested to be 0.1, which is used to prevent the weight of the relation edge from losing its meaning due to excessive decay. The optimal value for its upper bound is suggested to be 5.0, which is used to prevent the weight of a single relation edge from growing infinitely and dominating the reasoning process of the entire graph. This boundary is determined through training and verification with a large amount of historical data to ensure that the weight adjustment is carried out within an effective and controllable range.

[0031] The boundary values ​​for the weight rationality boundary are as follows: when the new candidate value of the connection strength weight is lower than the lower bound (0.1) of the preset weight rationality interval, the lower bound value of 0.1 is taken as the final connection strength weight; when the new candidate value of the connection strength weight is higher than the upper bound (5.0) of the preset weight rationality interval, the upper bound value of 5.0 is taken as the final connection strength weight. This boundary value application ensures that the weights of all relation edges in the relation graph are within a stable and interpretable numerical range.

[0032] The steps for generating fault propagation paths in the path construction module are as follows: Q1. Based on the initial relation graph after iterative updates, relation edges whose connection strength weight exceeds the preset weight threshold of the target system are selected and marked as high-weight key edges. At the same time, all relation nodes marked as abnormal states at the current time are identified as candidate path starting points, forming a key subgraph structure connected by high-weight key edges. The optimal value for the preset weight threshold is recommended to be 0.7. This threshold is used to filter the relation edges with significant influence in the initial relation graph. The setting is based on the statistical distribution of the connection strength weight of relation edges on the actual fault propagation path in the historical fault data of the target system. The upper quartile is taken to ensure that the selected high-weight key edges can effectively represent potential strong correlations, while avoiding the introduction of too much noise due to the threshold being too low or the breakage of the critical path due to the threshold being too high.

[0033] Methods for forming key subgraph structures: Q101. Based on the initial relation graph after iterative update, first traverse all relation edges in the graph, filter out relation edges with connection strength weight greater than or equal to the preset weight threshold (0.7), mark these edges as high-weight key edges, and at the same time identify all relation nodes marked as abnormal states at the current time as candidate path starting points. Q102. Using the selected high-weight key edges as the skeleton, extract the largest connected subgraph containing the starting points of all candidate paths and the relational nodes reachable through the high-weight key edges, forming a key subgraph structure. This structure retains the strongly connected regions in the original relational graph that are most likely related to the current anomaly, providing a focused search space for subsequent path tracing.

[0034] Q2. Based on the key subgraph structure, starting from the starting point of each candidate path marked as an abnormal state, traverse backwards along the high-weight key edge to trace all possible causal upstream relationship nodes, and record the relationship nodes and relationship edges passed through in real time during the traversal process to form multiple initial potential fault propagation chains. Q3. Evaluate all initial potential fault propagation chains and calculate the comprehensive propagation strength index of each initial potential fault propagation chain. The comprehensive propagation strength index is calculated by geometrically averaging the connection strength weights of all relational edges on the initial potential fault propagation chain. At the same time, it is corrected by combining the centrality index of each relational node on the initial potential fault propagation chain in the target system topology. Based on the corrected comprehensive propagation strength index, all initial potential fault propagation chains are sorted in descending order, and the top few initial potential fault propagation chains with a preset proportion are selected as fault propagation paths. The complete sequence of relation nodes, relation edge sequence, and corresponding comprehensive propagation strength index of each fault propagation path are output in a structured manner.

[0035] The optimal value for the preset ratio is recommended to be 0.2, which means selecting the top 20% of the initial potential fault propagation chains after sorting the comprehensive propagation intensity index in descending order as the final output fault propagation path. This ratio is determined based on the statistical regularity of the proportion of effective paths in historical fault review data, aiming to balance the comprehensiveness and refinement of the result set and ensure that the most likely fault propagation scenario is output.

[0036] The method for forming the initial potential fault propagation chain in step Q2 is as follows: Q201. Using the candidate path starting point marked as an abnormal state in the key subgraph structure as the starting node of the current traversal, initialize an empty current propagation path record, and at the same time create a set of nodes to be explored and add all starting nodes to the set of nodes to be explored. Q202. Take a starting node from the set of nodes to be explored as the current node. Among the relationship edges between the current node and other relationship nodes in the key subgraph structure, filter out the relationship edges pointing to the current node. If such relationship edges exist, mark the source relationship node connected by the relationship edge pointing to the current node as a new node to be explored and add it to the set of nodes to be explored. At the same time, add the current node, the relationship edge pointing to the current node and the corresponding source relationship node to the current propagation path record in the exploration order. Q203. Repeat step Q202 until the set of nodes to be explored is empty or the preset maximum backtracking depth is reached. At this time, the complete sequence of relation nodes and relation edges stored in the current propagation path record is output as an initial potential fault propagation chain. After the above process is completed for all starting nodes, multiple initial potential fault propagation chains are obtained.

[0037] The optimal value for the preset maximum backtracking depth is recommended to be 5. This depth limits the maximum number of steps to trace back from the starting point of the candidate path along the high-weight critical edge. Its setting takes into account the chain length range of typical distributed system service calls, which can cover most of the propagation levels of faults, and avoid the waste of computing resources and the increase of path independence caused by infinite backtracking or excessive depth.

[0038] The specific steps for correcting the centrality index of each relation node in the initial potential fault propagation chain within the target system topology in step Q3 are as follows: Q301. For each initial potential fault propagation chain, extract all relational nodes contained in the initial potential fault propagation chain. Calculate the basic centrality value based on the number of direct connections between each relational node and other relational nodes in the topology of the target system. Normalize the basic centrality value to obtain the node influence factor. Q302. The node impact factor and the comprehensive propagation intensity index of the initial potential fault propagation chain are fused and corrected. A preset topological weight coefficient is assigned to the node impact factor, and its complement weight coefficient is assigned to the comprehensive propagation intensity index. The product of the node impact factor and the topological weight coefficient and the product of the comprehensive propagation intensity index and the complement weight coefficient are added together to obtain the corrected comprehensive propagation intensity index. The optimal value of the preset topology weight coefficient is recommended to be 0.3. This coefficient is used to balance the contribution weights of node influence factors and comprehensive propagation intensity index in the correction process. This means that when calculating the corrected comprehensive propagation intensity index, the centrality of a node in the topology structure accounts for 30% of the weight, while the original path propagation intensity accounts for 70% of the weight. This allocation is based on a quantitative assessment of the comparison between the importance of topology criticality and path intensity in the root cause analysis of historical failures.

[0039] The complement weight coefficient is the difference weight coefficient obtained by subtracting the topology weight coefficient from one. The complement weight coefficient and the topology weight coefficient together constitute a complete weight allocation system, ensuring that the corrected comprehensive propagation strength index can reflect the propagation strength of the fault propagation path itself and the topological importance of the node in a balanced way.

[0040] The calculation steps for the basic centrality value in step Q301 are as follows: Q3011. For each relation node in the topology of the target system, count the total number of direct connection edges between the relation node and all other relation nodes to obtain the original connection degree of the relation node, and use the original connection degree as the initial input for calculating the basic centrality value. Q3012. Divide the original connectivity degree of each relation node by the normalized baseline value preset by the target system. The normalized baseline value is the historical maximum connectivity degree of all relation nodes in the topology of the target system. This yields a standardized connectivity ratio between 0 and 1. Multiply the standardized connectivity ratio by 100 to convert it into a percentage value to obtain the basic centrality value of the relation node.

[0041] The root cause localization method in the root cause localization module is as follows: R1. Receive the fault propagation path and its corresponding comprehensive propagation strength index from the path construction module, extract the basic centrality value of all relation nodes in each fault propagation path, and perform weighted fusion of the basic centrality value and the comprehensive propagation strength index according to the preset weight to calculate the root cause suspicion index of each relation node. The steps for calculating the root cause suspicion index are as follows: R101. For each relation node on the fault propagation path, extract its basic centrality value and the comprehensive propagation strength index of the path. Multiply the basic centrality value by a preset weight (0.4) and multiply the comprehensive propagation strength index by its complement weight (0.6) to obtain two weighted scores. R102. Add the two weighted scores above and normalize them so that the final root cause suspicion index falls within the range of 0 to 1. This index comprehensively reflects the importance of the node in the network and its correlation strength in specific fault propagation scenarios.

[0042] R2. Mark relationship nodes whose root cause suspicion index exceeds a preset dynamic threshold as candidate root cause nodes. Combine the historical fault mode features recorded in the fault mode library to calculate the matching degree value between each candidate root cause node and the historical fault mode features. Compare the matching degree value with the preset matching degree threshold, filter out all qualified candidate root cause nodes that exceed the matching degree threshold, and select the node with the largest matching degree value from the qualified candidate root cause nodes to determine the fault root cause. The node with the largest matching degree value is the node with the highest degree of matching with the current fault features.

[0043] A preset dynamic threshold is used to screen candidate root cause nodes with significant root cause suspicion index. Its optimal value is not a fixed value, but is determined based on the dynamic characteristics of the root cause suspicion index sequence calculated from all current relational nodes. Usually, the average value of the sequence is taken plus one standard deviation. This method can adapt to the distribution differences of suspicion index under different fault scenarios and ensure effective differentiation in high-suspicion node groups.

[0044] The fault mode library is a structured knowledge base that stores the features of historically verified fault cases. Each record includes the type of the root cause node, typical propagation path patterns, related abnormal performance indicators, log error keyword sequences, and finally verified effective handling strategies. The system performs multi-dimensional matching between the current candidate root cause node and its contextual features and the records in the fault mode library, and calculates the matching degree value to assist in decision-making.

[0045] The optimal value for the preset matching threshold is recommended to be 0.75. This threshold is used to determine whether the match between the candidate root cause node and historical cases in the failure mode library is significant enough. The matching is considered valid only if the matching value exceeds this threshold. This setting is based on the balance between recall and precision of a large number of test cases, and aims to ensure that the root cause localization results have a high degree of confidence.

[0046] The root cause localization module, after locating the root cause of the fault, also includes generating processing suggestions, which are then fed back to the data acquisition module to guide the focus and strategy adjustment of subsequent data acquisition, forming a closed-loop processing flow.

[0047] The steps for generating handling suggestions are as follows: First, based on the identified root cause node of the fault, query the handling strategy templates recorded in the historical cases with the highest matching degree in the fault mode library. Then, combine the real-time load status, resource availability, and operation and maintenance policy rules of the current system to instantiate and adjust the parameters in the template. For example, adjust the batch size of service restarts according to the current CPU idle rate, or delay the execution of certain non-urgent operations according to business peak information. Finally, generate specific, executable handling suggestion instructions that fit the current system status, and feed them back to the data acquisition module to optimize the focus of subsequent monitoring.

[0048] Example In a specific operational event, the data acquisition module collected multi-source data from a pre-defined target system in real time, including log event performance metrics and topology data. It then performed timestamp alignment, outlier removal, and format standardization on this multi-source data to generate a standardized multi-source data stream. The data fusion module then used a large AI model to perform real-time relationship reasoning on this standardized multi-source data stream, constructing an initial relationship graph.

[0049] When a new standardized multi-source data stream is input, the module first identifies the various data entities contained therein and the relationships between them, and calculates the incremental strength of the newly identified relationships between entities relative to the existing relationships between entities in the current initial relationship graph: based on the new data, newly emerging relationships between entities are identified, and feature matching is performed with existing relationships to calculate multi-dimensional feature similarity. The similarity is compared with a dynamic similarity threshold to filter candidate associations, and the absolute difference is calculated and weighted average normalized to obtain the incremental strength.

[0050] Subsequently, a time decay factor is introduced to correct the incremental intensity. The time decay factor is determined based on the interval between the new data timestamp and the latest timestamp of the map. When the interval is less than or equal to the first time threshold of 30 seconds, the factor is 1; when the interval is greater than 30 seconds and less than or equal to the second time threshold of 10 minutes, the factor is 0.5; and when the interval is greater than 10 minutes, the factor is 0. The instantaneous relationship intensity is obtained by multiplying the incremental intensity by the time decay factor.

[0051] Next, the instantaneous relationship strength is weighted and fused with the current connection strength weight of the corresponding relationship edge in the initial relationship graph. A dynamic influence coefficient is assigned to the instantaneous relationship strength, and a static compensation coefficient is assigned to the current connection strength weight. The product sum is calculated to obtain new candidate values ​​for connection strength weights. Finally, the new candidate values ​​are compared with the weight rationality boundary. If the candidate value is within the range of the lower bound of 0.1 to the upper bound of 5.0, the weight is directly updated; otherwise, the boundary value is used to complete the adjustment.

[0052] The path construction module, based on the iteratively updated initial relationship graph, filters relationship edges with connection strength weights exceeding a preset weight threshold of 0.7 and marks them as high-weight critical edges. Simultaneously, it identifies anomalous relationship nodes as candidate path starting points, forming a critical subgraph structure. Based on this structure, it performs a reverse traversal from each anomalous node along the high-weight critical edges, with a maximum backtracking depth of 5, recording in real-time the initial potential fault propagation chain formed by nodes and edges.

[0053] All initial potential fault propagation chains are evaluated, and the comprehensive propagation strength index of each chain is calculated, which is the geometric mean of the connection strength weights of all relation edges on the chain. At the same time, the centrality index of each relation node on the chain is combined for correction. The node influence factor is assigned a topological weight coefficient of 0.3, and the comprehensive propagation strength index is assigned a complement weight coefficient of 0.7. The weighted sum is calculated to obtain the corrected index, and the top 20% of chains are selected as fault propagation paths.

[0054] The root cause localization module receives these paths and indicators, calculates the root cause suspicion index for each relation node, which is to multiply the basic centrality value by a preset weight of 0.4, multiply the comprehensive propagation strength index by a complement weight of 0.6, perform weighted scoring and normalization, mark nodes whose indices exceed a preset dynamic threshold as candidate root cause nodes, calculate the matching degree value in conjunction with the fault mode library, filter nodes that exceed the matching degree threshold of 0.75, select the node with the largest matching degree value as the root cause of the fault, and finally generate processing suggestions to be fed back to the data acquisition module.

[0055] This embodiment uses real-time data acquisition and dynamic quality assessment, combined with AI large model to perform multi-source data fusion and relationship reasoning, to automatically construct accurate fault propagation paths and locate root causes, forming a closed-loop optimization, significantly improving fault handling efficiency and system reliability, and reducing manual intervention.

[0056] The above description is merely an embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent structural or procedural transformations made based on the content of the present invention's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of the present invention.

Claims

1. An AI large model-based full-automatic log analysis and fault processing system, characterized in that: It includes a data acquisition module, a data fusion module, a path construction module, and a root cause localization module; The data acquisition module is used to acquire multi-source data from a preset target system in real time, and to preprocess the multi-source data to generate a standardized multi-source data stream. The data fusion module uses an AI big data model to perform real-time relationship reasoning on log events, performance indicators and topology in the standardized multi-source data stream, constructs an initial relationship graph, and iteratively updates the initial relationship graph according to the new standardized multi-source data stream, adjusting the attributes of relationship nodes and the connection strength weights of relationship edges in the initial relationship graph. The path construction module, based on the iteratively updated initial relationship graph, filters out the relationship edges whose connection strength weight values ​​exceed the preset weight threshold of the target system, marks the relationship edges as high-weight key edges, and takes the relationship node corresponding to the detected abnormal indicators or error logs as the starting point, performs reverse tracing and path splicing along the high-weight key edges until it backtracks to the point where no new starting point is generated, thus generating a fault propagation path. The root cause localization module analyzes the distribution pattern of the connection strength weights of all the relation edges in the fault propagation path, sorts the relation nodes by centrality in the topology, and compares them with a preset fault mode library to locate the root cause of the fault.

2. The AI large model-based full-automatic log analysis and fault processing system according to claim 1, characterized in that: The preset target system in the data acquisition module is specifically a software system containing one or more interconnected service components; The data acquisition module includes preprocessing the multi-source data by performing timestamp alignment, outlier removal, and format standardization on log events, performance metrics, and topology data from different data sources to generate a standardized multi-source data stream.

3. The AI large model-based full-automatic log analysis and fault processing system according to claim 2, characterized in that: The method for constructing the initial relationship graph in the data fusion module is as follows: define the record entries, performance indicators, and specific components or connections in the topology of each independent log event in the standardized multi-source data stream as independent data entities, and mine the temporal, causal, and spatial relationships between the data entities. The temporal, causal, and spatial relationships are defined as relationships between entities. The data entities are used as relationship nodes, and the relationships between entities are used as relationship edges to construct the initial relationship graph.

4. The AI large model-based full-automatic log analysis and fault processing system according to claim 3, characterized in that: The steps for adjusting the connection strength weights in the data fusion module are as follows: S1. Based on the new input standardized multi-source data stream, identify each data entity contained therein and the relationship between the entities, and calculate the incremental strength of the newly identified relationship between the entities relative to the existing relationship between the entities in the current initial relationship graph; S2. A preset time decay factor is introduced to correct the incremental intensity. The time decay factor is determined based on the interval length between the timestamp of the newly input standardized multi-source data stream and the latest timestamp of the current record of the initial relationship graph. A first time threshold and a second time threshold are preset. When the interval length is less than or equal to the first time threshold, the time decay factor is 1; when the interval length is greater than the first time threshold and less than or equal to the second time threshold, the time decay factor is 0.5; when the interval length is greater than the second time threshold, the time decay factor is 0. The incremental intensity is multiplied by the time decay factor determined accordingly to obtain the time-corrected instantaneous relationship intensity. S3. Weighted fusion of the instantaneous relationship strength with the current connection strength weight of the corresponding relationship edge in the initial relationship graph; assign a preset dynamic influence coefficient to the instantaneous relationship strength, and assign a static compensation coefficient equal to one minus the dynamic influence coefficient to the current connection strength weight; multiply the instantaneous relationship strength by the dynamic influence coefficient, and multiply the current connection strength weight by the static compensation coefficient, add the two product results, and calculate the new candidate value of the connection strength weight of the relationship edge; S4. Compare the new candidate value of the connection strength weight with the preset weight rationality boundary. If the new candidate value of the connection strength weight is within the weight rationality boundary, it is directly updated as the connection strength weight of the relation edge. If the new candidate value of the connection strength weight exceeds the weight rationality boundary, the boundary value of the weight rationality boundary is taken as the connection strength weight, thus completing the adjustment and update of the connection strength weight of the relation edge in the initial relation graph.

5. The AI large model-based full-automatic log analysis and fault processing system according to claim 4, characterized in that: The steps for generating the fault propagation path in the path construction module are as follows: Q1. Based on the iteratively updated initial relation graph, the relation edges whose connection strength weight exceeds the preset weight threshold of the target system are selected and marked as high-weight key edges. At the same time, all relation nodes marked as abnormal at the current time are identified as candidate path starting points, forming a key subgraph structure connected by the high-weight key edges. Q2. Based on the key subgraph structure, starting from the starting point of each candidate path marked as an abnormal state, traverse backwards along the high-weight key edge to trace all possible causal upstream relationship nodes, and record the relationship nodes and relationship edges passed through in real time during the traversal process to form multiple initial potential fault propagation chains. Q3. Evaluate all the initial potential fault propagation chains and calculate the comprehensive propagation strength index of each initial potential fault propagation chain. The comprehensive propagation strength index is calculated by geometrically averaging the connection strength weights of all the relational edges on the links of the initial potential fault propagation chain. At the same time, it is corrected by combining the centrality index of each relational node on the links of the initial potential fault propagation chain in the target system topology. Based on the corrected comprehensive propagation strength index, all the initial potential fault propagation chains are sorted in descending order, and a number of the initial potential fault propagation chains with the highest ranking according to a preset proportion are selected as the fault propagation paths. The complete sequence of relation nodes, the sequence of relation edges, and the corresponding comprehensive propagation strength index for each fault propagation path are output in a structured manner.

6. The AI large model-based full-automatic log analysis and fault processing system according to claim 5, characterized in that: The method for forming the initial potential fault propagation chain in step Q2 is as follows: Q201. Using the candidate path starting point marked as abnormal state in the key subgraph structure as the starting node of the current traversal, initialize an empty current propagation path record, and at the same time create a set of nodes to be explored and add all the starting nodes to the set of nodes to be explored. Q202. Take a starting node from the set of nodes to be explored as the current node, and filter out the relationship edges pointing to the current node from the relationship edges between the current node and other relationship nodes in the key subgraph structure. If such a relational edge exists, the source relational node connected to the relational edge pointing to the current node is marked as a new node to be explored and added to the set of nodes to be explored. At the same time, the current node, the relational edge pointing to the current node, and the corresponding source relational node are added to the current propagation path record in the exploration order. Q203. Repeat step Q202 until the set of nodes to be explored is empty or the preset maximum backtracking depth is reached. At this time, the complete sequence of relation nodes and relation edges stored in the current propagation path record is output as an initial potential fault propagation chain. After the above process is completed for all the starting nodes, multiple initial potential fault propagation chains are obtained.

7. The AI large model-based full-automatic log analysis and fault processing system according to claim 6, characterized in that: The specific steps for correcting the centrality index of each relation node in the initial potential fault propagation chain within the target system topology in step Q3 are as follows: Q301. For each of the initial potential fault propagation chains, extract all the relational nodes contained in the initial potential fault propagation chain, calculate the basic centrality value based on the number of direct connections between each relational node and other relational nodes in the topology of the target system, and normalize the basic centrality value to obtain the node influence factor. Q302. The node influence factor is fused and corrected with the comprehensive propagation intensity index of the initial potential fault propagation chain. A preset topological weight coefficient is assigned to the node influence factor, and its complement weight coefficient is assigned to the comprehensive propagation intensity index. The product of the node influence factor and the topological weight coefficient and the product of the comprehensive propagation intensity index and the complement weight coefficient are added together to obtain the corrected comprehensive propagation intensity index. The complement weight coefficient is a difference weight coefficient obtained by subtracting the topological weight coefficient.

8. The full-automatic log analysis and fault processing system based on an AI large model according to claim 7, characterized in that: The calculation steps for the basic centrality value in step Q301 are as follows: Q3011. For each relation node in the topology of the target system, count the total number of direct connection edges between the relation node and all other relation nodes to obtain the original connection degree of the relation node, and use the original connection degree as the initial input for calculating the basic centrality value. Q3012. Divide the original connectivity degree of each relation node by the normalized baseline value preset by the target system. The normalized baseline value is the historical maximum connectivity degree of all relation nodes in the topology of the target system to obtain a normalized connectivity ratio between 0 and 1. Multiply the normalized connectivity ratio by 100 to convert it into a percentage value to obtain the basic centrality value of the relation node.

9. The full-automatic log analysis and fault processing system based on an AI large model according to claim 8, characterized in that: The method for locating the root cause of the fault in the root cause localization module is as follows: R1. Receive the fault propagation path and its corresponding comprehensive propagation intensity index from the path construction module, extract the basic centrality value of all relation nodes in each fault propagation path, and perform weighted fusion of the basic centrality value and the comprehensive propagation intensity index according to a preset weight to calculate the root cause suspicion index of each relation node. R2. Mark the relationship nodes whose root cause suspicion index exceeds the preset dynamic threshold as candidate root cause nodes, and calculate the matching degree value between each candidate root cause node and the historical fault mode features recorded in the fault mode library. The matching degree value is compared with a preset matching degree threshold, and all qualified candidate root cause nodes that exceed the matching degree threshold are filtered out. The node with the largest matching degree value is selected from the qualified candidate root cause nodes and determined as the root cause of the fault.

10. The full-automatic log analysis and fault processing system based on an AI large model according to claim 9, characterized in that: After locating the root cause of the fault in the root cause localization module, the module also includes generating processing suggestions and feeding the processing suggestions back to the data acquisition module.

Images