Fault insight and self-healing method and system based on chain control

Abstract

The invention relates to the technical field of intelligent operation and maintenance, in particular to a fault insight and self-healing method and system based on chain control. According to the chain control-based fault insight and self-healing method, indexes, logs and tracking data of a micro-service environment are dynamically collected and fused; the method comprises the following steps: dynamically constructing and visualizing a service dependent topology, realizing dynamic topology construction and intelligent visual rendering, performing intelligent root cause analysis and predictive diagnosis based on multi-dimensional data, realizing self-adaptive early warning, generating an intelligent decision, performing closed-loop learning optimization based on an execution result, and realizing automatic fault repair and continuous optimization. According to the fault insight and self-healing method and system based on chain control, a predictive early warning mechanism is established, automatic fault self-healing is realized, and through intelligent and automatic means, manual intervention is reduced, the operation and maintenance cost is reduced, the average repair time is shortened, the service continuity is guaranteed, and the system availability is improved.

Description

Technical Field

[0001] This invention relates to the field of intelligent operation and maintenance technology, and in particular to a fault insight and self-healing method and system based on chain control. Background Technology

[0002] As enterprises deepen their digital transformation, traditional monolithic application architectures, due to their bloated nature, lack of scalability, and rigid technology stacks, can no longer meet the demands of rapid iteration and elastic scaling. Microservice architecture, by breaking down applications into a set of small, autonomous services, each running in an independent process and communicating through lightweight mechanisms (such as HTTP / RPC), significantly improves system flexibility, maintainability, and scalability. The maturity of containerization technologies and container orchestration platforms (such as Kubernetes) has further accelerated the adoption of microservice architectures.

[0003] However, while microservice architecture brings many advantages, it also introduces significant operational complexity:

[0004] 1) Complex service dependencies. As the number of services increases, the call chain becomes too long, the dependency relationships become a network, and the failure propagation path is difficult to predict.

[0005] 2) Difficulty in fault location. A request may span multiple services, and an anomaly in any link may lead to a complete failure. Traditional monitoring tools struggle to quickly locate the root cause.

[0006] 3) High operation and maintenance costs. It requires the deployment, monitoring, scaling up and down, and fault recovery of a large number of services, and is highly dependent on the experience of operation and maintenance personnel.

[0007] 4) System availability challenges. Delays in fault detection and recovery directly impact user experience and business continuity.

[0008] Several microservice governance solutions exist in the industry, but they still have many shortcomings:

[0009] 1) APM (Application Performance Monitoring) based solutions. Examples include SkyWalking, Pinpoint, and Zipkin, which collect link data through distributed tracing technology to provide performance metrics and dependency graphs. However, their functionality mainly focuses on data collection and visualization, lacking in-depth analysis and automated response capabilities.

[0010] 2) Service Mesh-based solutions. Examples include service mesh frameworks like Istio and lightweight service meshes like Linkerd, which use sidecar proxies for traffic management, observation, and security. While these solutions provide rich telemetry data, they still rely on external systems or manual intervention for intelligent analysis, root cause analysis, and self-healing.

[0011] 3) Rule-based monitoring and alerting solutions. For example, the monitoring system Prometheus combined with the alert management tool AlertManager supports threshold-based alerting rules, but it cannot handle complex and related faults, has a high false alarm and false negative rate, and lacks self-healing capabilities.

[0012] 4) Traditional automated operation and maintenance scripts. Scripts are typically written using automated operation and maintenance tools such as Ansible and distributed remote execution systems like SaltStack to handle common faults. However, they lack deep integration with real-time monitoring data, making it impossible to achieve precise triggering and intelligent decision-making.

[0013] Based on the existing microservice governance technologies mentioned above, the main shortcomings can be summarized as follows:

[0014] 1) The problem of data silos is serious. Link data, log data, and indicator data are scattered in different systems and lack correlation analysis.

[0015] 2) Lack of intelligent analysis capabilities. Most systems still rely on preset thresholds or human experience, and cannot achieve predictive analysis and root cause localization.

[0016] 3) Early warning and self-healing are disconnected. Manual intervention is still required after an alarm is triggered, making it impossible to form a closed loop of "discovery-analysis-repair".

[0017] 4) Poor scalability and adaptability. Rules and scripts need to be written in advance, making it difficult to adapt to dynamically changing microservice environments.

[0018] To address the above problems, this invention proposes a fault insight and self-healing method and system based on chain control. Summary of the Invention

[0019] To overcome the shortcomings of existing technologies, this invention provides a simple and efficient chain-controlled fault detection and self-healing method and system.

[0020] This invention is achieved through the following technical solution:

[0021] A fault insight and self-healing method based on chain control includes the following steps:

[0022] Step S1: Build a multimodal link data collector (MLDC) to dynamically collect and integrate metrics, logs and tracing data from the microservice environment;

[0023] The Multimodal Link Data Acquisition Unit (MLDC) uses an Adaptive Smart Probe (ASA) for data acquisition.

[0024] During the data collection process, multidimensional feature encoding technology is used to encode indicator data, log content and tracking information into standardized feature vectors. A cross-modal attention mechanism is used to achieve semantic alignment and correlation fusion between different data sources to form a unified semantic data view.

[0025] In step S1, the Adaptive Smart Probe (ASA) has a built-in lightweight decision engine that dynamically adjusts the collection strategy according to the service criticality level.

[0026] For core services with a Service Criticality Level (SCL) of not less than 8, a full data collection mode will be enabled, with the collection frequency increased to 10 seconds per time, and deep tracking function will be enabled; for non-critical services with a Service Criticality Level (SCL) of not less than 3, a sampling data collection mode will be adopted, with the frequency reduced to 60 seconds per time.

[0027] Step S2: Based on the collected data, dynamically construct and visualize the service dependency topology to achieve dynamic topology construction and intelligent visualization rendering;

[0028] The Topology Building and Visualization Engine (TVE) automatically builds and dynamically updates the call chain topology diagram between microservice components and applications based on real-time collection of various runtime data in the distributed environment.

[0029] The call chain topology diagram is used to display the call relationships between components and applications, present performance indicators, and reflect the success and failure status of calls in real time.

[0030] In step S2, at the visualization level, a color coding mechanism is used to distinguish the call status: successfully called links are clearly marked in green, while failed calls are highlighted in red, enabling maintenance personnel to quickly locate abnormal links.

[0031] The call chain topology diagram supports drilling down layer by layer from the global view to the local links, allows users to zoom and pan the view, and click on any node or link to view detailed runtime data, providing real-time visualized microservice runtime status monitoring capabilities.

[0032] Step S3: Build an Intelligent Analysis Engine (IAE) to perform intelligent root cause analysis and predictive diagnosis based on multidimensional data;

[0033] In terms of intelligent root cause analysis, the Intelligent Analysis Engine (IAE) is based on graph neural network-based root cause localization technology. By analyzing the potential causal relationships between historical and real-time data, it locates the source of the anomaly and its propagation path, and calculates the contribution of each potential factor to the current failure.

[0034] In predictive diagnostics, the Intelligent Analytics Engine (IAE) integrates multi-scale time-series features, including second-level, minute-level, and hour-level features, and identifies abnormal patterns based on time-series anomaly detection. Through time-series modeling and dynamic weight adjustment, it can predict the long-term trends and short-term fluctuations of key indicators.

[0035] In step S3, the first step is to compare the abnormal deviations of each factor with historical and real-time data, and combine their position and intensity in the causal propagation path to calculate the specific impact weight of each factor on the final failure index, and output it in the form of contribution score.

[0036] The final result is a root cause ranking list, which clearly shows the influence ratio of each factor, and provides the variable relationship between each factor and related results, enabling intuitive effect value estimation, thereby accurately locating the core root cause and quantifying its influence.

[0037] The prediction results are output in a probabilistic form, and the confidence level of an anomaly occurrence is calculated; when the confidence level exceeds a user-defined threshold, an alert is automatically triggered.

[0038] Step S4: Construct an early warning rule hub (ARC) to achieve adaptive early warning and generate intelligent decisions;

[0039] The Early Warning Rule Hub (ARC) uses a dynamic risk scoring mechanism as the basis for triggering early warnings. The risk scoring mechanism is dynamically calculated from three dimensions, and the calculation formula is as follows:

[0040] Risk Score (RS) = Severity × Probability × Impact

[0041] Among them, the severity of the anomaly is determined by a comprehensive assessment of the degree to which the indicator deviates from the normal baseline and the criticality level of the business link in which it is located.

[0042] Probability of occurrence: Calculated based on the anomaly occurrence confidence level output by the predictive diagnostic module and the occurrence frequency in the same historical period;

[0043] Scope of impact: Quantify the number of affected businesses, service instances, and users through real-time topology analysis;

[0044] When the risk score exceeds the dynamically adjusted threshold, different levels of warnings are automatically generated according to the pre-set warning strategy to ensure that the warning information matches the actual business risks.

[0045] When an alert is triggered, based on the current system state, the best handling solution is recommended from a multi-objective optimization model, including three relevant handling solutions: shortest recovery time, minimum resource consumption, and minimum business impact. The algorithm is used to find the Pareto optimal solution set among them.

[0046] In step S4, a continuously updated fault handling suggestion knowledge base is constructed, in which each fault handling suggestion is structured to include operation type, expected effect and execution cost.

[0047] Step S5: Construct a self-healing actuator (SEE), perform closed-loop learning optimization based on the execution results, and realize automated fault repair and continuous optimization;

[0048] The Self-Healing Actuator (SEE) has built-in process control and safety management mechanisms;

[0049] Before performing any remediation operations, the entire process of the solution is pre-verified in an isolated sandbox environment to simulate real business traffic and execution environment, ensuring the effectiveness and security of the solution and effectively preventing business risks caused by misoperation.

[0050] After verification, based on the preset strategy, a gradual deployment approach is adopted, including canary releases and batch execution, to gradually promote the repair actions to the production environment. The execution status is monitored in real time during the process, and one-click pause and rollback are supported to maximize the security of operation and maintenance.

[0051] All self-healing operations generate structured execution logs and audit reports;

[0052] Construct an end-to-end closed-loop learning mechanism to evaluate and optimize the Intelligent Analytics Engine (IAE) based on the feedback execution results.

[0053] In step S5, the results of each self-healing operation are fully recorded and evaluated in multiple dimensions. An optimization sample library is constructed based on the decision context of key indicators and the early warning rule hub (ARC). The key indicators include execution status, execution time, resource consumption, business indicator recovery curve, and impact on surrounding systems.

[0054] By leveraging AI technology to conduct in-depth analysis of historical self-healing data in the optimization sample library, key patterns in successful handling are identified, the root causes of failed operations are summarized, the most effective repair strategies under different fault scenarios are discovered, and the fault handling suggestion knowledge base is automatically updated to achieve continuous iteration and effectiveness improvement of self-healing strategies.

[0055] This chain-controlled fault insight and self-healing system, used to implement the above methods, includes:

[0056] Multimodal Link Data Acquisition (MLDC): Employs a distributed architecture design, including adaptive smart probes and cross-modal attention fusion modules;

[0057] Adaptive intelligent probes are responsible for dynamically adjusting data collection strategies and frequencies based on predefined service criticality levels;

[0058] The cross-modal attention fusion module is responsible for feature encoding and semantic alignment of metrics, logs, and tracking data to form a unified multi-dimensional feature vector.

[0059] Topology Building and Visualization Engine (TVE): Responsible for dynamically building and visualizing service dependency topology based on real-time collected link data, realizing dynamic topology building and intelligent visualization rendering;

[0060] Intelligent Analysis Engine (IAE): Includes root cause localization module and predictive diagnostic module;

[0061] The root cause localization module is responsible for causal analysis based on graph neural networks and historical and real-time data, calculating the contribution score of each potential factor to the fault and generating a root cause ranking.

[0062] The predictive diagnostic module is responsible for predicting the future state of key indicators based on multi-scale time series analysis and outputting the confidence level of anomalies.

[0063] Early Warning Rule Hub (ARC): Responsible for intelligently assessing fault risks using a dynamic risk scoring mechanism, achieving adaptive early warning, and generating intelligent decisions;

[0064] Self-Healing Actuator (SEE): Built-in process control and safety management mechanisms, including a sandbox pre-verification module, a progressive execution module, and a closed-loop learning module;

[0065] The sandbox pre-validation module is responsible for simulating and validating the effectiveness and safety of the response plan in an isolated environment.

[0066] The progressive execution module is responsible for performing repair operations using canary releases or batch releases, and supports pausing and rolling back during the execution process;

[0067] The closed-loop learning module is responsible for recording and evaluating the results of each self-healing operation and using reinforcement learning to optimize subsequent handling strategies.

[0068] A chain-controlled fault detection and self-healing device includes a memory and a processor; the memory is used to store a computer program, and the processor is used to implement the method when executing the computer program.

[0069] A readable storage medium storing a computer program that, when executed by a processor, implements the method.

[0070] The beneficial effects of this invention are as follows: This chain-controlled fault insight and self-healing method and system not only realizes the dynamic construction and visualization of the global link topology, but also realizes intelligent fault diagnosis and root cause localization, establishes a predictive early warning mechanism, and realizes automated fault self-healing. Through intelligent and automated means, it reduces manual intervention, lowers operation and maintenance costs, shortens the average repair time, ensures business continuity, and improves system availability. Attached Figure Description

[0071] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0072] Appendix Figure 1 This is a schematic diagram of the fault insight and self-healing method based on chain control of the present invention. Detailed Implementation

[0073] To enable those skilled in the art to better understand the technical solutions of this invention, the technical solutions in the embodiments of this invention will be clearly and completely described below in conjunction with the embodiments of this invention. Obviously, the described embodiments are merely some embodiments of this invention, and not all embodiments. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this invention.

[0074] This chain-controlled fault insight and self-healing method includes the following steps:

[0075] Step S1: Build a multimodal link data collector (MLDC) to dynamically collect and integrate metrics, logs and tracing data from the microservice environment;

[0076] The Multimodal Link Data Acquisition Unit (MLDC) uses an Adaptive Smart Probe (ASA) for data acquisition.

[0077] During the data collection process, multidimensional feature encoding technology is used to encode indicator data, log content and tracking information into standardized feature vectors. A cross-modal attention mechanism is used to achieve semantic alignment and correlation fusion between different data sources, forming a unified semantic data view and providing high-quality input for subsequent analysis.

[0078] In step S1, the Adaptive Smart Probe (ASA) has a built-in lightweight decision engine that dynamically adjusts the collection strategy according to the service criticality level (e.g., request rate, data throughput, and other key indicators).

[0079] For core services with a Service Criticality Level (SCL) of not less than 8, a full data collection mode will be enabled, with the collection frequency increased to 10 seconds per time, and deep tracking function will be enabled; for non-critical services with a Service Criticality Level (SCL) of not less than 3, a sampling data collection mode will be adopted, with the frequency reduced to 60 seconds per time.

[0080] Step S2: Based on the collected data, dynamically construct and visualize the service dependency topology to achieve dynamic topology construction and intelligent visualization rendering;

[0081] The Topology Building and Visualization Engine (TVE) automatically builds and dynamically updates the call chain topology diagram between microservice components and applications based on real-time collection of various runtime data in the distributed environment.

[0082] The call chain topology diagram is used to display the call relationships between components and applications, comprehensively present performance indicators including call time, throughput, error rate, etc., and reflect the success and failure status of calls in real time.

[0083] In step S2, at the visualization level, an intuitive color coding mechanism is used to distinguish the call status: successfully called links are clearly marked in green, while failed calls are highlighted in red, enabling maintenance personnel to quickly locate abnormal links.

[0084] The call chain topology map supports drilling down layer by layer from the global view to the local links. Users can zoom and pan the view and click on any node or link to view its detailed runtime data, such as call sequence, dependency path and real-time performance statistics, providing real-time visualized microservice runtime status monitoring capabilities.

[0085] Step S3: Build an Intelligent Analysis Engine (IAE) to perform intelligent root cause analysis and predictive diagnosis based on multidimensional data;

[0086] In terms of intelligent root cause analysis, the Intelligent Analysis Engine (IAE) is based on graph neural network-based root cause localization technology. By analyzing the potential causal relationships between historical and real-time data, it locates the source of the anomaly and its propagation path, and calculates the contribution of each potential factor to the current failure.

[0087] In predictive diagnostics, the Intelligent Analytics Engine (IAE) integrates multi-scale time-series features, including second-level, minute-level, and hour-level features, and identifies abnormal patterns based on time-series anomaly detection. Through time-series modeling and dynamic weight adjustment, it achieves accurate prediction of the long-term trends and short-term fluctuations of key indicators.

[0088] In step S3, the first step is to compare the abnormal deviations of each factor with historical and real-time data, and combine their position and intensity in the causal propagation path to accurately calculate the specific impact weight of each factor (such as CPU utilization, memory usage, request throughput of key services, database connection pool utilization, cross-service call latency, frequency of occurrence of specific business error codes, etc.) on the final failure index, and output it in the form of contribution score (0-1).

[0089] The final result is a root cause ranking list, which clearly shows the influence ratio of each factor and provides the variable relationship between each factor and the relevant results, enabling intuitive effect value estimation (such as "if a factor changes by a certain amount, it will lead to a change of a certain result by a certain amount"), thereby accurately locating the core root causes and quantifying their influence.

[0090] The prediction results are output in a probabilistic form, and the confidence level of anomaly occurrence is calculated. When the confidence level exceeds a custom threshold (such as 0.85), an early warning is automatically triggered to achieve the effect of early intervention.

[0091] Step S4: Construct an early warning rule hub (ARC) to achieve adaptive early warning and generate intelligent decisions;

[0092] The Early Warning Rule Centralization (ARC) employs a dynamic risk scoring mechanism as the basis for triggering early warnings, abandoning the traditional fixed threshold. The risk scoring mechanism is dynamically calculated from three dimensions, as shown in the following formula:

[0093] Risk Score (RS) = Severity × Probability × Impact

[0094] Among them, the severity of the anomaly is determined by a comprehensive assessment of the degree to which the indicator deviates from the normal baseline and the criticality level of the business link in which it is located.

[0095] Probability of occurrence: Calculated based on the anomaly occurrence confidence level output by the predictive diagnostic module and the occurrence frequency in the same historical period;

[0096] Scope of impact: Quantify the number of affected businesses, service instances, and users through real-time topology analysis;

[0097] When the risk score exceeds the dynamically adjusted threshold, different levels of warnings (such as alert, warning, and severe) are automatically generated according to the pre-set warning strategy to ensure that the warning information matches the actual business risks.

[0098] When an alert is triggered, based on the current system state, the best handling solution is recommended from a multi-objective optimization model. This includes three related handling solutions: shortest recovery time, minimum resource consumption, and minimum business impact. The algorithm finds the Pareto optimal solution set, rather than a single answer, to provide operations and maintenance personnel with multiple trade-off options.

[0099] Build a continuously updated fault handling suggestion knowledge base, in which each fault handling suggestion is structured to include operation type (such as restarting service, scaling up nodes, switching traffic), expected effect (how much risk score can be reduced or what indicators can be restored) and execution cost (time required, resource cost, potential side effects);

[0100] Step S5: Construct a self-healing actuator (SEE), perform closed-loop learning optimization based on the execution results, and realize automated fault repair and continuous optimization;

[0101] The Self-Healing Actuator (SEE) has a built-in comprehensive process control and safety management mechanism;

[0102] Before performing any remediation operations, the entire process of the solution is pre-verified in an isolated sandbox environment to simulate real business traffic and execution environment, ensuring the effectiveness and security of the solution and effectively preventing business risks caused by misoperation.

[0103] After verification, based on the preset strategy, a gradual deployment approach is adopted, including canary releases and batch execution, to gradually promote the repair actions to the production environment. The execution status is monitored in real time during the process, and one-click pause and rollback are supported to maximize the security of operation and maintenance.

[0104] All self-healing operations generate structured execution logs and audit reports, recording "when, where, why, what operations were performed, and what the results were," providing the operations team with a completely transparent view of the operations.

[0105] Construct an end-to-end closed-loop learning mechanism to evaluate and optimize the Intelligent Analytics Engine (IAE) based on the feedback execution results.

[0106] In step S5, the results of each self-healing operation are fully recorded and evaluated in multiple dimensions. An optimization sample library is formed based on the decision context of key indicators and the early warning rule hub (ARC). The key indicators include execution status (success / failure), execution time, resource consumption, business indicator recovery curve, and impact on surrounding systems.

[0107] By leveraging AI technologies (such as reinforcement learning and causal inference) to conduct in-depth analysis of historical self-healing data in the optimization sample library, key patterns in successful handling are identified, the root causes of failed operations are summarized, the most effective repair strategies under different fault scenarios are discovered, and the fault handling suggestion knowledge base is automatically updated to achieve continuous iteration and effectiveness improvement of self-healing strategies.

[0108] This chain-controlled fault insight and self-healing system, used to implement the above methods, includes:

[0109] Multimodal Link Data Acquisition (MLDC): Employs a distributed architecture design, including adaptive smart probes and cross-modal attention fusion modules;

[0110] Adaptive intelligent probes are responsible for dynamically adjusting data collection strategies and frequencies based on predefined service criticality levels;

[0111] The cross-modal attention fusion module is responsible for feature encoding and semantic alignment of metrics, logs, and tracking data to form a unified multi-dimensional feature vector.

[0112] As the data perception cornerstone of the system, the Multimodal Link Data Collector (MLDC) undertakes the key task of comprehensive data collection and fusion, and is responsible for the unified collection, fusion and standardized processing of multi-source heterogeneous data in the microservice environment.

[0113] Its core capabilities are reflected in four aspects: ensuring zero business interference by collecting service performance indicators non-intrusively through lightweight probes; deeply integrating with mainstream distributed tracing systems to build complete call chain data; achieving automatic association between logs and tracing information through intelligent log agents; synchronously collecting infrastructure layer resource status and event information; and the Multimodal Link Data Collector (MLDC) adopts an adaptive sampling mechanism, which can dynamically adjust the collection frequency according to the system load, ensuring data integrity and avoiding the impact of the collection process on business performance, providing high-quality, multi-dimensional, real-time and reliable data input for upper-layer analysis.

[0114] Topology Building and Visualization Engine (TVE): Responsible for dynamically building and visualizing service dependency topology based on real-time collected link data, realizing dynamic topology building and intelligent visualization rendering;

[0115] The Topology Building and Visualization Engine (TVE) acts as the "visual hub" of the system, transforming complex microservice dependencies into an intuitive visual representation.

[0116] The Topology Building and Visualization Engine (TVE) uses graph computing algorithms to dynamically construct multi-layered service topology graphs, supporting drill-down from the application layer to the instance layer. Through innovative visualization coding technology, it uses color mapping to represent health status, line thickness to represent traffic volume, and animation effects to display real-time data flow, enabling operations and maintenance personnel to quickly grasp the overall system status. The TVE also provides rich interactive functions, supporting topology search, path tracing, and historical backtracking, greatly improving the efficiency of fault location and system understanding, and providing intuitive and reliable visual support for operations and maintenance decisions.

[0117] Intelligent Analysis Engine (IAE): Includes root cause localization module and predictive diagnostic module;

[0118] The root cause localization module is responsible for causal analysis based on graph neural networks and historical and real-time data, calculating the contribution score of each potential factor to the fault and generating a root cause ranking.

[0119] The predictive diagnostic module is responsible for predicting the future state of key indicators based on multi-scale time series analysis and outputting the confidence level of anomalies.

[0120] The Intelligent Analytics Engine (IAE) is the core of the entire system, integrating various machine learning algorithms to achieve deep data analysis and insights. The IAE employs a multi-model fusion architecture and includes four key analytical capabilities:

[0121] Time-series-based anomaly detection can identify a variety of anomaly patterns;

[0122] Root cause localization based on graph neural networks can accurately identify the source of a fault;

[0123] Predict the scope of fault propagation;

[0124] It supports predictive judgments about the future state of the system.

[0125] The Intelligent Analysis Engine (IAE) continuously optimizes model performance through a continuous learning mechanism, and can automatically extract feature patterns from historical data to achieve intelligent diagnosis and prediction of complex fault scenarios, providing powerful analytical support for automated operation and maintenance.

[0126] Early Warning Rule Hub (ARC): Responsible for intelligently assessing fault risks using a dynamic risk scoring mechanism, achieving adaptive early warning, and generating intelligent decisions;

[0127] This includes a multi-objective optimization module, which recommends a set of optimal disposal solutions from a disposal suggestion knowledge base that simultaneously considers recovery time, resource costs, and business impact.

[0128] The Early Warning Rule Hub (ARC) is the core of this system's intelligent decision-making, responsible for transforming monitoring data into actionable operational insights. It abandons the traditional fixed-threshold-based alarm method, employing a dynamic risk scoring mechanism to intelligently assess fault risks and automatically generate tiered alerts. Based on these alerts, the ARC can recommend the best set of solutions for operations personnel from a multi-objective optimization perspective, comprehensively considering recovery time, resource costs, and business impact. It also provides structured suggestions including operations, expected effects, and costs, thereby achieving a leap from passive alarms to proactive decision-making, and from single-point response to global trade-offs.

[0129] Self-Healing Actuator (SEE): Built-in process control and safety management mechanisms, including a sandbox pre-verification module, a progressive execution module, and a closed-loop learning module;

[0130] The sandbox pre-validation module is responsible for simulating and validating the effectiveness and safety of the response plan in an isolated environment.

[0131] The progressive execution module is responsible for performing repair operations using canary releases or batch releases, and supports pausing and rolling back during the execution process;

[0132] The closed-loop learning module is responsible for recording and evaluating the results of each self-healing operation and using reinforcement learning to optimize subsequent handling strategies.

[0133] The Self-Healing Actuator (SEE), as the automated operation and maintenance terminal of the system, is the final execution link in the intelligent operation and maintenance system. It is responsible for receiving the best handling solution recommended by the Early Warning Rule Center (ARC) and transforming it into safe, reliable, and traceable automated repair operations. Based on the execution results, it performs closed-loop learning and optimization to achieve automated fault repair and continuous optimization. It not only strives to achieve rapid fault response and lossless recovery, but also promotes the evolution of the system from "automation" to "intelligence" through continuous machine learning to optimize the self-healing strategy.

[0134] A chain-controlled fault insight and self-healing device includes a memory and a processor; the memory is used to store a computer program, and the processor is used to execute the method implemented when the computer program is executed.

[0135] A readable storage medium on which a computer program is stored, and a method for implementing the computer program when executed by a processor.

[0136] Compared with existing technologies, this chain-controlled fault insight and self-healing method and system has the following characteristics:

[0137] 1) It enables dynamic construction and visualization of the global link topology. Through multi-source data collection, it automatically generates service dependency graphs, supporting real-time updates and interactive queries.

[0138] 2) Intelligent fault diagnosis and root cause localization are achieved. AI algorithms are used to analyze link data, automatically identify abnormal patterns, locate the root cause service, and predict the impact of faults.

[0139] 3) A predictive early warning mechanism has been established. Based on historical data and real-time trends, potential risks can be identified in advance, triggering precise alerts.

[0140] 4) Automated fault self-healing is achieved. Through pre-set self-healing scripts and an automated execution framework, common faults can be automatically recovered without human intervention.

[0141] 5) Reduced operation and maintenance costs and improved system availability. Intelligent and automated methods reduced manual intervention, shortened mean time to repair (MTTR), and ensured business continuity.

[0142] The foregoing has provided a detailed description of a chain-controlled fault detection and self-healing method and system, as exemplified by this invention. This section uses specific examples to illustrate the principles and implementation methods of the invention. These examples are merely for the purpose of helping to understand the core ideas of this invention. All other embodiments obtained by those skilled in the art without creative effort, without departing from the principles of this invention, should fall within the scope of protection of this invention.

Claims

1. A fault insight and self-healing method based on chain control, characterized in that: Includes the following steps: Step S1: Build a multimodal data collector (MLDC) to dynamically collect and integrate metrics, logs, and tracing data from the microservice environment; The Multimodal Link Data Acquisition Unit (MLDC) uses an Adaptive Smart Probe (ASA) for data acquisition. During the data collection process, multidimensional feature encoding technology is used to encode indicator data, log content and tracking information into standardized feature vectors. A cross-modal attention mechanism is used to achieve semantic alignment and correlation fusion between different data sources to form a unified semantic data view. Step S2: Based on the collected data, dynamically construct and visualize the service dependency topology to achieve dynamic topology construction and intelligent visualization rendering; TVE, the topology building and visualization engine, automatically builds and dynamically updates the call chain topology diagram between microservice components and applications based on real-time collection of various operational data in the distributed environment. The call chain topology diagram is used to display the call relationships between components and applications, present performance indicators, and reflect the success and failure status of calls in real time. Step S3: Build an intelligent analysis engine (IAE) to perform intelligent root cause analysis and predictive diagnosis based on multidimensional data; In terms of intelligent root cause analysis, the Intelligent Analysis Engine (IAE) is based on graph neural network-based root cause localization technology. By analyzing the potential causal relationships between historical and real-time data, it locates the source of the anomaly and its propagation path, and calculates the contribution of each potential factor to the current failure. In terms of predictive diagnostics, the Intelligent Analysis Engine (IAE) integrates multi-scale time-series features, including second-level, minute-level, and hour-level features. Based on time series anomaly detection, it identifies abnormal patterns and, through time series modeling and dynamic weight adjustment, enables the prediction of long-term trends and short-term fluctuations of key indicators. Step S4: Construct the early warning rule hub ARC to achieve adaptive early warning and generate intelligent decisions; The Early Warning Rule Center (ARC) uses a dynamic risk scoring mechanism as the basis for triggering early warnings. The risk scoring mechanism is dynamically calculated from three dimensions, and the calculation formula is as follows: Risk Score (RS) = Severity × Probability × Impact Among them, the severity of the anomaly is determined by a comprehensive assessment of the degree to which the indicator deviates from the normal baseline and the criticality level of the business link in which it is located. Probability of occurrence: Calculated based on the anomaly occurrence confidence level output by the predictive diagnostic module and the occurrence frequency in the same historical period; Scope of impact: Quantify the number of affected businesses, service instances, and users through real-time topology analysis; When the risk score exceeds the dynamically adjusted threshold, different levels of warnings are automatically generated according to the pre-set warning strategy to ensure that the warning information matches the actual business risks. When an alert is triggered, based on the current system state, the best handling solution is recommended from a multi-objective optimization model, including three relevant handling solutions: shortest recovery time, minimum resource consumption, and minimum business impact. The algorithm is used to find the Pareto optimal solution set among them. Step S5: Construct a self-healing actuator (SEE), perform closed-loop learning optimization based on the execution results, and realize automated fault repair and continuous optimization. The self-healing actuator SEE has built-in process control and safety management mechanisms; Before performing any remediation operations, the entire process of the solution is pre-verified in an isolated sandbox environment to simulate real business traffic and execution environment, ensuring the effectiveness and security of the solution and effectively preventing business risks caused by misoperation. After verification, based on the preset strategy, a gradual deployment approach is adopted, including canary releases and batch execution, to gradually promote the repair actions to the production environment. The execution status is monitored in real time during the process, and one-click pause and rollback are supported to maximize the security of operation and maintenance. All self-healing operations generate structured execution logs and audit reports; Construct an end-to-end closed-loop learning mechanism to evaluate and optimize the Intelligent Analysis Engine (IAE) based on the feedback execution results.

2. The fault insight and self-healing method based on chain control according to claim 1, characterized in that: In step S1, the adaptive smart probe ASA has a built-in lightweight decision engine that dynamically adjusts the collection strategy according to the service criticality level. For core services with a Service Criticality Level (SCL) of not less than 8, a full data collection mode will be enabled, with the collection frequency increased to 10 seconds per time, and deep tracking function will be enabled; for non-critical services with a Service Criticality Level (SCL) of not less than 3, a sampling data collection mode will be adopted, with the frequency reduced to 60 seconds per time.

3. The fault insight and self-healing method based on chain control according to claim 1, characterized in that: In step S2, at the visualization level, a color coding mechanism is used to distinguish the call status: successfully called links are clearly marked in green, while failed calls are highlighted in red, enabling maintenance personnel to quickly locate abnormal links. The call chain topology diagram supports drilling down layer by layer from the global view to the local links, allows users to zoom and pan the view, and click on any node or link to view detailed runtime data, providing real-time visualized microservice runtime status monitoring capabilities.

4. The fault insight and self-healing method based on chain control according to claim 1, characterized in that: In step S3, the first step is to compare the abnormal deviations of each factor with historical and real-time data, and combine their position and intensity in the causal propagation path to calculate the specific impact weight of each factor on the final failure index, and output it in the form of contribution score. The final result is a root cause ranking list, which clearly shows the influence ratio of each factor, and provides the variable relationship between each factor and related results, enabling intuitive effect value estimation, thereby accurately locating the core root cause and quantifying its influence. The prediction results are output in a probabilistic form, and the confidence level of an anomaly occurrence is calculated; when the confidence level exceeds a user-defined threshold, an alert is automatically triggered.

5. The fault insight and self-healing method based on chain control according to claim 1, characterized in that: In step S4, a continuously updated fault handling suggestion knowledge base is constructed, in which each fault handling suggestion is structured to include operation type, expected effect and execution cost.

6. The fault insight and self-healing method based on chain control according to claim 5, characterized in that: In step S5, the results of each self-healing operation are fully recorded and evaluated in multiple dimensions. An optimization sample library is formed based on the decision context of key indicators and the early warning rule hub ARC. The key indicators include execution status, execution time, resource consumption, business indicator recovery curve, and impact on surrounding systems. By leveraging AI technology to conduct in-depth analysis of historical self-healing data in the optimization sample library, key patterns in successful handling are identified, the root causes of failed operations are summarized, the most effective repair strategies under different fault scenarios are discovered, and the fault handling suggestion knowledge base is automatically updated to achieve continuous iteration and effectiveness improvement of self-healing strategies.

7. A fault detection and self-healing system based on chain control, characterized in that: For implementing the method as described in any one of claims 1 to 6, comprising: Multimodal Link Data Acquisition (MLDC): Employs a distributed architecture design, including adaptive smart probes and cross-modal attention fusion modules; The adaptive smart probe is responsible for dynamically adjusting the data collection strategy and frequency according to the predefined service criticality level; The cross-modal attention fusion module is responsible for feature encoding and semantic alignment of metrics, logs, and tracking data to form a unified multi-dimensional feature vector; TVE (Topology Building and Visualization Engine): Responsible for dynamically building and visualizing service dependency topology based on real-time collected link data, realizing dynamic topology building and intelligent visualization rendering; Intelligent Analysis Engine (IAE): Includes a root cause localization module and a predictive diagnostics module; The root cause localization module is responsible for calculating the contribution score of each potential factor to the fault and generating a root cause ranking based on graph neural networks and causal analysis of historical and real-time data. The predictive diagnostic module is responsible for predicting the future state of key indicators based on multi-scale time series analysis and outputting the confidence level of anomaly occurrence. The Early Warning Rule Center (ARC) is responsible for intelligently assessing fault risks using a dynamic risk scoring mechanism, achieving adaptive early warning, and generating intelligent decisions. Self-healing actuator (SEE): Built-in process control and safety management mechanisms, including a sandbox pre-verification module, a progressive execution module, and a closed-loop learning module; The sandbox pre-validation module is responsible for simulating and validating the effectiveness and safety of the treatment plan in an isolated environment; The progressive execution module is responsible for performing repair operations using canary releases or batch methods, and supports pausing and rolling back during the execution process; The closed-loop learning module is responsible for recording and evaluating the results of each self-healing operation and using reinforcement learning to optimize subsequent handling strategies.

8. A fault detection and self-healing device based on chain control, characterized in that: It includes a memory and a processor; the memory is used to store a computer program, and the processor is used to execute the computer program to implement the method as described in any one of claims 1 to 6.

9. A readable storage medium, characterized in that: The readable storage medium stores a computer program that, when executed by a processor, implements the method as described in any one of claims 1 to 6.

Images