Method, device and storage medium for handling running framework service failure
Patent Information
- Application Number
- CN202611443933.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-09-17
- Publication Date
- 2026-10-20
AI Technical Summary
[0005]本申请的主要目的在于提供一种运行框架服务故障的处理方法、设备和存储介质,旨在解决故障处理效率低下的技术问题
[0016]本申请提供了一种运行框架服务故障的处理方法,包括通过在运行框架各服务节点部署采集代理以确定稳定运行阶段的正常基线拓扑快照,在检测到服务异常时以异常信号触发故障时刻依赖数据的获取并得到故障依赖拓扑快照,将故障依赖拓扑快照与正常基线拓扑快照进行对比并结合运行框架故障规则库筛选生成故障根因报告,再依据故障根因报告在隔离策略库中匹配目标隔离策略并执行以处理对应服务故障的技术路径,解决了现有服务故障定位方法无法精准捕获运行时依赖关系变化、难以快速定位依赖类故障、故障隔离与根因诊断缺乏联动机制、针对运行框架专属故障适配性不足导致故障处置效率低下的技术问题,提升了故障根因定位的精准度与自动化水平,缩短了平均故障修复时间,增强了运行框架微服务系统的整体可用性与运行稳定性,有效降低了人工故障排查与处置的成本。
Smart Images

Figure FT_1 
Figure FT_2 
Figure FT_3
Abstract
Description
Technical Field
[0001] This application relates to the field of computer network technology, and in particular to a method, device and storage medium for handling service failures in a runtime framework. Background Technology
[0002] In enterprise-level microservice runtime fault location and isolation scenarios, the ability to accurately capture runtime dependencies, build dependency topology snapshots, automatically diagnose fault root causes, and dynamically and accurately isolate faults directly affects the fault repair efficiency, service availability, and business continuity of distributed service systems.
[0003] In related technologies, microservice fault diagnosis and isolation are carried out by manually sorting log information and setting up circuit breaker and degradation rules. This method matches log and indicator anomalies based on preset rules, and the operation and maintenance personnel manually judge the root cause of the fault and then execute fixed circuit breaker and degradation actions. This method is difficult to adapt to the complex operation and maintenance scenarios under the distributed microservice architecture, resulting in low fault handling efficiency.
[0004] The above content is only used to help understand the technical solution of this application and does not represent an admission that the above content is prior art. Summary of the Invention
[0005] The main objective of this application is to provide a method, device, and storage medium for handling failures of runtime framework services, aiming to solve the technical problem of low efficiency in failure handling.
[0006] To achieve the above objectives, this application proposes a method for handling runtime framework service failures, the method comprising: By deploying collection agents on each service node of the runtime framework, normal baseline topology snapshots are determined during the stable operation phase. When a service anomaly is detected, the anomaly signal triggers the acquisition of fault-dependent data to obtain a fault-dependent topology snapshot. By comparing the fault-dependent topology snapshot with the normal baseline topology snapshot and combining the filtering of the runtime framework fault rule base, a fault root cause report is generated. Based on the root cause report, a target isolation policy is matched in the isolation policy library and executed to handle the corresponding service failure through the target isolation policy.
[0007] In one embodiment, based on the deployment of the collection agent on each service node of the runtime framework, and in conjunction with dependency-related events, the collection channel for runtime dependency data is activated; Through the acquisition channel, the dependency topology data, component version information, and resource consumption data of the internal components of the service are determined, forming the original runtime dependency data stream; The stable operation phase is determined according to the service health status indicators. During the stable operation phase, the dependent data streams in the original runtime are periodically aggregated to generate candidate baseline topology snapshots. Verify the consistency of the candidate baseline topology snapshots, and filter out the candidate baseline topology snapshots that cover all service nodes and have dependencies, and use them as the normal baseline topology snapshots.
[0008] In one embodiment, when a service anomaly is detected, the anomaly signal is classified and judged according to its anomaly type, duration, and impact range, and the target anomaly signal is obtained after filtering out false alarm signals. In response to the target abnormal signal, the collection agent of the service node is invoked to obtain the dependency topology data, component version information and resource consumption data at the time of the failure, and the corresponding service node is associated to obtain the fault data stream; The fault data stream is structured and aggregated to generate the fault-dependent topology snapshot, which is then associated and stored with the corresponding normal baseline topology snapshot.
[0009] In one embodiment, based on the anomaly type, duration, and impact range of the abnormal signal, multi-dimensional feature fusion and vectorized encoding processing are performed to obtain an abnormal feature vector; The abnormal feature vectors are matched with the abnormal threshold library, and the target anomalies corresponding to the abnormal feature vectors are classified into Level 1 severe anomalies, Level 2 warning anomalies, and Level 3 prompt anomalies according to the scope of influence and severity. The correlation between the initially classified anomaly levels and the historical anomaly pattern library is verified to determine whether the target anomaly is a known jitter or instantaneous fluctuation, and false alarm signals are filtered out and the anomaly level is corrected. The target anomaly signal is generated based on the revised anomaly level and its corresponding snapshot acquisition strategy.
[0010] In one embodiment, the fault-dependent topology snapshot is compared with the normal baseline topology snapshot using multi-dimensional differential analysis to determine the difference dataset; The differences in the difference dataset are matched with the fault rule base of the running framework to generate a set of candidate fault root causes with a comprehensive matching score. Based on the dependency topology graph, the candidate fault root cause set is inferred to infer the fault propagation path. The source component is traced back from the abnormal node along the dependency link, and intermediate abnormal nodes in the propagation process are removed to obtain the fault root cause ranking result. Based on the failure root cause ranking results, the root cause information, propagation path, and confidence score are integrated to obtain the failure root cause report.
[0011] In one embodiment, the differences in the difference dataset are classified according to their difference types, and then filtered against the rule conditions in the runtime framework fault rule base to determine a subset of candidate rules corresponding to the current difference type. Based on the set of rule condition features of the candidate rule subset, weights are assigned to the core features and auxiliary features respectively to obtain the rule weights; The feature items in the difference data are compared one by one with the rule condition features, and scores are assigned according to the degree of feature matching. The feature matching score is the percentage of the sum of the weighted scores of the hit features divided by the total score of the rule condition features. Based on the candidate rules in the subset of candidate rules, calculate the historical fault similarity score and the impact range weight score respectively; The feature matching score, the historical fault similarity score, and the influence range weight score are weighted and fused according to preset weights to obtain the comprehensive matching score corresponding to the candidate rule. Based on the comprehensive matching score, low-confidence candidates are filtered out, and candidate root causes of failure with scores higher than the matching threshold are retained, generating a set of candidate root causes of failure with the comprehensive matching score.
[0012] In one embodiment, based on the root cause type and root cause component identifier in the root cause report, a target isolation policy corresponding to the root cause type is matched in the isolation policy library; The target isolation strategy is parameterized based on the root cause component identifier and the scope of the fault impact, and an isolation execution instruction is generated. The isolation execution command is sent to the corresponding service node, the service node is isolated according to the target isolation strategy, and the isolation execution status information is obtained; Based on the isolation execution status information, if the service returns to normal, the isolation is confirmed to be successful; if the isolation is ineffective or causes a new exception, the policy is rolled back to the state before isolation.
[0013] In one embodiment, the entire process data of this fault handling is structured and stored in the fault case library to generate fault case records; The adjustment rules for the normal baseline topology snapshot are updated using historical case data from the fault case library, so that the normal baseline topology snapshot can adapt to changes in service load. The root causes of the confirmed cases in the historical case data are matched with the fault rule library of the operating framework. The feature weights and scoring parameters of the successfully matched rules are adjusted, and corresponding fault rules are added for the unmatched fault modes to update the fault rule library of the operating framework. Based on the isolation execution status information, the applicable scenarios and priorities of successful isolation strategies are increased, and the parameters of strategies that are ineffective or trigger rollback are adjusted and downgraded to improve the adaptability and success rate of isolation strategies.
[0014] In addition, to achieve the above objectives, this application also proposes a processing device for runtime framework service failures, the processing device for runtime framework service failures comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the runtime framework service failure processing method as described above.
[0015] In addition, to achieve the above objectives, this application also proposes a storage medium, which is a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements the steps of the above-described method for handling runtime framework service failures.
[0016] This application provides a method for handling service failures in a runtime framework. The method involves deploying acquisition agents on each service node of the runtime framework to determine normal baseline topology snapshots during stable operation. When a service anomaly is detected, an anomaly signal triggers the acquisition of dependency data at the time of the failure, resulting in a failure dependency topology snapshot. The failure dependency topology snapshot is compared with the normal baseline topology snapshot, and a failure root cause report is generated by filtering based on the runtime framework's failure rule base. Finally, based on the failure root cause report, a target isolation policy is matched in the isolation policy base and executed to handle the corresponding service failure. This method solves the technical problems of existing service failure location methods, such as the inability to accurately capture runtime dependency changes, difficulty in quickly locating dependency-type failures, lack of linkage between failure isolation and root cause diagnosis, and insufficient adaptability to runtime framework-specific failures, leading to low efficiency in failure handling. It improves the accuracy and automation level of failure root cause location, shortens the average failure repair time, enhances the overall availability and operational stability of the runtime framework microservice system, and effectively reduces the cost of manual fault investigation and handling.
[0017] In summary, this application solves the technical problem of low fault handling efficiency by deploying a data acquisition agent to determine normal baseline topology snapshots, comparing them with fault snapshots, and generating root cause reports and matching and executing isolation strategies. This improves the accuracy and speed of root cause location and enhances the stability of system operation. Attached Figure Description
[0018] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.
[0019] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0020] Figure 1 A flowchart illustrating the first embodiment of the method for handling service failures within the framework of this application; Figure 2 This is a schematic diagram of the overall process of this application; Figure 3 A flowchart illustrating the fourth embodiment of the method for handling service failures within the operational framework of this application; Figure 4 A flowchart illustrating the sixth embodiment of the method for handling service failures within the operational framework of this application; Figure 5 A flowchart illustrating the eighth embodiment of the method for handling service failures of the operating framework of this application; Figure 6 A schematic diagram of the structure of the equipment for handling faults in the operating framework service of this application.
[0021] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation
[0022] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.
[0023] In related technologies, microservice fault diagnosis and isolation are carried out by manually sorting log information and setting up circuit breaker and degradation rules. This method matches log and indicator anomalies based on preset rules, and the operation and maintenance personnel manually judge the root cause of the fault and then execute fixed circuit breaker and degradation actions. This method is difficult to adapt to the complex operation and maintenance scenarios under the distributed microservice architecture, resulting in low fault handling efficiency.
[0024] This application provides a solution: First, by deploying collection agents on each service node of the runtime framework, a normal baseline topology snapshot during the stable operation phase is determined. Then, when a service anomaly is detected, the acquisition of fault-dependent data is triggered by the anomaly signal to obtain a fault-dependent topology snapshot. The fault-dependent topology snapshot is then compared with the normal baseline topology snapshot, and a fault root cause report is generated by combining the runtime framework fault rule base for filtering. Finally, based on the fault root cause report, a target isolation policy is matched in the isolation policy base and executed to handle the corresponding service fault through the target isolation policy.
[0025] It should be noted that the executing entity in this embodiment can be a computing service device with data processing, network communication, and program execution functions, such as a tablet computer, personal computer, or mobile phone, or an electronic device capable of performing the above functions, or a device for handling runtime framework service failures. The following description uses a device for handling runtime framework service failures as an example to illustrate this embodiment and the subsequent embodiments.
[0026] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.
[0027] This application provides a method for handling runtime framework service failures, referring to... Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the method for handling service failures within the framework of this application.
[0028] In this embodiment, the method for handling runtime framework service failures includes steps S10 to S40: Step S10: Determine the normal baseline topology snapshot for the stable operation phase by deploying collection agents on each service node of the running framework.
[0029] The data collection agent is a lightweight execution component deployed on each service node of the runtime framework to intercept and collect runtime dependency events and state data. A normal baseline topology snapshot is structured baseline data describing the full-link dependencies, component version configurations, and resource baseline status under stable service operation conditions. Examples include inter-component call topology relationships, assembly version identifiers, service node resource usage baseline values, and cross-service call link weights. The stable operation phase refers to a continuous runtime period where core service business indicators are stable, there are no abnormal alarms, and dependencies do not fluctuate drastically.
[0030] In this embodiment, normal baseline topology snapshots can be constructed in three ways. The first is a steady-state threshold-triggered full-scale construction. By real-time monitoring of core indicators such as service request success rate, response latency, and resource volatility, when all indicators are within the normal threshold range for a consecutive preset period, the system is determined to be in a stable operation phase. Then, all nodes are invoked to collect all dependency and status data, aggregating and generating a complete normal baseline topology snapshot. This method offers high snapshot integrity and strong baseline consistency. The second is a sliding window incremental dynamic update. A fixed-length sliding time window is set to continuously collect dependency data from each node. At the end of each window, dependency stability, version consistency, and resource volatility are statistically analyzed. When the stability of multiple consecutive windows meets the standard, the aggregated topology data within the window is iteratively updated to a normal baseline topology snapshot. This method does not require a separate steady-state determination cycle and can adapt to smooth service changes. The third is a business partition-based multi-baseline construction. The operating framework is divided into multiple independent business partitions according to business domains, service clusters, or core call chains. Stable operating status is independently determined for each partition, and a partition baseline snapshot is generated. These are then integrated to form a normal baseline topology snapshot covering the entire framework. Each partition baseline supports independent updates. This approach avoids the impact of single-partition fluctuations on the global benchmark.
[0031] Once the data collection agent is deployed and started running, it begins to continuously collect runtime dependency data and status indicators of each node, and simultaneously executes the determination process for the stable operation phase.
[0032] For example, there are two methods for determining the stable operation phase and generating baseline snapshots. The first is a centralized unified determination and generation, where the framework's central node pulls the operating metrics of all service nodes, centrally performs steady-state determination, and after the determination is passed, a unified collection command is issued. All nodes synchronously collect data and report it to the central node to aggregate and generate a baseline snapshot. This method has unified logic, consistent benchmark standards, and is less prone to partitioning bias. The second method is a distributed autonomous determination and reporting, where each service node autonomously detects its local operating metrics and determines its local steady state. When more than a preset proportion of nodes determine their local steady state, the global system confirms that it has entered the stable operation phase. Each node generates its local baseline data and reports it to the global baseline snapshot. This method has low central node overhead and strong scalability.
[0033] Once the stable operation phase is determined, a normal baseline topology snapshot can be generated based on the following methods.
[0034] In one alternative approach, the system's preset baseline generation template and data verification rules are retrieved. All collected dependency and status data are then standardized and aggregated according to the template structure. After integrity verification, a normal baseline topology snapshot is directly generated. This method is logically intuitive and executes quickly.
[0035] In another alternative approach, the collected multi-node dependency data is first deduplicated and mapped to construct a global component dependency graph. Then, component version information and resource baseline data are overlaid, and attribute labels are added to the nodes and links in the graph. Subsequently, steady-state verification is performed to remove fluctuating data, ultimately generating a normal baseline topology snapshot with attribute labels. This method provides a clear topology structure and complete attribute information, supporting subsequent refined fault comparison and tracing.
[0036] In an exemplary scheme for determining a normal baseline topology snapshot, a lightweight collection agent is first deployed on all service nodes of the runtime framework. Each collection agent is configured with dependency event interception rules, data reporting cycle, and local cache capacity, establishing a communication link between the collection agent and the distributed snapshot database. Subsequently, a steady-state detection process is initiated, with the central detection node pulling core operational metrics from each service node in real time, including business request success rate, average response latency, CPU load volatility, and memory usage volatility. An initial steady-state assessment is performed; when the core operational metrics of all service nodes are within the preset normal threshold range for three consecutive collection cycles, it is preliminarily determined that the system has entered a globally stable operation phase. Next, dependency stability verification is performed, statistically analyzing the full set of component call relationships in the current collection cycle and comparing it with the call relationship set of the previous cycle to calculate dependency overlap. When the overlap is higher than the preset stable threshold and no new abnormal call chains are added, it is confirmed that the system is currently in a valid stable operation phase. Subsequently, a full data collection command was issued, invoking all node collection agents to synchronously collect component dependency topology, assembly version information, and service resource usage data. The collected multi-node data underwent timestamp alignment, deduplication, correlation, and structured aggregation to generate a normal baseline topology snapshot containing a component dependency graph, version information repository, and resource baseline values. Finally, an integrity check was performed on the generated baseline snapshot to confirm its coverage of all service nodes and core business call chains. After successful check, the snapshot was stored in a distributed snapshot database and marked with a unique version number, serving as benchmark data for subsequent fault comparison and analysis.
[0037] Step S20: When a service anomaly is detected, the fault-dependent data is obtained by triggering the fault-dependent topology snapshot with the anomaly signal.
[0038] Service anomalies refer to abnormal operational states in which core operational metrics of a service deviate from normal ranges, leading to decreased business availability. Anomaly signals are trigger control signals generated by the detection system to initiate fault handling procedures. Fault dependency topology snapshots are structured snapshot data collected at the moment the anomaly occurs, reflecting the service dependencies and operational status under fault conditions. Examples include component call relationships of the abnormal node, abnormal component version configurations, fault link resource usage data, and abnormal call link identifiers.
[0039] In this embodiment, fault dependency data collection can be triggered in three ways. First, there is immediate collection triggered by abnormal events. When the detection system detects a service anomaly and generates a valid anomaly signal, it immediately sends an immediate collection command to the collection agent of the corresponding service node. The collection agent freezes the current dependency data and status data and reports it, aggregating and generating a fault dependency topology snapshot. This method has a fast response speed, high snapshot time accuracy, and can capture the complete state at the moment of the fault. Second, there is anomaly tracing and retrospective collection. After receiving an anomaly signal, the system retrospectively retrieves the collection cache data within a preset time period before the anomaly occurred, extracts multiple sets of dependency status data before and after the anomaly occurred, selects the valid data closest to the time of the fault occurrence using the anomaly trigger time as the time anchor, and aggregates and generates a fault dependency topology snapshot. This method avoids data loss at the moment of the anomaly and restores the dependency change process before and after the fault occurred. Third, there is anomaly hierarchical targeted collection. The collection granularity is determined according to the level and impact range of the anomaly signal. Level 1 severe anomalies trigger full collection of all nodes; Level 2 warning anomalies only trigger collection of nodes in the anomaly-related links; and Level 3 alert anomalies only collect data from the anomaly node itself, aggregating and generating fault dependency topology snapshots of the corresponding granularity as needed. This approach can reduce unnecessary data collection overhead and minimize the impact of fault handling on normal operations.
[0040] When the fault handling module receives the abnormal signal, it begins to perform an abnormality validity verification and matches the corresponding collection strategy according to the abnormality level, and starts the process of acquiring fault-dependent data.
[0041] For example, there are two methods for collecting and aggregating fault-dependent data. The first is synchronous centralized collection and aggregation, where a central node uniformly issues collection instructions, and all target nodes synchronously execute collection and report data. After receiving all the data, the central node uniformly performs timestamp alignment and structured aggregation to generate a fault-dependent topology snapshot. This method has high data consistency and a unified snapshot structure, facilitating subsequent comparative analysis. The second method is asynchronous distributed collection and aggregation, where each target node autonomously completes collection and local structured processing after receiving the collection instructions, and asynchronously reports to the central node. The central node gradually aggregates the data according to the node dimension, and after all node data has been reported, it integrates and generates a fault-dependent topology snapshot. This method places less instantaneous pressure on the central node.
[0042] Once the original dependency data at the time of the failure is obtained, a snapshot of the failure dependency topology can be generated based on the following methods.
[0043] In one alternative approach, following the data structure template of a normal baseline topology snapshot, the collected raw fault data undergoes field mapping and format conversion to directly generate a fault-dependent topology snapshot with a structure completely identical to the baseline snapshot. This method offers a unified format, allowing direct use for subsequent differential comparisons and resulting in high execution efficiency.
[0044] In another alternative approach, the collected raw fault data is first annotated to identify indicators and links that significantly deviate from the normal range. Then, it is standardized and aggregated according to the structure of the baseline snapshot to generate a fault-dependent topology snapshot with pre-annotated anomalies. This method can identify anomalous areas in advance, improving the efficiency of subsequent root cause analysis.
[0045] In an exemplary scheme for obtaining fault-dependent topology snapshots, the system first receives raw anomaly signals reported by the detection system and parses the anomaly type, occurrence node, affected links, and alarm level information from the anomaly signals. Anomaly classification is performed, categorizing anomalies into Level 1 severe anomalies, Level 2 warning anomalies, and Level 3 alert anomalies based on the number of service nodes affected and the degree of damage to core services. Simultaneously, false alarm signals are filtered by comparing with historical instantaneous jitter patterns, generating valid anomaly trigger signals. Subsequently, a corresponding collection strategy is matched according to the anomaly level: Level 1 anomalies generate full-node full-data collection instructions, Level 2 anomalies generate related link targeted collection instructions, and Level 3 anomalies generate single-node simplified collection instructions. The collection instructions are sent to the collection agent of the corresponding service node. Upon receiving the instructions, the collection agent reads locally cached dependency event data and real-time runtime data, extracts component dependencies, version configurations, and resource usage data at the time of the anomaly occurrence, and reports them in a structured manner. The reported multi-node fault data is timestamped, invalid data is removed, and duplicate data is deduplicated. It is then standardized and integrated according to the field structure of a normal baseline topology snapshot to generate a fault-dependent topology snapshot with a consistent data structure. Finally, the fault-dependent topology snapshot is associated and bound with the corresponding version of the normal baseline topology snapshot, and stored in the snapshot database for subsequent differential comparison and root cause analysis.
[0046] Step S30: Compare the fault-dependent topology snapshot with the normal baseline topology snapshot, and generate a fault root cause report by combining the filtering of the runtime framework fault rule base.
[0047] The runtime framework fault rule base is a structured knowledge base storing the characteristic conditions, judgment logic, root cause mapping relationships, and matching weights of various runtime framework faults. The fault root cause report is a structured diagnostic output file containing fault root cause location results, fault type, propagation path, and confidence score. Examples include root cause component identification, dependency conflict fault type, fault propagation chain, and root cause confidence score.
[0048] In this embodiment, root cause screening and diagnosis can be carried out in three ways. The first is feature-by-feature rule matching diagnosis, which compares the fault snapshot with the baseline snapshot item by item to extract the difference features. Each difference feature is then matched against the rule conditions in the fault rule base. Once a matching rule is found, candidate root causes are generated, and the final root cause report is output after sorting by matching degree. This method is logically intuitive and highly interpretable. The second is topology propagation tracing diagnosis, which starts with the abnormal node in the fault snapshot and traces back along the dependency call chain. It analyzes the abnormal change points in the chain by combining the normal dependency relationship of the baseline snapshot, and simultaneously matches the propagation path rules in the fault rule base to locate the source component and triggering cause of the fault. This method can uncover indirect faults caused by dependency propagation. The third is multi-dimensional weighted scoring diagnosis, which calculates the fault feature matching degree from three dimensions: dependency relationship difference, version configuration difference, and abnormal resource status. Combined with the weight coefficients of each rule in the fault rule base, the comprehensive confidence of each candidate root cause is calculated, and the root cause with the highest confidence is selected to generate a complete report. This method has high diagnostic accuracy and can effectively distinguish the primary and secondary relationships of multiple root cause faults.
[0049] Once the root cause diagnosis module obtains the fault-dependent topology snapshot and the corresponding normal baseline topology snapshot, it begins differential comparison and rule matching to initiate the root cause diagnosis process.
[0050] For example, snapshot differential comparison can be implemented in two ways. The first is full-field item-by-item differential comparison, which compares all fields of the fault snapshot with the baseline snapshot one by one, extracting all differences across all dimensions to form a full differential dataset. This method provides comprehensive and complete difference extraction, supporting refined root cause analysis. The second is anomaly region targeted differential comparison, which first locates pre-labeled anomaly regions in the fault snapshot, and then performs differential comparison only on the anomaly regions and their associated links to extract core differences, forming a targeted differential dataset. This method requires less computation and has a faster comparison speed.
[0051] Once the differential dataset is extracted, the root causes of failures can be filtered using the following methods in conjunction with the failure rule base.
[0052] In one alternative approach, all rules in the runtime framework's fault rule base are retrieved. The difference items in the difference dataset are matched one by one with the condition items of each rule. The number of hit features for each rule is counted, and the candidate root cause set is obtained by sorting the hit counts. This method is logically simple and executes quickly.
[0053] In another alternative approach, the differential dataset is first categorized according to its differential dimensions. Then, corresponding rule subsets from the fault rule base for each dimension are matched. A feature matching score is calculated for each rule subset, and a comprehensive matching score is obtained by weighting the scores with dimension weights. Finally, the candidate root cause set is obtained by sorting the comprehensive matching scores. This method offers high matching accuracy and demonstrates the diagnostic value of differences across different dimensions.
[0054] In an exemplary scheme for generating a root cause report, a normal baseline topology snapshot, which is associated with and bound to the fault dependency topology snapshot, is first retrieved to establish a component node mapping relationship and a call chain mapping relationship between the two. A multi-dimensional differential comparison is performed, comparing the increase or decrease of component call relationships and the connectivity status of the links from the dependency topology perspective, the changes in assembly versions and differences in configuration parameters from the version perspective, and the deviation and abnormal fluctuations in resource usage from the resource perspective. All differences are extracted and structurally integrated into a difference dataset. Subsequently, the framework's fault rule library is loaded and run. This rule library contains multiple types of fault rules, including dependency conflicts, version compatibility, resource bottlenecks, and assembly loading anomalies. Each rule is configured with feature conditions, matching weights, and root cause mapping relationships. Each difference in the difference dataset is matched one by one with the rule conditions in the rule library, calculating the feature matching score for each rule. A comprehensive matching score is obtained by combining historical fault similarity and fault impact range weights, and a candidate root cause set is generated based on the scores from high to low. Next, fault propagation path reasoning is performed, tracing back along the dependency call chain from the abnormal node to verify the rationality of the propagation of candidate root causes, eliminating intermediate abnormal nodes and fault symptom items, and correcting the root cause ranking results. Finally, based on the root cause result ranked first, the root cause component identifier, fault type, propagation path, confidence score, and preliminary handling suggestions are integrated to generate a structured fault root cause report, which is then output to the fault isolation execution module.
[0055] Step S40: Based on the fault root cause report, match the target isolation policy in the isolation policy library and execute it to handle the corresponding service fault through the target isolation policy.
[0056] The isolation policy library stores a collection of policies containing handling operations, execution processes, parameter configurations, and rollback mechanisms for various types of faults. A target isolation policy is a handling and execution plan that matches the root cause of the current fault and is used to eliminate the impact of the fault and restore service availability. Examples include component hot-swap policies, traffic circuit breaker routing policies, dynamic resource allocation policies, and service instance restart policies.
[0057] In this embodiment, isolation strategies can be matched and executed in three ways. The first is root cause direct matching execution, which directly matches the corresponding standard isolation strategy from the isolation strategy library based on the root cause type and root cause component in the fault root cause report, and executes it directly according to a preset execution process. This method is fast and the handling process is standardized. The second is impact-scope adaptive customized execution, which combines the fault root cause with the impact scope of the current fault and business priorities, parameterizes the matched basic isolation strategy, customizes the scope, intensity, and execution rhythm of isolation execution, generates a target isolation strategy adapted to the current fault, and then executes it. This method has strong adaptability and can avoid excessive isolation affecting normal business. The third is step-by-step verification gradient execution, which breaks down the matched isolation strategy into multiple gradient execution steps. After each step is executed, the service status change is verified. If the fault is alleviated, the next gradient is executed; if ineffective, an alternative strategy is switched, gradually completing the fault isolation and handling. This method has controllable risk and can reduce the probability of secondary faults caused by isolation operations.
[0058] Once the isolation execution module receives the root cause report of the fault, it begins to perform policy matching and parameter configuration, and initiates the fault isolation and handling process.
[0059] For example, there are two methods for issuing and executing isolation strategies. The first is centralized unified issuance and execution, where a central execution node issues isolation instructions uniformly, all target nodes receive the instructions synchronously and execute isolation operations, and the execution results are uniformly reported to the central node. This method ensures consistent execution, strong global control, and facilitates unified verification of isolation effectiveness. The second method is distributed autonomous collaborative execution, where each target node autonomously executes local isolation operations after receiving the isolation instructions, and nodes synchronize their execution status through a message mechanism to collaboratively complete global fault isolation. This method offers high execution flexibility.
[0060] Once the target isolation strategy is matched, fault isolation and effect verification can be performed in the following ways.
[0061] In one alternative approach, all isolation operations are executed at once according to the preset process of the target isolation strategy. After execution, the service status is checked uniformly to confirm whether the fault has been eliminated. This method has a short execution process and fast processing speed.
[0062] In another alternative approach, operations are executed step-by-step according to the execution gradient of the target isolation strategy. The service status is checked after each step, and the execution parameters for the next step are adjusted based on the status feedback, until the fault is eliminated or all steps are completed. This method offers process control and can be dynamically adjusted based on real-time status.
[0063] In an exemplary scheme for implementing a target isolation strategy, the root cause type, root cause component, scope of impact, and related business priority information from the fault root cause report are first parsed. An isolation strategy library is then loaded, containing three categories: component-level, link-level, and node-level isolation strategies. Each strategy is configured with applicable scenarios, execution parameters, effect verification rules, and rollback mechanisms. Initial strategy matching is performed by filtering a subset of candidate isolation strategies corresponding to the root cause type, and then further refining the matching based on the root cause component and scope of impact to obtain a preliminarily adapted target isolation strategy. Next, strategy parameter customization is implemented by adjusting the strategy's execution scope, execution duration, and resource quota based on the number of nodes affected by the fault and business priority, generating parameterized isolation execution instructions. These instructions specify the isolation object, operation type, execution order, and effect verification nodes. The isolation execution instructions are then distributed to the corresponding service nodes in the execution order. Component-related faults trigger dynamic component unloading or hot replacement of compatible versions; link-related faults trigger traffic circuit breaking and backup route forwarding; and resource-related faults trigger dynamic resource allocation and service instance expansion. During execution, core service metrics and alarm status are collected in real time. Perform isolation effectiveness verification by comparing core business metrics and abnormal alarm status before and after isolation. If the metrics return to normal and alarms are cleared, isolation is confirmed as successful, and the success weight and applicable scenario tag of the isolation strategy in the isolation strategy library are updated synchronously. If the metrics do not improve or trigger new abnormal alarms, the strategy is immediately rolled back to the pre-isolation state, and an alternative isolation strategy is automatically switched. Finally, record the entire process data of this fault isolation and store it in the fault case library for subsequent iterative optimization of strategies and rules.
[0064] Further, please refer to Figure 2 , Figure 2This is a schematic diagram of the overall process of this application. The method for fault location and automatic isolation of network framework services based on runtime dependency snapshots (.NET, Dot NET) sequentially executes the following steps after startup: deployment of runtime dependency collection agents, construction and storage of normal baseline dependency snapshots, detection of alarms triggering anomaly detection, and after anomaly detection is confirmed, the following steps are executed sequentially: real-time fault dependency snapshot collection, multi-dimensional comparative analysis of fault snapshots and baseline snapshots, output of root cause reports combined with a dedicated rule base and intelligent diagnostics, matching of automated isolation strategies and execution of isolation operations, fault data collection and self-learning optimization of the baseline and rule base, ultimately completing the entire process. In specific implementation, a runtime dependency data collection module is first deployed. Lightweight collection agents are deployed on each node of the network framework service. By hooking assembly loading events, object creation events of dependency injection (DI) containers, and interceptors for cross-service calls, the module collects in real-time component dependencies, version information, call chain data, and cross-service dependency call topology data within the service. Subsequently, normal baseline dependency snapshots are constructed. During the stable operation phase of the service, runtime dependency data is periodically collected to build a baseline snapshot of the dependency topology under normal conditions, including the internal component dependency graph, cross-service call topology graph, and component version information database, which is then stored in a distributed snapshot database. Next, anomaly detection is triggered by detection alerts. In conjunction with detection and alerting systems such as Application Insights and Prometheus, the service runtime status is monitored in real time. When service anomalies such as response timeouts, error rate spikes, and memory leak alerts are detected, the real-time dependency snapshot collection process is triggered. After confirming the anomaly detection, a real-time fault dependency snapshot is collected to obtain complete runtime dependency data at the time of the fault, including current component dependency relationships, call chain status, component version changes, resource consumption correlation data, and to generate a real-time fault dependency topology snapshot. Next, a multi-dimensional comparative analysis of the fault snapshot and the baseline snapshot is performed. This involves comparing the real-time fault snapshot with the normal baseline snapshot from multiple dimensions, including component dependency difference analysis, version compatibility analysis, call chain anomaly analysis, and resource association analysis. Specifically, this identifies newly added or missing dependent components and changes in the dependency chain; identifies version conflicts using version compatibility rules; locates timeout and error nodes in the call chain and identifies fault propagation paths using dependency graphs; and associates processor and memory resource usage data with dependent components to pinpoint the root cause of resource leak-type faults. After the comparison is complete, a root cause report is output using a network framework-specific fault rule library and intelligent diagnostics. This library includes dependency conflict rules from NuGet Package Manager and assembly loading anomaly rules. A large language model is used to intelligently analyze the comparison results, outputting a structured fault root cause report.Next, based on the root cause localization results, an automated isolation strategy is matched and executed. The corresponding target isolation strategy is matched against a pre-defined isolation strategy library, and automated isolation operations are performed. If the root cause is a single component failure, isolation is achieved by dynamically uninstalling the component or replacing it with a compatible version. If the root cause is a cross-service dependency failure, traffic isolation of the failed service is achieved by calling service governance components such as Polly (a circuit breaker degradation component) and YARP (Yet Another Reverse Proxy) for routing and forwarding. If the root cause is a resource leak-related failure, component-level resource reclamation or service instance restart isolation is triggered. Finally, snapshot data self-learning optimization is performed, collecting full-process data from each fault localization and isolation process, updating the dynamic adjustment rules of the normal baseline snapshot, and optimizing the network framework-specific fault rule library to improve the accuracy of subsequent fault localization and the adaptability of isolation strategies. The system architecture corresponding to this method includes a runtime dependency acquisition agent, a snapshot storage database, a snapshot comparison and analysis engine, a root cause intelligent diagnosis module, an automated isolation execution module, and a self-learning optimization module. These modules communicate through a RESTful API or message queue, supporting distributed deployment and horizontal scaling. This solution achieves multiple technical benefits: First, it can accurately locate network framework service faults. Through comparative analysis of runtime dependency snapshots, it can quickly locate faults caused by specific scenarios such as dependency conflicts, version incompatibility, and cross-service call anomalies, improving the accuracy by more than 60% compared to existing solutions. Second, it achieves automated fault isolation, linking fault root cause location with isolation strategies. Fault isolation can be completed without manual intervention, reducing the mean time to repair (MTTR) by more than 70%. Third, it adapts to the characteristics of the network framework ecosystem, performing specific optimizations for its dependency management mechanism and runtime characteristics, resulting in a significantly better ability to identify service-specific fault types than general fault location solutions. Fourth, it possesses dynamic baseline optimization capabilities, continuously optimizing normal baseline snapshots and fault rule bases through a self-learning mechanism, improving the system's adaptability to dynamic service changes. Fifth, it adopts a low-intrusive data collection method. The data collection agent uses a method of hooking runtime events, which does not require modification of business code and has less than 5% impact on service performance, making it suitable for production environment deployment.
[0065] Second Embodiment This embodiment provides an exemplary scheme for constructing a normal baseline topology snapshot. In this example, a collection agent is first deployed on each service node of the runtime framework, and a runtime dependency data collection channel is activated in conjunction with dependency association events. Then, the component dependency topology, version information, and resource usage data are obtained through the collection channel to form the original runtime dependency data stream. Subsequently, a stable running phase is determined according to service health status indicators, and the dependency data stream is periodically aggregated to generate candidate baseline topology snapshots. Finally, a standard normal baseline topology snapshot is obtained through consistency verification and screening. Step S10 includes steps A11 to A14: Step A11: Based on the deployment of the collection agent on each service node of the running framework, and combined with the dependency association event, activate the collection channel for runtime dependency data.
[0066] Step A12: Through the acquisition channel, determine the dependency topology data, component version information, and resource usage data of the internal components of the service to form the original runtime dependency data stream.
[0067] Step A13: Determine the stable operation phase according to the service health status indicators, and periodically aggregate the dependent data streams during the original runtime to generate candidate baseline topology snapshots within the stable operation phase.
[0068] Step A14: Verify the consistency of the candidate baseline topology snapshots, and filter the candidate baseline topology snapshots that cover all service nodes and have dependencies, and use them as the normal baseline topology snapshots.
[0069] Dependency-related events are native runtime events that reflect the establishment, modification, and resolution of component dependencies during the runtime framework's operation. They are the signal source that triggers dependency data collection. Examples include assembly load events, dependency injection container object creation events, cross-service call request events, and component version change events. Collection channels are dedicated data collection links established between collection agents and runtime events for continuously acquiring dependency data; they are the carriers of raw dependency data transmission. Examples include assembly load event hooking channels, dependency injection object creation interception channels, cross-service call interception channels, and resource state sampling channels. The raw runtime dependency data stream is a sequence of unaggregated, native runtime dependency and state data continuously output by the collection channels, forming the raw data foundation for building baseline snapshots. Examples include sequential component call records, assembly load logs, component version identifiers, real-time resource usage samples, and cross-service call chain records.
[0070] Service health status metrics are a set of core operational indicators used to quantitatively assess service stability and determine whether it is in a normal steady state; they are the basis for determining stable operation. Examples include business request success rate, average response latency, error rate volatility, CPU load volatility, and memory usage volatility. Candidate baseline topology snapshots are aggregated versions of dependency topology snapshots generated during the stable operation phase, awaiting verification; they are alternative data objects for normal baseline topology snapshots. Examples include single-period aggregated snapshots, multi-period average snapshots, and partition aggregated snapshots. Consistency verification is a process used to verify the node coverage integrity, dependency stability, and data validity of candidate baseline topology snapshots; it is the core step in selecting qualified baseline snapshots. Examples include node coverage verification, dependency overlap verification, data integrity verification, and resource value volatility verification.
[0071] In this example, when activating the runtime dependency data collection channel by combining dependency-related events, all types of dependency-related events can be bound at once using a unified event hooking method to establish a full-dimensional collection channel and achieve synchronous collection of all dependency data. Alternatively, by using a segmented event on-demand mounting method, corresponding types of dependency-related events can be selectively mounted according to the business type and detection requirements of the service node to establish a lightweight targeted collection channel, thereby completing the activation configuration of the collection channel.
[0072] After activating the data collection channels, a continuous data collection process for raw runtime dependencies is initiated. Each collection channel captures component dependency topology data, component version information, and resource usage data. The multi-channel data is then aligned with a unified timestamp to form a continuous raw runtime dependency data stream. Subsequently, the service's global operational status is continuously monitored based on service health indicators to determine if it has entered a stable operation phase. Within the confirmed stable operation phase, the raw dependency data stream is aggregated at fixed intervals to generate candidate baseline topology snapshots for the corresponding period. Finally, consistency checks are performed on all candidate baseline topology snapshots, and snapshots that meet coverage and stability requirements are selected as the official normal baseline topology snapshots. This steady-state pre-judgment and multi-level consistency checks ensure the baseline accuracy and global representativeness of the snapshots, preventing deviations in the baseline due to instantaneous service fluctuations.
[0073] For example, there are two ways to obtain normal baseline topology snapshots through candidate baseline topology snapshot filtering. The first is a progressive filtering method based on sequential verification over a period of time. Following the generation sequence of candidate baseline topology snapshots, consistency checks are performed sequentially, starting with the first candidate snapshot of the period. After each candidate snapshot is verified, it is simultaneously determined whether the snapshot meets three requirements: full node coverage, stable dependencies, and acceptable resource fluctuations. If the requirements are met, the snapshot is included in the qualified candidate set, and the number of consecutive qualified snapshots is continuously accumulated. If the requirements are not met, the current consecutive qualified accumulation is terminated, the accumulated count is reset, and the verification of candidate snapshots for the next period begins. After all candidate snapshots for all periods have been verified, the group of candidate snapshots with the longest consecutive qualified duration is selected, and the snapshot of the middle period is taken as the final normal baseline topology snapshot. This method uses the logic of sequential verification and progressive filtering based on consecutive qualifications. Through progressive verification over a period of time and continuous steady-state verification, it ensures that the selected baseline snapshot corresponds to a truly continuous and stable operating phase, avoiding benchmark deviations caused by instantaneous steady-state conditions.
[0074] The second method is multi-dimensional clustering batch verification and screening. A full read and multi-dimensional feature extraction are performed on all candidate baseline topology snapshots. Clustering is performed based on three dimensions: node coverage, dependency overlap, and resource stability. Candidate snapshots with features in the same stable range and temporally consecutive are grouped into a single steady-state snapshot cluster, and fluctuation boundary nodes between clusters are marked, generating multiple steady-state snapshot clusters that are computationally independent of each other. Parallel verification is then initiated simultaneously for all the divided steady-state snapshot clusters. Within each steady-state snapshot cluster, the consistency index of all candidate snapshots within the cluster is verified one by one, and it is verified whether all snapshots within the cluster fully meet the preset stability criteria. The data coverage range within the snapshot cluster that meets the overall cluster stability requirements is statistically analyzed. After all parallel verification and statistics for all steady-state snapshot clusters are completed, the median snapshot of the snapshot cluster with the longest coverage period and the highest stability is selected as the normal baseline topology snapshot. This method employs multi-dimensional feature clustering and full-cluster parallel verification computation logic. By pre-emptively clustering steady-state features, the stable operating range of the service is locked in advance, reducing the number of invalid single-snapshot verifications and improving the overall efficiency of baseline snapshot screening.
[0075] Furthermore, in the distributed cluster scenario of the network service operation framework in the e-commerce business domain, a lightweight runtime dependency collection agent is deployed on all service nodes of the operation framework. This agent, combined with three core dependency-related events—assembly loading events, dependency injection (DI) container object creation events, and cross-service call interceptors—activates a full-dimensional runtime dependency data collection channel covering component calls, version configurations, and resource status. This channel collects and determines in real-time the dependency topology data, component version information, and resource usage data in terms of processors and memory. After being aligned with a unified timestamp, a time-series continuous raw runtime dependency data stream is formed. Service operation phases are continuously determined based on service health indicators such as business request success rate, average response latency, and error rate volatility. Within the identified stable operation phase, the raw runtime dependency data stream undergoes fixed-period structured aggregation processing to generate corresponding periodic candidate baseline topology snapshots. The consistency of all candidate baseline topology snapshots is verified by three rules: node coverage integrity, dependency overlap, and resource indicator fluctuation. Candidate baseline topology snapshots that cover all service nodes and have stable dependencies are selected as the benchmark normal baseline topology snapshots for subsequent fault comparison and diagnosis.
[0076] Third Embodiment This embodiment provides an exemplary scheme for generating fault-dependent topology snapshots. In this example, the detected service anomaly signals are first classified and judged according to anomaly type, duration, and impact range, and false alarms are filtered to obtain the target anomaly signal. Then, in response to the target anomaly signal, a data acquisition agent is invoked to obtain multi-dimensional dependency data at the time of the fault and associated with nodes to obtain a fault data stream. Finally, the fault data stream is subjected to structured aggregation and associated with the corresponding normal baseline snapshot for storage, thereby generating a fault-dependent topology snapshot that is accurately aligned with the time of the fault. Step S20 includes steps B11 to B13: Step B11: When a service anomaly is detected, the anomaly signal is classified and judged according to its anomaly type, duration, and impact range. False alarm signals are filtered out to obtain the target anomaly signal.
[0077] Step B12: In response to the target abnormal signal, call the collection agent of the service node to obtain the dependent topology data, component version information and resource usage data at the time of the fault, and associate the corresponding service node to obtain the fault data stream.
[0078] Step B13: Perform structured aggregation on the fault data stream to generate the fault-dependent topology snapshot, and store it in association with the corresponding normal baseline topology snapshot.
[0079] The target anomaly signal is a control signal that, after being graded and filtered for false alarms, is confirmed to be valid and used to formally trigger fault snapshot collection. It serves as the triggering basis for initiating fault data collection. Examples include Level 1 severe anomaly signals, Level 2 warning anomaly signals, and Level 3 alert anomaly signals. The fault data stream is a collection of original dependencies and status data reported by each service node's collection agent, carrying node identifiers and timestamps at the time of the fault. It is the data source for generating fault dependency topology snapshots. Examples include abnormal node component call records, fault-time assembly version data, node resource usage sampling values, and cross-service call anomaly link data. Structured aggregation is a process that aligns, deduplicates, and correlates the scattered original fault data from multiple nodes according to a unified topology structure. It is the core processing step in forming standardized fault snapshots. Examples include node data timestamp alignment, dependency link deduplication and correlation, component version information integration, and resource data attribute mounting. Associated storage is a storage method that establishes a mapping and binding relationship between the generated fault dependency topology snapshot and the corresponding version of the normal baseline topology snapshot before storing it in the database. It is the foundation for ensuring data consistency in subsequent comparative analysis. Examples include binding snapshot pairs by fault number, associating baseline versions by timestamp, and mapping snapshot data by node dimension.
[0080] In this example, when performing a graded judgment on abnormal signals to filter false alarms and obtain the target abnormal signal, a multi-dimensional feature weighted judgment method can be used. This method comprehensively assigns corresponding weights to three core features: abnormal type, duration, and impact range. Weighted scoring is then used to achieve abnormal grading and false alarm filtering. Alternatively, a historical abnormal pattern matching and verification method can be used to compare the characteristics of the current abnormal signal with historically diagnosed false alarm patterns, eliminating false alarm signals that match the characteristics of instantaneous jitter or indicator spikes, thereby completing the generation of the target abnormal signal.
[0081] Upon receiving the target anomaly signal, the data collection process for the fault-dependent data is initiated. Collection instructions are sent to the collection agents of the corresponding service nodes. Each collection agent extracts the dependent topology data, component version information, and resource usage data at the time of the fault. The data is then bound and associated with the identifier of its respective service node before being reported. The data is aggregated to form a fault data stream. Subsequently, the fault data stream is processed by timestamp alignment, invalid data removal, and structured aggregation to generate a standard format fault dependent topology snapshot. At the same time, a mapping relationship is established between the fault snapshot and the corresponding version of the normal baseline topology snapshot before associated storage. In this way, through hierarchical triggering and targeted collection, the time accuracy and data integrity of the fault snapshot are ensured, and invalid collection is avoided from consuming system resources.
[0082] For example, there are two ways to generate fault-dependent topology snapshots by aggregating fault data streams. The first is sequential node aggregation. Following the priority order of nodes affected by the fault, starting with the core business node, the original fault data of each node is read sequentially, and data verification, topology mapping, and attribute mounting are performed node by node. After data aggregation of each node is completed, the topology nodes and links of that node are simultaneously integrated into the global fault topology structure, and the node coverage and link completeness of the global snapshot are continuously updated. If a node with missing or abnormal data is encountered, it is marked as a node to be completed, the current node is skipped, and the aggregation process for the next node begins. After all target nodes have completed sequential aggregation, the default placeholder data of the marked nodes is completed, ultimately generating a complete fault-dependent topology snapshot. This method uses a processing logic of priority sequential aggregation and incremental topology fusion, ensuring that the core business links complete snapshot construction first through priority aggregation of core nodes and gradual construction of the global topology.
[0083] The second method is partitioned parallel batch aggregation. All target service nodes are partitioned according to business domain or cluster partitions. Based on the partition to which a node belongs, the fault data stream is split into multiple independent partition data subsets, and the cross-regional call link boundaries between partitions are marked, generating multiple partition aggregation tasks without computational dependencies. Then, parallel processing is simultaneously initiated for all partition aggregation tasks. Within each partition, node data verification, dependency topology construction, and attribute information mounting are independently completed, generating a partition-level fault topology snapshot. After all partition parallel aggregation processing is completed, topology splicing and link association between partitions are performed based on the cross-regional call link boundaries, summarizing to generate a global fault dependency topology snapshot. This method adopts the processing logic of business partition task splitting and full partition parallel aggregation. Through the pre-partition task splitting, the global aggregation is decomposed into multiple sets of parallel subtasks, reducing the computational pressure of single-node aggregation and improving the generation efficiency of fault snapshots in large-scale clusters.
[0084] Furthermore, in a distributed microservice cluster scenario within a network service runtime framework, when the alarm system detects service anomalies such as service response timeouts or spikes in error rates, it performs multi-dimensional hierarchical judgment based on the anomaly type, duration, and coverage of affected service nodes. After filtering false alarms using a historical instantaneous jitter pattern library, it obtains the valid target anomaly signal. Responding to this target anomaly signal, it invokes the runtime dependency acquisition agent for each service node within the corresponding affected area to accurately retrieve the service's internal component dependency topology data, assembly version information, and CPU (Central Processing Unit) and memory resource usage data at the time of the fault trigger. This data is then bound and associated with the corresponding service node identifier to obtain a time-aligned fault data stream. The fault data stream undergoes invalid data removal, dependency link deduplication, and topology-structured aggregation to generate a standard-format fault dependency topology snapshot. This snapshot is then uniquely mapped to the corresponding normal baseline topology snapshot of the same version and stored accordingly, providing precisely aligned paired benchmark data for subsequent multi-dimensional differential comparisons and intelligent root cause diagnosis.
[0085] Fourth embodiment This embodiment provides an exemplary scheme for anomaly signal classification and target anomaly signal generation. In this example, multi-dimensional features of the anomaly signal are first extracted and fused with vectorized encoding to obtain anomaly feature vectors. Then, these feature vectors are matched with anomaly threshold libraries to complete a preliminary three-level anomaly classification. Subsequently, a correlation check is performed using a historical anomaly pattern library to filter false alarms and correct the anomaly level. Finally, the target anomaly signal is generated based on the corrected anomaly level and the corresponding snapshot acquisition strategy. Please refer to... Figure 3 , Figure 3 This is a flowchart illustrating the fourth embodiment of the method for handling service failures within the runtime framework of this application. Step B11 includes steps C11 to C14: Step C11: Based on the abnormality type, duration, and impact range of the abnormal signal, perform multi-dimensional feature fusion and vectorization encoding to obtain the abnormal feature vector.
[0086] Step C12: Match the abnormal feature vector with the abnormal threshold library, and classify the target anomalies corresponding to the abnormal feature vector into Level 1 severe anomalies, Level 2 warning anomalies, and Level 3 prompt anomalies according to the scope of influence and severity.
[0087] Step C13: Verify the correlation between the initially classified anomaly levels and the historical anomaly pattern library, determine whether the target anomaly is a known jitter or instantaneous fluctuation, filter false alarm signals and correct the anomaly level.
[0088] Step C14: Generate the target anomaly signal based on the corrected anomaly level and its corresponding snapshot acquisition strategy.
[0089] Anomaly feature vectors are structured feature sets formed by standardizing and encoding the multi-dimensional attribute features of anomaly signals. They are the core input data for anomaly level determination and false alarm verification. Examples include anomaly type encoding values, anomaly duration values, the number of affected service nodes, error rate variation, and resource usage deviation. Anomaly threshold library is a structured parameter library pre-stored with boundary conditions and grading rules for anomaly determination at each level. It serves as the basis for initial anomaly grading. Examples include threshold ranges for Level 1 severe anomalies, Level 2 warning anomalies, Level 3 alert anomalies, and impact range grading thresholds. Historical anomaly pattern library is a feature knowledge base storing historical confirmed fault cases, known false alarm cases, and instantaneous fluctuation feature patterns. It serves as the verification basis for false alarm filtering and level correction. Examples include instantaneous fluctuation patterns during business peaks, network jitter false alarm patterns, instantaneous anomaly patterns during assembly loading, and historical confirmed fault feature templates. Snapshot collection strategies are a set of fault data collection ranges, collection granularities, and execution rules corresponding to different anomaly levels. They are the supporting execution rules for generating target anomaly signals. For example, there are strategies for collecting all data from all nodes in Level 1 anomalies, targeted data collection from related links in Level 2 anomalies, and simplified data collection from single nodes in Level 3 anomalies.
[0090] In this example, when performing multi-dimensional feature fusion and vectorization encoding on anomaly signals to obtain anomaly feature vectors, a simultaneous full-dimensional encoding approach can be used. This involves extracting features from all dimensions, such as anomaly type, duration, and impact range, all at once, and simultaneously performing standardization, normalization, and numerical encoding to generate a complete anomaly feature vector. Alternatively, a progressive dimensional encoding approach can be used. First, the anomaly type is extracted to complete the basic classification encoding. Then, the duration and impact range features are sequentially superimposed to gradually complete the feature dimensions, accumulating to generate a complete anomaly feature vector, thus completing the vectorization processing of anomaly features.
[0091] After obtaining the anomaly feature vector, a preliminary anomaly classification process is initiated. The anomaly feature vector is matched against thresholds at various levels in the anomaly threshold library. Based on a comprehensive judgment of the impact range and severity, the target anomaly is classified into three levels: Level 1 severe anomaly, Level 2 warning anomaly, and Level 3 alert anomaly. Subsequently, the preliminary anomaly levels are compared with a historical anomaly pattern library to verify their correlation. The similarity between the current anomaly features and known jitter and instantaneous fluctuation patterns is compared to determine whether the target anomaly is a false alarm. False alarm signals are filtered, and anomalies with inaccurate classification are corrected. Finally, the corrected anomaly level is matched with the corresponding snapshot acquisition strategy to generate a target anomaly signal carrying a level identifier and acquisition rules. This dual mechanism of multi-dimensional feature quantification classification combined with historical pattern verification improves the accuracy of anomaly judgment, reduces the probability of false alarms caused by instantaneous fluctuations, and avoids invalid acquisition consuming system resources.
[0092] For example, there are two implementation methods for anomaly classification and false alarm verification. The first is a time-sequential progressive serial classification verification. Following the temporal order of feature encoding, preliminary classification, pattern verification, and level correction, the judgment process is executed step-by-step sequentially, starting from the generation of the anomaly feature vector. After each judgment step is completed, the intermediate result of that step is synchronously output and passed to the next stage. If the preliminary classification stage determines it to be a level three warning anomaly, it directly enters the pattern verification stage. If the pattern verification stage determines it to be a known false alarm pattern, the judgment process is terminated directly and the anomaly signal is filtered. If the verification confirms the anomaly is valid and the level is accurate, it enters the final signal generation stage. After all the classification verification processes are completed, the corrected anomaly level and corresponding acquisition strategy are output, generating the target anomaly signal. This method uses a time-locked, step-by-step serial progressive judgment logic with breakpoint-based dynamic termination. By adhering to the progressive advancement of the judgment process and early termination for invalid anomalies, unnecessary subsequent computational overhead is reduced, ensuring the execution efficiency of anomaly judgment.
[0093] The second approach is a multi-dimensional joint parallel hierarchical verification. It performs full-dimensional feature parsing and splitting on the anomaly feature vector, simultaneously splitting it into threshold matching feature subsets and pattern verification feature subsets, generating independent hierarchical judgment tasks and pattern verification tasks. Simultaneously, two parallel tasks are initiated: preliminary hierarchical calculation and historical pattern correlation verification. The hierarchical task, based on the threshold matching feature subset and an anomaly threshold library, completes the preliminary classification of three anomaly levels. The verification task, based on the pattern verification feature subset and a historical anomaly pattern library, completes false alarm pattern matching and similarity calculation. After both parallel tasks are completed, the preliminary hierarchical results and pattern verification results are merged. False alarm signals are filtered, and anomalies with hierarchical deviations are corrected. Finally, the corrected results are used to match the acquisition strategy and generate the target anomaly signal. This method employs a multi-dimensional task splitting and dual-link parallel verification logic. The pre-emptive feature dimension splitting enables parallel execution of the hierarchical and verification stages, shortening the overall time consumption for anomaly judgment. Furthermore, the cross-validation of dual-dimensional results improves the accuracy of hierarchical judgment.
[0094] Furthermore, in the core transaction service cluster scenario of the network service operation framework, when the detection and alarm system detects a service anomaly and generates a raw anomaly signal, it performs multi-dimensional feature fusion and numerical normalization vectorization encoding based on the anomaly type, duration, and coverage of the affected service nodes of the anomaly signal, resulting in an anomaly feature vector containing multi-dimensional quantitative features. This anomaly feature vector is then matched item by item with a pre-built anomaly threshold library. Based on the breadth of the impact and the severity of business damage, the target anomaly corresponding to the anomaly feature vector is initially classified into three levels: Level 1 severe anomaly, Level 2 warning anomaly, and Level 3 alert anomaly. Subsequently, the initially classified anomaly levels are compared with a historical anomaly pattern library to perform feature correlation verification. Known feature patterns such as instantaneous transaction peak jitter, false network fluctuations, and instantaneous assembly loading anomalies are matched to determine whether the target anomaly is a known jitter or instantaneous fluctuation. Invalid false alarm signals are filtered out, and anomalies with biased level determinations are corrected. Finally, based on the corrected anomaly level, a corresponding differentiated snapshot collection strategy is matched to generate a target anomaly signal carrying an anomaly level identifier and collection rule parameters, used to accurately trigger the hierarchical collection of subsequent fault-dependent topology snapshots.
[0095] Fifth Embodiment This embodiment provides an exemplary scheme for intelligent diagnosis of fault root causes and generation of fault root cause reports. In this example, firstly, a difference dataset is extracted by multi-dimensional differential comparison between fault-dependent topology snapshots and normal baseline topology snapshots. Then, the difference data is matched with the fault rule base of the running framework to generate a set of candidate fault root causes with a comprehensive matching score. Subsequently, fault propagation path reasoning is performed based on the dependency topology graph to remove intermediate abnormal nodes and correct the root cause ranking. Finally, the root cause information, propagation path, and confidence score are integrated to obtain a structured fault root cause report. Step S30 includes steps D11~D14: Step D11: Perform a multi-dimensional differential comparison between the fault-dependent topology snapshot and the normal baseline topology snapshot to determine the difference dataset.
[0096] Step D12: Perform feature matching between each difference in the difference dataset and the fault rule base of the running framework to generate a set of candidate fault root causes with a comprehensive matching score.
[0097] Step D13: Based on the dependency topology graph, perform fault propagation path reasoning on the candidate fault root cause set, trace back the source component from the abnormal node along the dependency link, and remove intermediate abnormal nodes in the propagation process to obtain the fault root cause ranking result.
[0098] Step D14: Based on the fault root cause ranking results, integrate the root cause information, propagation path, and confidence score to obtain the fault root cause report.
[0099] The difference dataset is a structured collection of all deviations from the normal baseline, extracted after multi-dimensional comparison of the fault snapshot and the baseline snapshot. It serves as the foundational analysis object for fault root cause matching and diagnosis. Examples include dependency relationship differences, component version differences, call chain anomalies, and resource usage deviations. The candidate fault root cause set is a collection of multiple potential fault root causes, each carrying a matching score, generated after rule-based feature matching. It serves as a candidate object for propagation path reasoning and ranking optimization. Examples include component fault root cause candidates, dependency conflict root cause candidates, resource bottleneck root cause candidates, and version incompatibility root cause candidates. Fault propagation path reasoning is an analysis process based on the dependency topology graph, tracing the fault propagation chain backward from the abnormal nodes. It is used to distinguish between the fault source and intermediate propagation nodes and is a core step in correcting the root cause ranking. Examples include dependency chain reverse tracing, fault propagation path verification, and removal of intermediate abnormal nodes.
[0100] In this example, when performing multi-dimensional difference comparison between the fault snapshot and the baseline snapshot to determine the difference dataset, a full-dimensional item-by-item difference approach can be used. This involves sequentially comparing all fields across the three dimensions of dependency topology, version configuration, and resource status to fully extract all differences and form a complete difference dataset. Alternatively, a targeted difference approach can be used to locate pre-labeled abnormal nodes and related links in the fault snapshot. Then, difference comparison is performed only on the abnormal region and its upstream and downstream links to extract core differences and form a targeted difference dataset, thus completing the extraction of difference data.
[0101] After obtaining the difference dataset, a fault rule feature matching process is initiated. Each difference in the difference dataset is matched against the rule conditions in the fault rule library of the running framework. The feature matching degree of each rule is calculated, and a comprehensive matching score is calculated by combining historical similarity and influence range weights. A candidate fault root cause set with scores is generated based on the score ranking. Subsequently, fault propagation path reasoning is performed on the candidate fault root cause set based on the dependency topology graph. Starting from each abnormal node, the source component is traced back along the dependency call chain to verify the rationality of the propagation of each candidate root cause, eliminating intermediate abnormal nodes in the fault propagation process, and correcting the ranking results of the candidate root causes. Finally, based on the fault root cause ranking results, root cause information, propagation path, and confidence score are integrated to generate a structured fault root cause report. This dual diagnostic mechanism of rule matching combined with propagation path reasoning effectively distinguishes between fault sources and apparent nodes, significantly improving the accuracy of root cause localization and avoiding misjudging intermediate propagation nodes as fault root causes.
[0102] For example, there are two implementation methods for fault root cause diagnosis and report generation. The first is a link-progressive serial diagnostic reasoning. Following the temporal order of differential comparison, rule matching, path reasoning, and report generation, the diagnostic process is executed sequentially, starting from the extraction of the differential dataset. After each diagnostic step is completed, the intermediate results of that step are output synchronously and passed to the next step. If the rule matching step generates only one candidate root cause, the process directly proceeds to the path reasoning step. If the path reasoning step verifies that the candidate root cause lacks propagation rationality, the candidate is directly eliminated, and the verification process for the next lower priority candidate is initiated. After all diagnostic steps are completed, the final root cause ranking results, propagation paths, and confidence scores are integrated to generate a fault root cause report. This method employs a time-locked, step-by-step serial progression and candidate-by-candidate verification reasoning logic. By adhering to the progressive advancement of the diagnostic process and verifying each candidate root cause, the logical rigor of the root cause diagnosis is ensured.
[0103] The second approach is dimensional hierarchical parallel diagnostic reasoning. Dimensional splitting and feature hierarchical division are performed on the differential dataset, simultaneously dividing it into rule-matching feature subsets and path-reasoning feature subsets, generating independent rule-matching and propagation path reasoning tasks. Two parallel tasks are initiated simultaneously: rule-based feature matching computation and topology graph path tracing. The matching task uses the rule-matching feature subset and the fault rule base to match and score candidate root causes, while the reasoning task uses the path-reasoning feature subset and the dependent topology graph to reverse-trace the fault propagation link and mark intermediate nodes. After both parallel tasks are completed, the candidate root cause set and propagation path reasoning results are merged, intermediate abnormal nodes are removed, and the root cause ranking is corrected, finally generating a complete fault root cause report. This method employs dimensional hierarchical task splitting and dual-link parallel diagnostic reasoning logic. The pre-emptive feature dimensional splitting enables parallel execution of rule matching and path reasoning, shortening the overall time consumption of root cause diagnosis. Simultaneously, cross-validation of dual-dimensional results improves the accuracy of root cause localization.
[0104] For example, in the distributed microservice cluster verification test scenario of the network service runtime framework of the bank's core accounting system, testers pre-injected a NuGet PackageManager dependency version conflict fault into the core accounting component. The fault propagated to two front-end transaction service nodes, triggering a total of 12 abnormal alarms. The differences in dependency relationships, component version differences, call chain anomalies, etc., in the difference dataset obtained by differential comparison were matched with a pre-built runtime framework fault rule base for multi-dimensional feature matching. The rule base covers NuGet dependency conflict rules, assembly loading anomaly rules, resource leakage judgment rules, etc. Through the weighted calculation of feature matching degree, historical fault similarity, and impact scope weight, a set of candidate fault root causes with a comprehensive matching degree score was generated. In this test, a total of 5 candidate fault root causes were matched, with the initial highest comprehensive matching degree being 89 points. Based on the global dependency topology graph, fault propagation path reasoning is performed on the candidate root cause set. Starting from the two front-end transaction nodes where anomalies were detected, the fault source component is traced backward along the service call dependency chain. One intermediate routing node is identified as an intermediate abnormal node caused by fault propagation and is removed. The ranking priority of candidate root causes is then adjusted to obtain the final fault root cause ranking result. Based on the adjusted fault root cause ranking result, the root cause component identifier, fault type, complete fault propagation path, and comprehensive confidence score are integrated to generate a structured fault root cause report.
[0105] Sixth Embodiment This embodiment provides an exemplary scheme for fault rule feature matching and comprehensive matching score calculation. In this example, the differences in the difference dataset are first categorized by difference type and filtered against the fault rule library of the running framework to obtain a subset of candidate rules. Then, weights are assigned to the rule conditions based on core and auxiliary features. Subsequently, feature-by-feature comparisons are performed, and scores are awarded based on the degree of matching to obtain a feature matching score. Simultaneously, the weighted scores of historical fault similarity and impact range are calculated. These are then weighted and fused to obtain a comprehensive matching score. Finally, low-confidence candidates are filtered out using a matching score threshold to generate a set of candidate fault root causes with a comprehensive matching score. Please refer to... Figure 4 , Figure 4 This is a flowchart illustrating the sixth embodiment of the method for handling service failures within the runtime framework of this application. Step D11 includes steps E11 to E16: Step E11: Classify the differences in the difference dataset according to the difference type, filter them against the rule conditions in the fault rule base of the running framework, and determine the candidate rule subset corresponding to the current difference type.
[0106] Step E12: Based on the rule condition feature set of the candidate rule subset, assign weights to the core features and auxiliary features respectively to obtain the rule weights.
[0107] Step E13: Compare the feature items in the difference data with the rule condition features one by one, and score them according to the degree of feature matching. The feature matching score is the percentage of the sum of the weighted scores of the matched features divided by the total score of the rule condition features.
[0108] Step E14: Calculate the historical fault similarity score and the influence range weight score according to the candidate rules in the subset of candidate rules.
[0109] Step E15: The feature matching score, the historical fault similarity score, and the influence range weight score are weighted and fused according to preset weights to obtain the comprehensive matching score corresponding to the candidate rule.
[0110] Step E16: Filter low-confidence candidates based on the comprehensive matching score, retain candidate root causes of failure with scores higher than the matching threshold, and generate a set of candidate root causes of failure with the comprehensive matching score.
[0111] A candidate rule subset is a set of rules selected from the full-scale runtime framework's fault rule base based on the difference type, and is related to the current difference type. It represents the scope of feature matching and scoring calculations. Examples include dependency conflict rule subsets, version compatibility rule subsets, resource leak rule subsets, and assembly loading exception rule subsets. Rule weights are sets of weight coefficients assigned to the core and auxiliary features in the rule conditions, used to measure the contribution of different features to fault determination. Examples include the weight percentage of core features, the weight percentage of auxiliary features, and the weight coefficient of feature items. Feature matching degree tiered scoring is a scoring mechanism that divides the matching degree between difference features and rule condition features into different levels and corresponds to different scoring rules; it is the core method for calculating feature matching degree. Examples include full score for complete matches, proportional score for partial matches, and deduction of corresponding scores for conflicting features. Historical fault similarity scoring is a quantitative indicator that measures the similarity between the current difference feature pattern and the feature patterns of historically diagnosed fault cases, used to improve the empirical accuracy of fault determination. Examples include historical case feature overlap score and historical fault pattern matching score.
[0112] The impact scope weight score is a dynamically adjusted weight score based on the number of service nodes involved in the current anomaly and the degree of business impact, used to reflect the impact of the fault scale on root cause determination. Examples include node coverage weight and business damage weight. The comprehensive matching score is a comprehensive quantitative indicator obtained by fusing feature matching degree, historical fault similarity, and impact scope weight according to preset weights. It is the core criterion for measuring the credibility of candidate root causes. Examples include rule comprehensive matching degree percentage and candidate root cause confidence score. The matching degree threshold is a preset minimum comprehensive matching degree score used to filter qualified candidate root causes, used to filter out false positives with low confidence. Examples include first-level fault matching threshold, second-level fault matching threshold, and general qualified threshold.
[0113] In this example, when classifying the discrepancy data by type and filtering the rule base to obtain a subset of candidate rules, one approach is to iterate through the entire rule base and match each rule type against the current discrepancy type, filtering out all rules that match the type to form a subset of candidate rules. Alternatively, a quick mapping approach using type indexes can be used, based on a pre-established index mapping relationship between discrepancy types and rule categories, to directly locate the rule set of the corresponding type and quickly generate a subset of candidate rules, thus completing the initial screening of candidate rules.
[0114] After obtaining the subset of candidate rules, the calculation process for feature matching degree and comprehensive score is initiated. For each rule in the subset of candidate rules, the feature set of the rule conditions is first extracted, and weights are assigned to the core features and auxiliary features respectively to obtain the rule weight. Then, the feature items in the difference data are compared with the feature conditions one by one, and scores are calculated according to the degree of feature matching to obtain the feature matching degree score of the rule. At the same time, the historical fault similarity score and the influence range weight score corresponding to the rule are calculated separately. The three scores are weighted and fused according to the preset weights to obtain the comprehensive matching degree score. Finally, low-confidence candidates are filtered according to the preset matching degree threshold, and candidate fault root causes with scores higher than the threshold are retained to generate a set of candidate fault root causes with comprehensive matching degree scores. In this way, through the mechanism of multi-dimensional weighted scoring and threshold filtering, the accuracy of candidate root cause determination is improved and the interference of low-confidence candidates on subsequent diagnosis is reduced.
[0115] For example, the implementation of comprehensive matching score calculation and candidate root cause screening includes two methods. The first is single-rule serial item-by-item calculation. Following the priority order of the candidate rule subsets, starting with the highest priority rule, the feature set and weight configuration of each individual candidate rule are extracted sequentially. After completing the feature matching score calculation for each rule, the historical fault similarity score and influence range weight score of that rule are calculated simultaneously. Then, a weighted fusion is performed to obtain the comprehensive matching score of that rule, which is simultaneously compared with a matching score threshold. If it is lower than the threshold, it is directly eliminated; if it is higher than the threshold, it is included in the candidate fault root cause set. After all candidate rules have completed item-by-item calculation and screening, a final candidate fault root cause set with a comprehensive matching score is generated. This method uses a single-rule serial item-by-item calculation and real-time threshold filtering processing logic. Through full-dimensional calculation and real-time screening of each rule, low-confidence rules are eliminated in advance, reducing subsequent invalid calculations and ensuring the logical rigor of the score calculation.
[0116] The second approach is dimensional parallel batch computation. The scoring calculation task for all candidate rules is split dimensionally into three independent tasks: feature matching degree calculation, historical fault similarity calculation, and influence range weight calculation. These generate computationally independent parallel subtasks. Batch computation is initiated simultaneously across all three dimensions: feature matching degree calculation completes feature comparison and hierarchical scoring for all rules; historical similarity calculation completes feature matching calculation between all rules and historical cases; and influence range calculation completes range weight calculation for all rules. Once all three dimensions' batch computations are complete, a weighted fusion of the three scores is uniformly applied to all candidate rules to obtain a comprehensive matching degree score for each rule. Threshold filtering is then uniformly performed to generate a set of candidate fault root causes with a comprehensive matching degree score. This method employs dimensional task decomposition and full-dimensional parallel batch computation processing logic. By splitting the computational dimensions beforehand, it achieves parallel execution of multi-dimensional scoring, shortening the total computation time for scoring in multi-rule scenarios. Simultaneously, batch unified computation improves the consistency of scoring criteria.
[0117] For example, in a fault diagnosis test scenario for a microservice cluster within the network service runtime framework of a production execution system, testers pre-injected an assembly loading anomaly fault, obtaining 24 difference data items through snapshot differential comparison. The differences in this dataset were categorized into three types: dependency, version configuration, and resource status. These differences were then matched against the rule conditions in the runtime framework fault rule library, which contains 128 rules, to determine a subset of 32 candidate rules corresponding to the current difference type. Based on the feature set of each rule condition in the candidate rule subset, weights were assigned according to a 70% weighting for core features and a 30% weighting for auxiliary features, resulting in a rule weight configuration for each rule. The feature items in the difference data were compared one by one with the rule condition features, and a tiered scoring system was implemented, where complete matches received full marks, partial matches received marks proportionally, and conflicting features had their marks deducted. The feature matching score was calculated as the percentage of the sum of the weighted scores of the hit features divided by the total score of the rule condition features. In this test, the highest feature matching degree for a single rule reached 92%. For each candidate rule in the subset of candidate rules, a historical fault similarity score is calculated by matching it against the historical fault case library. Simultaneously, an impact range weight score is calculated by considering the impact range of the anomaly across 8 service nodes and 3 core production lines. These two scores are 87 and 85 respectively. The feature matching score, historical fault similarity score, and impact range weight score are then weighted and merged according to preset weights of 50%, 30%, and 20% to obtain a comprehensive matching score for each candidate rule, with the highest comprehensive matching score being 89.1. Low-confidence candidates are filtered based on a preset matching score threshold of 70, eliminating 11 candidate rules with scores below the threshold. Candidate fault root causes with scores above the matching score threshold are retained, ultimately generating a candidate fault root cause set containing 21 candidate root causes with comprehensive matching scores.
[0118] Seventh Embodiment This embodiment provides an exemplary scheme for automated fault isolation strategy matching and execution. In this example, the corresponding target isolation strategy is first matched in the isolation strategy library based on the root cause type and root cause component identifier in the fault root cause report. Then, the target isolation strategy is parameterized according to the root cause component identifier and the fault impact range to generate isolation execution instructions. Subsequently, the instructions are sent to the corresponding service nodes to execute the isolation operation and obtain the isolation execution status information. Finally, the isolation effect is verified according to the execution status. If the service returns to normal, the isolation is confirmed to be successful. If the isolation is invalid or causes a new anomaly, the strategy is rolled back and restored to the state before isolation. Step S40 includes steps F11 to F14: Step F11: Based on the root cause type and root cause component identifier in the root cause report, match the target isolation policy corresponding to the root cause type in the isolation policy library.
[0119] Step F12: Configure the target isolation strategy parameterized according to the root cause component identifier and the scope of the fault impact, and generate isolation execution instructions.
[0120] Step F13: Send the isolation execution command to the corresponding service node, perform the isolation operation of the service node according to the target isolation strategy, and obtain the isolation execution status information.
[0121] Step F14: Based on the isolation execution status information, if the service returns to normal, the isolation is confirmed to be successful; if the isolation is invalid or causes a new exception, the policy is rolled back to restore the state before isolation.
[0122] Isolation execution commands are structured control instructions carrying the isolation object, execution parameters, operation sequence, and verification rules. They are control signals that trigger service nodes to perform isolation operations. Examples include component unloading commands, traffic circuit breaker commands, resource reclamation commands, and service restart commands. Isolation execution status information is a collection of service running status, operation execution progress, and exception alarm information collected during and after the isolation operation. It serves as the basis for verifying the isolation effect. Examples include isolation operation execution progress, service request success rate, response latency metrics, number of exception alarms, and resource utilization levels. Policy rollback is a fault-tolerance mechanism that reverts all isolation operations and restores the service to its pre-isolation state when isolation operations are ineffective or trigger new exceptions. It is used to reduce the risk of secondary failures in isolation operations. Examples include component reload rollback, traffic routing recovery rollback, resource configuration restoration rollback, and service instance recovery rollback.
[0123] In this example, when matching the target isolation strategy based on the root cause type and root cause component identifier, the root cause can be precisely mapped, matching the corresponding standard isolation strategy in the isolation strategy library based on the root cause type and component identifier. Alternatively, a type generalization adaptation method can be used, where if no precise matching strategy is available, a general isolation strategy of the same type can be matched based on the root cause category, thus completing the initial matching of the target isolation strategy.
[0124] After obtaining the target isolation strategy, the strategy parameterization configuration process is initiated. The isolation object is located based on the root cause component identifier. The scope of isolation execution nodes, operational intensity, and execution sequence are adjusted according to the scope of the fault's impact, generating an isolation execution command with complete execution parameters. The isolation execution command is then sent to the corresponding service nodes, which complete the isolation operation according to the target isolation strategy's execution rules. Real-time collection of operation progress and service operation metrics yields isolation execution status information. Finally, the isolation effect is verified based on the isolation execution status information. If core business metrics return to normal and no new alarms are added, isolation is confirmed as successful. If metrics do not improve or new abnormal alarms are triggered, the strategy is rolled back, restoring the operating state to its pre-isolation state. This approach, through precise root cause matching combined with parameterized adaptation and rollback fault tolerance mechanisms, improves the scenario adaptability of the isolation strategy and reduces the risk of secondary faults caused by isolation operations.
[0125] For example, there are two ways to implement fault isolation strategy execution and effect verification. The first is a time-sequential sequential isolation execution. Following the sequence of strategy matching, parameter configuration, command issuance, isolation execution, and effect verification, the isolation process is executed sequentially, starting from the parsing of the root cause report. After each execution step is completed, the intermediate result of that step is output synchronously and passed to the next step. If a new abnormal alarm is detected in the isolation execution stage, a rollback process is directly triggered, terminating subsequent isolation operations and gradually restoring the state to its pre-isolation state. After all isolation processes are completed, the final isolation execution result and effect verification conclusion are output. This method uses a time-locked, step-by-step sequential execution logic with real-time rollback for abnormalities. By closely following the progressive execution process and responding to abnormalities in real time, it ensures the full controllability of the isolation operation.
[0126] The second method is partitioned collaborative parallel isolation execution. All service nodes within the fault's impact range are divided into business domains or cluster partitions. The global isolation task is broken down into multiple independent partition isolation subtasks, each independently completing policy adaptation, parameter configuration, and isolation execution. Simultaneously, parallel isolation execution is initiated across all partitions. Service nodes within each partition autonomously complete local isolation operations and report their execution status in real time. The central node collects the isolation execution status information from each partition and performs global effect verification. If some partitions fail to isolate or trigger an exception, only the policy rollback for that partition is triggered, without affecting the normal isolation execution of other partitions. After all partition isolation operations are completed, the global isolation execution results and effect verification conclusions are summarized. This method employs business partition task splitting and multi-partition parallel collaborative execution logic. By implementing parallel execution of multi-node isolation operations through pre-defined partition task division, it significantly shortens the total isolation time for large-scale faults. Simultaneously, the partition-level independent rollback mechanism improves the fault tolerance of isolation execution.
[0127] For example, in the network service runtime framework order microservice cluster fault isolation test under the e-commerce platform's peak promotion stress test scenario, testers pre-injected a NuGet package manager dependency version conflict fault into the core order calculation component, spreading the fault to three order service nodes, causing the order submission error rate to rise to 18%. Based on the dependency conflict root cause type and the root cause component identifier of the order calculation component in the fault root cause report, a compatible version hot replacement target isolation strategy corresponding to the root cause type was matched from the isolation strategy library, which includes three types of strategies: component replacement, traffic circuit breaking, and resource reclamation. According to the root cause component identifier and the impact range of the three service nodes and two business partitions affected by the fault, the target isolation strategy was parameterized with the execution node range, replacement version number, and traffic switching sequence, generating an isolation execution command carrying complete execution parameters. The isolation execution command was issued to the corresponding three service nodes, and the service node isolation operations of dynamically unloading the old version component, hot loading the compatible version component, and smoothly switching back traffic were executed in sequence according to the target isolation strategy. Service error rate, response latency, and component loading status data were collected in real time to obtain isolation execution status information. The effectiveness was verified based on the isolation execution status information. Test data showed that after isolation was implemented, the order submission error rate dropped to below 0.3% within 12 seconds, and core business indicators returned to normal, confirming successful isolation. If anomalies such as component loading failure or a continuously increasing error rate occur during the isolation process, the strategy rollback mechanism is automatically triggered to reload the original version of the components and restore the original traffic routing, returning to the running state before isolation.
[0128] Eighth embodiment This embodiment provides an exemplary scheme for self-learning optimization of the entire fault handling process. In this example, the entire fault handling process data is first structured and organized, then stored in a fault case library to generate fault case records. Next, the adjustment rules for the normal baseline topology snapshot are updated based on historical case data to achieve baseline adaptive service load changes. Subsequently, the diagnosed root cause is matched with the runtime framework fault rule library to adjust rule weights and add missing rules to complete the iterative update of the rule library. Finally, the applicable scenarios, priorities, and execution parameters of the isolation strategy library are optimized based on the isolation execution effect, comprehensively improving the adaptability and execution success rate of the isolation strategy. Please refer to... Figure 5 , Figure 5 This is a flowchart illustrating the eighth embodiment of the method for handling service failures within the runtime framework of this application. Following step S40, steps G11-G14 are also included: Step G11: After the entire process data of this fault handling is structured and organized, it is stored in the fault case library to generate fault case records.
[0129] Step G12: Update the adjustment rules of the normal baseline topology snapshot using historical case data in the fault case library, so that the normal baseline topology snapshot adapts to changes in service load.
[0130] Step G13: Match the confirmed root causes of the historical case data with the fault rule library of the operating framework, adjust the feature weights and scoring parameters of the successfully matched rules, and add corresponding fault rules for the unmatched fault modes to update the fault rule library of the operating framework.
[0131] Step G14: Based on the isolation execution status information, increase the applicable scenarios and priorities of successful isolation strategies, and adjust and downgrade the parameters of strategies that are ineffective or trigger rollbacks to improve the adaptability and success rate of isolation strategies.
[0132] The fault case library is a structured case knowledge base storing data from the entire fault handling process, diagnostic results, isolation execution effects, and optimization parameters. It serves as the data foundation for self-learning optimization. Examples include fault characteristic data, root cause diagnosis results, isolation strategy execution records, effect verification data, and optimization parameter records. Fault case records are standardized case entries generated after structuring and organizing the data from a single fault handling process; they are the smallest storage unit in the fault case library. Examples include fault number, fault type, root cause information, snapshot version, isolation strategy, execution effect, and optimization suggestions. The adjustment rules for normal baseline topology snapshots are dynamic configuration rules used to control the baseline snapshot update cycle, steady-state judgment threshold, and data aggregation granularity. They are the core basis for achieving baseline adaptation to service load changes. Examples include dynamic adjustment rules for steady-state judgment thresholds, adaptive rules for snapshot update cycles, and baseline correction rules for load fluctuations.
[0133] Updating the runtime framework's fault rule base involves iteratively optimizing existing rules by adjusting their feature weights and scoring parameters, and adding corresponding rules for new fault modes. This is a core step in improving the accuracy of the rule base's diagnostics. Examples include adjusting rule feature weights, correcting scoring parameters, adding new fault rules, and eliminating outdated rules. Isolation strategy adaptability optimization is a process of dynamically adjusting the applicable scenarios, priorities, and execution parameters of strategies based on their effectiveness. This improves the scenario matching and execution success rate of isolation strategies. Examples include expanding the applicable scenarios of strategies, prioritizing successful strategies, adjusting parameters for failed strategies, and downgrading inefficient strategies.
[0134] In this example, when structuring the data of the entire fault handling process to generate fault case records, a complete fault case record can be generated by collecting all data from the entire process—from anomaly detection, snapshot collection, root cause diagnosis to isolation execution and effect verification—in a full-process, full-volume archiving manner, and then standardizing and organizing the data according to a unified structure. Alternatively, a streamlined archiving method can be used, extracting core elements such as fault characteristics, root cause results, isolation strategies, and execution effects to generate lightweight fault case records, thereby completing the storage of fault cases in the database.
[0135] After generating fault case records and storing them in the fault case database, the baseline adjustment rule optimization process is initiated. Using historical case data from the database, the normal operation characteristics and fluctuation range under different load levels are analyzed. The steady-state judgment threshold, update cycle, and aggregation granularity of the normal baseline topology snapshot are updated to adapt to dynamic changes in service load. Simultaneously, the fault rule database iterative optimization process is initiated. The diagnosed root causes from historical case data are matched with the runtime framework fault rule database. For successfully matched rules, their feature weights and scoring parameters are adjusted. For new, unmatched fault modes, corresponding fault rules are added, completing the update of the runtime framework fault rule database. Finally, the isolation strategy optimization process is initiated. Based on isolation execution status information, the applicable scenario tags for successfully isolated strategies are expanded and their matching priority is increased. For strategies that are ineffective or trigger rollbacks, execution parameters are adjusted and priority is downgraded. This comprehensively improves the adaptability and success rate of isolation strategies. Through a data-driven self-learning mechanism, the baseline, rule database, and strategy database are continuously iterated and optimized, constantly improving the accuracy of fault diagnosis and the effectiveness of isolation handling.
[0136] For example, the implementation of self-learning optimization and iterative updates of each module includes two methods. The first is a sequential optimization update in a process-chain manner. Following the chronological order of case entry, baseline rule update, fault rule base update, and isolation strategy optimization, the optimization process is executed sequentially step by step, starting from the generation of fault case records. After each module's optimization update is completed, the update result of that module is output synchronously and passed to the next step. If a fault rule is added in the rule base update step, the correlation verification and adjustment of the baseline rule are triggered synchronously. If the strategy optimization step determines that the strategy is invalid, the weight correction of the corresponding rule is triggered synchronously. After all optimization steps are completed, the final update result and optimization effect statistics of the entire module are output. This method adopts a time-locked, step-by-step, sequential, and correlated adjustment optimization logic. By adhering to the progressive advancement of the optimization process and the correlation and linkage between modules, it ensures the logical consistency and parameter synergy of each module's optimization.
[0137] The second approach is modular decoupling and parallel optimization updates. The self-learning optimization task is decoupled and split into three independent modules: baseline rule optimization, fault rule base optimization, and isolation strategy optimization. Each module independently performs optimization calculations based on shared case data from the fault case base. Parallel optimization updates of the three modules are initiated simultaneously. The baseline rule module dynamically corrects baseline adjustment rules based on historical load case data; the rule base module adjusts rule weights and generates new rules based on confirmed root cause case data; and the strategy module optimizes strategy adaptability and adjusts priorities based on isolated execution case data. After all three modules have completed their parallel optimizations, a unified parameter consistency check is performed between modules to correct conflicting configuration parameters, and the optimization update results for all modules are output. This method employs modular decoupling and multi-module parallel optimization processing logic. By decoupling the modules upfront, parallel execution of each module's optimization is achieved, shortening the total optimization time for batch cases. Simultaneously, independent parallel computation enhances the professionalism of each module's optimization.
[0138] For example, in the monthly self-learning optimization test scenario of the distributed microservice cluster of the network service operation framework of the accounting system, a total of 32 service failure handling processes were triggered during the test period. The data of each failure handling process, from anomaly detection, snapshot collection, root cause diagnosis to isolation execution and effect verification, was standardized and structured and stored in the failure case library. This generated failure case records containing full-dimensional information including failure characteristics, root cause results, isolation strategies, and execution effects. A total of 32 valid failure case records were generated in this test. By analyzing the service load characteristics and normal fluctuation range during different peak transaction periods using historical case data in the failure case library, the steady-state judgment threshold, snapshot update cycle, and aggregation granularity adjustment rules of the normal baseline topology snapshot were updated, making the normal baseline topology snapshot adapt to the dynamic changes in service load. Test data showed that the load misjudgment rate of the optimized baseline snapshot decreased from 12% to 4%. The root causes of confirmed cases in historical case data were matched one by one with the runtime framework's fault rule base. The core feature weights and scoring parameters of the 26 successfully matched rules were adjusted. For the 6 new types of unmatched assembly loading anomaly fault modes, corresponding exclusive fault rules were added to complete the iterative update of the runtime framework's fault rule base. The root cause matching accuracy of the optimized rule base was significantly improved. Based on the execution status information of previous isolation attempts, the applicable business scenario tags of the 28 successfully isolated strategies were expanded and their matching priority was increased. The execution parameters of the 4 strategies that were ineffective in isolation or triggered rollbacks were adjusted and their priority downgraded to improve the adaptability and success rate of the isolation strategies.
[0139] This application provides a processing device for runtime framework service failures. The processing device for runtime framework service failures includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the runtime framework service failure processing method in the above embodiment 1.
[0140] The following is for reference. Figure 6 The diagram illustrates a structural schematic of a processing device suitable for implementing runtime service failures in embodiments of this application. The processing device for runtime service failures in embodiments of this application may include, but is not limited to, mobile terminals such as service circuit breaker / degradation devices, reverse proxy routing devices, managed runtime diagnostic devices, etc., as well as fixed terminals such as servers, compute nodes, server clusters, etc. Figure 6 The illustrated device for handling service failures within the runtime framework is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments described in this application.
[0141] like Figure 6 As shown, the processing device for handling framework service failures may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 1002 or a program loaded from storage device 1003 into random access memory (RAM) 1004. The random access memory 1004 also stores various programs and data required for the operation of the processing device for handling framework service failures. The processing unit 1001, ROM 1002, and RAM 1004 are interconnected via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to I / O interface 1006: input devices 1007 including, for example, touchscreens, touchpads, keyboards, mice, image sensors, microphones, accelerometers, gyroscopes, etc.; output devices 1008 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 1003 including, for example, magnetic tapes, hard disks, etc.; and communication devices 1009. Communication device 1009 allows the processing equipment for operating framework service faults to communicate wirelessly or wiredly with other devices to exchange data. Although the figure shows a processing equipment for operating framework service faults with various systems, it should be understood that it is not required to implement or possess all the systems shown. More or fewer systems can be implemented alternatively.
[0142] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from read-only memory 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.
[0143] The runtime framework service failure processing device provided in this application, employing the runtime framework service failure processing method described in the above embodiments, can solve the technical problem of low failure processing efficiency. Compared with the prior art, the beneficial effects of the runtime framework service failure processing device provided in this application are the same as those of the runtime framework service failure processing method provided in the above embodiments, and other technical features in the runtime framework service failure processing device are the same as those disclosed in the method of the previous embodiment, and will not be repeated here.
[0144] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.
[0145] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.
[0146] This application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, the computer-readable program instructions being used to execute the method for handling service failures of the runtime framework in the above embodiments.
[0147] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, radio frequency (RF), etc., or any suitable combination thereof.
[0148] The aforementioned computer-readable storage medium may be included in a processing device for runtime framework service failures; or it may exist independently and not assembled into a processing device for runtime framework service failures.
[0149] The aforementioned computer-readable storage medium carries one or more programs. When these programs are executed by the processing device for runtime framework service failures, the processing device for runtime framework service failures performs the following actions: Deploying acquisition agents at each service node of the runtime framework to determine a normal baseline topology snapshot during stable operation; when a service anomaly is detected, triggering the acquisition of fault-dependent data with an anomaly signal to obtain a fault-dependent topology snapshot; comparing the fault-dependent topology snapshot with the normal baseline topology snapshot, and filtering using the runtime framework fault rule base to generate a fault root cause report; and matching and executing a target isolation policy in the isolation policy base based on the fault root cause report to process the corresponding service failure through the target isolation policy.
[0150] Computer program code for performing the operations of this application can be written in one or more programming languages or a combination thereof, including object-oriented programming languages such as Java, Smalltalk, and C++, as well as conventional procedural programming languages such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).
[0151] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation that may be implemented in systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.
[0152] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.
[0153] The readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., a computer program) for executing the above-described method for handling runtime framework service failures, thereby solving the technical problem of low failure handling efficiency. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as the beneficial effects of the runtime framework service failure handling method provided in the above embodiments, and will not be repeated here.
[0154] The above description is only a part of the embodiments of this application and does not limit the patent scope of this application. All equivalent structural transformations made under the technical concept of this application and using the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included in the patent protection scope of this application.
Claims
1. A method for handling service failures in a runtime framework, characterized in that, The method includes: By deploying collection agents on each service node of the runtime framework, normal baseline topology snapshots are determined during the stable operation phase. When a service anomaly is detected, the anomaly signal triggers the acquisition of fault-dependent data to obtain a fault-dependent topology snapshot. By comparing the fault-dependent topology snapshot with the normal baseline topology snapshot and combining the filtering of the runtime framework fault rule base, a fault root cause report is generated. Based on the root cause report, a target isolation policy is matched in the isolation policy library and executed to handle the corresponding service failure through the target isolation policy.
2. The method for handling runtime framework service failures as described in claim 1, characterized in that, The step of determining a normal baseline topology snapshot during stable operation by deploying a collection agent on each service node of the runtime framework includes: Based on the deployment of the collection agent on each service node of the runtime framework, and combined with dependency-related events, the collection channel for runtime dependency data is activated. Through the acquisition channel, the dependency topology data, component version information, and resource consumption data of the internal components of the service are determined, forming the original runtime dependency data stream; The stable operation phase is determined according to the service health status indicators. During the stable operation phase, the dependent data streams in the original runtime are periodically aggregated to generate candidate baseline topology snapshots. Verify the consistency of the candidate baseline topology snapshots, and filter out the candidate baseline topology snapshots that cover all service nodes and have dependencies, and use them as the normal baseline topology snapshots.
3. The method for handling runtime framework service failures as described in claim 1, characterized in that, The step of obtaining a fault-dependent topology snapshot by triggering the acquisition of fault-dependent data with an abnormal signal when a service anomaly is detected includes: When a service anomaly is detected, the anomaly signal is classified and judged according to its anomaly type, duration, and impact range. False alarm signals are filtered out to obtain the target anomaly signal. In response to the target abnormal signal, the collection agent of the service node is invoked to obtain the dependency topology data, component version information and resource consumption data at the time of the failure, and the corresponding service node is associated to obtain the fault data stream; The fault data stream is structured and aggregated to generate the fault-dependent topology snapshot, which is then associated and stored with the corresponding normal baseline topology snapshot.
4. The method for handling runtime framework service failures as described in claim 3, characterized in that, The step of classifying and judging the abnormal signal according to its abnormality type, duration, and impact range when a service abnormality is detected, and filtering out false alarm signals to obtain the target abnormal signal includes: Based on the anomaly type, duration, and impact range of the abnormal signal, multi-dimensional feature fusion and vectorized encoding processing are performed to obtain an anomaly feature vector; The abnormal feature vectors are matched with the abnormal threshold library, and the target anomalies corresponding to the abnormal feature vectors are classified into Level 1 severe anomalies, Level 2 warning anomalies, and Level 3 prompt anomalies according to the scope of influence and severity. The correlation between the initially classified anomaly levels and the historical anomaly pattern library is verified to determine whether the target anomaly is a known jitter or instantaneous fluctuation, and false alarm signals are filtered out and the anomaly level is corrected. The target anomaly signal is generated based on the revised anomaly level and its corresponding snapshot acquisition strategy.
5. The method for handling runtime framework service failures as described in claim 1, characterized in that, The step of comparing the fault-dependent topology snapshot with the normal baseline topology snapshot, and combining the filtering from the runtime framework fault rule base to generate a fault root cause report includes: The fault-dependent topology snapshot is compared with the normal baseline topology snapshot in a multi-dimensional difference to determine the difference dataset; The differences in the difference dataset are matched with the fault rule base of the running framework to generate a set of candidate fault root causes with a comprehensive matching score. Based on the dependency topology graph, the candidate fault root cause set is inferred to infer the fault propagation path. The source component is traced back from the abnormal node along the dependency link, and intermediate abnormal nodes in the propagation process are removed to obtain the fault root cause ranking result. Based on the failure root cause ranking results, the root cause information, propagation path, and confidence score are integrated to obtain the failure root cause report.
6. The method for handling runtime framework service failures as described in claim 5, characterized in that, The step of performing feature matching between each difference in the difference dataset and the fault rule base of the running framework to generate a candidate fault root cause set with a comprehensive matching score includes: The differences in the difference dataset are classified according to their difference types, and then filtered against the rule conditions in the fault rule base of the runtime framework to determine the candidate rule subset corresponding to the current difference type. Based on the set of rule condition features of the candidate rule subset, weights are assigned to the core features and auxiliary features respectively to obtain the rule weights; The feature items in the difference data are compared one by one with the rule condition features, and scores are assigned according to the degree of feature matching. The feature matching score is the percentage of the sum of the weighted scores of the matched features divided by the total score of the rule condition features. Based on the candidate rules in the subset of candidate rules, calculate the historical fault similarity score and the impact range weight score respectively; The feature matching score, the historical fault similarity score, and the influence range weight score are weighted and fused according to preset weights to obtain the comprehensive matching score corresponding to the candidate rule. Based on the comprehensive matching score, low-confidence candidates are filtered out, and candidate root causes of failure with scores higher than the matching threshold are retained, generating a set of candidate root causes of failure with the comprehensive matching score.
7. The method for handling runtime framework service failures as described in claim 1, characterized in that, The step of matching a target isolation policy in the isolation policy library and executing it based on the root cause report to handle the corresponding service failure through the target isolation policy includes: Based on the root cause type and root cause component identifier in the fault root cause report, a target isolation policy corresponding to the root cause type is matched in the isolation policy library. The target isolation strategy is parameterized based on the root cause component identifier and the scope of the fault impact, and an isolation execution instruction is generated. The isolation execution command is sent to the corresponding service node, the service node is isolated according to the target isolation strategy, and the isolation execution status information is obtained; Based on the isolation execution status information, if the service returns to normal, the isolation is confirmed to be successful; if the isolation is ineffective or causes a new exception, the policy is rolled back to the state before isolation.
8. The method for handling runtime framework service failures as described in claim 1, characterized in that, After the step of matching and executing a target isolation policy in the isolation policy library based on the root cause report of the fault to handle the corresponding service fault through the target isolation policy, the method for handling service faults of the runtime framework further includes: The entire process data of this fault handling was structured and stored in the fault case library to generate fault case records; The adjustment rules for the normal baseline topology snapshot are updated using historical case data from the fault case library, so that the normal baseline topology snapshot can adapt to changes in service load. The root causes of the confirmed cases in the historical case data are matched with the fault rule library of the operating framework. The feature weights and scoring parameters of the successfully matched rules are adjusted, and corresponding fault rules are added for the unmatched fault modes to update the fault rule library of the operating framework. Based on the isolation execution status information, the applicable scenarios and priorities of successful isolation strategies are increased, and the parameters of strategies that are ineffective or trigger rollback are adjusted and downgraded to improve the adaptability and success rate of isolation strategies.
9. A device for handling service failures in a runtime framework, characterized in that, The device for handling runtime framework service failures includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the method for handling runtime framework service failures as described in any one of claims 1 to 8.
10. A storage medium, characterized in that, The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, it implements the steps of the method for handling runtime framework service failures as described in any one of claims 1 to 8.