Cloud edge collaboration-based service fault root cause analysis method and system

Abstract

This application discloses a service failure root cause analysis method and system based on cloud-edge collaboration. The cloud server-side method includes: receiving a standardized feature data stream sent by at least one edge node; the feature data stream is obtained by real-time collection and preprocessing of key performance data by the edge node, and the key performance data includes at least one of the following: abnormal SQL monitoring data, slow SQL performance data, and slow response interface data; performing cross-instance and cross-service time-series correlation analysis and pattern mining based on the feature data stream to determine each root cause node of the service failure phenomenon currently existing in the business system; constructing service failure information for each root cause node; and sending the service failure information to a visualization module on the client for user viewing. Using this application, the analysis blind spots caused by data silos and lack of correlation in traditional methods are solved, greatly improving the accuracy and global perspective of root cause location. At the same time, the automation of root cause analysis ensures the stability of the business system.

Description

Technical Field

[0001] This application relates to the field of SaaS technology, and in particular to a method and system for root cause analysis of service failures based on cloud-edge collaboration. Background Technology

[0002] With the widespread adoption of SaaS (Software as a Service) in the digital transformation of the manufacturing industry, cloud-edge collaborative architecture has become the mainstream distributed deployment model supporting business systems. For example, in intelligent manufacturing production lines, the Manufacturing Execution System (MES) deploys its business services (such as order processing, equipment control, and quality inspection) in a distributed manner as microservices within a cloud-edge environment. In this complex and heterogeneous service architecture, a performance failure of any single service component located at the edge or on the cloud server can disrupt the entire production process; therefore, real-time monitoring of service components is necessary.

[0003] In related technologies, simple performance thresholds (such as CPU utilization) are set independently on edge nodes. When the monitored data exceeds the threshold, the edge node reports an alarm to the cloud. Alternatively, by collecting and centrally storing full logs and distributed tracing data, and relying on operations and maintenance experts after a failure, root cause inference is performed by manually tracing back and piecing together the service call chain and system status at the time of the failure.

[0004] However, monitoring methods that simply set performance thresholds lack mechanisms for collaborative correlation and fusion analysis of multi-source heterogeneous performance data across edge nodes, cloud edge layers, and service components. This leads to a fragmentation of the correlation between fault phenomena, making it difficult to accurately trace the original trigger point of cascading faults, and often resulting in misjudgments in root cause localization. Relying on operations and maintenance experts cannot provide real-time responses to sudden anomalies on the edge side. Summary of the Invention

[0005] This application provides a method and system for service failure root cause analysis based on cloud-edge collaboration. To provide a basic understanding of some aspects of the disclosed embodiments, a brief summary is given below. This summary is not intended as a general commentary, nor is it intended to identify key / important components or describe the scope of protection of these embodiments. Its sole purpose is to present some concepts in a simple form as a prelude to the detailed description that follows.

[0006] In a first aspect, embodiments of this application provide a service failure root cause analysis method based on cloud-edge collaboration, the method comprising: Receive a standardized feature data stream sent by at least one edge node; wherein the feature data stream is obtained by the edge node in real time by collecting and preprocessing key performance data, and the key performance data includes at least one of the following: abnormal SQL monitoring data, slow SQL performance data, and slow response interface data; Based on the feature data stream, perform cross-instance and cross-service temporal correlation analysis and pattern mining to identify each root cause node of the current service failure phenomenon in the business system. Construct service failure information for each root cause node; Service failure information is sent to the client's visualization module for users to view.

[0007] Secondly, embodiments of this application provide a service failure root cause analysis system based on cloud-edge collaboration, the system comprising: The feature data stream receiving module is used to receive a standardized feature data stream sent by at least one edge node; wherein, the feature data stream is obtained by the edge node in real time by collecting and preprocessing key performance data, and the key performance data includes at least one of the following: abnormal SQL monitoring data, slow SQL performance data, and slow response interface data; The temporal correlation analysis and pattern mining module is used to perform cross-instance and cross-service temporal correlation analysis and pattern mining based on feature data streams in order to identify each root cause node of the service failure phenomenon currently existing in the business system. The service failure information construction module is used to construct service failure information for each root cause node. The visualization module is used to send service failure information to the client for user viewing.

[0008] The technical solutions provided in this application embodiment may include the following beneficial effects: In this embodiment, on the one hand, by constructing a standardized feature data stream and uploading it to the cloud, the cloud analysis center receives unified format data from edge nodes across the entire network. Through temporal correlation analysis and pattern mining, it can identify potential spatiotemporal correlations based on dependencies between anomalies at multiple points in a global view, thereby accurately inferring the original trigger point of the fault and reconstructing the complete fault propagation path. This solves the analysis blind spots caused by data silos and lack of correlation in traditional methods, greatly improving the accuracy of root cause localization and the global perspective. On the other hand, edge nodes filter noise and aggregate events based on static thresholds and dynamic baseline rules, and immediately report when significant anomalies are detected. Simultaneously, second-level, fully automated deep root cause analysis is achieved on the cloud side. From the edge side's perception of anomalies to the cloud's analysis of the most likely root cause node and generation of a diagnostic report, the entire process is automated by the system. This automation of root cause analysis ensures the stability of the business system.

[0009] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0010] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0011] Figure 1 This is a schematic flowchart of a service failure root cause analysis method based on cloud-edge collaboration provided in an embodiment of this application; Figure 2 This is a schematic diagram of a key performance data preprocessing process provided in an embodiment of this application; Figure 3 This is a schematic block diagram illustrating a process for tracing multiple suspected fault propagation paths provided in an embodiment of this application; Figure 4 This is a schematic diagram illustrating the process of generating a set of candidate root cause nodes according to an embodiment of this application; Figure 5 This is a schematic diagram illustrating the final confidence score generation process provided in an embodiment of this application; Figure 6 This is a schematic diagram of a service failure information interface in a certain scenario provided by an embodiment of this application; Figure 7 This is a schematic diagram of the interface of a visualization module in a certain scenario provided in an embodiment of this application; Figure 8 This is a schematic diagram of the structure of a service failure root cause analysis system based on cloud-edge collaboration provided in an embodiment of this application; Figure 9 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0012] The following description and accompanying drawings fully illustrate specific embodiments of this application to enable those skilled in the art to practice them.

[0013] It should be understood that the described embodiments are merely some, not all, of the embodiments in this application. All other embodiments obtained by those skilled in the art based on the embodiments in this application without inventive effort are within the scope of protection of this application.

[0014] In the following description, when referring to the accompanying drawings, the same numbers in different drawings denote the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of systems and methods consistent with some aspects of this application as detailed in the appended claims.

[0015] In the description of this application, it should be understood that the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance. Those skilled in the art can understand the specific meaning of the above terms in this application based on the specific circumstances. Furthermore, in the description of this application, unless otherwise stated, "multiple" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship.

[0016] Currently, by independently setting simple performance thresholds (such as CPU utilization) on edge nodes, when monitored data exceeds the threshold, the edge nodes report alarm information to the cloud. Alternatively, by collecting and centrally storing full logs and distributed tracing data, after a failure occurs, operations and maintenance experts can manually trace and piece together the service call chain and system status at the time of the failure to infer the root cause.

[0017] The inventors realized that monitoring methods that simply set performance thresholds lack a mechanism for collaborative correlation and fusion analysis of multi-source heterogeneous performance data across edge nodes, cloud-edge layers, and service components. This leads to a fragmentation of the correlation between fault phenomena, making it difficult to accurately trace the original trigger point of cascading faults, and often resulting in misjudgments in root cause localization. Relying on operations and maintenance experts cannot provide real-time responses to sudden anomalies on the edge side.

[0018] To address the aforementioned issues, this application provides a cloud-edge collaborative service failure root cause analysis method and system to resolve the problems existing in the related technologies. In this application's embodiments, on one hand, by constructing a standardized feature data stream and uploading it to the cloud, the cloud analysis center receives unified format data from edge nodes across the entire network. Through temporal correlation analysis and pattern mining, it can identify potential spatiotemporal correlations based on dependencies between anomalies at multiple points in a global view, thereby accurately inferring the original trigger point of the fault and reconstructing the complete fault propagation path. This solves the analysis blind spots caused by data silos and lack of correlation in traditional methods, greatly improving the accuracy and global perspective of root cause localization. On the other hand, edge nodes filter noise and aggregate events based on static thresholds and dynamic baseline rules, and immediately report when significant anomalies are detected. Simultaneously, second-level, fully automated deep root cause analysis is achieved on the cloud side. From the edge side perceiving anomalies to the cloud analyzing the most likely root cause node and generating a diagnostic report, the entire process is automated. This automation of root cause analysis ensures the stability of the business system. The following exemplary embodiments provide a detailed description.

[0019] The following will be combined with the appendix Figure 1 -Appendix Figure 7 This application provides a detailed description of the cloud-edge collaborative service failure root cause analysis method provided in its embodiments. This method can be implemented using a computer program and can run on a cloud-edge collaborative service failure root cause analysis system based on the von Neumann architecture. This computer program can be integrated into applications or run as a standalone utility application.

[0020] Please see Figure 1 This document provides a flowchart illustrating a service failure root cause analysis method based on cloud-edge collaboration, applicable to cloud servers. Figure 1 As shown, the method in this application embodiment includes the following steps: S101, receive a standardized feature data stream sent by at least one edge node; wherein the feature data stream is obtained by the edge node in real time by collecting and preprocessing key performance data, and the key performance data includes at least one of the following: abnormal SQL monitoring data, slow SQL performance data, and slow response interface data; Edge nodes are computing entities deployed between the industrial site where the business system resides and the core cloud network within the network topology. These include edge servers, industrial smart gateways, or embedded industrial control computers. Logically, they are configured with data acquisition agents and preprocessing modules to capture, clean, and initially aggregate local data. Abnormal SQL monitoring data records and analyzes SQL execution anomalies captured by database or middleware probes, including syntax errors, deadlocks, and permission errors. Slow SQL performance data collects SQL statements whose execution time exceeds a preset threshold from the application or database side, and correlates them with their execution plans, resource consumption (CPU, IO), and other contextual information. Slow response interface data is obtained by monitoring the response time, error rate, and call chain of application programming interfaces (APIs) through a microservice gateway to identify performance bottlenecks.

[0021] Among them, the business system can be a manufacturing execution system (MES system).

[0022] In some embodiments of this application, at least one edge node collects abnormal SQL monitoring data, slow SQL performance data, and slow response interface data, performs preprocessing operations to obtain a standardized feature data stream, and then sends the standardized feature data stream to the cloud server, which receives the standardized feature data stream sent by at least one edge node.

[0023] In some embodiments of this application, the specific process of edge nodes collecting and preprocessing key performance data in real time includes: capturing each abnormal event corresponding to abnormal SQL monitoring data, slow SQL performance data, and slow response interface data in real time through lightweight agents pre-integrated in the database driver layer, middleware probe, and application programming interface gateway; encapsulating each abnormal event into a structured log containing a timestamp, service identifier, instance identifier, performance indicators, and original context information; filtering the structured log according to preset filtering rules to obtain filtered events; counting similar events existing in the filtered events within a preset short time window, aggregating key performance indicators to obtain aggregated data; and converting the aggregated data into a unified format to obtain a feature data stream.

[0024] The lightweight agent is a software module deployed within or around the database driver layer, middleware probe, or application programming interface (API) gateway. This module features low resource consumption, low invasiveness, and high concurrency processing capabilities. Structured logs transform unstructured or semi-structured raw monitoring data into machine-readable, easily parsed standard data objects according to a predefined schema. Pre-configured filtering rules are a set of pre-configured strategies used to filter out valid events that meet specific anomaly characteristics from massive amounts of raw events. Similar events are a set of anomalous events with the same attributes across preset dimensions.

[0025] In one possible implementation, a manufacturing enterprise's cloud-based MES system consists of multiple microservices, where the order service relies on the inventory service and a database cluster. Operations personnel want to monitor and automatically identify the root causes of order creation failures in real time. Agent_DB is integrated into the database driver layer (e.g., JDBC Driver). Agent_Middleware is integrated into the middleware probe (e.g., Tomcat / Jetty container). Agent_Gateway is integrated into the API gateway (e.g., SpringGateway / Nginx). When a user initiates an order creation request, Agent_DB listens for an SQL statement `UPDATEinventory SET count = count – 1 WHERE id = ?` whose execution time reaches 2.5 seconds (far exceeding normal levels), classifying it as an abnormal event. Agent_DB immediately encapsulates this abnormal event into a structured log in JSON format as follows: { "timestamp": "2023-10-27T10:00:05.123Z", "service_id": "order-service-v1", "instance_id": "node-mes-03", "metric_type": "slow_sql", "metric_value": 2500, / / Unit: milliseconds "context_info": { "sql_text": "UPDATE inventory...", "error_code": null, "stack_trace": "at com.db.Driver.execute..." } }

[0026] At this point, Rule A (static threshold): SQL execution time > 1000ms. Rule B (dynamic baseline): Average response time + 3 standard deviation < actual response time. The system receives the above logs, and Rule A determines that 2500ms > 1000ms, so the verification passes. The system simultaneously receives another log with an execution time of 800ms, which Rule A determines fails and needs to be discarded. The final result is a filtered list of events, containing only records of severe timeouts. The preset short time window is 60 seconds. Grouping is done by {service_id, metric_type}. At this point, within the window from 10:00:00 to 10:01:00, there are 15 slow SQL events that meet the criteria, all belonging to order-service-v1. Total number of events of the same type = 15. Average value = (2500 + 2600 + ... + 2400) / 15 ≈ 2520ms. Maximum value = 2800ms. The final aggregated data record is generated as follows: {service_id: "order-service-v1", type: "slow_sql", count: 15, avg: 2520, max: 2800}. The system converts the above aggregated record into a unified Protobuf format, adds a sequence number and timestamp, and pushes it to the cloud server. The characteristic data stream format is as follows: message FeatureStream { int64 window_start = 1; / / Window start time string service_id = 2; / / order-service-v1 string event_type = 3; / / slow_sql int32 occurrence_count = 4; / / 15 float avg_metric = 5; / / 2520.0 float max_metric = 6; / / 2800.0 }

[0027] The preset filtering rules include static threshold rules and dynamic baseline rules.

[0028] In some embodiments of this application, the specific process of filtering structured logs according to preset filtering rules to obtain filtered events includes: applying static threshold rules to perform preliminary screening of structured logs to determine whether the execution time of SQL statements exceeds a first preset threshold and whether the application programming interface response error code belongs to a preset error code set, thereby obtaining initial performance data filtered by static threshold rules; and performing secondary judgment on the initial performance data based on dynamic baseline rules; wherein, the dynamic baseline rules are calculated by analyzing historical performance index data, and the dynamic baseline rules are used to characterize the performance fluctuation range of the service under normal conditions; in the secondary judgment process, when the performance index in the initial performance data exceeds the preset proportion of the corresponding dynamic baseline value, it is judged as an abnormal event, and each abnormal event is output as a filtered event.

[0029] In one possible implementation, the filtering process is divided into two phases: a static phase and a dynamic judgment phase. In the static phase, the system performs rapid filtering of the real-time incoming structured logs based on fixed rules, aiming to remove obviously normal data and capture identified hard faults. In the dynamic judgment phase, the system performs intelligent secondary verification on the initial performance data that has passed the first phase of filtering. This aims to address relative anomalies that static thresholds cannot handle (e.g., although it does not exceed 1000ms, during off-peak hours, an operation that normally takes 50ms suddenly takes 200ms, which is also considered an anomaly).

[0030] For example, in the static phase, the first preset threshold is set to 1000 milliseconds (ms). The agent captures a structured log showing that the execution time of a certain inventory deduction SQL statement is 1500ms. The system compares 1500ms with 1000ms. Since 1500ms > 1000ms, this log is statically filtered and marked as initial performance data. If another SQL statement takes 800ms to execute, it is discarded and not processed further.

[0031] For example, in the dynamic judgment phase, suppose there is an API response log with a response time of 350ms. This data does not exceed the static threshold (assuming the static threshold is set to 2000ms), but in this embodiment, it is assumed that it passed some kind of static filtering (or we only focus on the correction of the initial data by dynamic rules). The current dynamic baseline value is 200ms. For example, it is set to 150% of the baseline value (i.e., 1.5 times). The judgment threshold is calculated as: 200ms × 1.5 = 300ms. The actual performance metric 350ms > the judgment threshold 300ms. The system determines this event as an abnormal event. Although 350ms may not seem extremely slow globally, it represents a significant performance degradation compared to the historical baseline at the current moment.

[0032] In some embodiments of this application, the specific process of aggregating key performance indicators (KPIs) to obtain aggregated data from similar events among the filtered events includes: grouping the filtered events according to preset dimensions to obtain multiple groups; the preset dimensions include event type, service identifier, instance identifier, anomaly indicator name, and their combinations; counting the number of events in each group according to the preset event type to obtain the total number of occurrences of similar events within a preset time window; calculating the key performance indicators (KPIs) among similar events, where the KPIs include at least the average performance indicator and the maximum performance indicator determined from similar events; using the total number of occurrences and key performance indicators of each group as aggregated data records for each group; and summarizing the aggregated data records of each group to form aggregated data.

[0033] In one possible implementation, during a one-minute time window from 10:00:00 to 10:01:00, the manufacturing enterprise's production data monitoring service experienced a surge of slow SQL requests and interface timeouts due to database connection pool congestion. Within a preset short time window (e.g., 1 minute), the system received 50 filtered exception event logs. The system first categorized these 50 logs into different buckets based on a predefined grouping key. The preset dimensions are {service identifier, instance identifier, event type}. The system then iterated through the 50 logs, extracting these three fields for hash matching. The grouping results are shown below: Group A Service = production-monitor, Instance = node-01, Type = slow SQL (30 records).

[0034] Group B: Service = production-monitor, Instance = node-02, Type = slow SQL (15 records).

[0035] Group C: Service = production-monitor, Instance = node-01, Type = Interface Timeout (5 entries).

[0036] At this point, events within each group are counted to quantify the frequency of failures. For group A: the system counts the number of log entries in this group. Calculation result: Total occurrences = 30 times. This indicates that node-01 experienced 30 slow SQL queries within 1 minute, which is a high-frequency anomaly. The system extracts specific performance values ​​(such as SQL execution time and interface response time) from each log entry within the group and performs statistical calculations to quantify the severity of the failure. For group A (slow SQL): the data source is the `execution_time` field extracted from the 30 log entries. Assume the data is {2.1s, 2.5s, ..., 5.0s}.

[0037] Calculate the average: s.

[0038] Calculate the maximum value: s.

[0039] Finally, the above statistical results are packaged into a single structured aggregated data record, replacing the original 30 log entries. For example, the aggregated record for group A is as follows: { "window_time": "10:00:00 - 10:01:00", "dimensions": { "service": "production-monitor", "instance": "node-01", "type": "Slow SQL" }, "metrics": { "count": 30, / / Total number of occurrences "avg_value": 2.8, / / Average performance metric (seconds) "max_value": 5.0 / / Maximum performance metric (seconds) } }

[0040] For example Figure 2 As shown, Figure 2This is a schematic diagram of a key performance data preprocessing process provided in this application. First, various abnormal events generated during system operation are captured in real time through three types of data sources: a lightweight proxy at the database driver layer, a middleware probe, and an API gateway proxy, serving as the raw input for analysis. The captured raw abnormal events are immediately encapsulated into structured logs. These logs contain five key fields: timestamp (recording the time the event occurred), service identifier (indicating the service), instance identifier (indicating the specific running node), performance metrics (such as execution time and error codes), and original context information. The system applies static threshold rules to perform preliminary filtering of the structured logs. Specifically, it determines whether the execution time of an SQL statement exceeds a preset threshold T1, or whether the API response error code belongs to a preset error code set. Events that meet either condition are retained, forming initial performance data. A second judgment is then made on the initial performance data. The system first calculates historical performance data to generate a dynamic baseline (reflecting the normal fluctuation range of the service), and then determines whether the current performance metric exceeds a preset proportion of this baseline value. Only events that pass this checkpoint are ultimately confirmed as filtered events, effectively eliminating occasional, non-abnormal performance fluctuations. Events are divided into multiple logical groups based on event type, service / instance identifier, and anomaly metric name. The total number of occurrences for each group within a preset time window is counted to quantify the failure frequency. Statistical values ​​of key performance indicators (KPIs) within each group are calculated, including at least the average and maximum values, to reflect the severity of the failure. Finally, the total number of occurrences for each group and the aggregated KPIs are summed to form the aggregated data, i.e., the final standardized feature data stream, for subsequent time-series correlation analysis in the cloud.

[0041] S102, perform cross-instance and cross-service temporal correlation analysis and pattern mining based on feature data streams to determine each root cause node of the service failure phenomenon currently existing in the business system; In some embodiments of this application, the specific process of performing cross-instance and cross-service temporal correlation analysis and pattern mining based on feature data streams to determine each root cause node of the current service failure phenomenon in the business system includes: taking the services, service instances, database instances, or application programming interfaces of the business system as nodes, and the call relationships, data dependency relationships, or physical deployment relationships between the nodes as edges between the nodes, and labeling each node with abnormal indicators based on feature data streams to obtain a service dependency graph with temporal attributes; performing temporal correlation analysis on the abnormal events of each node within a preset sliding time window, and calculating the correlation between different abnormal indicator sequences on each node to identify multiple suspicious failure propagation paths with correlation; performing pattern mining on multiple suspicious failure propagation paths based on the service dependency graph to infer the set of candidate root cause nodes that best explain the global abnormal phenomenon; calculating the confidence score of each candidate root cause node in the set of candidate root cause nodes, and sorting the set of candidate root cause nodes based on the confidence scores to obtain at least one root cause node.

[0042] The service dependency graph is a graph-based data model where nodes represent logical or physical components in a business system (such as services, service instances, database instances, and APIs), and edges represent static topological relationships between these components (such as calls, data dependencies, and physical deployments). Unlike static topology graphs, each node in this graph is labeled with anomaly metrics (such as error rate and latency) calculated from feature data streams, and these metrics have explicit timestamps or time window attributes. This makes the graph structure contain not only topological information but also dynamic health status that changes over time. A preset sliding time window is used in time-series data analysis to extract dynamic time segments from continuous time-series data. Suspicious fault propagation paths are potential fault propagation links that may lead to global anomalies, derived from the service dependency graph and time-series correlation analysis results. The candidate root cause node set is a group of nodes most likely to cause the current service failure, selected through pattern mining and preliminary inference. Root cause nodes are one or more nodes with the highest confidence scores, determined by the system as the root cause of the current service failure.

[0043] In some embodiments of this application, the specific process of performing time-series correlation analysis on abnormal events of each node and calculating the correlation between different abnormal indicator sequences on each node to identify multiple suspicious fault propagation paths with correlation includes: traversing all abnormal events to be analyzed that converge within a preset sliding time window; extracting abnormal indicator sequences of each node within the sliding time window from all abnormal events to be analyzed; the abnormal indicator sequences include one or more of error rate sequences, request latency sequences, and SQL execution time sequences, each abnormal indicator sequence consisting of abnormal indicator values ​​at consecutive time points or time slices; for each abnormal indicator sequence, calculating the statistical correlation coefficient between any two sequences, the statistical correlation coefficient being used to quantify the degree of linear correlation between the two abnormal indicator sequences; identifying target sequence pairs whose statistical correlation coefficient exceeds a preset threshold; and tracing the path of the target node pairs corresponding to the target sequence pairs according to the dependency relationships between nodes in the service dependency graph to obtain multiple suspicious fault propagation paths with correlation.

[0044] For example Figure 3 As shown, Figure 3 This application provides a schematic flowchart of a process for tracing multiple suspicious fault propagation paths. First, a sliding time window is set, and all abnormal events to be analyzed within the window are aggregated. Then, three types of abnormal indicator sequences—error rate, request latency, and SQL execution time—are extracted from each node. Next, the statistical correlation coefficient between the sequences (such as the Pearson / Spearman coefficient) is calculated. Valid sequence pairs are filtered by determining whether the correlation exceeds a preset threshold. If the correlation meets the threshold, it is identified as a target sequence pair, and then traced according to service dependency paths, ultimately outputting multiple suspicious fault propagation paths. If the correlation does not reach the threshold, it is directly filtered out, thereby achieving accurate identification and hierarchical filtering of fault propagation paths.

[0045] In one possible implementation, at a certain moment, the platform monitoring system triggers an alarm, indicating that the order processing service response has slowed down. At this point, the system needs to analyze data from the past 5 minutes to pinpoint the root cause. The system first iterates through all the abnormal events to be analyzed within a 5-minute sliding time window. For the order service, the system extracts its request latency sequence. Assuming it's segmented into 30-second time slices, sequence A is obtained: [200ms, 210ms, 250ms, 500ms, 800ms, 1200ms,...]. For the inventory service, the system extracts its error rate sequence, resulting in sequence B: [0.1%, 0.2%, 0.5%, 1.0%, 5.0%, 10.0%,...]. For the payment gateway, the system extracts its SQL execution time sequence (assuming the payment gateway has an internal database instance), resulting in sequence C: [50ms, 60ms, 80ms, 100ms, 150ms, 200ms,...]. The system calculates the statistical correlation coefficient (e.g., using the Pearson correlation coefficient) between any two sequences extracted above. The correlation coefficient is calculated between the order service request latency sequence A and the inventory service error rate sequence B. The result is 0.92 (strong positive correlation; the closer the value is to 1, the stronger the correlation). The correlation coefficient is also calculated between the order service request latency sequence A and the payment gateway SQL execution time sequence C. The result is 0.35 (weak correlation). The system compares the calculated correlation coefficients with a preset threshold (e.g., 0.8). The decision logic is as follows: Since 0.92 > 0.8, the sequence pair (A, B) is identified as the target sequence pair. Since 0.35 < 0.8, the sequence pair (A, C) is filtered out. The system then performs path tracing for the target sequence pairs based on the topological relationships in the service dependency graph. The service dependency graph shows that the order service depends on the inventory service (i.e., the order service calls the inventory service). At this point, because sequence A (order service delay) and sequence B (inventory service error) are highly correlated and have a call dependency, the system will construct a path from the inventory service to the order service. Ultimately, the system outputs a suspicious fault propagation path: from the inventory service to the order service. This indicates that the anomaly in the inventory service (increased error rate) most likely caused the delay in the order service.

[0046] In some embodiments of this application, the specific process of inferring the candidate root cause node set that best explains the global anomaly by performing pattern mining on multiple suspicious fault propagation paths based on the service dependency graph includes: starting from the endpoint node of each suspicious fault propagation path, backtracking in the direction of the dependency relationship indicated by the edge on the service dependency graph; during the backtracking process, calculating the probability score of each node in the service dependency graph becoming the root cause of the fault; selecting nodes whose probability scores exceed a set threshold as the candidate root cause node set; verifying the candidate root cause node set to determine the degree of decrease in the explanatory power of the global anomaly after removing any node from the set, and obtaining the verification result; based on the verification result, selecting the smallest subset that can best explain the global anomaly as the candidate root cause node set.

[0047] For example Figure 4 As shown, Figure 4 This is a schematic diagram illustrating the process of generating a candidate root cause node set provided in this application. First, starting from the endpoint node, the fault propagation path is traced backwards, and the root cause probability score of each node in the path is calculated. Nodes with scores greater than a set threshold are selected to form a candidate root cause node set. Next, the importance of each node in the set is verified. By removing the node and assessing the degree of decline in its global anomaly explanatory power, it is determined whether it is a critical node. If it is a critical node, it is retained in the candidate set; otherwise, it is removed. Finally, a verified candidate root cause node set is obtained. This method combines probability scoring with causal intervention assessment to ensure that the selected root cause nodes are both highly probable and have a critical impact, improving the accuracy and interpretability of root cause localization.

[0048] In one possible implementation, the system detects widespread timeouts (global anomaly) at the user order placement interface. The suspected fault propagation path is: Database Instance A → Inventory Service → Order Service → Gateway Service. Within a preset time window, the error rate of the Gateway Service spikes to 30%, with an average response time exceeding 3 seconds. The system starts from the endpoint of the suspected path (i.e., the node with the most obvious anomaly) and backtracks along the dependency relationships on the service dependency graph. The starting point of this backtracking is the Gateway Service. Based on the edges in the service dependency graph, the system traces back to the upstream caller or dependent of the Gateway Service—the Order Service. Continuing the backtracking, it finds that the Order Service depends on the Inventory Service. Continuing the backtracking, it finds that the Inventory Service depends on Database Instance A. The backtracking path is: Gateway Service ← Order Service ← Inventory Service ← Database Instance A.

[0049] During the reverse backtracking process, the system calculates a probability score for each node to be a root cause of the failure. This score is calculated based on factors such as the severity of the anomaly, its position in the path, and its historical failure frequency. For example, database instance A, being at the top of the path, has a probability score of 0.95 because its CPU utilization reached 95% during the anomaly (far exceeding the usual 60%). The inventory service's error rate increased due to database call timeouts, resulting in a probability score of 0.70. The order service's latency was caused by inventory service call timeouts, resulting in a probability score of 0.50. The gateway service is only a symptom node of the anomaly, with a probability score of 0.10. The system includes nodes with probability scores exceeding a set threshold (e.g., 0.60) in the candidate root cause node set. The selection result is that database instance A (0.95 > 0.60) and the inventory service (0.70 > 0.60) are selected, forming the candidate root cause node set {database instance A, inventory service}.

[0050] Assuming the failure of database instance A is removed (i.e., assuming the database returns to normal), system simulations show that anomalies in the inventory service, order service, and gateway service all disappear, but the global anomaly explanation capability decreases by 90%. Assuming the failure of the inventory service is removed, system simulations show that anomalies in the order service and gateway service are alleviated, but latency in the inventory service itself persists, and the global anomaly explanation capability only decreases by 40%. Based on the verification results, the system selects the smallest subset that can best explain the global anomalies as the candidate root cause node set. The removal of database instance A causes the largest decrease in global anomaly explanation capability (90%), and since it is a single node, it constitutes the smallest subset. Final result: The candidate root cause node set is determined to be {database instance A}.

[0051] In some embodiments of this application, the specific process of calculating the confidence score of each candidate root cause node in the candidate root cause node set includes: obtaining multiple scoring dimension indicators for each candidate root cause node in the candidate root cause node set; the multiple scoring dimension indicators include at least the root cause node's own anomaly severity indicator, the number and anomaly degree indicators of downstream nodes affected by the root cause node, and the topological criticality indicator of the root cause node in the inferred fault propagation path; calculating the individual score of each scoring dimension indicator; using preset weighting coefficients, weighting and summing the individual scores of each scoring dimension indicator to obtain the preliminary confidence score of each candidate root cause node; correcting the preliminary confidence score of each candidate root cause node to obtain the final confidence score of each candidate root cause node.

[0052] Specifically, the individual score of the root cause node's own anomaly severity index. The calculation expression is: ; in, The error rate increases by a factor of 1. In response to the increase in latency, The rate of decrease in throughput, For the weighting coefficients, take... , It is a normalization function that maps the actual values ​​to the interval [0, 100].

[0053] Specifically, the number of downstream nodes affected by the root cause node and the individual score of the anomaly index. The calculation expression is: ; in The number of directly and indirectly affected downstream nodes. This represents the total number of nodes in the system. The average anomaly level of downstream nodes. For the weighting coefficients, take... .

[0054] Specifically, the individual scores of the root node in the topological criticality index of the inferred fault propagation path. The expression for the calculation is:

[0055] in, For normalized degree centrality, For normalized betweenness centrality, For normalized compactness centrality, For the weighting coefficients, take... .

[0056] Specifically, the correction expression for adjusting the initial confidence score is as follows:

[0057] in, The final confidence score for each candidate root cause node. The initial confidence score for each candidate root cause node. It is a time consistency factor, which characterizes the proximity between the fault start time and the node anomaly start time; It is a historical frequency factor, representing the frequency of historical failures of a node. It is a repair verification factor used to simulate the degree to which repairing this node improves the overall anomaly.

[0058] For example, Table 1 shows the multiple scoring dimensions for each candidate root cause node in a given scenario.

[0059] Table 1

[0060] For example Figure 5 As shown, Figure 5 This is a schematic diagram illustrating the final confidence score generation process provided in this application. First, three scoring dimensions of candidate root cause nodes are obtained: their own anomaly severity, downstream influence range (number × anomaly degree), and topological criticality (location along the propagation path). Then, individual scores for each dimension are calculated. Next, the individual scores are weighted and summed using preset weighting coefficients to obtain a preliminary confidence score for each candidate root cause node. Subsequently, the preliminary confidence scores are corrected, and finally, the ranked final confidence scores and corresponding root cause nodes are output. This method achieves objective and quantifiable ranking and location of root cause nodes through multi-dimensional index fusion and confidence correction.

[0061] S103, Construct service failure information for each root cause node; Service failure information refers to the structured failure diagnosis report generated after root cause analysis. It contains complete failure analysis results, including the identification, type, and location of the root cause node, quantitative data of abnormal indicators (normal values ​​vs. outliers), failure propagation path and timeline, impact scope and business loss assessment, remediation suggestions and verification methods, and relevant evidence chains (logs, monitoring graphs, correlation matrices).

[0062] In one possible implementation, the analysis engine traces the fault propagation path backward, discovering that all anomalies point to a specific database instance. Therefore, the root cause is identified as the instance in the MES database service executing the SQL statement "update vehicle VIN code binding workstation". The system collects and integrates information from multiple dimensions around this root cause to construct a complete service fault information report. This report will include the following core contents: root cause identifier, anomaly indicator details, fault impact scope, and fault propagation path.

[0063] The root cause node identifier includes the service name (e.g., MES-DB-Primary), instance ID (e.g., db-inst-03), and node type (e.g., database instance). Anomaly metric details include the anomaly type (e.g., slow SQL), SQL statement summary (e.g., UPDATE vehicle_station SET status = 'occupied' WHERE vin = ? AND station_id = ?), and key performance metrics (average execution time, maximum execution time, number of anomaly events). The scope of the failure impact includes directly affected services (e.g., the / api / v1 / production / report interface of MES-API-Gateway), downstream anomaly metrics, and business impact. The failure propagation path includes a path description.

[0064] For example, service failure information in one scenario, such as Figure 6 As shown.

[0065] S104 sends service failure information to the client's visualization module for user review.

[0066] In some embodiments of this application, the visualization module is used to receive a fault analysis request input by the user. The fault analysis request includes at least a Uniform Resource Locator (URI) and a fault time range. Based on the URI and the fault time range, the module queries and displays service fault information. The service fault information includes at least the root cause information of the fault located from one or more of the following dimensions: cyclic remote procedure calls, abnormal SQL, abnormal response, slow interface, and slow SQL.

[0067] For example, a visualization module for a given scenario, such as Figure 7 As shown.

[0068] In this embodiment, on the one hand, by constructing a standardized feature data stream and uploading it to the cloud, the cloud analysis center receives unified format data from edge nodes across the entire network. Through temporal correlation analysis and pattern mining, it can identify potential spatiotemporal correlations based on dependencies between anomalies at multiple points in a global view, thereby accurately inferring the original trigger point of the fault and reconstructing the complete fault propagation path. This solves the analysis blind spots caused by data silos and lack of correlation in traditional methods, greatly improving the accuracy of root cause localization and the global perspective. On the other hand, edge nodes filter noise and aggregate events based on static thresholds and dynamic baseline rules, and immediately report when significant anomalies are detected. Simultaneously, second-level, fully automated deep root cause analysis is achieved on the cloud side. From the edge side's perception of anomalies to the cloud's analysis of the most likely root cause node and generation of a diagnostic report, the entire process is automated by the system. This automation of root cause analysis ensures the stability of the business system.

[0069] The following are system embodiments of this application, which can be used to execute the method embodiments of this application. For details not disclosed in the system embodiments of this application, please refer to the method embodiments of this application.

[0070] Please see Figure 8 This illustration shows a schematic diagram of a cloud-edge collaborative service failure root cause analysis system provided in an exemplary embodiment of this application. This cloud-edge collaborative service failure root cause analysis system can be implemented as all or part of an electronic device through software, hardware, or a combination of both. System 1 includes a feature data stream receiving module 10, a time-series correlation analysis and pattern mining module 20, a service failure information construction module 30, and a visualization module 40.

[0071] The feature data stream receiving module 10 is used to receive a standardized feature data stream sent by at least one edge node; wherein the feature data stream is obtained by the edge node in real time by collecting and preprocessing key performance data, and the key performance data includes at least one of the following: abnormal SQL monitoring data, slow SQL performance data, and slow response interface data; The temporal correlation analysis and pattern mining module 20 is used to perform cross-instance and cross-service temporal correlation analysis and pattern mining based on feature data streams to determine each root cause node of the service failure phenomenon currently existing in the business system. Service failure information construction module 30 is used to construct service failure information for each root cause node; The visualization module 40 is a visualization display module used to send service failure information to the client for user viewing.

[0072] It should be noted that the cloud-edge collaborative service failure root cause analysis system provided in the above embodiments is only illustrated by the division of the above functional modules when executing the cloud-edge collaborative service failure root cause analysis method. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the cloud-edge collaborative service failure root cause analysis system and the cloud-edge collaborative service failure root cause analysis method embodiments provided in the above embodiments belong to the same concept, and the implementation process is detailed in the method embodiments, which will not be repeated here.

[0073] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0074] In this embodiment, on the one hand, by constructing a standardized feature data stream and uploading it to the cloud, the cloud analysis center receives unified format data from edge nodes across the entire network. Through temporal correlation analysis and pattern mining, it can identify potential spatiotemporal correlations based on dependencies between anomalies at multiple points in a global view, thereby accurately inferring the original trigger point of the fault and reconstructing the complete fault propagation path. This solves the analysis blind spots caused by data silos and lack of correlation in traditional methods, greatly improving the accuracy of root cause localization and the global perspective. On the other hand, edge nodes filter noise and aggregate events based on static thresholds and dynamic baseline rules, and immediately report when significant anomalies are detected. Simultaneously, second-level, fully automated deep root cause analysis is achieved on the cloud side. From the edge side's perception of anomalies to the cloud's analysis of the most likely root cause node and generation of a diagnostic report, the entire process is automated by the system. This automation of root cause analysis ensures the stability of the business system.

[0075] This application also provides a computer-readable medium having program instructions stored thereon, which, when executed by a processor, implement the cloud-edge collaborative service failure root cause analysis method provided in the above-described method embodiments.

[0076] This application also provides a computer program product containing instructions that, when run on a computer, cause the computer to execute the cloud-edge collaborative service failure root cause analysis method of the above-described method embodiments.

[0077] Please see Figure 9 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 9 As shown, the electronic device 1000 may include a memory 1001. Various functions of the electronic device 1000 and data processing are performed by running or executing instructions, programs, code sets, or instruction sets stored in the memory 1001, and by calling data stored in the memory 1001.

[0078] The memory 1001 may include random access memory (RAM) or read-only memory. Optionally, the memory 1001 may include a non-transitory computer-readable storage medium. The memory 1001 can be used to store instructions, programs, code, code sets, or instruction sets. The memory 1001 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as touch function, sound playback function, image playback function, etc.), instructions for implementing the above-described method embodiments, etc.; the data storage area may store data involved in the above-described method embodiments, etc. Figure 9 As shown, the memory 1001, which serves as a computer storage medium, may include an operating system, a network communication module, a user interface module, and a cloud-edge collaborative service failure root cause analysis application.

[0079] exist Figure 9 In the electronic device 1000 shown, the cloud-edge collaborative service failure root cause analysis application stored in the memory 1001 is invoked, and the following operations are performed: Receive a standardized feature data stream sent by at least one edge node; wherein the feature data stream is obtained by the edge node in real time by collecting and preprocessing key performance data, and the key performance data includes at least one of the following: abnormal SQL monitoring data, slow SQL performance data, and slow response interface data; Based on the feature data stream, perform cross-instance and cross-service temporal correlation analysis and pattern mining to identify each root cause node of the current service failure phenomenon in the business system. Construct service failure information for each root cause node; Service failure information is sent to the client's visualization module for users to view.

[0080] In one embodiment, when the processor 1001 performs real-time acquisition and preprocessing of key performance data, it specifically performs the following operations: Through lightweight agents pre-integrated in the database driver layer, middleware probes, and application programming interface gateway, each abnormal event corresponding to abnormal SQL monitoring data, slow SQL performance data, and slow response interface data is captured in real time. Each exception event is encapsulated into a structured log containing a timestamp, service identifier, instance identifier, performance metrics, and original context information; The structured logs are filtered according to preset filtering rules to obtain the filtered events; Within a preset short time window, count the similar events that exist in the filtered events, aggregate key performance indicators, and obtain aggregated data. The aggregated data is converted into a unified format to obtain a feature data stream.

[0081] In one embodiment, when the processor 1001 filters the structured logs according to preset filtering rules to obtain the filtered events, it specifically performs the following operations: The structured logs are initially filtered using static threshold rules to determine whether the execution time of the SQL statement exceeds the first preset threshold and whether the application programming interface response error code belongs to the preset error code set, so as to obtain the initial performance data after being filtered by the static threshold rules. Based on dynamic baseline rules, the initial performance data is re-evaluated; the dynamic baseline rules are calculated by analyzing historical performance index data and are used to characterize the performance fluctuation range of the service under normal conditions. During the secondary judgment process, when the performance indicators in the initial performance data exceed the preset proportion of the corresponding dynamic baseline value, they are judged as abnormal events, and each abnormal event is output as a filtered event.

[0082] In one embodiment, when processor 1001 aggregates key performance indicators to obtain aggregated data for similar events found in the events after counting and filtering, it specifically performs the following operations: The filtered events are grouped according to preset dimensions to obtain multiple groups; the preset dimensions include event type, service identifier, instance identifier, abnormal indicator name and their combination; According to the preset event type, the number of events in each group is counted to obtain the total number of times the same type of event occurs within the preset time window; Calculate key performance indicators (KPIs) for similar events. The KPIs should include at least the average value of the performance indicators and the maximum value of the performance indicators from similar events. The total number of occurrences and key performance indicators for each group are recorded as aggregated data for each group. The aggregated data records of each group are summarized to form the aggregated data.

[0083] In one embodiment, when processor 1001 performs cross-instance and cross-service temporal correlation analysis and pattern mining based on feature data streams to determine each root cause node of the current service failure phenomenon in the business system, it specifically performs the following operations: Using services, service instances, database instances, or application programming interfaces of the business system as nodes, and the call relationships, data dependencies, or physical deployment relationships between nodes as edges between nodes, and anomaly indicators based on feature data flow are marked on each node to obtain a service dependency graph with time-series attributes. Within a preset sliding time window, time-series correlation analysis is performed on abnormal events at each node, and the correlation between different abnormal indicator sequences at each node is calculated to identify multiple suspicious fault propagation paths with correlation. Based on the service dependency graph, pattern mining is performed on multiple suspicious fault propagation paths to infer the set of candidate root cause nodes that best explain global anomalies. Calculate the confidence score of each candidate root cause node in the candidate root cause node set, and sort the candidate root cause node set based on the confidence score to obtain at least one root cause node.

[0084] In one embodiment, when the processor 1001 performs time-series correlation analysis on abnormal events of each node and calculates the correlation between different abnormal indicator sequences on each node to identify multiple suspicious fault propagation paths with correlation, it specifically performs the following operations: Iterate through all abnormal events to be analyzed that converge within the preset sliding time window; From all the abnormal events to be analyzed, extract the abnormal indicator sequence of each node within the sliding time window; the abnormal indicator sequence includes one or more of the following: error rate sequence, request delay sequence, SQL execution time sequence, and each abnormal indicator sequence consists of abnormal indicator values ​​in continuous time points or time slices. For each abnormal indicator sequence, calculate the statistical correlation coefficient between any two sequences. The statistical correlation coefficient is used to quantify the degree of linear correlation between two abnormal indicator sequences. Identify target sequence pairs whose statistical correlation coefficients exceed a preset threshold; Based on the dependency relationships between nodes in the service dependency graph, the target sequence pairs and their corresponding target node pairs are traced to obtain multiple related suspicious fault propagation paths.

[0085] In one embodiment, when the processor 1001 performs pattern mining on multiple suspicious fault propagation paths based on the service dependency graph to infer the set of candidate root cause nodes that best explain the global anomaly, it specifically performs the following operations: Starting from the end node of each suspected fault propagation path, backtrack in reverse along the dependency direction indicated by the edge on the service dependency graph; During the reverse backtracking process, the probability score of each node in the service dependency graph becoming the root cause of the failure is calculated; Nodes whose probability scores exceed a set score threshold are selected as the set of candidate root cause nodes. The set of candidate root cause nodes is validated to determine the extent to which the explanatory power of global anomalies decreases after removing any node from the set, and the validation results are obtained. Based on the verification results, the smallest subset that can best explain the global anomalies is selected as the candidate root cause node set.

[0086] In one embodiment, when the processor 1001 calculates the confidence score of each candidate root cause node in the candidate root cause node set, it specifically performs the following operations: Obtain multiple scoring dimensions for each candidate root cause node in the candidate root cause node set; the multiple scoring dimensions include at least the root cause node's own anomaly severity index, the number and anomaly degree of downstream nodes affected by the root cause node, and the topological criticality index of the root cause node in the inferred fault propagation path. Calculate the individual score for each rating dimension indicator; By using preset weighting coefficients, the individual scores of each scoring dimension are weighted and summed to obtain the preliminary confidence score of each candidate root cause node. The initial confidence score of each candidate root cause node is corrected to obtain the final confidence score of each candidate root cause node.

[0087] In this embodiment, on the one hand, by constructing a standardized feature data stream and uploading it to the cloud, the cloud analysis center receives unified format data from edge nodes across the entire network. Through temporal correlation analysis and pattern mining, it can identify potential spatiotemporal correlations based on dependencies between anomalies at multiple points in a global view, thereby accurately inferring the original trigger point of the fault and reconstructing the complete fault propagation path. This solves the analysis blind spots caused by data silos and lack of correlation in traditional methods, greatly improving the accuracy of root cause localization and the global perspective. On the other hand, edge nodes filter noise and aggregate events based on static thresholds and dynamic baseline rules, and immediately report when significant anomalies are detected. Simultaneously, second-level, fully automated deep root cause analysis is achieved on the cloud side. From the edge side's perception of anomalies to the cloud's analysis of the most likely root cause node and generation of a diagnostic report, the entire process is automated by the system. This automation of root cause analysis ensures the stability of the business system.

[0088] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program for service failure root cause analysis based on cloud-edge collaboration can be stored in a computer-readable storage medium. When executed, the program can include the processes of the embodiments of the above methods. The storage medium for the program for service failure root cause analysis based on cloud-edge collaboration can be a magnetic disk, optical disk, read-only memory, or random access memory, etc.

[0089] The above-disclosed embodiments are merely preferred embodiments of this application and should not be construed as limiting the scope of this application. Therefore, any equivalent variations made in accordance with the claims of this application shall still fall within the scope of this application.

Claims

1. A service failure root cause analysis method based on cloud-edge collaboration, characterized in that, Applied to cloud servers, the method includes: Receive a standardized feature data stream sent by at least one edge node; wherein the feature data stream is obtained by the edge node in real time by collecting and preprocessing key performance data, and the key performance data includes at least one of the following: abnormal SQL monitoring data, slow SQL performance data, and slow response interface data; Based on the characteristic data stream, cross-instance and cross-service temporal correlation analysis and pattern mining are performed to determine each root cause node of the service failure phenomenon currently existing in the business system. Construct service failure information for each root cause node; The service failure information is sent to the client's visualization module for user viewing.

2. The method according to claim 1, characterized in that, Real-time acquisition and preprocessing of key performance data, including: The system uses a lightweight agent pre-integrated in the database driver layer, middleware probe, and application programming interface gateway to capture each abnormal event in real time, corresponding to the abnormal SQL monitoring data, slow SQL performance data, and slow response interface data. Each of the above abnormal events is encapsulated into a structured log containing a timestamp, service identifier, instance identifier, performance metrics, and original context information; The structured logs are filtered according to preset filtering rules to obtain filtered events; Within a preset short time window, count the similar events that exist in the filtered events, aggregate key performance indicators, and obtain aggregated data. The aggregated data is converted into a unified format to obtain a feature data stream.

3. The method according to claim 2, characterized in that, The preset filtering rules include static threshold rules and dynamic baseline rules; The step of filtering the structured logs according to preset filtering rules to obtain filtered events includes: The static threshold rules are applied to perform preliminary screening on the structured logs to determine whether the execution time of the SQL statement exceeds the first preset threshold and whether the application programming interface response error code belongs to the preset error code set, so as to obtain the initial performance data after being filtered by the static threshold rules. Based on the dynamic baseline rules, the initial performance data is subjected to a secondary judgment; wherein, the dynamic baseline rules are calculated by analyzing historical performance index data, and the dynamic baseline rules are used to characterize the performance fluctuation range of the service under normal conditions; During the secondary determination process, when the performance index in the initial performance data exceeds the preset proportion of the corresponding dynamic baseline value, it is determined to be an abnormal event, and each abnormal event is output as a filtered event.

4. The method according to claim 2, characterized in that, Among the events that are filtered by counting, similar events are aggregated, and key performance indicators are aggregated to obtain aggregated data, including: The filtered events are grouped according to preset dimensions to obtain multiple groups; the preset dimensions include event type, service identifier, instance identifier, anomaly indicator name and their combination; According to the preset event type, the number of events in each group is counted to obtain the total number of times the same type of event occurs within the preset time window; Calculate the key performance indicators (KPIs) for the similar events, wherein the KPIs include at least the average value of the performance indicators and the maximum value of the performance indicators determined from the similar events; The total number of occurrences and the key performance indicators of each group are recorded as aggregated data for each group. The aggregated data records of each group are summarized to form the aggregated data.

5. The method according to claim 1, characterized in that, The step of performing cross-instance and cross-service temporal correlation analysis and pattern mining based on the feature data stream to determine each root cause node of the current service failure phenomenon in the business system includes: Using the services, service instances, database instances, or application programming interfaces of the business system as nodes, and the call relationships, data dependencies, or physical deployment relationships between the nodes as edges between the nodes, and anomaly indicators based on the characteristic data flow are marked on each node, a service dependency graph with time-series attributes is obtained. Within a preset sliding time window, a time-series correlation analysis is performed on the abnormal events of each node, and the correlation between different abnormal indicator sequences on each node is calculated to identify multiple suspicious fault propagation paths with correlation. Based on the service dependency graph, pattern mining is performed on the multiple suspicious fault propagation paths to infer the set of candidate root cause nodes that best explain the global anomaly. Calculate the confidence score of each candidate root cause node in the candidate root cause node set, and sort the candidate root cause node set based on the confidence score to obtain at least one root cause node.

6. The method according to claim 5, characterized in that, The step of performing time-series correlation analysis on the abnormal events of each node and calculating the correlation between different abnormal indicator sequences on each node to identify multiple suspicious fault propagation paths with correlation includes: Iterate through all the abnormal events to be analyzed that converge within the preset sliding time window; From all the abnormal events to be analyzed, extract the abnormal indicator sequence of each node within the sliding time window; the abnormal indicator sequence includes one or more of the error rate sequence, request delay sequence, and SQL execution time sequence, and each abnormal indicator sequence consists of abnormal indicator values ​​in continuous time points or time slices. For each of the abnormal indicator sequences, a statistical correlation coefficient is calculated between any two sequences. The statistical correlation coefficient is used to quantify the degree of linear correlation between the two abnormal indicator sequences. Identify target sequence pairs whose statistical correlation coefficients exceed a preset threshold; Based on the dependency relationships between nodes in the service dependency graph, the target node pairs corresponding to the target sequence pairs are traced to obtain multiple related suspicious fault propagation paths.

7. The method according to claim 5, characterized in that, Based on the service dependency graph, pattern mining is performed on the multiple suspicious fault propagation paths to infer the set of candidate root cause nodes that best explain the global anomaly, including: Starting from the end node of each suspected fault propagation path, backtrack backward along the dependency direction indicated by the edge on the service dependency graph; During the reverse backtracking process, the probability score of each node in the service dependency graph becoming the root cause of the failure is calculated; Nodes whose probability scores exceed a set score threshold are selected as the set of candidate root cause nodes. The set of candidate root cause nodes is verified to determine the extent to which the explanatory power of removing any node from the set decreases for the global anomaly, and the verification results are obtained. Based on the verification results, the smallest subset that can best explain the global anomaly is selected as the candidate root cause node set.

8. The method according to claim 5, characterized in that, The calculation of the confidence score for each candidate root cause node in the candidate root cause node set includes: Obtain multiple scoring dimensions for each candidate root cause node in the candidate root cause node set; the multiple scoring dimensions include at least the root cause node's own anomaly severity index, the number and anomaly degree of downstream nodes affected by the root cause node, and the topological criticality index of the root cause node in the inferred fault propagation path. Calculate the individual score for each rating dimension indicator; By using preset weighting coefficients, the individual scores of each scoring dimension indicator are weighted and summed to obtain the preliminary confidence score of each candidate root cause node. The initial confidence score of each candidate root cause node is corrected to obtain the final confidence score of each candidate root cause node.

9. The method according to claim 1, characterized in that, The visualization module is used to receive fault analysis requests input by users. The fault analysis requests include at least a Uniform Resource Locator (URI) and a fault time range. Based on the URI and the fault time range, the module queries and displays service fault information. The service fault information includes at least root cause information located from one or more of the following dimensions: looping remote procedure calls, abnormal SQL, abnormal responses, slow interfaces, and slow SQL.

10. A service failure root cause analysis system based on cloud-edge collaboration, characterized in that, The system includes: A feature data stream receiving module is used to receive a standardized feature data stream sent by at least one edge node; wherein the feature data stream is obtained by the edge node in real time by collecting and preprocessing key performance data, and the key performance data includes at least one of the following: abnormal SQL monitoring data, slow SQL performance data, and slow response interface data; The temporal correlation analysis and pattern mining module is used to perform cross-instance and cross-service temporal correlation analysis and pattern mining based on the feature data stream, so as to determine each root cause node of the service failure phenomenon currently existing in the business system. The service failure information construction module is used to construct the service failure information for each root cause node. The visualization module is used to send the service failure information to the client for visualization display, so that the user can view it.

Images