Cloud service assurance method and system based on eBPF observable data and automated self-healing

Abstract

This invention provides a cloud service assurance method and system based on eBPF observable data and a prediction model. The method includes: collecting historical observable operational status data of the kernel, containers, and application layers and mapping and associating them; aligning and concatenating the associating data to generate a time-series feature vector; using the time-series feature vector to train and test the constructed model, calculating the residuals based on the predicted values ​​and corresponding actual observed values ​​during the testing period, and analyzing the residual probability distribution characteristics; using the model's prediction results for cloud services to calculate the residuals, and generating anomaly confidence scores based on the residual probability distribution characteristics; under the condition that the anomaly confidence score is greater than a threshold, analyzing the anomaly type and its impact range, then matching a remediation plan in the policy library and automatically executing the remediation plan; monitoring the execution effect of the remediation plan and feeding it back to the model; calculating a stability index based on key stability indicators to form a stability change curve, and then evaluating the execution effect of the remediation plan.

Description

Technical Field

[0001] This invention relates to the field of cloud service assurance technology, and in particular to a cloud service assurance method and system based on eBPF observable data and automated self-healing. Background Technology

[0002] With the booming development of the digital economy, cloud-native architecture has become the mainstream choice for enterprise digital transformation. Against the backdrop of widespread application of cloud-native architecture, the deep integration of microservice decomposition, containerized deployment, and dynamic scheduling technologies has driven business systems to achieve a leapfrog upgrade from traditional monolithic architectures to distributed and elastic architectures. However, this architectural innovation has also made the operational logic, dependencies, and resource scheduling patterns of business systems increasingly complex, bringing unprecedented challenges to ensuring the stability of cloud services.

[0003] Currently, the core challenges facing cloud service stability assurance systems are concentrated in four dimensions: First, monitoring granularity struggles to penetrate the underlying "black box." Traditional monitoring systems focus on macro-level indicators (such as CPU utilization, memory usage, and interface latency), lacking the ability to capture micro-level behaviors at the kernel level. Underlying performance bottlenecks such as system call latency, kernel scheduling blocking, kernel-mode network protocol stack packet loss, and kernel-mode storage I / O queuing are difficult to perceive, leading to difficulties in locating the root cause of problems. Second, anomaly detection exhibits a passive and lagging characteristic. Traditional alarm mechanisms based on static thresholds or fixed rules can only trigger a response after significant deterioration in business indicators, failing to accurately identify complex and hidden time-series anomalies. This makes it difficult to support the implementation of predictive maintenance, missing the critical window for proactive fault intervention. Third, there is a lack of automated self-healing capabilities. There is a significant disconnect between the "monitoring" and "control" links. Even if the existing system can detect anomalies, it still requires manual intervention to complete root cause analysis, fault diagnosis, and repair execution. This not only results in low response efficiency but also leads to a significant increase in the mean time to repair (MTBT). Fourth, the lack of a unified quantitative evaluation benchmark means that the industry has not yet formed a stability quantitative indicator that can comprehensively cover the actual effect of the entire process from monitoring and detection to self-healing. This makes it impossible to scientifically evaluate the overall effectiveness of the protection system and restricts the continuous optimization of stability protection capabilities. Summary of the Invention

[0004] To address the shortcomings of existing technologies, such as the difficulty in penetrating the underlying "black box" of monitoring granularity, the passive and lagging nature of anomaly detection, the lack of automated self-healing capabilities, and the absence of a unified quantitative evaluation benchmark, this invention provides a cloud service assurance method based on eBPF observable data and automated self-healing. By deeply integrating system kernel-level observable data provided by eBPF technology with an AI time-series prediction model, a closed loop for cloud service assurance is constructed, encompassing data collection, anomaly prediction, intelligent decision-making, and automated execution. This achieves accurate predictive anomaly detection, automatically matches and executes repair strategies, and uses a comprehensive stability index to quantitatively evaluate and optimize the assurance effect, significantly improving the reliability of cloud service assurance. This invention also provides a cloud service assurance system based on eBPF observable data and automated self-healing.

[0005] The technical solution of the present invention is as follows:

[0006] A cloud service assurance method based on eBPF observable data and automated self-healing includes the following steps:

[0007] The steps for collecting historical observable data across the entire link are as follows: Historical observable runtime status data of the kernel layer is collected using eBPF kernel probes mounted on the CPU scheduling, network, storage, and kernel system call paths; historical observable runtime status data of the container layer is collected using independent sampling methods; and historical observable runtime status data of the application layer is collected using OpenTelemetry or Zipkin end-to-end tracing. Based on timestamps and request identifiers, the historical observable runtime status data of the kernel layer, container layer, and application layer are mapped and associated across the entire link to form an end-to-end historical observable dataset associated with the business request to the system kernel.

[0008] The steps for end-to-end multidimensional data fusion and feature extraction are as follows: The historical observable operational status data of each layer associated with the end-to-end mapping are preprocessed by structuring, normalizing, and unifying the time granularity; based on the timestamp and request identifier, the preprocessed data of each layer are aligned within the same time window or the same request link dimension to obtain multiple sets of historical time-series data of the same business scenario or the same time segment; the aligned sets of historical time-series data are spliced ​​according to the time sequence or feature dimension to generate multiple sets of multidimensional unified time-series feature vectors containing system status and business performance with labels of the actual observation value at the next moment;

[0009] Anomaly prediction model training steps: Construct a temporal deep learning model based on the Transformer network; train and test the temporal deep learning model using the multiple sets of multidimensional unified temporal feature vectors to obtain the anomaly prediction model; calculate the residual based on the predicted value output by the anomaly prediction model during the testing period and the corresponding actual observation value at the next time moment, and obtain the residual probability distribution characteristics based on the statistical analysis of the residual;

[0010] Cloud service anomaly prediction steps: Real-time or periodic collection of observable operational status data of the kernel layer, container layer, and application layer of the cloud service platform under test; full-link mapping and association of the real-time observable operational status data of each layer based on timestamps and request identifiers; preprocessing, data alignment, and feature concatenation processing according to the same method as in the full-link multidimensional data fusion and feature extraction steps to obtain a unified time-series feature vector of the cloud service platform under test; inputting the unified time-series feature vector of the cloud service platform under test into a trained and tested anomaly prediction model; the model outputs the prediction result of the observable operational status data of the cloud service platform at the next moment; by calculating the residual between the prediction result and the corresponding actual observation value in the unified time-series feature vector of the cloud service platform under test, and combining the probability distribution characteristics of the residual, generating the anomaly confidence level corresponding to the prediction result;

[0011] The self-healing decision generation and execution steps are as follows: Under the condition that the anomaly confidence level is greater than a preset anomaly confidence level threshold, the target feature dimension to which the maximum residual corresponding to the anomaly confidence level belongs is determined through the interpretable method of the anomaly prediction model. The anomaly type corresponding to the target feature dimension is determined by combining it with a preset anomaly type rule base. The anomaly impact range corresponding to the target feature dimension is determined using dimension drill-down and topological association methods. Based on the anomaly type and anomaly impact range, the optimal repair solution is matched in a preset strategy base, and the matched repair solution is automatically executed by calling a cloud platform or Kubernetes API. The execution effect of the repair solution is monitored and fed back to the anomaly prediction model to achieve adaptive fine-tuning of the anomaly prediction model.

[0012] Stability demonstration and evaluation steps: Real-time acquisition of key stability indicators, including business availability, average response latency, and anomaly confidence; calculation of stability index based on the key stability indicators; construction of a two-dimensional coordinate axis with the stability index and time as the axis, mapping the stability index corresponding to each time point to the two-dimensional coordinate axis, connecting adjacent data points to form a stability change curve to visualize the stability change trend of cloud services; evaluation of the implementation effect of the remediation plan based on the stability change curve.

[0013] Preferably, in the self-healing decision generation and execution step, if multiple repair solutions are matched in the preset strategy library based on the anomaly type and the anomaly impact range, the repair solution with the highest priority is selected as the optimal repair solution.

[0014] The monitoring and repair scheme's execution effect and its feedback to the anomaly prediction model, enabling adaptive fine-tuning of the anomaly prediction model, includes: collecting observable operational status data of the kernel layer, container layer, and application layer after the optimal repair scheme is executed; if the data after execution has all recovered to within the preset normal observable operational status threshold range, and the stability change curve within the corresponding time period shows a steady upward trend over time, then the optimal repair scheme is determined to have been executed successfully; if the data after execution has not all recovered to within the preset normal observable operational status threshold range, or the stability change curve within the corresponding time period does not show a steady upward trend over time, then the optimal repair scheme is determined to have failed.

[0015] Based on the execution results of this optimal repair scheme, the observable runtime status data collected during the execution period of the optimal repair scheme, including the kernel layer, container layer, and application layer, as well as their corresponding target feature dimensions and the optimal repair scheme, are marked: if the execution is successful, it is marked as a successful case; if the execution fails, it is marked as a failed case. The failed cases include false positive cases, false negative cases, and cases where the repair is ineffective.

[0016] The successful cases are added to the training set of the anomaly prediction model, and the anomaly prediction model is incrementally trained or its parameters are fine-tuned periodically using the training set containing the successful cases.

[0017] The preset anomaly confidence threshold is dynamically adjusted based on the ratio of the total number of false alarms and false negatives within a preset time period to the total number of all repaired cases within that time period.

[0018] Based on the success rate, average repair efficiency, and changes in the stability index of the repair solutions corresponding to each anomaly type within a preset time period, the priority of each repair solution under the corresponding anomaly type in the preset strategy library is dynamically adjusted.

[0019] Preferably, in the full-link historical observable data collection step, the historical observable runtime status data of the kernel layer is kernel layer event data including kernel system calls, TCP latency, and CPU scheduling blockage collected through eBPF kernel probes; the historical observable runtime status data of the container layer is data including container instance resource utilization, IO latency, and network packet loss rate collected through fixed-frequency timed sampling; and the historical observable runtime status data of the application layer is data including request latency and error rate of the business request link collected through OpenTelemetry or Zipkin full-link tracing.

[0020] Preferably, in the full-link multidimensional data fusion and feature extraction step, the preprocessing step of structuring, normalizing, and unifying the time granularity of the historical observable operational status data of each layer after full-link mapping includes: structuring, normalizing, and time-aligning the kernel layer event data and container layer historical observable operational status data after full-link mapping; and normalizing the application layer historical observable operational status data after full-link mapping.

[0021] Preferably, in the self-healing decision generation and execution step, the step of determining the anomaly type corresponding to the target feature dimension by combining a preset anomaly type rule base, and determining the anomaly influence range corresponding to the target feature dimension by using dimension drill-down and topological association methods includes:

[0022] Based on the target feature dimension, the corresponding anomaly type is matched in the preset anomaly type rule base, which pre-stores the correspondence between each feature dimension and the anomaly type.

[0023] Perform a dimension drill-down operation on the target feature dimension to obtain the source dimension attribute corresponding to the target feature dimension. The source dimension attribute includes at least one of Pod, Node, Service, and Deployment deployment unit.

[0024] By combining the collected end-to-end observable operational status data, a topology correlation analysis is performed on the abnormal business links associated with the target feature dimensions to identify the common topology nodes of each abnormal business link. The common topology nodes include at least one of a gateway and a database instance.

[0025] By combining the source dimension attribute results of the drill-down tracing and the common topology node results of the topology association analysis, the scope of the abnormal influence corresponding to the target feature dimension is determined.

[0026] Preferably, in the self-healing decision generation and execution step, if no repair solution corresponding to the anomaly type and the scope of anomaly impact is found in the preset strategy library, a repair solution adapted to the anomaly type and the scope of anomaly impact is dynamically generated, specifically including:

[0027] Kernel layer event data is collected using diagnostic tools or eBPF kernel probes, and the collected kernel layer event data is analyzed in depth to locate the root cause corresponding to the anomaly confidence level.

[0028] The root cause and the observable operational status data obtained from the cloud service anomaly prediction steps are input into a decision engine containing an operation and maintenance knowledge graph or causal graph, and the decision engine infers and generates a repair action chain.

[0029] Using a sandbox testing environment or a trained cost / risk assessment model, the pre-execution impact prediction analysis is performed on the remediation strategies to be executed corresponding to the remediation action chain, and the assessment results including execution cost and cross-service impact are obtained.

[0030] Based on the repair action chain and the evaluation results, a repair plan adapted to the anomaly type and the scope of its impact is generated.

[0031] Preferably, in the self-healing decision generation and execution steps, the optimal repair scheme includes resource scaling up / down, service rollback, node restart, configuration hot update, and traffic switching. The execution form of the optimal repair scheme is a set of K8s commands or API call sequences.

[0032] Preferably, in the stability demonstration and evaluation step, when calculating the stability index based on the key stability indicators, a weighted calculation method is used, and the formula for calculating the stability index is:

[0033] ,

[0034] Where S represents the stability index, A represents service availability, L represents the average response latency, and C represents the anomaly confidence level. Weights representing business availability The penalty coefficient representing the average response delay. The penalty coefficient represents the degree of confidence in anomalies.

[0035] Preferably, in the self-healing decision generation and execution step, the step of dynamically adjusting the preset anomaly confidence threshold based on the ratio of the total number of false positives and false negatives to the total number of all repair cases within a preset time period includes: recording the repair scheme and the execution result of the repair scheme corresponding to each anomaly type; if the ratio of the total number of false positives to the total number of all repair cases within the preset time period is higher than the preset false positive safety threshold, then the anomaly confidence threshold is increased; if the ratio of the total number of false negatives to the total number of all repair cases within the preset time period is higher than the preset false negative safety threshold, then the anomaly confidence threshold is decreased.

[0036] The step of dynamically adjusting the priority of each repair scheme in the preset strategy library for each anomaly type based on the execution success rate, average repair efficiency, and post-repair stability index changes of the repair schemes corresponding to each anomaly type within a preset time period includes: for the same anomaly type, statistically analyzing the execution success rate, average repair efficiency, and average recovery speed and magnitude of the post-repair stability index of each corresponding repair scheme within the preset time period; if the average repair efficiency, stability index recovery speed, and magnitude of the first repair scheme corresponding to the anomaly type are all improved compared to the second repair scheme corresponding to the anomaly type, then the priority of the first repair scheme is adjusted to be higher than the priority of the second repair scheme; if the execution success rate of the first repair scheme corresponding to the anomaly type is lower than a preset execution success rate threshold, then the priority of the first repair scheme is reduced.

[0037] A cloud service assurance system based on eBPF observable data and automated self-healing includes, in sequence, a full-link historical observable data acquisition module, a full-link data fusion and feature extraction module, an anomaly prediction model training module, a cloud service anomaly prediction module, a self-healing decision generation and execution module, and a stability display and evaluation module; wherein,

[0038] The end-to-end historical observable data acquisition module is used to collect historical observable runtime status data of the kernel layer through eBPF kernel probes mounted on the CPU scheduling, network, storage, and kernel system call paths; to collect historical observable runtime status data of the container layer through independent sampling; and to collect historical observable runtime status data of the application layer through OpenTelemetry or Zipkin end-to-end tracing. Based on timestamps and request identifiers, the historical observable runtime status data of the kernel layer, container layer, and application layer are mapped and associated in the end-to-end end-to-end associated historical observable dataset from business requests to the system kernel.

[0039] The end-to-end data fusion and feature extraction module is used to perform structured, normalized, and time-granularized preprocessing on the historical observable operational status data of each layer associated with the end-to-end mapping; based on the timestamp and request identifier, the preprocessed data of each layer is aligned within the same time window or the same request link dimension to obtain multiple sets of historical time-series data of the same business scenario or the same time segment; the aligned sets of historical time-series data are spliced ​​together according to the time sequence or feature dimension to generate multiple sets of multi-dimensional unified time-series feature vectors containing system status and business performance with labels of the actual observation value at the next moment;

[0040] The anomaly prediction model training module is used to construct a temporal deep learning model based on a Transformer network; train and test the temporal deep learning model using the multiple sets of multidimensional unified temporal feature vectors to obtain an anomaly prediction model; calculate the residual based on the predicted value output by the anomaly prediction model during the testing period and the corresponding actual observation value at the next time moment, and obtain the residual probability distribution characteristics based on the statistical analysis of the residual.

[0041] The cloud service anomaly prediction module is used to collect real-time observable operational status data of the kernel layer, container layer, and application layer of the cloud service platform under test in real time or periodically. Based on timestamps and request identifiers, it performs end-to-end mapping and association of the real-time observable operational status data of each layer, and performs preprocessing, data alignment, and feature concatenation processing according to the same method as in the end-to-end multidimensional data fusion and feature extraction module to obtain a unified time-series feature vector of the cloud service platform under test. This unified time-series feature vector is then input into a trained and tested anomaly prediction model, which outputs a prediction result of the observable operational status data of the cloud service platform at the next moment. By calculating the residual between the prediction result and the corresponding actual observed value in the unified time-series feature vector of the cloud service platform under test, and combining the probability distribution characteristics of the residual, an anomaly confidence level corresponding to the prediction result is generated.

[0042] The self-healing decision generation and execution module is used to: determine the target feature dimension to which the maximum residual corresponding to the anomaly confidence level belongs through the interpretable method of the anomaly prediction model when the anomaly confidence level is greater than a preset anomaly confidence level threshold; determine the anomaly type corresponding to the target feature dimension by combining a preset anomaly type rule base; and determine the anomaly impact range corresponding to the target feature dimension by using dimension drill-down and topological association methods. Based on the anomaly type and anomaly impact range, it matches the optimal repair scheme in a preset strategy base and automatically executes the matched repair scheme by calling the cloud platform or Kubernetes API. It monitors the execution effect of the repair scheme and feeds it back to the anomaly prediction model to achieve adaptive fine-tuning of the anomaly prediction model.

[0043] The stability display and evaluation module is used to acquire key stability indicators, including business availability, average response latency, and anomaly confidence, in real time; calculate a stability index based on the key stability indicators; construct a two-dimensional coordinate axis with the stability index and time as the axis, and map the stability index corresponding to each time point to the two-dimensional coordinate axis, connecting adjacent data points to form a stability change curve to visually display the stability change trend of cloud services; and evaluate the execution effect of the repair solution based on the stability change curve.

[0044] The beneficial effects of this invention are as follows:

[0045] This invention provides a cloud service assurance method based on eBPF observable data and automated self-healing. This method overcomes the kernel-level data collection barrier through eBPF kernel probes, accurately capturing low-level micro-operational data such as CPU scheduling, network, storage, and kernel system calls. This breaks the "black box" dilemma of low-level performance issues, achieving zero-intrusion, high-performance collection of kernel events. Simultaneously, through layered collection (kernel layer, container layer, application layer) and end-to-end mapping, using timestamps and request identifiers as links, a complete end-to-end data link for business requests is constructed. This achieves data linkage from application-layer business performance to low-level kernel behavior, providing a foundation for subsequent model learning, anomaly prediction, and root cause analysis. This approach provides a comprehensive, coherent, and uninterrupted high-quality historical data source, effectively avoiding the analytical bias caused by single-level data. Through structured and normalized preprocessing, it eliminates format differences and noise interference between different levels of data, improving data standardization and usability. Data alignment and splicing integrates time-series data scattered across various levels into a multi-dimensional unified time-series feature vector, achieving a deep fusion of system operating status and business performance. This allows the feature vector to reflect both the operational health of the underlying system and the actual performance of the upper-level business, significantly improving the ability of subsequent anomaly prediction models to capture complex business operation patterns and providing high-quality feature support for accurate prediction. This approach utilizes a Transformer network to construct a temporal deep learning model, leveraging its ability to capture dependencies in long-term time-series data. Compared to traditional time-series models, it is better suited to the multi-dimensional and dynamic temporal characteristics of cloud services, improving prediction accuracy. Through multi-dimensional unified temporal feature vector training, residual statistics, analysis, and residual probability distribution characteristic mining, it not only achieves iterative optimization and performance verification of the model, ensuring its generalization ability and reliability, but also provides a scientific basis for the subsequent quantitative evaluation of anomaly confidence, avoiding misjudgments caused by relying solely on predicted values ​​and making anomaly identification more objective. The reuse of the previously described data processing methods ensures the consistency between the test data and training data. Consistent processing logic effectively avoids prediction bias caused by differences in processing procedures, improving the accuracy and stability of anomaly prediction. By collecting data in real time / periodically and generating anomaly confidence scores, dynamic monitoring and quantitative assessment of cloud service operation status are achieved. Compared with traditional static threshold or rule-based alarm mechanisms, potential anomaly risks can be identified in advance. By using model-interpretable methods to locate target feature dimensions, combined with rule bases, dimension drill-down, and topology correlation analysis, accurate determination of anomaly types and clear definition of impact scope are achieved, avoiding interference of repair operations with normal business. The optimal repair solution is matched based on anomaly type and impact scope and automatically executed by calling the cloud platform or Kubernetes API without manual intervention, significantly shortening the fault response and repair cycle. At the same time, accurate matching improves the repair success rate, minimizing the impact of anomalies on cloud service continuity and ensuring stable business operation.Meanwhile, the closed-loop design of feedback on repair effects and adaptive fine-tuning of the model can continuously optimize the anomaly prediction model based on actual operating conditions, improve the model's adaptability and prediction accuracy, and gradually improve the self-healing strategy. By constructing a stability index through key stability indicators and visually generating stability change curves, operations and maintenance personnel can intuitively and in real time grasp the stability trend of cloud services, breaking through the subjectivity and lag of traditional assessment methods, and realizing quantitative assessment and visual monitoring of repair effects. Based on the stability change curve, the execution results of the repair plan are evaluated, and the repair effect is quantified, providing clear data basis for the continuous optimization of operation and maintenance decisions and model strategies, and achieving long-term improvement in cloud service stability. This invention achieves penetration into the "black box" of the system kernel layer by combining eBPF technology and residual probability distribution characteristics. It involves full-link historical observable data collection, full-link data fusion and feature extraction, anomaly prediction model training, observable operational state anomaly prediction, self-healing decision generation and execution, and stability display and evaluation. This allows the anomaly prediction model to accurately locate the root cause of cloud service anomalies, predict anomalies in advance, and achieve predictive maintenance. Simultaneously, by combining dimensional drill-down and topology correlation methods, it achieves accurate identification of the type and scope of anomalies, rapid matching of repair solutions, and efficient repair. A comprehensive stability index is used to detect the execution effect of the repair solution, thereby quantitatively evaluating and optimizing the assurance effect, significantly improving the reliability of cloud service assurance.

[0046] This invention employs a priority mechanism to select the optimal repair solution, ensuring that the best solution is used for the current anomaly type and its impact scope. This shortens the fault repair cycle, improves repair efficiency, and minimizes the additional impact of repair operations on business operations, reducing resource waste and ensuring the rapid recovery capability of cloud services in fault scenarios. It uses a dual standard of data recovery achievement and stability change curve trend to determine the success or failure of repair, accurately identifying the long-term effectiveness and stability of repair operations, avoiding misjudging instantaneous recovery as success and ignoring hidden fluctuation risks after repair, thus improving the accuracy and reliability of repair result judgment. By incorporating successful cases into the training set and performing incremental training or parameter fine-tuning, a "practice-feedback-optimization" model iteration closed loop is constructed. This allows the anomaly prediction model to continuously absorb effective data from real business scenarios, dynamically adapt to changes in the cloud business operating status, and continuously improve the model's prediction accuracy and generalization ability. Dynamically adjusting the preset anomaly confidence threshold based on the proportion of false positives and false negatives allows the preset anomaly confidence threshold to accurately match the current business operating characteristics, finding the optimal balance between false positives and false negatives, improving the adaptability and flexibility of anomaly identification, and achieving adaptive optimization of the preset anomaly confidence threshold. Prioritization is adjusted based on core indicators such as execution success rate, repair efficiency, and stability index changes. This allows for the continuous elimination of inefficient and low-success-rate repair solutions, ensuring that highly adaptable and efficient repair solutions are always given high priority. This guarantees that the strategy library can keep up with business changes and the evolution of abnormal scenarios.

[0047] This invention leverages eBPF kernel probes to specifically attach to the core paths of CPU scheduling, network, storage, and system calls. It non-intrusively collects key kernel-level event data such as kernel system calls, TCP latency, and CPU scheduling congestion, overcoming the limitations of traditional monitoring methods that struggle to penetrate the underlying "black box" and collect incomplete kernel-level data. This allows for the accurate capture of underlying operational bottlenecks and potential anomalies. Simultaneously, it focuses on the core resource status of container instances and the core operational performance of services. It collects data such as container resource utilization, IO latency, and network packet loss rate through fixed-frequency periodic sampling, and collects data such as request latency and error rate of service request links through OpenTelemetry or Zipkin full-link tracing. This achieves full-link, multi-dimensional data coverage from the kernel layer, container middleware layer to the service application layer, ensuring the relevance and completeness of the collected data.

[0048] This invention performs structured processing on kernel-level event data and container-level historical observable runtime status data after end-to-end mapping and association. This transforms non-standardized and fragmented low-level data into structured data in a unified format, eliminating analytical obstacles caused by format confusion. Simultaneously, normalization processing is performed to eliminate differences in units and numerical ranges between different data, ensuring comparability and adaptability across dimensions. Furthermore, time alignment processing ensures precise temporal matching between kernel-level event data and container-level historical observable runtime status data, resolving cross-layer data temporal misalignment issues. Separate normalization processing is performed on the application-level historical observable runtime status data after end-to-end mapping and association, achieving numerical adaptation with the pre-processed data from the kernel and container layers. Ultimately, this achieves unified format, consistent units, and synchronized temporal sequence across the entire multi-dimensional data chain, effectively avoiding fusion bias caused by data heterogeneity and providing reliable data support for generating high-quality, multi-dimensional unified temporal feature vectors.

[0049] This invention matches target feature dimensions with a pre-defined rule base for anomaly types. Leveraging the pre-stored correspondence between feature dimensions and anomaly types, it achieves rapid and accurate anomaly type determination, avoiding the subjectivity and lag of manual diagnosis and improving anomaly identification efficiency. Through dimension drill-down operations, it traces source dimension attributes such as Pods, Nodes, and Services, accurately locating the underlying deployment unit of the anomaly and clarifying its source. Combining observable operational status data across the entire chain, it conducts topology correlation analysis to identify common topology nodes in the anomaly business chain, such as gateways and database instances, clarifying the propagation path and key nodes of the anomaly within the business chain. Finally, by integrating the results of both analyses, it grasps both the underlying source of the anomaly and its impact nodes in the business chain, achieving a comprehensive and accurate definition of the anomaly's impact scope. This avoids inadequate or excessive repair solutions due to biased assessment of the impact scope, providing a reliable basis for subsequently matching the optimal repair solution and controlling the repair impact boundary.

[0050] When no corresponding repair solution is found in the preset strategy library for the anomaly type and its impact range, this invention can dynamically generate a repair solution adapted to the anomaly type and its impact range. This involves: collecting and deeply analyzing kernel-level event data using diagnostic tools or eBPF kernel probes; accurately locating the root cause of the anomaly by leveraging the underlying penetration capabilities of eBPF technology; using a decision engine integrating an operational knowledge graph or causal graph to fused the root cause with real-time observable data to infer the repair action chain, fully reusing operational experience and causal logic to ensure the scientific validity and feasibility of the repair action chain; conducting pre-execution predictions through a sandbox testing environment or a trained cost / risk assessment model to anticipate the execution cost and cross-service impact of the repair strategy, effectively avoiding risks such as secondary failures and resource waste caused by repair actions, and ensuring the safety of the repair operation; and finally generating a repair solution by combining the repair action chain and assessment results, achieving targeted responses to new and complex anomalies, improving the self-healing capability of the entire process of anomaly location, solution generation, and risk prediction, and further enhancing the comprehensiveness and reliability of cloud service assurance.

[0051] This invention records repair plans and execution results according to anomaly type, and dynamically adjusts the confidence threshold based on the proportion of false positives and false negatives within a preset time period. If the proportion of false positives exceeds the preset false positive safety threshold, the threshold is increased; if the proportion of false negatives exceeds the preset false negative safety threshold, the threshold is decreased. This can accurately balance the risks of false positives and false negatives in anomaly detection, avoid the problem that fixed thresholds cannot adapt to changes in business operation status, and make the anomaly detection standard highly matched with the actual business scenario.

[0052] This invention targets the same anomaly type and conducts multi-dimensional statistical comparisons based on the success rate of repair schemes, average repair efficiency, and the recovery speed and magnitude of post-repair stability index. It prioritizes repair schemes with better performance and depriors those with lower success rates, thus enabling the selection and elimination of superior repair schemes. This ensures that high-quality and efficient repair schemes are matched and executed first, effectively improving the overall success rate and execution efficiency of self-healing repair and promoting the self-iteration and optimization of the preset strategy library.

[0053] The present invention also relates to a cloud service assurance system based on eBPF observable data and automated self-healing. This system corresponds to the aforementioned cloud service assurance method based on eBPF observable data and automated self-healing. It can be understood as a system that implements the aforementioned cloud service assurance method based on eBPF observable data and automated self-healing. This system is modularly implemented through a microservice architecture and provides services through RESTful API or gRPC interface, making it easy to integrate into existing cloud service platforms. The system comprises a full-link historical observable data acquisition module, a full-link data fusion and feature extraction module, an anomaly prediction model training module, a cloud service anomaly prediction module, a self-healing decision generation and execution module, and a stability display and evaluation module. These modules work collaboratively and offer the following advantages: 1. Deep observability and accurate root cause localization: Utilizing eBPF technology, it securely and efficiently collects low-level event data such as system calls, network, and scheduling at the kernel layer, breaking through the limitations of traditional application-layer monitoring and providing unprecedented data depth and accuracy for anomaly detection and root cause analysis; 2. Predictive rather than passive detection: Based on deep learning models such as Transformer, it performs joint analysis of multi-dimensional time-series data integrating kernel, container, and application layers, enabling early prediction of potential faults and shifting from "treating existing problems" to "preventing future problems"; 3. Intelligent closed-loop self-healing: By linking anomaly prediction results with a pre-set strategy library, it achieves a fully automated closed loop from "detection-decision-execution-verification," significantly reducing the mean time to recovery (MTBF). Recovery (MTTR) reduces reliance on manual operations and improves business availability; Fourth, quantifiable stability assessment and continuous optimization: It innovatively proposes a comprehensive stability index to quantify the system's protection effect, providing clear data basis for continuous optimization of operation and maintenance decisions and model strategies. Attached Figure Description

[0054] Figure 1 This is a flowchart of the cloud service assurance method based on eBPF observable data and automated self-healing according to the present invention.

[0055] Figure 2 This is a timing diagram of the cloud service assurance method based on eBPF observable data and automated self-healing according to the present invention.

[0056] Figure 3 This is a schematic diagram illustrating the principle of multi-dimensional observable data acquisition and correlation in this invention.

[0057] Figure 4 This is a structural block diagram of the cloud service assurance system based on eBPF observable data and automated self-healing according to the present invention. Detailed Implementation

[0058] This invention discloses a cloud service assurance method based on eBPF (extended Berkeley Packet Filter) observable data and automated self-healing, such as... Figure 1 and Figure 2 As shown, it includes the following steps:

[0059] I. Steps for collecting historical observable data across the entire link: Collect historical observable runtime status data of the kernel layer through eBPF kernel probes mounted on the CPU scheduling, network, storage, and kernel system call paths; collect historical observable runtime status data of the container layer through independent sampling; and collect historical observable runtime status data of the application layer through OpenTelemetry or Zipkin full-link tracing. Based on timestamps and request identifiers, perform full-link mapping and association of the historical observable runtime status data of the kernel layer, container layer, and application layer to form an end-to-end full-link associated historical observable dataset from business requests to the system kernel.

[0060] Specifically, such as Figure 3 As shown, the historical observable runtime state data of the kernel layer is mounted on the CPU scheduling and network (corresponding) via eBPF kernel probes. Figure 3 Network packet sending and receiving), storage (corresponding to) Figure 3The kernel layer event data collected along the storage I / O and kernel system call paths includes kernel system calls, TCP latency, CPU scheduling blockage, etc., such as sched_switch, sys_enter_write, etc.; the historical observable runtime status data of the container layer is the resource and performance data of container instances collected at fixed frequencies (such as 1s / time or 2s / time), including data on resource utilization, I / O latency, network packet loss rate, etc., and the associated fields can be Pod IP, container ID, namespace, etc.; the historical observable runtime status data of the application layer is the data of the business request chain (end-to-end chain of business requests) collected through OpenTelemetry or Zipkin full-link tracing (such as Trace ID, microservice call path, interface time, error code, etc.), and the data is analyzed to generate the request latency, error rate, etc. of the business request chain, and the associated fields can be TraceID (also known as request ID), user ID, transaction serial number, etc. Ultimately, these three elements are mapped together, using dimensions such as timestamps and request identifiers (e.g., Trace IDs) to perform a full-link mapping of historical observable runtime status data across the kernel layer, container layer, and application layer, forming a complete data association from "business request → container → kernel". For example, when a payment request times out (application layer exception), the association can pinpoint whether there is a sudden increase in IO latency in the container corresponding to the request within the same time window (container layer data), or whether there are events such as network congestion or excessively long kernel system call times in the kernel layer (kernel layer data). This allows for rapid identification of the root cause of the problem (such as application code defects, insufficient container resources, or underlying system bottlenecks), achieving end-to-end observability from business exceptions to underlying root causes.

[0061] II. Steps for end-to-end multidimensional data fusion and feature extraction: The historical observable operational status data of each layer associated with the end-to-end mapping are preprocessed by structuring, normalizing, and unifying the time granularity; based on the timestamps and request identifiers, the preprocessed data of each layer are aligned within the same time window or the same request link dimension to obtain multiple sets of historical time-series data for the same business scenario or the same time segment; the aligned sets of historical time-series data are concatenated according to the time sequence or feature dimension to generate multiple sets of multidimensional unified time-series feature vectors (including system status and business performance, such as kernel system call time, container I / O latency, request latency, error rate, etc.) with labels of the actual observation values ​​at the next moment.

[0062] The specific steps for preprocessing the historical observable operational status data of each layer after full-link mapping and association include: structuring (e.g., converting the original kernel layer event data into key-value pair format), normalizing (e.g., scaling the kernel layer event data and the historical observable operational status data of the container layer using Z-score or Min-Max methods to eliminate dimensional differences), and time alignment (i.e., unifying the time granularity, such as unifying sampling to 1 second / time) for the historical observable operational status data of the application layer after full-link mapping and association; and normalizing the historical observable operational status data of the application layer after full-link mapping and association to ensure that it is consistent with the system layer data (including kernel layer event data and historical observable operational status data of the container layer) in terms of numerical scale.

[0063] III. Anomaly Prediction Model Training Steps: Construct a temporal deep learning model based on networks such as Transformer and LSTM; train and test the temporal deep learning model using the multiple sets of multidimensional unified temporal feature vectors to obtain the anomaly prediction model; calculate the residual based on the predicted value output by the anomaly prediction model during the testing period and the corresponding actual observation value at the next time moment, and obtain the residual probability distribution characteristics based on the statistical analysis of the residual.

[0064] Specifically, the multiple sets of multidimensional unified temporal feature vectors are divided into training set and test set according to a preset ratio. The training set is used to train the temporal deep learning model to obtain a trained anomaly prediction model. The test set is used to test the trained anomaly prediction model. The residual is calculated based on the absolute value of the difference between the predicted value output by the anomaly prediction model during the test and the corresponding actual observation value at the next time moment. The residual probability distribution characteristics are obtained based on the statistical analysis of the residual.

[0065] The absolute value of the difference between the predicted value output by the anomaly prediction model during the test (hereinafter referred to as the model predicted value) and the corresponding actual observed value at the next moment (hereinafter referred to as the actual value) is the residual of this prediction. That is, the prediction residual is calculated as: Residual = |Actual value − Model predicted value|. Then, a Gaussian distribution model can be constructed based on the statistics and analysis of all residual data, thereby obtaining the residual probability distribution characteristics of the anomaly prediction model.

[0066] IV. Cloud Service Anomaly Prediction Steps: Real-time or periodically collect observable operational status data from the kernel layer, container layer, and application layer of the cloud service platform under test. Based on timestamps and request identifiers, perform end-to-end mapping and association of the real-time observable operational status data for each layer. Then, perform preprocessing, data alignment, and feature concatenation using the same methods as in the end-to-end multidimensional data fusion and feature extraction steps to obtain a unified temporal feature vector for the cloud service platform under test. Input the unified temporal feature vector into the trained and tested anomaly prediction model. The model outputs the prediction result of the observable operational status data of the cloud service platform at the next moment. Calculate the residual between the prediction result and the corresponding actual observed value in the unified temporal feature vector under test, and combine the residual probability distribution characteristics to generate the anomaly confidence level corresponding to the prediction result.

[0067] This invention utilizes the residual probability distribution characteristics to calculate the probability density value of the residual between the predicted result and the corresponding actual observed value in the multidimensional unified time-series feature vector to be measured, and converts it into anomaly confidence. The larger the residual, the lower the probability density value and the higher the anomaly confidence; the smaller the residual, the higher the probability density value and the lower the anomaly confidence.

[0068] The higher the anomaly confidence level, the higher the probability of cloud services experiencing anomalies within a certain period of time in the future. Therefore, an anomaly confidence level threshold can be preset (the initial threshold can be set based on experience, such as 95%). When the anomaly confidence level is higher than the preset anomaly confidence level threshold, it is judged as an anomaly and an anomaly alarm is issued, automatically triggering the self-healing decision generation and execution steps.

[0069] V. Self-Healing Decision Generation and Execution Steps: Under the condition that the anomaly confidence level is greater than the preset anomaly confidence level threshold, the target feature dimension to which the maximum residual corresponding to the anomaly confidence level belongs is determined through the interpretable method of the anomaly prediction model (such as SHAP value, feature importance ranking), and the anomaly type corresponding to the target feature dimension is determined in combination with the preset anomaly type rule base, and the anomaly impact range corresponding to the target feature dimension is determined by dimension drill-down and topology association methods; based on the anomaly type (such as resource bottleneck, service unavailability) and the anomaly impact range, the optimal repair solution (such as container replica scaling, service rollback, node restart) is matched in the preset strategy base, and the matched repair solution is automatically executed by calling the cloud platform or Kubernetes API; the execution effect of the repair solution is monitored and fed back to the anomaly prediction model to realize the adaptive fine-tuning of the anomaly prediction model.

[0070] Furthermore, if multiple repair solutions are matched in the preset strategy library based on the anomaly type and the anomaly impact range, the repair solution with the highest priority is selected as the optimal repair solution. Priority is stored in the preset strategy library as an attribute value of the repair solution. If no repair solution corresponding to the anomaly type and anomaly impact range is matched in the preset strategy library, or if the matched optimal repair solution fails after execution, a repair solution adapted to the anomaly type and anomaly impact range is dynamically generated, specifically including:

[0071] First, more detailed diagnostic tools or eBPF kernel probes are used to collect finer-grained kernel-level event data, and the collected kernel-level event data is analyzed in depth to locate the root cause corresponding to the anomaly confidence level.

[0072] Then, the root cause and the observable operational status data obtained from the cloud service anomaly prediction steps are input into a decision engine containing an operations and maintenance knowledge graph or causal graph, which then infers and generates a repair action chain. For example, one repair action chain is: "Storage IO latency anomaly -> Possible cause: Disk queue full -> Recommendation: Clean up or expand disk capacity".

[0073] Then, using the sandbox testing environment or a trained cost / risk assessment model, the pre-execution impact prediction analysis is performed on the remediation strategies to be executed corresponding to the remediation action chain, and the assessment results including execution cost and cross-service impact are obtained.

[0074] Finally, based on the remediation action chain and the evaluation results, a specific, executable remediation plan adapted to the anomaly type and its impact scope is generated. The generated remediation plan can automatically determine whether it needs to be reviewed by operations and maintenance personnel based on the risk scenario (urgency): in non-urgent scenarios, the remediation action is executed only after review by operations and maintenance personnel; in urgent scenarios, the remediation action is executed automatically.

[0075] Furthermore, the specific steps of determining the anomaly type corresponding to the target feature dimension by combining a preset anomaly type rule base, and determining the anomaly influence range corresponding to the target feature dimension by using dimension drill-down and topological association methods, as described in this embodiment of the invention, include:

[0076] First, based on the target feature dimension, the corresponding exception type is matched in the preset exception type rule base, which pre-stores the correspondence between each feature dimension and the exception type. For example, if the features "container I / O latency" and "kernel system call time (including read / write time)" contribute extremely highly, they can be identified as "storage I / O bottleneck" exceptions in the preset exception type rule base.

[0077] Then, a dimension drill-down operation is performed on the target feature dimension to obtain the source dimension attribute corresponding to the target feature dimension. The source dimension attribute includes at least one of the following: Pod, Node, Service, and Deployment. The influence range of these four types of source dimension attributes increases from smallest to largest. For example, to determine which Pods, Nodes, and Services the "container IO latency" anomaly data comes from, if all these anomaly containers belong to the same Deployment, then the influence range is the corresponding microservice.

[0078] Next, combining the collected end-to-end observable operational status data, a topology correlation analysis is performed on the abnormal business links associated with the target feature dimensions to identify the common topology nodes of each abnormal business link. These common topology nodes include at least one of a gateway or a database instance. For example, if it is found that all abnormal business links pass through a specific gateway or database instance, then all upstream services related to that gateway or database instance can be located.

[0079] Finally, by combining the source dimension attribute results of the drill-down tracing and the common topology node results of the topology association analysis, the scope of the anomaly impact corresponding to the target feature dimension is determined.

[0080] For example, if the target characteristic dimension (i.e., the anomalous characteristic, such as an extremely high CPU soft interrupt rate or memory allocation failure) only appears on one or a few worker nodes in the cluster, it indicates that the problem may be limited to the hardware of these nodes or the host operating system, and the impact is at the "node level". If the target characteristic dimension (such as a surge in request latency or an increase in error rate for a specific microservice) is strongly correlated with a certain Kubernetes Service Deployment or Namespace, while other services in the cluster are normal, it indicates that the problem may originate from a code update, configuration error, or failure of a dependent service of that service, and the impact is at the "service level".

[0081] Furthermore, the optimal repair scheme in this embodiment of the invention (i.e., the repair scheme in the preset strategy library) includes resource scaling up and down (the scaling up degree is determined by the value of the anomaly confidence level, including light scaling up: adding 1 Deployment replica; medium scaling up: increasing the number of Deployment replicas by 50%, such as from 4 Deployment replicas to 6 Deployment replicas; heavy scaling up: increasing the number of Deployment replicas by 100%, such as from 2 Deployment replicas to 4 Deployment replicas), service rollback (microservice rollback to the previous stable version without errors), node restart, configuration hot update, and traffic switching. The execution form of the optimal repair scheme is a set of K8s commands or API call sequences.

[0082] For example, if the exception is of the "CPU / memory resource bottleneck" type, the following remedial solutions can be matched: "Adjust the Pod's resource limits (resources.limits) to allocate more CPU or memory to the container," "Increase the number of replicas of the corresponding Deployment," or "Evict the affected Pod and schedule it to a node with more abundant resources." It should be noted that the above remedial solution descriptions are explanatory notes, and each remedial solution is executed as a sequence of Kubernetes commands or API calls.

[0083] Furthermore, the specific steps for monitoring the effectiveness of the remediation plan and feeding it back to the anomaly prediction model to achieve adaptive fine-tuning of the anomaly prediction model include:

[0084] After the optimal repair solution is executed, observable running status data of the kernel layer, container layer, and application layer are collected. If all these data are restored to the preset normal observable running status threshold range, and the stability change curve within the corresponding time period shows a steady upward trend over time, then the optimal repair solution is determined to have been executed successfully. If the data after execution is not fully restored to the preset normal observable running status threshold range, or the stability change curve within the corresponding time period does not show a steady upward trend over time, then the optimal repair solution is determined to have failed.

[0085] Based on the execution results of this optimal repair scheme, the observable runtime status data collected during the execution period of the optimal repair scheme, including the kernel layer, container layer, and application layer, as well as their corresponding target feature dimensions and the optimal repair scheme, are marked: if the execution is successful, it is marked as a successful case; if the execution fails, it is marked as a failed case. The failed cases include false positive cases, false negative cases, and cases where the repair is ineffective; these successful or failed cases are stored.

[0086] The successful cases are used as reinforcement data for the normal state and added to the training set of the anomaly prediction model to help the model better learn the "recovery after anomaly" pattern; the anomaly prediction model is incrementally trained or its parameters are fine-tuned periodically using the training set containing the successful cases to adapt it to the latest state and dynamic changes of the cloud business platform.

[0087] The preset anomaly confidence threshold is dynamically adjusted based on the ratio of the total number of false positives and false negatives to the total number of all repair cases within a preset time period. Specific steps include: recording the repair schemes and execution results corresponding to each anomaly type; if the ratio of the total number of false positives to the total number of all repair cases within the preset time period is higher than the preset false positive safety threshold (i.e., the self-healing decision generation and execution process is frequently triggered, but subsequent verification shows the cloud business platform itself has no real problems, leading to unnecessary resource consumption and operational interference), it indicates that the anomaly prediction model is too sensitive, and the anomaly confidence threshold should be increased to reduce the sensitivity of the anomaly prediction model; if the ratio of the total number of false negatives to the total number of all repair cases within the preset time period is higher than the preset false negative safety threshold (i.e., the real fault was not predicted by the anomaly prediction model but was discovered through other means (such as business alarms), it indicates that the anomaly prediction model may not be sensitive enough, and the anomaly confidence threshold should be decreased to capture weaker anomaly signals);

[0088] Based on the execution success rate, average repair efficiency, and post-repair stability index changes of the repair solutions corresponding to each anomaly type within a preset time period, the priority of each repair solution under the corresponding anomaly type in the preset strategy library is dynamically adjusted. Specific steps include: for the same anomaly type, statistically analyzing the execution success rate, average repair efficiency, and average recovery speed and magnitude of the post-repair stability index of each corresponding repair solution within the preset time period; if the average repair efficiency, stability index recovery speed, and magnitude of the first repair solution corresponding to the anomaly type are all improved compared to the second repair solution corresponding to the anomaly type, then the priority of the first repair solution is adjusted to be higher than the priority of the second repair solution; if the execution success rate of the first repair solution corresponding to the anomaly type is lower than a preset execution success rate threshold, then the priority of the first repair solution is reduced.

[0089] It is worth noting that if the optimal remediation plan is marked as a successful case, and this optimal remediation plan was dynamically generated, then the execution plan, its corresponding exception type, and the scope of its impact will be added to the preset policy library to achieve dynamic updates of the preset policy library. The preset policy library also supports custom updates by operations and maintenance personnel, and has good scalability.

[0090] VI. Stability Display and Evaluation Steps: Real-time acquisition of key stability indicators, including service availability, average response latency, and anomaly confidence; calculation of a stability index based on these key stability indicators; construction of a two-dimensional coordinate axis using the stability index and time as the axis, mapping the stability index corresponding to each time point to the two-dimensional coordinate axis, connecting adjacent data points to form a stability change curve, thus visually displaying the stability change trend of cloud services; and evaluation of the implementation effect of the remediation plan based on the stability change curve.

[0091] In this embodiment of the invention, when calculating the stability index based on the aforementioned key stability indicators, a weighted calculation method is used. The formula for calculating the stability index is as follows:

[0092] ,

[0093] Where S represents the stability index, A represents business availability, derived from the cloud business platform's operational status after the remediation plan is implemented (e.g., whether services have returned to normal and resources are available), L represents average response latency, derived from historical observable operational status data at the application layer (e.g., time-series statistics of request latency), and C represents anomaly confidence. The weights representing business availability increase the stability index S. The penalty coefficient representing the average response delay. The penalty coefficient represents the anomaly confidence level. The higher the average response delay penalty coefficient or the larger the anomaly confidence level penalty coefficient, the smaller the stability index S.

[0094] The embodiments of the present invention can adjust the weights according to the needs of the business scenario; for example, in a financial payment scenario, the weights can be increased. Prioritizing business availability, low-latency services can increase [the potential for growth]. (To strictly control the average response delay); the weighting coefficients can be dynamically optimized through the stability change curve. For example, when a certain type of anomaly causes the stability index S to continuously decrease, the weighting coefficients can be increased. To enhance sensitivity to this type of anomaly.

[0095] This invention can evaluate the effectiveness of a repair solution based on a stability change curve, i.e., the overall health of the cloud service platform after repair, providing decision support for operations and maintenance personnel. For example... Figure 2 As shown, this invention realizes a complete automated process from data acquisition to closed-loop feedback. The operating status of the cloud business platform is monitored and analyzed in real time. Once the anomaly prediction model predicts a potential anomaly, the self-healing decision generation and execution steps will immediately intervene and perform repair actions. Finally, the effectiveness of the entire "data acquisition → data fusion and feature extraction → anomaly detection → self-healing decision" chain of this intervention is evaluated in reverse through the stability index, forming a continuously optimized intelligent operation and maintenance closed loop.

[0096] Based on the same inventive concept, one or more embodiments of this specification also provide a cloud service assurance system based on eBPF observable data and predictive models. Since the principle of solving the problem by the cloud service assurance system based on eBPF observable data and predictive models is similar to that of the aforementioned cloud service assurance method based on eBPF observable data and predictive models, the implementation of the cloud service assurance system based on eBPF observable data and predictive models can refer to the aforementioned implementation of the cloud service assurance method based on eBPF observable data and predictive models, and the repeated parts will not be described again.

[0097] Figure 4 This is a block diagram illustrating the structure of a cloud service assurance system based on eBPF observable data and automated self-healing, provided for one or more embodiments of this specification. Figure 4 As shown, the cloud service assurance system based on eBPF observable data and automated self-healing includes, in sequence, a full-link historical observable data acquisition module 1, a full-link data fusion and feature extraction module 2, an anomaly prediction model training module 3, a cloud service anomaly prediction module 4, a self-healing decision generation and execution module 5, and a stability display and evaluation module 6. Among these,

[0098] The end-to-end historical observable data acquisition module 1 includes a kernel probe unit 101, a container monitoring unit 102, and an application tracing unit 103. The kernel probe unit 101 is used to collect historical observable runtime status data of the kernel layer through eBPF kernel probes mounted on the CPU scheduling, network, storage, and kernel system call paths. The container monitoring unit 102 is used to collect historical observable runtime status data of the container layer through independent sampling. The application tracing unit 103 is used to collect historical observable runtime status data of the application layer through OpenTelemetry or Zipkin end-to-end tracing, and to perform end-to-end mapping and association of the historical observable runtime status data of the kernel layer, container layer, and application layer based on timestamps and request identifiers to form an end-to-end end-to-end associated historical observable dataset from business requests to the system kernel.

[0099] The end-to-end data fusion and feature extraction module 2 includes a system indicator processing unit 201 and an application indicator fusion unit 202. The system indicator processing unit 201 is used to perform structured, normalized, and time-granular unified preprocessing on the historical observable operational status data of each layer associated with the end-to-end mapping. The application indicator fusion unit 202 is used to align the preprocessed data of each layer under the same time window or the same request link dimension based on the timestamp and request identifier to obtain multiple sets of historical time-series data of the same business scenario or the same time segment. The aligned sets of historical time-series data are spliced ​​according to the time sequence or feature dimension to generate multiple sets of multi-dimensional unified time-series feature vectors containing system status and business performance with labels of the actual observation value at the next moment.

[0100] The anomaly prediction model training module 3 includes a temporal modeling unit 301, which is used to construct a temporal deep learning model based on a Transformer network; to train and test the temporal deep learning model using the multiple sets of multidimensional unified temporal feature vectors to obtain an anomaly prediction model; to calculate the residual based on the predicted value output by the anomaly prediction model during the test and the corresponding actual observation value at the next time moment, and to obtain the residual probability distribution characteristics based on the statistical analysis of the residual;

[0101] The cloud service anomaly prediction module 4 includes an anomaly prediction unit 401, which is used to collect real-time observable operational status data of the kernel layer, container layer, and application layer of the cloud service platform under test in real time or periodically. Based on timestamps and request identifiers, the real-time observable operational status data of each layer are mapped and associated in the entire link. The data is then preprocessed, aligned, and feature-stitched using the same method as in the entire link multidimensional data fusion and feature extraction module to obtain a multidimensional unified temporal feature vector under test. The multidimensional unified temporal feature vector under test is input into the trained and tested anomaly prediction model, and the model outputs the prediction result of the observable operational status data of the cloud service platform at the next moment. By calculating the residual between the prediction result and the corresponding actual observation value in the multidimensional unified temporal feature vector under test, and combining the probability distribution characteristics of the residual, the anomaly confidence level corresponding to the prediction result is generated.

[0102] The self-healing decision generation and execution module 5 includes a strategy generation unit 501, an execution engine unit 502, and a verification feedback unit 503. The strategy generation unit 501 is used to determine the target feature dimension to which the maximum residual corresponding to the anomaly confidence level belongs, using an interpretable method of the anomaly prediction model, when the anomaly confidence level is greater than a preset anomaly confidence level threshold. It then determines the anomaly type corresponding to the target feature dimension by combining a preset anomaly type rule base, and uses dimensional drill-down and topological association methods to determine the anomaly impact range corresponding to the target feature dimension. The execution engine unit 502 is used to match the optimal repair scheme in the preset strategy base based on the anomaly type and anomaly impact range, and automatically execute the matched repair scheme by calling a cloud platform or Kubernetes API. The verification feedback unit 503 is used to monitor the execution effect of the repair scheme and feed it back to the anomaly prediction model, enabling adaptive fine-tuning of the anomaly prediction model.

[0103] The stability display and evaluation module 6 includes an indicator sampling unit 601, a weight calculation unit 602, and a trend analysis unit 603. The indicator sampling unit 601 is used to acquire key stability indicators, including business availability, average response latency, and anomaly confidence, in real time. The weight calculation unit 602 is used to calculate a stability index based on the key stability indicators. The trend analysis unit 603 constructs a two-dimensional coordinate axis with the stability index and time as the axis, maps the stability index corresponding to each time point to the two-dimensional coordinate axis, connects adjacent data points to form a stability change curve, visualizes the stability change trend of cloud services, and evaluates the execution effect of the repair solution based on the stability change curve.

[0104] It should be noted that the specific embodiments described above enable those skilled in the art to more fully understand the present invention, but do not limit the present invention in any way. Therefore, although the present invention has been described in detail with reference to the accompanying drawings and embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the present invention. In short, all technical solutions and improvements that do not depart from the spirit and scope of the present invention should be covered within the protection scope of the present invention patent.

Claims

1. A cloud service assurance method based on eBPF observable data and automated self-healing, characterized in that, Includes the following steps: The steps for collecting historical observable data across the entire link are as follows: historical observable runtime status data of the kernel layer is collected by eBPF kernel probes mounted on the CPU scheduling, network, storage and kernel system call paths; historical observable runtime status data of the container layer is collected by independent sampling; and historical observable runtime status data of the application layer is collected by OpenTelemetry or Zipkin full-link tracing. Based on timestamps and request identifiers, the historical observable runtime status data of the kernel layer, container layer, and application layer are mapped and associated across the entire link to form an end-to-end historical observable dataset associated with the business request to the system kernel. The steps for end-to-end multidimensional data fusion and feature extraction are as follows: The historical observable operational status data of each layer associated with the end-to-end mapping are preprocessed by structuring, normalizing and unifying the time granularity. Based on the timestamp and request identifier, the preprocessed data of each layer are aligned under the same time window or the same request link dimension to obtain multiple sets of historical time series data of the same business scenario or the same time segment; the aligned sets of historical time series data are spliced ​​according to the time sequence order or feature dimension to generate multiple sets of multidimensional unified time series feature vectors containing system status and business performance with labels of the actual observation value of the next moment. Anomaly prediction model training steps: Construct a temporal deep learning model based on the Transformer network; Train and test the temporal deep learning model using the multiple sets of multidimensional unified temporal feature vectors to obtain the anomaly prediction model; The residuals are calculated based on the predicted values ​​output by the anomaly prediction model during the test and the corresponding actual observation values ​​at the next time step, and the residual probability distribution characteristics are obtained based on the statistical analysis of the residuals. Cloud service anomaly prediction steps: Real-time or periodic collection of observable operational status data of the kernel layer, container layer, and application layer of the cloud service platform under test; full-link mapping and association of the real-time observable operational status data of each layer based on timestamps and request identifiers; preprocessing, data alignment, and feature concatenation processing according to the same method as in the full-link multidimensional data fusion and feature extraction steps to obtain a unified time-series feature vector of the cloud service platform under test; inputting the unified time-series feature vector of the cloud service platform under test into a trained and tested anomaly prediction model; the model outputs the prediction result of the observable operational status data of the cloud service platform at the next moment; by calculating the residual between the prediction result and the corresponding actual observation value in the unified time-series feature vector of the cloud service platform under test, and combining the probability distribution characteristics of the residual, generating the anomaly confidence level corresponding to the prediction result; The self-healing decision generation and execution steps are as follows: Under the condition that the anomaly confidence level is greater than a preset anomaly confidence level threshold, the target feature dimension to which the maximum residual corresponding to the anomaly confidence level belongs is determined through the interpretable method of the anomaly prediction model. The anomaly type corresponding to the target feature dimension is determined by combining it with a preset anomaly type rule base. The anomaly impact range corresponding to the target feature dimension is determined using dimension drill-down and topological association methods. Based on the anomaly type and anomaly impact range, the optimal repair solution is matched in a preset strategy base, and the matched repair solution is automatically executed by calling a cloud platform or Kubernetes API. The execution effect of the repair solution is monitored and fed back to the anomaly prediction model to achieve adaptive fine-tuning of the anomaly prediction model. Stability demonstration and evaluation steps: Real-time acquisition of key stability indicators, including business availability, average response latency, and anomaly confidence; calculation of the stability index based on the key stability indicators; A two-dimensional coordinate axis is constructed using the stability index and time as the axis, and the stability index corresponding to each time point is mapped to the two-dimensional coordinate axis. Adjacent data points are connected to form a stability change curve to visualize the stability change trend of cloud services. The execution effect of the repair scheme is evaluated based on the stability change curve.

2. The method according to claim 1, characterized in that, In the self-healing decision generation and execution step, if multiple repair solutions are matched in the preset strategy library based on the anomaly type and the anomaly impact range, the repair solution with the highest priority is selected as the optimal repair solution. The monitoring and repair scheme's execution effect and its feedback to the anomaly prediction model, enabling adaptive fine-tuning of the anomaly prediction model, includes: collecting observable operational status data of the kernel layer, container layer, and application layer after the optimal repair scheme is executed; if the data after execution has all recovered to within the preset normal observable operational status threshold range, and the stability change curve within the corresponding time period shows a steady upward trend over time, then the optimal repair scheme is determined to have been executed successfully; if the data after execution has not all recovered to within the preset normal observable operational status threshold range, or the stability change curve within the corresponding time period does not show a steady upward trend over time, then the optimal repair scheme is determined to have failed. Based on the execution results of this optimal repair scheme, the observable runtime status data collected during the execution period of the optimal repair scheme, including the kernel layer, container layer, and application layer, as well as their corresponding target feature dimensions and the optimal repair scheme, are marked: if the execution is successful, it is marked as a successful case; if the execution fails, it is marked as a failed case. The failed cases include false positive cases, false negative cases, and cases where the repair is ineffective. The successful cases are added to the training set of the anomaly prediction model, and the anomaly prediction model is incrementally trained or its parameters are fine-tuned periodically using the training set containing the successful cases. The preset anomaly confidence threshold is dynamically adjusted based on the ratio of the total number of false alarms and false negatives within a preset time period to the total number of all repaired cases within that time period. Based on the success rate, average repair efficiency, and changes in the stability index of the repair solutions corresponding to each anomaly type within a preset time period, the priority of each repair solution under the corresponding anomaly type in the preset strategy library is dynamically adjusted.

3. The method according to claim 1 or 2, characterized in that, In the full-link historical observable data collection step, the historical observable runtime status data of the kernel layer is kernel layer event data including kernel system calls, TCP latency, and CPU scheduling blockage, collected through eBPF kernel probes; the historical observable runtime status data of the container layer is data including container instance resource utilization, IO latency, and network packet loss rate, collected through fixed-frequency timed sampling; and the historical observable runtime status data of the application layer is data including request latency and error rate of the business request link, collected through OpenTelemetry or Zipkin full-link tracing.

4. The method according to claim 3, characterized in that, In the full-link multidimensional data fusion and feature extraction step, the preprocessing step of structuring, normalizing, and unifying the time granularity of the historical observable running status data of each layer after full-link mapping includes: structuring, normalizing, and time-aligning the kernel layer event data and container layer historical observable running status data after full-link mapping; and normalizing the application layer historical observable running status data after full-link mapping.

5. The method according to claim 1 or 2, characterized in that, In the self-healing decision generation and execution step, the step of determining the anomaly type corresponding to the target feature dimension by combining a preset anomaly type rule base, and determining the anomaly influence range corresponding to the target feature dimension by using dimension drill-down and topological association methods includes: Based on the target feature dimension, the corresponding anomaly type is matched in the preset anomaly type rule base, which pre-stores the correspondence between each feature dimension and the anomaly type. Perform a dimension drill-down operation on the target feature dimension to obtain the source dimension attribute corresponding to the target feature dimension. The source dimension attribute includes at least one of Pod, Node, Service, and Deployment deployment unit. By combining the collected end-to-end observable operational status data, a topology correlation analysis is performed on the abnormal business links associated with the target feature dimensions to identify the common topology nodes of each abnormal business link. The common topology nodes include at least one of a gateway and a database instance. By combining the source dimension attribute results of the drill-down tracing and the common topology node results of the topology association analysis, the scope of the abnormal influence corresponding to the target feature dimension is determined.

6. The method according to claim 1 or 2, characterized in that, In the self-healing decision generation and execution step, if no repair solution corresponding to the anomaly type and the scope of anomaly impact is found in the preset strategy library, a repair solution adapted to the anomaly type and the scope of anomaly impact is dynamically generated, specifically including: Kernel layer event data is collected using diagnostic tools or eBPF kernel probes, and the collected kernel layer event data is analyzed in depth to locate the root cause corresponding to the anomaly confidence level. The root cause and the observable operational status data obtained from the cloud service anomaly prediction steps are input into a decision engine containing an operation and maintenance knowledge graph or causal graph, and the decision engine infers and generates a repair action chain. Using a sandbox testing environment or a trained cost / risk assessment model, the pre-execution impact prediction analysis is performed on the remediation strategies to be executed corresponding to the remediation action chain, and the assessment results including execution cost and cross-service impact are obtained. Based on the repair action chain and the evaluation results, a repair plan adapted to the anomaly type and the scope of its impact is generated.

7. The method according to claim 1 or 2, characterized in that, In the self-healing decision generation and execution steps, the optimal repair solution includes resource scaling, service rollback, node restart, configuration hot update, and traffic switching. The execution form of the optimal repair solution is a set of K8s commands or API call sequences.

8. The method according to claim 1 or 2, characterized in that, In the stability demonstration and evaluation step, when calculating the stability index based on the key stability indicators, a weighted calculation method is used. The formula for calculating the stability index is as follows: ， Where S represents the stability index, A represents service availability, L represents the average response latency, and C represents the anomaly confidence level. Weights representing business availability The penalty coefficient representing the average response delay. The penalty coefficient represents the degree of confidence in anomalies.

9. The method according to claim 2, characterized in that, In the self-healing decision generation and execution step, the step of dynamically adjusting the preset anomaly confidence threshold based on the ratio of the total number of false positives and false negatives to the total number of all repair cases within a preset time period includes: recording the repair scheme and the execution result of each anomaly type; if the ratio of the total number of false positives to the total number of all repair cases within the preset time period is higher than the preset false positive safety threshold, then the anomaly confidence threshold is increased; if the ratio of the total number of false negatives to the total number of all repair cases within the preset time period is higher than the preset false negative safety threshold, then the anomaly confidence threshold is decreased. The step of dynamically adjusting the priority of each repair scheme in the preset strategy library for each anomaly type based on the execution success rate, average repair efficiency, and post-repair stability index changes of the repair schemes corresponding to each anomaly type within a preset time period includes: for the same anomaly type, statistically analyzing the execution success rate, average repair efficiency, and average recovery speed and magnitude of the post-repair stability index of each corresponding repair scheme within the preset time period; if the average repair efficiency, stability index recovery speed, and magnitude of the first repair scheme corresponding to the anomaly type are all improved compared to the second repair scheme corresponding to the anomaly type, then the priority of the first repair scheme is adjusted to be higher than the priority of the second repair scheme; if the execution success rate of the first repair scheme corresponding to the anomaly type is lower than a preset execution success rate threshold, then the priority of the first repair scheme is reduced.

10. A cloud service assurance system based on eBPF observable data and automated self-healing, characterized in that, This includes, in sequence, a full-link historical observable data acquisition module, a full-link data fusion and feature extraction module, an anomaly prediction model training module, a cloud service anomaly prediction module, a self-healing decision generation and execution module, and a stability display and evaluation module; among which, The end-to-end historical observable data acquisition module is used to collect historical observable runtime status data of the kernel layer through eBPF kernel probes mounted on the CPU scheduling, network, storage, and kernel system call paths; to collect historical observable runtime status data of the container layer through independent sampling; and to collect historical observable runtime status data of the application layer through OpenTelemetry or Zipkin end-to-end tracing. Based on timestamps and request identifiers, the historical observable runtime status data of the kernel layer, container layer, and application layer are mapped and associated in the end-to-end end-to-end associated historical observable dataset from business requests to the system kernel. The end-to-end data fusion and feature extraction module is used to perform structured, normalized, and time-granularized preprocessing on the historical observable operational status data of each layer associated with the end-to-end mapping; based on the timestamp and request identifier, the preprocessed data of each layer is aligned within the same time window or the same request link dimension to obtain multiple sets of historical time-series data of the same business scenario or the same time segment; the aligned sets of historical time-series data are spliced ​​together according to the time sequence or feature dimension to generate multiple sets of multi-dimensional unified time-series feature vectors containing system status and business performance with labels of the actual observation value at the next moment; The anomaly prediction model training module is used to construct a temporal deep learning model based on a Transformer network; train and test the temporal deep learning model using the multiple sets of multidimensional unified temporal feature vectors to obtain an anomaly prediction model; calculate the residual based on the predicted value output by the anomaly prediction model during the testing period and the corresponding actual observation value at the next time moment, and obtain the residual probability distribution characteristics based on the statistical analysis of the residual. The cloud service anomaly prediction module is used to collect real-time observable operational status data of the kernel layer, container layer, and application layer of the cloud service platform under test in real time or periodically. Based on timestamps and request identifiers, it performs end-to-end mapping and association of the real-time observable operational status data of each layer, and performs preprocessing, data alignment, and feature concatenation processing according to the same method as in the end-to-end multidimensional data fusion and feature extraction module to obtain a unified time-series feature vector of the cloud service platform under test. This unified time-series feature vector is then input into a trained and tested anomaly prediction model, which outputs a prediction result of the observable operational status data of the cloud service platform at the next moment. By calculating the residual between the prediction result and the corresponding actual observed value in the unified time-series feature vector of the cloud service platform under test, and combining the probability distribution characteristics of the residual, an anomaly confidence level corresponding to the prediction result is generated. The self-healing decision generation and execution module is used to: determine the target feature dimension to which the maximum residual corresponding to the anomaly confidence level belongs through the interpretable method of the anomaly prediction model when the anomaly confidence level is greater than a preset anomaly confidence level threshold; determine the anomaly type corresponding to the target feature dimension by combining a preset anomaly type rule base; and determine the anomaly impact range corresponding to the target feature dimension by using dimension drill-down and topology association methods. Based on the anomaly type and anomaly impact range, it matches the optimal repair scheme in a preset strategy base and automatically executes the matched repair scheme by calling the cloud platform or Kubernetes API. It monitors the execution effect of the repair scheme and feeds it back to the anomaly prediction model to achieve adaptive fine-tuning of the anomaly prediction model. The stability display and evaluation module is used to acquire key stability indicators, including business availability, average response latency, and anomaly confidence, in real time; calculate a stability index based on the key stability indicators; construct a two-dimensional coordinate axis with the stability index and time as the axis, and map the stability index corresponding to each time point to the two-dimensional coordinate axis, connecting adjacent data points to form a stability change curve to visually display the stability change trend of cloud services; and evaluate the execution effect of the repair solution based on the stability change curve.

Images