Cloud native fault diagnosis method and system based on enhanced fine tuning of large model agent

Abstract

The invention discloses a cloud native fault diagnosis method and system based on large model intelligent agent enhanced fine tuning, and belongs to the technical field of intelligent operation and maintenance and artificial intelligence. The system comprises a data preprocessing module which is used for actively injecting a fault into a cloud native environment and preprocessing multi-modal operation and maintenance data before and after the fault occurs; the scene construction module is used for constructing a fault scene sample based on the preprocessed multi-modal operation and maintenance data; the virtual environment module is used for constructing a fault diagnosis interaction environment; and the agent training module is used for optimizing a diagnosis strategy through model fine tuning and reinforcement learning. According to the method, high-accuracy, high-efficiency and interpretable fault root cause positioning of the cloud native application can be realized.

Description

Technical Field

[0001] This invention relates to the fields of intelligent operation and maintenance and artificial intelligence technology, specifically to a cloud-native fault diagnosis method and system based on large model intelligent agent enhancement and fine-tuning. Background Technology

[0002] With the widespread application of cloud-native technologies, modern information technology systems are exhibiting highly distributed, dynamic, and complex characteristics. Cloud-native applications, represented by microservice architectures, typically consist of dozens or even hundreds of interdependent services, and their operational status is monitored through various observable data such as metrics, logs, and distributed tracing. When a system failure occurs, operations and maintenance personnel need to quickly locate the root cause from massive amounts of multimodal data, which poses a significant challenge to traditional operations and maintenance models.

[0003] Observability data collection for cloud-native systems has gradually become standardized. In the field of metric monitoring, Prometheus, based on a pull model and multi-dimensional labeling system, has become the de facto standard, but it faces scalability bottlenecks in ultra-large-scale cluster scenarios. Regarding distributed tracing, OpenTelemetry achieves cross-system link association through a unified API and protocol specification, providing a foundation for fault propagation path analysis. Log aggregation mainly relies on the EFK technology stack for centralized processing. However, these data exhibit significant differences in time precision, sampling frequency, and format specifications, requiring standardization before they can be used for intelligent analysis.

[0004] Traditional fault diagnosis methods mainly include rule-based expert systems and machine learning-based statistical models. Rule-based methods automate the process through predefined diagnostic procedures and threshold-based alarms, but their adaptability is limited by the rule coverage and they perform poorly when faced with unknown fault types. Traditional machine learning methods, such as random forests and support vector machines, can learn fault patterns from historical data, but their feature engineering relies on manual design and they struggle to handle unstructured log text and complex service dependencies.

[0005] Breakthroughs in large language models have opened up new possibilities for fault diagnosis. While general-purpose large language models possess powerful reasoning and language understanding capabilities, their knowledge primarily comes from publicly available internet corpora, limiting their understanding of cloud-native operations and maintenance. Efficient parameter fine-tuning techniques make it possible to adapt large models to specific domains with limited computing resources. Fault diagnosis is essentially a sequential decision-making process, requiring an agent to dynamically select analysis tools based on system state, progressively collect evidence, and ultimately determine the root cause. Reinforcement learning, through interaction with the environment and learning from reward signals, can optimize the agent's decision-making strategies. In recent years, algorithms such as PPO, DPO, and GRPO have made significant progress in the reinforcement fine-tuning of large language models, providing a theoretical foundation for building autonomous diagnostic agents.

[0006] The existing technology has the following main problems: First, standardizing multimodal data and constructing high-quality training samples are challenging. Metrics, logs, and link data generated in cloud-native environments suffer from inconsistencies in temporal precision, sampling frequencies, and format heterogeneity. Existing methods lack a unified cross-modal temporal alignment mechanism and efficient large-scale data processing capabilities. More critically, while existing fault injection techniques can generate fault data, they lack a systematic method for constructing scenario samples. They exhibit significant shortcomings in areas such as call chain causal filtering, fault propagation pattern recognition, and ensuring sample diversity, leading to incomplete training data coverage and a high risk of data leakage, severely impacting the model's generalization ability and diagnostic accuracy.

[0007] Secondly, large language models suffer from insufficient domain adaptability and reliability in fault diagnosis tasks. While general-purpose large language models possess powerful reasoning capabilities, their understanding of cloud-native operations and maintenance expertise is limited. Existing hint engineering and simple fine-tuning methods struggle to deeply infuse domain knowledge into the model. More seriously, the model is prone to generating illusions during the diagnostic process, providing unverified root cause speculations. The lack of effective constraint mechanisms to ensure the traceability and reliability of diagnostic conclusions makes it difficult to apply the diagnostic results in practical production environments.

[0008] Finally, cultivating the sequential decision-making ability of intelligent agents faces multiple challenges. Fault diagnosis is essentially a multi-step sequential decision-making process, requiring intelligent agents to dynamically select analysis tools and gradually collect evidence based on the system state. Existing methods mainly rely on supervised learning to imitate expert behavior, but expert data is scarce and labeling is costly, making it difficult to cover complex and ever-changing fault scenarios. More importantly, intelligent agents lack effective constraints on the rationality of their decisions during the diagnostic process, easily generating logically inconsistent tool call sequences, and struggling to learn autonomous exploration strategies in unknown scenarios. This prevents intelligent agents from achieving a progressive improvement from basic interaction norms to accurate diagnostic capabilities.

[0009] Therefore, designing a cloud-native application fault diagnosis method that can efficiently process multimodal operation and maintenance data, construct high-quality training scenarios, achieve efficient domain adaptation, and realize the autonomous evolution of intelligent agents through reinforcement learning has important practical significance and application value. Summary of the Invention

[0010] This invention provides a cloud-native fault diagnosis method and system based on large-model intelligent agent reinforcement fine-tuning. Through multimodal data standardization processing, scenario-based training data construction, efficient parameter fine-tuning, and reinforcement learning-driven intelligent agent evolution, it achieves high accuracy, high efficiency, and interpretable fault root cause localization.

[0011] To achieve the above objectives, the present invention adopts the following technical solution.

[0012] A cloud-native fault diagnosis method based on large-model intelligent agent reinforcement fine-tuning, characterized in that the method includes: Actively inject faults into the cloud-native environment and preprocess the multimodal operation and maintenance data before and after the fault occurs; Based on the preprocessed multimodal operation and maintenance data, a sample of fault scenarios is constructed; Build an interactive environment for fault diagnosis; The teacher model is used to perform multiple rounds of interaction based on the fault scenario samples in the fault diagnosis interactive environment to generate expert interaction trajectories, and the base model is then subjected to format specification supervision and fine-tuning based on the expert interaction trajectories. The dialogue prompts are input into the base model after format specification supervision and fine-tuning for inference, and the tool call instructions obtained from the inference are executed in the virtual environment to obtain environmental feedback and reward signals. Then, the reasoning process of the base model, the tool call instructions, the environmental feedback and reward signals are packaged into an interaction trajectory. The dialogue prompts include: system instructions, virtual environment state description and tool call history. By combining the interaction trajectory with the format specification-supervised fine-tuning of the base model, a multi-stage reinforcement fine-tuning is performed to obtain the optimized fault diagnosis large model; among which, the stage includes: format constraint reinforcement learning stage and ranking selection reinforcement learning stage; The optimized fault diagnosis model is used to perform fault diagnosis on the cloud-native application under test.

[0013] Furthermore, the multimodal data includes: indicator data, log data, and link data; Based on the preprocessed multimodal operation and maintenance data, fault scenario samples are constructed, including: The preprocessed multimodal operation and maintenance data is encapsulated into standardized experimental data objects; wherein, the standardized experimental data objects include: scenario metadata, standardized indicator data, standardized log data, and standardized call chain data, and the scenario metadata includes: fault unique identifier, timestamp, fault level, fault cause, and fault component name; Filter standardized call chain data based on scenario metadata; For the filtered standardized call chain data, extract the call chain structure features; Based on the call chain structure characteristics, the call chain call pattern is obtained, and the call chain is sampled according to the call pattern to generate several fault scenario samples; Using a fixed random seed, the fault scenario samples are divided into training scenario set, validation scenario set, and test scenario set.

[0014] Furthermore, the standardized call chain data is filtered based on scenario metadata, including: Based on the timestamp, filter out standardized call chain data prior to the fault occurrence; The number of call steps in the call chain is calculated based on the standardized call chain data, and standardized call chain data with a number of call steps less than the minimum complexity threshold is filtered out. Filter out standardized call chain data that are not associated with the faulty component at the container level, service level, or node level.

[0015] Furthermore, based on the call chain structure characteristics, the call pattern of the call chain is obtained, and call chain sampling is performed according to the call pattern to generate several fault scenario samples, including: Cluster the call chain structure features; where each cluster corresponds to a call pattern; Calculate the sampling quota for each cluster based on the total sampling budget and the number of clusters; Independent random sampling is performed in each cluster according to the sampling quota, and corresponding fault scenario samples are generated based on the sampling results.

[0016] Furthermore, the tool invocation instructions include: an indicator query tool, a log analysis tool, and a result submission tool; The step of executing the inference-derived tool invocation instructions in a virtual environment to obtain environmental feedback and reward signals includes: When the tool call command is an indicator query tool or a log analysis tool, a process reward score is assigned by determining whether the environmental feedback returned by the indicator query tool or the log analysis tool is valid feedback. When the tool invocation instruction is a result submission tool, the final result reward score is obtained by determining whether the environmental feedback returned by the result submission tool can pass the evidence-based constraint check and calculating the ranking of the environmental feedback in the output list.

[0017] Furthermore, by determining whether the environmental feedback returned by the result submission tool passes the evidence-based constraint check, and calculating the ranking of the environmental feedback in the output list, the final result reward score is obtained, including: The reward score is calculated based on the environmental feedback; Determine if the true root cause is being explored along the path; If the true root cause is not in the path of exploration, the final result reward score is set to zero; If the true root cause is in the exploration path, the recall score is calculated based on the ranking of the original result reward score in the output list. After applying time decay penalty and hallucination penalty to the recall score, the final result reward score is obtained. The time decay penalty is used to penalize the number of diagnostic rounds, and the hallucination penalty is used to negatively penalize unexplored components and duplicate components included in the output list.

[0018] Furthermore, the execution process of the format constraint reinforcement learning phase includes: Based on the legality and validity of tool call instructions and the closed-loop nature of diagnostic tasks, a format scorer is initialized; Parallel virtual environment instances are generated and divided into several groups; within each group, the virtual environment instances share the same fault scenario sample. The format specification supervises the fine-tuning of the base model to interact with each virtual environment instance, generating several complete trajectories, and using the format scorer to obtain the process reward score for each complete trajectory; Based on the process reward scores of all complete trajectories within the group, calculate the advantage value of each complete trajectory relative to the group average. Based on the aforementioned dominance value, the base model parameters are adjusted by maximizing the generation probability of high-dominance trajectories and suppressing the generation probability of low-dominance trajectories.

[0019] Furthermore, the execution process of the ranking-based reinforcement learning phase includes: Based on the accuracy and efficiency of the diagnosis, initialize a recall scorer; Parallel virtual environment instances are generated and divided into several groups; within each group, the virtual environment instances share the same fault scenario sample. The base model after format constraint reinforcement learning interacts with each virtual environment instance to generate several complete trajectories, and the recall scorer is used to obtain the final result reward score for each complete trajectory. Using the average final result reward score of all complete tracks within the group as the baseline, calculate the advantage value of each complete track relative to the group average. Based on the aforementioned dominance value, the base model parameters are adjusted by maximizing the generation probability of high-dominance trajectories and suppressing the generation probability of low-dominance trajectories.

[0020] A cloud-native fault diagnosis system based on large model intelligent agent reinforcement fine-tuning, the system comprising: The data preprocessing module is used to proactively inject faults into the cloud-native environment and preprocess multimodal operation and maintenance data before and after the fault occurs. The scenario construction module is used to construct fault scenario samples based on preprocessed multimodal operation and maintenance data; The virtual environment module is used to build an interactive environment for fault diagnosis; The agent training module is used for: The teacher model is used to perform multiple rounds of interaction based on the fault scenario samples in the fault diagnosis interactive environment to generate expert interaction trajectories, and the base model is then subjected to format specification supervision and fine-tuning based on the expert interaction trajectories. The dialogue prompts are input into the base model after format specification supervision and fine-tuning for inference, and the tool call instructions obtained from the inference are executed in the virtual environment to obtain environmental feedback and reward signals. Then, the reasoning process of the base model, the tool call instructions, the environmental feedback and reward signals are packaged into an interaction trajectory. The dialogue prompts include: system instructions, virtual environment state description and tool call history. By combining the interaction trajectory with the format specification-supervised fine-tuning of the base model, a multi-stage reinforcement fine-tuning is performed to obtain the optimized fault diagnosis large model; among which, the stage includes: format constraint reinforcement learning stage and ranking selection reinforcement learning stage; Among them, the optimized fault diagnosis model is used to complete the fault diagnosis of the cloud-native application under test.

[0021] An electronic device includes: a processor and a memory storing computer program instructions; the processor, when executing the computer program instructions, implements the cloud-native fault diagnosis method based on large model intelligent agent reinforcement fine-tuning as described above.

[0022] Compared with existing technologies, this invention systematically solves the core challenges of cloud-native fault diagnosis through a two-layer architecture design of data processing and agent fine-tuning. At the data level, it innovatively proposes a three-layer temporal alignment mechanism and a structural feature-based intelligent sampling method. Combined with a global model training strategy and scenario-level dataset partitioning, it ensures both the standardized quality and processing efficiency of multimodal data, while also guaranteeing the diversity coverage and generalization ability of training samples. At the agent level, it pioneeringly applies multi-stage progressive reinforcement learning to diagnostic tasks driven by large language models. Through three progressive stages—format standardization, policy optimization, and accuracy fine-tuning—along with an evidence-based constraint mechanism and a dual-track reward design, it effectively suppresses model illusions and optimizes decision-making strategies. The application of a group relative policy optimization algorithm achieves stable and efficient policy learning, providing reliable advantage estimation in sparse reward scenarios. Experimental results show that the method of this invention significantly outperforms existing methods in key indicators such as top-rank accuracy, top-five accuracy, and average reciprocal ranking on fault diagnosis tasks of multiple real-world microservice systems, while also exhibiting good interpretability and real-time performance. Attached Figure Description

[0023] Figure 1 This is a schematic diagram of the overall technical architecture of the present invention.

[0024] Figure 2 This is a flowchart of the multimodal data preprocessing process.

[0025] Figure 3 A flowchart for constructing and intelligently sampling fault scenarios.

[0026] Figure 4 This is a diagram of a multi-stage, gradual fine-tuning strategy.

[0027] Figure 5 This is a sequence diagram of interactions between an intelligent agent and a virtual environment.

[0028] Figure 6 Flowchart for reward calculation. Detailed Implementation

[0029] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be described in further detail below with reference to the accompanying drawings.

[0030] like Figure 1 As shown, this invention provides a cloud-native application fault diagnosis method and system based on a large model intelligent agent. The entire system starts from the "start" node, goes through the processing of two core parts, and finally outputs the "root cause localization intelligent agent" and reaches the "end" state.

[0031] The system comprises four core modules: data preprocessing module, scene construction module, virtual environment module, and agent training module.

[0032] This invention divides the architecture into two main parts based on data processing and agent capabilities: The first part is the data processing and scenario generation layer, which includes the following processing flow: First, "fault injection and data acquisition" is performed, where various faults are actively injected into the cloud-native environment through the fault injection platform, while simultaneously collecting multimodal operation and maintenance data generated by the system; then, the data flows into the "data preprocessing module" to standardize the raw heterogeneous data, including time-series alignment, format normalization, and feature engineering; next, it enters the "scenario construction module," which constructs high-quality fault scenario samples from the standardized data through intelligent sampling and filtering strategies; finally, "high-quality fault scenarios" are generated as the output of the first part.

[0033] The second part is a scenario-based reinforcement and fine-tuning agent layer. It receives high-quality fault scenario data output from the first part and includes two parallel processing modules: a "virtual environment module" responsible for constructing an interactive environment for fault diagnosis, providing tool call interfaces and reward calculation mechanisms; and an "agent training module" responsible for domain adaptation and reinforcement learning training of the base large language model. The two modules work together. The virtual environment provides the agent with an interactive space and feedback signals, while the agent training module uses these signals to optimize the model strategy, ultimately outputting a "root cause localization agent" with autonomous diagnostic capabilities.

[0034] The entire architecture embodies a progressive design philosophy, moving from coarse data processing to intelligent fine processing, ensuring both data quality and model performance.

[0035] (a) Data preprocessing module.

[0036] The data preprocessing module is used to implement standardized processing of multimodal operation and maintenance data, including five processing steps S1-S5.

[0037] In this embodiment of the invention, step S1: multimodal operation and maintenance data acquisition and preprocessing, specifically includes: like Figure 2 As shown, the system starts from the "Start" node and first performs "raw multimodal data input". The system actively injects various types of faults such as network latency, resource exhaustion, and service dependency failure through a fault injection platform, and collects metrics, logs, and trace data before and after the fault in real time. The adapter factory pattern is used to create dedicated adapters for data sources of different formats, realizing a unified data loading interface and shielding the differences in underlying data formats.

[0038] In this embodiment of the invention, step S2: standardization processing of indicator data, specifically includes: The system performs "data source adaptation and standardization" processing, followed by "time window slicing." Based on the principle of temporal locality, fault records within the same date are grouped, and all data within the time window is loaded into memory at once. Then, precise time window slicing is performed on each fault record. Next, the system enters the "data type" judgment node, where "time series alignment and Z-Score standardization" processing is performed on the indicator data branch. A three-layer time series alignment mechanism is implemented: the first layer is intelligent timestamp standardization, where the system automatically infers the time precision and uniformly converts it into a standard time object; the second layer is adaptive sampling frequency unification, which resamples or maintains the original sampling interval based on the characteristics of the data source; the third layer is global feature dimension alignment, where the system performs a full dataset scan to build a global feature pattern dictionary and dynamically generates filler sequences to complete missing features.

[0039] In this embodiment of the invention, step S3: log data template parsing and standardization, specifically includes: In the implementation case, the system performs "template parsing and statistical processing" for log data branches. A dataset-level global model training strategy is adopted to train a unified log parsing model across the entire dataset to generate log templates, and a log pattern learning model is also trained. In the record-level template matching stage, the global parsing model is used to perform template matching for each fault scenario to generate template sequences, and then the pattern learning model is used to perform anomaly detection and mark abnormal events.

[0040] Based on the above processing, the standardized log data content that the system finally extracts and encapsulates includes the following dimensions: 1) Raw log text: This saves the original statements in the log, providing complete contextual semantic information and avoiding the loss of key parameters due to templates. 2) Log templates, structured event patterns extracted by the parsing model, are used to abstract unstructured text into unified event categories and eliminate noise interference from variables; 3) Abnormal status label, a binary or probabilistic label predicted by the log pattern learning model based on the temporal or statistical features of the template sequence, used to indicate whether the current log entry belongs to an abnormal event; 4) Timestamps are used to accurately record the time when logs are generated, serving as a time reference for cross-modal data time-series alignment.

[0041] In this embodiment of the invention, step S4: link data processing and service-level metric derivation, specifically includes: For the data branch in the link, the system first performs field mapping and normalization processing to establish a mapping relationship from the original fields of heterogeneous data sources to standard fields, so as to solve the problem of inconsistent naming of corresponding key attributes (such as unifying span_id and id as SpanID) for different acquisition protocols.

[0042] Subsequently, the execution chain is restructured. Based on the global trace identifier (TraceID), all spans belonging to the same request are aggregated, and a tree-like parent-child dependency relationship is constructed within the call chain using the span identifier (SpanID) and parent span identifier (ParentSpanID). At the same time, the resource tags of each span are parsed to establish the association between logical spans and physical resources (such as Containers and Nodes).

[0043] Finally, the performance indicators are derived by performing statistical calculations based on the response status codes (such as 200, 500) and latency data of Span, and then merging them into the indicator system described in step S2.

[0044] Based on the above processing, the standardized data content that is finally output and encapsulated in this step specifically includes: 1) Standardized call chain topology: A complete call tree structure built on TraceID and ParentSpanID, used to characterize the propagation path of requests between microservices; 2) Resource association mapping: It clarifies the specific service name and container instance identifier corresponding to each Span node, which is used to locate the physical facility from the logical call; 3) Service-level derived metrics: These include request success rate based on status code statistics and average response time calculated based on latency data, used to supplement the dimensions of metric monitoring data.

[0045] In this embodiment of the invention, step S5: data encapsulation and persistent storage, specifically includes: After the three data branches are processed, they converge at the "Multimodal Data Fusion" node and then enter the "Experimental Data Object Encapsulation" stage. The system assembles the data processed in the previous steps into standardized experimental data objects, which are named fault scenario files. This file is a complete digital snapshot characterizing a specific fault injection event, recording all full multimodal observation data generated by the microservice system for that fault from the start of the fault injection to the fault recovery period.

[0046] This object consists of the following two parts: 1) Fault Meta-information: The source of this information depends on the application scenario. In offline training scenarios, the fault meta-information comes from the injection configuration parameters provided by the fault injection platform, specifically including the unique fault identifier, fault occurrence timestamp, fault level, preset root cause, and fault component name. This information serves as a supervision signal (Label) to construct labeled simulation training samples, and in subsequent multi-stage progressive reinforcement fine-tuning, it serves as a benchmark ground truth to verify the prediction accuracy of the large model and calculate learning rewards. However, in real fault diagnosis scenarios, due to the lack of prior injection operations, the root cause, component name, and level fields in the fault meta-information are initialized to empty, with only the timestamp for recording retained. These fields are then filled in after the agent completes inference. 2) Multimodal data payload: This corresponds to the "three types of data tables" generated in the preceding steps, specifically referring to the index data table standardized by step S2, the log data table parsed and marked by step S3, and the link topology data table reconstructed by step S4.

[0047] Finally, the "scenario-level persistence" operation is performed to serialize and store the experimental data objects, with each fault scenario having its own independent file. At the same time, a global index file is generated to organize the entire dataset, containing metadata, a list of scenarios, and a list of samples, ultimately reaching the "end" state.

[0048] (ii) Scene construction module.

[0049] The scenario building module is used to build fault scenario samples from fault scenarios, including three processing steps S1-S3.

[0050] In this embodiment of the invention, step S1, the three-level funnel screening strategy, is specifically designed to construct a high-quality training dataset from standardized experimental data objects. Here, the data granularity is first defined: the fault scenario serves as a macro-level data container, encapsulating the complete system state (full metrics and logs) within the fault window period, as well as the set of all call chains occurring during this period (typically containing thousands of traces); the "fault sample" refers to the atomic input unit of a single diagnostic task performed by the agent, consisting of a single call chain (trace) to be analyzed as the core thread, and jointly constructed with globally shared metrics and log context.

[0051] Based on the above definition, and given that the agent's diagnostic logic is based on deep analysis of a single abnormal link context, the system needs to implement a one-to-many instantiation and cleaning process from "fault scenarios" to "fault samples." The system traverses the entire link set within the "fault scenario," treating each independent trace as a potential fault sample instance. While retaining global metrics and log data, it sequentially performs the following three-level filtering to eliminate invalid instances and construct a high-quality training set: 1) Temporal Causality Filtering: Based on the fault injection timestamp in the metadata as a hard constraint, historical request links with start times earlier than the fault injection time are strictly filtered out. This step ensures that the traces of all selected samples are products of the fault occurrence, conforming to the causal temporal logic of "fault leading to anomaly"; 2) Complexity and Validity Screening: Based on statistical analysis of typical microservice call depths, a minimum span threshold is set. Simple chains with too few call steps and insufficient information density are filtered out to ensure that the selected samples have sufficient analytical value and can support the agent's reasoning process; 3) Multi-dimensional correlation screening: Topology correlation detection is performed using the fault component name (Ground Truth) in the metadata. The system checks whether the Trace call path passes through the fault injection component or has a direct / indirect dependency relationship with it (including container-level and service-level correlations). This step aims to eliminate "background noise" links that occurred during the fault but are irrelevant to the current fault, ensuring that the features of the training samples are strongly correlated with the fault label.

[0052] In this embodiment of the invention, step S2, which involves calling chain structure feature identification and unsupervised clustering, specifically aims to remove highly redundant homogeneous samples through structural feature recognition. This step includes the following specific processing flow: First, a "call chain feature vectorization" process is performed. The system constructs a global service dictionary, mapping all microservices in the system to a fixed-dimensional vector space. For each filtered candidate call chain, it is abstracted into a high-dimensional "service call feature vector." In this vector, each dimension corresponds to a microservice, and the value of the dimension represents the call frequency of that service in the current call chain. Through this operation, the unstructured set of call chains is converted into a computable feature matrix in batches.

[0053] Secondly, unsupervised clustering analysis is performed. To balance computational efficiency and clustering quality, the system dynamically selects the algorithm kernel based on the sample size of the feature matrix: when the number of samples is below a preset threshold, the standard K-Means algorithm is used to ensure convergence accuracy; when the number of samples exceeds the threshold, it automatically switches to the Mini-Batch K-Means algorithm, which reduces memory usage and improves computation speed through batch processing iteration.

[0054] Finally, the "intelligent optimization of clustering parameters" process is executed. For the selection of the number of cluster centers (K value), a heuristic search strategy based on the silhouette coefficient is adopted. The system iteratively tries different K values ​​within a preset range, calculates the average silhouette coefficient of the current clustering results, and introduces an early stopping mechanism: when the improvement in the silhouette coefficient after multiple consecutive iterations falls below a set threshold, the search is automatically terminated, and the current optimal K value and corresponding clustering results are output, thereby automatically discovering various hidden fault propagation patterns.

[0055] In this embodiment of the invention, step S3: hierarchical sampling and dataset construction, specifically aims to utilize the call chain clustering results obtained in step S2 to solve the sample structure imbalance problem caused by long-tail distribution, and to construct a robust, leak-proof training set. This step includes the following key processing logic: First, perform "hierarchical quota sampling". The system receives the call chain clusters output in step S2 and treats them as different link request patterns. Addressing the common data imbalance phenomenon in actual operation and maintenance data (i.e., simple health check links constitute the vast majority, while complex fault links are a minority), the system calculates the sampling quota for each cluster based on the total budget. Random sampling is performed independently within each cluster, forcibly increasing the probability of selecting low-frequency complex patterns. This step directly affects the call chain sample set, ensuring that the constructed dataset can evenly cover various sparse but critical call topologies in the system, avoiding high-frequency simple patterns dominating model training.

[0056] Second, a "scenario-level dataset partitioning" process is implemented. When dividing the sampled data into training and test sets, the system abandons the traditional sample-level random splitting method and implements strict scenario-level partitioning. Given that multiple call chain samples under the same failure scenario share the exact same environmental state background (i.e., share the same time-series metrics and log context), if they are split into different datasets, the model can easily cheat by memorizing the environmental background rather than reasoning logic (i.e., data leakage). Therefore, this step mandates that all call chain samples from the same failure scenario, along with their associated metrics and log data, must be treated as an indivisible whole and completely belong to either the training or test set.

[0057] Then, the "Scenario-level Dataset Partitioning" node is executed. A fault scenario-level strategy is adopted to avoid data leakage, and a fixed random seed is used to ensure the reproducibility of the partitioning. Different call chains from the same fault scenario have similar system state backgrounds and fault propagation characteristics. If they are distributed across the training and test sets, it will lead to serious data leakage problems.

[0058] Finally, the process reaches the "Generate High-Quality Fault Scenario Dataset" node, implementing three-level hierarchical metadata management and constructing a central index for the dataset. The three levels of metadata are specifically defined as follows: 1) Dataset-level metadata: describing the macroscopic statistical characteristics and partitioning structure of the entire dataset, including the total number of samples, the distribution statistics of fault types, and the partitioning list of training and test sets; 2) Scenario-level metadata: describing the global attributes of a single fault scenario file, including the ground truth label of the fault injection (such as root cause component, fault type), the shared system environment context index (pointing to the metrics and log data blocks under this scenario), and the dataset partition to which this scenario belongs (training or testing); 3) Sample-level metadata: describing the characteristics of the smallest atomic unit of the agent's input, including the unique identifier of a single call chain (TraceID), the cluster ID to which the sample belongs (used to characterize its call pattern category), and the association pointer pointing to the parent fault scenario. All metadata is structured and serialized and stored as the central index file of the dataset, ultimately reaching the "End" state.

[0059] (iii) Virtual environment module.

[0060] The virtual environment module is used to build an interactive environment for fault diagnosis, which includes four processing steps: S1-S4.

[0061] In this embodiment of the invention, step S1: environment state initialization, specifically includes: like Figure 5As shown, the virtual environment module illustrates the interaction process between the agent and the virtual environment through sequence diagrams. The system constructs the interaction flow based on the "Reaction and Reasoning (ReAct)" paradigm, which includes four core participants: the agent, the virtual environment, the tool manager, and the state object.

[0062] The entire interaction begins with a “session reset”, triggered by the control program, which loads a fault sample file from the fault samples as the data base. The state object acts as the manager of the session context, constructs the service call chain topology based on the currently loaded fault samples, initializes the component state bitmap (to track which nodes have been explored), and encapsulates the empty interaction history and the initial system topology into an Initial Observation and returns it to the agent, marking the official start of a new diagnostic episode of the Partially Observable Markov Decision Process (POMDP).

[0063] In this embodiment of the invention, step S2: tool design and execution specifically involves the system entering a multi-round iterative loop of "reasoning-action-observation". In each round of interaction, the agent reasons based on the current context history and generates the next action intention. The large model generates a "tool call request" in JSON format. After receiving the request, the tool manager executes the corresponding logic. The system defines three types of core tools: 1) Metric Query Tool: This tool detects anomalies in metrics for services, containers, and nodes. It not only calculates abnormal deviations of metrics using statistical methods (such as 3-Sigma and Z-Score) but also utilizes time-series shape matching algorithms to identify abnormal fluctuation patterns (such as sudden increases, sudden decreases, step-like increases, and oscillations). Crucially, it includes a built-in "time-series-to-text translation module," which converts abstract numerical fluctuation characteristics into natural language descriptions understandable by larger models (e.g., "CPU usage experienced a step-like surge at 10:05 and remained high"). The final metric report generated by this tool includes: the name of the abnormal metric, a quantified deviation score, the specific anomaly timestamp, and a semantic text description of the fluctuation pattern.

[0064] 2) Log Analysis Tool: This tool aggregates abnormal logs for containers. Based on the log parsing model pre-trained in step S3, it performs pattern matching and filtering on log streams within a specified time window. The final log report output by this tool includes: the template content of logs marked as abnormal (retaining key variables), the frequency of occurrence of various abnormal logs within the query window, and the temporal distribution information of abnormal clusters (such as the first and last occurrence times). This statistical information helps the agent determine the severity and frequency of faults.

[0065] 3) Results Submission Tool: Used to submit the final diagnostic conclusion. Calling this tool marks the termination of the current diagnostic task.

[0066] The context update and state transition mechanism after tool execution is as follows: When the agent calls the indicator query tool or log analysis tool, the tool manager generates an "indicator report" or "log report" containing the aforementioned detailed fields. The virtual environment encapsulates these reports into new environmental observations and appends them to the agent's input context, forming an accumulated interaction trajectory to support the agent in performing deeper reasoning in the next round. This loop continues until the agent calls the "result submission tool," at which point the virtual environment triggers the result reward calculation logic (i.e., calls the algorithm described in step S3), calculates the final score of this diagnosis, and ends the current session.

[0067] like Figure 6 As shown, the system constructs a dual-track evaluation system consisting of "process reward" and "result reward," which aims to guide the agent's strategy optimization from two dimensions: the standardization of interactive behavior and the accuracy of diagnostic conclusions.

[0068] For tool call branches, the system executes the "Process Reward Calculation" node. The process reward provides immediate feedback for each tool call, with zero reward for valid calls and negative penalties for invalid calls. A valid call refers to the first query of an unchecked component or a call that conforms to the tool usage guidelines. Invalid calls include violations such as repeatedly querying checked components or using logging tools on non-container components.

[0069] First, a process reward is defined to quantify the quality of the agent's behavior during the diagnostic process. This reward is applied throughout the multi-round interaction cycle, providing an immediate score for each tool call made by the agent. If the agent performs a logically sound and interface-compliant operation (such as querying metrics of unexplored components for the first time or analyzing logs appropriately), the system provides a positive incentive or zero penalty; if redundant, irregular, or invalid operations are performed, a negative penalty is imposed. The accumulation of process rewards reflects the agent's "professionalism" in using tools to collect evidence.

[0070] Secondly, a result reward is defined to quantify the accuracy of the agent's final diagnostic conclusion. When the agent calls the "result submission tool," it signifies the end of the diagnostic task, and the system immediately triggers the result reward calculation logic. The agent must submit a ranking list of potential root causes of failure in the submission tool, ordered from highest to lowest confidence. The system performs the following two-step evaluation based on this list: 1) Evidence-based constraint verification: The system first checks whether the root cause components in the list appear in the agent's exploration trajectory (i.e., the set of checked components). If the true root cause has not been explored, it is considered a model "guess" or "illusion," and the result reward is directly determined to be zero, thus forcing the agent to follow the logic of "evidence first, conclusion later." 2) Ranking accuracy calculation: Under the premise of passing the evidence-based verification, the system calculates the rank of the true root cause of failure in the agent's submitted list. Using reciprocal rank or Top-K hit rate as a quantitative indicator, the higher the ranking of the true root cause (i.e., the closer it is to Top-1), the higher the reward score is given; conversely, the lower the ranking, the score decays non-linearly.

[0071] Ultimately, the system uses the process rewards accumulated during the interaction and the result rewards calculated when the task ends as the optimization objectives of reinforcement learning algorithms (such as GRPO), driving the agent to continuously evolve so that it can maximize the accuracy of root cause localization while ensuring operational norms. This corresponds to the "final reward output" node and the "update agent policy" endpoint in the diagram.

[0072] Ultimately, all branches converge at the "Final Reward Output" node, and then reach the "Update Agent Policy" endpoint.

[0073] In this embodiment of the invention, step S4: the implementation of the stage transition logic, specifically includes: At the end of each round, the virtual environment performs a "reward calculation" operation and then returns "reward + observation + termination flag" information to the agent. The system implements a three-phase transition logic: the initial phase allows indicator query and log analysis tools to explore, with the goal of collecting evidence; the middle phase additionally allows a result submission tool, which can choose to submit or continue exploration; the final phase mandates the invocation of the result submission tool. After a successful invocation of the indicator query tool, the component is moved from the "pending inspection" state to the "inspected" state and added to the exploration path. When the result submission tool is invoked, an evidence-based constraint check is performed to ensure that the diagnostic conclusions are based on the actual collected evidence.

[0074] (iv) Intelligent agent training module.

[0075] The agent training module is used to optimize diagnostic strategies through model fine-tuning and reinforcement learning, and includes four processing steps, S1-S4.

[0076] In this embodiment of the invention, step S1 above: trajectory generation and supervised fine-tuning based on the teacher model, specifically includes: like Figure 4 As shown, the system starts from the "Start" node and first loads the "Base Large Language Model" as the target model to be optimized. This step utilizes the teacher model, whose reasoning ability is superior to the base model, as the core driving model of the agent, and performs multiple rounds of fault diagnosis drills (Rollout) in the fault scenarios of the training set generated by the scene construction module. The teacher model, through simulated interaction with the virtual environment, fully records the thought chain reasoning process, tool call sequence, and final diagnostic conclusion when facing a specific fault context, thereby generating a high-quality expert interaction trajectory. Subsequently, the system cleans and formats the collected interaction trajectories to construct a structured supervised fine-tuning dataset containing instructions, environmental state context, and the teacher model's reasoning path. Based on this dataset, supervised fine-tuning of the base large language model is carried out using quantized low-rank adaptation technology. The weights of the pre-trained model are quantized to reduce resource consumption, and a low-rank adapter matrix is ​​introduced for gradient updates at key layers. By minimizing the difference between the base model generation strategy and the teacher demonstration trajectory, knowledge injection of expert diagnostic logic and tool usage norms is achieved, enabling the base model to have initial fault diagnosis capabilities and laying a strategic foundation for the subsequent reinforcement learning stage. This step corresponds to the "format specification supervision and fine-tuning" stage in the figure.

[0077] In this embodiment of the invention, step S2: online interaction and experience generation based on the current policy, specifically involves: the system loading the model fine-tuned in step S1 as the current agent policy and initiating a high-concurrency online interaction sampling process. The system calls the aforementioned "virtual environment module" for multiple instantiations, constructing multiple independent fault diagnosis interaction sessions, each loading specific fault scenario data. The agent, as the diagnostic subject, uses a context manager to assemble system instructions, current environment state, and historical operation records into structured prompts for each parallel environment instance, and generates action responses containing thought chains and tool calls through the inference engine. The virtual environment module receives and executes these instructions, returning the corresponding execution results and immediate reward signals. The system monitors and records this complete state-action-reward loop in real time, transforming the entire multi-turn dialogue process of the agent under the current policy into standardized experience trajectory data, storing it in the experience replay buffer, thereby providing sufficient trial and error samples for subsequent group-based reinforcement learning algorithms.

[0078] In this embodiment of the invention, step S3, the two-stage progressive reinforcement fine-tuning based on group relative policy optimization, specifically involves: the system loading the model trained in step S1 as the initial policy network, and employing the group relative policy optimization algorithm to implement two progressive stages of reinforcement training—from "format compliance" to "diagnostic accuracy"—through a closed-loop process of "sampling-evaluation-update." Throughout step S3, the group relative policy optimization algorithm (GRPO) serves as the core driving engine. Its execution logic is as follows: for each input fault scenario, the agent samples and generates a set of parallel diagnostic trajectories based on the current policy. The system calculates the advantage function of each trajectory within the group, standardizing it by subtracting the average reward of the group of trajectories from the reward value of a single trajectory and dividing by the standard deviation. This mechanism eliminates the dependence on an additional value network, directly utilizing the relative advantage within the group to update the policy, reducing training memory usage and improving stability. The specific training stages are as follows: Phase 1: Format Constraint Reinforcement Learning. This phase aims to address the formatting errors and logical inconsistencies that agents often exhibit when deviating from expert guidance. The system employs a format scorer as the primary reward function. The model autonomously samples and generates multiple candidate tool call paths for fault scenarios. The format scorer quantitatively evaluates the correctness of each path, and the reward signal comprehensively considers three dimensions: syntactic validity (e.g., correct JSON format), tool parameter validity, and task closure. By maximizing the likelihood probability of high-scoring trajectories, the agent is forced to strictly adhere to tool call specifications during exploration, ensuring its fundamental ability to interact stably in complex environments.

[0079] Phase 2: Ranking Preference Reinforcement Learning. After the agent masters the interaction norms, the optimization focus shifts to improving the accuracy of root cause localization. This phase introduces a recall scorer as the core reward signal, combining evidence-based constraints and a ranking recall mechanism to evaluate diagnostic quality. The agent generates a final ranking list containing multiple candidate root causes. The recall scorer first implements evidence-based constraint checks. If the true root cause is not in the exploration path (i.e., has not been verified by the tool), the reward is forcibly set to zero to prevent guessing behavior; if the true root cause has been explored, a high reward is calculated based on its ranking in the list. Simultaneously, a time decay function is applied to penalize excessively long diagnostic rounds, and points are deducted for illusory outputs (such as non-existent components), thereby guiding the agent to learn a "fast, accurate, and stable" expert-level diagnostic strategy.

[0080] The cloud-native application fault diagnosis system provided by this invention adopts a modular design, with each module interacting through a standard interface. The system deploys a fault injection platform and configures a data acquisition system in a cloud-native environment; it runs a data preprocessing module to batch process fault data and generate standardized experimental data objects; a scenario construction module constructs training and test datasets; an agent training module performs domain knowledge injection and reinforcement learning training; and the trained model is integrated into the operation and maintenance platform. When a system fault occurs, the platform automatically collects multimodal data, calls the data preprocessing module for standardization, and then drives the agent model to perform fault diagnosis, outputting a complete diagnostic report including root cause ranking, confidence score, diagnostic inference path, and repair suggestions.

[0081] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A cloud-native fault diagnosis method based on large model agent reinforcement fine-tuning, characterized in that, The method comprises: actively injecting faults into a cloud-native environment, and preprocessing multi-modal operation and maintenance data before and after the occurrence of the faults; constructing fault scenario samples based on the preprocessed multi-modal operation and maintenance data; constructing a fault diagnosis interactive environment; performing multi-round interaction based on the fault scenario samples in the fault diagnosis interactive environment using a teacher model, generating an expert interaction track, and performing format specification supervision fine-tuning on a base model based on the expert interaction track; inputting a dialogue prompt to the format specification supervision fine-tuned base model for reasoning, executing tool invocation instructions obtained through reasoning in a virtual environment to obtain environment feedback and reward signals, and packaging the reasoning thought process of the base model, the tool invocation instructions, the environment feedback, and the reward signals into an interaction track; wherein the dialogue prompt comprises system instructions, virtual environment state descriptions, and tool invocation histories; performing multi-stage reinforcement fine-tuning on the format specification supervision fine-tuned base model in combination with the interaction track to obtain an optimized fault diagnosis large model; wherein the stages include a format constraint reinforcement learning stage and a ranking optimization reinforcement learning stage; performing fault diagnosis on a to-be-tested cloud-native application based on the optimized fault diagnosis large model.

2. The method of claim 1, wherein, The multi-modal data comprises index data, log data, and link data; constructing fault scenario samples based on the preprocessed multi-modal operation and maintenance data comprises: encapsulating the preprocessed multi-modal operation and maintenance data into standardized experimental data objects; wherein the standardized experimental data objects comprise scenario metadata, standardized index data, standardized log data, and standardized call chain data, and the scenario metadata comprises a fault unique identifier, a timestamp, a fault level, a fault reason, and a fault component name; filtering the standardized call chain data based on the scenario metadata; extracting call chain structure features from the filtered standardized call chain data; obtaining call patterns of the call chains according to the call chain structure features, and performing call chain sampling according to the call patterns to generate a plurality of fault scenario samples; dividing the fault scenario samples into a training scenario set, a verification scenario set, and a test scenario set using a fixed random seed.

3. The method of claim 1, wherein, filtering the standardized call chain data based on the scenario metadata comprises: filtering out the standardized call chain data occurring before the fault according to the timestamp; calculating the number of call steps of the call chain based on the standardized call chain data, and filtering out the standardized call chain data with a number of call steps less than a minimum complexity threshold; filtering out the standardized call chain data that does not exist in container-level, service-level, or node-level association with the fault component.

4. The method of claim 1, wherein, obtaining call patterns of the call chains according to the call chain structure features, and performing call chain sampling according to the call patterns to generate a plurality of fault scenario samples comprises: clustering the call chain structure features; wherein each cluster corresponds to a call pattern; calculating a sampling quota for each cluster according to the total sampling budget and the number of clusters; performing independent random sampling in each cluster according to the sampling quota, and generating corresponding fault scenario samples according to the sampling results.

5. The method of claim 1, wherein, The tool invocation instructions comprise index query tools, log analysis tools, and result submission tools; The tool calling instruction obtained by reasoning is executed in a virtual environment to obtain environment feedback and a reward signal, including: In the case that the tool calling instruction is an index query tool or a log analysis tool, a process reward score is given by judging whether the environment feedback returned by the index query tool or the log analysis tool is valid feedback; In the case that the tool calling instruction is a result submission tool, a final result reward score is obtained by judging whether the environment feedback returned by the result submission tool passes the evidence constraint check and calculating the ranking of the environment feedback in an output list.

6. The method of claim 5, wherein, The final result reward score is obtained by judging whether the environment feedback returned by the result submission tool passes the evidence constraint check and calculating the ranking of the environment feedback in an output list, including: An original result reward score is calculated according to the environment feedback; It is judged whether the real root cause is in the exploration path; In the case that the real root cause is not in the exploration path, the final result reward score is set to zero; In the case that the real root cause is in the exploration path, a recall score is calculated according to the ranking of the original result reward score in the output list, and after time decay punishment and illusion punishment are performed on the recall score, the final result reward score is obtained; wherein the time decay punishment is used to punish the number of diagnosis rounds, and the illusion punishment is used to negatively punish unexplored components and repeated components contained in the output list.

7. The method of claim 6, wherein, The execution process of the format constraint reinforcement learning stage includes: A format scorer is initialized based on the legality and validity of the tool calling instruction and the closed-loop nature of the diagnosis task; Parallel virtual environment instances are generated, and the virtual environment instances are divided into groups; wherein the virtual environment instances in each group share the same fault scene sample; The base model supervised by the format specification is allowed to interact with each virtual environment instance to generate complete trajectories, and a process reward score of each complete trajectory is obtained using the format scorer; Based on the process reward scores of all complete trajectories in the group, an advantage value of each complete trajectory relative to the average level in the group is calculated; Based on the advantage value, the generation probability of high-advantage trajectories is maximized and the generation probability of low-advantage trajectories is suppressed to adjust the base model parameters.

8. The method of claim 6, wherein, The execution process of the ranking optimization reinforcement learning stage includes: A recall scorer is initialized based on the accuracy and efficiency of diagnosis; Parallel virtual environment instances are generated, and the virtual environment instances are divided into groups; wherein the virtual environment instances in each group share the same fault scene sample; The base model after the format constraint reinforcement learning is allowed to interact with each virtual environment instance to generate complete trajectories, and a final result reward score of each complete trajectory is obtained using the recall scorer; The average final result reward score of all complete trajectories in the group is taken as a baseline to calculate an advantage value of each complete trajectory relative to the average level in the group; Based on the advantage value, the generation probability of high-advantage trajectories is maximized and the generation probability of low-advantage trajectories is suppressed to adjust the base model parameters.

9. A cloud-native fault diagnosis system based on large model agent reinforcement fine-tuning, characterized in that, The system includes: The data preprocessing module is configured to actively inject faults into a cloud-native environment and preprocess multi-modal operation and maintenance data before and after the faults occur. The scene construction module is configured to construct fault scene samples based on the preprocessed multi-modal operation and maintenance data. The virtual environment module is configured to construct a fault diagnosis interactive environment. The agent training module is configured to: generate expert interaction trajectories by performing multiple rounds of interaction in the fault diagnosis interactive environment based on the fault scene samples using a teacher model, and perform format specification supervision fine-tuning on the base model based on the expert interaction trajectories; input a dialogue prompt into the format specification supervision fine-tuned base model for reasoning, execute the tool invocation instructions obtained through reasoning in the virtual environment to obtain environment feedback and reward signals, and package the reasoning thought process of the base model, the tool invocation instructions, the environment feedback, and the reward signals into interaction trajectories; wherein the dialogue prompt includes system instructions, virtual environment state descriptions, and tool invocation histories; perform multi-stage reinforcement fine-tuning on the format specification supervision fine-tuned base model in combination with the interaction trajectories to obtain an optimized fault diagnosis large model; wherein the stages include a format constraint reinforcement learning stage and a ranking and optimization reinforcement learning stage. The optimized fault diagnosis large model is used to complete fault diagnosis of a to-be-tested cloud-native application.

10. An electronic device, comprising: The electronic device includes a processor and a memory storing computer program instructions; and the processor executes the computer program instructions to implement the cloud-native fault diagnosis method based on large model agent reinforcement fine-tuning according to any one of claims 1-8.

Images