A method and apparatus for troubleshooting operations and maintenance intelligent agents based on large language models
By introducing explicit causal chains and fault cause-effect graph matching, counterfactual testing, functional semantic matching, and cost scoring into Kafka operations and maintenance, the problem of insufficient fault location capabilities in Kafka operations and maintenance is solved, enabling efficient and secure fault diagnosis and tool selection, adapting to different deployment scenarios, and quickly learning new types of faults.
Patent Information
- Application Number
- CN202611475384.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-09-21
- Publication Date
- 2026-11-17
AI Technical Summary
Existing technologies in Kafka operations and maintenance suffer from insufficient fault location capabilities, low tool invocation efficiency, poor long-term memory management, insufficient security, and poor business adaptability. They cannot effectively distinguish between the correlation and causation of faults and lack access control and audit tracking.
By combining explicit causal chains and fault cause-effect graph matching with counterfactual testing, along with functional semantic matching, cost scoring, and output pattern verification, we achieve intelligent and efficient tool selection. Furthermore, through hierarchical management of long short-term memory and hierarchical permission control of RBAC, we ensure the security and traceability of operation and maintenance, and adapt to the characteristics of Kafka business.
It significantly improves the accuracy of root cause location of faults, enables intelligent and efficient tool selection, avoids endless troubleshooting loops, maintains consistency in long-term interactions, ensures the safety and traceability of operation and maintenance, adapts to different deployment scenarios, and quickly learns new types of faults.
Smart Images

Figure CN123114833A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of intelligent operation and maintenance technology, and in particular to a method and device for troubleshooting operation and maintenance intelligent agents based on a large language model. Background Technology
[0002] As a widely used distributed message queue, Apache Kafka's operational complexity increases significantly with the size of the cluster. Troubleshooting typically requires operations personnel to possess deep domain knowledge and manually execute numerous commands to gradually pinpoint the root cause, a process that is time-consuming and error-prone.
[0003] In existing technologies, intelligent agents that combine reasoning and action using large language models (such as the ReAct framework) propose a general intelligent agent solution that allows the large language model to alternately generate reasoning trajectories and actions, and to think and call external tools while executing tasks. However, this solution is a general reasoning framework and does not build a dedicated knowledge base and tools for the Kafka operation and maintenance domain. The reasoning process does not distinguish between the correlation and causality of faults, lacks explicit causal chains and counterfactual tests, and lacks operation and maintenance-oriented cache invalidation, permission hierarchies, and auditing mechanisms.
[0004] The Retrieval Enhancement Generation (RAG) scheme generates answers by retrieving relevant documents from external knowledge bases as context input to a large language model to alleviate model illusion. However, its retrieval results are expressed as static document blocks, without modeling the causal chain of "phenomenon → root cause" of the fault, and it is only for generating answers, without agent tool calls and execution loops for Kafka operations and maintenance.
[0005] Agent orchestration and tool invocation frameworks (such as LangChain and AutoGPT) provide general components such as orchestration, tool invocation, and memory modules, but none of them have built-in Kafka-specific knowledge bases and tool adaptations. Tool selection lacks a multi-level screening mechanism for functional semantic matching, cost scoring, and output pattern verification, and there is no RBAC permission hierarchical and auditing function for operation and maintenance.
[0006] The long-term memory management and vector retrieval scheme utilizes vector databases and vector models to achieve text vectorization and similarity retrieval, providing agents with long-term memory and knowledge retrieval capabilities. However, this scheme is a general retrieval and storage method, and does not organize causal chains based on the operational experience of "fault phenomenon → root cause". It also lacks a memory compression and forgetting mechanism based on three-dimensional scoring of importance, access frequency, and timeliness.
[0007] Message queue operation and maintenance monitoring and fault handling solutions (such as the official Kafka command-line tool combined with Prometheus + Grafana) are based on rules and thresholds to trigger alarms and automate processing, but they do not introduce semantic reasoning of large language models, lack a "phenomenon → root cause" causal chain, and lack cross-tool collaborative orchestration, cache invalidation indexing and RBAC permission hierarchical capabilities.
[0008] In summary, the existing technology has the following drawbacks: (1) Insufficient reasoning and fault location capabilities, prone to reasoning chain breakage, difficult to establish causal relationship between fault phenomena and root causes, unable to distinguish the correlation and causality of faults, and limited self-correction capabilities; (2) Poor tool calling efficiency and adaptability, lack of targeted tool selection, cumbersome calling process, lack of caching mechanism, and weak multi-tool collaboration capability; (3) Weak long-term planning and context management capabilities, long-term memory decays with the extension of interaction, and lack of memory retrieval and compression mechanisms; (4) Insufficient credibility and security, existence of illusion problems, and lack of permission management and audit tracking; (5) Poor business adaptability and generalization capabilities, not customized for Kafka business characteristics, and unable to quickly learn new faults. Summary of the Invention
[0009] In view of this, embodiments of this application provide a method and apparatus for troubleshooting operational intelligence agents based on a large language model. This method can solve the problems of broken agent inference chains and the inability to distinguish between fault correlation and causation, achieving accurate Kafka fault location; it also solves the problems of poor agent compatibility with Kafka tools and APIs, and low calling efficiency, enabling efficient and orderly tool calls; it addresses the problems of insufficient long-term memory and weak context management capabilities of agents, maintaining consistency over long interactions; it resolves issues of agent illusion, low credibility, and operational security risks, achieving trusted inference and traceable operations; and it addresses the problems of poor agent business adaptability and insufficient generalization ability, adapting to different deployment scenarios and quickly learning new types of faults.
[0010] The technical solution of this application embodiment is implemented as follows: In a first aspect, embodiments of this application provide a fault diagnosis method for an operation and maintenance intelligent agent based on a large language model, comprising the following steps: Receive operation and maintenance requests, parse and generate standardized requirement instructions; Based on the standardized requirement instructions, task planning is performed to generate a priority-based task queue; The iterative loop of execution reasoning and tool invocation is performed. In each iteration, the large language model generates reasoning trajectories and action intentions based on the current short-term memory. Based on the stated action intent, the optimal tool is selected from the tool registry through functional semantic matching, cost scoring, and output mode sufficiency verification. Execute the selected tools to operate on the target cluster and obtain observations; The observed results are matched with explicit causal chains and fault causal graphs in the knowledge base that point from the fault phenomenon to the root cause, and causal inference and counterfactual testing are performed to locate the root cause.
[0011] Secondly, embodiments of this application also provide a fault diagnosis device for an operation and maintenance intelligent agent based on a large language model, the device comprising: The parsing module is used to receive operation and maintenance requirements and parse them to generate standardized requirement instructions; The generation module is used to plan tasks according to the standardized requirement instructions and generate a priority task queue. The first execution module is used to execute the iterative loop of reasoning and tool invocation. In each iteration, the large language model generates the reasoning trajectory and action intention based on the current short-term memory. The selection module is used to select the optimal tool from the tool registry based on the action intent, through functional semantic matching, cost scoring, and output mode sufficiency verification. The second execution module is used to execute the selected tool, operate on the target cluster, and obtain observation results; The third execution module is used to match the observation results with the explicit causal chain and fault causal graph in the knowledge base that points from the fault phenomenon to the root cause, and to perform causal inference and counterfactual testing to locate the root cause.
[0012] Thirdly, embodiments of this application also provide an electronic device, including: a processor, a storage medium, and a bus, wherein the storage medium stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor communicates with the storage medium via the bus, and the processor executes the machine-readable instructions to perform the fault diagnosis method for operation and maintenance intelligent agents based on large language models as described in any of the first aspects.
[0013] Fourthly, embodiments of this application also provide a computer-readable storage medium storing a computer program, wherein the computer program is executed by a processor to perform the fault diagnosis method for operation and maintenance intelligent agents based on a large language model as described in any one of the first aspects.
[0014] The embodiments of this application have the following beneficial effects: By combining explicit causal chains with fault cause-effect graph matching and counterfactual testing, the system effectively distinguishes between fault correlation and causality, significantly improving the accuracy of root cause localization. A three-tiered screening mechanism—functional semantic matching, cost scoring, and output pattern sufficiency verification—enables intelligent and efficient tool selection. An evidence chain backtracking self-correction mechanism endows the agent with self-correction capabilities, preventing endless troubleshooting loops. A hierarchical management system for long short-term memory and a three-dimensional scoring forgetting mechanism effectively manage contextual information, maintaining consistency over long interactions. RBAC permission hierarchical management and full-link audit tracing ensure the security and traceability of operational processes. A dedicated knowledge base and tool adaptation tailored to Kafka business characteristics enable excellent adaptability to different deployment scenarios and rapid learning capabilities for new types of faults. Attached Figure Description
[0015] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.
[0016] Figure 1 This is a flowchart illustrating steps S101-S106 provided in the embodiments of this application; Figure 2 This is a flowchart illustrating steps S201-S202 provided in the embodiments of this application; Figure 3 This is a flowchart illustrating steps S301-S303 provided in the embodiments of this application; Figure 4 This is a flowchart illustrating steps S401-S403 provided in the embodiments of this application; Figure 5 This is a schematic diagram of the fault diagnosis device for operation and maintenance intelligent agents based on large language models provided in the embodiments of this application; Figure 6 This is a schematic diagram of the composition structure of the electronic device provided in the embodiments of this application. Detailed Implementation
[0017] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. It should be understood that the accompanying drawings in this application are for illustrative and descriptive purposes only and are not intended to limit the scope of protection of this application. Furthermore, it should be understood that the schematic drawings are not drawn to scale. The flowcharts used in this application illustrate operations implemented according to some embodiments of this application. It should be understood that the operations in the flowcharts may not be implemented in sequence, and steps without logical contextual relationships may be reversed or implemented simultaneously. In addition, those skilled in the art, guided by the content of this application, may add one or more other operations to the flowcharts, or remove one or more operations from the flowcharts.
[0018] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.
[0019] Furthermore, the described embodiments are merely some, not all, of the embodiments of this application. The components of the embodiments of this application described and illustrated herein can typically be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of the application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.
[0020] In the following description, the terms "first, second, third" are used merely to distinguish similar objects and do not represent a specific ordering of objects. It is understood that "first, second, third" may be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.
[0021] It should be noted that the term "comprising" will be used in the embodiments of this application to indicate the presence of the features declared thereafter, but does not exclude the addition of other features.
[0022] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application and is not intended to limit this application.
[0023] See Figure 1 , Figure 1This is a flowchart illustrating steps S101-S106 of the fault diagnosis method for operation and maintenance intelligent agents based on a large language model provided in this application embodiment. It will be combined with... Figure 1 Steps S101-S106 shown will be explained.
[0024] In step S101, maintenance requirements are received and standardized requirement instructions are generated by parsing. In step S102, task planning is performed according to the standardized requirement instructions to generate a priority task queue; In step S103, an iterative loop of reasoning and tool invocation is executed. In each iteration, the large language model generates reasoning trajectory and action intention based on the current short-term memory. In step S104, based on the action intent, the optimal tool is selected from the tool registry through functional semantic matching, cost scoring, and output mode sufficiency verification. In step S105, the selected tool is executed to operate on the target cluster and obtain observation results; In step S106, the observation results are matched with the explicit causal chain and fault causal graph in the knowledge base that points from the fault phenomenon to the root cause, and causal inference and counterfactual testing are performed to locate the root cause.
[0025] In this embodiment, the system first receives maintenance requests submitted by operations and maintenance personnel via natural language, command line, or a visual interface. The request parsing chain then parses these requests into structured, standardized request instructions. The data structure of these instructions includes fields such as fault type, target cluster, keywords, priority, and user role. Next, the task planning module receives these instructions and, in conjunction with cluster configuration and historical records, decomposes the complex troubleshooting objectives into a series of executable sub-tasks. These sub-tasks are then prioritized and output as a task queue, ensuring that the troubleshooting work is carried out in an orderly and efficient manner.
[0026] After task planning is completed, this embodiment performs an iterative loop of reasoning and tool invocation. In each iteration, the large language model combines the short-term memory of the current session with the task objective to perform deep reasoning, generating a "reasoning trajectory," recording its thought process, and producing a clear "action intent," i.e., what operation needs to be performed next. This action intent drives an intelligent tool selection process, using a three-level screening mechanism of "functional semantic matching, cost scoring, and output pattern sufficiency verification" to select the optimal tool to execute the intent from the tool registry. Once the tool is selected, the system executes it on the target Kafka cluster, for example, querying topic status, obtaining consumer group offsets, or checking Broker configuration, and obtaining the returned observation results.
[0027] Finally, this embodiment performs a deep matching between the observations obtained from the tool execution and a specially constructed operations and maintenance knowledge base. The core of this knowledge base lies in "explicit causal chains," which are predefined ordered sequences of events pointing from fault phenomena to the root cause, such as "ZooKeeper anomaly → Broker disconnection → Partition Leader has no available replicas → Partition offline." The system maps the observed phenomena to nodes in a "fault causal graph" constructed from multiple causal chains, determining whether the phenomena are causally related or merely correlated by searching for directed paths within the graph structure. To ensure the reliability of the conclusions, the system also performs "counterfactual testing," such as asking the large model, "If this root cause did not exist, would the fault phenomenon still occur?", thereby ultimately pinpointing the root cause of the fault accurately.
[0028] In some embodiments, if the root cause is not located and the maximum number of iterations is not exceeded in the iterative loop of reasoning and tool invocation, self-correction is triggered; wherein, the self-correction is based on the action observation evidence chain accumulated in this investigation, backtracks to locate the first suspicious step, attributes the error type and adjusts the reasoning direction or tool strategy, and then returns to the reasoning step of the iterative loop.
[0029] In complex Kafka troubleshooting, the agent may need to go through multiple iterations. If an error occurs in the inference or tool selection of a certain round, it may cause the subsequent troubleshooting direction to deviate completely, ultimately failing to locate the root cause. In this embodiment, a maximum number of iterations (e.g., 10 times) is set to prevent infinite loops. Under this premise, when the root cause is not successfully located within the loop, self-correction is actively triggered.
[0030] The core basis of self-correction is the "chain of evidence." The chain of evidence is a chronological record of all "action-observation" steps accumulated during a troubleshooting process, comprehensively documenting what the agent did and what results it obtained at each step. When error correction is triggered, the system traces back this chain of evidence to locate the first suspicious step. This "suspicious" judgment can be based on anomalies in the observed results, contradictions in the reasoning logic, or deviations from typical failure patterns in the knowledge base. After locating the error, the system attributes it to the type of error, such as an incorrect reasoning direction, inappropriate tool selection, or incorrect tool parameters. It then adjusts the reasoning direction or tool strategy accordingly and returns to the iterative loop's reasoning steps with the corrected context. This approach significantly improves the success rate and robustness of fault location.
[0031] In some embodiments, the cost score in the selection of the optimal tool is based on the formula c_t=0.4. d_t+0.3 r_t+0.3 The calculation is p_t, where c_t is the tool cost score, d_t is the normalized call time, r_t is the normalized resource consumption, and p_t is the normalized risk level. The smaller the cost score, the higher the priority.
[0032] Here, after the system filters out a batch of candidate tools through functional semantic matching, it needs to select the optimal one. This requires considering not only whether the tool can complete the task, but also the cost and risk of executing the tool. This embodiment designs a quantitative cost scoring formula: c_t=0.4 d_t+0.3 r_t+0.3 p_t and c_t represent the final cost score of tool t. The smaller the score, the lower the overall cost of the tool, and the more likely it should be selected. This score is derived by weighted summation of three dimensionless indicators normalized to the 0-1 range: d_t is the normalized call time, reflecting the speed of tool execution; r_t is the normalized resource consumption, reflecting the tool's consumption of system resources such as CPU and memory; and p_t is the normalized risk level, reflecting the degree of risk that executing the tool may have on the production cluster. The weight allocation (0.4, 0.3, 0.3) considers execution time as the most important factor, while also taking into account resource consumption and operational risk. Through this formula, the system can automatically and objectively make the optimal decision among multiple available tools. For example, when querying a list of topics, if a lightweight API query tool and a resource-intensive JMX monitoring and collection tool can both complete the task, the cost scoring mechanism will tend to choose the API tool with shorter execution time and lower resource consumption, thereby achieving efficient and low-risk operation and maintenance.
[0033] In some embodiments, the method further includes: For long-term memory, the following processing is performed: based on experience in cross-session history retrieval, using the formula s_m=0.7 cos(q,e_m)+0.2 i_m+0.1 a_m calculates the memory retrieval score, where s_m is the retrieval score, cos(q,e_m) is the cosine similarity between the query vector and the memory vector, i_m is the memory importance, and a_m is the memory timeliness score; and is based on the formula f_m=0.5. (1-i_m)+0.3 (1 / (1+n_m))+0.2 ρ_m calculates the forgetting score, where f_m is the forgetting score, n_m is the number of visits, and ρ_m is the age ratio. Memories that exceed the forgetting threshold or lifespan are cleared.
[0034] Long-term memory stores past troubleshooting cases and operational experience. When performing new troubleshooting, the system needs to retrieve the most relevant historical experience from long-term memory as a reference. This retrieval process uses a comprehensive retrieval scoring formula s_m=0.7. cos(q,e_m)+0.2 i_m+0.1 a_m quantifies the relevance of memory entries. In this formula, s_m is the final retrieval score of memory m. It consists of three weighted components: cos(q,e_m) is the cosine similarity between the current query vector q and the memory vector e_m, with the highest weight (0.7), ensuring that the retrieved content is semantically highly relevant to the current fault; i_m is the importance score of the memory, with a weight of 0.2, used to prioritize the recall of cases marked as key or classic; a_m is the timeliness score of the memory, with a weight of 0.1, making newer experiences more likely to be retrieved. In addition to retrieval, this embodiment also manages the capacity of the memory bank through forgetting to prevent infinite expansion. The forgetting score is calculated using the formula f_m=0.5. (1-i_m)+0.3 (1 / (1+n_m))+0.2 ρ_m is calculated as follows: f_m is the forgetting score; the higher the score, the more likely the memory should be cleared. (1-i_m) means that less important memories are more easily forgotten. (1 / (1+n_m)) corresponds to a lower number of visits n_m, resulting in a higher score and greater likelihood of forgetting. ρ_m is the age ratio, representing the relative time since the last visit; older memories are more easily forgotten. When the forgetting score f_m of a memory exceeds a set threshold, or its existence exceeds its lifespan, the system will clear it, achieving dynamic and intelligent memory management.
[0035] In some embodiments, see Figure 2 , Figure 2 This is a flowchart illustrating steps S201-S202 provided in the embodiments of this application. When the selected tool is executed to operate on the target cluster and obtain observation results, the method further includes steps S201-S202, which will be explained in conjunction with each step.
[0036] In step S201, access control permissions are intercepted and cached based on roles; In step S202, if the operation request hits the read cache and the permissions are granted, the cached result is returned directly; otherwise, the real tool is called, the result is written to the cache, and an audit log containing end-to-end tracing identifiers is generated.
[0037] This embodiment performs role-based access control before executing any tool operations. The system authenticates the user by calling the `checkPermission` interface based on the user's assigned role during login (e.g., administrator, operator, viewer) and the specific content of the operation request. Permitted operations can proceed, preventing unauthorized actions; for example, a regular user cannot execute commands to modify the Broker configuration.
[0038] After passing permission checks, the system performs caching checks to improve efficiency. The tool adaptation layer maintains a read cache with a configurable lifespan (e.g., 30 seconds) and is managed using an LRU (Least Recently Used) strategy, with a maximum of 10,000 entries. If the current operation request (such as querying the status of a topic) hits the cache and the result is still valid, the system will directly return the cached result, thus avoiding duplicate queries to the Kafka cluster, significantly reducing cluster load and improving response speed. Only when the cache is not hit will the system execute the actual tool call and interact with the target Kafka cluster. After the call is completed, the system writes the new call result to the cache for subsequent use; then it generates an audit log containing a full-link tracing identifier and writes it to the operation log repository. The trace_id is associated with all operation records, ensuring traceability for every operation and meeting the security audit requirements of enterprise-level operations and maintenance.
[0039] In some embodiments, see Figure 3 , Figure 3 This is a flowchart illustrating steps S301-S303 provided in the embodiments of this application. The causal inference and counterfactual test can be performed through steps S301-S303, which will be explained in conjunction with each step.
[0040] In step S301, the fault phenomena observed by the tool are mapped to nodes in a fault cause-effect graph constructed from multiple cause-effect chains; In step S302, if there is a directed path between the phenomenon nodes in the graph, it is determined to be a causal relationship; if there is only co-occurrence without a path, it is determined to be a correlation. In step S303, a counterfactual question is posed to the large language model to confirm the causal relationship by posing a query on whether the fault phenomenon would disappear if the root cause did not exist.
[0041] Failures in Kafka systems are often concurrent; for example, a partition going offline may simultaneously be accompanied by increased Broker CPU usage and ZooKeeper session timeouts. Simple correlation analysis cannot determine which is the root cause and which is a derivative phenomenon. This embodiment addresses this problem by constructing a "failure cause-effect graph." This graph is a directed graph where nodes represent various failure events (such as "ZooKeeper anomaly," "Broker disconnection," and "partition offline"), and directed edges represent causal relationships formed by multiple explicit causal chains.
[0042] When performing causal inference, the system maps multiple fault phenomena observed through tools to corresponding nodes in a fault causal graph. Then, the system checks whether a directed path exists between these activated phenomenon nodes in the graph. If a directed path exists, for example, starting from the "ZooKeeper anomaly" node and following a directed edge to "Broker disconnected," and then to "partition offline," then a causal relationship is determined between these phenomena. Conversely, if two phenomenon nodes appear simultaneously in the graph but there is no directed path connecting them, the system determines they are only related, possibly caused by another undiscovered common cause, and therefore no conclusion is drawn. To further confirm, this embodiment also performs a "counterfactual test." It constructs a counterfactual question, such as "Assuming ZooKeeper did not malfunction, would the partition offline problem still exist?" and submits this question to the large language model. The large model reasones based on its internalized world knowledge and operational logic; if the answer is "no," then from a counterfactual perspective, it confirms that "ZooKeeper anomaly" is the root cause of "partition offline." The dual verification mechanism of "graph structure path judgment + counterfactual reasoning" provided in this embodiment greatly improves the accuracy of root cause localization.
[0043] In some embodiments, see Figure 4 , Figure 4 This is a flowchart illustrating steps S401-S403 provided in the embodiments of this application. The method further includes steps S401-S403, which will be explained in conjunction with each step.
[0044] In step S401, after locating the root cause, a remediation plan is generated based on the solutions in the knowledge base; In step S402, based on the operational risk level, ordinary operations are executed automatically, critical operations are executed after user confirmation, and high-risk operations are only suggested to be executed manually. In step S403, the tool is called again for verification after the repair, and the root cause and repair solution identified in this investigation are stored as long-term memory.
[0045] Once the root cause of the fault is identified, the system will combine the solutions and inspection steps associated with the root cause in the knowledge base to automatically generate a repair plan, including specific operation steps, commands or API calls to be executed, and parameters.
[0046] Before executing a repair plan, the system categorizes repair operations into different levels: for low-risk "ordinary operations" (such as querying status), the system can execute them automatically; for "critical operations" with some risk (such as restarting a consumer group), the system needs to generate a confirmation request, which is then executed only after secondary confirmation by operations personnel; for high-risk "high-danger operations" (such as modifying Broker core configuration or reassigning partitions), the system only returns operation suggestions to the user, who then judges and executes them manually, thereby minimizing the risk of automatic operations escalating the fault. After the repair operation is executed, the system re-enters an "iterative loop of inference and tool invocation," calling verification tools to check whether the previous fault symptoms have been eliminated. Finally, regardless of whether the repair is successful or not, the entire process of this investigation, including the ultimate root cause, the repair plan adopted, and its effects, will be stored as a new long-term memory. This approach enables the agent to learn from each practical experience, continuously enriching its experience base, and responding more quickly and accurately when encountering similar faults in the future.
[0047] In some embodiments, the knowledge base employs a dual-path recall using dense vectors and sparse keywords, and sorts the two recall results using a reciprocal ranking fusion algorithm. After filtering with a cosine similarity threshold, the relevant knowledge entries are provided as context to the large language model.
[0048] The knowledge base provided in this embodiment adopts a hybrid retrieval strategy, namely, dual-path recall using dense vectors and sparse keywords, and the results are fused using a reciprocal ranking fusion algorithm.
[0049] Specifically, when the system needs to retrieve knowledge related to the current fault from the knowledge base, it initiates two recall requests simultaneously. One is "dense vector recall," which uses an embedding model to convert both the query text and knowledge entries in the knowledge base into high-dimensional vectors. Then, it retrieves the semantically closest knowledge by calculating the cosine similarity between vectors, which can capture synonyms and deep semantic relationships. The other is "sparse keyword recall" (such as the BM25 algorithm), which performs retrieval based on word matching and is very effective for precisely matching proper nouns (such as specific Kafka parameter names or error codes). Each of the two recalls returns a ranked list of documents. Then, the system uses the inverse ranking fusion algorithm s_{rrf}(d)=∑_{r∈R}1 / (k+rank_r(d)) to fuse the two ranked lists. Here, R is the set of recall channels, rank_r(d) is the ranking of document d in channel r, and k is a constant (60 in this scheme) used to smooth ranking differences. The algorithm comprehensively considers a document's ranking across different channels; the higher the ranking, the higher the score. After merging and ranking, the system also sets a cosine similarity threshold (default 0.80) for filtering, eliminating entries with insufficient semantic relevance. Finally, the selected high-quality, highly relevant knowledge entries are provided as context, along with the original question, to the large language model, thereby generating accurate and reliable answers or inference conclusions, effectively suppressing model illusions.
[0050] In summary, the embodiments of this application have the following beneficial effects: By combining explicit causal chains with fault cause-effect graph matching and counterfactual testing, the system effectively distinguishes between fault correlation and causality, significantly improving the accuracy of root cause localization. A three-tiered screening mechanism—functional semantic matching, cost scoring, and output pattern sufficiency verification—enables intelligent and efficient tool selection. An evidence chain backtracking self-correction mechanism endows the agent with self-correction capabilities, preventing endless troubleshooting loops. A hierarchical management system for long short-term memory and a three-dimensional scoring forgetting mechanism effectively manage contextual information, maintaining consistency over long interactions. RBAC permission hierarchical management and full-link audit tracing ensure the security and traceability of operational processes. A dedicated knowledge base and tool adaptation tailored to Kafka business characteristics enable excellent adaptability to different deployment scenarios and rapid learning capabilities for new types of faults.
[0051] Based on the same inventive concept, this application also provides a fault diagnosis device for operation and maintenance intelligent agents based on large language models, which corresponds to the fault diagnosis method for operation and maintenance intelligent agents based on large language models in the first embodiment. Since the principle of the device in this application is similar to the fault diagnosis method for operation and maintenance intelligent agents based on large language models, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be described again.
[0052] like Figure 5 As shown, Figure 5 This is a schematic diagram of the structure of the operation and maintenance intelligent agent fault diagnosis device 500 based on a large language model provided in this application embodiment. The operation and maintenance intelligent agent fault diagnosis device 500 based on a large language model includes: The parsing module 501 is used to receive operation and maintenance requirements and parse them to generate standardized requirement instructions; The generation module 502 is used to perform task planning based on the standardized requirement instructions and generate a priority task queue. The first execution module 503 is used to execute the iterative loop of reasoning and tool invocation. In each iteration, the large language model generates reasoning trajectory and action intention based on the current short-term memory. Selection module 504 is used to select the optimal tool from the tool registry based on the action intent, through functional semantic matching, cost scoring and output mode sufficiency verification. The second execution module 505 is used to execute the selected tool, operate on the target cluster, and obtain observation results; The third execution module 506 is used to match the observation results with the explicit causal chain and fault causal graph in the knowledge base that points from the fault phenomenon to the root cause, and to perform causal inference and counterfactual testing to locate the root cause.
[0053] Those skilled in the art should understand that Figure 5 The functions of each unit in the fault diagnosis device 500 based on a large language model can be understood by referring to the relevant description of the fault diagnosis method based on a large language model. Figure 5 The functions of each unit in the fault diagnosis device 500 based on a large language model can be implemented by a program running on a processor or by specific logic circuits.
[0054] In one possible implementation, if the root cause is not located and the maximum number of iterations is not exceeded in the iterative loop of reasoning and tool invocation, self-correction is triggered; wherein, the self-correction is based on the action observation evidence chain accumulated in this investigation, backtracks to locate the first suspicious step, attributes the error type and adjusts the reasoning direction or tool strategy, and then returns to the reasoning step of the iterative loop.
[0055] In one possible implementation, the cost score in the selection of the optimal tool is based on the formula c_t=0.4. d_t+0.3 r_t+0.3 The calculation is p_t, where c_t is the tool cost score, d_t is the normalized call time, r_t is the normalized resource consumption, and p_t is the normalized risk level. The smaller the cost score, the higher the priority.
[0056] In one possible implementation, the device further includes a processing module 507, for: For long-term memory, the following processing is performed: based on experience in cross-session history retrieval, using the formula s_m=0.7 cos(q,e_m)+0.2 i_m+0.1 a_m calculates the memory retrieval score, where s_m is the retrieval score, cos(q,e_m) is the cosine similarity between the query vector and the memory vector, i_m is the memory importance, and a_m is the memory timeliness score; and is based on the formula f_m=0.5. (1-i_m)+0.3 (1 / (1+n_m))+0.2 ρ_m calculates the forgetting score, where f_m is the forgetting score, n_m is the number of visits, and ρ_m is the age ratio. Memories that exceed the forgetting threshold or lifespan are cleared.
[0057] In one possible implementation, the second execution module 505 is further configured to: Role-based access control, permission interception, and cache judgment; If the operation request hits the read cache and the permissions are granted, the cached result is returned directly; otherwise, the actual tool call is executed, the result is written to the cache, and an audit log containing end-to-end tracing identifiers is generated.
[0058] In one possible implementation, the third execution module 506 is specifically used for: Map the fault phenomena observed by the tool to nodes in a fault cause-effect graph constructed from multiple causal chains; If there is a directed path between the phenomenon nodes in the graph, it is determined to be a causal relationship; if there is only co-occurrence without a path, it is determined to be a correlation. By posing a counterfactual question—whether the fault would disappear if the root cause did not exist—a query is made to the large language model to confirm the causal relationship.
[0059] In one possible implementation, the device further includes a repair module 508, for: After locating the root cause, a remediation plan is generated based on the solutions in the knowledge base; Based on the operational risk level, routine operations are executed automatically, critical operations are executed after user confirmation, and high-risk operations are only suggested to be executed manually. After the repair, the tool is called again for verification, and the root cause and repair solution found in this investigation are stored as long-term memory.
[0060] In one possible implementation, the knowledge base employs a dual-path recall using dense vectors and sparse keywords, and sorts the two recall results using a reciprocal ranking fusion algorithm. After filtering with a cosine similarity threshold, the relevant knowledge entries are provided as context to the large language model.
[0061] The aforementioned fault diagnosis device for operations and maintenance agents based on large language models effectively distinguishes between fault correlation and causality by combining explicit causal chain and fault causal graph matching with counterfactual testing, significantly improving the accuracy of root cause localization. Through a three-level screening mechanism of functional semantic matching, cost scoring, and output mode sufficiency verification, it achieves intelligent and efficient tool selection. The evidence chain backtracking self-correction mechanism endows the agent with self-correction capabilities, avoiding deadlocks in the troubleshooting process. Through hierarchical management of long short-term memory and a three-dimensional scoring forgetting mechanism, it effectively manages contextual information and maintains consistency over long interactions. Through RBAC permission hierarchical management and full-link audit tracing, it ensures the security and traceability of operations and maintenance. Through a dedicated knowledge base and tool adaptation tailored to Kafka business characteristics, it achieves good adaptability to different deployment scenarios and rapid learning capabilities for new types of faults.
[0062] like Figure 6 As shown, Figure 6 This is a schematic diagram of the composition structure of the electronic device 600 provided in the embodiments of this application. The electronic device 600 includes: The device includes a processor 601, a storage medium 602, and a bus 603. The storage medium 602 stores machine-readable instructions that can be executed by the processor 601. When the electronic device 600 is running, the processor 601 communicates with the storage medium 602 via the bus 603. The processor 601 executes the machine-readable instructions to perform the steps of the fault diagnosis method for operation and maintenance intelligent agents based on large language models described in the embodiments of this application.
[0063] In practical applications, the various components in the electronic device 600 are coupled together via a bus 603. It is understood that the bus 603 is used to achieve communication between these components. In addition to a data bus, the bus 603 also includes a power bus, a control bus, and a status signal bus. However, for clarity, in... Figure 6 The general designated all buses as Bus 603.
[0064] The aforementioned electronic devices effectively distinguish between fault correlation and causality by combining explicit causal chaining and fault cause-effect graph matching with counterfactual testing, significantly improving the accuracy of root cause localization. A three-level screening mechanism—functional semantic matching, cost scoring, and output pattern sufficiency verification—enables intelligent and efficient tool selection. An evidence chain backtracking self-correction mechanism endows the agent with self-correction capabilities, avoiding endless troubleshooting loops. A hierarchical management system for long short-term memory and a three-dimensional scoring forgetting mechanism effectively manage contextual information, maintaining consistency over long interactions. RBAC permission hierarchical management and full-link audit tracing ensure the security and traceability of operational procedures. A dedicated knowledge base and tool adaptation tailored to Kafka business characteristics enable good adaptation to different deployment scenarios and rapid learning capabilities for new types of faults.
[0065] This application also provides a computer-readable storage medium storing executable instructions. When the executable instructions are executed by at least one processor 601, the fault diagnosis method for operation and maintenance intelligent agents based on large language models described in this application is implemented.
[0066] In some embodiments, the storage medium may be a magnetic random access memory (FRAM), a read-only memory (ROM), or a programmable read-only memory (PROM). Erasable Programmable Read-Only Memory (EPROM) Electrically Erasable Programmable Read-Only Memory (EEPROM) Read-only memory, flash memory, magnetic surface storage, optical disc, or CD-ROM ROM, Compact Disc Read It can be a memory such as a memory only; or it can be a device that includes one or any combination of the above-mentioned memories.
[0067] In some embodiments, executable instructions may take the form of a program, software, software module, script, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a standalone program or as a module, component, subroutine, or other unit suitable for use in a computing environment.
[0068] As an example, executable instructions may, but do not necessarily, correspond to files in the file system. They may be stored as part of a file that holds other programs or data, for example, in one or more scripts in a HyperText Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple collaborating files (e.g., a file that stores one or more modules, subroutines, or code sections).
[0069] As an example, executable instructions can be deployed to execute on a single computing device, or on multiple computing devices located in one location, or on multiple computing devices distributed across multiple locations and interconnected via a communication network.
[0070] The aforementioned computer-readable storage medium, through explicit causal chaining and fault causal graph matching combined with counterfactual testing, effectively distinguishes between fault correlation and causality, significantly improving the accuracy of root cause localization. A three-level screening mechanism—functional semantic matching, cost scoring, and output pattern sufficiency verification—enables intelligent and efficient tool selection. An evidence chain backtracking self-correction mechanism endows the agent with self-correction capabilities, avoiding endless troubleshooting loops. Long Short-Term Memory (LSTM) hierarchical management and a three-dimensional scoring forgetting mechanism effectively manage contextual information, maintaining consistency over long interactions. RBAC (Relative Backtracking Account) permission hierarchy and full-link audit tracing ensure the security and traceability of operational processes. A dedicated knowledge base and tool adaptation tailored to Kafka business characteristics enable good adaptation to different deployment scenarios and rapid learning capabilities for new types of faults.
[0071] In the several embodiments provided in this application, it should be understood that the disclosed methods and electronic devices can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple units or components may be combined, or integrated into another system, or some features may be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed may be through some interfaces, and the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.
[0072] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.
[0073] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.
[0074] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a processor-executable, non-volatile, computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, a platform server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, ROM, RAM, magnetic disks, or optical disks.
[0075] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.
Claims
1. A fault diagnosis method for operation and maintenance intelligent agents based on large language models, characterized in that, Includes the following steps: Receive operation and maintenance requests, parse and generate standardized requirement instructions; Based on the standardized requirement instructions, task planning is performed to generate a priority-based task queue; The iterative loop of execution reasoning and tool invocation is performed. In each iteration, the large language model generates reasoning trajectories and action intentions based on the current short-term memory. Based on the stated action intent, the optimal tool is selected from the tool registry through functional semantic matching, cost scoring, and output mode sufficiency verification. Execute the selected tools to operate on the target cluster and obtain observations; The observed results are matched with explicit causal chains and fault causal graphs in the knowledge base that point from the fault phenomenon to the root cause, and causal inference and counterfactual testing are performed to locate the root cause.
2. The method according to claim 1, characterized in that, In the iterative loop of reasoning and tool invocation, if the root cause is not located and the maximum number of iterations is not exceeded, self-correction is triggered. The self-correction is based on the action observation evidence chain accumulated in this investigation, backtracks to locate the first suspicious step, attributes the error type, and adjusts the reasoning direction or tool strategy before returning to the reasoning step of the iterative loop.
3. The method according to claim 1, characterized in that, In the selection of the optimal tool, cost scoring is based on the formula c_t=0.4 d_t+0.3 r_t+0.3 The calculation is p_t, where c_t is the tool cost score, d_t is the normalized call time, r_t is the normalized resource consumption, and p_t is the normalized risk level. The smaller the cost score, the higher the priority.
4. The method according to claim 1, characterized in that, The method further includes: For long-term memory, the following processing is performed: based on experience in cross-session history retrieval, using the formula s_m=0.7 cos(q,e_m)+0.2 i_m+0.1 a_m calculates the memory retrieval score, where s_m is the retrieval score, cos(q,e_m) is the cosine similarity between the query vector and the memory vector, i_m is the memory importance, and a_m is the memory timeliness score; and is based on the formula f_m=0.
5. (1-i_m)+0.3 (1 / (1+n_m))+0.2 ρ_m calculates the forgetting score, where f_m is the forgetting score, n_m is the number of visits, and ρ_m is the age ratio. Memories that exceed the forgetting threshold or lifespan are cleared.
5. The method according to claim 1, characterized in that, When executing the selected tool to operate on the target cluster and obtain observation results, the method further includes: Role-based access control, permission interception, and cache judgment; If the operation request hits the read cache and the permissions are granted, the cached result is returned directly; otherwise, the actual tool call is executed, the result is written to the cache, and an audit log containing end-to-end tracing identifiers is generated.
6. The method according to claim 1, characterized in that, The aforementioned causal inference and counterfactual testing include: Map the fault phenomena observed by the tool to nodes in a fault cause-effect graph constructed from multiple causal chains; If there is a directed path between the phenomenon nodes in the graph, it is determined to be a causal relationship; if there is only co-occurrence without a path, it is determined to be a correlation. By posing a counterfactual question—whether the fault would disappear if the root cause did not exist—a query is made to the large language model to confirm the causal relationship.
7. The method according to claim 1, characterized in that, The method further includes: After locating the root cause, a remediation plan is generated based on the solutions in the knowledge base; Based on the operational risk level, routine operations are executed automatically, critical operations are executed after user confirmation, and high-risk operations are only suggested to be executed manually. After the repair, the tool is called again for verification, and the root cause and repair solution found in this investigation are stored as long-term memory.
8. The method according to claim 1, characterized in that, The knowledge base employs a dual-path recall strategy using dense vectors and sparse keywords. The results from both paths are then ranked using a reciprocal ranking fusion algorithm. After filtering with a cosine similarity threshold, relevant knowledge items are provided as context to the large language model.
9. A fault diagnosis device for operation and maintenance intelligent agents based on a large language model, characterized in that, The device includes: The parsing module is used to receive operation and maintenance requirements and parse them to generate standardized requirement instructions; The generation module is used to plan tasks according to the standardized requirement instructions and generate a priority task queue. The first execution module is used to execute the iterative loop of reasoning and tool invocation. In each iteration, the large language model generates the reasoning trajectory and action intention based on the current short-term memory. The selection module is used to select the optimal tool from the tool registry based on the action intent, through functional semantic matching, cost scoring, and output mode sufficiency verification. The second execution module is used to execute the selected tool, operate on the target cluster, and obtain observation results; The third execution module is used to match the observation results with the explicit causal chain and fault causal graph in the knowledge base that points from the fault phenomenon to the root cause, and to perform causal inference and counterfactual testing to locate the root cause.
10. An electronic device, characterized in that, include: The device includes a processor, a storage medium, and a bus. The storage medium stores machine-readable instructions executable by the processor. When the electronic device is running, the processor communicates with the storage medium via the bus, and the processor executes the machine-readable instructions to perform the fault diagnosis method for an operation and maintenance intelligent agent based on a large language model as described in any one of claims 1 to 8.
