Operation and maintenance question-answering system and method based on large language model workflow

Abstract

The invention discloses an operation and maintenance question-answering system and method based on a large language model workflow, and mainly relates to the technical field of information security. Comprising a question semantic classification module used for performing semantic analysis and classification on questions through a text classification large language model when the questions are input, and identifying categories of the questions; the process selection and execution module is used for selecting and executing a corresponding process according to the semantic category of the question to generate a preliminary answer; and the answer generation and formatting module is used for processing the preliminary answers through a text generation large language model and formatting the preliminary answers into final answers which can be understood by the user. The method has the beneficial effects that the accuracy of operation and maintenance knowledge retrieval can be improved, problems can be converted into operation and maintenance instructions to execute related operations, meanwhile, the method has the capability of analyzing and deciding operation and maintenance problems, and the response quality and the capability of processing complex problems are remarkably improved.

Description

Technical Field

[0001] The present invention relates to the field of intelligent IT system operation and maintenance technology, and specifically to an operation and maintenance question-answering system and method based on a large language model workflow. Background Art

[0002] With the rapid development of information technology and automation technology, the operation and maintenance of IT systems is becoming increasingly complex. Operation and maintenance engineers not only need to quickly diagnose and repair system failures, but also need to handle daily maintenance tasks, optimize system performance, and respond to emergencies. However, most existing operation and maintenance question-and-answer systems rely on rule-based QA pairs or knowledge graphs. With the rapid development of information technology and automation technology, the operation and maintenance of IT systems is becoming increasingly complex. Operation and maintenance engineers not only need to quickly diagnose and repair system failures, but also need to handle daily maintenance tasks, optimize system performance, and respond to emergencies. To meet this challenge, traditional operation and maintenance question-and-answer systems usually rely on rule-based QA pairs or knowledge graphs.

[0003] These operation and maintenance question-answering systems have the following problems: 1. Limited retrieval accuracy: Traditional operation and maintenance question-answering systems usually rely on static knowledge bases or keyword matching-based technologies, which often fail to accurately understand the deep semantics of questions when faced with natural language input. Even if knowledge graphs are introduced, it is still difficult to obtain accurate answers for complex or incomplete question descriptions, and the accuracy and comprehensiveness of knowledge retrieval are limited; 2. Lack of intelligent task execution capabilities: The current operation and maintenance question-and-answer system can only provide reference answers and cannot directly convert user questions into executable operation and maintenance instructions. This limitation requires operation and maintenance personnel to manually complete a large number of operation steps when solving problems, which reduces efficiency. In particular, when dealing with emergency situations or cross-system operations, the process cannot be effectively automated; 3. Experience is difficult to solidify and pass on: A large amount of knowledge and experience has been accumulated in operation and maintenance work, but traditional question-answering systems cannot structure and systematically apply this implicit knowledge to the analysis of practical problems. As a result, experienced engineers still need to rely on personal experience when solving problems, while junior engineers lack effective technical support and find it difficult to handle complex problems independently. 4. Unable to adapt to dynamically changing operation and maintenance scenarios: With the continuous evolution of IT architecture, such as cloud computing and microservice architecture, systems have become more distributed and dynamic. Traditional operation and maintenance question-and-answer systems lack the ability to perform real-time analysis and dynamically adapt, and are unable to effectively cope with the diversification and complexity of operation and maintenance scenarios.

[0004] In view of the above problems, how to design a system that can improve the accuracy of operation and maintenance knowledge retrieval while also having the ability to analyze and make decisions on operation and maintenance problems is an issue that needs to be urgently addressed in this field. Summary of the invention

[0005] The purpose of the present invention is to provide an operation and maintenance question-answering system and method based on a large language model workflow, which can not only improve the accuracy of operation and maintenance knowledge retrieval, but also convert questions into operation and maintenance instructions to perform related operations. At the same time, it has the ability to analyze and make decisions on operation and maintenance issues, which significantly improves the quality of answers and the ability to handle complex problems.

[0006] In order to achieve the above object, the present invention is implemented through the following technical solutions: On the one hand, the present invention provides an operation and maintenance question-answering system based on a large language model workflow, comprising: The question semantic classification module is used to: when a question is input, analyze and classify the question semantics through the text classification language model to identify the category of the question; The process selection and execution module is used to: select and execute the corresponding process according to the semantic category of the question to generate a preliminary answer; The answer generation and formatting module is used to: process the preliminary answers through the text generation large language model and format them into the final answers that can be understood by the user.

[0007] On the other hand, a method for an operation and maintenance question-answering system based on a large language model workflow is provided, based on the above-mentioned operation and maintenance question-answering system based on a large language model workflow, comprising the following steps: S1: Use the text classification language model to semantically analyze and classify the input question and identify the semantic category of the question; S2: According to the semantic category of the question identified in step S1, select and execute the corresponding process to generate a preliminary answer; S3: The preliminary answer generated in step S2 is processed through a text generation large language model and formatted into a final answer.

[0008] Preferably, in step S1, the question categories include: knowledge retrieval category, operation and maintenance task category and operation and maintenance analysis category.

[0009] Preferably, in step S2, according to the semantic category of the question identified in step S1, a corresponding process is selected and executed, specifically: For knowledge retrieval questions, the execution process is as follows: if the question is about knowledge query, the RAG process is executed to retrieve knowledge through the knowledge base or search engine and generate answers; For operation and maintenance task-related problems, the execution process is as follows: For operation and maintenance problems involving operation execution, the operation and maintenance agent process is executed to convert the problem into calling the operation and maintenance agent to execute the relevant operation tasks; For operation and maintenance analysis issues, the execution process is as follows: If the problem involves operation and maintenance decision-making or analysis, execute the operation and maintenance analysis process, combine system logs and monitoring data information, analyze and provide solutions.

[0010] Preferably, the RAG process comprises the following steps: The question text is encoded into an index vector and keywords are extracted through the text encoding model. Through vector search and keyword search of the operation and maintenance knowledge base, the most relevant text list is obtained; The re-ranking model is used to re-rank the text list according to relevance to obtain the retrieved text data; Use text to generate a large language model, use the retrieved text data as context, and generate preliminary answers; The answer matching evaluation model is called. If the evaluation passes, the answer is returned. If the evaluation fails, the search engine tool is called to search, and the recall model is used for re-ranking. The text generation large language model is called to generate a new answer as the return result.

[0011] Preferably, the Agent process includes the following steps: Use a large language model for text classification to perform semantic analysis and classification of questions to determine the specific type of question and processing requirements; Based on the text classification results, the system routes the problem to the appropriate operation and maintenance agent; The operation and maintenance agent performs corresponding operations or queries in the system according to the type of problem and key information; The operation and maintenance agent organizes the operation results into answers.

[0012] Preferably, the operation and maintenance agent includes but is not limited to a script execution agent, an indicator data query agent, a log data query agent, an operation and maintenance inspection agent, and an API call agent; The script execution agent uses a large language model to extract the execution command in the question and the server address that executes the command, and then performs a command compliance check. If the check passes, the command is executed on the target server and the execution result is returned as the answer. If the check fails, the command is returned as non-compliant and cannot be executed. The indicator data query agent is: extract the indicator ID, time range and server where the indicator data is located in the question through the large language model, then splice it into query SQL through the SQL template, and perform SQL compliance verification. If the verification passes, the query SQL is executed on the target indicator database and the query result is returned as the answer; if the verification fails, it returns that the query is non-compliant and cannot be executed; The log data query agent is as follows: it uses a large language model to extract the log ID, time range, and location of the log data in the question, and then uses a SQL template different from the SQL template in the above-mentioned indicator data query agent to splice the query SQL, and performs SQL compliance verification; if the verification passes, the query SQL is executed on the target log database different from the above-mentioned indicator data query agent, and the query result is returned as the answer; if the verification fails, it returns that the query is non-compliant and cannot be executed; The operation and maintenance inspection agent extracts the inspection content in the question through a large language model, executes the corresponding predefined operation and maintenance inspection procedures according to the inspection content, generates a text report based on the inspection results, and returns it as an answer. The API call agent uses a large language model to extract the API link address and parameters in the question, then performs parameter verification and concatenates them into a call URL, and finally executes the API call and returns the result as the answer.

[0013] Preferably, the operation and maintenance analysis process includes the following steps: Input questions and prompt words into the operation and maintenance language model for analysis; The O&M language model has been trained to have strong O&M knowledge and problem analysis capabilities; The operation and maintenance big language model generates the final answer based on the analysis results of the problem, including an in-depth analysis of the problem, recommended solutions and operation steps.

[0014] Preferably, training the operation and maintenance large language model includes the following steps: The basic large language model is pre-trained with a specialized corpus in the field of operation and maintenance to grasp the basic knowledge and terminology related to operation and maintenance; The model uses the operation and maintenance task instruction data to fine-tune the supervisory instructions so that it can understand and process specific operation and maintenance instructions and operation steps; The model is trained for preference optimization through operation and maintenance experience data to improve its handling effect and decision-making ability for actual operation and maintenance problems.

[0015] Preferably, in step S3, formatting the content includes: Convert answers to rich text format; Organize content through paragraphs, headings, bullet points, etc. Convert data and information into tabular form.

[0016] Compared with the prior art, the beneficial effects of the present invention are: The present invention proposes an operation and maintenance question-answering system and method based on a large language model workflow, which can not only improve the accuracy of operation and maintenance knowledge retrieval, but also convert questions into operation and maintenance instructions to perform related operations. At the same time, it has the ability to analyze and make decisions on operation and maintenance issues, which significantly improves the quality of answers and the ability to handle complex problems. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 It is a schematic diagram of the system structure of the present invention; Figure 2 is a flow chart of the method of the present invention; Figure 3 is a business process diagram of the present invention; Figure 4 is a RAG flow chart of the present invention; Figure 5 It is the operation and maintenance Agent flow chart of the present invention; Figure 6 It is the operation and maintenance analysis flow chart of the present invention. DETAILED DESCRIPTION

[0018] The present invention will be further described below in conjunction with specific embodiments. It should be understood that these embodiments are only used to illustrate the present invention and are not intended to limit the scope of the present invention. In addition, it should be understood that after reading the content taught by the present invention, those skilled in the art can make various changes or modifications to the present invention, and these equivalent forms fall within the scope limited by the application equally.

[0019] In the present invention, terms such as "upper", "lower", "left", "right", "front", "back", "vertical", "horizontal", "side", "bottom", etc. indicate directions or positional relationships based on the directions or positional relationships shown in the accompanying drawings. They are relational words determined only for the convenience of describing the structural relationships of the various parts or elements of the present invention, and do not specifically refer to any part or element in the present invention and should not be understood as limitations on the present invention.

[0020] Embodiment 1: like Figure 1 As shown, this embodiment provides an operation and maintenance question-answering system based on a large language model workflow, including: The question semantic classification module is used to: when a question is input, analyze and classify the question semantics through the text classification language model to identify the category of the question; The process selection and execution module is used to: select and execute the corresponding process according to the semantic category of the question to generate a preliminary answer; The answer generation and formatting module is used to: process the preliminary answers through the text generation large language model and format them into the final answers that can be understood by the user.

[0021] like Figure 2As shown, this embodiment also provides a method for applying the above-mentioned operation and maintenance question-answering system based on the large language model workflow, comprising the following steps: S1: Use the text classification language model to perform semantic analysis and classification on the input question and identify the semantic category of the question; S2: According to the semantic category of the question identified in step S1, select and execute the corresponding process to generate a preliminary answer; S3: The preliminary answer generated in step S2 is processed through a text generation large language model and formatted into a final answer.

[0022] like Figure 3 As shown, this embodiment shows the specific process of semantic classification of operation and maintenance problems input by users. The present invention performs semantic analysis on input problems through a text classification language model and automatically classifies the problems into different types to match the corresponding processing flow: The prompt is: "Please classify the operation and maintenance issues entered by the user according to their types. The classification results can only be selected in the following three categories: Knowledge retrieval questions: These questions involve searching for existing knowledge or documents. Usually, users want to obtain explanations and solutions to certain errors or phenomena. Operation and maintenance agent type questions: These questions require specific operations to be performed, such as querying system status or performing system operation tasks; Operation and maintenance analysis issues: These issues involve complex analysis of system failures or abnormal situations, and usually require comprehensive analysis of multi-dimensional operation and maintenance data to provide solutions. "; When the operation and maintenance engineer enters the following question, the system will process it as follows: Question 1: "What is the cause and solution for the live broadcast error code 51002?" The system uses a text classification language model to perform semantic analysis on the question and identifies that the question is a query for existing knowledge. Error code 51002 is a common error type in the operation and maintenance field, and the system determines that this type of problem is a knowledge retrieval problem. Therefore, the system executes the RAG (retrieval enhanced generation) process to retrieve relevant documents or historical records through the internal knowledge base and provide a solution.

[0023] Question 2: "Query the latest CPU usage of host A" Based on the key words in the question, "Query the CPU usage of host A", the system classifies the question as an operation and maintenance task that requires specific operations. The system routes the question to the appropriate operation and maintenance agent, which queries the latest CPU usage of host A in the monitoring system and returns the result to the user.

[0024] Question 3: "MySQL data master and slave are not synchronized, what is the reason and how to solve it?" This problem involves complex system failure analysis. The text classification large language model identifies this problem as an operation and maintenance analysis problem. The system calls the operation and maintenance analysis process, analyzes relevant logs and monitoring data, and combines the decision-making ability of the large language model to generate and return the cause analysis and solution of the failure.

[0025] Embodiment 2: like Figure 4 As shown, this embodiment demonstrates how the system generates an answer through the RAG (retrieval enhanced generation) process for the knowledge retrieval question "What is the cause and solution of the live broadcast error code 51002?"

[0026] When the operation and maintenance engineer enters the question "What is the cause and solution of live broadcast error code 51002?", the system performs the following steps: (1) Text encoding and keyword extraction First, the system encodes the input text through a text encoding model, converts the question text into a high-dimensional index vector, and extracts the keywords "live broadcast", "error code 51002", "cause", and "solution". The text encoding model used in this embodiment is bce-embedding-base_v1.

[0027] (2) Vector search and keyword search The system uses keywords and vectors to search the operation and maintenance knowledge base and find documents or historical records related to "error code 51002". The documents in the knowledge base include various formats (such as WORD, EXCEL, PDF, etc.). These documents have been converted into searchable text data after OCR and NLP processing, and are segmented, word-segmented, and vector-indexed according to paragraphs, titles, and semantics.

[0028] (3) Document reordering The system re-ranks the multiple relevant documents retrieved. The re-ranking model used scores the documents based on the relevance between the semantics of the question and the document content, and ranks the most relevant documents first. At this point, the system may find multiple relevant documents, such as "technical documentation for error code 51002" and "operation and maintenance records for solving this error in history". The re-ranking model used in this embodiment is bce-reranker-base_v1.

[0029] (4) Preliminary answer generation The system calls the text generation large language model and uses the most relevant documents as context to answer the user's question. The system generates a preliminary answer to the error code 51002. The answer includes the cause of the error code (such as abnormal network connection) and the corresponding solution steps (such as checking the network configuration, restarting related services, etc.). Prompt words such as: "You are an operation and maintenance expert. Use the following content as your knowledge and put it in <context>< / context> Inside XML tags; <context> < / context> When answering a user: If you don’t know, say you don’t know. If you don’t know when you are unsure, seek clarification; Avoid mentioning that you took the information out of context; Please answer in Chinese. "; Based on the prompt word and document context, the large language model generates preliminary answers, such as: “Reason: Error code 51002 is usually caused by an unstable network connection leading to a live broadcast interruption; Solution: Please check the network configuration, restart related services or servers, and ensure that the network connection is normal. ".

[0030] (5) Answer matching evaluation The system evaluates the generated preliminary answers through the answer matching evaluation model. If the answer has a high matching degree, the system will directly return the answer. If the evaluation fails, the system calls the search engine tool to conduct further network search.

[0031] (6) Search engine assisted retrieval (optional step) If there is not enough information in the internal knowledge base, the system will call the search engine tool to search the Internet resources. The retrieved external data will be re-ranked by the recall model and the text-generated large language model will generate new answers again.

[0032] (7) Answer returned The final answer will be returned to the operation and maintenance engineer, including the cause of error code 51002, the corresponding solution and reference document link, etc.

[0033] Embodiment 3: like Figure 5 As shown, this embodiment shows how the system performs relevant operations and generates answers through the operation and maintenance Agent process for the operation and maintenance task type problem of "querying the latest CPU usage of host A".

[0034] (1) Text analysis and classification When the operation and maintenance engineer inputs "query the latest CPU usage of host A", the system first performs semantic analysis on the question and prompt words through the text classification large language model, and outputs the recommended "indicator query type" operation and maintenance agent.

[0035] (2) Index data query agent The indicator ID, time range and server where the indicator data is located in the question are extracted through the large language model, which are: CPU usage, latest, and host A. Then the query SQL is spliced ​​into SQL through the SQL template and SQL compliance is checked. For example, the generated SQL is: "SELECT cpu_usage FROM cpu_metrics WHERE host_name = 'A' ORDERBY timestamp DESC LIMIT 1;" If the SQL compliance check passes, the query SQL is executed on the target indicator database and the query result is returned as the answer; if the SQL compliance check fails, it is returned that the query is non-compliant and cannot be executed.

[0036] (3) Answer returned Get the CPU usage data of host A, for example, the current CPU usage is "75%". Then, the indicator data query agent returns the data to the system, for example: "The latest CPU usage of host A is: 75%.".

[0037] Embodiment 4: like Figure 6 As shown, this embodiment shows how the system handles the operation and maintenance analysis problem of "MYSQL data master-slave is out of sync, what is the reason, and how to solve it?" and generates analysis results and solutions through the operation and maintenance analysis process.

[0038] (1) Question input and prompt word setting The operation and maintenance engineer inputs the question: "MySQL data master and slave are not synchronized. What is the reason and how to solve it?" The system inputs the question and prompt words into the operation and maintenance language model. An example of prompt words is: "You are an operation and maintenance expert. Please answer the questions raised by users professionally and accurately; If you don't know the answer, reply 'I don't know'; Please answer in Chinese. "

[0039] (2) Model analysis and processing After receiving the question and prompt words, the operation and maintenance language model conducts in-depth analysis. Based on the MYSQL master-slave synchronization mechanism and related fault points involved in the question, the model combines its training knowledge to diagnose the problem and generate the final answer: "MYSQL data master-slave asynchrony may be caused by the following reasons: network delay, excessive load on the master server, inconsistent configuration; It is recommended to follow the steps below to solve the problem: 1. Check the network connection between the master and slave servers to ensure it is stable; 2. Monitor and optimize the load of the main server; 3. Confirm the configuration consistency of the master and slave servers; 4. If the problem persists, try restarting the synchronization process. ".

[0040] (3) Answer returned The system returns the generated final answer, providing an in-depth analysis of the problem and a detailed solution.

[0041] Embodiment 5: like Figure 6 As shown, this embodiment shows how to train a large operation and maintenance language model to improve its performance and accuracy in handling operation and maintenance problems. The training process includes three main stages: pre-training, supervised instruction fine-tuning, and preference optimization.

[0042] (1) Pre-training stage First, the basic large language model uses Qwen2-72B-Instruct and is pre-trained using a specialized corpus in the field of operation and maintenance. The goal of this stage is to enable the model to master basic knowledge and terminology related to operation and maintenance. The model is pre-trained using LORA (Low-Rank Adaptation) technology to ensure that the model can understand the professional terminology and basic concepts in the field of operation and maintenance, laying the foundation for subsequent fine-tuning and optimization.

[0043] The pre-training data is in plain text format, for example: "'In computer systems, RAID (Redundant Array of Independent Disks) is a technology that combines multiple hard drives together to improve data redundancy and performance.'; 'System monitoring tools usually include indicators such as CPU usage, memory usage, and disk I / O to help operation and maintenance personnel understand the system status in real time. '".

[0044] (2) Supervisory instruction fine-tuning stage After pre-training is completed, the model enters the supervised instruction fine-tuning stage. In this stage, the operation and maintenance task instruction data is used for fine-tuning to further improve the model's ability to understand specific operation and maintenance instructions and operation steps. LORA technology is used to fine-tune the model at this stage to ensure that the model can handle complex instructions and operation steps in actual operation and maintenance tasks.

[0045] The supervised instruction fine-tuning data sample is: { "type": "chatml", "messages": [ { "role": "system", "content": "You are Xiaohai, an artificial intelligence assistant created by Haikan Co., Ltd., focusing on solving operation and maintenance problems." }, { "role": "user", "content": "MySQL data master and slave are not synchronized, what is the reason and how to solve it?" }, { "role": "assistant", "content": "MYSQL data master-slave asynchrony may be caused by the following reasons: xxxxxx" } ], "source": "self-made" } (3) Preference optimization training phase Finally, the model is trained for preference optimization through operation and maintenance experience data. DPO (Direct Preference Optimization) technology is used in this stage to improve the processing effect and decision-making ability of the model in actual operation and maintenance scenarios. DPO technology optimizes the performance of the model in actual operation and maintenance scenarios so that it can better adapt to specific operation and maintenance needs and user preferences, thereby improving the accuracy and effectiveness of the model in handling operation and maintenance problems.

[0046] The preference optimization data sample is: { "conversations": [{ "from": "human", "value": "MySQL data master and slave are not synchronized, what is the reason and how to solve it?" }], "chosen": { "from": "gpt", "value": "MYSQL data master-slave asynchrony may be caused by the following reasons: xxxxx" }, "rejected": { "from": "gpt", "value": "MYSQL master-slave out of sync may be due to xxxxx" } } The above is a specific description of the preferred implementation of the present invention, but the present invention is not limited to the described embodiments. Those skilled in the art may make various equivalent modifications or substitutions without violating the spirit of the present invention. These equivalent modifications or substitutions are all included in the scope defined by the claims of this application.

Claims

1. An operation and maintenance question-answering system based on a large language model workflow, characterized in that: include: The question semantic classification module is used to: when a question is input, analyze and classify the question semantics through the text classification language model to identify the category of the question; The process selection and execution module is used to: select and execute the corresponding process according to the semantic category of the question to generate a preliminary answer; The answer generation and formatting module is used to: process the preliminary answers through the text generation large language model and format them into the final answers that can be understood by the user.

2. An operation and maintenance question-answering method based on a large language model workflow, characterized in that: An operation and maintenance question-answering system based on a large language model workflow according to claim 1, comprising the following steps: S1: Use the text classification language model to perform semantic analysis and classification on the input question and identify the semantic category of the question; S2: According to the semantic category of the question identified in step S1, select and execute the corresponding process to generate a preliminary answer; S3: The preliminary answer generated in step S2 is processed through a text generation large language model and formatted into a final answer.

3. According to claim 2, a maintenance question-answering method based on a large language model workflow is characterized in that: In step S1, the question categories include: knowledge retrieval category, operation and maintenance task category, and operation and maintenance analysis category.

4. According to claim 3, the operation and maintenance question-answering method based on a large language model workflow is characterized in that: In step S2, according to the semantic category of the question identified in step S1, a corresponding process is selected and executed, specifically: For knowledge retrieval questions, the execution process is as follows: if the question is about knowledge query, the RAG process is executed to retrieve knowledge through the knowledge base or search engine and generate answers; For operation and maintenance task-related problems, the execution process is as follows: For operation and maintenance problems involving operation execution, the operation and maintenance agent process is executed to convert the problem into calling the operation and maintenance agent to execute the relevant operation tasks; For operation and maintenance analysis issues, the execution process is as follows: If the problem involves operation and maintenance decision-making or analysis, execute the operation and maintenance analysis process, combine system logs and monitoring data information, analyze and provide solutions.

5. According to claim 4, a maintenance question-answering method based on a large language model workflow is characterized in that: The RAG process includes the following steps: The question text is encoded into an index vector and keywords are extracted through the text encoding model. Through vector search and keyword search of the operation and maintenance knowledge base, the most relevant text list is obtained; The re-ranking model is used to re-rank the text list according to relevance to obtain the retrieved text data; Use text to generate a large language model, use the retrieved text data as context, and generate preliminary answers; The answer matching evaluation model is called. If the evaluation passes, the answer is returned. If the evaluation fails, the search engine tool is called to search, and the recall model is used for re-ranking. The text generation large language model is called to generate a new answer as the return result.

6. The operation and maintenance question-answering method based on a large language model workflow according to claim 4, characterized in that: The Agent process includes the following steps: Use a large language model for text classification to perform semantic analysis and classification of questions to determine the specific type of question and processing requirements; Based on the text classification results, the system routes the problem to the appropriate operation and maintenance agent; The operation and maintenance agent performs corresponding operations or queries in the system according to the type of problem and key information; The operation and maintenance agent organizes the operation results into answers.

7. The operation and maintenance question-answering method based on a large language model workflow according to claim 4, characterized in that: The operation and maintenance agent includes but is not limited to a script execution agent, an indicator data query agent, a log data query agent, an operation and maintenance inspection agent, and an API call agent; The script execution agent is: extract the execution command in the question and the server address of the command through the large language model, and then perform command compliance verification. If the verification passes, the command is executed on the target server and the execution result is returned as the answer; If the verification fails, the command is returned as non-compliant and cannot be executed; The indicator data query agent is: extract the indicator ID, time range and server where the indicator data is located in the question through the large language model, then splice it into query SQL through the SQL template, and perform SQL compliance verification. If the verification passes, the query SQL is executed on the target indicator database and the query result is returned as the answer; If the verification fails, the query is returned as non-compliant and cannot be executed; The log data query agent is: extract the log ID, time range and location of the log data in the question through the large language model, and then splice it into query SQL through the SQL template different from the above indicator data query agent, and perform SQL compliance verification; if the verification passes, execute the query SQL on the target log database different from the above indicator data query agent, and return the query result as the answer; If the verification fails, the query is returned as non-compliant and cannot be executed; The operation and maintenance inspection agent extracts the inspection content in the question through a large language model, executes the corresponding predefined operation and maintenance inspection procedures according to the inspection content, generates a text report based on the inspection results, and returns it as an answer. The API call agent uses a large language model to extract the API link address and parameters in the question, then performs parameter verification and concatenates them into a call URL, and finally executes the API call and returns the result as the answer.

8. The operation and maintenance question-answering method based on a large language model workflow according to claim 4, characterized in that: The operation and maintenance analysis process includes the following steps: Input questions and prompt words into the operation and maintenance language model for analysis; The O&M language model has been trained to have strong O&M knowledge and problem analysis capabilities; The operation and maintenance big language model generates the final answer based on the analysis results of the problem, including an in-depth analysis of the problem, recommended solutions and operation steps.

9. The operation and maintenance question-answering method based on a large language model workflow according to claim 8, characterized in that: Training the operation and maintenance language model includes the following steps: The basic large language model is pre-trained with a specialized corpus in the field of operation and maintenance to grasp the basic knowledge and terminology related to operation and maintenance; The model uses the operation and maintenance task instruction data to fine-tune the supervisory instructions so that it can understand and process specific operation and maintenance instructions and operation steps; The model is trained for preference optimization through operation and maintenance experience data to improve its handling effect and decision-making ability for actual operation and maintenance problems.

10. The operation and maintenance question-answering method based on a large language model workflow according to claim 2, characterized in that: In step S3, formatting the content includes: Convert answers to rich text format; Organize content through paragraphs, headings, bullet points, etc. Convert data and information into tabular form.

Images