Data center, fault prediction method, electronic equipment, medium and program product

Abstract

The invention discloses a data center, a fault prediction method, electronic equipment, a medium and a program product, and is applied to the technical field of fault diagnosis. The method comprises the following steps: dividing a fault prediction task into a data collection sub-task, a data analysis sub-task, a fault diagnosis sub-task and a result generation sub-task; the data collection subtask is sent to a data collection agent to collect operation state data of the server, and the original operation state data and the data analysis subtask are sent to a data analysis agent to carry out data processing on the operation state data; and sending the operation state data and the fault diagnosis subtask to a fault diagnosis agent for fault diagnosis, and finally sending a fault diagnosis result and a result generation subtask to a result generation agent to generate a fault prediction result. The problem that complex faults cannot be accurately predicted in related technologies can be solved, and the fault prediction precision of the data center can be effectively improved.

Description

Technical Field

[0001] The present invention relates to the technical field of fault diagnosis, and in particular to a data center and a fault prediction method, an electronic device, a non-volatile storage medium, and a computer program product. Background Art

[0002] As the scale of data centers continues to increase, server fault diagnosis and fault prediction can quickly identify and resolve potential problems in data centers, prevent system crashes and data loss, and ensure business continuity and stability.

[0003] In order to ensure the normal operation of the data center, related technologies use artificial intelligence algorithms to quickly predict whether the server in the data center is faulty. However, this method trains a model for each type of fault task to make predictions. The prediction accuracy of a single fault task type of the server is high, but it cannot accurately predict complex fault tasks and cannot effectively guarantee the accurate prediction of data center faults.

[0004] In view of this, effectively improving the fault prediction accuracy of data centers is a technical problem that technical personnel in this field need to solve.

[0005] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present application, and therefore may include information that does not constitute the prior art known to ordinary technicians in the field. Summary of the invention

[0006] The present invention provides a data center and a fault prediction method, an electronic device, a non-volatile storage medium, and a computer program product, which effectively improve the fault prediction accuracy of the data center.

[0007] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0008] In one aspect, the present invention provides a fault prediction method, comprising:

[0009] The fault prediction task is divided into a data collection subtask, a data analysis subtask, a fault diagnosis subtask and a result generation subtask; the data collection subtask is sent to a data collection agent so that the data collection agent collects the original operating status data of each server in the data center; when the data collection agent completes the data collection subtask, the original operating status data and the data analysis subtask are sent to the data analysis agent so that the data analysis agent processes the original operating status data to obtain operating status data that meets the preset quality conditions; when the data analysis agent completes the data analysis subtask, the original operating status data and the data analysis subtask are sent to the data analysis agent so that the data analysis agent processes the original operating status data to obtain operating status data that meets the preset quality conditions; task, sending the operating status data and the fault diagnosis subtask to the fault diagnosis agent, so that the fault diagnosis agent performs fault diagnosis on the operating status data; when the fault diagnosis agent completes the fault diagnosis subtask, sending the fault diagnosis result and the result generation subtask to the result generation agent, so that the result generation agent generates a fault prediction result according to the fault diagnosis result and prior knowledge; wherein the data collection agent, the data analysis agent, the fault diagnosis agent, and the result generation agent are agents based on language models, which are configured with corresponding functions to enable them to complete tasks autonomously.

[0010] In a first exemplary embodiment, the fault diagnosis agent includes at least a first fault diagnosis agent, a second fault diagnosis agent and a hybrid fault diagnosis agent, and the data collection agent includes at least a first data collection agent and a second data collection agent. The operating status data and the fault diagnosis subtask are sent to the fault diagnosis agent so that the fault diagnosis agent performs fault diagnosis on the operating status data, including: sending the first operating status data collected by the first data collection agent to the first fault diagnosis agent, sending the second operating status data collected by the second data collection agent to the second fault diagnosis agent, and sending the first operating status data and the second operating status data to the hybrid fault diagnosis agent so that the first fault diagnosis agent, the second fault diagnosis agent and the hybrid fault diagnosis agent simultaneously perform fault diagnosis on the operating status data.

[0011] In a second exemplary embodiment, there are multiple fault diagnosis agents, and sending the fault diagnosis results and result generation subtasks to the result generation agent includes: obtaining historical task execution data of each fault diagnosis agent; the historical task execution data includes at least the number of historical tasks, each historical task execution parameter, each historical task behavior parameter and interaction history parameter; determining the decision weight of the corresponding fault diagnosis agent to perform the current task based on the historical task execution data of each fault diagnosis agent; and determining the fault diagnosis result to be input into the result generation agent based on the fault diagnosis results of each fault diagnosis agent and the decision weight corresponding to the fault diagnosis result.

[0012] In a third exemplary embodiment, the decision weight of the corresponding fault diagnosis agent to execute the current task is determined based on the historical task execution data of each fault diagnosis agent, including: determining the task execution score of the current historical task based on the current historical task success rate, the current historical task execution speed, and the resource consumption efficiency of the current historical task, and determining the performance factor of each fault diagnosis agent based on the task execution score and task difficulty score corresponding to each historical task of each fault diagnosis agent; determining the trust factor of each fault diagnosis agent based on the interaction success rate, protocol compliance rate, and communication success rate of each fault diagnosis agent in the process of completing each historical task; determining the experience value of each fault diagnosis agent based on the total amount of historical task processing of each fault diagnosis agent, the total number of target historical tasks that meet the preset complexity conditions, and the total running time; determining the decision weight for executing the current task based on the performance factor, trust factor and experience value of each fault diagnosis agent and their respective weight coefficients.

[0013] In a fourth exemplary embodiment, the fault diagnosis agent runs multiple tasks simultaneously, and the fault diagnosis agent performs fault diagnosis on the running status data, including: when insufficient computing resources and / or memory resources are detected, obtaining the workload information and task running information of the fault diagnosis agent; reading the task type and task impact of each task from the task running information, and determining the urgency score of each task according to the task type and task impact of each task; determining the workload in the process of executing each task according to the workload information, and determining the load score according to the workload quantity and workload resource demand; determining the priority of each task according to the urgency score, load score and corresponding weight coefficient of each task; and performing fault diagnosis on the running status data based on the priority of the fault diagnosis subtask.

[0014] In a fifth exemplary embodiment, there are multiple fault diagnosis agents, the result generating agent includes an expert agent and an interactive agent, the result generating subtask includes a diagnosis result generating subtask and a report generating subtask, and the fault diagnosis result and the result generating subtask are sent to the result generating agent so that the result generating agent generates a fault prediction result based on the fault diagnosis result and prior knowledge, including: sending the fault diagnosis result of each fault diagnosis agent and the diagnosis result generating subtask to the expert agent so that the expert agent generates a total fault diagnosis result based on the fault diagnosis results of each fault diagnosis agent and the expert knowledge base; when the diagnosis result generating subtask is completed, sending the report generating subtask and the total fault diagnosis result to the interactive agent so that the interactive agent generates a fault prediction result based on the total fault diagnosis result and artificial interaction information using a learning retrieval enhanced generation method.

[0015] In the sixth exemplary embodiment, it also includes: obtaining the workload and resource consumption information of the data collection agent, the data analysis agent, the fault diagnosis agent and the result generation agent; determining the resource allocation weights for the data collection agent, the data analysis agent, the fault diagnosis agent and the result generation agent at the current moment according to the workload and resource consumption information of the data collection agent, the data analysis agent, the fault diagnosis agent and the result generation agent; and adjusting the resource allocation of the data collection agent, the data analysis agent, the fault diagnosis agent and the result generation agent according to the current resource allocation weights.

[0016] In the seventh exemplary embodiment, it also includes: sending the original operating status data to the first target end through the first learning interface, so that the first target end checks the original operating status data; when feedback information that the original operating status data is erroneous is received, receiving the correct original operating status data fed back by the first target end through the first learning interface, and sending the first learning task and the correct original operating status data to the data collection agent, and sending the correct original operating status data as the original operating status data to the data collection agent; sending the operating status data to the second target end through the second learning interface, so that the second target end checks the operating status data; when feedback information that the operating status data is erroneous is received, receiving the correct operating status data fed back by the second target end through the second learning interface, and sending the second learning task and the correct operating status data to the data collection agent, and sending the correct operating status data to the data collection agent. The data is sent to the fault diagnosis agent as running status data; the fault prediction result is sent to the third target end through the third learning interface, so that the third target end reviews the fault prediction result; when feedback information that the fault prediction result is wrong is received, the correct fault prediction result fed back by the third target end is received through the third learning interface, and the third learning task and the correct fault prediction result are sent to the result generation agent, and the correct fault prediction result is output as the fault prediction result; the fault diagnosis result is sent to the fourth target end through the fourth learning interface, so that the fourth target end corrects the fault diagnosis result based on the chaos engineering self-evolution mechanism; when feedback information that the fault diagnosis result is wrong is received, the correct fault diagnosis result fed back by the fourth target end is received through the fourth learning interface, and the fourth learning task and the correct fault diagnosis result are sent to the fault diagnosis agent, and the correct fault diagnosis result is sent to the result generation agent.

[0017] The present invention also provides an electronic device, comprising a processor, wherein the processor is used to implement the steps of any of the above-mentioned fault prediction methods when executing a computer program stored in a memory.

[0018] The present invention also provides a non-volatile storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of any of the above-mentioned fault prediction methods are implemented.

[0019] The present invention also provides a computer program product, comprising a computer program / instruction, which implements the steps of any of the above-mentioned fault prediction methods when executed by a processor.

[0020] Finally, the present invention also provides a data center, including a fault prediction system; the fault prediction system includes a manager agent, a data collection agent, a data analysis agent, a fault diagnosis agent, an expert agent and an interactive agent; wherein the manager agent respectively communicates data with the data collection agent, the data analysis agent, the fault diagnosis agent, the expert agent and the interactive agent, and executes the steps of the fault prediction method as described above; the data collection agent is used to collect the original operating status data of each server in the data center after receiving the data collection subtask; the data analysis agent is used to process the original operating status data after receiving the data analysis subtask to obtain operating status data that meets the preset quality conditions; the fault diagnosis agent is used to perform fault diagnosis on the operating status data after receiving the fault diagnosis subtask; the expert agent is used to generate a total fault diagnosis result by integrating the fault diagnosis results of each fault diagnosis agent after receiving the diagnosis result generation subtask; the interactive agent is used to generate a fault prediction result based on the total fault diagnosis result, prior knowledge and interactive information after receiving the report generation subtask.

[0021] In a first exemplary embodiment, the data collection agent includes an interface calling function, a monitoring tool calling function and a tracking data function; wherein the interface calling function is used to obtain the server's operation log data by calling a remote control interface; the monitoring tool calling function is used to call a monitoring tool interface to obtain server performance indicator data; and the tracking data function is used to obtain the call chain data corresponding to the target identifier implanted by the tracking system.

[0022] In a second exemplary embodiment, the data analysis agent includes a data processing function, a log parsing tool calling function and a data cleaning tool calling function; wherein the data analysis agent filters the operation log data whose length is less than a preset length threshold by calling the data processing function; performs log parsing on the remaining operation log data by calling the log parsing tool function, and removes invalid data and redundant data in the operation log data; the data analysis agent also deletes incomplete call chain data by calling the data processing function; the data analysis agent also deletes redundant data in the server performance indicator data by calling the data processing function, and completes the server performance indicator data by calling the data cleaning tool function.

[0023] In a third exemplary embodiment, the fault prediction system also includes a knowledge database; the knowledge database includes a structured knowledge sub-base and an unstructured sub-base; the historical knowledge that has been learned by the manager agent and / or the data collection agent and / or the data analysis agent and / or the fault diagnosis agent and / or the expert agent and / or the interactive agent is obtained, and target historical knowledge that has not been accessed within a preset time period is selected from the historical knowledge; for a first type of target historical knowledge belonging to a structured data type, the first type of target historical knowledge is converted into vector data and stored in the structured knowledge sub-base; for a second type of target historical knowledge belonging to an unstructured data type, the second type of target historical knowledge is converted into text data and stored in the unstructured knowledge sub-base.

[0024] The advantage of the technical solution provided by the present invention is that the entire fault prediction process of the data center is decomposed into subtasks, simplifying and optimizing the difficulty of fault prediction of complex computer systems. Each agent is only responsible for a subdivided subtask, which not only reduces the amount of tasks that a single agent needs to undertake, but also ensures the accuracy and efficiency of task execution, and achieves efficient and high-precision fault prediction. With the advanced large language model as the core engine, it drives multiple agents to work together, accurately respond to complex fault situations that may occur during server operation, achieve high-precision prediction of complex fault tasks, effectively ensure accurate prediction of data center faults, and ensure safe, stable and reliable operation of the data center.

[0025] In addition, the present invention also provides corresponding electronic devices, non-volatile storage media, computer program products and data centers for implementing the fault prediction method, further making the method more practical. The electronic devices, non-volatile storage media, computer program products and data centers have corresponding advantages.

[0026] The technical features mentioned above, the technical features to be mentioned below, and the technical features shown separately in the drawings can be arbitrarily combined with each other, as long as the combined technical features are not contradictory. All feasible feature combinations are technical contents clearly recorded in this article. Any of the multiple sub-features contained in the same sentence can be applied independently, and does not have to be applied together with other sub-features. It should be understood that the above general description and the detailed description below are only exemplary and cannot limit the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] In order to more clearly illustrate the technical solutions of the present invention or related technologies, the following briefly introduces the drawings required for use in the embodiments or related technical descriptions. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0028] Figure 1 A schematic diagram of a flow chart of a fault prediction method provided by the present invention;

[0029] Figure 2 A schematic flow chart of another fault prediction method provided by the present invention;

[0030] Figure 3 A schematic diagram of an exemplary architecture of a fault prediction system for a data center provided by the present invention;

[0031] Figure 4 Another exemplary architecture diagram of the fault prediction system for a data center provided by the present invention;

[0032] Figure 5 A structural diagram of a specific implementation of the fault prediction device provided by the present invention;

[0033] Figure 6 A structural diagram of a specific implementation of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0034] In order to enable those skilled in the art to better understand the technical solution of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. Among them, the terms "first", "second", "third", "fourth", etc. in the specification and the above-mentioned drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations of the two are intended to cover non-exclusive inclusions. The term "exemplary" means "used as an example, embodiment or illustrative". Any embodiment described here as "exemplary" is not necessarily interpreted as being superior or better than other embodiments.

[0035] As the core infrastructure of computer information systems, data centers are not only key hubs for storing and processing massive amounts of data, but also support a variety of key business applications and services. With the rapid development of artificial intelligence technology, the Internet of Things, and cloud technology, the scale of data centers is getting larger and larger, and the requirements for availability and reliability are getting higher and higher. In order to prevent computer information system crashes and data loss, ensure business continuity and stability, and ensure the normal operation of data centers, fault diagnosis and fault prediction will be performed on each server in the data center. Through timely fault diagnosis, abnormal equipment can be discovered in advance, potential problems in the data center can be quickly solved, and failures in the data center can be effectively prevented. For example, by monitoring and analyzing parameters such as temperature, voltage, and vibration of the server, possible hardware failures can be predicted, and maintenance and replacement can be performed before the failure occurs, thereby avoiding unexpected server downtime and business data loss. In addition, by predicting server failures in advance and reducing the frequency of emergency repairs and hardware replacement, maintenance costs can be reduced, resource allocation can be optimized, and energy utilization efficiency can be improved, thereby improving overall operational efficiency. For example, the predictive maintenance system can automatically adjust the workload according to the health of the server to avoid global performance degradation caused by single point failures. By monitoring the health of servers and other infrastructure equipment in real time, energy use can be optimized and unnecessary energy consumption can be reduced.

[0036] In order to ensure the normal operation of the data center, traditional methods usually perform fault diagnosis or fault prediction based on statistical analysis and expert rules, such as threshold-based fault diagnosis methods, statistical model-based fault diagnosis methods, and expert rule-based fault diagnosis methods. Among them, the threshold-based fault diagnosis method predefines a threshold for each performance indicator. When there is an indicator exceeding the threshold during the operation of the server, it is considered that the server has a fault.

[0037] This method only judges faults based on preset thresholds, has limited information utilization capabilities, lacks a comprehensive and in-depth analysis of the system's operating status, and has a high probability of misjudging or missing faults. In addition, reasonable threshold setting requires rich experience and expertise, and as the system's operating environment changes, the threshold needs to be adjusted continuously, which not only cannot guarantee the fault diagnosis effect, but also has a high maintenance difficulty. The fault diagnosis method based on statistical models uses statistical models to analyze the distribution of server operating data and identify abnormal data points. This method relies on assumptions about data distribution, but the data distribution in actual applications is complex and changeable, and it is difficult to meet these assumptions, and the model is very likely to fail. In addition, before building the model, it is necessary to have an in-depth understanding of the system's operating mechanism and data characteristics, which makes the construction of the statistical model complex and time-consuming. Furthermore, when processing high-dimensional data, the performance of this method will significantly decrease, or even fail to work properly, limiting its application in complex system fault diagnosis. The fault diagnosis method based on expert rules converts the expert's knowledge and experience into rules and uses the rules to diagnose server faults. Although this method can effectively utilize the deep knowledge and experience of domain experts to quickly locate problems, it is difficult to fully capture and express complex, implicit, and context-dependent knowledge, resulting in reduced fault diagnosis accuracy. In addition, the rule base has high maintenance costs and needs to be continuously updated to adapt to technological development and environmental changes, and is prone to errors. The conflicts and redundancies between rules are complex and difficult to resolve, affecting the consistency of diagnostic results and resulting in reduced diagnostic accuracy. In addition, the application of fixed rule sets makes this method unable to quickly adapt to new situations and unknown faults, and is prone to misleading. Fault diagnosis of complex systems requires processing of complex rule combinations and logical judgments, which is inefficient, and the system complexity is proportional to the computational cost.

[0038] In order to solve the problems of traditional fault diagnosis methods, related technologies predict and diagnose server faults based on artificial intelligence algorithms. Fault prediction methods based on artificial intelligence algorithms include methods based on traditional machine learning and methods based on deep learning. Among them, methods based on traditional machine learning, such as using XGboost (Extreme Gradient Boost) and LightGBM (Light Gradient Boosting Machine), judge and predict faults. Although such methods can effectively use the advantages of data-driven to improve prediction accuracy, they have high requirements on the quality and quantity of training data. In the case of missing or incomplete data, the model performance will be significantly reduced, resulting in inaccurate final fault prediction results. When facing new fault modes or changing system environments, such models need to be retrained or adjusted to ensure model performance, and their flexibility and adaptability are poor. In addition, in order to avoid unstable model performance and calculation delays, in actual industrial applications, the real-time and stability of the model also need special attention, which increases maintenance costs and complexity. Methods based on deep learning, such as using convolutional neural networks, long short-term memory networks, Transformers and other deep learning methods, judge and predict faults. Although such methods can improve the accuracy and automation of fault prediction, they are also highly dependent on training data. When the amount of data is small, the model performance may drop significantly. Secondly, the hyperparameter tuning process of deep learning models is complex and requires a lot of experiments and expertise, which increases the difficulty of model development and maintenance. Finally, no matter which of the above artificial intelligence algorithms is used, it is only for a single task type, that is, each task requires a separate training neural network model to predict this type of fault. As the scale of data centers increases and the complexity of computer systems increases, fault tasks are often complex, so related technologies cannot achieve accurate prediction of data center failures.

[0039] In view of this, in order to solve the problems existing in the related technologies, the present invention decomposes the entire fault prediction process of the data center into subtasks, simplifies and optimizes the difficulty of fault prediction of complex computer systems, uses advanced large language models as the core engine, drives multiple agents to work together, and each agent is only responsible for a subdivided subtask, accurately responds to complex fault situations that may occur during server operation, and achieves high-precision prediction of complex fault tasks to ensure the safe, stable and reliable operation of the data center. After introducing the technical solution of the present invention, various non-limiting embodiments of the present invention are described in detail below. In order to better illustrate the present invention, numerous specific details are given in the specific embodiments below. Those skilled in the art should understand that the present invention can also be implemented without these specific details. In other examples, methods, means, components and circuits well known to technicians in the relevant field are not described in detail in order to highlight the main purpose of the present invention.

[0040] First see Figure 1 , Figure 1A flow chart of a fault prediction method provided in this embodiment is provided. Based on any existing large language model, the agent is equipped with planning, memory, tool use and other functions, and the agent can complete the task autonomously with the help of language processing ability, thereby obtaining a manager agent, a data collection agent, a data analysis agent, a fault diagnosis agent, and a result generation agent. The manager agent, the data collection agent, the data analysis agent, the fault diagnosis agent, and the result generation agent identify the tasks to be performed and the execution method through the capabilities of task planning, memory and external tools, so as to achieve autonomous decision-making and complete the corresponding tasks. Among them, planning refers to the process in which the agent decomposes a complex task into a series of subtasks and formulates a corresponding action plan when performing a task, which not only helps the agent understand the task goal, but also can adjust and optimize according to feedback during the execution process, so as to efficiently complete the task. The role of planning in the agent includes: dividing a complex task into multiple subtasks and solving them step by step, which is similar to the way humans think about solving problems. Self-criticism and reflection based on existing actions, learning from mistakes to optimize the next action. Adjustment and optimization are made according to the feedback of the external environment during the execution process to ensure the efficient completion of the task. Memory includes short-term memory and long-term memory. Short-term memory refers to contextual information during the execution of a task, including the return value of the tool and the completed reasoning path. Short-term memory is similar to the working memory in the human brain, which can store information currently being processed and help perform complex cognitive tasks. Short-term memory is usually limited by the length of the context window, has a limited impact range, and has a short duration. Long-term memory can store information for a considerable period of time, ranging from a few days to decades, and has a basically unlimited storage capacity. Long-term memory includes episodic memory and semantic memory, which is the memory of facts and events, which can be consciously recalled; it also includes unconscious skills and routines that are automatically executed. In other words, the manager agent, data collection agent, data analysis agent, fault diagnosis agent, and result generation agent of the present invention are all based on language models, and are configured with corresponding functions for each of them to enable them to complete tasks autonomously. As for the functions configured for the manager agent, data collection agent, data analysis agent, fault diagnosis agent, and result generation agent, in addition to the functions recorded in the following embodiments, those skilled in the art can flexibly configure them according to the actual application scenario, which does not affect the implementation of the present invention. After obtaining the manager agent, data collection agent, data analysis agent, fault diagnosis agent, and result generation agent, this embodiment uses the manager agent as the execution subject to complete the fault prediction method. This embodiment may include the following contents:

[0041] S101: When a fault prediction task is detected, the fault prediction task is divided into a data collection subtask, a data analysis subtask, a fault diagnosis subtask and a result generation subtask.

[0042] In this embodiment, the fault prediction task can be triggered by a fault prediction request or by a script according to a preset rule. As long as the data center is detected to have a fault prediction task, this step is executed. In this embodiment, the fault prediction task is divided into four subtasks: data collection subtask, data analysis subtask, fault diagnosis subtask and result generation subtask. Different subtasks are performed by different agents. The data collection subtask is used to trigger the data collection agent to perform the data collection task, the data analysis subtask is used to trigger the data analysis agent to perform the data analysis task, the fault diagnosis subtask is used to trigger the fault diagnosis agent to perform the fault diagnosis task, and the result generation subtask is used to trigger the result generation agent to perform the task of generating prediction results.

[0043] S102: Send the data collection subtask to the data collection agent, so that the data collection agent collects the original operating status data of each server in the data center.

[0044] The data collection agent in this step is equipped with planning, memory and tool use functions to enable it to complete the data collection task. Among them, equipping it with planning refers to how and which tasks it should collect when performing data collection tasks. For example, the data collection task can be divided into multiple subtasks according to the device to be collected, and each subtask is responsible for collecting the operating data of a device. The tools configured by the data collection agent include but are not limited to external application program interfaces, calculators, search tools, code executors, database query tools, and knowledge bases. External data can be obtained through external application program interfaces. Calculators, search tools, code executors, and database query tools can help data collection agents perform operations such as calculations, searches, code execution, and database queries to obtain the required information and data. Information can be quickly retrieved and stored through the knowledge base, realizing the ability to retain and recall information for a long time. The original operating status data includes but is not limited to various performance indicator data such as CPU utilization, memory usage, disk input / output performance, as well as the server's operating log data and data tracked by the tracking system used for calling.

[0045] S103: When the data collection agent completes the data collection subtask, it sends the original operating status data and the data analysis subtask to the data analysis agent, so that the data analysis agent processes the original operating status data to obtain operating status data that meets the preset quality conditions.

[0046] The data analysis agent in this step is equipped with planning, memory and tool use functions to enable it to complete the data analysis task. The planning of the data analysis agent refers to how it should handle the task and how to analyze the data when performing the data analysis task. For example, the data analysis task can be divided according to the type of data to be analyzed, and the same type of data can be divided into a subtask, each subtask is responsible for processing a type of data. The tools configured by the data analysis agent include but are not limited to external application program interfaces, calculators, search tools, code executors, database query tools, and knowledge bases. External data can be obtained through external application program interfaces. Calculators, search tools, code executors, and database query tools can help data analysis agents perform operations such as calculations, searches, code execution, and database queries to obtain the required information and data. Information can be quickly retrieved and stored through the knowledge base, realizing the ability to retain and recall information for a long time. The preset quality condition is pre-set to measure the degree of processing of the original running status data by the data analysis agent, that is, the quality of the running status data processed by the data analysis agent, such as whether it contains redundant data, whether it is complete, whether it is missing, and the minimum conditions to be achieved. Those skilled in the art may make a flexible determination based on actual conditions, and the present invention does not impose any limitation on this.

[0047] S104: When the data analysis agent completes the data analysis subtask, it sends the operating status data and the fault diagnosis subtask to the fault diagnosis agent, so that the fault diagnosis agent performs fault diagnosis on the operating status data.

[0048] The fault diagnosis agent in this step is equipped with planning, memory and tool use functions to enable it to complete the fault diagnosis task. The fault diagnosis agent is equipped with planning, which means how it should perform fault diagnosis when performing the fault diagnosis task, and which data should be processed in which way. For example, the fault diagnosis task can be decomposed according to the type of data that needs to be diagnosed, and the same type of data can be divided into a subtask. Each subtask is responsible for processing a type of data. For each subtask, which model or tool is used for analysis, such as image data, can be analyzed using an image processing model, such as text data, can be analyzed using a text data model. The tools configured by the fault diagnosis agent include but are not limited to external application program interfaces, calculators, search tools, code executors, database query tools, and knowledge bases. External data can be obtained through external application program interfaces. Calculators, search tools, code executors, and database query tools can help the fault diagnosis agent perform operations such as calculation, search, code execution, and database query, so as to obtain the required information and data.

[0049] S105: When the fault diagnosis agent completes the fault diagnosis subtask, it sends the fault diagnosis result and the result generation subtask to the result generation agent, so that the result generation agent generates a fault prediction result according to the fault diagnosis result and prior knowledge.

[0050] The result generation agent in this step is equipped with planning, memory and tool use functions to enable it to generate the fault prediction result. The result generation agent is equipped with planning, which means how it should generate the prediction result and what prior knowledge it should obtain when performing the result generation task. The tools configured by the result generation agent include but are not limited to external application program interfaces, calculators, search tools, code executors, database query tools, and knowledge bases. External data can be obtained through external application program interfaces. Calculators, search tools, code executors, and database query tools can help data collection agents perform calculations, searches, code execution, and database queries to obtain the required information and data. The knowledge base can be used to quickly retrieve and store information, and the ability to retain and recall information for a long time can be achieved. Prior knowledge refers to the external data required to generate fault prediction results, including but not limited to expert experience, fault support records, product documents, and web crawler results.

[0051] In the technical solution provided in this embodiment, the entire fault prediction process of the data center is decomposed into subtasks, simplifying and optimizing the difficulty of fault prediction of complex computer systems. Each agent is only responsible for a subdivided subtask, which not only reduces the amount of tasks that a single agent needs to undertake, but also ensures the accuracy and efficiency of task execution, and achieves efficient and high-precision fault prediction. With the advanced large language model as the core engine, it drives multiple agents to work together, accurately respond to complex fault situations that may occur during server operation, achieve high-precision prediction of complex fault tasks, effectively ensure accurate prediction of data center faults, and ensure safe, stable and reliable operation of the data center.

[0052] In order to further improve the efficiency of fault prediction, based on the above embodiment, the present invention further provides multiple fault diagnosis agents to perform fault diagnosis subtasks in parallel, which may include the following contents:

[0053] like Figure 2As shown, the fault diagnosis agent includes at least a first fault diagnosis agent, a second fault diagnosis agent and a hybrid fault diagnosis agent, and the data collection agent includes at least a first data collection agent and a second data collection agent. The first operating status data collected by the first data collection agent is sent to the first fault diagnosis agent, the second operating status data collected by the second data collection agent is sent to the second fault diagnosis agent, and the first operating status data and the second operating status data are sent to the hybrid fault diagnosis agent, so that the first fault diagnosis agent, the second fault diagnosis agent and the hybrid fault diagnosis agent can perform fault diagnosis on the operating status data at the same time.

[0054] In this embodiment, the first fault diagnosis agent, the second fault diagnosis agent, and the hybrid fault diagnosis agent have programming and communication capabilities through configuration planning, and use programming capabilities to call the required relevant machine learning models, and use communication capabilities to state the reasons for the diagnosis or prediction conclusions, so that the final result is interpretable, thereby solving the problem that the methods based on artificial intelligence in related technologies are all "black box" modes that lead to lack of interpretability. The first fault diagnosis agent, the second fault diagnosis agent, and the hybrid fault diagnosis agent can call various algorithms to analyze and predict the collected and cleaned operating status data to determine whether there is a fault and the type of fault, so as to identify whether the data center and each server are in normal operating mode or abnormal mode. The difference is that the first fault diagnosis agent and the second fault diagnosis agent analyze, diagnose and predict a single operating status data, and the hybrid fault diagnosis agent fuses all operating status data and analyzes, diagnoses and predicts the fused operating status data. If the running status data is running log data, models such as LightGBM and long short-term memory network can be called from the code base to judge and predict the running status data for faults; if the running status data is call chain data, graph neural network is called from the code base to model the call chain data and give fault diagnosis results. If the running status data is performance indicator data, isolated forest, standard deviation and other methods are called from the code base to judge outliers, and Prophet (time series prediction model) is used to diagnose and predict faults. The first fault diagnosis agent, the second fault diagnosis agent, and the hybrid fault diagnosis agent can not only instantly call advanced algorithms to perform deep learning and pattern recognition on massive data through efficient data processing engines to accurately determine whether a data center has failed, but also further identify the fault type, severity and potential impact range. Fault types such as CPU overload, memory leak or disk failure can be pre-divided into multiple levels of severity and configured with corresponding standards. When the first fault diagnosis agent, the second fault diagnosis agent, and the hybrid fault diagnosis agent detect abnormal patterns, they can generate alarm prompt information carrying the fault type.

[0055] On the basis of fault diagnosis, the first fault diagnosis agent, the second fault diagnosis agent, and the hybrid fault diagnosis agent can also have built-in predictive analysis algorithms, combining historical fault trends with the current operating status of the data center to predict possible future faults. Exemplarily, the first fault diagnosis agent, the second fault diagnosis agent, and the hybrid fault diagnosis agent use time series analysis of historical data and current operating status data to predict whether there is performance degradation in the data center, so as to take preventive measures in advance to avoid the occurrence of faults. Perform root cause analysis on the operating status data, find the root cause of the fault through correlation analysis, and propose targeted solutions to ensure the high availability and stability of the data center and improve the task execution efficiency of the data center.

[0056] Furthermore, the first fault diagnosis agent, the second fault diagnosis agent, and the hybrid fault diagnosis agent are configured with student-teacher planning to enable self-optimization capabilities, and through continuous learning, the accuracy of fault diagnosis and prediction efficiency are improved, the prediction accuracy and response speed are improved, and the stability, reliability and high availability of data center operations are ensured.

[0057] Based on the above embodiment, when there are multiple fault diagnosis agents performing fault diagnosis simultaneously, in order to further improve the accuracy of the final fault prediction result, based on the above embodiment, the present invention also sets the importance of each fault diagnosis agent in the fault diagnosis result decision process, which may include the following contents:

[0058] Obtain the historical task execution data of each fault diagnosis agent; determine the decision weight of the corresponding fault diagnosis agent to execute the current task based on the historical task execution data of each fault diagnosis agent; determine the fault diagnosis result of the input result generation agent based on the fault diagnosis result of each fault diagnosis agent and its corresponding decision weight.

[0059] Among them, the historical task execution data may at least include the number of historical tasks, the execution parameters of each historical task, the behavior parameters of each historical task and the interaction history parameters; the historical task execution parameters may at least include the task success rate, the task execution speed, and the task resource consumption efficiency; the historical task behavior parameters may include the number of rule violations, the number of non-compliance with the protocol, and the number of non-execution of tasks; the interaction history parameters may include the interaction of each fault diagnosis intelligent agent in the process of completing each historical task. Exemplarily, the decision weight can be jointly determined by performance factors, trust factors and experience factors. The task execution score of the current historical task is determined based on the success rate of the current historical task, the execution speed of the current historical task, and the resource consumption efficiency of the current historical task. The performance factor of each fault diagnosis agent is determined based on the task execution score and task difficulty score corresponding to each historical task of each fault diagnosis agent; the trust factor of each fault diagnosis agent is determined based on the interaction success rate, protocol compliance rate and communication success rate of each fault diagnosis agent in the process of completing each historical task; the experience value of each fault diagnosis agent is determined based on the total amount of historical tasks processed by each fault diagnosis agent, the total number of target historical tasks that meet the preset complexity conditions, and the total running time; the decision weight for executing the current task is determined based on the performance factor, trust factor and experience value of each fault diagnosis agent and their respective weight coefficients.

[0060] As an efficient way to determine decision weights, the present invention can store the collaborative decision weight relationship formula locally in advance, and calculate the decision weight of each fault diagnosis agent by calling the collaborative decision weight relationship formula. The collaborative decision weight relationship formula can be expressed as:

[0061] ;

[0062] in, is the decision weight of the i-th fault diagnosis agent in collaborative decision making, α, β, γ are weight coefficients assigned according to their respective importance, is the task execution score of the jth task, is the difficulty weight of the jth task, m is the number of historical tasks used in the performance factor calculation process, is the number of successful interactions in the kth time, is the total number of k-th interactions, p is the total number of interactions, q is the total number of target historical tasks, is the complexity score of the lth task, is the total running time of the i-th fault diagnosis agent.

[0063] In this embodiment, the decision weight of the i-th fault diagnosis agent in the collaborative decision-making can be obtained by Sure.

[0064] is the performance factor of the i-th fault diagnosis agent, is the trust factor of the i-th fault diagnosis agent, Experience i is the experience value of the i-th fault diagnosis agent. The performance factor can be evaluated by the performance of the fault diagnosis agent in historical tasks. For example, it can be determined based on factors such as task success rate, task completion speed, resource consumption efficiency, etc. The performance factor can be determined by the following relationship:

[0065] .

[0066] The trust factor can be evaluated based on the historical behavior and interaction history of the fault diagnosis agent, including the cooperation record with other agents, the degree of compliance with the protocol, and the accuracy of historical communication. The trust factor of the fault diagnosis agent can be calculated by the following relationship:

[0067] .

[0068] The experience value can be determined based on the number of tasks in which the fault diagnosis agent participates, the types of complex problems handled, and the running time in the data center. The experience value can be calculated by the following relationship:

[0069] .

[0070] Since each agent is based on a large-scale language model, each agent performs tasks at the same time, which will result in running the language model in parallel at the same time. There may be insufficient computing and memory resources, resulting in the failure of successful task execution, which may include the following:

[0071] For the scenario where the fault diagnosis agent runs multiple tasks simultaneously, when insufficient computing resources and / or memory resources are detected, the workload information and task running information of the fault diagnosis agent are obtained; the task type and task impact of each task are read from the task running information, and the urgency score of each task is determined according to the task type and task impact of each task; the workload in the process of executing each task is determined according to the workload information, and the load score is determined according to the number of workloads and the workload resource demand; the priority of each task is determined according to the urgency score, load score and corresponding weight coefficient of each task; based on the priority of the fault diagnosis subtask, fault diagnosis is performed on the running status data.

[0072] In this embodiment, whenever there is a shortage of computing resources and memory resources, the tasks run by each agent need to calculate their respective priorities according to this embodiment, and execute each task in order of their respective priorities. As an efficient priority calculation method, the priority calculation relationship can be pre-stored locally, and the priority of each task can be calculated by calling the priority. The priority calculation relationship can be expressed as:

[0073] ;

[0074] in, is the priority of the ith task, For the current workload, is the urgency of the i-th task, α, β are weight coefficients.

[0075] In order to further improve the accuracy of prediction results and solve the problem that the prediction results lack interpretability due to the black box mode of the artificial intelligence-based methods in the related art, the present invention also provides the following implementation methods:

[0076] There are multiple fault diagnosis agents, and the result generating agents include expert agents and interactive agents. The result generating subtasks include diagnosis result generating subtasks and report generating subtasks. The fault diagnosis results and diagnosis result generating subtasks of each fault diagnosis agent are sent to the expert agent, so that the expert agent generates a total fault diagnosis result based on the fault diagnosis results of each fault diagnosis agent and the expert knowledge base. When the diagnosis result generating subtask is completed, the report generating subtask and the total fault diagnosis result are sent to the interactive agent, so that the interactive agent generates a fault prediction result based on the total fault diagnosis result and the manual interaction information by using the learning retrieval enhanced generation method.

[0077] In this embodiment, taking into account the self-consistency of each fault diagnosis agent, that is, independently performing fault diagnosis and fault prediction, which is prone to conflicts, this embodiment uses an expert agent to communicate and collaborate effectively with each fault diagnosis agent, integrates the analysis results of each fault diagnosis agent, and combines relevant knowledge to form an accurate and comprehensive overall fault diagnosis result.

[0078] The interactive intelligent body will generate fault prediction results based on the total fault diagnosis results of the expert intelligent body, and interact with users such as operation and maintenance personnel to answer various questions of the operation and maintenance personnel. Through continuous interaction with the operation and maintenance personnel, the operation and maintenance personnel will be provided with the required information and operation and maintenance knowledge. In order to effectively alleviate the hallucination problem of the language model and generate reference solutions and fault causes with fault repair suggestions, the interactive intelligent body of this embodiment uses RAG (Relevance Analysis and Generation) technology to retrieve and analyze multi-source information such as expert experience, fault support records, product documents, and web crawler results. Through this process, the interactive intelligent body generates a detailed fault prediction result, and can also point out the specific cause of the fault and the scope of influence. At the same time, the interactive intelligent body can also give reasonable repair suggestions based on the fault prediction results and the actual situation. For example, for the identified hardware fault, the interactive intelligent body will provide the replacement steps of the relevant components; for software problems, it will provide corresponding repair or configuration adjustment guidelines. The interactive intelligent body presents these suggestions to the operation and maintenance personnel in an easy-to-understand form through natural language processing technology to ensure that they can efficiently perform repair tasks, thereby helping the operation and maintenance personnel to solve problems quickly and efficiently.

[0079] It can be seen from the above that the intelligent interaction method of this embodiment not only simplifies the complex diagnosis and repair process, but also improves the efficiency and quality of operation and maintenance work.

[0080] Since each agent is based on a large-scale language model and each agent performs tasks at the same time, this may result in insufficient computing and memory resources when the language model is run in parallel at the same time. In order to avoid errors or errors caused by insufficient resources, based on the above embodiments, the present invention also provides an implementation method for dynamic resource allocation, which may include the following contents:

[0081] Obtain the workload and resource consumption information of the data collection agent, data analysis agent, fault diagnosis agent and result generation agent; determine the resource allocation weights for the data collection agent, data analysis agent, fault diagnosis agent and result generation agent at the current moment according to the workload and resource consumption information of the data collection agent, data analysis agent, fault diagnosis agent and result generation agent; adjust the resource allocation of the data collection agent, data analysis agent, fault diagnosis agent and result generation agent according to the current resource allocation weights.

[0082] Since it involves private enterprise data, the base model uses an open source model and is deployed on a private machine. Compared with a closed-source commercial model, the open source model has fewer parameters, so its performance is not as good as that of the commercial model. In addition, whether it is an open source model or a closed-source commercial model, it is trained based on a general field and is relatively unfamiliar to the vertical data center operation and maintenance field, which may cause hallucinations. That is, the language model used by each intelligent agent may generate seemingly reasonable but actually inaccurate or completely wrong information for unfamiliar content, resulting in a reduction in the accuracy of the final fault prediction results. On the basis of the above-mentioned embodiment using RAG to obtain external knowledge to improve the accuracy of fault prediction results, the present invention also provides an implementation method for each intelligent agent to improve performance through continuous learning, which may include the following contents:

[0083] The original operating status data is sent to the first target end through the first learning interface so that the first target end checks the original operating status data; when feedback information that there are errors in the original operating status data is received, the correct original operating status data fed back by the first target end is received through the first learning interface, and the first learning task and the correct original operating status data are sent to the data collection agent, and the correct original operating status data is sent to the data collection agent as the original operating status data. The operating status data is sent to the second target end through the second learning interface so that the second target end checks the operating status data; when feedback information that there is an error in the operating status data is received, the correct operating status data fed back by the second target end is received through the second learning interface, and the second learning task and the correct operating status data are sent to the data collection agent, and the correct operating status data is sent as the operating status data to the fault diagnosis agent; the fault prediction result is sent to the third target end through the third learning interface so that the third target end reviews the fault prediction result; when feedback information that there is an error in the fault prediction result is received, the correct fault prediction result fed back by the third target end is received through the third learning interface, and the third learning task and the correct fault prediction result are sent to the result generation agent, and the correct fault prediction result is output as the fault prediction result. The fault diagnosis result is sent to the fourth target end through the fourth learning interface, so that the fourth target end corrects the fault diagnosis result based on the chaos engineering self-evolution mechanism; when feedback information that the fault diagnosis result is erroneous is received, the correct fault diagnosis result fed back by the fourth target end is received through the fourth learning interface, and the fourth learning task and the correct fault diagnosis result are sent to the fault diagnosis agent, and the correct fault diagnosis result is sent to the result generation agent.

[0084] In this embodiment, a third-party interface is reserved at each agent, such as the Teacher-Student interface. The third party, that is, each target end, can be manually reviewed or checked, or can be reviewed or tested through a cloud server. If the third party detects that there are errors in the results of each agent, it can correct them and feed back the correct data to the corresponding agent so that it can learn the teacher's correction results. This not only prevents the agent from propagating errors backward, but also enables the agent to continue learning and improve performance. Exemplarily, when the data collection agent obtains performance data and operation log data, the first learning interface allows manual inspection of the integrity and accuracy of the data, and then hand it over to the data analysis agent for processing after ensuring that there are no errors. Secondly, in the process of completing data and removing redundant data, the data analysis agent can manually review the analysis results through the second learning interface to ensure that the data cleaning and preprocessing work meet the standards. In terms of interactive agents that interact with operation and maintenance personnel, the corresponding learning interface ensures that the generated diagnostic reports and repair suggestions are manually reviewed to avoid unnecessary maintenance operations caused by erroneous information. Manual feedback not only provides an instant error correction mechanism, but also improves the self-learning ability and accuracy of each agent through continuous correction and optimization. In the task allocation, resource management and coordination process of the manager agent with other agents, the reserved learning interface also provides an audit interface to ensure that the entire data center can detect and solve potential problems in a timely manner while operating efficiently, avoiding data center failures and resource waste. For the fault diagnosis agent, since manual intervention cannot easily detect errors, this embodiment adopts the self-evolution mechanism of chaos engineering to simulate data center failures. When it is determined that the fault diagnosis result is wrong, it can be corrected by the results of chaos engineering to achieve the purpose of self-evolution.

[0085] As can be seen from the above, this embodiment, by introducing a learning interface at each intelligent agent, is conducive to improving the performance of each intelligent agent, effectively preventing error propagation, improving the accuracy of the final fault prediction result, and ensuring the stability and reliability of the entire data center.

[0086] The present invention also provides a data center, which may include a fault prediction system. The fault prediction system may be deployed on a master node server in the data center or on a server with the best performance in the data center, which does not affect the implementation of the present invention.

[0087] See also Figure 3The fault prediction system may include a manager agent, a data collection agent, a data analysis agent, a fault diagnosis agent, an expert agent, and an interactive agent. The manager agent communicates with the data collection agent, the data analysis agent, the fault diagnosis agent, the expert agent, and the interactive agent respectively; the data collection agent communicates with the data analysis agent, the data analysis agent communicates with the fault diagnosis agent, the fault diagnosis agent communicates with the expert agent, and the expert agent communicates with the interactive agent. The number of data collection agents, data analysis agents, and fault diagnosis agents can be set according to actual conditions, such as Figure 4 As shown, three data collection agents, three data analysis agents, and three fault diagnosis agents can be set respectively, and the data collection agents, the data analysis agents, and the fault diagnosis agents correspond one to one.

[0088] In this embodiment, when the manager agent is executed, the steps of the fault prediction method described in any of the above embodiments are implemented, and the manager agent is used to coordinate the data collection agent, the data analysis agent, the fault diagnosis agent, the expert agent and the interactive agent, and realize the interaction between the data collection agent, the data analysis agent, the fault diagnosis agent, the expert agent and the interactive agent, so as to ensure the interaction between the data collection agent, the data analysis agent, the fault diagnosis agent, the expert agent and the interactive agent, thereby ensuring the smooth execution of various tasks such as data collection, analysis, diagnosis and prediction. For example, when the data collection agent completes data collection, the manager agent notifies the data analysis agent to start data processing, and when the fault diagnosis agent outputs the fault diagnosis result, the manager agent transmits the fault diagnosis result to the interactive agent that interacts with the operation and maintenance personnel to generate a detailed prediction result and a fault prediction result of the repair suggestion. By coordinating the data collection agent, the data analysis agent, the fault diagnosis agent, the expert agent and the interactive agent to work together, the manager agent can effectively improve the overall operation efficiency and response speed of the data center. The data collection agent is used to collect the original operating status data of each server in the data center after receiving the data collection subtask; the data analysis agent is used to process the original operating status data after receiving the data analysis subtask to obtain operating status data that meets the preset quality conditions; the fault diagnosis agent is used to perform fault diagnosis on the operating status data after receiving the fault diagnosis subtask; the expert agent is used to generate a total fault diagnosis result by integrating the fault diagnosis results of each fault diagnosis agent after receiving the diagnosis result generation subtask; the interactive agent is used to generate a fault prediction result based on the total fault diagnosis result, prior knowledge and interactive information after receiving the report generation subtask.

[0089] Exemplarily, the manager agent is the core of the entire fault prediction system, which is used to at least realize the allocation of tasks, management of resources and coordination of interactions between agents. For task allocation: the manager agent can allocate different tasks to corresponding agents according to the needs and task priorities of the data center. For example, when performance data needs to be collected, the manager agent sends the data collection task to the data collection agent to perform the task; when data analysis is required, the task is transferred to the data analysis agent for execution. The manager agent is used to manage and optimize all computing and memory resources of the data center: monitor the workload and resource consumption of each agent, and ensure the reasonable allocation and efficient utilization of computing resources and memory resources of the data center. For example, when there is an agent with an overload, the manager agent can adopt load balancing methods such as load balancing to achieve load balancing, or dynamically adjust resource allocation when necessary to maintain the stability and efficiency of the system. The manager agent can ensure the efficient operation of the data center and guarantee the stability, reliability and availability of the data center by allocating tasks, managing resources and coordinating interactions.

[0090] Exemplarily, the data collection agent can be configured with planning and tools to at least have an interface call function, a monitoring tool call function and a tracking data function; wherein the interface call function is used to obtain the server's operation log data by calling the remote control interface, such as calling RedFish (out-of-band management protocol) and other protocols; the monitoring tool call function is used to call the monitoring tool interface to obtain server performance indicator data, such as having the ability to call tools such as rometheus (service monitoring system and time series database); the tracking data function is used to obtain the call chain data corresponding to the target identifier implanted by the tracking system, such as realizing the tracking data function through programming functions. In this embodiment, the data collection agent can collect various performance indicator data such as CPU utilization, memory usage, disk performance, etc., the server's operation log data and call chain data by calling various protocol frameworks and APIs (Application Programming Interfaces) of the server. For the operation log data, the standard RedFish interface or the customized http (Hypertext Transfer Protocol) interface programming can be used to collect data. For tracing systems such as Jaeger (tracing system name) and Zipkin (tracing system name), specific identification information such as ID for tracing is implanted in specific locations, and then the data collection agent collects data through programming. For performance indicator data, performance indicator data can be collected by calling the interfaces of monitoring tools such as Prometheus (monitoring tool name) and Graphite (monitoring tool name) and performing programming.

[0091] Exemplarily, the data analysis agent includes a data processing function, a log parsing tool calling function, and a data cleaning tool calling function. The data processing function can compile programs of corresponding functions through programming capabilities, such as log parsing tools such as LogParser (tool name). Among them, the data analysis agent filters the operation log data whose length is less than the preset length threshold by calling the data processing function, thereby avoiding the short operation log data from affecting the accuracy of subsequent data analysis due to incomplete information. The remaining operation log data is parsed by calling the log parsing tool function, and invalid data and redundant data in the operation log data are removed. Due to some reasons, the call chain information may be incomplete, and the data analysis agent can also delete incomplete call chain data by calling the data processing function; the data analysis agent also deletes redundant data in the server performance indicator data by calling the data processing function, and completes the server performance indicator data by calling the data cleaning tool function, such as calling pandas (tool name) in python (compiled language) to complete the missing data.

[0092] In this embodiment, the data analysis agent is used to process and optimize the original operating status data collected by the data collection agent. Considering that the original operating status data may be missing, redundant, incomplete, inconsistent, etc., in order to avoid these low-quality data affecting the accuracy and effectiveness of subsequent fault diagnosis, this embodiment can first call various data cleaning and preprocessing tools through the data analysis agent to complete the missing data. For example, the missing data can be filled by interpolation or a prediction algorithm based on historical data. Secondly, the data analysis agent can remove redundant data by finding duplicate records and merging similar data, thereby reducing the overhead of data storage and processing. For the operation log data, in order to further analyze and mine, the data analysis agent also needs to parse the operation log data, extract key fields, format timestamps, and identify event patterns, and convert unstructured log data into structured data.

[0093] Considering that each intelligent agent of the fault prediction system will generate some new knowledge in the process of learning, and this new knowledge includes short-term knowledge and long-term knowledge, the fault prediction system will automatically memorize short-term knowledge, while the long-term knowledge system cannot memorize, and long-term knowledge can play a certain role in other dialogues and tasks, thereby supplementing the deficiency of short-term knowledge memory, so that the fault prediction system can maintain the integrity of memory for a considerable period of time. Based on the above embodiment, the fault prediction system of the present invention may also include a knowledge database; the knowledge database includes a structured knowledge sub-base and an unstructured sub-base; obtain the historical knowledge that the manager intelligent agent and / or the data collection intelligent agent and / or the data analysis intelligent agent and / or the fault diagnosis intelligent agent and / or the expert intelligent agent and / or the interactive intelligent agent have learned, and select the target historical knowledge that has not been accessed within a preset time period from the historical knowledge; for the first type of target historical knowledge belonging to the structured data type, the first type of target historical knowledge is converted into vector data and stored in the structured knowledge sub-base; for the second type of target historical knowledge belonging to the unstructured data type, the second type of target historical knowledge is converted into text data and stored in the unstructured knowledge sub-base.

[0094] In this embodiment, long-term knowledge refers to knowledge that has been learned but is no longer used in the near future, and the preset time period can be flexibly adjusted according to the length of the near future in different application scenarios. Since the fault prediction system limitations cannot be directly memorized, this embodiment uses a knowledge database for storage. For structured knowledge, in order to facilitate the conversion of such knowledge into vectors, a format such as markdown (lightweight markup language) can be used directly for storage. For unstructured knowledge, it can be stored in text form. In order to ensure data accuracy, long-term knowledge can be manually verified regularly, and after confirmation, it can be converted into vectors and stored in the knowledge database for use next time or in other tasks.

[0095] As can be seen from the above, this embodiment completes the unified management of server faults in the data center based on the collaboration of multiple agents, which solves the current situation where each server fault management module acts independently and cannot work in a coordinated and unified manner, so that fault management is integrated into a large system module and works intelligently and collaboratively. The fault prediction system uses an advanced large language model as the core engine to drive multiple agents to work collaboratively. It can flexibly call a variety of tools and interfaces to accurately respond to complex fault situations that may occur during server operation. Through the powerful capabilities of deep learning and natural language processing, the accuracy of fault identification and prediction accuracy can be significantly improved, thereby effectively solving the difficulty of fault diagnosis for complex tasks in the data center, greatly reducing the workload of fault diagnosis and prediction, and at the same time making each link of operation and maintenance an organic whole, effectively avoiding the fragmentation of the entire data center.

[0096] The present invention also provides a corresponding device for the fault prediction method, which further makes the method more practical. Among them, the device can be described from the perspective of functional modules and hardware. The fault prediction device provided by the present invention is introduced below. The device is used to implement the fault prediction method provided by the present invention. In this embodiment, the fault prediction device may include or be divided into one or more program modules, and the one or more program modules are stored in a storage medium and executed by one or more processors to complete the fault prediction method disclosed in Example 1. The program module referred to in this embodiment refers to a series of computer program instruction segments that can complete specific functions, which is more suitable for describing the execution process of the fault prediction device in the storage medium than the program itself. The following description will specifically introduce the functions of each program module of this embodiment. The fault prediction device described below and the fault prediction method described above can be referenced to each other.

[0097] From the perspective of functional modules, see Figure 5 , Figure 5 The structural diagram of the fault prediction device provided in this embodiment under a specific implementation manner, the device may include:

[0098] The task decomposition module 501 is used to divide the fault prediction task into a data collection subtask, a data analysis subtask, a fault diagnosis subtask and a result generation subtask when the fault prediction task is detected.

[0099] The data collection module 502 is used to send the data collection subtask to the data collection agent so that the data collection agent collects the original operating status data of each server in the data center.

[0100] The collection and analysis module 503 is used to send the original operating status data and the data analysis subtask to the data analysis agent when the data collection agent completes the data collection subtask, so that the data analysis agent processes the original operating status data to obtain operating status data that meets the preset quality conditions.

[0101] The fault diagnosis module 504 is used to send the operating status data and the fault diagnosis subtask to the fault diagnosis agent when the data analysis agent completes the data analysis subtask, so that the fault diagnosis agent can perform fault diagnosis on the operating status data.

[0102] The fault prediction result output module 505 is used to send the fault diagnosis result and the result generation subtask to the result generation agent when the fault diagnosis agent completes the fault diagnosis subtask, so that the result generation agent generates a fault prediction result based on the fault diagnosis result and prior knowledge; wherein, the data collection agent, the data analysis agent, the fault diagnosis agent, and the result generation agent are based on the language model, and are configured with corresponding functions to enable each agent to complete the task autonomously.

[0103] Exemplarily, in some implementations of the present embodiment, the above-mentioned fault diagnosis module 504 is also used for: when the fault diagnosis agent includes at least a first fault diagnosis agent, a second fault diagnosis agent and a hybrid fault diagnosis agent, and the data collection agent includes at least a first data collection agent and a second data collection agent, the first operating status data collected by the first data collection agent is sent to the first fault diagnosis agent, the second operating status data collected by the second data collection agent is sent to the second fault diagnosis agent, and the first operating status data and the second operating status data are sent to the hybrid fault diagnosis agent, so that the first fault diagnosis agent, the second fault diagnosis agent and the hybrid fault diagnosis agent can simultaneously perform fault diagnosis on the operating status data.

[0104] Illustratively, in some other implementations of the present embodiment, the above-mentioned fault prediction result output module 505 is also used for: when there are multiple fault diagnosis agents, obtaining the historical task execution data of each fault diagnosis agent; the historical task execution data at least includes the number of historical tasks, the execution parameters of each historical task, the behavior parameters of each historical task and the interaction history parameters; according to the historical task execution data of each fault diagnosis agent, determining the decision weight of the corresponding fault diagnosis agent to perform the current task; according to the fault diagnosis results of each fault diagnosis agent and the corresponding decision weights, determining the fault diagnosis results of the input result generating agent.

[0105] As an exemplary implementation of the above embodiment, the above fault prediction result output module 505 can also be used to: determine the task execution score of the current historical task based on the current historical task success rate, the current historical task execution speed, and the resource consumption efficiency of the current historical task, and determine the performance factor of each fault diagnosis agent based on the task execution score and task difficulty score corresponding to each historical task of each fault diagnosis agent; determine the trust factor of each fault diagnosis agent based on the interaction success rate, protocol compliance rate, and communication success rate of each fault diagnosis agent in the process of completing each historical task; determine the experience value of each fault diagnosis agent based on the total amount of historical task processing of each fault diagnosis agent, the total number of target historical tasks that meet the preset complexity conditions, and the total running time; determine the decision weight for executing the current task based on the performance factor, trust factor and experience value of each fault diagnosis agent and their respective weight coefficients.

[0106] Exemplarily, in some other implementations of the present embodiment, the above-mentioned fault diagnosis module 504 is also used for: when the fault diagnosis agent runs multiple tasks simultaneously, when insufficient computing resources and / or memory resources are detected, obtaining the workload information and task running information of the fault diagnosis agent; reading the task type and task impact of each task from the task running information, and determining the urgency score of each task according to the task type and task impact of each task; determining the workload in the process of executing each task according to the workload information, and determining the load score according to the workload quantity and the workload resource demand; determining the priority of each task according to the urgency score, load score and corresponding weight coefficient of each task; and performing fault diagnosis on the operating status data based on the priority of the fault diagnosis subtask.

[0107] Exemplarily, in some other implementations of the present embodiment, the above-mentioned fault prediction result output module 505 is also used for: when there are multiple fault diagnosis agents, the result generating agent includes an expert agent and an interactive agent, and the result generating subtask includes a diagnosis result generating subtask and a report generating subtask, the fault diagnosis result of each fault diagnosis agent and the diagnosis result generating subtask are sent to the expert agent, so that the expert agent generates a total fault diagnosis result based on the fault diagnosis results of each fault diagnosis agent and the expert knowledge base; when the diagnosis result generating subtask is completed, the report generating subtask and the total fault diagnosis result are sent to the interactive agent, so that the interactive agent uses the learning retrieval enhanced generation method to generate a fault prediction result based on the total fault diagnosis result and the manual interaction information.

[0108] Exemplarily, in some other implementations of the present embodiment, the above-mentioned device may also include a resource allocation module, which can be used to: obtain the workload and resource consumption information of the data collection agent, the data analysis agent, the fault diagnosis agent and the result generation agent; determine the resource allocation weights for the data collection agent, the data analysis agent, the fault diagnosis agent and the result generation agent at the current moment according to the workload and resource consumption information of the data collection agent, the data analysis agent, the fault diagnosis agent and the result generation agent; and adjust the resource allocation of the data collection agent, the data analysis agent, the fault diagnosis agent and the result generation agent according to the current resource allocation weights.

[0109] Exemplarily, in some other implementations of this embodiment, the above-mentioned device may also include a learning module, which can be used to: send the original operating status data to the first target end through the first learning interface, so that the first target end checks the original operating status data; when receiving feedback information that there is an error in the original operating status data, receive the correct original operating status data fed back by the first target end through the first learning interface, and send the first learning task and the correct original operating status data to the data collection intelligent agent, and send the correct original operating status data as the original operating status data to the data collection intelligent agent; send the operating status data to the second target end through the second learning interface, so that the second target end checks the operating status data. ; When receiving feedback information that there is an error in the running state data, receive the correct running state data fed back by the second target end through the second learning interface, send the second learning task and the correct running state data to the data collection agent, and send the correct running state data as the running state data to the fault diagnosis agent; send the fault prediction result to the third target end through the third learning interface so that the third target end can review the fault prediction result; when receiving feedback information that there is an error in the fault prediction result, receive the correct fault prediction result fed back by the third target end through the third learning interface, send the third learning task and the correct fault prediction result to the result generation agent, and output the correct fault prediction result as the fault prediction result. Send the fault diagnosis result to the fourth target end through the fourth learning interface so that the fourth target end can correct the fault diagnosis result based on the self-evolution mechanism of chaos engineering; when receiving feedback information that there is an error in the fault diagnosis result, receive the correct fault diagnosis result fed back by the fourth target end through the fourth learning interface, send the fourth learning task and the correct fault diagnosis result to the fault diagnosis agent, and send the correct fault diagnosis result to the result generation agent.

[0110] The fault prediction device mentioned above is described from the perspective of functional modules. Furthermore, the present invention also provides an electronic device, which is described from the perspective of hardware. Figure 6 The following is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention in one implementation manner. Figure 6 As shown, the electronic device includes a memory 60 for storing a computer program; and a processor 61 for implementing the steps of the fault prediction method mentioned in any of the above embodiments when executing the computer program.

[0111] Among them, the processor 61 may include one or more processing cores, such as a 4-core processor, an 8-core processor, and the processor 61 may also be a controller, a microcontroller, a microprocessor or other data processing chip. The processor 61 may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). The processor 61 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 61 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 61 may also include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning.

[0112] The memory 60 may include one or more computer non-volatile storage media, which may be non-transitory. The memory 60 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices and flash memory storage devices. In some embodiments, the memory 60 may be an internal storage unit of an electronic device, such as a hard disk of a server. In other embodiments, the memory 60 may also be an external storage device of an electronic device, such as a plug-in hard disk equipped on a server, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. Further, the memory 60 may also include both an internal storage unit and an external storage device of an electronic device. The memory 60 may not only be used to store application software and various types of data installed in the electronic device, such as: the code of the program in the process of executing the fault prediction method, etc., but also be used to temporarily store data that has been output or is to be output. In this embodiment, the memory 60 is at least used to store the following computer program 601, wherein, after the computer program is loaded and executed by the processor 61, the relevant steps of the fault prediction method disclosed in any of the aforementioned embodiments can be implemented. In addition, the resources stored in the memory 60 may also include an operating system 602 and data 603, and the storage method may be temporary storage or permanent storage. The operating system 602 may include Windows, Unix, Linux, etc. The data 603 may include but is not limited to data corresponding to the fault prediction result.

[0113] In some embodiments, the electronic device may further include a display screen 62, an input / output interface 63, a communication interface 64 or a network interface, a power supply 65 and a communication bus 66. Among them, the display screen 62 and the input / output interface 63, such as a keyboard, belong to the user interface, and exemplary user interfaces may also include standard wired interfaces, wireless interfaces, etc. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, and an OLED (Organic Light-Emitting Diode) touch device, etc. The display may also be appropriately referred to as a display screen or a display unit, which is used to display information processed in the electronic device and to display a visual user interface. The communication interface 64 may exemplarily include a wired interface and / or a wireless interface, such as a WI-FI interface, a Bluetooth interface, etc., which are generally used to establish a communication connection between the electronic device and other electronic devices. The communication bus 66 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 6 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0114] Those skilled in the art will understand that Figure 6 The structure shown in the figure does not constitute a limitation on the electronic device, and may include more or fewer components than those shown in the figure, for example, it may also include a sensor 67 for realizing various functions.

[0115] It is understandable that if the fault prediction method in the above embodiment is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a non-volatile storage medium. Based on such an understanding, the technical solution of the present invention is essentially or the part that contributes to the relevant technology or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium to execute all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes but is not limited to: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), electrically erasable programmable ROM, register, hard disk, multimedia card, card-type memory (such as SD or DX memory, etc.), magnetic memory, removable disk, CD-ROM, magnetic disk or optical disk, etc. Various media that can store program codes. Based on this, the present invention also provides a non-volatile storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the fault prediction method recorded in any of the above embodiments are executed.

[0116] It is understandable that if the fault prediction method in the above embodiment is implemented in the form of a software functional unit and sold or used as an independent product, the computer software product does not need to be stored in a physical storage medium. For example, it can be directly transmitted to a computer or other device with information processing capabilities through a wired network or a wireless network to execute all or part of the steps of the methods of each embodiment of the present invention. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the relevant technology or all or part of the technical solution can be embodied in the form of a software product. Based on this, the present invention also provides a computer program product that stores a computer program. When the computer program is executed by a processor, the steps of the fault prediction method described in any of the above embodiments are performed.

[0117] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the hardware disclosed in the embodiment, including electronic devices, non-volatile storage media and computer program products, since they correspond to the methods disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part description.

[0118] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0119] The above is a detailed introduction to a data center and fault prediction method, electronic device, non-volatile storage medium, and computer program product provided by the present invention. Specific examples are used herein to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. It should be pointed out that based on the embodiments of the present invention, for ordinary technicians in this technical field, all other embodiments obtained without creative work are within the scope of protection of the present invention. Without departing from the principles of the present invention, the present invention may also be improved and modified in a number of ways, and these improvements and modifications also fall within the scope of protection of the present invention.

Claims

1. A fault prediction method, characterized in that: include: The fault prediction task is divided into data collection subtask, data analysis subtask, fault diagnosis subtask and result generation subtask; Sending the data collection subtask to the data collection agent so that the data collection agent collects the original operation status data of each server in the data center; When the data collection agent completes the data collection subtask, the original operating status data and the data analysis subtask are sent to the data analysis agent, so that the data analysis agent processes the original operating status data to obtain operating status data that meets the preset quality conditions; When the data analysis agent completes the data analysis subtask, the operating state data and the fault diagnosis subtask are sent to the fault diagnosis agent, so that the fault diagnosis agent performs fault diagnosis on the operating state data; When the fault diagnosis agent completes the fault diagnosis subtask, the fault diagnosis result and the result generation subtask are sent to the result generation agent, so that the result generation agent generates a fault prediction result according to the fault diagnosis result and prior knowledge; Among them, the data collection agent, the data analysis agent, the fault diagnosis agent, and the result generation agent are agents based on language models, which are configured with corresponding functions to enable them to complete tasks autonomously.

2. The fault prediction method according to claim 1, characterized in that: The fault diagnosis agent at least includes a first fault diagnosis agent, a second fault diagnosis agent and a mixed fault diagnosis agent, the data collection agent at least includes a first data collection agent and a second data collection agent, and the running state data and the fault diagnosis subtask are sent to the fault diagnosis agent so that the fault diagnosis agent performs fault diagnosis on the running state data, including: The first operating status data collected by the first data collection agent is sent to the first fault diagnosis agent, the second operating status data collected by the second data collection agent is sent to the second fault diagnosis agent, and the first operating status data and the second operating status data are sent to the hybrid fault diagnosis agent, so that the first fault diagnosis agent, the second fault diagnosis agent and the hybrid fault diagnosis agent can perform fault diagnosis on the operating status data at the same time.

3. The fault prediction method according to claim 1, characterized in that: There are multiple fault diagnosis agents, and sending the fault diagnosis results and result generation subtasks to the result generation agent includes: Obtaining historical task execution data of each fault diagnosis agent; the historical task execution data at least includes the number of historical tasks, execution parameters of each historical task, behavior parameters of each historical task and interaction history parameters; According to the historical task execution data of each fault diagnosis agent, the decision weight of the corresponding fault diagnosis agent to execute the current task is determined; According to the fault diagnosis results of each fault diagnosis agent and its corresponding decision weight, the fault diagnosis result of the result generation agent is determined to be input.

4. The fault prediction method according to claim 3, characterized in that: The step of determining the decision weight of the corresponding fault diagnosis agent to execute the current task according to the historical task execution data of each fault diagnosis agent includes: Determine the task execution score of the current historical task according to the success rate of the current historical task, the execution speed of the current historical task, and the resource consumption efficiency of the current historical task, and determine the performance factor of each fault diagnosis agent according to the task execution score and task difficulty score corresponding to each historical task of each fault diagnosis agent; Determine the trust factor of each fault diagnosis agent according to the interaction success rate, protocol compliance rate, and communication success rate of each fault diagnosis agent in the process of completing each historical task; Determine the experience value of each fault diagnosis agent based on the total amount of historical tasks processed by each fault diagnosis agent, the total number of target historical tasks that meet the preset complexity conditions, and the total running time; According to the performance factor, trust factor and experience value of each fault diagnosis agent and their respective weight coefficients, the decision weight for executing the current task is determined.

5. The fault prediction method according to claim 1, characterized in that: The fault diagnosis agent runs multiple tasks simultaneously, and the fault diagnosis agent performs fault diagnosis on the running status data, including: When insufficient computing resources and / or memory resources are detected, workload information and task running information of the fault diagnosis agent are obtained; Reading the task type and task impact of each task from the task running information, and determining the urgency score of each task according to the task type and task impact of each task; Determine the workload during the execution of each task according to the workload information, and determine the load score according to the workload quantity and workload resource demand; Determine the priority of each task based on its urgency score, load score and corresponding weight coefficient; Based on the priority of the fault diagnosis subtask, fault diagnosis is performed on the operating status data.

6. The fault prediction method according to claim 1, characterized in that: The fault diagnosis agent is multiple, the result generation agent includes an expert agent and an interactive agent, the result generation subtask includes a diagnosis result generation subtask and a report generation subtask, and the fault diagnosis result and the result generation subtask are sent to the result generation agent so that the result generation agent generates a fault prediction result according to the fault diagnosis result and prior knowledge, including: Sending the fault diagnosis results of each fault diagnosis agent and the diagnosis result generation subtask to the expert agent, so that the expert agent generates a total fault diagnosis result according to the fault diagnosis results of each fault diagnosis agent and the expert knowledge base; When the diagnosis result generation subtask is completed, the report generation subtask and the overall fault diagnosis result are sent to the interactive agent, so that the interactive agent uses the learning retrieval enhancement generation method to generate a fault prediction result based on the overall fault diagnosis result and manual interaction information.

7. The fault prediction method according to claim 1, characterized in that: Also includes: Acquire workload and resource consumption information of the data collection agent, the data analysis agent, the fault diagnosis agent, and the result generation agent; Determine resource allocation weights for the data collection agent, the data analysis agent, the fault diagnosis agent, and the result generation agent at the current moment according to the workload and resource consumption information of the data collection agent, the data analysis agent, the fault diagnosis agent, and the result generation agent; According to the current resource allocation weights, the resource allocation of the data collection agent, the data analysis agent, the fault diagnosis agent and the result generation agent is adjusted.

8. The fault prediction method according to any one of claims 1 to 7, characterized in that: Also includes: Sending the original running status data to a first target end through a first learning interface, so that the first target end checks the original running status data; When receiving feedback information that the original running status data contains errors, receiving the correct original running status data fed back by the first target end through the first learning interface, sending the first learning task and the correct original running status data to the data collection agent, and sending the correct original running status data as the original running status data to the data collection agent; Sending the running status data to the second target end through the second learning interface so that the second target end checks the running status data; when receiving feedback information that the running status data is wrong, receiving the correct running status data fed back by the second target end through the second learning interface, and sending the second learning task and the correct running status data to the data collection agent, and sending the correct running status data as the running status data to the fault diagnosis agent; Sending the fault prediction result to the third target end through the third learning interface so that the third target end reviews the fault prediction result; when receiving feedback information that the fault prediction result is wrong, receiving the correct fault prediction result fed back by the third target end through the third learning interface, sending the third learning task and the correct fault prediction result to the result generating agent, and outputting the correct fault prediction result as the fault prediction result; The fault diagnosis result is sent to the fourth target end through the fourth learning interface, so that the fourth target end corrects the fault diagnosis result based on the chaos engineering self-evolution mechanism; when feedback information that the fault diagnosis result is erroneous is received, the correct fault diagnosis result fed back by the fourth target end is received through the fourth learning interface, and the fourth learning task and the correct fault diagnosis result are sent to the fault diagnosis agent, and the correct fault diagnosis result is sent to the result generation agent.

9. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the processor is used to implement the steps of the fault prediction method according to any one of claims 1 to 8 when executing a computer program stored in the memory.

10. A non-volatile storage medium, characterized in that: The non-volatile storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the fault prediction method according to any one of claims 1 to 8 are implemented.

11. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instruction is executed by a processor, the steps of the fault prediction method according to any one of claims 1 to 8 are implemented.

12. A data center, characterized in that: Includes failure prediction system; The fault prediction system includes a manager agent, a data collection agent, a data analysis agent, a fault diagnosis agent, an expert agent and an interactive agent; The manager agent performs data communication with the data collection agent, the data analysis agent, the fault diagnosis agent, the expert agent and the interaction agent, and executes the steps of the fault prediction method according to any one of claims 1 to 8; The data collection agent is used to collect the original operation status data of each server in the data center after receiving the data collection subtask; The data analysis agent is used to process the original operation status data after receiving the data analysis subtask to obtain the operation status data that meets the preset quality conditions; The fault diagnosis agent is used to perform fault diagnosis on the operating status data when receiving the fault diagnosis subtask; The expert agent is used to generate a total fault diagnosis result by integrating the fault diagnosis results of each fault diagnosis agent upon receiving the diagnosis result generation subtask; The interactive agent is used to generate a fault prediction result based on the overall fault diagnosis result, prior knowledge and interactive information when receiving the report generation subtask.

13. The data center according to claim 12, characterized in that: The data collection agent includes an interface calling function, a monitoring tool calling function and a data tracking function; Among them, the interface calling function is used to obtain the server's operation log data by calling the remote control interface; the monitoring tool calling function is used to call the monitoring tool interface to obtain the server performance indicator data; the tracking data function is used to obtain the call chain data corresponding to the target identifier implanted by the tracking system.

14. The data center according to claim 12, characterized in that: The data analysis agent includes data processing functions, log analysis tool calling functions and data cleaning tool calling functions; The data analysis agent filters the operation log data whose length is less than a preset length threshold by calling the data processing function; performs log parsing on the remaining operation log data by calling the function of the log parsing tool, and removes invalid data and redundant data in the operation log data; The data analysis agent also deletes incomplete call chain data by calling a data processing function; The data analysis agent also deletes redundant data in the server performance indicator data by calling the data processing function, and completes the server performance indicator data by calling the function of the data cleaning tool.

15. The data center according to claim 12, characterized in that: The fault prediction system further comprises a knowledge database; the knowledge database comprises a structured knowledge sub-base and an unstructured knowledge sub-base; Acquire the historical knowledge that has been learned by the manager agent and / or the data collection agent and / or the data analysis agent and / or the fault diagnosis agent and / or the expert agent and / or the interactive agent, and select target historical knowledge that has not been accessed within a preset time period from the historical knowledge; For the first type of target historical knowledge belonging to the structured data type, converting the first type of target historical knowledge into vector data and storing it in the structured knowledge sub-base; For the second type of target historical knowledge belonging to the unstructured data type, the second type of target historical knowledge is converted into text data and stored in the unstructured knowledge sub-base.

Images