Fault positioning method and device and storage medium

By combining Transformer, GCN and DQN methods, reinforcement learning agents are formed, which solves the problem of existing fault positioning technology relying on manual intervention and inaccurate positioning, and realizes more efficient fault positioning and automated monitoring, improving system stability and diagnostic accuracy.

CN118916203APending Publication Date: 2024-11-08CHINA TELECOM DIGITAL INTELLIGENCE TECH CO LTD
View PDF 0 Cites 2 Cited by

Patent Information

Application Number
CN202411038655.8
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-31
Publication Date
2024-11-08

AI Technical Summary

Technical Problem

The existing fault positioning technology relies on high degree of manual intervention, and the positioning is not accurate enough, and monitoring efficiency and accuracy need to be improved.

Method used

Using a combination of Transformer, GCN and DQN, the system topology and virtual machine dependencies are represented through graph structures, and the node relationship is encoded using graph convolution networks to define the Markov decision-making process to form a reinforcement learning agent, and the intelligence is trained through reinforcement learning algorithms. The body is fault-located.

Benefits of technology

It realizes more accurate and automated monitoring and diagnosis, reduces dependence on manual intervention, improves the stability and availability of the system, and improves the accuracy and intelligence of monitoring and diagnosis.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118916203A_ABST
    Figure CN118916203A_ABST
Patent Text Reader

Abstract

The invention relates to a fault positioning method and device, and the method achieves the automatic monitoring and diagnosis of a virtualization platform through the introduction of TRG and reinforcement learning, and reduces the degree of dependence on manual intervention. The TRG intelligent agent can receive system data in real time and automatically adjust operation parameters and execute fault repairing operation according to a learned strategy, and the stability and usability of the system are improved. The intelligent agent is trained through a reinforcement learning algorithm, so that the intelligent agent can make appropriate decisions in different system states, and the accuracy and the intelligent degree of monitoring and diagnosis are improved. The invention further provides a fault positioning device and a storage medium.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the field of computing technology, and in particular relates to a fault location method, device and storage medium. Background Art

[0002] Fault location technology originated from the development of computer technology and the popularization of the Internet. In the early days, IT systems and network equipment were mainly maintained and managed manually. With the continuous advancement and development of information technology, IT systems and network environments have become more and more complex and large, and traditional manual management methods can no longer meet business needs. For this reason, fault location technology has gradually emerged. Its core idea is to achieve real-time monitoring and maintenance of IT systems and network equipment through automated tools and monitoring systems to improve system reliability and stability, reduce maintenance costs and improve operational efficiency.

[0003] The development of fault location technology is inseparable from the support of technologies such as big data, cloud computing, and artificial intelligence. In the era of big data, fault location technology can quickly discover system anomalies and failures through the analysis and processing of massive data, and take timely measures to repair them. In the era of cloud computing, fault location technology can realize the monitoring and management of distributed and virtualized systems, and improve the efficiency and flexibility of fault location.

[0004] However, in the existing technology, the degree of reliance on manual intervention is still high in fault location technology, which cannot be fully automated, the positioning is not accurate enough, and the monitoring efficiency and accuracy need to be improved. Summary of the invention

[0005] In view of the above deficiencies in the prior art, the purpose of the invention is to provide a fault location method and device to achieve more accurate and intelligent monitoring and diagnosis, thereby improving monitoring efficiency and accuracy compared to traditional methods.

[0006] A first aspect of the present invention provides a fault location method, comprising:

[0007] S1, represents the system topology and the dependencies between virtual machines as a graph structure;

[0008] S2, using a graph convolutional network GCN to encode the graph structure data to capture the relationship between nodes;

[0009] S3, taking the encoded graph structure data as the environment state, defines a Markov decision process MDP to form a reinforcement learning agent;

[0010] S4, training the agent through a reinforcement learning algorithm DQN;

[0011] S5, locate the fault based on the trained agent.

[0012] The fault location method of the present invention adopts a method combining Transformer, GCN and DQN, and the deep learning algorithm automatically learns the operating characteristics and abnormal behaviors of the system, automates judgment, and realizes more accurate and automated monitoring.

[0013] Furthermore, the topology of the system and the dependencies between virtual machines are represented as a graph structure including:

[0014] Each virtual machine is a node in the graph, and the node attributes include the performance indicators and log information of the virtual machine. The dependency relationship between the virtual machines is represented by the edges.

[0015] Furthermore, the graph structure data is encoded using a graph convolutional network (GCN) to capture the relationship between nodes, including:

[0016] The running state of the virtual machine is encoded through multi-layer GCN, and the representation of nodes and edges is obtained to characterize the current state of the system.

[0017] Furthermore, the encoded graph structure data is used as the environment state to define a Markov decision process MDP to form a reinforcement learning agent including:

[0018] The agent decides how to adjust the operating parameters of the virtual machine and perform fault repair operations based on the current system status and the learned strategies.

[0019] Furthermore, training the agent by using the reinforcement learning algorithm DQN includes:

[0020] The simulation environment provides different system states and failure conditions;

[0021] The agent interacts with the simulated environment and adjusts its strategy based on reward signals to achieve optimal monitoring and diagnosis effects.

[0022] Furthermore, the agent interacts with the simulated environment and adjusts its strategy based on the reward signal to achieve optimal monitoring and diagnosis effects, including:

[0023] Collect data from system logs, performance monitoring tools, or network traffic analysis tools;

[0024] A graph neural network is constructed through the GCN, and a multi-layer graph convolution layer is defined to extract the features of nodes and edges;

[0025] Determine the Transformer encoder, use the GCN output as input, design a reward function, and give feedback based on system performance indicators or fault recovery;

[0026] A predetermined DQN algorithm is selected, a policy network and a value network of the agent are initialized, a reward signal is calculated, and the policy network and the value network are optimized, wherein the reward signal includes performance improvement and / or fault elimination.

[0027] Furthermore, locating the fault based on the trained agent includes:

[0028] The agent receives and encodes the operating data of the system in real time, and automatically adjusts the operating parameters of the virtual machine and performs fault repair operations according to the current system state and the learned strategy. A second aspect of the present invention provides a device for implementing a fault location method, comprising:

[0029] A graph structure building module configured to represent the topology of the system and the dependency relationship between virtual machines as a graph structure;

[0030] An encoding module is configured to encode the graph structure data using a graph convolutional network (GCN) to capture the relationship between nodes;

[0031] A building module is configured to use the encoded graph structure data as the environment state, define a Markov decision process MDP, and form a reinforcement learning agent;

[0032] A training module, configured to train the agent through a reinforcement learning algorithm DQN;

[0033] The localization module is configured to locate the fault according to the trained agent.

[0034] A third aspect of the present invention provides a device for implementing a fault location method, comprising:

[0035] Memory, processor and user interface;

[0036] Memory for storing computer programs;

[0037] User interface, used to interact with users;

[0038] A processor is used to read a computer program in a memory. When the processor executes the computer program, it realizes:

[0039] Represent the system topology and dependencies between virtual machines as a graph structure;

[0040] Encode the graph structure data using a graph convolutional network (GCN) to capture the relationship between nodes;

[0041] The encoded graph structure data is used as the environment state, and a Markov decision process MDP is defined to form a reinforcement learning agent;

[0042] Training the agent through a reinforcement learning algorithm DQN;

[0043] The fault is located based on the trained agent.

[0044] According to a fourth aspect of the present invention, a processor-readable storage medium stores a computer program, and the above-mentioned fault location method is implemented when the processor executes the computer program.

[0045] The beneficial effects of the present invention are as follows:

[0046] The method and device described in the present invention, by introducing TRG and reinforcement learning, realize automatic monitoring and diagnosis of the virtualization platform, and reduce the dependence on manual intervention. The TRG agent can receive system data in real time, and automatically adjust the operating parameters and perform fault repair operations according to the learned strategy, thereby improving the stability and availability of the system. The agent is trained by the reinforcement learning algorithm, so that it can make appropriate decisions under different system states, improving the accuracy and intelligence of monitoring and diagnosis. Compared with traditional rule-based or manual adjustment methods, the method of the present invention introduces graph neural networks and reinforcement learning technology, which can learn the operating status of the system and the optimal adjustment strategy from the data, thereby realizing automated monitoring and fault repair, better coping with complex system and network environments, improving the accuracy and intelligence of monitoring and diagnosis, and reducing dependence on manual intervention. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] The accompanying drawings are only used to illustrate specific embodiments and are not considered to limit the present invention. In the entire drawings, the same reference symbols represent the same components. Obviously, the drawings described below are only some embodiments recorded in the embodiments of the present invention. For those of ordinary skill in the art, other drawings can also be obtained based on these drawings.

[0048] Figure 1 A flow chart of a fault location method according to an embodiment of the present invention;

[0049] Figure 2 is a flow chart of the agent training process according to an embodiment of the present invention;

[0050] Figure 3 This is an example flow chart of a fault location method according to an embodiment of the present invention;

[0051] Figure 4 A schematic diagram of a fault location device according to an embodiment of the present invention;

[0052] Figure 5 The figure is a schematic diagram of another fault location device according to an embodiment of the present invention.

[0053] Reference numerals:

[0054] S1, step 1; S2, step 2; S3, step 3; S4, step 4; S5, step 5; S201, step 201; S202, step 202; S203, step 203; S204, step 204; S301, step 301; S302, step 302; S303, step 303; S304, step 304; 401, graph structure construction module; 402, encoding module; 403, construction module; 404, training module; 405, positioning module; 501, processor; 502, memory; 503, user interface; 504, bus interface. DETAILED DESCRIPTION

[0055] In order to enable those skilled in the art to better understand the technical solutions in the embodiments of the present invention, the technical solutions of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all of the embodiments. It should be understood that these descriptions are only exemplary and are not intended to limit the scope of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in the field without making creative work should fall within the scope of protection of the present invention.

[0056] Furthermore, in the following description, descriptions of well-known structures and technologies are omitted to avoid unnecessarily obscuring the concepts disclosed in the present invention.

[0057] In the description of the present invention, it should be noted that, unless otherwise clearly specified and limited, the orientation or position relationship indicated by the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inside", "outside", etc. is based on the orientation or position relationship shown in the drawings, which is only for the convenience of describing the present invention and simplifying the description, and does not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation of the present invention. In addition, the terms "first", "second", and "third" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance. The terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be a connection between the two elements. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0058] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Instead, they are merely examples of methods and systems consistent with some aspects of the present invention as detailed in the appended claims.

[0059] First, some of the terms used in this case are explained.

[0060] 1.GCN, or graph convolutional network, is the abbreviation of Graph Convolution Network.

[0061] 2.DQN, short for Deep Q-Network, is an algorithm based on deep learning and reinforcement learning.

[0062] 3. Transformer, a deep learning model architecture for natural language processing (NLP) and other sequence-to-sequence tasks. The Transformer architecture introduces a self-attention mechanism.

[0063] With the rapid development of information technology, the scale and complexity of systems are increasing, and traditional rule-based monitoring and diagnosis methods have gradually exposed limitations, such as high reliance on manual intervention and inability to achieve full automation. In order to solve these problems, the fault location method of the present invention uses TRG (i.e., a combination of Transformer, GCN and DQN) deep learning algorithm to automatically learn the operating characteristics of the system and automatically judge abnormal behaviors, so as to achieve more accurate and automated monitoring. In the present invention, TRG is also called Reinforcement Graph Neural Network.

[0064] Embodiments are given below in conjunction with the accompanying drawings.

[0065] In a first aspect, the present invention provides a fault location method, specifically, as follows: Figure 1 As shown, the method includes steps S1 to S5:

[0066] S1, represents the system topology and the dependencies between virtual machines as a graph structure;

[0067] S2, using a graph convolutional network GCN to encode the graph structure data to capture the relationship between nodes;

[0068] S3, taking the encoded graph structure data as the environment state, defines a Markov decision process MDP to form a reinforcement learning agent;

[0069] S4, training the agent through a reinforcement learning algorithm DQN;

[0070] S5, locate the fault based on the trained agent.

[0071] The fault location method of the present invention combines Transformer, GCN and DQN to form a TRG agent, which captures the system topology and component dependencies through GCN, and implements decision-making and optimization with DQN, thereby realizing root cause analysis, fault location and automatic repair of abnormal behaviors. The TRG agent can receive system data in real time, and automatically adjust operating parameters and perform fault repair operations according to the learned strategy, thereby improving the stability and availability of the system. The agent is trained by the reinforcement learning algorithm, so that it can make appropriate decisions under different system states, thereby improving the accuracy and intelligence of monitoring and diagnosis.

[0072] As an optional example, S1 represents the system topology and the dependency relationship between virtual machines as a graph structure including:

[0073] Each virtual machine is a node in the graph, and the node attributes include the performance indicators and log information of the virtual machine. The dependency relationship between the virtual machines is represented by the edges.

[0074] As an optional example, S2 uses a graph convolutional network GCN to encode graph structure data to capture the relationships between nodes, including:

[0075] The running state of the virtual machine is encoded through multi-layer GCN, and the representation of nodes and edges is obtained to characterize the current state of the system.

[0076] As an optional example, S3 uses the encoded graph structure data as the environment state, defines a Markov decision process MDP, and forms a reinforcement learning agent including:

[0077] The agent decides how to adjust the operating parameters of the virtual machine and perform fault repair operations based on the current system status and the learned strategies.

[0078] After S1 to S3, the TRG agent composed of Transformer, GCN and DQN is constructed, and then the agent is trained. After the training is completed, the fault can be located based on the trained agent.

[0079] As an optional example, training the agent using the reinforcement learning algorithm DQN in S4 includes:

[0080] The simulation environment provides different system states and failure conditions;

[0081] The agent interacts with the simulated environment and adjusts its strategy based on reward signals to achieve optimal monitoring and diagnosis effects.

[0082] In the embodiment of the present invention, the process in which the agent interacts with the simulation environment and adjusts the strategy according to the reward signal to achieve the best monitoring and diagnosis effect is also a training process. As an optional example, the training process is as follows: Figure 2 As shown, including S201 to S204:

[0083] S201. Collect data from system logs, performance monitoring tools, or network traffic analysis tools. For example, obtain system components and their dependency graphs, including nodes and edges. Process missing values, outliers, and duplicate data for the collected data. Standardize performance indicator data, such as CPU usage, memory usage, etc. Represent components as nodes and dependencies as edges. Add attributes to nodes and edges, such as performance indicators, log information, etc.

[0084] S202. Build a graph neural network through GCN and define a multi-layer graph convolution layer to extract the features of nodes and edges. Use the graph convolution layer to encode the graph structure data. Generate high-dimensional representations of nodes and edges. Use historical data to train the GCN model and optimize the representation of nodes and edges.

[0085] S203. Determine the Transformer encoder, use the GCN output as input, design a reward function, and give feedback based on system performance indicators or fault recovery conditions. In this step, it can specifically include: using a self-attention mechanism to capture long-distance dependencies. Define a multi-head attention layer to enhance feature representation capabilities. Use a multi-layer Transformer encoder for further encoding. Add position encoding to retain the position information of the graph structure. Use the encoded graph structure data as the state of the MDP, including node features, edge features, and global system status. Define system adjustment configuration parameters and restart services. Construct an action space that includes all possible operations. Design a reward function and give feedback based on system performance indicators and fault recovery conditions. Reward signals include performance improvement and fault elimination.

[0086] S204, select a predetermined DQN algorithm, initialize the agent's policy network and value network, calculate the reward signal, and optimize the policy network and value network, wherein the reward signal includes performance improvement and / or fault elimination. In this step, the predetermined DQN algorithm may be an algorithm selected based on pre-set indicators. The agent interacts with the environment and collects training data. Calculate the reward signal and optimize the policy network and value network. Use the experience replay buffer (Replay Buffer) to store interaction data. Update network parameters based on small batch samples. The agent identifies faults based on the system status. Use the trained model to predict the fault location. The agent selects the optimal repair strategy. Automatically perform repair operations, such as adjusting configurations and restarting services. Verify the decision-making effect of the agent in a test environment. Optimize the agent's strategy based on the test results.

[0087] After S201 to S204, the training of the intelligent agent is completed. Specifically, locating the fault according to the trained intelligent agent in S5 includes:

[0088] The intelligent agent receives and encodes the system's operating data in real time, and automatically adjusts the operating parameters of the virtual machine and performs fault repair operations based on the current system status and learned strategies.

[0089] In the present invention, once the intelligent agent TRG is trained, the trained TRG is deployed to the actual integrated test platform. The TRG intelligent agent receives the system's operating data in real time and encodes it. According to the current system status and the learned strategy, the intelligent agent TRG automatically adjusts the operating parameters of the virtual machine and performs fault repair operations. The online monitoring and diagnosis of the integrated test platform is realized, the stability and security of the system are improved, and the dependence on manual intervention is reduced.

[0090] It should be noted that, in the present invention, the agent refers to TRG, the agent, the agent TRG, and TRG all have the same meaning and can be replaced with each other.

[0091] In order to more clearly describe the fault location method of the present invention, an example of implementing the method of the present invention using Python language is given below. Figure 3 As shown in the figure, the implementation process of this example includes the steps of graph structure modeling, graph neural network encoding, reinforcement learning modeling and agent training:

[0092] S301, constructing a system graph. This step may include two steps: encoding the system state as attributes of nodes and edges, and constructing an adjacency matrix to represent the dependency relationship of the graph. In this step, the functions of the above method embodiments S1 and S2 are implemented, that is, the topological structure of the system and the dependency relationship between virtual machines are represented as a graph structure, and the graph convolutional network GCN is used to encode the graph structure data to capture the relationship between nodes.

[0093] S302, define a reinforcement learning agent. This step uses the graph neural network encoding GCN to perform reinforcement learning modeling. This step implements the function of S3 in the above method embodiment, that is, using the encoded graph structure data as the environment state, defining a Markov decision process MDP, and forming a reinforcement learning agent.

[0094] S303, training the agent. During the training process, a maximum number of training times can be pre-set, that is, the following steps are repeated until the maximum number of cycles is reached:

[0095] Get the current system status;

[0096] Coding system status;

[0097] Get the adjacency matrix;

[0098] Choose actions based on strategy;

[0099] Perform actions and observe environmental feedback;

[0100] Encode the next system state;

[0101] Calculate TD error;

[0102] Update the parameters.

[0103] In the implementation of the present invention, TD error, also known as temporal difference error, is used to measure the difference between the predicted value and the actual value. In the training process of reinforcement learning, the role of TD error is to reduce this error by continuously adjusting the parameters of the algorithm, so that the prediction effect of the algorithm is getting better and better. As the training progresses, the trend of TD error should be gradually decreasing, which indicates that the difference between the predicted value and the actual value is gradually shrinking.

[0104] S304, online monitoring and diagnosis. After the above S301 to S303, the training of the agent TRG is completed, and the fault location can be performed based on the agent. Specifically, the following steps may be included:

[0105] Get the current system status;

[0106] Coding system status;

[0107] Get the adjacency matrix;

[0108] Choose actions based on strategy;

[0109] Perform actions and observe environmental feedback;

[0110] If abnormal behavior is found, fault location and automatic repair are carried out.

[0111] The above fault location process implemented in Python language realizes a system monitoring based on graph neural network and reinforcement learning. The specific code implementation can be: First, the system components and their dependencies are constructed into a graph structure through the SystemGraph class, and the system status is encoded. Then the GCN class is used to process the graph data to learn and extract features. The reinforcement learning agent RLAgent uses these features to predict and select the best action. The whole process trains the agent through the train_agent function, and implements online monitoring and fault handling in the monitor_and_diagnose function. This framework aims to support automated fault location through intelligent decision support systems, although specific environmental interaction functions such as state acquisition and action execution need to be implemented according to the actual application scenario.

[0112] In the integrated test platform, this method implements a system that uses reinforcement graph neural network (TRG) for online monitoring, fault location and automatic repair. First, the topology and component dependencies of the system are represented as a graph structure through graph structure modeling. Then, the graph structure data is encoded using graph neural network (GCN) to capture the relationship between nodes and the operating status of the system. Next, through reinforcement learning modeling and agent training, the ability of the agent to make appropriate decisions under different system states is realized. Finally, through online monitoring and diagnosis, the agent can perform automatic repair operations based on the current system state and the learned strategy. Through the combination of reinforcement learning agent and graph neural network, the system can perform real-time abnormal behavior analysis and fault location, improving the system's automatic fault location capability. Based on the learned strategy, the agent can automatically adjust the operating parameters and perform fault repair operations, reducing the need for manual intervention and improving the reliability and stability of the system.

[0113] Compared with traditional monitoring and diagnosis technologies, the improvements of this example are: using graph neural networks to encode the system's topology and component dependencies can more accurately capture the relationship between nodes and the system's operating status. Introducing reinforcement learning agents for decision-making and optimization can automatically learn the optimal monitoring and repair strategies, improving the system's automation level. Online monitoring and diagnosis can detect abnormal behavior in a timely manner and locate and automatically repair faults, reducing the impact of faults on the system and the workload of maintenance personnel.

[0114] Use as Figure 3 The fault location method implemented in Python as shown in the figure uses, for example, a data set containing a graph structure of a simulated communication network fault. The specific process data is as follows:

[0115] 1. Dataset size: There are 1,000 samples in total, 800 of which are used for training and 200 for validation.

[0116] 2. Number of nodes: Each sample contains 50 network device nodes.

[0117] 3. Number of edges: Each sample contains about 200 connection edges between devices. Node characteristics: Each node has a 3-dimensional vector containing the performance indicators of the device (such as CPU usage, memory usage, latency). Edge characteristics: Each edge has a 2-dimensional vector containing the bandwidth and transmission rate of the connection.

[0118] 4. Design a TRG model, including a 3-layer GCN as a GCN encoder and a DQN agent using DQN. During the training process, adjust the hyperparameters such as learning rate and number of neural network layers.

[0119] During the training process, try different learning rates, such as 0.001, 0.01, 0.1, etc. Suppose you find that a learning rate of 0.01 can make the model converge better.

[0120] On the validation set, the following evaluation indicators were obtained:

[0121] Accuracy: 0.93

[0122] Recall: 0.91

[0123] Precision: 0.92;

[0124] F1-score: 0.915

[0125] Among them, the full name of F1 score is F1 Score, which is a reference indicator value after the deep learning training reaches a conclusion.

[0126] In order to realize automatic fault location, the above method can be used to realize its function well under the integrated testing tool.

[0127] The fault location method described in the present invention realizes automated network fault detection and repair through TRG and DQN methods. Through structure-aware fault detection and reinforcement learning optimization strategies, the code improves the accuracy of fault detection and the degree of automation of network repair, providing a more intelligent and efficient solution for network fault location in the integrated test platform. By encoding the network topology structure through the TRG model, abnormal behaviors and faults in the network can be automatically detected, reducing dependence on manual intervention and improving the accuracy and intelligence level of fault detection. Based on the decision-making and optimization capabilities of the DQN agent, the model can perform automatic repair operations according to the learned strategies, improving the automated repair capabilities of network faults, and reducing the impact of faults on the system and the workload of maintenance personnel.

[0128] Compared with traditional network fault detection and repair technologies, the improvement of this method is that the network topology is encoded through graph neural networks, which can more accurately capture the relationship between nodes and the operating status of the system, and improve the accuracy of fault detection. The introduction of reinforcement learning agents for decision-making and optimization can automatically learn the optimal network repair strategy, improving the automation of network fault repair. A larger data set is used for training and evaluation, which improves the generalization ability and robustness of the model.

[0129] In the second aspect, based on the same inventive concept, another specific embodiment of the present invention discloses a fault location device, such as Figure 4 As shown, including:

[0130] A graph structure building module 401 is configured to represent the topology of the system and the dependency relationship between virtual machines as a graph structure;

[0131] An encoding module 402 is configured to encode the graph structure data using a graph convolutional network (GCN) to capture the relationship between nodes;

[0132] A construction module 403 is configured to use the encoded graph structure data as an environment state, define a Markov decision process MDP, and form a reinforcement learning agent;

[0133] A training module 404 is configured to train the agent through a reinforcement learning algorithm DQN;

[0134] The positioning module 405 is configured to locate the fault according to the trained intelligent agent.

[0135] As an optional example, the graph structure building module 401 is further configured to:

[0136] Each virtual machine is a node in the graph. The node attributes contain the performance indicators and log information of the virtual machine. The dependencies between virtual machines are represented by edges.

[0137] As an optional example, the encoding module 402 is further configured to encode the running state of the virtual machine through multi-layer GCN and obtain representations of nodes and edges to represent the current state of the system.

[0138] As an optional example, the construction module 403 is further configured to determine how to adjust the operating parameters of the virtual machine and perform fault recovery operations according to the current system state and the learned policy.

[0139] As an optional example, the training module 404 is further configured to:

[0140] The simulation environment provides different system states and failure conditions;

[0141] The agent interacts with the simulated environment and adjusts its strategy based on reward signals to achieve optimal monitoring and diagnosis effects.

[0142] As an optional example, the training module 404 is further configured to:

[0143] Collect data from system logs, performance monitoring tools, or network traffic analysis tools;

[0144] A graph neural network is constructed through the GCN, and a multi-layer graph convolution layer is defined to extract the features of nodes and edges;

[0145] Determine the Transformer encoder, use the GCN output as input, design a reward function, and give feedback based on system performance indicators or fault recovery;

[0146] A predetermined DQN algorithm is selected, a policy network and a value network of the agent are initialized, a reward signal is calculated, and the policy network and the value network are optimized, wherein the reward signal includes performance improvement and / or fault elimination.

[0147] As an optional example, the positioning module 405 is further configured to:

[0148] Receive and encode system operation data in real time, and automatically adjust the operating parameters of the virtual machine and perform fault repair operations based on the current system status and learned strategies.

[0149] It should be noted that the graph structure building module 401 in the present device embodiment can implement all the method steps of the above method embodiment S1 and achieve the same technical effect, and the similarities will not be repeated here;

[0150] It should be noted that the encoding module 402 in the present device embodiment can implement all the method steps of the above method embodiment S2 and achieve the same technical effect, and the similarities will not be repeated here;

[0151] It should be noted that the construction module 403 in the present device embodiment can implement all the method steps of the above method embodiment S3 and achieve the same technical effect, and the similarities will not be repeated here.

[0152] It should be noted that the training module 404 in the present device embodiment can implement all the method steps of the above method embodiment S4 and achieve the same technical effect, and the similarities will not be repeated here;

[0153] It should be noted that the positioning module 405 in the embodiment of the present device can implement all the method steps of the above method embodiment S5 and achieve the same technical effect, and the similarities will not be repeated here.

[0154] It should be noted that the present device embodiment can implement all the method steps of the above method embodiment, achieve the same technical effects, and solve the same technical problems, and the similarities will not be repeated here.

[0155] In a third aspect, the present invention provides an electronic device, such as Figure 5 As shown, it includes: a memory 502 and one or more processors 501.

[0156] One or more application programs are stored in the memory 502 , and the one or more application programs are suitable for being executed by the one or more processors 501 to implement the above-mentioned fault location method.

[0157] like Figure 5 As shown, the processor 501 is connected to the memory 503 , such as via a bus 504 .

[0158] This structure does not constitute a limitation on the embodiments of the present invention.

[0159] Processor 501 may be a CPU, a general purpose processor, a DSP, an ASIC, an FPGA or other programmable logic device, a transistor logic device, a hardware component or any combination thereof. It may implement or execute various exemplary logic blocks, modules and circuits described in conjunction with the disclosure of the present invention. Processor 501 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0160] The bus 504 may include a path to transmit information between the above components. The bus 504 may be a PCI bus or an EISA bus, etc. The bus 504 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 5 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0161] The memory 502 may be a ROM or other type of static storage device that can store static information and instructions, a RAM or other type of dynamic storage device that can store information and instructions, or an EEPROM, a CD-ROM or other optical disk storage, an optical disk storage (including a compressed optical disk, a laser disk, an optical disk, a digital versatile disk, a Blu-ray disk, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited to these.

[0162] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program can be loaded and executed by a processor to implement the fault location method described in the first aspect.

[0163] The applicant of the present invention has made a detailed explanation and description of the implementation examples of the present invention in conjunction with the drawings in the specification. However, those skilled in the art should understand that the above implementation examples are only preferred implementation schemes of the present invention, and the detailed description is only to help readers better understand the spirit of the present invention, but not to limit the scope of protection of the present invention. On the contrary, any improvements or modifications based on the inventive spirit of the present invention should fall within the scope of protection of the present invention.

[0164] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the embodiments of the present invention, rather than to limit them. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions recorded in the above embodiments, or replace some of the technical features therein by equivalents; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention. Any changes or replacements that can be easily thought of by those skilled in the art within the technical scope disclosed in the present invention should be covered within the protection scope of the present invention.

Claims

1. A fault location method, characterized in that: include: S1, represents the system topology and the dependencies between virtual machines as a graph structure; S2, using a graph convolutional network GCN to encode the graph structure data to capture the relationship between nodes; S3, taking the encoded graph structure data as the environment state, defines a Markov decision process MDP to form a reinforcement learning agent; S4, training the agent through a reinforcement learning algorithm DQN; S5, locate the fault based on the trained agent.

2. A fault location method according to claim 1, characterized in that: In S1, the system topology and the dependencies between virtual machines are represented as a graph structure including: Each virtual machine is a node in the graph, and the node attributes include the performance indicators and log information of the virtual machine. The dependency relationship between the virtual machines is represented by the edges.

3. A fault location method according to claim 2, characterized in that: In S2, a graph convolutional network GCN is used to encode the graph structure data to capture the relationship between nodes, including: The running state of the virtual machine is encoded through multi-layer GCN, and the representation of nodes and edges is obtained to characterize the current state of the system.

4. A fault location method according to claim 1, characterized in that: In S3, the encoded graph structure data is used as the environment state, and a Markov decision process MDP is defined to form a reinforcement learning agent including: The agent decides how to adjust the operating parameters of the virtual machine and perform fault repair operations based on the current system status and the learned strategies.

5. A fault location method according to claim 1, characterized in that: In S4, training the agent by using the reinforcement learning algorithm DQN includes: The simulation environment provides different system states and failure conditions; The agent interacts with the simulated environment and adjusts its strategy based on reward signals to achieve optimal monitoring and diagnosis effects.

6. A fault location method according to claim 5, characterized in that: The agent interacts with the simulated environment and adjusts the strategy according to the reward signal to achieve the best monitoring and diagnosis effect, including: Collect data from system logs, performance monitoring tools, or network traffic analysis tools; A graph neural network is constructed through the GCN, and a multi-layer graph convolution layer is defined to extract the features of nodes and edges; Determine the Transformer encoder, use the GCN output as input, design a reward function, and give feedback based on system performance indicators or fault recovery; A predetermined DQN algorithm is selected, a policy network and a value network of the agent are initialized, a reward signal is calculated, and the policy network and the value network are optimized, wherein the reward signal includes performance improvement and / or fault elimination.

7. A fault location method according to claim 1, characterized in that: The fault location according to the trained intelligent agent in S5 includes: The intelligent agent receives and encodes the system's operating data in real time, and automatically adjusts the operating parameters of the virtual machine and performs fault repair operations according to the current system status and the learned strategies.

8. A fault location device, characterized in that: Used to implement the fault location method according to any one of claims 1 to 7, the device comprises: A graph structure building module configured to represent the topology of the system and the dependency relationship between virtual machines as a graph structure; An encoding module is configured to encode the graph structure data using a graph convolutional network (GCN) to capture the relationship between nodes; A building module is configured to use the encoded graph structure data as the environment state, define a Markov decision process MDP, and form a reinforcement learning agent; A training module, configured to train the agent through a reinforcement learning algorithm DQN; The localization module is configured to locate the fault according to the trained agent.

9. A fault detection device, characterized in that: including a memory, a processor and a user interface; The memory is used to store computer programs; The user interface is used to interact with the user; The processor is configured to read the computer program in the memory, and when the processor executes the computer program, implements: Represent the system topology and dependencies between virtual machines as a graph structure; Encode the graph structure data using a graph convolutional network (GCN) to capture the relationship between nodes; The encoded graph structure data is used as the environment state, and a Markov decision process MDP is defined to form a reinforcement learning agent; Training the agent through a reinforcement learning algorithm DQN; The fault is located based on the trained agent.

10. A processor-readable storage medium, characterized in that: The processor-readable storage medium stores a computer program, and when the processor executes the computer program, the fault handling method according to any one of claims 1 to 7 is implemented.

Citation Information

Cited By

  • Equipment fault detection data analysis method and system for digital factory

    CN119829957A

  • A method and system for analyzing data of equipment fault detection in a digital factory

    CN119829957B