Distributed system network anomaly detection and recovery method based on VAE and DQN

Abstract

The invention discloses a distributed system network anomaly detection and recovery method based on VAE and DQN. The method comprises a fault detection core module and a path recovery core module. The method comprises the following steps: firstly, constructing a reference model of network behaviors by learning probability characteristics of network equipment and connection which run normally through VAE, realizing real-time monitoring of a network state, and accurately identifying an anomaly and positioning a fault position; and secondly, after an anomaly is detected, the DQN intelligently selects an optimal recovery path through a reinforcement learning strategy according to a real-time network state and historical data, and optimizes a decision process through a target network and an experience playback mechanism. According to the method, the fault detection precision and recovery speed of the distributed system in a complex environment are remarkably improved, the high stability and continuous operation capability of the system are ensured, and the method is suitable for the fields of intelligent transportation, industrial control, intelligent power grids and the like with extremely high requirements on high real-time performance and high reliability.

Description

Technical Field

[0001] The present invention belongs to the field of distributed computing, and specifically is a distributed system network anomaly detection and recovery method of VAE (Variational Autoencoder) and DQN (Deep Q Network). Background Art

[0002] With the rapid development of emerging technologies such as the Internet of Things, smart transportation, and smart grids, distributed systems have become an important part of network communications. Such systems achieve real-time data collection and processing through widely distributed smart devices, sensors, and nodes, greatly improving the efficiency of information transmission and computing. However, with the expansion of system scale, the sharp increase in the number of devices, and the diversity and heterogeneity of the system, the network structure of distributed systems has become more and more complex. Since these devices are distributed in different geographical areas, the network connection between nodes may be affected by factors such as network congestion, hardware failure, and bad weather, resulting in unstable or interrupted network connections. These network connection or node failures often lead to service interruptions, data loss, and even affect the overall performance of the system. Therefore, in order to ensure the reliability and stability of distributed systems, real-time monitoring and fault detection of network status are particularly important.

[0003] Traditional network detection methods mainly rely on rule models or static threshold judgments, and judge the health status of network connections by setting a series of rules or thresholds. For example, in a traditional IP network, when a router detects a network failure, the following processing flow is usually triggered: First, the router that detects the failure propagates the failure information to other routers in the network, notifying the entire network of the failed path or node. The routers that receive the failure information will recalculate their respective routing entries based on the new network topology to ensure that the data flow can continue to be transmitted by bypassing the failed network connection. When recalculating routing entries, routers usually use the shortest path algorithm (such as the Dijkstra algorithm or the Bellman-Ford algorithm) to update the routing table. The Dijkstra algorithm regards the network as a weighted graph, in which nodes represent routers, edges represent network paths, and weights usually represent the cost of the path (such as delay, bandwidth, or packet loss rate). When a network path fails, the router removes the path and reruns the Dijkstra algorithm to find the shortest path to each destination node. The Bellman-Ford algorithm uses an iterative method, in which each router gradually updates the routing entries based on the routing information of its neighbors until the routing table of the entire network converges. Although these two algorithms can effectively update the routing table, in large networks, the recalculation process may consume a lot of resources, resulting in slow fault recovery. This passive fault handling method is difficult to meet the real-time and high-efficiency requirements of modern distributed systems.

[0004] For example, in intelligent transportation systems, distributed devices such as roadside sensors, traffic cameras, and traffic lights are spread everywhere, providing data support for urban traffic flow monitoring, signal control, and accident warning. These devices are interconnected through the network to collect and transmit data in real time. However, with the surge in the amount of urban traffic data and the increase in the number of devices, the system faces severe challenges in reliability and real-time performance. Distributed devices in the transportation system are located in different geographical areas, and network connections are often affected by changes in the external environment, such as network congestion, equipment aging, sudden hardware failures, and even bad weather, which can cause instability or even interruption of network connections. In this environment, traditional fault detection and recovery methods mainly rely on passive detection methods, which will cause delays in the recovery process and make it difficult to meet the real-time requirements of the transportation system. Secondly, traditional routing updates are usually based on static thresholds or fixed algorithms, ignoring the dynamic and complex network status in the transportation network. Therefore, the recovery path is often not fully optimized, which may increase the network burden. Summary of the invention

[0005] The present invention proposes a distributed system network anomaly detection and recovery method based on variational autoencoder and deep Q network, which uses VAE to monitor network connections in real time and performs intelligent path recovery through DQN to achieve fast and accurate network fault detection and efficient recovery, thereby improving the reliability and stability of the distributed system.

[0006] The technical solution to achieve the purpose of the present invention is: a distributed system network anomaly detection and recovery method based on VAE and DQN, comprising the following steps:

[0007] Step 1: Construct a distributed system network equipment operation parameter dataset and use the parameter dataset to train a network fault detection model based on variational autoencoder;

[0008] Step 2: Use the state data of the distributed system to train the DQN model so that the DQN model can achieve intelligent path selection;

[0009] Step 3: Collect the operating parameter data of the distributed system network equipment in real time, and use the trained network fault detection model based on variational autoencoder to detect whether the parameter data is abnormal. If there is an abnormality, delete the faulty node or path in the distributed system, and use the trained DQN model to select the path for operation.

[0010] Preferably, a distributed system network device operation parameter data set is constructed, and a specific method for using the parameter data set to train a network fault detection model based on a variational autoencoder is as follows:

[0011] Step 1.1: Collect the operating parameters of the devices that are operating normally in the distributed system network and build a data set;

[0012] Step 1.2: Use the dataset to train a network fault detection model based on a variational autoencoder to generate latent space and reconstruct data;

[0013] Step 1.3: Minimize the reconstruction error to obtain the trained network fault detection model and set the judgment threshold of the trained network fault detection model to distinguish the device status.

[0014] Preferably, the network fault detection model based on variational autoencoder includes an encoder and a decoder, wherein the encoder is used to map the number of operating parameters of normal equipment to a low-dimensional latent space, and output a mean vector μ and a standard vector σ to represent the probability distribution of the latent space, and the latent space sampling generates a latent vector ); The decoder is used to decode and reconstruct the latent vector z, and reconstruct the original data through a hidden layer.

[0015] Preferably, the specific method of minimizing the reconstruction error to obtain a trained network fault detection model is:

[0016] Calculate the operating parameter data in the data set and reconstructing data The loss function between and is used, and back propagation is used to adjust the parameters of the network fault detection model based on the variational autoencoder so that the reconstructed data is close to the original input. That is, when the loss function is less than the training threshold, the training is stopped.

[0017] Preferably, the loss function is specifically:

[0018] L

[0019] In the formula, N is the total amount of operating parameter data, d is the dimension of the data, is the VAE output mean vector, is the standard deviation vector of VAE output.

[0020] Preferably, the specific method of training the DQN model using the state data of the distributed system is:

[0021] Step 2.1: Express the state vector of the distributed system at time t as In each state, the action set A adopted is defined as the path selection set, denoted as ;

[0022] Step 2.2: Initialize the estimated Q network and the target Q network, and copy the parameters of the estimated Q network to the target Q network;

[0023] Step 2.3: Input the state vector to the estimated Q network using -greedy strategy selects actions from a set of actions and receive instant rewards and the next state , and transfer the state sample Deposit into experience pool middle;

[0024] Step 2.4: Replay from the Experience Pool Randomly sample a batch of samples and calculate the current target The target value of the network parameter update, the loss function is calculated based on the estimated Q network and the target Q network ;

[0025] Use gradient descent to update the estimated Q network parameters In order to minimize the loss function, the parameters of the estimated Q network are copied to the target Q network every certain number of steps, and the process returns to step 2.3. When the preset maximum number of training steps is reached, the training is stopped to obtain the trained DQN model.

[0026] Compared with the prior art, the present invention has the following significant advantages:

[0027] (1) High fault detection accuracy: The variational autoencoder VAE in the present invention significantly improves the sensitivity and accuracy of detection through probability modeling and automatic learning of judgment criteria by reconstruction error.

[0028] (2) Path optimization intelligence: The present invention implements path recovery through DQN, and uses the state, action and reward mechanism to dynamically select the optimal path, effectively bypass the fault area, quickly restore network connection, and significantly shorten the recovery time.

[0029] (3) Highly adaptable to complex network environments: The VAE and DQN models in the present invention can be continuously updated and optimized based on new network data, so that the system can operate effectively in a complex and dynamic distributed network environment, enhancing the adaptability of the network.

[0030] (4) Personalized adaptation to edge node requirements: Through personalized training and model optimization, the present invention provides customized detection and recovery solutions for the specific requirements of different edge nodes, thereby improving the flexibility and reliability of the system.

[0031] The present invention is described in further detail below in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] FIG1 is a network fault detection model structure based on a variational autoencoder according to the present invention.

[0033] FIG2 is a flowchart of network fault detection and recovery based on VAE and deep Q network of the present invention. DETAILED DESCRIPTION

[0034] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments.

[0035] The typical application scenario of the present invention is fault detection and recovery in edge computing networks, especially for real-time detection and recovery tasks of network faults and equipment anomalies in distributed systems. The method is mainly used in network systems that require high real-time performance and high reliability, such as intelligent transportation systems, industrial Internet of Things, smart grids, and data center networks. By monitoring the network status and equipment operating status, rapid fault detection and intelligent recovery can be achieved to ensure the stability and continuity of the network in complex environments.

[0036] For the above application scenarios, on the one hand, it is necessary to have fault detection capabilities to identify the type and specific location of the fault by continuously monitoring the network status. In a distributed network, the operating status of network connections and devices may be abnormal due to various reasons. Therefore, it is necessary to monitor indicators such as bandwidth, latency, and packet loss rate to promptly detect abnormal behaviors that deviate from the benchmark. When a fault occurs, it should be able to accurately identify the type and specific location of the fault to provide accurate positioning information for subsequent recovery operations. On the other hand, it is necessary to select a suitable recovery path after detecting the fault to ensure rapid recovery and stable operation of the network. Since fault recovery involves rerouting the data flow, it is necessary to comprehensively consider multiple factors, such as network topology, current network status, bandwidth and latency requirements, etc. In the process of selecting the recovery path, it is necessary not only to select a path that bypasses the fault, but also to optimize the transmission performance of the path to ensure that data can continue to be transmitted with minimal delay and packet loss.

[0037] The present invention proposes a distributed system network anomaly detection and recovery method based on VAE and DQN. First, VAE builds a baseline model of network behavior by learning the probabilistic characteristics of normal device operation status. When the distributed system network deviates from the normal distribution, VAE identifies the anomaly by calculating the reconstruction error (MSE). After the fault is detected, DQN dynamically optimizes path selection and resource scheduling based on real-time network status and historical data through reinforcement learning strategies, and intelligently implements rapid fault recovery and adaptive adjustment. The present invention cleverly combines the efficient anomaly detection capability of VAE and the intelligent decision-making advantages of DQN, ensuring the real-time response capability, stability and fault recovery efficiency of the distributed system network in complex environments, and significantly improving the robustness and resilience of the network, including the following steps:

[0038] Step 1: Construct a distributed system network equipment operating parameter data set and use the data set to train a network fault detection model based on a variational autoencoder. The specific steps are as follows:

[0039] Step 1.1: Collect normal equipment operation parameters and build a data set.

[0040] Collect the operating parameters of the normally operating devices in the distributed system network to form a basic data set. The collected data includes delay τ, packet loss rate P loss , load U, bandwidth B, throughput T, request response time R t 、CPU utilization U cpu , Error log frequency F err and power consumption P parameter.

[0041] Step 1.2: Use the dataset to train a network fault detection model based on a variational autoencoder to generate latent space and reconstruct data;

[0042] As shown in Figure 1, the operating parameter data of the normal equipment The input is to the network fault detection model based on variational autoencoder, which includes an encoder and a decoder. The encoder maps the number of operating parameters of normal equipment to a low-dimensional latent space and outputs a mean vector μ and a standard vector σ, which represent the probability distribution of the latent space. The latent space sampling generates a latent vector ), this latent vector will be used to reconstruct the data. The decoder is used to decode and reconstruct the latent vector z, and reconstruct the original data through a hidden layer .

[0043] Step 1.3: Minimize the reconstruction error to obtain the trained network fault detection model and set the judgment threshold of the trained network fault detection model to distinguish the device status.

[0044] By calculating the operating parameter data of the original normal equipment and reconstructing data The loss function between , and the back propagation is used to continuously adjust the parameters of the network fault detection model based on the variational autoencoder, so that the reconstructed data is as close to the original input as possible, that is, when the loss function is less than the training threshold, the training is stopped; where the loss function L of the network fault detection model based on the variational autoencoder is:

[0045] L

[0046] In the formula, N is the total amount of operating parameter data, d is the dimension of the data, is the VAE output mean vector, is the standard deviation vector of VAE output.

[0047] The loss function consists of two parts. The first part is the reconstruction error, and the second part is the KL divergence, which is used to measure the difference between the posterior distribution and the prior distribution.

[0048] After the model training is completed, a batch of data sets containing 20% ​​abnormal data and 80% normal equipment data sets are used as test sets to determine the appropriate error threshold δ. This threshold is set manually, and data below this error is considered normal data, while data above the threshold is considered abnormal data.

[0049] Step 2: Use the state data of the distributed system to train the DQN model so that the DQN model can achieve intelligent path selection. The main purpose of the DQN model is to achieve efficient network failure recovery by learning the state-action-reward model of network operation. The specific process is:

[0050] Step 2.1: Denote the state vector of the distributed system at time t as s t , the state vector of the distributed system at time t includes: network connection status, network bandwidth, network delay, packet loss rate and network load, that is, the state vector Expressed as , where is the network bandwidth at time t, is the network delay at time t, is the packet loss rate, is the network load at time t;

[0051] In each state, the action set A adopted is defined as the path selection set, denoted as .

[0052] Step 2.2: Initialize the estimated Q network and the target Q network , including setting the initial state And initialize the experience replay pool , Exploration Rate , learning rate , Discount Factor And experience replay pool . The Q network will be estimated The parameters are copied to the target Q network ;

[0053] Step 2.3: Input the state vector to the estimated Q network using -greedy strategy selection action and receive instant rewards and the next state , and transfer the state sample Deposit into experience pool middle;

[0054] In order to evaluate the quality of each action, a reward function r is designed t, the goal is to select the path with maximum bandwidth, minimum delay and lowest packet loss rate. The reward function at time t is defined as:

[0055]

[0056] Among them, α is the bandwidth weight factor, β is the delay weight factor, is the packet loss rate weight factor.

[0057] Step 2.4: Replay from the Experience Pool Randomly sample a batch from Samples of , calculate the current target The target value of the network parameter update, specifically, in state S t Execute an action After that, calculate the target Q value corresponding to the target Q network , the specific calculation formula is:

[0058]

[0059] In the formula, are the parameters of the target Q network, is the weight factor.

[0060] The Q value represents the expected cumulative reward that the agent can obtain by taking a specific action and following a certain strategy in a given state.

[0061] Calculate the loss function based on the estimated Q network and the target Q network , the specific formula is:

[0062]

[0063] Use gradient descent to update the estimated Q network parameters To minimize the loss function, at certain steps, copy the parameters of the estimated Q network to the target Q network and return to step 2.3; in addition, the target Network and Estimation The network structure is the same, but its parameters are updated less frequently, thus stabilizing Value calculation to avoid unstable training caused by frequent changes.

[0064] When the preset maximum number of training steps is reached, the training is stopped and the trained DQN model is obtained.

[0065] Through the above training process, the deep Q network gradually learns the optimal path selection strategy, making the Q value close to the actual cumulative reward. Ultimately, the DQN model can dynamically select efficient paths in complex network environments to achieve fast fault recovery and optimize network performance, meeting the application requirements of high real-time and high reliability.

[0066] Step 3: Collect the operating parameter data of the distributed system network equipment in real time, and use the trained network fault detection model based on variational autoencoder to detect whether the parameter data is abnormal. If there is an abnormality, delete the faulty node or path in the distributed system, and use the trained DQN model to select the path for operation. The specific steps are:

[0067] Step 3.1: Use the trained variational autoencoder to continuously collect device parameters and detect anomalies.

[0068] Continuously collect device operating parameters from edge nodes , including the delay τ t , Packet loss rate P loss , load U, bandwidth B, throughput T, request response time R t 、CPU utilization U cpu , Error log frequency F err , Packet loss rate P loss And power consumption P, these numbers are dynamically input into the VAE model. The VAE network will reconstruct the data for ', once the VAE reconstructs the data With the original data When the MSE variance δ exceeds the set threshold, it is determined that the network connection is abnormal and then enters the fault recovery phase.

[0069] Step 3.2: Fault recovery and path optimization based on DQN model.

[0070] After entering the fault recovery phase, DQN is based on the state Network Status Generate a set of optional recovery paths. The set of optional recovery paths does not include a failed node or path in a distributed system network device;

[0071] Select actions using the trained DQN model .in Represents the different recovery paths that can be selected in the current state of the distributed system network. After executing the selected recovery path, the recovery results will be monitored and the instant recovery reward R will be recorded. t , build an experience replay mechanism to store state transition samples , and randomly extract samples from the experience pool to update the Q network parameters to re-optimize the recovery strategy.

[0072] After the fault is recovered, the network status is continuously monitored. If a new anomaly is detected, steps 3.1 to 3.2 are repeated to achieve dynamic and intelligent network fault recovery.

[0073] The present invention continuously collects device operation data from edge nodes and updates VAE and DQN in real time. When VAE detects that the reconstruction error δ exceeds the threshold, the network is determined to be abnormal and fault recovery is triggered. Subsequently, DQN selects the optimal recovery path based on the current state. After the recovery is performed, the original abnormal position will be skipped and the optimal recovery path will be selected and the recovery result will be monitored. The result will be added to the experience pool to update the experience replay pool, from which samples are randomly selected to update the Q network parameters to re-optimize the recovery strategy.

[0074] The intelligent network abnormal state detection and recovery method based on VAE and DQN proposed in this invention has shown significant advantages in the application of edge computing networks. VAE achieves efficient anomaly detection by modeling the probability distribution of normal network states, can quickly identify potential faults and provide accurate positioning information; while DQN dynamically generates the optimal recovery plan through reinforcement learning, avoids faulty equipment or network paths, and selects alternative paths to avoid local faults causing the overall failure of the distributed system network.

[0075] This invention cleverly combines the high detection sensitivity of VAE and the path optimization capability of DQN, which not only significantly improves the detection accuracy and recovery efficiency, but also enhances the robustness and adaptability of the network system in complex scenarios. Compared with the traditional static rule-based detection and recovery method, this solution has outstanding advantages in real-time, reliability and adaptability, and is suitable for high-reliability application scenarios such as intelligent transportation, industrial Internet of Things, and smart grids.

Claims

1. A distributed system network anomaly detection and recovery method based on VAE and DQN, characterized in that: The following steps are involved: Step 1: Construct a distributed system network equipment operation parameter dataset and use the parameter dataset to train a network fault detection model based on variational autoencoder; Step 2: Use the state data of the distributed system to train the DQN model so that the DQN model can achieve intelligent path selection; Step 3: Collect the operating parameter data of the distributed system network equipment in real time, and use the trained network fault detection model based on variational autoencoder to detect whether the parameter data is abnormal. If there is an abnormality, delete the faulty node or path in the distributed system, and use the trained DQN model to select the path for operation.

2. According to claim 1, the distributed system network anomaly detection and recovery method based on VAE and DQN is characterized in that: The specific method of constructing a distributed system network equipment operation parameter data set and using the parameter data set to train a network fault detection model based on a variational autoencoder is as follows: Step 1.1: Collect the operating parameters of the devices that are operating normally in the distributed system network and build a data set; Step 1.2: Use the dataset to train a network fault detection model based on a variational autoencoder to generate latent space and reconstruct data; Step 1.3: Minimize the reconstruction error to obtain the trained network fault detection model and set the judgment threshold of the trained network fault detection model to distinguish the device status.

3. The distributed system network anomaly detection and recovery method based on VAE and DQN according to claim 2 is characterized in that: The network fault detection model based on variational autoencoder includes an encoder and a decoder. The encoder is used to map the operating parameter number of normal equipment to a low-dimensional latent space, and output a mean vector μ and a standard vector σ to represent the probability distribution of the latent space. The latent space sampling generates a latent vector ), ) indicates that z is sampled from a normal distribution, the mean of the normal distribution is μ, and the standard deviation is σ; the decoder is used to decode and reconstruct the latent vector z, and reconstruct the original data through a hidden layer.

4. The distributed system network anomaly detection and recovery method based on VAE and DQN according to claim 2 is characterized in that: The specific method of minimizing the reconstruction error to obtain a trained network fault detection model is: Calculate the operating parameter data in the data set and reconstructing data The loss function between and is used, and back propagation is used to adjust the parameters of the network fault detection model based on the variational autoencoder so that the reconstructed data is close to the original input. That is, when the loss function is less than the training threshold, the training is stopped.

5. The distributed system network anomaly detection and recovery method based on VAE and DQN according to claim 2 is characterized in that: The loss function is specifically: L In the formula, N is the total amount of operating parameter data, d is the dimension of the data, is the VAE output mean vector, is the standard deviation vector of VAE output.

6. The distributed system network anomaly detection and recovery method based on VAE and DQN according to claim 1 is characterized in that: The specific method of training the DQN model using the state data of the distributed system is: Step 2.1: Express the state vector of the distributed system at time t as In each state, the action set A adopted is defined as the path selection set, denoted as ; Step 2.2: Initialize the estimated Q network and the target Q network, and copy the parameters of the estimated Q network to the target Q network; Step 2.3: Input the state vector to the estimated Q network using -greedy strategy selects actions from a set of actions and receive instant rewards and the next state , and transfer the state sample Deposit into experience pool middle; Step 2.4: Replay from the Experience Pool Randomly sample a batch of samples and calculate the current target The target value of the network parameter update, the loss function is calculated based on the estimated Q network and the target Q network ; Update the estimated Q network parameters using gradient descent In order to minimize the loss function, the parameters of the estimated Q network are copied to the target Q network every certain number of steps, and the process returns to step 2.

3. When the preset maximum number of training steps is reached, the training is stopped to obtain the trained DQN model.

7. The distributed system network anomaly detection and recovery method based on VAE and DQN according to claim 6 is characterized in that: Instant Rewards Specifically: Among them, α is the bandwidth weight factor, β is the delay weight factor, is the packet loss rate weight factor.

8. The distributed system network anomaly detection and recovery method based on VAE and DQN according to claim 6 is characterized in that: The target Q value corresponding to the target Q network , the specific calculation formula is: In the formula, are the parameters of the target Q network, is the weight factor.

9. The distributed system network anomaly detection and recovery method based on VAE and DQN according to claim 8 is characterized in that: Loss Function The specific calculation formula is: In the formula, is the number of samples in a batch, Indicates in status Next action After that, the estimated cumulative rewards that can be obtained in the future.

10. The distributed system network anomaly detection and recovery method based on VAE and DQN according to claim 1 is characterized in that: The state vector of a distributed system at time t includes: network connection status, network bandwidth, network delay, packet loss rate, and network load.

Images