Network fault trend prediction method and system based on artificial intelligence

Abstract

The invention relates to the crossing field of network technology and artificial intelligence, in particular to a network fault trend prediction method and system based on artificial intelligence, and the method comprises the steps: collecting multi-modal network data, generating a causal relationship pair through a rule engine, carrying out the processing through an improved real-time compression algorithm, and carrying out the deduplication, deletion complementation and abnormal value filtering. Distributing and screening feature weights according to a causal graph model; generating a standardized data set; constructing a twinborn environment to capture network topology changes; updating the causal graph model, simulating multi-fault coupling evolution, generating and verifying a fault causal chain through causal reasoning, and predicting a fault trend by means of an edge-cloud hierarchical model. A diffusion path is determined through path analysis, early warning logic is adjusted, the optimal disposal action is selected through reinforcement learning, the effect matching degree is calculated, and strategies and parameters are updated. According to the invention, through technologies of multi-modal data fusion, causal reasoning, hierarchical model and the like, effects of data quality improvement, accurate fault positioning, efficient early warning disposal and stable model adaptation are realized.

Description

Technical Field

[0001] This invention relates to the intersection of network technology and artificial intelligence, and in particular to a method and system for predicting network fault trends based on artificial intelligence. Background Technology

[0002] As network scale continues to expand and architectural complexity increases, network fault prediction technology is steadily developing towards intelligence. Early fault prediction relied heavily on manually preset rules and simple statistical models, capable of judging only a few known fault types and exhibiting extremely poor adaptability to complex network environments. The growth in data volume has driven the widespread application of machine learning algorithms, improving prediction accuracy by mining statistical features. However, limited by single-modal data input and sample size, it is difficult to capture multi-dimensional correlations and dynamic changes in the network. In recent years, emerging technologies such as deep learning, digital twins, and federated learning have gradually been integrated into the field of fault prediction, attempting to achieve multi-source data integration, network state reproduction, and cross-node joint training, pushing the technology towards adaptive precision. However, the overall technical system is still not mature enough, and there is still considerable room for improvement in its practicality and stability in complex scenarios.

[0003] The shortcomings of existing technologies are becoming increasingly apparent in practical deployments. Data processing lacks efficient integration and filtering methods; redundant information and outliers in multimodal data easily interfere with model training; data compression often results in the loss of key correlation information, making it difficult to balance processing efficiency and data validity. In fault evolution analysis, most methods can only identify surface fault phenomena, failing to accurately trace the root cause and propagation path. Their ability to reproduce the evolutionary logic of multiple coupled faults is weak, leading to a lack of targeted response measures. At the model optimization level, the scarcity of fault samples directly limits the model's generalization ability, and cross-node data sharing faces privacy constraints, making comprehensive model optimization difficult. Furthermore, the prediction and response stages are disconnected; the model cannot dynamically adjust based on actual response results, making it difficult to adapt to real-time changes in network topology and operational status, hindering continuous improvement in early warning accuracy and response efficiency. Summary of the Invention

[0004] The purpose of this invention is to overcome the above-mentioned problems and provide a network fault trend prediction method and system based on artificial intelligence. To achieve the above objective, this invention adopts the following technical solution:

[0005] The network fault trend prediction method based on artificial intelligence includes the following steps:

[0006] Step S1: Collect multimodal network data, generate causal relationship pairs through a rule engine, and process the data using an improved real-time compression algorithm;

[0007] Step S2: Perform deduplication, missing data completion, and outlier filtering operations on the collected and compressed data. Assign feature weights based on the causal graph model and filter features whose weights meet the preset filtering conditions to generate a standardized dataset.

[0008] Step S3: Construct a twin environment based on a standardized dataset, capture network topology changes and update the causal graph model, simulate the multi-fault coupling evolution to reproduce the fault chain logic;

[0009] Step S4: Analyze the fault evolution data using a causal reasoning algorithm, generate a fault causal chain, and verify its accuracy and completeness in a twin environment;

[0010] Step S5: Use an edge-cloud layered model to predict fault trends, determine fault propagation paths through path analysis algorithms, and adjust early warning logic.

[0011] Step S6: Select the optimal action through reinforcement learning algorithm, calculate the degree of matching between the action effect and the expected result, and update the relevant strategies and parameters;

[0012] Step S7: Generate virtual fault samples through generative adversarial networks, and optimize the model's generalization ability based on shared parameters through longitudinal federated learning.

[0013] Furthermore, in step S1, the collected multimodal network data includes network traffic data, device hardware status data, topology change data, and application layer log data. A rule engine analyzes the correlations between the multimodal data, generates causal pairs, and appends them to the collected data. An improved real-time compression algorithm is then used to process the data, and this improved real-time compression algorithm satisfies the compression ratio calculation formula:

[0014]

[0015] Where η is the real-time compression ratio; L0 is the total amount of data before compression; and L1 is the total amount of data after compression.

[0016] Further, in step S2, the collected and compressed multimodal network data undergoes deduplication, missing data completion, and outlier filtering operations. A standardization method is used to convert the data to a uniform numerical range. Based on the causal graph model, the causal correlation degree of each feature is calculated and weights are assigned. Features whose weights meet preset filtering conditions are selected to generate a standardized dataset. The formula for calculating the causal weights of features in the causal graph model is:

[0017]

[0018] Where, ω i C represents the causal weight of the i-th feature; i,∑ The sum of the causal correlations between the i-th feature and all other features; m fThis represents the total number of features in the valid feature set determined based on the strength of causal relationships between features.

[0019] Furthermore, in step S3, a twin environment synchronized with the real network in time and space is constructed on the generated standardized dataset. This environment captures changes in the real network topology in real time, synchronously updates the node connection relationships and corresponding weights of the causal graph model, simulates the multi-fault coupling evolution process, and reproduces the fault chain reaction logic. Topology change capture is achieved by real-time monitoring of network node connection status and link parameters. Node updates in the causal graph model follow the priority of node correlation, and the node correlation calculation formula is:

[0020]

[0021] Where, γ ij Let T be the degree of association between the i-th node and the j-th node; ij denoted as , where is the real-time interaction frequency between the i-th node and the j-th node; m is the total number of nodes in the causal graph model.

[0022] Furthermore, in step S4, the fault evolution data output by the twin environment is analyzed using a causal reasoning algorithm to generate a complete fault causal chain from the root cause to the surface phenomenon. The starting conditions of the fault causal chain are reproduced in the twin environment to verify the accuracy and completeness of the fault causal chain. The causal reasoning algorithm traces the fault triggering source layer by layer based on the node association relationship and weight of the causal graph model to form a continuous and unbroken causal sequence. The verification process is completed by comparing the results reproduced in the twin environment with the actual fault phenomenon.

[0023] Furthermore, in step S5, a lightweight model is deployed at the edge to process real-time data and output early warning information, while a spatiotemporal model is deployed at the cloud to fuse real-time data transmitted from multiple edge nodes to predict the global fault trend. A path analysis algorithm is used to determine the fault propagation path and dynamically adjust the early warning logic. The probability calculation formula for the fault propagation path is:

[0024]

[0025] Among them, P pq γ represents the probability that a fault propagates from node p to node q; pq Let λ be the degree of association between the p-th node and the q-th node; p μ represents the probability of failure at the p-th node. q is the vulnerability parameter of the q-th node, which is a parameter characterizing the node's ability to resist fault propagation and is determined by the node's hardware configuration, operating status, and redundancy design; k is the number of downstream associated nodes corresponding to the p-th node; j is a summation variable used to distinguish downstream nodes.

[0026] Furthermore, in step S6, a reinforcement learning algorithm is used to select the optimal action based on the current fault state, compare the action effect with the expected result of the fault causal chain to calculate the matching degree, and transmit the matching result to the twin environment and model optimization module to update relevant strategies and parameters. The reinforcement learning algorithm selects suitable actions from the preset action library based on the fault type and the current network state. The matching degree is calculated based on the fit between the fault mitigation effect after action and the expected effect. The matching result is used simultaneously to adjust the twin environment parameters and model training strategy.

[0027] Further, in step S7, virtual fault samples are generated based on the real data distribution using a generative adversarial network (GAN). A longitudinal federated learning approach is employed to share virtual sample parameters among nodes. The virtual samples and shared parameters are used to optimize the model's generalization ability. The GAN includes a generator and a discriminator. The distribution loss of the virtual samples output by the generator is calculated as follows:

[0028]

[0029] Among them, L G This represents the generator loss value. For a probability distribution P z The mathematical expectation of the random noise variable z is calculated using the combination of (z); z is a random noise variable; P z (z) is the probability distribution that the random noise variable z follows; G(z) is the virtual fault sample generated by the generator based on the random noise variable z; D(G(z)) is the discrimination probability of the discriminator on the virtual fault sample output by the generator; a gradient pruning strategy is introduced during the generator training process to adaptively limit the gradient magnitude during the training process.

[0030] Furthermore, the AI-based network fault trend prediction system includes an edge causality labeling module, a causality weight allocation module, a topology causality constraint module, a causality chain verification module, a hierarchical prediction module, a treatment effect attribution module, and a virtual sample federated sharing module. The edge causality labeling module transmits data on associated causal pairs to the causality weight allocation module. The causality weight allocation module outputs a standardized dataset to the topology causality constraint module. The topology causality constraint module provides fault evolution data and twin environment data to the causality chain verification module and the hierarchical prediction module, respectively. The output data of the causality chain verification module and the hierarchical prediction module are both transmitted to the treatment effect attribution module. The treatment effect attribution module sends feedback data to the topology causality constraint module and the virtual sample federated sharing module, respectively. The virtual sample federated sharing module outputs optimized parameters to the causality weight allocation module and the hierarchical prediction module. The topology causality constraint module feeds back updated causal graph data to the causality weight allocation module.

[0031] Furthermore, the edge causal labeling module integrates a multi-source data acquisition interface, a rule engine, and an improved real-time compression algorithm module. The multi-source data acquisition interface synchronously acquires network traffic, device hardware status, topology changes, and application layer log data. The rule engine analyzes the correlation between multimodal data, generates one-to-one causal relationship pairs, and binds them to the original data. The improved real-time compression algorithm module processes data based on compression ratio calculation formulas, preserving causal relationship attributes.

[0032] The causal weight allocation module includes a data cleaning module, a normalization module, a causal graph model module, and a feature selection module. The data cleaning module performs deduplication, missing data completion, and outlier filtering operations on the data processed by the edge causal labeling module. The normalization module maps the cleaned data to a unified numerical range. The causal graph model module assigns feature causal weights to the data based on the sum of the causal correlations between each feature and other features. The feature selection module retains features whose weights meet the preset selection criteria and generates a standardized dataset, which is then transmitted to the topological causal constraint module.

[0033] The advantages of this invention are:

[0034] 1. This invention collects multimodal network data and generates causal relationship pairs using a rule engine. It also optimizes data processing efficiency by combining an improved real-time compression algorithm. Simultaneously, it accurately calculates the degree of causal correlation of features and assigns weights based on a causal graph model, and selects effective features to generate a standardized dataset. This achieves efficient cleaning of network data and focus on core features, avoids invalid data from interfering with the analysis, provides high-quality data support for fault prediction, and significantly improves the reliability and relevance of the input data for the prediction model.

[0035] 2. This invention constructs a twin environment that is synchronized with the real network in time and space by using a standardized dataset. It captures topological changes in real time and updates the causal graph model to simulate the multi-fault coupling evolution process. Then, the causal reasoning algorithm traces the source of the fault to generate a complete causal chain and verifies it in the twin environment. This achieves accurate reproduction of the fault evolution logic and clear definition of causal relationships, solves the problem of difficulty in clarifying the fault propagation path, and greatly improves the accuracy and completeness of fault root cause location.

[0036] 3. This invention adopts an edge-cloud layered model architecture. The edge side quickly processes real-time data to output early warnings, while the cloud integrates multi-node data to predict global trends. It combines path analysis algorithms with node vulnerability parameters to determine the fault propagation path, realizing layered and accurate prediction of fault trends and dynamic adjustment of early warning logic. This ensures both the real-time nature of local early warnings and the comprehensiveness of global predictions, effectively improving the timeliness and pertinence of network fault early warnings.

[0037] 4. This invention generates virtual fault samples through generative adversarial networks, securely shares parameters among nodes by relying on vertical federated learning, and introduces a gradient pruning strategy in generator training to optimize training stability. This significantly improves the model's generalization ability in scenarios where fault samples are scarce, avoids training bias due to insufficient samples, and ensures data privacy and security. It also enables the prediction model to maintain stable and efficient fault prediction performance in complex network environments.

[0038] 5. This invention utilizes reinforcement learning algorithms to select the optimal handling action based on the fault state, compares the handling effect with the expected result to calculate the degree of matching, and transmits the result back to the twin environment and model optimization module to update relevant strategies and parameters. This allows the fault prediction and handling process to form a virtuous cycle of dynamic iteration, improving the effectiveness of fault handling and the model's ability to adapt to dynamic changes in the network, thus helping the network system to quickly restore stable operation. Attached Figure Description

[0039] The accompanying drawings, which form part of this application, are used to provide a further understanding of the application and to make other features, objects, and advantages of the application more apparent. The illustrative embodiments and descriptions of this application are used to explain the application and do not constitute an undue limitation of the application.

[0040] In the attached diagram:

[0041] Figure 1 This is a flowchart of the network fault trend prediction method based on artificial intelligence in Example 1.

[0042] Figure 2 This is a flowchart of data collection and annotation in the AI-based network fault trend prediction method in Example 1.

[0043] Figure 3 This is a framework diagram of the network fault trend prediction system based on artificial intelligence in Example 1. Detailed Implementation

[0044] The present invention will now be described in detail and specifically through specific embodiments to enable a better understanding of the invention. However, the following embodiments do not limit the scope of protection of the present invention.

[0045] Example 1

[0046] like Figure 1 As shown, the network fault trend prediction method based on artificial intelligence includes the following steps:

[0047] Step S1: Collect multimodal network data, generate causal relationship pairs through a rule engine, and process the data using an improved real-time compression algorithm;

[0048] Step S2: Perform deduplication, missing data completion, and outlier filtering operations on the collected and compressed data. Assign feature weights based on the causal graph model and filter features whose weights meet the preset filtering conditions to generate a standardized dataset.

[0049] Step S3: Construct a twin environment based on a standardized dataset, capture network topology changes and update the causal graph model, simulate the multi-fault coupling evolution to reproduce the fault chain logic;

[0050] Step S4: Analyze the fault evolution data using a causal reasoning algorithm, generate a fault causal chain, and verify its accuracy and completeness in a twin environment;

[0051] Step S5: Use an edge-cloud layered model to predict fault trends, determine fault propagation paths through path analysis algorithms, and adjust early warning logic.

[0052] Step S6: Select the optimal action through reinforcement learning algorithm, calculate the degree of matching between the action effect and the expected result, and update the relevant strategies and parameters;

[0053] Step S7: Generate virtual fault samples through generative adversarial networks, and optimize the model's generalization ability based on shared parameters through longitudinal federated learning.

[0054] In specific embodiments, multimodal network data acquisition employs mature and easily deployable general solutions for different data types. Network traffic data is acquired by deploying open-source network probes on the mirror ports of the core switch, supporting real-time capture of TCP and UDP protocol data. Device hardware status data is collected using the device's built-in hardware management interface or network management protocol, covering key indicators such as CPU utilization, memory usage, and hard disk temperature. Topology change data is obtained by periodically scanning network devices using network scanning tools, acquiring device connectivity relationships at fixed time intervals. Application layer log data is collected in real-time by a log collection agent deployed on each server node, transmitting application runtime logs to the data processing node.

[0055] The rules engine is built using an open-source rules engine framework and includes a built-in configurable rule set for network fault domains. Rule design is based on expert experience and historical fault data. For example, if the traffic on a certain port exceeds a preset threshold for a continuous period (the threshold is set according to a reasonable proportion of the port's historical peak traffic), and the corresponding device's CPU utilization also exceeds a certain proportion, a causal relationship pair between traffic anomalies and port overload is automatically generated. Each relationship pair includes triggering conditions, correlation characteristics, and confidence levels, which are directly appended to the metadata fields of the collected data, achieving a stable binding between data and causal information.

[0056] The improved real-time compression algorithm employs a combination of causal correlation filtering and lossless compression. First, the collected data is grouped according to causal relationships, retaining data fields directly related to causal pairs while eliminating duplicate and redundant information. Then, the filtered data is compressed using a classic lossless compression algorithm, recording the location index of key data during compression to prevent loss of correlation information during decompression. This algorithm can be directly deployed in embedded systems of edge gateways, adapting to low-computing-power environments.

[0057] The data processing phase follows a fixed workflow of deduplication, completion, and filtering. Deduplication is achieved by comparing the unique identifiers of the data, which are composed of timestamps, device identifiers, and data types, eliminating completely duplicate records. Missing data completion uses a nearest-neighbor interpolation method, selecting multiple features with the strongest causal relationship to the missing data as references, and calculating the completed value through a weighted average. Outlier filtering adopts statistical principles, combined with a causal graph model to determine whether outlier data is a precursor to a fault. If an outlier has no causal relationship with other features, it is directly removed; otherwise, it is retained and marked.

[0058] The causal graph model is built on an open-source graph learning framework. Nodes correspond to feature terms in the network data, and edge weights are calculated using mutual information values. Model training uses historical fault data, iteratively optimizing the causal correlation strength between features. Weight allocation is directly obtained by normalizing the correlation coefficients output by the model. Preset conditions for feature selection are determined through cross-validation, selecting weight thresholds that minimize the fault prediction error on the validation set, ensuring that the selected core features maximize model performance.

[0059] The twin environment is built using container orchestration and visualization monitoring tools, with ordinary cloud servers used for hardware, simulating the core and access layer devices of a real network. Virtualization technology replicates the topology, link bandwidth, and device performance parameters of the real network, while the visualization tool displays the twin environment's operational status in real time. Topology change detection is triggered by the network monitoring tool's alarm mechanism; once changes such as device offline or link interruption are detected, the interface is immediately invoked to update the node connection relationships and weights of the cause-effect graph model.

[0060] Multi-fault coupling evolution simulation is implemented through pre-set fault scenario scripts, supporting the combined triggering of common faults such as hardware failures, traffic attacks, and link interruptions. The scripts can be configured with fault occurrence time, impact range, and propagation rate to reproduce the temporal process of fault chain reactions. The causal inference algorithm adopts a simplified probabilistic graphical model. It first locates the terminal nodes in the causal graph through fault phenomena, and then traverses backward based on node weights to calculate the fault contribution of each upstream node. The node with the highest contribution is the root cause of the fault. The generated causal chain contains verifiable information such as node identifiers, trigger times, and propagation paths.

[0061] The edge-cloud layered model adopts a deployment architecture of edge nodes and cloud server clusters. The edge nodes use low-power edge computing devices and deploy lightweight deep learning models to process only local real-time data for a short period of time and output local fault warnings. The cloud uses a cluster of multiple high-performance servers to deploy deep neural network models, periodically summarize data from each edge node, and combine it with historical fault data to predict global trends.

[0062] The path analysis algorithm is based on the classic shortest path algorithm and optimizes it by using node correlation, failure probability, and vulnerability parameters as path weighting factors. It calculates the diffusion probability by traversing all possible paths, and the path with the highest probability is the optimal diffusion path. The early warning logic is adjusted by modifying the early warning thresholds of edge nodes. When the diffusion probability of a path exceeds a set value, the early warning trigger thresholds of all nodes on that path are automatically lowered, triggering an early warning.

[0063] The reinforcement learning algorithm uses a simplified version of the deep Q-network algorithm, implemented based on a mainstream deep learning framework. The pre-defined action library contains multiple common fault handling actions, each with a corresponding execution script. For example, the bandwidth adjustment action calls the network device's management interface to modify port bandwidth limits, and the redundant link switching action switches the network routing table via configuration files. Action selection is based on the principle of maximizing the value function, which is updated by the similarity between the handling effect and the expected result. The action value network parameters are updated iteratively after each fault is handled.

[0064] Both the generator and discriminator in the generative adversarial network (GAN) employ fully connected neural network structures. The generator's input is random noise, the number of hidden layer nodes is set according to a reasonable gradient, and a common nonlinear activation function is used. The output dimension is consistent with that of real faulty samples. The discriminator's input dimension is the same as the generator's output dimension, the number of hidden layer nodes gradually decreases, and a linear activation function with leakage is used. The output dimension is the discrimination probability. During training, an adaptive optimizer is used, with a reasonable learning rate set, and stable virtual samples are generated after sufficient iterations.

[0065] Vertical federated learning is implemented based on an open-source federated learning framework. Each node trains a local model of the generative adversarial network locally, transmitting only the model parameters to the federated server via an encrypted transmission protocol. Parameter aggregation uses a weighted average method, with weights set according to the number of samples at each node. The aggregated global parameters are then distributed back to each node to update the model. The entire process does not require the transmission of raw data, effectively protecting data privacy, and can be directly deployed across network nodes in different regions.

[0066] Furthermore, such as Figure 2As shown, in step S1, the collected multimodal network data includes network traffic data, device hardware status data, topology change data, and application layer log data. A rule engine analyzes the correlations between the multimodal data, generates causal pairs, and appends them to the collected data. An improved real-time compression algorithm is used to process the data, and this improved real-time compression algorithm satisfies the compression ratio calculation formula:

[0067]

[0068] Where η is the real-time compression ratio; L0 is the total amount of data before compression; and L1 is the total amount of data after compression.

[0069] In specific embodiments, the selection of tools for multimodal network data acquisition fully considers compatibility and deployment costs. The network traffic acquisition tool is directly integrated into the embedded system of the core switch, supporting high-bandwidth line-speed acquisition without requiring additional hardware. The acquisition interface for device hardware status data is standard on the server, the network management protocol supports mainstream network devices, and the acquisition frequency can be flexibly set through configuration files. The topology change data scanning tool is deployed on a cloud server, scanning the network through network probing requests, limiting the scanning scope to the target network segment to avoid network congestion. The application layer log data acquisition agent is containerized, supporting interface with the log output interfaces of mainstream application servers, and using reliable transmission protocols to ensure data integrity.

[0070] The rule engine framework is deployed on data processing nodes. Rule sets are stored in a common text format and support online updates. Rule condition judgments employ a multi-feature combination model. For example, if port traffic reaches a certain value (referenced to historical peak values ​​for the port), and CPU utilization and memory usage exceed reasonable limits, a causal relationship of insufficient port resources is triggered. Correlation features are selected from fields directly related to the fault, such as the source and destination ports of traffic data, and the number of cores in hardware data, ensuring the accuracy of the causal relationship. Causal relationship pairs are appended using a common data exchange format, embedding the metadata portion of the data without altering the original data structure, facilitating parsing by subsequent processing modules.

[0071] The implementation process of the improved real-time compression algorithm consists of three steps: First, data grouping, which divides the data into multiple groups according to the identifier of the causal relationship pair, with each group containing data that triggers the same causal relationship; second, field filtering, which retains key fields in each group of data, such as timestamps, trigger features, and device identifiers, while removing duplicate descriptive fields; and third, lossless compression, which performs sliding window compression on the filtered fields, with the window size and matching length threshold set according to the data characteristics to balance compression efficiency and speed.

[0072] The compression ratio η directly reflects the degree of data compression. L0 is the total number of bytes of the original data before grouping, including valid information, redundant data, and noise. L1 is the total number of bytes after processing by the compression algorithm. During compression, the algorithm focuses on retaining key information related to causality and eliminates meaningless redundant data, reducing data transmission and storage overhead while ensuring that core data required for subsequent analysis is not lost. Its design is particularly suitable for edge network environments with limited bandwidth, achieving a balance between data processing and transmission efficiency and data validity. The algorithm code can be written in a general-purpose programming language, and the compiled executable file is small in size, occupying very little storage space.

[0073] Further, in step S2, the collected and compressed multimodal network data undergoes deduplication, missing data completion, and outlier filtering operations. A standardization method is used to convert the data to a uniform numerical range. Based on the causal graph model, the causal correlation degree of each feature is calculated and weights are assigned. Features whose weights meet preset filtering conditions are selected to generate a standardized dataset. The formula for calculating the causal weights of features in the causal graph model is:

[0074]

[0075] Where, ω i C represents the causal weight of the i-th feature; i,∑ The sum of the causal correlations between the i-th feature and all other features; m f This represents the total number of features in the valid feature set determined based on the strength of causal relationships between features.

[0076] In a specific embodiment, the collected and compressed multimodal data first enters the data cleaning module, which is implemented based on a general data processing library and deployed on the data processing node. Deduplication is achieved by constructing a composite primary key, which is composed of a timestamp, device identifier, and data type. When a duplicate primary key is detected, the latest data is retained, and historical duplicate records are deleted to avoid duplicate data consuming computing resources.

[0077] Missing data completion employs a nearest neighbor interpolation method that combines causal relationships, with the number of nearest neighbors set according to the data distribution characteristics. First, a causal graph model is used to select the features with the strongest causal relationship to the missing data. Then, the most similar records of these features to the current data are selected from historical data, and the weighted average of the missing fields is calculated. The higher the similarity, the greater the weight, ensuring that the error after completion is controlled within a reasonable range.

[0078] Outlier filtering employs statistical principles combined with causal verification. First, the mean μ and standard deviation σ of each feature are calculated, and data exceeding a reasonable range are marked as suspected anomalies. Then, a causal graph model is used to determine whether the suspected anomaly data has a strong causal relationship with other features. If such a relationship exists, it is considered valid precursor data for faults, retained, and marked. If not, it is considered noise data and directly removed. For example, if the instantaneous traffic of a device exceeds the statistical range but has no causal relationship with CPU utilization and no other features support this traffic anomaly, it is judged as noise data.

[0079] The standardization method employs Z-score standardization. The specific calculation process involves first calculating the mean and standard deviation of each feature, and then transforming each data point to an interval with a mean of 0 and a standard deviation of 1 using a standardization formula. This process is implemented using a general data preprocessing library. The fitting process is based on the training set data to avoid data leakage, and the standardized feature data can be directly input into the model for weight calculation.

[0080] The causal graph model is constructed based on an open-source graph learning framework. The specific steps are as follows: Define nodes, taking each data feature as a model node, with each node containing attributes such as feature name and data type; Calculate edge weights, using mutual information values ​​to measure the strength of the causal relationship between two nodes. The mutual information value is calculated using a general mathematical calculation library function, with a value range of 0-1, where a larger value indicates a stronger relationship; Model training, using historical fault data to iteratively optimize edge weights, setting reasonable training rounds and learning rates to ensure that the model can accurately reflect the true causal relationship between features.

[0081] Feature causal weight ω i The calculation is achieved by normalizing the edge weights. First, the weights of all edges at each node are summed to obtain C. i,∑ Then C of all nodes i,∑ Summation, ω for each node i That is, itself C i,∑ The ratio to the sum. The m of the effective feature set. f Isolated nodes are identified by removing isolated nodes, which are nodes whose edge weights are all less than the minimum value, ensuring that all core features involved in the weight calculation have actual causal relationships.

[0082] The preset conditions for feature selection are determined through cross-validation. Historical data is divided into multiple folds, and models are trained and prediction errors are calculated for each fold. The weight threshold that minimizes the average error is selected. ω is retained during the selection process. i Features exceeding this threshold are used to generate a standardized dataset. The dataset is in a general text format, with each row corresponding to one data record and each column corresponding to a filtered feature. It can be directly used for twin environment construction and model training.

[0083] Furthermore, in step S3, a twin environment synchronized with the real network in time and space is constructed on the generated standardized dataset. This environment captures real-time changes in the real network topology and synchronously updates the node connection relationships and corresponding weights of the causal graph model. It simulates the multi-fault coupling evolution process to reproduce the fault chain reaction logic. Topology change capture is achieved by real-time monitoring of network node connection status and link parameters. Node updates in the causal graph model follow the priority of node correlation, and the node correlation calculation formula is:

[0084]

[0085] Where, γ ij Let T be the degree of association between the i-th node and the j-th node; ij denoted as , where is the real-time interaction frequency between the i-th node and the j-th node; m is the total number of nodes in the causal graph model.

[0086] In a specific embodiment, the hardware deployment of the twin environment uses two well-configured general-purpose servers deployed within the same local area network, with network latency controlled to an extremely low level. Containers are orchestrated using container orchestration tools, with one server running multiple containers to simulate core layer devices and the other running more containers to simulate access layer devices. Communication between containers is achieved through a bridged network, and link bandwidth is limited to common network bandwidth standards using network traffic control tools to simulate the bandwidth constraints of a real network.

[0087] At the software level of the twin environment, a combination of visual monitoring tools and metric collection tools is used to monitor the container's CPU, memory usage, and network traffic in real time. The monitoring data is updated at fixed time intervals, providing a clear view of the twin environment's operational status. Spatiotemporal synchronization between the real network and the twin environment is achieved through a network time protocol, ensuring that their timestamps are consistent and that synchronization errors are kept to a minimum.

[0088] Topology change detection is achieved through a combination of network monitoring tools and scripts. The network monitoring tools send network probe requests at fixed intervals to detect device online status and periodically request device port connection status via network management protocols. When a device is detected to be offline or a port status change is detected, the script is triggered to call the interface of the graph learning framework to update the node connections and edge weights of the causal graph model. For example, when the status of a switch port changes from available to unavailable, the connection edge between the node corresponding to that port and the downstream device node is immediately deleted, and the edge weights of the upstream nodes are adjusted simultaneously.

[0089] The multi-fault coupling evolution simulation allows for the creation of fault scenarios via scripts, supporting three modes: single-fault triggering, dual-fault superposition, and multi-fault cascading. Single-fault triggering, such as simulating a server crash, involves the script stopping the corresponding container using container management commands. Dual-fault superposition, such as simultaneously simulating a server crash and link interruption, involves stopping the container while simultaneously disconnecting the corresponding link using a flow control tool. Multi-fault cascading, such as simulating a server crash causing downstream terminal traffic congestion, involves first stopping the server container and then sending a large number of data packets to the terminal containers using a traffic generation tool. Fault triggering conditions can be configured via configuration files, including fault type, occurrence time, duration, and impact scope. The script can run directly on the server in the twin environment without additional hardware support.

[0090] In causal graph models, node updates are prioritized based on node affinity γ. ij Determined, γ ij The frequency T of interactions between nodes per unit time ij Calculate, T ij This represents the sum of the number of connections and data transmission packets between two nodes during this time period, where m is the total number of nodes in the causal graph model. The correlation degree calculation is implemented using a data processing library, and T is periodically statistically analyzed. ij And update γ ij Nodes with higher correlation rankings are set to high priority and processed first during updates to ensure the causal graph model can quickly respond to changes in key nodes. The data for calculating node correlation is directly derived from network monitoring data in the twin environment. Container network traffic data collected by the metric collection tool contains interaction information between nodes, which can be extracted through an interface to calculate T. ij No additional data collection tools are required, resulting in low cost and easy maintenance. The causal graph model update process is executed asynchronously in the background, without affecting the normal operation of the twin environment. Update latency is controlled within a short period of time, ensuring that the model can promptly reflect the actual network conditions.

[0091] Furthermore, in step S4, the fault evolution data output by the twin environment is analyzed using a causal reasoning algorithm to generate a complete fault causal chain from the root cause to the surface phenomenon. The starting conditions of the fault causal chain are reproduced in the twin environment to verify the accuracy and completeness of the fault causal chain. The causal reasoning algorithm traces the fault triggering source layer by layer based on the node association relationship and weight of the causal graph model to form a continuous and unbroken causal sequence. The verification process is completed by comparing the results reproduced in the twin environment with the actual fault phenomenon.

[0092] In a specific embodiment, the causal inference algorithm employs a simplified Bayesian network, implemented based on a general probabilistic graphical model library. The network structure consists of three layers: the fault source layer, the intermediate propagation layer, and the fault phenomenon layer. The number of nodes in each layer is adjusted according to the actual network size. The conditional probability table is obtained through statistical analysis of historical fault data. For example, the probability that a certain source node causes a certain phenomenon node is calculated by statistically analyzing the number of times both occur simultaneously in a large number of historical fault records.

[0093] The specific process of generating the causal chain consists of three steps: First, fault phenomenon identification, which extracts fault phenomena from the fault evolution data output by the twin environment and locates the phenomenon nodes in the probabilistic graph network; Second, reverse tracing, which calculates the fault contribution of each upstream node based on the conditional probability table. The contribution is equal to the product of the conditional probability and the node correlation degree. The nodes are sorted from high to low contribution and the node with the highest contribution is selected as the candidate source; Third, verification and confirmation, which traverses all propagation paths of the candidate source. If a certain path can completely cover all fault phenomena, the node is determined to be the root cause of the fault, and a complete causal chain containing the source node, propagation nodes, phenomenon nodes, and trigger time is generated.

[0094] The specific implementation process for twin environment verification is as follows: First, configure the twin environment according to the starting conditions of the causal chain. For example, if the source of the causal chain is server CPU overload, then limit the CPU usage of the server container using resource limiting commands to simulate the overload state. Second, run the twin environment and observe whether the fault evolves according to the path described by the causal chain, such as whether it triggers phenomena like decreased port traffic or increased terminal access latency. Finally, compare the consistency between the reproduced results and the actual fault phenomena. If most phenomena are consistent, the causal chain is determined to be accurate. If they are inconsistent, adjust the conditional probability table of the probabilistic graphical network and regenerate the causal chain until the verification is successful.

[0095] During the verification process, the comparison of fault phenomena is achieved through scripts. Fault phenomenon data from the reproduction process and actual fault phenomenon data are extracted, and their similarity is calculated. The similarity is measured using the Jaccard coefficient, with a value ranging from 0 to 1. This verification process can be executed automatically without manual intervention. The verification duration is set according to the fault evolution cycle, meeting the needs of real-time fault analysis. Those skilled in the art can adapt it to different network environments by adjusting the number of layers and nodes in the probabilistic graphical network.

[0096] Furthermore, in step S5, a lightweight model is deployed at the edge to process real-time data and output early warning information, while a spatiotemporal model is deployed at the cloud to fuse real-time data transmitted from multiple edge nodes to predict the global fault trend. A path analysis algorithm is used to determine the fault propagation path and dynamically adjust the early warning logic. The probability calculation formula for the fault propagation path is:

[0097]

[0098] Among them, P pq γ represents the probability that a fault propagates from node p to node q; pq Let λ be the degree of association between the p-th node and the q-th node; p μ represents the probability of failure at the p-th node. q is the vulnerability parameter of the q-th node, which is a parameter characterizing the node's ability to resist fault propagation and is determined by the node's hardware configuration, operating status, and redundancy design; k is the number of downstream associated nodes corresponding to the p-th node; j is a summation variable used to distinguish downstream nodes.

[0099] In a specific embodiment, the edge nodes of the edge-cloud layered model utilize low-power edge computing devices. These devices are small in size and consume little power, allowing for direct deployment in network racks within data centers, thus meeting the deployment requirements of edge scenarios. The lightweight deep learning model deployed on the edge nodes is optimized based on a lightweight deep learning framework. It selects multiple key features for input dimensions, employs two hidden layers, and outputs dimensions corresponding to common fault types. The model file is small in size, occupies minimal storage space, and boasts fast inference speed, with single inference time controlled within a short period.

[0100] The model training for edge nodes employs a transfer learning approach. First, a base model is trained in the cloud using large-scale historical fault data. Then, the model parameters are transferred to the edge nodes for fine-tuning using local, real-time data from the edge nodes over a short period. Appropriate fine-tuning rounds and learning rates are set to ensure the model adapts to the local data distribution of the edge nodes. The real-time data processing flow for edge nodes is as follows: local data is collected periodically, standardized, and then input into the model. The model outputs the fault type and warning level. When the warning level exceeds a set value, a warning message is sent to the administrator terminal via a message transmission protocol.

[0101] The cloud-based system utilizes a cluster of multiple high-performance servers to support parallel training and inference of the model. The deep neural network model deployed in the cloud is implemented based on a general-purpose deep learning library, with a multi-layer encoder, a reasonable number of attention heads, a moderate hidden layer dimension, and a controllable model file size. It supports batch processing of data from multiple edge nodes, processing input data from multiple edge nodes at a time within a short period of time.

[0102] The global trend prediction process in the cloud is as follows: Real-time data and local early warning results uploaded by each edge node are periodically received, aggregated, and combined with long-term historical fault data. This data is then input into a deep neural network model, which outputs a global fault trend for the future, including possible fault types, impact range, and probability of occurrence. The prediction results are pushed to the network management platform via a network interface for administrators to view and make decisions.

[0103] The path analysis algorithm is based on the classic shortest path algorithm and is optimized as follows: First, construct a fault propagation graph, where nodes are network devices and the weight of an edge is equal to the node's correlation γ. pq Fault occurrence probability λ p Node vulnerability parameter μ q The product of λ p μ is calculated based on the historical failure frequency of the node. q The first step involves a comprehensive calculation based on the node's hardware redundancy and load rate. The second step involves calculating the shortest path from the faulty node to all downstream nodes, which is the most likely path for the fault to propagate. The third step involves dynamically adjusting the warning logic to lower the warning threshold for nodes on the propagation path. For example, if the original warning threshold was a certain traffic value, it can be adjusted to a lower value to trigger the warning earlier.

[0104] The algorithm is implemented using a general network analysis library and can be directly deployed on cloud servers. It boasts a short computation time per run, quickly outputting fault propagation paths and adjusted early warning logic, thus meeting the demands of real-time prediction. (Node vulnerability parameter μ) q The calculation data comes from the device's hardware configuration information and operating status data, which can be obtained through network management protocols and server monitoring tools. No additional data collection equipment is required, resulting in low cost.

[0105] Furthermore, in step S6, a reinforcement learning algorithm is used to select the optimal action based on the current fault state, compare the action effect with the expected result of the fault causal chain to calculate the matching degree, and transmit the matching result to the twin environment and model optimization module to update relevant strategies and parameters. The reinforcement learning algorithm selects suitable actions from the preset action library based on the fault type and the current network state. The matching degree is calculated based on the fit between the fault mitigation effect after action and the expected effect. The matching result is used simultaneously to adjust the twin environment parameters and model training strategy.

[0106] In this specific embodiment, a simplified version of the deep Q-network algorithm is used for reinforcement learning. It is implemented based on a mainstream deep learning framework and deployed on one of the servers in a cloud server cluster. The algorithm's experience replay pool size is set to a reasonable value, the batch size is configured according to computing resources, the discount factor is set to a value close to 1, the learning rate is set to a relatively small value, and the target network update frequency is set according to the number of iterations to ensure the algorithm's stability and convergence speed.

[0107] The pre-defined action library is designed based on real-world network fault handling scenarios and includes multiple core actions. Each action corresponds to a specific execution script and applicable scenario: 1. Restart device: Log in to the device via a remote login protocol and execute the restart command, suitable for device unresponsiveness faults; 2. Adjust bandwidth: Modify the bandwidth limit of the router port via a network management protocol, suitable for traffic overload faults; 3. Switch redundant links: Switch traffic to a backup link by modifying the network routing table, suitable for primary link interruption faults; 4. Optimize resource allocation: Adjust the CPU and memory quotas of containers via container management commands, suitable for resource shortage faults; 5. Close abnormal ports: Disable the ports corresponding to abnormal traffic via firewall configuration commands, suitable for port attack faults; 6. Clean up log files: Delete excessively large log files on the server via file management commands, suitable for insufficient disk space faults; 7. Restart application services: Restart application processes via service management commands, suitable for application crash faults; 8. Issue rate limiting rules: Limit the access frequency of abnormal addresses via reverse proxy configuration files, suitable for distributed attack faults.

[0108] The scripts in the action library are all written in a general-purpose scripting language and can be executed directly on network devices or servers. They support parameterized configuration; for example, a script to adjust bandwidth can be flexibly adjusted by passing in the port number and target bandwidth value. The execution results of the scripts are returned to the reinforcement learning algorithm via a remote login protocol, including information such as execution success or failure status and execution time.

[0109] The matching degree is calculated using a cosine similarity algorithm, which converts the handling effect data and expected result data into feature vectors respectively. The handling effect data includes indicators such as fault mitigation time, changes in resource consumption, and business recovery rate. The expected result data is set based on historical successful handling cases. For example, if the expected result for a fault is a high business recovery rate within a short period of time, and the actual business recovery rate reaches an even higher percentage within a shorter period of time after handling, then the cosine similarity of the feature vectors is high, indicating a good matching degree.

[0110] The feedback process for matching results is as follows: After the algorithm executes a action, it waits for a period of time to collect data on the action's effect and calculates the matching degree. If the matching degree is high, it indicates that the action is effective, and the action value network parameters are updated according to the current learning rate. If the matching degree is medium, the learning rate is adjusted and the parameters are updated. If the matching degree is low, it indicates that the action is ineffective, the experience is discarded, and the parameters are not updated. The parameter update process is executed asynchronously, which does not affect the algorithm's real-time decision-making and ensures that the algorithm can continuously optimize the action strategy.

[0111] The parameter adjustments for the twin environment and model optimization module are implemented through an interface. After the matching results are fed back, the algorithm calls the interface of the deep learning framework to adjust the fault simulation parameters of the twin environment, and at the same time adjusts the training parameters of the model, so that the model and the twin environment can continuously adapt to changes in network state and improve the effectiveness of fault handling.

[0112] Further, in step S7, virtual fault samples are generated based on the real data distribution using a generative adversarial network (GAN). A longitudinal federated learning approach is employed to share virtual sample parameters among nodes. The virtual samples and shared parameters are used to optimize the model's generalization ability. The GAN includes a generator and a discriminator. The distribution loss of the virtual samples output by the generator is calculated as follows:

[0113]

[0114] Among them, L G This represents the generator loss value. For a probability distribution P z The mathematical expectation of the random noise variable z is calculated using the combination of (z); z is a random noise variable; P z (z) is the probability distribution that the random noise variable z follows; G(z) is the virtual fault sample generated by the generator based on the random noise variable z; D(G(z)) is the discrimination probability of the discriminator on the virtual fault sample output by the generator; a gradient pruning strategy is introduced during the generator training process to adaptively limit the gradient magnitude during the training process.

[0115] In a specific embodiment, both the generator and discriminator of the generative adversarial network adopt a fully connected neural network structure, are implemented based on mainstream deep learning frameworks, and are deployed on cloud servers to accelerate the training process by utilizing the computing power of the servers.

[0116] The generator's specific structure is as follows: the input layer is high-dimensional random noise z, which follows a normal distribution P with a mean of 0 and a variance of 1. z(z); The hidden layer consists of two layers, with the first layer having a larger number of nodes and the second layer having a decreasing number of nodes. Both layers use non-linear activation functions, and a dropout mechanism is added to prevent overfitting. The output layer has the same dimension as the real fault samples and uses the hyperbolic tangent activation function, which maps the output value to a fixed interval, consistent with the distribution of the standardized real sample data. The generator's role is to convert random noise into virtual samples similar to the distribution of real fault samples. The discriminator's specific structure is as follows: the input layer has the same dimension as the generator's output layer; the hidden layer consists of two layers, with the first layer having the same number of nodes as the generator's output layer and the second layer having a decreasing number of nodes. Both layers use linear activation functions with leakage to avoid gradient vanishing; the output layer is one-dimensional and uses the sigmoid activation function. The discriminator's role is to distinguish whether the input sample is a real fault sample or a virtual sample. The sigmoid activation function of the output layer outputs the discrimination probability D(G(z)), which ranges from 0 to 1. The closer the value is to 1, the more likely the discriminator considers the input to be a real sample.

[0117] Generator loss L G The computation is implemented through built-in functions of the deep learning framework, with the core being minimization. The goal is to ensure that the virtual samples generated by the generator can deceive the discriminator as much as possible, making D(G(z)) close to 1. During training, the generator and discriminator are trained alternately. First, the discriminator is trained while the generator parameters are fixed, then the generator is trained while the discriminator parameters are fixed. The total number of training rounds is set to a sufficient value, and L is calculated after each training round. G Training is stopped when the value is within a reasonable range to ensure the quality of the generated virtual samples.

[0118] The gradient clipping strategy is implemented as follows: During the backpropagation of the generator, the gradient norm value is monitored through the gradient clipping function of the deep learning framework. A reasonable gradient norm threshold is set, which is determined through cross-validation to avoid gradient explosion or gradient vanishing. When the gradient norm exceeds the threshold, the gradient vector is scaled proportionally to make the gradient norm equal to the threshold; when the gradient norm is less than or equal to the threshold, the gradient remains unchanged. This strategy is directly integrated into the generator's training process without additional hardware or software support, effectively improving training stability.

[0119] Vertical federated learning is implemented based on an open-source federated learning framework. The nodes participating in federated training include multiple edge gateways and cloud servers, and each node deploys a local model of a generative adversarial network. Communication between nodes uses an encrypted transmission protocol to ensure the security of parameter transmission, and common ports are selected for communication to avoid port conflicts.

[0120] The specific process of federated training is as follows: First, local training: each node trains the model using local fault samples, sets a reasonable number of training rounds, and saves the local model parameters. Second, parameter uploading: each node encrypts its local model parameters and uploads them to the federated server, with an upload frequency of once every few rounds of local training. Third, parameter aggregation: after receiving the parameters from all nodes, the federated server performs a weighted average based on the number of samples from each node; the more samples, the greater the weight, generating global model parameters. Fourth, parameter distribution: the federated server encrypts the global model parameters and distributes them to each node, which then updates its local model using the global parameters. Fifth, iterative training: the above steps are repeated, setting a total number of federated training rounds to ensure that the global model can integrate sample information from each node, improving generalization ability.

[0121] Each node's local samples include real fault samples and some virtual samples. Real fault samples are derived from the node's historical fault records, while virtual samples are derived from the generator's output. The sample quantity ratio is set reasonably to ensure the authenticity of the samples while supplementing any shortage of samples. During federated training, each node only uploads model parameters and does not transmit raw sample data, effectively protecting data privacy. This approach is suitable for cross-departmental and cross-regional network fault prediction scenarios.

[0122] Furthermore, such as Figure 3 As shown, the AI-based network fault trend prediction system includes an edge causality labeling module, a causality weight allocation module, a topology causality constraint module, a causality chain verification module, a hierarchical prediction module, a treatment effect attribution module, and a virtual sample federated sharing module. The edge causality labeling module transmits data on associated causal pairs to the causality weight allocation module. The causality weight allocation module outputs a standardized dataset to the topology causality constraint module. The topology causality constraint module provides fault evolution data and twin environment data to the causality chain verification module and the hierarchical prediction module, respectively. The output data of the causality chain verification module and the hierarchical prediction module are transmitted to the treatment effect attribution module. The treatment effect attribution module sends feedback data to the topology causality constraint module and the virtual sample federated sharing module, respectively. The virtual sample federated sharing module outputs optimized parameters to the causality weight allocation module and the hierarchical prediction module. The topology causality constraint module feeds back updated causal graph data to the causality weight allocation module.

[0123] In a specific embodiment, the deployment of each module of the system fully considers functional requirements and resource costs, and adopts a distributed deployment architecture of edge plus cloud.

[0124] The edge causality labeling module is deployed on a low-power edge gateway. This device supports multiple communication protocols and can directly connect to network probes and sensors to collect multimodal data. The module's software is developed using a general-purpose programming language, integrating interfaces for various data acquisition tools. It supports data acquisition, causal relationship generation, and compression processing, while keeping memory usage within a reasonable range to adapt to the low-computing-power environment of the edge gateway. Data transmission between the edge causality labeling module and the causality weight allocation module uses a message transmission protocol, setting an appropriate quality of service level to ensure data transmission reliability. The transmission frequency is set at fixed time intervals to avoid network congestion.

[0125] The causal weight allocation module and the topological causal constraint module are deployed on two balanced cloud servers. These two modules have a large computational load and require cloud computing power support. The causal weight allocation module integrates a general data processing library and is responsible for data preprocessing and feature weight calculation; the topological causal constraint module integrates container orchestration tools, graph learning frameworks, and other tools and is responsible for building the twin environment and updating the causal graph model. Data transmission between the two modules uses the TCP protocol, with transmission latency controlled within a short time to ensure real-time data processing.

[0126] The causal chain verification module, hierarchical prediction module, treatment effect attribution module, and virtual sample federated sharing module are all deployed on a cloud server cluster, leveraging the cluster's computing power to accelerate algorithm training and inference. The causal chain verification module implements causal inference and verification based on a probabilistic graphical model library; the hierarchical prediction module integrates a lightweight deep learning framework and a general-purpose deep learning library, supporting collaboration between edge and cloud models; the treatment effect attribution module uses a general-purpose programming language to write the matching degree calculation and parameter update logic; and the virtual sample federated sharing module integrates an open-source federated learning framework, responsible for parameter interaction during federated training. Data transmission between these modules uses an in-memory database as a caching middleware to reduce data transmission latency and improve system response speed.

[0127] The data flow sequence between modules is as follows: the edge causality labeling module transmits data to the causality weight allocation module at fixed time intervals; the causality weight allocation module outputs standardized datasets to the topology causality constraint module at fixed time intervals; the topology causality constraint module provides data to the causality chain verification module and the hierarchical prediction module at fixed time intervals; the causality chain verification module and the hierarchical prediction module output results to the treatment effect attribution module at fixed time intervals; the treatment effect attribution module feeds back data to the topology causality constraint module and the virtual sample federated sharing module at fixed time intervals; the virtual sample federated sharing module outputs optimized parameters to the causality weight allocation module and the hierarchical prediction module after a fixed number of federated training rounds; and the topology causality constraint module feeds back updated causal graph data to the causality weight allocation module at fixed time intervals.

[0128] The system's fault tolerance mechanism is achieved through multiple backups: the data collected by the edge causality labeling module is stored locally and simultaneously uploaded to the cloud for backup; the standardized dataset of the causality weight allocation module is stored on the high-speed hard drive of the cloud server, supporting data recovery; the model parameters of each module are backed up to cloud storage regularly, with the backup frequency set to once a day, to avoid data loss leading to system failure. When a module fails, it can be quickly restored using backup data, maintaining the stable operation of the system.

[0129] Furthermore, the edge causal labeling module integrates a multi-source data acquisition interface, a rule engine, and an improved real-time compression algorithm module. The multi-source data acquisition interface synchronously acquires network traffic, device hardware status, topology changes, and application layer log data. The rule engine analyzes the correlation between multimodal data, generates one-to-one causal relationship pairs, and binds them to the original data. The improved real-time compression algorithm module processes data based on compression ratio calculation formulas, preserving causal relationship attributes.

[0130] The causal weight allocation module includes a data cleaning module, a normalization module, a causal graph model module, and a feature selection module. The data cleaning module performs deduplication, missing data completion, and outlier filtering operations on the data processed by the edge causal labeling module. The normalization module maps the cleaned data to a unified numerical range. The causal graph model module assigns feature causal weights to the data based on the sum of the causal correlations between each feature and other features. The feature selection module retains features whose weights meet the preset selection criteria and generates a standardized dataset, which is then transmitted to the topological causal constraint module.

[0131] In this specific embodiment, the internal components of the edge causality labeling module all utilize open-source tools, making configuration simple and maintenance easy. The multi-source data acquisition interface is implemented using network programming in a general-purpose programming language, supporting the parsing of protocols such as TCP / IP, SNMP, and Syslog: the TCP / IP protocol is used to receive application-layer log data from the log collection agent, with a suitable port configured; the SNMP protocol is used to collect device hardware status data, supporting common SNMP versions and allowing for the setting of a custom community name; the Syslog protocol is used to receive system logs from network devices, with a standard port configured. Data reception via the interface employs a multi-threaded approach, with each protocol corresponding to an independent thread to avoid data reception conflicts. The number of threads is set appropriately to support simultaneous data transmission from multiple devices.

[0132] The rule engine's expert rule base is stored in a common text format. The rule base contains multiple core rules covering common fault scenarios such as traffic anomalies, hardware failures, and link interruptions. The rule structure includes a rule identifier, trigger condition, associated features, and a causal relationship name. For example, the rule identifier might be a specific code; the trigger condition might be port traffic exceeding a certain value and CPU utilization exceeding a reasonable percentage; the associated features might be the port number and the number of CPU cores; and the causal relationship name might be "Traffic Anomaly - Port Overload." The rule base supports online updates. Modified rule files can be uploaded to the edge gateway via a file transfer protocol to replace the original files without requiring a module restart.

[0133] The improved real-time compression algorithm module is written in a general-purpose programming language and integrates open-source code from a classic lossless compression algorithm. The optimized code structure is concise and can be directly compiled into an executable file for execution on the edge gateway. The module's workflow is as follows: First, it receives data from multi-source data acquisition interfaces and causal relationship pairs from the rule engine, grouping them according to the causal relationship pairs. Then, it filters key fields for each group of data, including timestamps, device identifiers, and trigger features. Finally, it calls a lossless compression function to compress the data and outputs the compressed byte stream. During compression, the position index of key fields is stored in the header of the compressed data. During decompression, key fields can be quickly extracted based on the index, ensuring that causal relationship information is not lost.

[0134] The internal components of the causal weight allocation module work together to form a complete data processing flow. The data cleaning module is implemented based on a general data processing library and processes data in the order of deduplication, completion, and filtering: deduplication is achieved through the deduplication function of the data processing library, based on a combination of timestamps and device identifiers; completion is achieved through an interpolation function, using linear interpolation combined with causal features; and filtering is achieved through Boolean indexing, based on statistical principles to screen out abnormal data.

[0135] The normalization module uses the standardization class from a general data preprocessing library. It first calls the fitting function to fit the training set data, calculates the mean and standard deviation, and then calls the transformation function to standardize the cleaned data. The standardized dataset is stored in array format for easy subsequent processing.

[0136] The causal graph model module is implemented based on an open-source graph learning framework. First, graph data is constructed using graph data classes, including node features, edge indices, and edge weights. Then, the model is trained using graph convolutional layers to learn the strength of causal relationships between features. The batch size, learning rate, and training epochs of the model training are set according to reasonable values. After training, the causal weights of each feature are output.

[0137] The feature selection module uses the weights output by the causal graph model to filter features with weights greater than a threshold, generating a standardized dataset. The dataset uses a general text format, with the first line containing the feature name, followed by rows of data records. Each record includes a timestamp, device identifier, and the filtered feature value. The feature selection module transmits the dataset to the topology causal constraint module via a data transmission protocol, returning an acknowledgment signal upon successful transmission. The operational status of each component is monitored by a monitoring script. The script checks the component's process at fixed time intervals; if the process is missing, it automatically restarts the component and logs the changes to a specified path. The monitoring script can be set to start automatically on system boot, ensuring continuous component operation and improving system stability and reliability.

[0138] The specific embodiments of the present invention have been described in detail above, but they are merely examples, and the present invention is not equivalent to the specific embodiments described above. For those skilled in the art, any equivalent modifications and substitutions to the present invention are also within the scope of the present invention. Therefore, all equivalent transformations and modifications made without departing from the spirit and scope of the present invention should be covered within the scope of the present invention.

Claims

1. A network fault trend prediction method based on artificial intelligence, characterized in that, Includes the following steps: Step S1: Collect multimodal network data, generate causal relationship pairs through a rule engine, and process the data using an improved real-time compression algorithm; Step S2: Perform deduplication, missing data completion, and outlier filtering operations on the collected and compressed data. Assign feature weights based on the causal graph model and filter features whose weights meet the preset filtering conditions to generate a standardized dataset. Step S3: Construct a twin environment based on a standardized dataset, capture network topology changes and update the causal graph model, simulate the multi-fault coupling evolution to reproduce the fault chain logic; Step S4: Analyze the fault evolution data using a causal reasoning algorithm, generate a fault causal chain, and verify its accuracy and completeness in a twin environment; Step S5: Use an edge-cloud layered model to predict fault trends, determine fault propagation paths through path analysis algorithms, and adjust early warning logic. Step S6: Select the optimal action through reinforcement learning algorithm, calculate the degree of matching between the action effect and the expected result, and update the relevant strategies and parameters; Step S7: Generate virtual fault samples through generative adversarial networks, and optimize the model's generalization ability based on shared parameters through longitudinal federated learning.

2. The network fault trend prediction method based on artificial intelligence according to claim 1, characterized in that, In step S1, the collected multimodal network data includes network traffic data, device hardware status data, topology change data, and application layer log data. A rule engine analyzes the correlations between the multimodal data, generates causal pairs, and appends them to the collected data. An improved real-time compression algorithm is then used to process the data, and this improved real-time compression algorithm satisfies the compression ratio calculation formula: Where η is the real-time compression ratio; L0 is the total amount of data before compression; and L1 is the total amount of data after compression.

3. The network fault trend prediction method based on artificial intelligence according to claim 2, characterized in that, In step S2, the collected and compressed multimodal network data undergoes deduplication, missing data completion, and outlier filtering. A standardization method is used to convert the data to a uniform numerical range. Based on a causal graph model, the causal correlation degree of each feature is calculated and weights are assigned. Features whose weights meet preset filtering conditions are selected to generate a standardized dataset. The formula for calculating the causal weights of features in the causal graph model is: where ω i is the causal weight of the i-th feature; C i,∑ is the sum of causal correlations of the i-th feature with all other features; m f is the total number of features in the effective feature set determined based on the strength of causal correlations among features.

4. The network fault trend prediction method based on artificial intelligence according to claim 3, characterized in that, In step S3, a twin environment synchronized with the real network in time and space is constructed on the generated standardized dataset. This environment captures real-time changes in the real network topology and synchronously updates the node connection relationships and corresponding weights of the causal graph model. It simulates the multi-fault coupling evolution process to reproduce the fault chain reaction logic. Topology change capture is achieved by real-time monitoring of network node connection status and link parameters. Node updates in the causal graph model follow the priority of node correlation, and the node correlation calculation formula is: Where, γ ij Let T be the degree of association between the i-th node and the j-th node; ij denoted as , where is the real-time interaction frequency between the i-th node and the j-th node; m is the total number of nodes in the causal graph model.

5. The network fault trend prediction method based on artificial intelligence according to claim 4, characterized in that, In step S4, a causal reasoning algorithm is used to analyze the fault evolution data output by the twin environment, generating a complete fault causal chain from the root cause to the surface phenomenon. The starting conditions of the fault causal chain are reproduced in the twin environment to verify the accuracy and completeness of the fault causal chain. The causal reasoning algorithm is based on the node association relationship and weight of the causal graph model, tracing the fault triggering source layer by layer to form a continuous and unbroken causal sequence. The verification process is completed by comparing the results of the twin environment reproduction with the actual fault phenomenon.

6. The network fault trend prediction method based on artificial intelligence according to claim 5, characterized in that, In step S5, a lightweight model is deployed at the edge to process real-time data and output early warning information. A spatiotemporal model is deployed at the cloud to fuse real-time data transmitted from multiple edge nodes to predict the global fault trend. A path analysis algorithm is used to determine the fault propagation path and dynamically adjust the early warning logic. The probability calculation formula for the fault propagation path is: Among them, P pq γ represents the probability that a fault propagates from node p to node q; pq Let λ be the degree of association between the p-th node and the q-th node; p μ represents the probability of failure at the p-th node. q is the vulnerability parameter of the q-th node, which is a parameter characterizing the node's ability to resist fault propagation and is determined by the node's hardware configuration, operating status, and redundancy design; k is the number of downstream associated nodes corresponding to the p-th node; j is a summation variable used to distinguish downstream nodes.

7. The network fault trend prediction method based on artificial intelligence according to claim 6, characterized in that, In step S6, a reinforcement learning algorithm is used to select the optimal action based on the current fault state. The matching degree is calculated by comparing the action effect with the expected result of the fault causal chain. The matching result is transmitted to the twin environment and model optimization module to update relevant strategies and parameters. The reinforcement learning algorithm selects suitable actions from a preset action library based on the fault type and the current network state. The matching degree is calculated based on the fit between the fault mitigation effect after the action and the expected effect. The matching result is used to adjust the twin environment parameters and model training strategy.

8. The network fault trend prediction method based on artificial intelligence according to claim 7, characterized in that, In step S7, virtual fault samples are generated based on the real data distribution using a generative adversarial network (GAN). A longitudinal federated learning approach is employed to share virtual sample parameters among nodes. The virtual samples and shared parameters are used to optimize the model's generalization ability. The GAN includes a generator and a discriminator. The distribution loss of the virtual samples output by the generator is calculated as follows: Among them, L G This represents the generator loss value. For a probability distribution P z The mathematical expectation of the random noise variable z is calculated using the combination of (z); z is a random noise variable; P z G(z) is the probability distribution that the random noise variable z follows; G(z) is the virtual fault sample generated by the generator based on the random noise variable z; D(G(z)) is the discrimination probability of the discriminator on the virtual fault sample output by the generator; the generator training process introduces a gradient pruning strategy to adaptively limit the gradient magnitude during the training process.

9. An AI-based network fault trend prediction system, used to implement the AI-based network fault trend prediction method according to any one of claims 1 to 8, comprising an edge causality labeling module, a causality weight allocation module, a topology causality constraint module, a causality chain verification module, a hierarchical prediction module, a treatment effect attribution module, and a virtual sample federated sharing module; the edge causality labeling module transmits data of associated causal relationship pairs to the causality weight allocation module, the causality weight allocation module outputs a standardized dataset to the topology causality constraint module, the topology causality constraint module provides fault evolution data and twin environment data to the causality chain verification module and the hierarchical prediction module respectively, the output data of the causality chain verification module and the hierarchical prediction module are both transmitted to the treatment effect attribution module, the treatment effect attribution module sends feedback data to the topology causality constraint module and the virtual sample federated sharing module respectively, the virtual sample federated sharing module outputs optimization parameters to the causality weight allocation module and the hierarchical prediction module, and the topology causality constraint module feeds back updated causal graph data to the causality weight allocation module.

10. The network fault trend prediction system based on artificial intelligence according to claim 9, wherein the edge causal labeling module integrates a multi-source data acquisition interface, a rule engine, and an improved real-time compression algorithm module; the multi-source data acquisition interface synchronously acquires network traffic, device hardware status, topology changes, and application layer log data; the rule engine analyzes the correlation between multimodal data, generates one-to-one causal relationship pairs, and binds them to the original data; and the improved real-time compression algorithm module processes data based on compression ratio calculation formula, preserving causal correlation attributes. The causal weight allocation module includes a data cleaning module, a normalization module, a causal graph model module, and a feature selection module. The data cleaning module performs deduplication, missing data completion, and outlier filtering operations on the data processed by the edge causal labeling module. The normalization module maps the cleaned data to a unified numerical range. The causal graph model module assigns feature causal weights to the data based on the sum of the causal correlations between each feature and other features. The feature selection module retains features whose weights meet preset selection conditions and generates a standardized dataset, which is then transmitted to the topological causal constraint module.

Images