Network fault automatic positioning system and method based on multi-source data fusion and artificial intelligence algorithm

Abstract

The invention relates to the technical field of network fault diagnosis, in particular to an automatic network fault positioning system and method based on multi-source data fusion and an artificial intelligence algorithm, and the method comprises the steps: collecting network flow, equipment states, logs and topological structure data in real time; cleaning, denoising, standardization processing and feature extraction are carried out on the collected original data; extracting network health state key features, generating node embedding vectors, aggregating neighbor features, and splicing the aggregated neighbor features with equipment states and log semantic vectors; a fault diagnosis model is constructed based on deep learning and a knowledge graph technology, reinforcement learning and current network topology, equipment state and historical fault information are combined, a fault point is dynamically positioned, and a fault type is identified; and displaying the fault diagnosis result and the positioning information, and pushing alarm information to an operation and maintenance system. According to the invention, intelligent, accurate and efficient positioning of network faults is realized, the real-time performance and reliability of fault diagnosis are improved, and the operation and maintenance dependence and cost are reduced.

Description

Technical Field

[0001] This invention relates to the field of network fault diagnosis technology, and is a network fault automatic location system and method based on multi-source data fusion and artificial intelligence algorithms. Background Technology

[0002] With the acceleration of social digitalization, network systems have become the core infrastructure for enterprise operations, public services, and daily life. Their stability and reliability directly determine the normal operation of various businesses. Especially in key industries such as power companies, as the scale of production and services continues to expand, the scale of power dispatch data networks and internal networks is constantly expanding, and the topology is becoming increasingly complex. The number of devices, link connections, and data transmission volume in the network are growing exponentially, which significantly increases the probability of network failures, makes the types of failures more diverse, and greatly increases the difficulty of fault location and diagnosis.

[0003] Traditional network fault diagnosis methods primarily rely on manual experience and basic monitoring tools such as SNMP, Ping, and Traceroute. Operations personnel must meticulously examine massive amounts of log data, traffic information, and device status parameters to attempt to identify the fault. This approach has significant limitations. First, manual troubleshooting lacks a holistic perspective, often employing segmented detection, which can easily overlook related fault information, making it difficult to accurately pinpoint the root cause and frequently resulting in misdiagnosis or missed diagnosis. Second, many existing automated diagnostic tools are designed based on single indicators or simple rules, failing to adapt to complex and ever-changing network environments. They are ineffective in diagnosing complex faults caused by multiple factors. Third, fault diagnosis has long response times, heavily relying on the technical skills and experience of operations personnel. The efficiency and accuracy of diagnosis vary significantly among different personnel, making it difficult to meet the real-time fault handling requirements of critical business networks. Finally, traditional diagnostic methods lack intelligent analysis and self-learning capabilities, failing to summarize experience from historical fault data and optimize diagnostic strategies. This leads to repeated, tedious troubleshooting processes when similar faults recur, resulting in low operational efficiency and, in the long run, significant human resource costs and business losses for enterprises.

[0004] In critical industries such as the power sector, network failures can lead to delays in dispatch command transmission and data loss, thereby affecting the safety and stability of power production dispatch and causing serious consequences such as large-scale power outages, resulting in incalculable impacts on the social economy and public life. Therefore, the inefficiency, inaccuracy, and lack of real-time performance of traditional network fault diagnosis technologies have become key bottlenecks restricting the stable operation of network systems. There is an urgent need for a technical solution that can adapt to complex network environments and achieve intelligent and precise fault location to solve this problem. Summary of the Invention

[0005] This invention provides an automatic network fault location system and method based on multi-source data fusion and artificial intelligence algorithms, which overcomes the shortcomings of the prior art and can effectively solve the problems of low efficiency, poor accuracy and insufficient real-time performance in traditional network fault diagnosis.

[0006] One of the technical solutions of the present invention is achieved through the following measures: an automatic network fault location system based on multi-source data fusion and artificial intelligence algorithms, including a data acquisition module, a preprocessing module, a feature fusion module, an intelligent analysis module and a human-computer interaction module. Each module works collaboratively according to the data transmission logic to realize intelligent diagnosis and accurate location of network faults. The data acquisition module is used to collect network traffic, device status, logs, and topology data in real time. The preprocessing module is used to clean, denoise, standardize, and extract features from the raw data to construct multidimensional feature vectors. Among them, the feature fusion module is used to extract key features of network health status and generate a global feature vector; Among them, the intelligent analysis module builds a fault diagnosis model based on deep learning and knowledge graph technology, which is used to dynamically locate fault points and identify fault types by combining reinforcement learning; The human-computer interaction module is used to display fault diagnosis results and location information, and to push alarm information to the operation and maintenance system.

[0007] The following are further optimizations and / or improvements to one of the above-mentioned technical solutions: The aforementioned data acquisition module may include a topology discovery unit, used to construct physical topology and generate logical topology.

[0008] The aforementioned intelligent analysis module can incorporate a fault detection model, a root cause analysis model, and a location model, which are used to identify abnormal traffic patterns, infer fault propagation paths, and dynamically adjust location strategies, respectively.

[0009] The fault diagnosis model of the aforementioned intelligent analysis module can be constructed by combining the TCN and GAT models for intelligent diagnosis and localization of network faults.

[0010] The second technical solution of the present invention is achieved through the following measures: a method for automatic network fault location based on multi-source data fusion and artificial intelligence algorithms, comprising the following steps: Step 1: Data Acquisition: Real-time acquisition of network traffic, device status, logs, and topology data; Step 2: Data preprocessing: Clean, denoise, standardize, and extract features from the collected raw data to construct a multidimensional feature vector; Step 3, Feature Fusion: Extract key features of network health status, generate node embedding vectors, aggregate neighbor features and concatenate them with device status and log semantic vectors to form a global feature vector; Step 4, Intelligent Diagnosis and Localization: The fault diagnosis model, built based on deep learning and knowledge graph technology, combined with reinforcement learning and current network topology, device status and historical fault information, dynamically locates the fault point and identifies the fault type. Step 5: Output Results: Display the fault diagnosis results and location information, and push alarm information to the operation and maintenance system.

[0011] The following are further optimizations and / or improvements to the second technical solution of the above invention: During the data collection process in step one above, metadata of the entire network traffic can be collected, and throughput, packet loss rate, and latency jitter can be monitored in real time.

[0012] In the data preprocessing process of step two above, outlier detection can be performed, outliers can be marked and interpolated; at the same time, log text can be mapped to numerical features, and time domain, frequency domain and spatial domain features can be extracted to construct multidimensional feature vectors.

[0013] In the feature fusion process of step three above, node embedding vectors can be generated and neighbor features can be aggregated and concatenated with device status and log semantic vectors.

[0014] In the feature fusion process of step three above, key features of network health status can be extracted through the spatiotemporal correlation matrix.

[0015] In the intelligent diagnosis and localization process of step four above, the fault point can be dynamically and accurately located by combining the current network topology, device status and historical fault information through reinforcement learning.

[0016] This invention, through the deep integration of multi-source data fusion and artificial intelligence algorithms, constructs a systematic automatic network fault location scheme, significantly improving the intelligence and accuracy of network fault diagnosis. At the system level, the various modules work together to achieve fully automated processing from data acquisition to result output, eliminating excessive reliance on human experience. At the methodological level, standardized step design ensures the orderly and efficient execution of data processing, feature extraction, and intelligent diagnosis. Comprehensive acquisition of multi-source data provides rich foundational information for fault diagnosis, avoiding the limitations of single data dimensions; data preprocessing effectively improves data quality, providing reliable assurance for subsequent analysis; feature fusion technology can uncover deep correlations between data, forming global features that comprehensively reflect the network status; and the application of artificial intelligence algorithms enables intelligent identification of fault modes, accurate inference of fault paths, and dynamic location of fault points. Overall, this invention significantly shortens the response time for fault diagnosis, improves the accuracy and reliability of fault location, reduces operation and maintenance costs, and can adapt to diverse fault scenarios in complex network environments, providing strong support for the stable operation of network systems, especially suitable for critical industries such as power where network reliability requirements are extremely high. Attached Figure Description

[0017] Figure 1 This is a schematic diagram of the structure of a network fault automatic location system module based on multi-source data fusion and artificial intelligence algorithms according to an embodiment of the present invention.

[0018] Figure 2 This is a flowchart illustrating the automatic network fault location method based on multi-source data fusion and artificial intelligence algorithms according to an embodiment of the present invention.

[0019] Figure 3 This is a schematic diagram of the TCN model training process according to an embodiment of the present invention. Detailed Implementation

[0020] The present invention is not limited to the following embodiments, and the specific implementation can be determined according to the technical solution of the present invention and the actual situation.

[0021] The present invention will be further described below with reference to embodiments: Example 1: As Figure 1As shown in the figure, this embodiment provides an automatic network fault location system based on multi-source data fusion and artificial intelligence algorithms. It includes a data acquisition module, a preprocessing module, a feature fusion module, an intelligent analysis module, and a human-computer interaction module. Each module works collaboratively according to data transmission logic to achieve intelligent diagnosis and precise location of network faults. Specifically, the data acquisition module collects network traffic, device status, logs, and topology data in real time; the preprocessing module cleans, denoises, standardizes, and extracts features from the raw data to construct a multi-dimensional feature vector; the feature fusion module extracts key features of network health status and generates a global feature vector; the intelligent analysis module constructs a fault diagnosis model based on deep learning and knowledge graph technology, which is used to dynamically locate fault points and identify fault types using reinforcement learning; and the human-computer interaction module displays fault diagnosis results and location information, and pushes alarm information to the operation and maintenance system. Each module performs its own function while working together to form a complete fault location closed loop. The data acquisition module obtains comprehensive network operation data to provide a foundation for diagnosis; the preprocessing module optimizes data quality and avoids noise interference; the feature fusion module mines key information and enhances data value; the intelligent analysis module realizes core fault diagnosis and location; and the human-computer interaction module presents results intuitively and issues timely alarms, greatly improving fault handling efficiency and reducing manual intervention.

[0022] In this embodiment, the data acquisition module includes a topology discovery unit, used to construct the physical topology and generate the logical topology. The topology discovery unit can automatically sort out the device connection relationships and routing information in the network to form a comprehensive topology view, helping operation and maintenance personnel and diagnostic models to understand the overall network structure, providing spatial dimension reference for fault location, avoiding misjudgment of fault points due to unclear topology information, and improving the accuracy of fault location.

[0023] In this embodiment, the intelligent analysis module incorporates a fault detection model, a root cause analysis model, and a location model, which are used to identify abnormal traffic patterns, infer fault propagation paths, and dynamically adjust location strategies, respectively. The three models work collaboratively: the fault detection model quickly detects abnormal signals in the network, the root cause analysis model traces the root cause and propagation path of the fault, and the location model dynamically optimizes the location plan based on real-time conditions. The combination of these three models achieves intelligent processing throughout the entire process, from anomaly identification to root cause analysis and precise location, improving the depth and accuracy of fault diagnosis.

[0024] In this embodiment, the fault diagnosis model of the intelligent analysis module is constructed using a combination of TCN and GAT models for intelligent diagnosis and localization of network faults. The TCN model excels at processing time-series data and can effectively capture the temporal dimension features of data such as network traffic. The GAT model, on the other hand, has advantages in graph-structured data processing and can fully utilize the spatial correlation information of network topology. The combination of the two can balance feature extraction in both time and spatial dimensions, improving the model's ability to identify complex fault modes, thereby enhancing the accuracy and reliability of fault diagnosis. Figure 3 As shown, Figure 3 This is a schematic diagram of the TCN model training process according to an embodiment of the present invention. The diagram illustrates the training and application logic of the TCN model for network fault location. First, data acquisition and normalization are completed. Then, the processed data is divided into a test set and a training set. After the training set is input into the TCN model, the model is trained and tuned using the AdamW optimizer. At the same time, the training loop is closed by judging whether the termination condition is met. If the condition is not met, optimization continues. Once the condition is met, the optimized TCN model is obtained. Finally, the test set is input into the optimized model, and the fault location result is output.

[0025] In this invention, the data acquisition module collects network traffic metadata using the NetFlow / sFlow protocol through network traffic sensors, device status sensors, and log parsers, and monitors throughput, packet loss rate, and latency jitter in real time. The topology discovery unit automatically constructs the physical topology based on the LLDP / CDP protocol and generates the logical topology by combining the BGP / OSPF routing table. The preprocessing module performs outlier detection using the 3δ principle, marks and interpolates outliers, normalizes the traffic data to the [0,1] interval using the z-score standard normalization formula, maps the log text to numerical features, and extracts time, frequency, and spatial domains. The domain features construct multi-dimensional feature vectors; the feature fusion module extracts key features through spatiotemporal correlation matrices and graph embedding technology, and generates 128-dimensional node embedding vectors using the GraphSAGE algorithm. After aggregating neighbor features, it concatenates them with device status and log semantic vectors to form a global feature vector; the intelligent analysis module has built-in fault detection models, root cause analysis models, and localization models. It constructs a TCN and GAT combined fault diagnosis model based on deep learning and knowledge graph technology, and combines reinforcement learning to dynamically and accurately locate fault points; the human-computer interaction module provides a visual interface to display fault root causes, localization results, and repair suggestions, and supports API interfaces to push alarm information to the operation and maintenance system.

[0026] The specific implementation process of this system is as follows: First, a flow sensor (sFlow Agent) is deployed on the core switch, and a log agent is installed on the edge device to complete the deployment and configuration, collecting multi-dimensional data in real time. Then, data preprocessing is performed every hour to complete data cleaning and feature extraction. Next, historical data is divided into training, validation, and test sets in a 7:2:1 ratio. The AdamW optimizer and cross-entropy loss function are used to train the AI ​​model, and reinforcement learning is used to optimize the localization strategy. In the online inference stage, real-time data is input, and combined with the network topology and fault propagation path, a graph algorithm is used to locate the fault point and output the fault type (such as congestion, hardware failure, configuration error) and location coordinates. Finally, the diagnostic results are displayed through a visual interface, and repair suggestions are provided. Among them, the data acquisition module can use different protocol combinations to replace the NetFlow / sFlow protocol, and the fault diagnosis model of the intelligent analysis module can also use other machine learning algorithms to achieve the same function.

[0027] During operation, the data acquisition module collects network traffic, device status, logs, and topology data in real time and transmits the data to the preprocessing module. The preprocessing module cleans, denoises, standardizes, and extracts features from the raw data, constructing a multi-dimensional feature vector before sending it to the feature fusion module. The feature fusion module extracts key features of network health status, generates a global feature vector, and transmits it to the intelligent analysis module. The intelligent analysis module uses a fault diagnosis model built based on deep learning and knowledge graph technologies, combined with reinforcement learning, to dynamically locate fault points and identify fault types, sending the results to the human-machine interaction module. The human-machine interaction module displays the fault diagnosis results and location information and pushes alarm information to the operation and maintenance system, completing the entire fault location process. This embodiment, through efficient collaboration between modules and the application of advanced algorithms, achieves intelligent and accurate fault location, significantly improving operation and maintenance efficiency and ensuring the stable operation of the network system.

[0028] Example 2: Figure 2As shown, this embodiment provides an automatic network fault location method based on multi-source data fusion and artificial intelligence algorithms, including the following steps: Step 1, Data Acquisition: Real-time acquisition of network traffic, device status, logs, and topology data; Step 2, Data Preprocessing: Cleaning, denoising, standardizing, and feature extraction of the acquired raw data to construct a multi-dimensional feature vector; Step 3, Feature Fusion: Extracting key features of network health status, generating node embedding vectors, aggregating neighbor features, and concatenating them with device status and log semantic vectors to form a global feature vector; Step 4, Intelligent Diagnosis and Location: A fault diagnosis model based on deep learning and knowledge graph technology, combined with reinforcement learning and current network topology, device status, and historical fault information, dynamically locates fault points and identifies fault types; Step 5, Result Output: Displaying fault diagnosis results and location information, and pushing alarm information to the operation and maintenance system. This method follows a standardized procedure to automate the entire process from data acquisition to result output. Each step is interconnected: data acquisition provides the foundation for subsequent analysis, preprocessing ensures data quality, feature fusion mines key information, intelligent diagnosis enables core localization, and result output provides timely feedback. This effectively improves the efficiency and accuracy of fault location and reduces reliance on manual intervention.

[0029] In this embodiment, during the data acquisition process in step one, metadata of the entire network traffic is collected, and throughput, packet loss rate, and latency jitter are monitored in real time. Comprehensive collection of key indicator data related to the entire network traffic enables timely understanding of the real-time status of network transmission. Throughput reflects network transmission capacity, while packet loss rate and latency jitter reflect transmission quality. This data provides important evidence for fault detection, allowing for the rapid discovery of potential fault signals such as abnormal traffic, laying the foundation for subsequent fault localization.

[0030] In this embodiment, during the data preprocessing step two, outlier detection is performed, outliers are marked and interpolated; simultaneously, log text is mapped to numerical features, and time-domain, frequency-domain, and spatial-domain features are extracted to construct a multi-dimensional feature vector. Outlier detection and interpolation can remove abnormal interference from the original data, ensuring the accuracy and integrity of the data; mapping log text to numerical features and extracting multi-dimensional features can transform unstructured data into structured data that can be used for algorithm analysis, enriching feature dimensions, improving the effect of subsequent feature fusion and intelligent diagnosis, and making the constructed multi-dimensional feature vector more comprehensively reflect the network state.

[0031] In this embodiment, during the feature fusion process in step three, a node embedding vector is generated and neighbor features are aggregated, then concatenated with device status and log semantic vectors. The node embedding vector can capture the feature information of network nodes, and the aggregation of neighbor features can uncover the correlation between nodes. Combined with device status and log semantic vectors, the resulting global feature vector can comprehensively integrate multi-dimensional and multi-type feature information, avoiding the limitations of single features, providing more comprehensive and valuable input data for the intelligent diagnostic model, and improving the accuracy of fault identification and location.

[0032] In this embodiment, during the feature fusion process in step three, key features of network health status are extracted using a spatiotemporal correlation matrix. The spatiotemporal correlation matrix effectively mines the correlations between data in both time and space dimensions. The time dimension reflects the trend of data change, while the space dimension reflects the correlations between different nodes and devices. Based on this, the extracted key features more accurately reflect the essence of network health status, helping the model better identify fault modes and improve the accuracy of fault diagnosis.

[0033] In this embodiment, during the intelligent diagnosis and localization process in step four, the fault point is dynamically and accurately located by combining the current network topology, device status, and historical fault information through reinforcement learning. The current network topology and device status provide real-time network environment information, while historical fault information provides experiential reference for current diagnosis. The reinforcement learning algorithm can dynamically adjust the localization strategy based on real-time feedback, continuously optimize the localization results, and achieve dynamic and accurate localization of the fault point. This effectively addresses the dynamic changes in faults in complex network environments and improves the flexibility and accuracy of localization.

[0034] During operation, the system first performs data acquisition in step one, collecting real-time network traffic, device status, logs, and topology data. Next, it performs data preprocessing in step two, cleaning, denoising, standardizing, and extracting features from the raw data to construct a multi-dimensional feature vector. Then, it performs feature fusion in step three, extracting key features of network health status, generating node embedding vectors, aggregating neighbor features, and concatenating them with device status and log semantic vectors to form a global feature vector. Following this, it performs intelligent diagnosis and localization in step four, using a fault diagnosis model built based on deep learning and knowledge graph technology, combined with reinforcement learning, current network topology, device status, and historical fault information to dynamically locate fault points and identify fault types. Finally, it performs result output in step five, displaying the fault diagnosis results and location information, and pushing alarm information to the operation and maintenance system. This embodiment, through its clear and orderly step design and advanced technical means, achieves efficient and accurate network fault localization, adapts to the needs of complex network environments, and provides strong support for network operation and maintenance.

[0035] It should be noted that in this invention, multi-source data fusion refers to integrating and processing data from various sources and of different types, such as network traffic, device status, logs, and topology, to mine the correlation information between the data and form comprehensive data that fully reflects the network status; deep learning refers to using multi-layer neural network models to learn features and recognize patterns in the data, which can automatically mine deep patterns in the data; knowledge graph refers to the structured storage of network-related entities, relationships, and other knowledge, providing knowledge support for root cause analysis of faults; reinforcement learning refers to continuously learning and optimizing decision-making strategies through the interaction between intelligent agents and the environment, realizing the dynamic adjustment of fault location strategies; node embedding vector refers to mapping the feature information of network nodes to a low-dimensional vector space, which facilitates feature processing and analysis by algorithms; spatiotemporal correlation matrix refers to a matrix constructed by integrating time and space dimension information, used to mine the correlation relationships of data in the spatiotemporal dimension.

[0036] The above technical features constitute the embodiments of the present invention, which have strong adaptability and implementation effect. Unnecessary technical features can be added or removed according to actual needs to meet the needs of different situations.

Claims

1. A network fault automatic locating system based on multi-source data fusion and artificial intelligence algorithm, characterized in that, It comprises a data acquisition module, a preprocessing module, a feature fusion module, an intelligent analysis module, and a human-computer interaction module, which work cooperatively according to data transmission logic to realize intelligent diagnosis and accurate positioning of network faults. The data acquisition module is configured to acquire network traffic, device status, log, and topology structure data in real time. The preprocessing module is configured to clean, denoise, standardize, and extract features from the raw data to construct a multi-dimensional feature vector. The feature fusion module is configured to extract key features of network health status and generate a global feature vector. The intelligent analysis module is configured to construct a fault diagnosis model based on deep learning and knowledge graph technology, to dynamically locate fault points and identify fault types in combination with reinforcement learning. The human-computer interaction module is configured to display fault diagnosis results and positioning information, and to push alarm information to an operation and maintenance system.

2. The network fault automatic locating system based on multi-source data fusion and artificial intelligence algorithm according to claim 1, characterized in that, The data acquisition module comprises a topology discovery unit configured to construct a physical topology and generate a logical topology.

3. The network fault automatic locating system based on multi-source data fusion and artificial intelligence algorithm according to claim 1 or 2, characterized in that, The intelligent analysis module comprises a fault detection model, a root cause analysis model, and a positioning model, which are respectively configured to identify abnormal traffic patterns, deduce fault propagation paths, and dynamically adjust positioning strategies.

4. The network fault automatic locating system based on multi-source data fusion and artificial intelligence algorithm according to claim 1 or 2, characterized in that, The fault diagnosis model of the intelligent analysis module is constructed by combining TCN and GAT models, and is configured to intelligently diagnose and locate network faults.

5. A network fault automatic locating method based on multi-source data fusion and artificial intelligence algorithm, characterized in that, The method comprises the following steps: Step 1: Data acquisition: acquiring network traffic, device status, log, and topology structure data in real time; Step 2: Data preprocessing: cleaning, denoising, standardizing, and extracting features from the acquired raw data to construct a multi-dimensional feature vector; Step 3: Feature fusion: extracting key features of network health status, generating node embedding vectors, aggregating neighbor features, and concatenating device status and log semantic vectors to form a global feature vector; Step 4: Intelligent diagnosis and positioning: constructing a fault diagnosis model based on deep learning and knowledge graph technology, dynamically locating fault points and identifying fault types in combination with reinforcement learning, current network topology, device status, and historical fault information; Step 5: Result output: displaying fault diagnosis results and positioning information, and pushing alarm information to an operation and maintenance system.

6. The method for automatic network fault locating based on multi-source data fusion and artificial intelligence algorithm according to claim 5, characterized in that, In the data acquisition process of Step 1, full-network traffic metadata is acquired to monitor throughput, packet loss rate, and delay jitter in real time.

7. The network fault automatic locating method based on multi-source data fusion and artificial intelligence algorithm according to claim 5 or 6, characterized in that, In the data preprocessing process of Step 2, outlier detection is performed to mark and interpolate abnormal values; meanwhile, log texts are mapped into numerical features, and time-domain, frequency-domain, and spatial-domain features are extracted to construct a multi-dimensional feature vector.

8. The method of claim 5 or 6, wherein the method further comprises: In the feature fusion process of Step 3, node embedding vectors are generated and neighbor features are aggregated, and then concatenated with device status and log semantic vectors.

9. The network fault automatic locating method based on multi-source data fusion and artificial intelligence algorithm according to claim 5 or 6, characterized in that, In the feature fusion process of Step 3, key features of network health status are extracted through a spatio-temporal correlation matrix.

10. The method of claim 5 or 6, wherein the method is characterized by, In the intelligent diagnosis and positioning process of Step 4, fault points are dynamically and accurately located through reinforcement learning in combination with current network topology, device status, and historical fault information.

Images