Transmission network connectivity detection method based on deep learning
By building a multi-layer recurrent neural network model based on deep learning, real-time detection and update network connectivity is solved, the problems of low efficiency and high false alarm rate of traditional detection methods are solved, real-time perception of network status and accurate positioning of fault points are achieved, and operation and maintenance costs are reduced.
Patent Information
- Application Number
- CN202510593504.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-05-09
- Publication Date
- 2025-07-29
AI Technical Summary
Traditional network connectivity detection methods are inefficient and have high false alarm rates, making it difficult to detect and locate faults in a complex computer network environment in a timely manner.
Using a deep learning-based method, a multi-layer recurrent neural network model is built by collecting and preprocessing communication data packets, a multi-layer recurrent neural network model is detected in real time, and a model update and retraining is carried out to realize real-time perception and prediction of network state.
It improves the accuracy of detection, reduces network operation and maintenance costs, can quickly locate fault points, and adapt to various complex network environments.
Smart Images
Figure CN120389967A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of computer networks, and particularly to a method for detecting the connectivity of a transmission network based on deep learning. Background Art
[0002] With the rapid development of communication technologies, the scale and complexity of transmission networks have been continuously increasing, and the detection of network connectivity has become an important link to ensure the stable operation of the network. Traditional network connectivity detection methods mostly rely on manual inspections or simple protocol detections, which have problems such as low detection efficiency and high false alarm rates. In recent years, deep learning technologies have achieved remarkable results in fields such as natural language processing, enabling deep learning technologies to learn in a constantly changing and relatively complex computer network environment, thereby detecting the connectivity of the transmission network and locating network problems in a timely manner.
[0003] To overcome the traditional network connectivity problems, the present invention proposes a method for detecting the connectivity of a transmission network based on deep learning. Summary of the Invention
[0004] The present invention provides a simple and efficient method for detecting the connectivity of a transmission network based on deep learning to make up for the deficiencies of the prior art.
[0005] The present invention is realized through the following technical solutions:
[0006] A method for detecting the connectivity of a transmission network based on deep learning, comprising the following steps:
[0007] Step S1, collecting communication data packets
[0008] Using a network monitoring tool, capturing and collecting the plaintext data of network communication of network devices, i.e., the communication data between nodes;
[0009] The plaintext data of network communication of network devices includes the size, longitude, latitude, transmission time, device communication status, source address, and destination address information of the data packets;
[0010] The communication data between nodes includes the communication status of network nodes, the link relationship between nodes, and the traffic;
[0011] In the said step S1, an SNMP protocol network monitoring tool is used to capture and collect communication data packets from the network.
[0012] Step S2, preprocessing the communication data packets
[0013] Cleaning the collected communication data packets, slicing the communication data packets using a sliding window technique, and reducing the number of features for the extracted feature data using a dimensionality reduction technique;
[0014] In step S2, the collected communication data packets are pre-processed as follows:
[0015] S2.1. Clean the collected communication data packets. Use interpolation algorithms to repair and adjust numerical data containing errors and irregularities. Convert the raw network traffic data into a custom format suitable for subsequent analysis or model processing, ensuring that the data meets the input requirements of the recurrent neural network model (RNN).
[0016] S2.2. Use sliding window technology to process communication data packets, slicing them according to a custom, unified window size, and extracting time-related features from the slices that reflect the patterns and changes in network traffic, including communication status and inter-node link information.
[0017] S2.3. For the extracted feature data, dimensionality reduction technology is used to reduce the number of features and reduce the computational complexity, while retaining the information of the original data as much as possible to reduce the complexity of the model calculation.
[0018] In step S2.3, principal component analysis (PCA) or linear discriminant analysis (LDA) is used to reduce the number of features.
[0019] Step S3: Deep learning model construction and training
[0020] Select the recurrent neural network model (RNN) suitable for time series data processing and build a multi-layer neural network, where each layer is responsible for learning the features of different levels of data;
[0021] S3.1. Select a recurrent neural network (RNN) model suitable for time series data processing. When constructing the model, take preprocessed feature data, including communication status and inter-node link information, as input and output information reflecting the network connectivity status.
[0022] S3.2. Build a multi-layer neural network, where each layer is responsible for learning features at different levels of the data.
[0023] S3.2.1. Before building a multi-layer neural network, initialize the network parameters, including weights and biases.
[0024] Customize a basic universal weight value and bias number so that the network model can run initially;
[0025] S3.2.2. Quantify specific tasks and data characteristics in advance, design network layers that meet current feature requirements, select the Bayesian optimization algorithm for the hidden layer, and customize the appropriate probability model based on the currently selected feature information;
[0026] S3.2.3. Use the collected historical data to train the deep learning model, and continuously adjust the model parameters and structure during the training process to minimize the prediction error of the network and improve the prediction accuracy and performance of the model.
[0027] In step S3.2, the PReLU activation function is selected for the deep learning model. By using the characteristic that the negative interval of the PReLU activation function can also learn, the parameters are automatically adjusted according to different network characteristics, so as to select the optimal value during the learning process.
[0028] Step S4. Real-time transmission network connectivity detection
[0029] Deploy the trained transmission network connectivity model to the nodes in the network, and obtain relevant network information at each node;
[0030] The deep learning model generates local node information data of the current network according to the path data of each node in the current network, synchronizes each node to the database and marks the relevant status, and displays it to the network maintenance personnel through a large screen;
[0031] The network maintenance personnel monitor the network information in real time according to the status and location information and perform maintenance;
[0032] Step S5. Model update and retraining
[0033] Collect new network information data, and repeat the processing operations of steps S1 to S3 for the newly collected data, and then merge it with the original data set to form a larger data for retraining and optimizing the model;
[0034] During the retraining process, use the model parameters obtained from the previous training as the initial values to accelerate the training process of the new model and improve its performance.
[0035] A transmission network connectivity detection system based on deep learning for implementing the above method, including a communication data packet collection module, a communication data packet preprocessing module, a deep learning model construction and training module, a real-time transmission network connectivity detection module, and a model update and retraining module;
[0036] The communication data packet collection module is responsible for using network monitoring tools to capture and collect the clear text data of network communication of network devices, that is, the communication data between nodes;
[0037] The clear text data of network communication of network devices includes the size, longitude, latitude, transmission time, device communication status, source address, and destination address information of the data packet;
[0038] The communication data between nodes includes the communication status of network nodes, the link relationship between nodes, and the traffic;
[0039] The communication data packet preprocessing module is responsible for cleaning the collected communication data packets, slicing the communication data packets using the sliding window technique, and reducing the number of features for the extracted feature data using the dimensionality reduction technique;
[0040] The deep learning model construction and training module is responsible for selecting the recurrent neural network model RNN suitable for time series data processing, constructing a multi-layer neural network, and each layer of the network is responsible for learning different levels of features of the data;
[0041] The real-time transmission network connectivity detection module is responsible for deploying the trained transmission network connectivity model to the nodes in the network and obtaining relevant network information at each node;
[0042] The deep learning model generates the local node information data of the current network according to the path data of each node in the current network, synchronizes each node to the database and marks the relevant status, and displays it to the network maintenance personnel through a large screen;
[0043] The network maintenance personnel monitor the network information in real time according to the status and location information and perform maintenance;
[0044] The model update and retraining module is responsible for collecting new network information data, and calling the communication data packet collection module, the communication data packet preprocessing module, and the deep learning model construction and training module to process the newly collected data, and then merging it with the original data set to form a larger data for the retraining and optimization of the model;
[0045] During the retraining process, the model parameters obtained from the previous training are used as the initial values to accelerate the training process of the new model and improve its performance.
[0046] A transmission network connectivity detection device based on deep learning, characterized in that: it includes a memory and a processor; the memory is used to store a computer program, and the processor is used to implement the above method steps when executing the computer program.
[0047] A readable storage medium, characterized in that: a computer program is stored on the readable storage medium, and the computer program implements the above method steps when executed by a processor.
[0048] The beneficial effects of the present invention are: the transmission network connectivity detection method based on deep learning realizes the real-time perception and prediction of the network state through deep learning technology, can dynamically obtain the state information of the current network, thereby detecting the connectivity of the current network, timely discovering fault information and locating the fault location, reducing the network operation and maintenance cost, and being able to cope with various complex network environments. Description of the Drawings
[0049] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required in the description of the embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.
[0050] Figure 1 It is a schematic diagram of the transmission network connectivity detection system based on deep learning of the present invention. Specific embodiments
[0051] In order to enable those skilled in the art of this technology to better understand the technical solutions in the present invention, the following will clearly and completely describe the technical solutions in the embodiments of the present invention in combination with the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the scope of protection of the present invention.
[0052] The transmission network connectivity detection method based on deep learning includes the following steps:
[0053] Step S1, collect communication data packets
[0054] Use a network monitoring tool to capture and collect the clear text data of network communication of network devices, that is, the communication data between nodes;
[0055] The clear text data of network communication of network devices includes the size, longitude, latitude, transmission time, device communication status, source address, and target address information of the data packet;
[0056] The communication data between nodes includes the communication status of network nodes, the link relationship between nodes, and the traffic;
[0057] In the above step S1, use the SNMP protocol network monitoring tool to capture and collect communication data packets from the network.
[0058] It can also be collected through other network monitoring tools, and historical data and real-time data can be combined for comprehensive analysis to more accurately predict the network status.
[0059] Step S2, preprocess the communication data packets
[0060] Clean the collected communication data packets, use the sliding window technology to slice the communication data packets, and use the dimensionality reduction technology to reduce the number of features for the extracted feature data;
[0061] In step S2, the collected communication data packets are pre-processed as follows:
[0062] S2.1. Clean the collected communication data packets. Use interpolation algorithms to repair and adjust numerical data containing errors and irregularities. Convert the raw network traffic data into a custom format suitable for subsequent analysis or model processing, ensuring that the data meets the input requirements of the recurrent neural network model (RNN).
[0063] S2.2. Use sliding window technology to process communication data packets, slicing them according to a custom, unified window size, and extracting time-related features from the slices that reflect the patterns and changes in network traffic, including communication status and inter-node link information.
[0064] S2.3. For the extracted feature data, dimensionality reduction technology is used to reduce the number of features and reduce the computational complexity, while retaining the information of the original data as much as possible to reduce the complexity of the model calculation.
[0065] In step S2.3, principal component analysis (PCA) or linear discriminant analysis (LDA) is used to reduce the number of features.
[0066] Step S3: Deep learning model construction and training
[0067] Select the recurrent neural network model (RNN) suitable for time series data processing and build a multi-layer neural network, where each layer is responsible for learning the features of different levels of data;
[0068] In step S3, the deep learning model is constructed and trained as follows:
[0069] S3.1. Select a recurrent neural network (RNN) model suitable for time series data processing. When constructing the model, take preprocessed feature data, including communication status and inter-node link information, as input and output information reflecting the network connectivity status.
[0070] S3.2. Build a multi-layer neural network, where each layer is responsible for learning features at different levels of the data.
[0071] S3.2.1. Before building a multi-layer neural network, initialize the network parameters, including weights and biases.
[0072] Customize a basic universal weight value and bias number so that the network model can run initially;
[0073] S3.2.2. Quantify specific tasks and data characteristics in advance, design a network layer that meets the current feature requirements. Since the hidden layer has non-linear transformation and the complexity of the actual communication network is too high, the Bayesian optimization algorithm is selected, and a suitable probability model is custom-built according to the currently selected feature information;
[0074] S3.2.3. Use the collected historical data to train the deep learning model, and continuously adjust the model parameters and structure during the training process to minimize the prediction error of the network, improve the prediction accuracy and performance of the model.
[0075] In step S3.2, due to the high complexity of the actual network, the PReLU activation function is selected for the deep learning model. Utilizing the characteristic that the negative interval of the PReLU activation function can also learn, the parameters are automatically adjusted according to different network characteristics, so as to select the optimal value during the learning process.
[0076] Step S4. Real-time transmission network connectivity detection
[0077] Deploy the trained transmission network connectivity model to the nodes in the network, such as computer rooms. Obtain relevant network information at each node;
[0078] The deep learning model generates local node information data of the current network based on the path data of each node in the current network, synchronizes each node to the database and marks the relevant status, and displays it to the network maintenance personnel through a large screen;
[0079] The network maintenance personnel monitor the network information in real time according to the status and location information and perform maintenance;
[0080] Step S5. Model update and retraining
[0081] Due to the continuous change and complexity of the actual transmission network, it is necessary to change the data of the transmission network connectivity model and retrain and optimize the model.
[0082] Collect new network information data, and repeat the processing operations of steps S1 to S3 for the newly collected data, and then merge it with the original data set to form a larger data set for the retraining and optimization of the model;
[0083] During the retraining process, use the model parameters obtained from the previous training as the initial values to accelerate the training process of the new model and improve its performance.
[0084] The transmission network connectivity detection system based on deep learning, used to implement the above method, includes a communication data packet collection module, a communication data packet preprocessing module, a deep learning model construction and training module, a real-time transmission network connectivity detection module, and a model update and retraining module;
[0085] The communication data packet collection module is responsible for using network monitoring tools to capture and collect network device network communication plaintext data and communication data between nodes;
[0086] Network device network communication plaintext data includes data packet size, longitude, latitude, transmission time, device communication status, source address, and destination address information;
[0087] Communication data between nodes includes the communication status of network nodes, the link relationship between nodes, and traffic;
[0088] The communication data packet preprocessing module is responsible for cleaning the collected communication data packets, slicing the communication data packets using a sliding window technique, and using a dimensionality reduction technique to reduce the number of features extracted from the feature data;
[0089] The deep learning model construction and training module is responsible for selecting a recurrent neural network model (RNN) suitable for time series data processing and building a multi-layer neural network, where each layer of the network is responsible for learning features at different levels of the data;
[0090] The real-time transmission network connectivity detection module is responsible for deploying the trained transmission network connectivity model to nodes in the network and obtaining relevant network information at each node;
[0091] The deep learning model generates local node information data of the current network based on the path data of each node in the current network, synchronizes each node to the database and annotates the relevant status, and displays it to network maintenance personnel through the large screen;
[0092] Network maintenance personnel monitor network information in real time and perform maintenance based on status and location information;
[0093] The model update and retraining module is responsible for collecting new network information data and calling the communication data packet collection module, the communication data packet preprocessing module and the deep learning model construction and training module to process the newly collected data, and then merge it with the original data set to form a larger data set for retraining and optimization of the model;
[0094] During the retraining process, the model parameters obtained from the previous training are used as initial values to accelerate the training process of the new model and improve its performance.
[0095] The deep learning-based transmission network connectivity detection device includes a memory and a processor; the memory is used to store a computer program, and the processor is used to implement the above-mentioned method steps when executing the computer program.
[0096] A computer program is stored on the readable storage medium, and when the computer program is executed by a processor, the above-mentioned method steps are implemented.
[0097] Compared with the prior art, the transmission network connectivity detection method based on deep learning has the following characteristics:
[0098] 1). Improved accuracy: By using the transmission network connectivity model to dynamically obtain network information in real time and predict the states of network nodes for early warning information, network operation and maintenance personnel can obtain accurate fault points.
[0099] 2). Reduced network operation and maintenance costs: The present invention can reduce manual intervention and network operation and maintenance costs, quickly locate fault point information, and improve efficiency.
[0100] 3). Enhanced adaptability: The present invention does not depend on a specific network structure or protocol, has a wider applicability, and can cope with various complex network environments.
[0101] The above-described embodiments are only one of the specific implementation manners of the present invention, and the common changes and substitutions made by those skilled in the art within the scope of the technical solution of the present invention should be included in the protection scope of the present invention.
Claims
1. A method for detecting the connectivity of a transmission network based on deep learning, characterized in that: Including the following steps: Step S1: Collect communication data packets Using a network monitoring tool, capture and collect the clear text data of network communication of network devices to obtain the communication data between nodes; The clear text data of network communication of network devices includes the size, longitude, latitude, transmission time, device communication status, source address, and destination address information of the data packet; The communication data between nodes includes the communication status of network nodes, the link relationship between nodes, and the traffic; Step S2: Preprocess the communication data packets Clean the collected communication data packets, slice the communication data packets using the sliding window technique, and use the dimensionality reduction technique to reduce the number of features for the extracted feature data; Step S3: Construct and train a deep learning model Select the recurrent neural network model RNN suitable for processing time series data, construct a multi-layer neural network, and each layer of the network is responsible for learning different levels of features of the data; Step S4: Detect the connectivity of the real-time transmission network Deploy the trained transmission network connectivity model to the nodes in the network, and obtain relevant network information at each node; The deep learning model generates the local node information data of the current network based on the path data of each node in the current network, synchronizes each node to the database and marks the relevant status, and displays it to the network maintenance personnel through a large screen; The network maintenance personnel monitor the network information in real time according to the status and location information and perform maintenance; Step S5: Update and retrain the model Collect new network information data, repeat the processing operations of steps S1 to S3 for the newly collected data, and then merge it with the original data set to form a larger data set for retraining and optimizing the model; During the retraining process, use the model parameters obtained from the previous training as the initial values to accelerate the training process of the new model and improve its performance.
2. The method for detecting the connectivity of a transmission network based on deep learning according to claim 1, wherein: In step S1, an SNMP protocol network monitoring tool is used to capture and collect communication data packets from the network.
3. The method for detecting the connectivity of a transmission network based on deep learning according to claim 1, characterized in that: In step S2, the collected communication data packets are preprocessed, and the steps are as follows: S2.1: Clean the collected communication data packets. For the numerical data containing errors and irregularities, use the interpolation algorithm for repair and adjustment, and convert the original network traffic data into a custom format to make the data meet the input requirements of the recurrent neural network model RNN; S2.2: Use the sliding window technique to process the communication data packets, slice the communication data packets according to the custom unified window size, and extract the features related to time and capable of reflecting the patterns and changes of network traffic in the slice, including the communication status and the link information between nodes; S2.3: For the extracted feature data, use the dimensionality reduction technique to reduce the number of features, reduce the computational complexity, and retain the information of the original data at the same time to reduce the computational complexity of the model.
4. The method for detecting the connectivity of a transmission network based on deep learning according to claim 3, wherein: In step S2.3, the principal component analysis technique PCA or the linear discriminant analysis technique LDA is used to reduce the number of features.
5. The method for detecting the connectivity of a transmission network based on deep learning according to claim 1, wherein: In step S3, the deep learning model is constructed and trained, and the steps are as follows: S3.
1. Select a recurrent neural network model RNN suitable for time-series data processing; when constructing the model, use the preprocessed feature data, including communication status and link information between nodes, as input, and output information reflecting the network connectivity status. S3.
2. Construct a multi-layer neural network, where each layer of the network is responsible for learning different levels of features of the data. S3.2.
1. Before constructing the multi-layer neural network, initialize the parameters of the network, including weights and biases. Customize a basic general weight value and bias number to enable the initial operation of the network model. S3.2.
2. Quantify specific tasks and data characteristics in advance, design network layers that meet the current feature requirements, select the Bayesian optimization algorithm for the hidden layer, and customize and construct a suitable probability model according to the currently selected feature information. S3.2.
3. Use the collected historical data to train the deep learning model, and continuously adjust the model parameters and structure during the training process to minimize the prediction error of the network and improve the prediction accuracy and performance of the model.
6. The method for detecting the connectivity of a transmission network based on deep learning according to claim 5, wherein: In step S3.2, the PReLU activation function is selected for the deep learning model. Utilize the characteristic that the negative interval of the PReLU activation function can also learn, and automatically adjust the parameters according to different network characteristics, so as to select the optimal value during the learning process.
7. A transmission network connectivity detection system based on deep learning, characterized in that: A method for implementing any one of claims 1 to 6, including a communication packet collection module, a communication packet preprocessing module, a deep learning model construction and training module, a real-time transmission network connectivity detection module, and a model update and retraining module. The communication packet collection module is responsible for using network monitoring tools to capture and collect the plaintext data of network communication of network devices, namely the communication data between nodes. The plaintext data of network communication of network devices includes the size of the data packet, longitude, latitude, transmission time, device communication status, source address, and destination address information. The communication data between nodes includes the communication status of network nodes, the link relationship between nodes, and traffic. The communication packet preprocessing module is responsible for cleaning the collected communication packets, slicing the communication packets using the sliding window technique, and reducing the number of features for the extracted feature data using the dimensionality reduction technique. The deep learning model construction and training module is responsible for selecting a recurrent neural network model RNN suitable for time-series data processing and constructing a multi-layer neural network, where each layer of the network is responsible for learning different levels of features of the data. The real-time transmission network connectivity detection module is responsible for deploying the trained transmission network connectivity model to the nodes in the network and obtaining relevant network information at each node. The deep learning model generates the local node information data of the current network based on the path data of each node in the current network, synchronizes each node to the database and marks the relevant status, and displays it to the network maintenance personnel through a large screen. The network maintenance personnel monitor the network information in real time according to the status and location information and perform maintenance. The model update and retraining module is responsible for collecting new network information data, and invoking the communication data packet collection module, the communication data packet preprocessing module, and the deep learning model construction and training module to process the newly collected data, and then merging it with the original data set to form a larger data set for model retraining and optimization; During the retraining process, the model parameters obtained from the previous training are used as initial values to accelerate the training process of the new model and improve its performance.
8. A transmission network connectivity detection device based on deep learning, characterized in that: It includes a memory and a processor; the memory is used to store computer programs, and the processor is used to implement the method according to any one of claims 1 to 6 when executing the computer programs.
9. A readable storage medium, characterized in that: A computer program is stored on the readable storage medium, and when the computer program is executed by the processor, the method according to any one of claims 1 to 6 is implemented.