Network flow intelligent optimization method and system
By constructing a traffic prediction and anomaly detection model based on machine learning algorithms, the real-time monitoring and resource scheduling problems of existing network traffic management systems are solved, network resource optimization and problem early warning are realized, and network service quality and operational efficiency are improved.
Patent Information
- Application Number
- CN202510911265.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-07-02
- Publication Date
- 2025-11-14
AI Technical Summary
Existing network traffic management systems suffer from insufficient real-time monitoring capabilities, low levels of intelligent data analysis, rigid resource scheduling strategies, and ineffective early warning mechanisms, resulting in low network resource utilization and poor service quality.
By constructing a traffic prediction and anomaly detection model using machine learning algorithms, the collected traffic data is analyzed in real time to generate early warning information and automatically execute network optimization measures, including traffic redirection and bandwidth adjustment.
It enables real-time data collection, processing, and analysis of network traffic, improving the utilization rate of network resources and service quality, and enhancing operational efficiency.
Smart Images
Figure CN120956601A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of network communication technology, and in particular to a method and system for intelligent network traffic optimization. Background Technology
[0002] With the rapid development of 5G communication, IoT technology, and mobile internet applications, global network traffic is growing exponentially, posing a severe challenge to the carrying capacity and resource scheduling capabilities of existing network infrastructure. In traditional network management systems, operators mainly rely on fixed threshold alarm mechanisms and manual experience for traffic management, which suffers from problems such as delayed response and rigid resource allocation.
[0003] Current network traffic management solutions commonly used in the industry have the following technical shortcomings:
[0004] (1) Insufficient real-time monitoring capability: Existing systems mostly adopt periodic polling mechanisms (such as 5-10 minute sampling intervals), which makes it difficult to capture millisecond-level traffic mutation events in a timely manner, resulting in an anomaly identification accuracy rate of less than 75%;
[0005] (2) Low level of intelligence in data analysis: Static analysis mode based on rule engine cannot effectively handle dynamic traffic characteristics in complex network scenarios, especially lacking predictive decision-making ability when dealing with sudden traffic peaks;
[0006] (3) Fixed resource scheduling strategy: The existing SDN controller mainly uses the preset QoS strategy for bandwidth allocation, and lacks a dynamic resource adjustment mechanism based on service priority, resulting in network resource utilization being lower than 60% for a long time.
[0007] (4) Insufficient effectiveness of early warning mechanism: The false alarm rate of traditional SNMP-based monitoring systems is as high as 30%-40%, and the early warning information lacks effective correlation with operation and maintenance decisions. Summary of the Invention
[0008] To address the aforementioned issues, this invention provides a method and system for intelligent network traffic optimization. It utilizes a traffic prediction and anomaly detection model built using machine learning algorithms to detect traffic anomalies in collected traffic data and automatically executes network optimization measures based on the anomaly status. This enables real-time data collection, processing, and analysis of network traffic, achieving network resource optimization and problem early warning, and efficiently and accurately improving network service quality and operational efficiency.
[0009] To achieve the above objectives, the present invention provides a method for intelligent network traffic optimization, comprising:
[0010] Collect traffic data for the network devices to be monitored;
[0011] The traffic data is cleaned and format converted, and key feature data is extracted from the traffic data.
[0012] A traffic prediction and anomaly detection model is obtained by training a labeled training dataset based on machine learning algorithms.
[0013] The key feature data is input into the traffic prediction and anomaly detection model. Based on the traffic anomaly situation corresponding to the model analysis results, early warning information is generated, and preset network optimization measures are automatically executed based on the early warning information.
[0014] In the above technical solution, preferably, the process of collecting traffic data for the network device to be monitored specifically includes:
[0015] Data acquisition tools are used to collect traffic data from the network devices to be monitored through corresponding interfaces and protocols;
[0016] The traffic data is encrypted and transmitted using the TLS encryption algorithm. The traffic data includes traffic statistics, device status, and user behavior data.
[0017] In the above technical solution, preferably, the traffic data undergoes data cleaning and format conversion processing, and key feature data is extracted from the traffic data. The specific process includes:
[0018] The traffic data is cleaned using Apache Spark or Apache Flink.
[0019] The cleaned traffic data is formatted in a unified format and stored using HDFS or Apache Hive.
[0020] The stored traffic data is analyzed, and key feature data is extracted from it, including traffic peaks and packet sizes.
[0021] In the above technical solution, preferably, the machine learning algorithm is trained using a labeled training dataset to obtain a traffic prediction and anomaly detection model, and the specific process includes:
[0022] Based on machine learning frameworks such as TensorFlow or PyTorch, we use Long Short-Term Memory (LSTM) networks for time series prediction, Isolation Forest or One-Class SVM for detecting and identifying abnormal traffic patterns, and K-means for clustering traffic data to identify traffic patterns, thus constructing a traffic prediction and anomaly detection model.
[0023] The traffic prediction and anomaly detection model is trained using a pre-labeled training dataset until a preset convergence level is reached.
[0024] In the above technical solution, preferably, the key feature data is input into the traffic prediction and anomaly detection model, and early warning information is generated based on the traffic anomaly situation corresponding to the model analysis results. Pre-set network optimization measures are then automatically executed based on the early warning information. The specific process includes:
[0025] The key feature data is input into the traffic prediction and anomaly detection model. If the model outputs that there is a traffic anomaly, an early warning message is generated based on the traffic anomaly and an adaptive early warning algorithm.
[0026] Based on the warning information and preset thresholds and response strategies, preset network optimization measures are automatically executed via REST API, including traffic redirection and bandwidth adjustment.
[0027] Send alert information to preset users via email, SMS and / or system notification.
[0028] In the above technical solution, preferably, the intelligent network traffic optimization method further includes:
[0029] Generate visual charts or reports based on the analysis results of the model;
[0030] Data filtering, data querying, and report customization can be performed based on the charts or reports.
[0031] This invention also proposes a network traffic intelligent optimization system, which applies the network traffic intelligent optimization method disclosed in any of the above technical solutions, including:
[0032] The data acquisition module is used to collect traffic data from the network devices to be monitored.
[0033] The data processing module is used to perform data cleaning and format conversion on the traffic data, and to extract key feature data from the traffic data.
[0034] The model training module is used to train a traffic prediction and anomaly detection model based on a labeled training dataset using machine learning algorithms.
[0035] The anomaly detection module is used to input the key feature data into the traffic prediction and anomaly detection model, generate early warning information based on the traffic anomaly situation corresponding to the model analysis results, and automatically execute preset network optimization measures based on the early warning information.
[0036] In the above technical solution, preferably, the data processing module is specifically used for:
[0037] The traffic data is cleaned using Apache Spark or Apache Flink.
[0038] The cleaned traffic data is formatted in a unified format and stored using HDFS or Apache Hive.
[0039] The stored traffic data is analyzed, and key feature data is extracted from it, including traffic peaks and packet sizes.
[0040] In the above technical solution, preferably, the anomaly detection module is specifically used for:
[0041] The key feature data is input into the traffic prediction and anomaly detection model. If the model outputs that there is a traffic anomaly, an early warning message is generated based on the traffic anomaly and an adaptive early warning algorithm.
[0042] Based on the warning information and preset thresholds and response strategies, preset network optimization measures are automatically executed via REST API, including traffic redirection and bandwidth adjustment.
[0043] Send alert information to preset users via email, SMS and / or system notification.
[0044] In the above technical solution, preferably, the network traffic intelligent optimization system further includes a result display module, specifically used for:
[0045] Generate visual charts or reports based on the analysis results of the model;
[0046] Data filtering, data querying, and report customization can be performed based on the charts or reports.
[0047] Compared with the prior art, the beneficial effects of the present invention are as follows: the traffic prediction and anomaly detection model constructed by machine learning algorithm performs traffic anomaly detection on the collected traffic data, and automatically executes network optimization measures according to the abnormal state, realizing real-time data collection, processing and analysis of network traffic, realizing network resource optimization and problem early warning, and can efficiently and accurately improve network service quality and operational efficiency. Attached Figure Description
[0048] Figure 1 This is a flowchart illustrating a network traffic intelligent optimization method disclosed in an embodiment of the present invention. Detailed Implementation
[0049] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0050] The present invention will now be described in further detail with reference to the accompanying drawings:
[0051] like Figure 1 As shown, a network traffic intelligent optimization method according to the present invention includes:
[0052] Collect traffic data for the network devices to be monitored;
[0053] The traffic data is cleaned and format converted, and key feature data is extracted from the traffic data.
[0054] A traffic prediction and anomaly detection model is obtained by training a labeled training dataset based on machine learning algorithms.
[0055] Key feature data is input into the traffic prediction and anomaly detection model. Based on the traffic anomaly situation corresponding to the model analysis results, early warning information is generated, and preset network optimization measures are automatically executed based on the early warning information.
[0056] In this implementation, a traffic prediction and anomaly detection model constructed using machine learning algorithms is used to detect traffic anomalies in the collected traffic data. Based on the abnormal state, network optimization measures are automatically executed, enabling real-time data collection, processing, and analysis of network traffic. This achieves network resource optimization and problem early warning, and can efficiently and accurately improve network service quality and operational efficiency.
[0057] Specifically, network devices include routers, switches, and base stations, and data collection is achieved by installing data acquisition agents or sensors on these network devices.
[0058] In the above embodiments, preferably, traffic data is collected for the network devices to be monitored, and the specific process includes:
[0059] Data acquisition tools are used to collect traffic data from the network devices to be monitored through corresponding interfaces and protocols;
[0060] Traffic data is transmitted in encrypted form using the TLS encryption algorithm. This traffic data includes traffic statistics, device status, and user behavior data.
[0061] Specifically, the data acquisition tools include Prometheus or commercial platforms, the protocols include SNMP, NetFlow, and sFlow, and the data transmission uses TLS encryption to ensure data security.
[0062] In the above embodiments, preferably, the traffic data undergoes data cleaning and format conversion, and key feature data is extracted from the traffic data. The specific process includes:
[0063] The data stream is processed using the Apache Kafka streaming framework, and real-time data processing is performed using Apache Flink.
[0064] Use Apache Spark or Apache Flink to clean and process traffic data;
[0065] The cleaned traffic data is formatted in a unified way and stored using HDFS or Apache Hive.
[0066] The stored traffic data is analyzed, and key feature data is extracted from it, including traffic peaks and packet sizes.
[0067] In the above embodiments, preferably, a traffic prediction and anomaly detection model is obtained by training a labeled training dataset based on a machine learning algorithm. The specific process includes:
[0068] Based on machine learning frameworks such as TensorFlow or PyTorch, the Long Short-Term Memory (LSTM) network is used for time series prediction to predict traffic trends. Isolation Forest or One-Class SVM is used to detect and identify abnormal traffic patterns. K-means is used to cluster traffic data to identify traffic patterns, thus constructing a traffic prediction and anomaly detection model.
[0069] The traffic prediction and anomaly detection models are trained using a pre-labeled training dataset until a preset convergence level is reached.
[0070] In the above implementation, preferably, key feature data is input into the traffic prediction and anomaly detection model, and early warning information is generated based on the traffic anomaly situation corresponding to the model analysis results. Pre-set network optimization measures are then automatically executed based on the early warning information. The specific process includes:
[0071] Key feature data are input into the traffic prediction and anomaly detection model. If the model output indicates that there is an abnormal traffic situation, an early warning message is generated based on the traffic anomaly and an adaptive early warning algorithm.
[0072] Based on the early warning information and the preset dynamic thresholds and response strategies, the system automatically executes preset network optimization measures via REST API. These network optimization measures include traffic redirection and bandwidth adjustment.
[0073] Send alert information to preset users via email, SMS and / or system notification.
[0074] In the above embodiments, preferably, the intelligent network traffic optimization method further includes:
[0075] The user interface is implemented using the Vue.js or React front-end framework, and data visualization is built using the D3.js or Echarts data visualization library;
[0076] Based on the model analysis results, use JasperReports or Apache Superset reporting tools to generate custom charts or reports for visualization.
[0077] Data filtering, data querying, and report customization are based on charts or reports.
[0078] This invention also proposes a network traffic intelligent optimization system, which applies the network traffic intelligent optimization method disclosed in any of the above embodiments, including:
[0079] The data acquisition module is used to collect traffic data from the network devices to be monitored.
[0080] The data processing module is used to clean and convert the traffic data, and extract key feature data from the traffic data.
[0081] The model training module is used to train a traffic prediction and anomaly detection model based on a labeled training dataset using machine learning algorithms.
[0082] The anomaly detection module is used to input key feature data into the traffic prediction and anomaly detection model, generate early warning information based on the traffic anomalies corresponding to the model analysis results, and automatically execute preset network optimization measures based on the early warning information.
[0083] In the above embodiments, preferably, the data processing module is specifically used for:
[0084] Use Apache Spark or Apache Flink to clean and process traffic data;
[0085] The cleaned traffic data is formatted in a unified way and stored using HDFS or Apache Hive.
[0086] The stored traffic data is analyzed, and key feature data is extracted from it, including traffic peaks and packet sizes.
[0087] In the above embodiments, preferably, the anomaly detection module is specifically used for:
[0088] Key feature data are input into the traffic prediction and anomaly detection model. If the model output indicates that there is an abnormal traffic situation, an early warning message is generated based on the traffic anomaly and an adaptive early warning algorithm.
[0089] Based on the early warning information and preset thresholds and response strategies, preset network optimization measures are automatically executed via REST API. These network optimization measures include traffic redirection and bandwidth adjustment.
[0090] Send alert information to preset users via email, SMS and / or system notification.
[0091] In the above embodiments, preferably, the network traffic intelligent optimization system further includes a result display module, specifically used for:
[0092] Generate visual charts or reports based on the model analysis results;
[0093] Data filtering, data querying, and report customization are based on charts or reports.
[0094] The network traffic intelligent optimization system disclosed in the above embodiments has modules whose functions correspond to the steps of the network traffic intelligent optimization method disclosed in the above embodiments. During implementation, the above embodiments are referred to for operation, and will not be repeated here.
[0095] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.
Claims
1. A method for intelligent optimization of network traffic, characterized in that, include: Collect traffic data for the network devices to be monitored; The traffic data is cleaned and format converted, and key feature data is extracted from the traffic data. A traffic prediction and anomaly detection model is obtained by training a labeled training dataset based on machine learning algorithms. The key feature data is input into the traffic prediction and anomaly detection model. Based on the traffic anomaly situation corresponding to the model analysis results, early warning information is generated, and preset network optimization measures are automatically executed based on the early warning information.
2. The intelligent network traffic optimization method according to claim 1, characterized in that, The specific process of collecting traffic data from the network devices to be monitored includes: Data acquisition tools are used to collect traffic data from the network devices to be monitored through corresponding interfaces and protocols; The traffic data is encrypted and transmitted using the TLS encryption algorithm. The traffic data includes traffic statistics, device status, and user behavior data.
3. The intelligent network traffic optimization method according to claim 1, characterized in that, The traffic data undergoes data cleaning and format conversion, and key feature data is extracted from the traffic data. The specific process includes: The traffic data is cleaned using Apache Spark or Apache Flink. The cleaned traffic data is formatted in a unified format and stored using HDFS or Apache Hive. The stored traffic data is analyzed, and key feature data is extracted from it, including traffic peaks and packet sizes.
4. The intelligent network traffic optimization method according to claim 1, characterized in that, The machine learning algorithm is trained using a labeled training dataset to obtain a traffic prediction and anomaly detection model. The specific process includes: Based on machine learning frameworks such as TensorFlow or PyTorch, we use Long Short-Term Memory (LSTM) networks for time series prediction, Isolation Forest or One-Class SVM for detecting and identifying abnormal traffic patterns, and K-means for clustering traffic data to identify traffic patterns, thus constructing a traffic prediction and anomaly detection model. The traffic prediction and anomaly detection model is trained using a pre-labeled training dataset until a preset convergence level is reached.
5. The intelligent network traffic optimization method according to claim 1, characterized in that, The key feature data is input into the traffic prediction and anomaly detection model. Based on the traffic anomaly analysis results, early warning information is generated, and preset network optimization measures are automatically executed based on the early warning information. The specific process includes: The key feature data is input into the traffic prediction and anomaly detection model. If the model outputs that there is a traffic anomaly, an early warning message is generated based on the traffic anomaly and an adaptive early warning algorithm. Based on the warning information and preset thresholds and response strategies, preset network optimization measures are automatically executed via REST API, including traffic redirection and bandwidth adjustment. Send alert information to preset users via email, SMS and / or system notification.
6. The intelligent network traffic optimization method according to any one of claims 1 to 5, characterized in that, Also includes: Generate visual charts or reports based on the analysis results of the model; Data filtering, data querying, and report customization can be performed based on the charts or reports.
7. A network traffic intelligent optimization system, characterized in that, The network traffic intelligent optimization method as described in any one of claims 1 to 6 includes: The data acquisition module is used to collect traffic data from the network devices to be monitored. The data processing module is used to perform data cleaning and format conversion on the traffic data, and to extract key feature data from the traffic data. The model training module is used to train a traffic prediction and anomaly detection model based on a labeled training dataset using machine learning algorithms. The anomaly detection module is used to input the key feature data into the traffic prediction and anomaly detection model, generate early warning information based on the traffic anomaly situation corresponding to the model analysis results, and automatically execute preset network optimization measures based on the early warning information.
8. The intelligent network traffic optimization system according to claim 7, characterized in that, The data processing module is specifically used for: The traffic data is cleaned using Apache Spark or Apache Flink. The cleaned traffic data is formatted in a unified format and stored using HDFS or Apache Hive. The stored traffic data is analyzed, and key feature data is extracted from it, including traffic peaks and packet sizes.
9. The intelligent network traffic optimization system according to claim 8, characterized in that, The anomaly detection module is specifically used for: The key feature data is input into the traffic prediction and anomaly detection model. If the model outputs that there is a traffic anomaly, an early warning message is generated based on the traffic anomaly and an adaptive early warning algorithm. Based on the warning information and preset thresholds and response strategies, preset network optimization measures are automatically executed via REST API, including traffic redirection and bandwidth adjustment. Send alert information to preset users via email, SMS and / or system notification.
10. The intelligent network traffic optimization system according to any one of claims 7 to 9, characterized in that, It also includes a results display module, specifically used for: Generate visual charts or reports based on the analysis results of the model; Data filtering, data querying, and report customization can be performed based on the charts or reports.