Network traffic anomaly evaluation and early warning method and system based on artificial intelligence

Abstract

The present invention relates to the technical field of network security assessment, and specifically to an artificial intelligence-based network traffic anomaly assessment and early warning method and system. The method of the present invention comprehensively considers information of multiple dimensions such as the entropy of IP addresses, port number distribution characteristics, protocol traffic proportion, and traffic time series characteristics, and maps these characteristics into a feature matrix, then obtains an initial detection value by generating a Boolean matrix. Different operations are performed according to a comparison result between the initial detection value and a first value. When it is determined based on the initial detection value that there is no network traffic anomaly, a network traffic dynamic graph structure is constructed and segmented and analyzed, the shortest path changes between different nodes are obtained, and path change deviation detection is performed, thereby determining network traffic anomaly. Finally, different network anomaly early warning measures are taken according to different traffic anomaly levels, thereby improving the intelligence and accuracy of network traffic anomaly assessment, as well as the timeliness and flexibility of network anomaly early warning.

Description

Technical Field

[0001] The present invention relates to the technical field of network security assessment, and specifically to an artificial intelligence-based network traffic anomaly assessment and early warning method and system. Background Art

[0002] Network traffic often harbors various potential security threats, such as hacker attacks, virus transmission, and malware intrusion. Real-time monitoring and anomaly assessment of network traffic can promptly detect these anomalies, provide early warnings, and implement appropriate protective measures to prevent security incidents such as sensitive information leaks and system crashes, thereby protecting the data and assets on the network.

[0003] Modern cyberattacks are increasingly complex and diverse, often employing a combination of techniques and methods to evade traditional detection mechanisms. Manual evaluation requires significant time and effort to analyze network traffic data. Faced with massive amounts of data, manual processing is slow, making it difficult to promptly detect hidden anomalies. This can lead to delayed detection of issues and delays in optimal action. Furthermore, differences in professional background, experience, and personal judgment among different evaluators can lead to subjective and inconsistent evaluation results, impacting their accuracy and reliability. Summary of the Invention

[0004] The purpose of the present invention is to provide a network traffic anomaly evaluation and early warning method and system based on artificial intelligence.

[0005] The technical solutions of the present invention are as follows:

[0006] An artificial intelligence-based network traffic anomaly assessment and early warning method includes the following operations:

[0007] S1. Periodically obtain network traffic data of edge servers within a preset time range, where the network traffic data includes source IP information, destination IP information, port number information, and protocol information; and clean the network traffic data to obtain network traffic data to be processed.

[0008] S2. Based on the network traffic data to be processed, obtain the entropy of the IP address, the port number distribution characteristics, the protocol traffic ratio, and the traffic time series characteristics, map them into a feature matrix, and perform normalization processing to obtain a network traffic feature matrix; based on the method of generating a Boolean matrix, obtain an initial detection value of the network traffic feature matrix; if the initial detection value is a first value, execute S3; if the initial detection value is greater than the first value, the network traffic of the edge server is abnormal, and based on the network traffic feature matrix, obtain the traffic anomaly level, and issue a corresponding network anomaly warning according to the traffic anomaly level;

[0009] S3. Based on the network traffic data to be processed, a network traffic dynamic graph structure with several nodes as source IP, destination IP, port number and protocol is constructed; the network traffic dynamic graph structure is divided according to the preset time window, and arranged in chronological order to obtain several network traffic subgraph structures; based on all network traffic subgraph structures, the shortest path changes between the source IP node and the destination IP node, the shortest path changes between the source IP node and the protocol node, and the shortest path changes between the port number node and the protocol node are obtained, and the respective path deviations are obtained through path change deviation detection; if there is a path deviation greater than the corresponding path deviation threshold, the edge server network traffic is abnormal; based on all path deviations, the traffic anomaly level is obtained, and corresponding network anomaly warnings are issued according to the traffic anomaly level.

[0010] The frequency of obtaining network traffic data within the preset time range in S1 is based on the traffic load of the edge server and the evaluation requirements. The specific calculation formula is as follows:

[0011]

[0012] f is the frequency of obtaining network traffic data, α is the traffic load factor, U is the average value of data packets, R is the required resolution for evaluation, P is the data processing capacity of the edge server, and λ is the load rate of the edge server.

[0013] The specific method for obtaining the traffic time series characteristics in S2 is as follows: after grouping the respective traffic data according to the source IP, destination IP, port number and protocol, a multidimensional data structure is created; in the multidimensional data structure, each grouped traffic data is arranged in chronological order to form several traffic time series; each traffic time series is standardized to obtain several traffic time standardized series; based on the preset time series length, each traffic time standardized series is decomposed into a combination of sine and cosine waves of different frequencies to obtain several traffic complex number arrays; based on the several traffic complex number arrays, a spectrum diagram is drawn to obtain the statistical characteristics of the spectrum diagram to obtain the traffic time series characteristics.

[0014] The entropy of the IP address in S2 is obtained based on the probability distribution of the number of occurrences of the IP address in the network traffic data to be processed. The specific calculation formula is as follows:

[0015]

[0016] H is the entropy of the IP address, N i is the number of occurrences of the i-th IP address, and n is the total number of IP addresses.

[0017] In S1, data cleaning processing includes removing noise and duplicate data from the data, and supplementing missing values; when network data is missing, the network data is interpolated, and the interpolation processing operation is specifically as follows: the data other than the missing values ​​in the network data is used as samples, and the samples are input into the training decision tree for traversal, starting from the root node of the decision tree, and the corresponding branch is selected at each node according to the characteristic value of the sample until the leaf node is reached; if the leaf node stores the predicted label of the sample, the predicted label is used as the interpolation result of the missing value; if the leaf node stores numerical statistical information, the median is used as the interpolation result.

[0018] The path offset in S3 is obtained by the following formula:

[0019]

[0020] Q is the path deviation, β is the path node deviation weight, P1 is the node set of path 1, P2 is the node set of path 2, γ is the path length deviation weight, L1 is the length of path 1, L2 is the length of path 2, and J is the total number of paths.

[0021] The specific operations for issuing corresponding network anomaly warnings based on the traffic anomaly level are as follows: if the traffic anomaly level is a high-risk level, network blocking measures are taken for the edge server; if the traffic anomaly level is a medium-risk level, the network traffic rate of the edge server is limited to the traffic rate threshold; if the traffic anomaly level is a low-risk level, the source IP, destination IP, port number or protocol with a reputation value less than the reputation value threshold is deleted.

[0022] An artificial intelligence-based network traffic anomaly evaluation and early warning system, used to implement the above-mentioned artificial intelligence-based network traffic anomaly evaluation and early warning method, includes:

[0023] The module for generating network traffic data to be processed is used to periodically obtain network traffic data of edge servers within a preset time range. The network traffic data includes source IP information, destination IP information, port number information, and protocol information. The network traffic data is cleansed to obtain the network traffic data to be processed.

[0024] The initial detection and network anomaly warning module is used to obtain the entropy of the IP address, port number distribution characteristics, protocol traffic proportion, and traffic time series characteristics based on the network traffic data to be processed, map them into a feature matrix, and then perform normalization processing to obtain a network traffic feature matrix; based on the method of generating a Boolean matrix, obtain the initial detection value of the network traffic feature matrix; if the initial detection value is a first value, execute the refined detection and network anomaly warning module; if the initial detection value is greater than the first value, the edge server network traffic is abnormal, and based on the network traffic feature matrix, the traffic anomaly level is obtained, and the corresponding network anomaly warning is issued according to the traffic anomaly level;

[0025] The refined detection and network anomaly warning module constructs a network traffic dynamic graph structure with several nodes as source IP, destination IP, port number and protocol based on the network traffic data to be processed; the network traffic dynamic graph structure is divided according to the preset time window and arranged in chronological order to obtain several network traffic subgraph structures; based on all network traffic subgraph structures, the shortest path changes between the source IP node and the destination IP node, the shortest path changes between the source IP node and the protocol node, and the shortest path changes between the port number node and the protocol node are obtained, and the respective path deviations are obtained through path change deviation detection; if there is a path deviation greater than the corresponding path deviation threshold, the edge server network traffic is abnormal, and the traffic anomaly level is obtained based on all path deviations, and the corresponding network anomaly warning is issued according to the traffic anomaly level.

[0026] The beneficial effects of the present invention are:

[0027] The present invention provides an artificial intelligence-based network traffic anomaly evaluation and early warning method, which first obtains source IP information, destination IP information, port number information and protocol information within a preset time range at intervals, comprehensively collects key network traffic data, and further improves data quality after data cleaning processing; then, comprehensively considers information from multiple dimensions such as IP address entropy, port number distribution characteristics, protocol traffic proportion, traffic time series characteristics, etc., which can more comprehensively describe the characteristics of network traffic and avoid the one-sidedness of single feature analysis, thereby improving the accuracy of anomaly detection; and maps these features into a feature matrix and performs normalization processing, and then obtains an initial detection value by generating a Boolean matrix, and performs different operations according to the comparison result between the initial detection value and the first value, thereby improving the accuracy and intelligence of network traffic anomaly evaluation. Then, when it is judged that there is no network traffic anomaly based on the initial detection value, a dynamic graph structure of network traffic is constructed and segmented and analyzed. It can deeply explore the internal relationship and change law of network traffic from the perspective of graph structure, and by obtaining the shortest path changes between different nodes and performing path change deviation detection, it can more accurately discover subtle abnormal behaviors in network traffic, obtain the path deviation amount, judge whether the edge server network traffic is abnormal based on the path deviation amount, and obtain the traffic anomaly level, thereby improving the accuracy of traffic anomaly evaluation; finally, different early warning measures are taken according to different traffic anomaly levels, which can more accurately respond to different degrees of network traffic anomalies, avoid overreaction or underreaction, and improve the intelligence and accuracy of network traffic anomaly evaluation, as well as the timeliness and flexibility of network traffic anomaly warning. DETAILED DESCRIPTION

[0028] This embodiment provides an artificial intelligence-based network traffic anomaly assessment and early warning method, including the following operations:

[0029] S1. Periodically obtain network traffic data of edge servers within a preset time range, where the network traffic data includes source IP information, destination IP information, port number information, and protocol information; and clean the network traffic data to obtain network traffic data to be processed.

[0030] S2. Based on the network traffic data to be processed, obtain the entropy of the IP address, the port number distribution characteristics, the protocol traffic ratio, and the traffic time series characteristics, map them into a feature matrix, and perform normalization processing to obtain a network traffic feature matrix; based on the method of generating a Boolean matrix, obtain an initial detection value of the network traffic feature matrix; if the initial detection value is a first value, execute S3; if the initial detection value is greater than the first value, the network traffic of the edge server is abnormal, and based on the network traffic feature matrix, obtain the traffic anomaly level, and issue a corresponding network anomaly warning according to the traffic anomaly level;

[0031] S3. Based on the network traffic data to be processed, a network traffic dynamic graph structure with several nodes as source IP, destination IP, port number and protocol is constructed; the network traffic dynamic graph structure is divided according to the preset time window, and arranged in chronological order to obtain several network traffic subgraph structures; based on all network traffic subgraph structures, the shortest path changes between the source IP node and the destination IP node, the shortest path changes between the source IP node and the protocol node, and the shortest path changes between the port number node and the protocol node are obtained, and the respective path deviations are obtained through path change deviation detection; if there is a path deviation greater than the corresponding path deviation threshold, the edge server network traffic is abnormal; based on all path deviations, the traffic anomaly level is obtained, and corresponding network anomaly warnings are issued according to the traffic anomaly level.

[0032] S1. Periodically obtain network traffic data of edge servers within a preset time range, where the network traffic data includes source IP information, destination IP information, port number information, and protocol information; and perform data cleaning on the network traffic data to obtain network traffic data to be processed.

[0033] By obtaining source IP information, destination IP information, port number information and protocol information within a preset time range at intervals, it is possible to comprehensively collect key network traffic data, more accurately reflect the actual situation of network traffic, and is applicable to different network scenarios. After data cleaning processing, it further improves data quality and provides an accurate basis for subsequent analysis.

[0034] First, we periodically collect network traffic data of edge servers within a preset time range from multiple data sources such as the edge server's network interface, firewall, and intrusion detection system. The network traffic data includes: source IP information that can reflect the source of the tracking traffic, destination IP information that can reflect the target of the traffic, port information that can reflect specific network services, and protocol information that shows that different protocols have different normal traffic patterns.

[0035] For edge servers with complex and diverse traffic loads, a suitable data collection frequency helps to more accurately characterize the normal characteristics of network traffic. At the same time, in order to reduce resource consumption and improve detection timeliness, in the process of obtaining network traffic data of edge servers within a preset time range, the frequency of network traffic data acquisition within the preset time range is based on the traffic load of the edge server and evaluation requirements.

[0036] The frequency of obtaining network traffic data is obtained by the following formula:

[0037]

[0038] f is the frequency of obtaining network traffic data, α is the traffic load factor, which reflects the average traffic load of the edge server, U is the average value of data packets, R is the resolution required for evaluation, P is the data processing capacity of the edge server, which depends on the hardware performance of the server, including CPU, memory, storage and other factors, and λ is the edge server load rate, that is, the proportion of the server's currently used processing capacity to the total processing capacity.

[0039] Next, to improve stability during the detection process, the network traffic data is cleaned to obtain the network traffic data to be processed. Data cleaning includes removing noise and duplicate data and filling in missing values. When network data (source IP information, destination IP information, port information, or protocol information) is missing, the network data is interpolated to fill in the missing values.

[0040] The above interpolation processing operation is specifically as follows: taking the data other than missing values ​​in the network data as samples, inputting the samples into the training decision tree for traversal, starting from the root node of the decision tree, selecting the corresponding branch at each node according to the characteristic value of the sample until reaching the leaf node; wherein, the leaf node stores the predicted value or statistical information for the sample of that category, which is used as the interpolation result of the missing value; if the leaf node stores the predicted label of the sample of that category, then the label with the largest predicted value is used as the interpolation result of the missing value; if the leaf node stores numerical statistical information, such as mean, median, etc., then the mean or median is used as the interpolation result. The operation of training the decision tree in the training decision tree is specifically as follows: taking the complete data as the sample input, taking the data to be interpolated as the target variable, and using the training set data to train the decision tree model. During the training process, the decision tree learns the relationship between different samples and the target variable, and continuously recursively divides the nodes until the depth of the decision tree reaches the upper limit value, and the training ends.

[0041] S2. Based on the network traffic data to be processed, obtain the entropy of the IP address, port number distribution characteristics, protocol traffic proportion, and traffic time series characteristics, map them into a feature matrix, and perform normalization processing to obtain a network traffic feature matrix; based on the method of generating a Boolean matrix, obtain the initial detection value of the network traffic feature matrix; if the initial detection value is the first value, execute S3; if the initial detection value is greater than the first value, the network traffic of the edge server is abnormal, and based on the network traffic feature matrix, obtain the traffic anomaly level, and issue a corresponding network anomaly warning according to the traffic anomaly level.

[0042] By comprehensively considering information from multiple dimensions such as the entropy of IP addresses, port number distribution characteristics, protocol traffic proportion, and traffic time series characteristics, it is possible to more comprehensively describe the characteristics of network traffic, avoid the one-sidedness of single feature analysis, and thus improve the accuracy of anomaly detection; and map these features into a feature matrix and perform normalization processing, and then obtain the initial detection value by generating a Boolean matrix. According to the comparison result between the initial detection value and the first value, different operations are performed, thereby improving the accuracy and intelligence of network traffic anomaly evaluation.

[0043] First, based on the network traffic data to be processed, the entropy of the IP address, port number distribution characteristics, protocol traffic proportion, and traffic time series characteristics are obtained.

[0044] The entropy of IP addresses (including source and destination IP addresses) reflects the diversity of edge server addresses over a period of time (within a preset timeframe), indirectly reflecting the complexity of network activity. A higher entropy value indicates a more dispersed distribution of IP addresses. Edge servers communicate with a large number of different clients or servers, potentially involving a wide range of networks and multiple types of users or services. This indicates a more complex network environment in which edge servers operate, facing diverse sources and destinations of network traffic, and a higher degree of uncertainty regarding network traffic security.

[0045] The entropy of an IP address is obtained based on the probability distribution of the number of occurrences of the IP address in the network traffic data to be processed. The specific calculation formula is as follows:

[0046]

[0047] H is the entropy of the IP address, N i is the number of occurrences of the i-th IP address, and n is the total number of IP addresses.

[0048] Different applications and services typically use specific port numbers for communication. By analyzing the port number distribution characteristics, we can understand the main application types and service models running on the edge server. For example, if a large amount of traffic is concentrated on port 80 (HTTP) and port 443 (HTTPS), it means that the server mainly provides services related to web browsing; if there is a lot of traffic on port 22 (SSH), the server may be frequently used for remote login management; at the same time, abnormal port number usage may indicate potential security issues or abnormal activities. For example, if an uncommon port suddenly has a large amount of traffic, it may be that the server is being used by malware for data transmission, or that unauthorized services are running.

[0049] The specific method for obtaining the port number distribution characteristics is as follows: count the number of times each port number appears in the network traffic data, draw a line graph with time as the horizontal axis and the number of times the port number appears as the vertical axis, obtain the fluctuation characteristics of the number of times each port number appears from the line graph, and obtain the port number distribution characteristics; the fluctuation of port number usage can be observed through the line graph, and abnormal situations can be discovered in combination with the corresponding functions of the port numbers.

[0050] The specific method for obtaining the protocol traffic share is: protocol traffic share (i) = (traffic volume of protocol i / total traffic volume) × 100%, where total traffic volume is the sum of all protocol traffic volumes within a preset timeframe. Different network protocols are used in different application scenarios and service functions. Therefore, analyzing protocol traffic share can clearly understand the relative importance and distribution of various services carried by edge servers. For example, a high TCP traffic share indicates that the server primarily handles connection-oriented reliable data transmission services, such as file downloads and database access. A high UDP traffic share may indicate that the server is more involved in real-time services, such as video and audio streaming. Furthermore, observing protocol traffic share can assess whether the server is utilizing network resources effectively. For example, if a low-efficiency protocol's traffic share is excessively high, it indicates network anomalies such as traffic waste.

[0051] Traffic time series characteristics can reveal how edge server traffic changes over time. Sudden changes, unusual fluctuations, or deviations from normal patterns in the time series may indicate unusual traffic events or server failures. For example, a sudden, significant fluctuation in a previously stable traffic curve could indicate a cyberattack or a server failure, leading to partial service interruption and a sudden drop in traffic.

[0052] The method for obtaining traffic time series features is as follows: grouping the respective traffic data (traffic information) according to the different types of source IP, destination IP, port number, and protocol to create a multidimensional data structure; in the multidimensional data structure, the source IP, destination IP, port number, and protocol serve as keys, and the corresponding traffic data serves as values; arranging the traffic data after each group in the multidimensional data structure in chronological order to form several traffic time series; normalizing each traffic time series to obtain several traffic time normalized series; based on a preset time series length, decomposing each traffic time normalized series into a combination of sine and cosine waves of different frequencies to obtain several traffic complex number arrays; each element in the complex number array corresponds to a frequency component, and its modulus represents the amplitude of the frequency component, reflecting the energy of the frequency in the time series; based on the several traffic complex number arrays, plotting a spectrum graph to obtain the statistical features of the spectrum graph, including but not limited to the amplitude, mean, variance, skewness, and kurtosis of the spectrum, to describe the overall shape of the spectrum. Changes in these statistical features indicate changes in the traffic pattern, thus obtaining the traffic time series features.

[0053] Next, the IP address entropy, port number distribution characteristics, protocol traffic percentage, and traffic time series characteristics are mapped into a feature matrix. In this feature matrix, each column represents a different type of traffic data. For example, the first column represents the IP address entropy, and the fourth column represents the traffic time series characteristics. Each row represents a different dimension. The total number of dimensions for the port number distribution characteristics is the total number of port numbers, the total number of dimensions for the protocol traffic percentage is the total number of protocols, and the total number of dimensions for the traffic time series characteristics is the total number of traffic data types. For example, if the IP address entropy has only one dimension, then all data in the first column except the first row is set to 0.

[0054] Next, since the dimensions and value ranges of different features vary greatly, the feature matrix is ​​normalized so that all features are in the same dimension to facilitate subsequent comparison and obtain the network traffic feature matrix.

[0055] Subsequently, the initial detection value of the network traffic feature matrix is ​​obtained based on the method of generating a Boolean matrix. That is, a threshold matrix corresponding to the network traffic feature matrix is ​​constructed, and the threshold matrix is ​​compared with the network traffic feature matrix. Positions in the network traffic feature matrix that exceed the corresponding threshold in the corresponding threshold matrix are marked as anomalies. Based on all the marking information and the corresponding position data, a Boolean matrix is ​​obtained. The number of anomalies in the Boolean matrix is ​​counted to obtain the initial detection value.

[0056] Finally, if the initial detection value is greater than the first value (preferably 0), the edge server network traffic is abnormal. Based on the network traffic feature matrix, the traffic anomaly level is obtained, and a corresponding network anomaly warning is issued according to the traffic anomaly level.

[0057] The specific operation of obtaining the flow anomaly level is as follows: if the initial detection value is greater than the first value but less than the second value, the flow anomaly level is a low risk level; if the initial detection value is not less than the second value and not greater than the third value, the flow anomaly level is a medium risk level; if the initial detection value is greater than the third value, the flow anomaly level is a high risk level. The first value is less than the second value and less than the third value.

[0058] If the initial detection value is the first value (preferably 0), in order to improve the detection accuracy, the operation of performing refined detection according to the correlation between the flow data in S3 is executed.

[0059] S3. Based on the network traffic data to be processed, a network traffic dynamic graph structure with several nodes as source IP, destination IP, port number and protocol is constructed; the network traffic dynamic graph structure is divided according to the preset time window, and arranged in chronological order to obtain several network traffic subgraph structures; based on all network traffic subgraph structures, the shortest path changes between the source IP node and the destination IP node, the shortest path changes between the source IP node and the protocol node, and the shortest path changes between the port number node and the protocol node are obtained, and the respective path deviations are obtained through path change deviation detection; if there is a path deviation greater than the corresponding path deviation threshold, the edge server network traffic is abnormal, and the traffic anomaly level is obtained based on all path deviations, and the corresponding network anomaly warning is issued according to the traffic anomaly level.

[0060] By constructing a dynamic graph structure for network traffic and performing segmentation and analysis, we can deeply explore the intrinsic relationships and changing patterns of network traffic from the perspective of the graph structure. By obtaining the shortest path changes between different nodes and performing path change deviation detection, we can more accurately discover subtle abnormal behaviors in network traffic and obtain the path deviation amount. Based on the path deviation amount, we can judge whether the edge server network traffic is abnormal and obtain the traffic anomaly level, thereby improving the accuracy of traffic anomaly evaluation.

[0061] First, based on the network traffic data to be processed, a dynamic network traffic graph structure is constructed, with several nodes representing source IP, destination IP, port number, and protocol. The graph structure changes dynamically over time, intuitively reflecting the relationship between different network data at different times.

[0062] In the network traffic dynamic graph, if a source IP communicates with a destination IP using a specific port number, an edge is created between the source IP node and the corresponding port node, and between the destination IP node and the corresponding port node. If a port number uses a specific protocol for data transmission, an edge is created between the port number node and the corresponding protocol node. If an IP address communicates using a specific protocol, an edge is created between the IP node and the corresponding protocol node.

[0063] Next, the network traffic dynamic graph structure is segmented according to a preset time window into several time subgraph structures, which are arranged in chronological order to obtain several network traffic subgraph structures.

[0064] Then, based on the structure of all network traffic subgraphs, the shortest path changes between the source IP node and the destination IP node, the shortest path changes between the source IP node and the protocol node, and the shortest path changes between the port number node and the corresponding protocol node when using a specific protocol for data transmission are obtained.

[0065] Specifically, based on the structure of each network traffic subgraph, the shortest path between the source IP node and the destination IP node, the shortest path between the source IP node and the protocol node, and the shortest path between the port number node and the protocol node are obtained, and the shortest paths between several source IP nodes and destination IP nodes, the shortest paths between several source IP nodes and protocol nodes, and the shortest paths between several port number nodes and protocol nodes are obtained; the shortest paths between the source IP node and the destination IP node at adjacent times, the shortest paths between the source IP node and the protocol node at adjacent times, and the shortest paths between the port number node and the protocol node at adjacent times are counted, and the node changes and path length changes are obtained to obtain the shortest path changes between the source IP node and the destination IP node, the shortest path changes between the source IP node and the protocol node, and the shortest path changes between the port number node and the protocol node.

[0066] The shortest path acquisition method is as follows: create a distance array dist[] to store the shortest distance from the source node to each other node. Initially, set the distances of all nodes to infinity and the distance of the source node to 0, that is, dist[src] = 0, where src is the source node. Create a set S to store the nodes for which the shortest path has been determined. Initially, S is empty. Find the node u with the smallest distance that is not in S and add node u to set S, indicating that the shortest path from the source node to u has been found. For all outgoing edges (u, v) of node u, if dist[v]>dist[u]+w(u, v), update the value of dist[v] to dist[u]+w(u, v), indicating that a shorter path has been found from the source node to v.

[0067] When obtaining the shortest path between a source IP node and a destination IP node, the shortest path acquisition method is executed with the source IP node as the source node src. After execution, dist[dst] stores the shortest distance from the source IP node to the destination IP node, where dst is the destination IP node. To obtain the shortest path, you can start from the destination IP node and construct a path by backtracking to the predecessor nodes. For example, record the predecessor node prev[v] of each node. When updating dist[v], if a shorter path is found, update prev[v] = u at the same time. In this way, starting from dst, by continuously accessing the prev array, you can obtain the shortest path from the source IP node to the destination IP node.

[0068] When obtaining the shortest path between a source IP node and a protocol node, the shortest path acquisition method is executed with the source IP node as the source node src. After the execution, for each protocol node p, dist[p] stores the shortest distance from the source IP node to the protocol node. Similar to calculating the shortest path from the source IP node to the destination IP node, the specific shortest path is obtained by backtracking to the predecessor node prev.

[0069] To obtain the shortest path between a port number node and a protocol node, the source IP node is used as the source node src and the shortest path acquisition method is executed. After the execution is completed, for each protocol node p, dist[p] gives the shortest distance from the port number node to the protocol node. The shortest path from the port number node to the protocol node is traced back using the predecessor node prev array.

[0070] Next, the shortest path changes between the source IP node and the destination IP node, the shortest path changes between the source IP node and the protocol node, and the shortest path changes between the port number node and the protocol node are detected to obtain the path deviation result.

[0071] Specifically, based on the shortest path changes between the source IP node and the destination IP node, the shortest path changes between the source IP node and the protocol node, and the shortest path changes between the port number node and the protocol node, the respective path offsets are obtained.

[0072] The path offset is obtained by the following formula:

[0073]

[0074] Q is the path deviation, β is the path node deviation weight, P1 is the node set of path 1, P2 is the node set of path 2, γ is the path length deviation weight, L1 is the length of path 1, L2 is the length of path 2, and J is the total number of paths.

[0075] Finally, if there is a path deviation greater than the corresponding path deviation threshold, the edge server network traffic is abnormal. Based on all path deviations, the traffic anomaly level is obtained, and the corresponding network anomaly warning is issued according to the traffic anomaly level.

[0076] If the number of paths with a path deviation greater than the corresponding path deviation threshold is 1, the traffic anomaly level is a low risk level; if the number of paths with a path deviation greater than the corresponding path deviation threshold is 2, the traffic anomaly level is a medium risk level; if the number of paths with a path deviation greater than the corresponding path deviation threshold is 3, the traffic anomaly level is a high risk level.

[0077] If the traffic anomaly level is high risk, network blocking measures are implemented on the edge server, such as severing the connection between the source and destination IP addresses, or closing the relevant port numbers and network connections. If the traffic anomaly level is medium risk, the edge server network traffic rate is limited to the traffic rate threshold. If the traffic anomaly level is low risk, source IP addresses, destination IP addresses, port numbers, or protocols with reputation values ​​below the reputation threshold are deleted. The reputation value of a source IP address, destination IP address, port number, or protocol is determined based on the number of abnormal behaviors within a historical period.

[0078] This embodiment further provides an artificial intelligence-based network traffic anomaly assessment and early warning system, which is used to implement the above-mentioned artificial intelligence-based network traffic anomaly assessment and early warning method, including:

[0079] The module for generating network traffic data to be processed is used to periodically obtain network traffic data of edge servers within a preset time range. The network traffic data includes source IP information, destination IP information, port number information, and protocol information. The network traffic data is cleansed to obtain the network traffic data to be processed.

[0080] The initial detection and network anomaly warning module is used to obtain the entropy of the IP address, port number distribution characteristics, protocol traffic proportion, and traffic time series characteristics based on the network traffic data to be processed, map them into a feature matrix, and then perform normalization processing to obtain a network traffic feature matrix; based on the method of generating a Boolean matrix, obtain the initial detection value of the network traffic feature matrix; if the initial detection value is a first value, execute the refined detection and network anomaly warning module; if the initial detection value is greater than the first value, the edge server network traffic is abnormal, and based on the network traffic feature matrix, the traffic anomaly level is obtained, and the corresponding network anomaly warning is issued according to the traffic anomaly level;

[0081] The refined detection and network anomaly warning module constructs a network traffic dynamic graph structure with several nodes as source IP, destination IP, port number and protocol based on the network traffic data to be processed; the network traffic dynamic graph structure is divided according to the preset time window and arranged in chronological order to obtain several network traffic subgraph structures; based on all network traffic subgraph structures, the shortest path changes between the source IP node and the destination IP node, the shortest path changes between the source IP node and the protocol node, and the shortest path changes between the port number node and the protocol node are obtained, and the respective path deviations are obtained through path change deviation detection; if there is a path deviation greater than the corresponding path deviation threshold, the edge server network traffic is abnormal, and the traffic anomaly level is obtained based on all path deviations, and the corresponding network anomaly warning is issued according to the traffic anomaly level.

[0082] This embodiment provides an artificial intelligence-based network traffic anomaly assessment and early warning method, which first obtains source IP information, destination IP information, port number information and protocol information within a preset time range at intervals, comprehensively collects key network traffic data, and further improves data quality after data cleaning processing; then, comprehensively considers information from multiple dimensions such as IP address entropy, port number distribution characteristics, protocol traffic proportion, traffic time series characteristics, etc., which can more comprehensively describe the characteristics of network traffic and avoid the one-sidedness of single feature analysis, thereby improving the accuracy of anomaly detection; and maps these features into a feature matrix and normalizes them, and then obtains an initial detection value by generating a Boolean matrix. According to the comparison result between the initial detection value and the first value, different operations are performed, thereby improving the accuracy and accuracy of network traffic anomaly assessment. Intelligence; then, when it is judged that there is no network traffic anomaly based on the initial detection value, a dynamic graph structure of network traffic is constructed and segmented and analyzed, which can deeply explore the internal relationship and change law of network traffic from the perspective of graph structure, and by obtaining the shortest path changes between different nodes and performing path change deviation detection, it can more accurately discover subtle abnormal behaviors in network traffic, obtain the path deviation amount, judge whether the edge server network traffic is abnormal based on the path deviation amount, and obtain the traffic anomaly level, thereby improving the accuracy of traffic anomaly evaluation; finally, different measures are taken according to different traffic anomaly levels, which can more accurately respond to different degrees of network traffic anomalies, avoid overreaction or underreaction, and improve the intelligence and accuracy of network traffic anomaly evaluation, as well as the timeliness and flexibility of network traffic anomaly warning.

[0083] Although exemplary embodiments of the present invention have been shown and described in detail herein, many other variations or modifications consistent with the principles of the present invention may be directly determined or derived from the disclosure without departing from the spirit and scope of the present invention. Therefore, the scope of the present invention should be understood and deemed to cover all such other variations or modifications.

Claims

1. A network traffic anomaly evaluation and early warning method based on artificial intelligence, characterized in that: The following operations are included: S1. Periodically obtain network traffic data of edge servers within a preset time range, where the network traffic data includes source IP information, destination IP information, port number information, and protocol information; and clean the network traffic data to obtain network traffic data to be processed. S2. Based on the network traffic data to be processed, obtain the entropy of the IP address, the port number distribution characteristics, the protocol traffic ratio, and the traffic time series characteristics, map them into a feature matrix, and perform normalization processing to obtain a network traffic feature matrix; based on the method of generating a Boolean matrix, obtain the initial detection value of the network traffic feature matrix; If the initial detection value is the first value, execute S3; If the initial detection value is greater than the first value, the network traffic of the edge server is abnormal. Based on the network traffic feature matrix, the traffic anomaly level is obtained, and a corresponding network anomaly warning is issued according to the traffic anomaly level; S3. Based on the network traffic data to be processed, a network traffic dynamic graph structure is constructed with several nodes representing source IP, destination IP, port number, and protocol; the network traffic dynamic graph structure is divided according to a preset time window and arranged in chronological order to obtain several network traffic subgraph structures; Based on the structure of all network traffic subgraphs, the shortest path changes between the source IP node and the destination IP node, the shortest path changes between the source IP node and the protocol node, and the shortest path changes between the port number node and the protocol node are obtained. After path change deviation detection, the respective path deviations are obtained. If there is a path deviation greater than the corresponding path deviation threshold, the edge server network traffic is abnormal. Based on all path deviations, the traffic anomaly level is obtained, and the corresponding network anomaly warning is issued according to the traffic anomaly level.

2. The network traffic anomaly evaluation and early warning method based on artificial intelligence according to claim 1 is characterized in that: In S1, the frequency of acquiring network traffic data within the preset time period is based on the traffic load of the edge server and the evaluation requirements; the specific calculation formula is as follows: f is the frequency of obtaining network traffic data, α is the traffic load factor, U is the average value of data packets, R is the required resolution for evaluation, P is the data processing capacity of the edge server, and λ is the load rate of the edge server.

3. The network traffic anomaly evaluation and early warning method based on artificial intelligence according to claim 1 is characterized in that: In S2, the method for obtaining traffic time series characteristics is specifically as follows: After grouping the respective traffic data according to the source IP, destination IP, port number and protocol, a multidimensional data structure is created; the traffic data of each group in the multidimensional data structure is arranged in chronological order to form several traffic time series; each traffic time series is standardized to obtain several traffic time standardized series; based on the preset time series length, each traffic time standardized series is decomposed into a combination of sine and cosine waves of different frequencies to obtain several traffic complex number arrays; based on the several traffic complex number arrays, a spectrum graph is drawn to obtain the statistical characteristics of the spectrum graph and obtain the traffic time series characteristics.

4. The method for network traffic anomaly evaluation and early warning based on artificial intelligence according to claim 1 is characterized in that: In S2, the entropy of the IP address is obtained based on the probability distribution of the number of occurrences of the IP address in the network traffic data to be processed. The specific calculation formula is as follows: H is the entropy of the IP address, N i is the number of occurrences of the i-th IP address, and n is the total number of IP addresses.

5. The network traffic anomaly evaluation and early warning method based on artificial intelligence according to claim 1 is characterized in that: In S1, data cleaning includes removing noise and duplicate data from the data, and supplementing missing values; When network data is missing, the network data is interpolated. The interpolation operation is as follows: the data other than the missing values ​​in the network data is used as samples, and the samples are input into the training decision tree for traversal. Starting from the root node of the decision tree, the corresponding branch is selected at each node according to the characteristic value of the sample until the leaf node is reached; if the leaf node stores the predicted label of the sample, the predicted label is used as the interpolation result of the missing value; if the leaf node stores numerical statistical information, the median is used as the interpolation result.

6. The network traffic anomaly evaluation and early warning method based on artificial intelligence according to claim 1 is characterized in that: In S3, the path offset is obtained by the following formula: Q is the path deviation, β is the path node deviation weight, P t is the node set of path 1, P2 is the node set of path 2, γ is the path length deviation weight, L t is the length of path 1, L2 is the length of path 2, and J is the total number of paths.

7. The network traffic anomaly evaluation and early warning method based on artificial intelligence according to claim 1 is characterized in that: The specific operations for issuing corresponding network anomaly warnings based on traffic anomaly levels are as follows: If the traffic anomaly level is high risk, network blocking measures are taken for the edge server; if the traffic anomaly level is medium risk, the edge server network traffic rate is limited to the traffic rate threshold; If the traffic anomaly level is a low risk level, the source IP, destination IP, port number or protocol with a reputation value less than the reputation value threshold will be deleted.

8. An artificial intelligence-based network traffic anomaly evaluation and early warning system, used to implement the artificial intelligence-based network traffic anomaly evaluation and early warning method according to claim 1, characterized in that: include: The module for generating network traffic data to be processed is used to periodically obtain network traffic data of edge servers within a preset time range. The network traffic data includes source IP information, destination IP information, port number information, and protocol information. The network traffic data is cleansed to obtain the network traffic data to be processed. The initial detection and network anomaly warning module is used to obtain the entropy of IP addresses, port number distribution characteristics, protocol traffic proportion, and traffic time series characteristics based on the network traffic data to be processed. After mapping them into a feature matrix, normalization is performed to obtain the network traffic feature matrix. Based on the method of generating a Boolean matrix, the initial detection value of the network traffic feature matrix is ​​obtained. If the initial detection value is the first value, executing the refined detection and network anomaly warning modules; If the initial detection value is greater than the first value, the network traffic of the edge server is abnormal. Based on the network traffic feature matrix, the traffic anomaly level is obtained, and a corresponding network anomaly warning is issued according to the traffic anomaly level; The refined detection and network anomaly warning module constructs a network traffic dynamic graph structure based on the network traffic data to be processed, with several nodes being the source IP, destination IP, port number, and protocol. The network traffic dynamic graph structure is divided into preset time windows and arranged in chronological order to obtain several network traffic subgraph structures. Based on the structure of all network traffic subgraphs, the shortest path changes between the source IP node and the destination IP node, the shortest path changes between the source IP node and the protocol node, and the shortest path changes between the port number node and the protocol node are obtained. After path change deviation detection, the respective path deviations are obtained; if there is a path deviation greater than the corresponding path deviation threshold, the edge server network traffic is abnormal. Based on all path deviations, the traffic anomaly level is obtained, and the corresponding network anomaly warning is issued according to the traffic anomaly level.

Images