Network traffic anomaly evaluation and early warning method and system based on artificial intelligence

Abstract

The invention relates to the technical field of network security evaluation, in particular to a network traffic anomaly evaluation and early warning method and system based on artificial intelligence. According to the method, information of multiple dimensions such as entropy of an IP address, port number distribution characteristics, a protocol traffic proportion and traffic time sequence characteristics is comprehensively considered, after the characteristics are mapped into a characteristic matrix, an initial detection value is obtained through a Boolean matrix generation method, and according to a comparison result of the initial detection value and a first numerical value, the IP address is detected. Different operations are executed; and when it is judged that there is no network flow anomaly based on the initial detection value, constructing a network flow dynamic graph structure and carrying out segmentation analysis, obtaining the shortest path change between different nodes, and carrying out path change deviation detection, thereby carrying out network flow anomaly judgment. And finally, different network anomaly early warning measures are taken according to different traffic anomaly levels, so that the intelligence and accuracy of network traffic anomaly evaluation and the timeliness and flexibility of network anomaly early warning are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of network security assessment, and specifically to a method and system for network traffic anomaly assessment and early warning based on artificial intelligence. Background Art

[0002] Various potential security threats often hide in network traffic, such as hacker attacks, virus propagation, malware intrusion, etc. By performing real-time monitoring and anomaly assessment on network traffic, these abnormal behaviors can be discovered in a timely manner, early warnings can be given, and corresponding protection measures can be taken to prevent security incidents such as sensitive information leakage and system paralysis, and to protect the data and asset security in the network.

[0003] Modern network attack means are becoming increasingly complex and diverse, often using a combination of multiple technologies and methods to evade traditional detection mechanisms. Manual assessment requires a large amount of time and effort to analyze network traffic data. Facing a large amount of data, the manual processing speed is slow, it is difficult to discover abnormal information hidden in it in a timely manner, which easily leads to untimely problem discovery and delays the best processing opportunity; in addition, due to differences in professional backgrounds, experience levels, and personal judgment criteria among different assessors, the assessment results are subjective and inconsistent, affecting the accuracy and reliability of the assessment. Summary of the Invention

[0004] The purpose of the present invention is to provide a method and system for network traffic anomaly assessment and early warning based on artificial intelligence.

[0005] The technical solution of the present invention is as follows:

[0006] A method for network traffic anomaly assessment and early warning based on artificial intelligence includes the following operations:

[0007] S1. Periodically obtain network traffic data of edge servers within a preset time range. The network traffic data includes source IP information, destination IP information, port number information, and protocol information; the network traffic data is processed by data cleaning to obtain network traffic data to be processed;

[0008] S2. Based on the network traffic data to be processed, obtain the entropy of the IP address, the distribution characteristics of the port number, the protocol traffic ratio, and the traffic time series characteristics. After mapping to a feature matrix, perform normalization processing to obtain a network traffic feature matrix; based on the method of generating a Boolean matrix, obtain the initial detection value of the network traffic feature matrix; if the initial detection value is the first value, execute S3; if the initial detection value is greater than the first value, the network traffic of the edge server is abnormal. Based on the network traffic feature matrix, obtain the traffic anomaly level, and perform corresponding network anomaly early warning according to the traffic anomaly level;

[0009] S3. Based on the network traffic data to be processed, construct a network traffic dynamic graph structure with nodes being source IP, destination IP, port number, and protocol; divide the network traffic dynamic graph structure according to a preset time window, arrange them in chronological order, and obtain several network traffic sub-graph structures; based on all network traffic sub-graph structures, obtain the shortest path changes between the source IP node and the destination IP node, the shortest path changes between the source IP node and the protocol node, and the shortest path changes between the port number node and the protocol node. Through path change deviation detection, obtain their respective path deviation amounts; if there is a path deviation amount greater than the corresponding path deviation amount threshold, the edge server network traffic is abnormal; based on all path deviation amounts, obtain the traffic anomaly level, and perform corresponding network anomaly warnings according to the traffic anomaly level.

[0010] The acquisition frequency of network traffic data within the preset duration range in S1 is obtained based on the traffic load and evaluation requirements of the edge server; the specific calculation formula is as follows:

[0011]

[0012] f is the network traffic data acquisition frequency, α is the traffic load factor, U is the average value of data packets, R is the resolution required for evaluation, P is the data processing capacity of the edge server, and λ is the edge server load rate.

[0013] The method for obtaining the traffic time series features in S2 is specifically as follows: After grouping the respective traffic data according to the source IP, destination IP, port number, and protocol, create a multi-dimensional data structure; in the multi-dimensional data structure, arrange the traffic data after each grouping in chronological order to form several traffic time series; perform standardization processing on each traffic time series to obtain several traffic time standardization series; based on the preset time series length, decompose each traffic time standardization series into a combination of sine and cosine waves with different frequencies to obtain several traffic complex number arrays; based on several traffic complex number arrays, draw a spectrogram, obtain the statistical features of the spectrogram, and obtain the traffic time series features.

[0014] The entropy of the IP address in S2 is obtained based on the probability distribution corresponding to the occurrence times of the IP address in the network traffic data to be processed. The specific calculation formula is as follows:

[0015]

[0016] H is the entropy of the IP address, N i is the occurrence times of the i-th IP address, and n is the total number of IP addresses.

[0017] In S1, the data cleaning process includes removing noise and duplicate data from the data, as well as filling in missing values; when missing values occur in the network data, interpolation processing is performed on the network data. The specific operation of the interpolation processing is as follows: The data in the network data except for the missing values is used as a sample, and the sample is input into the trained decision tree for traversal. Starting from the root node of the decision tree, the corresponding branch is selected at each node according to the feature values of the sample until the leaf node is reached; if the leaf node stores the predicted label of the sample, the predicted label is used as the interpolation result of the missing value; if the leaf node stores numerical statistical information, the median is used as the interpolation result.

[0018] In S3, the path offset is obtained through the following formula:

[0019]

[0020] Q is the path deviation, β is the path node deviation weight, P1 is the set of nodes of path 1, P2 is the set of nodes of path 2, γ is the path length deviation weight, L1 is the length of path 1, L2 is the length of path 2, and J is the total number of paths.

[0021] The operation of performing corresponding network anomaly warnings according to the traffic anomaly level is as follows: If the traffic anomaly level is a high-risk level, network blocking measures are taken for the edge server; if the traffic anomaly level is a medium-risk level, the network traffic rate of the edge server is limited to the traffic rate threshold; if the traffic anomaly level is a low-risk level, the source IP, or destination IP, or port number or protocol with a reputation value less than the reputation value threshold is deleted.

[0022] An artificial intelligence-based network traffic anomaly evaluation and warning system for implementing the above-mentioned artificial intelligence-based network traffic anomaly evaluation and warning method, including:

[0023] A to-be-processed network traffic data generation module, used to periodically obtain the network traffic data of the edge server within a preset time range. The network traffic data includes source IP information, destination IP information, port number information, and protocol information; the network traffic data is subjected to data cleaning processing to obtain the to-be-processed network traffic data;

[0024] Initial detection and network anomaly warning module, which is used to obtain the entropy of the IP address, the distribution characteristics of port numbers, the proportion of protocol traffic, and the traffic time series characteristics based on the network traffic data to be processed. After mapping them into a feature matrix and performing normalization processing, a network traffic feature matrix is obtained; based on the method of generating a Boolean matrix, the initial detection value of the network traffic feature matrix is obtained; if the initial detection value is the first value, the refined detection and network anomaly warning module is executed; if the initial detection value is greater than the first value, the edge server network traffic is abnormal. Based on the network traffic feature matrix, the traffic anomaly level is obtained, and corresponding network anomaly warnings are given according to the traffic anomaly level.

[0025] Refined detection and network anomaly warning module, which constructs a network traffic dynamic graph structure with nodes of source IP, destination IP, port number, and protocol based on the network traffic data to be processed; divides the network traffic dynamic graph structure according to a preset time window and arranges them in chronological order to obtain several network traffic sub-graph structures; based on all network traffic sub-graph structures, the shortest path changes between the source IP node and the destination IP node, the shortest path changes between the source IP node and the protocol node, and the shortest path changes between the port number node and the protocol node are obtained. After path change deviation detection, the respective path deviation amounts are obtained; if there is a path deviation amount greater than the corresponding path deviation amount threshold, the edge server network traffic is abnormal. Based on all path deviation amounts, the traffic anomaly level is obtained, and corresponding network anomaly warnings are given according to the traffic anomaly level.

[0026] The beneficial effects of the present invention are as follows:

[0027] A method for evaluating and warning network traffic anomalies based on artificial intelligence provided by the present invention first obtains source IP information, destination IP information, port number information, and protocol information within a preset time range at intervals, comprehensively collects key network traffic data, and further improves the data quality after data cleaning and processing; then, considering multiple dimensions of information such as the entropy of the IP address, the distribution characteristics of the port number, the proportion of protocol traffic, and the characteristics of the traffic time series, it can more comprehensively describe the characteristics of network traffic, avoid the one-sidedness of single-feature analysis, and thus improve the accuracy of anomaly detection; and maps these features into a feature matrix and performs normalization processing, then obtains an initial detection value by generating a Boolean matrix, and performs different operations according to the comparison result of the initial detection value and the first value, improving the accuracy and intelligence of network traffic anomaly evaluation; then, when it is determined based on the initial detection value that there is no network traffic anomaly, a network traffic dynamic graph structure is constructed and segmented and analyzed, which can deeply explore the internal relationship and change law of network traffic from the perspective of the graph structure, and by obtaining the change of the shortest path between different nodes and performing path change deviation detection, it is possible to more accurately discover subtle abnormal behaviors in network traffic, obtain the path deviation amount, judge whether the edge server network traffic is abnormal according to the path deviation amount, and obtain the traffic anomaly level, improving the accuracy of traffic anomaly evaluation; finally, different warning measures are taken according to different traffic anomaly levels, which can more accurately respond to different degrees of network traffic anomalies, avoid overreaction or underreaction, and improve the intelligence and accuracy of network traffic anomaly evaluation, as well as the timeliness and flexibility of network traffic anomaly warning. Specific implementation mode

[0028] This embodiment provides a method for evaluating and warning network traffic anomalies based on artificial intelligence, including the following operations:

[0029] S1. Periodically obtain the network traffic data of the edge server within a preset time range. The network traffic data includes source IP information, destination IP information, port number information, and protocol information; the network traffic data is subjected to data cleaning and processing to obtain the network traffic data to be processed;

[0030] S2. Based on the network traffic data to be processed, obtain the entropy of the IP address, the distribution characteristics of the port number, the proportion of protocol traffic, and the characteristics of the traffic time series. After mapping to a feature matrix, perform normalization processing to obtain a network traffic feature matrix; based on the method of generating a Boolean matrix, obtain the initial detection value of the network traffic feature matrix; if the initial detection value is the first value, execute S3; if the initial detection value is greater than the first value, the edge server network traffic is abnormal. Based on the network traffic feature matrix, obtain the traffic anomaly level, and perform corresponding network anomaly warnings according to the traffic anomaly level;

[0031] S3. Based on the network traffic data to be processed, construct a dynamic network traffic graph structure with nodes being source IP, destination IP, port number, and protocol; divide the dynamic network traffic graph structure according to a preset time window, and arrange them in chronological order to obtain several network traffic sub-graph structures; based on all the network traffic sub-graph structures, obtain the shortest path changes between the source IP node and the destination IP node, the shortest path changes between the source IP node and the protocol node, and the shortest path changes between the port number node and the protocol node. After path change deviation detection, obtain their respective path deviation amounts; if there is a path deviation amount greater than the corresponding path deviation amount threshold, the network traffic of the edge server is abnormal; based on all the path deviation amounts, obtain the traffic anomaly level, and issue corresponding network anomaly warnings according to the traffic anomaly level.

[0032] S1. Periodically obtain the network traffic data of the edge server within a preset duration. The network traffic data includes source IP information, destination IP information, port number information, and protocol information; the network traffic data is processed through data cleaning to obtain the network traffic data to be processed.

[0033] By obtaining the source IP information, destination IP information, port number information, and protocol information within a preset duration at intervals, it is possible to comprehensively collect the key data of network traffic, more accurately reflect the real situation of network traffic, and is applicable to different network scenarios. After data cleaning processing, the data quality is further improved, providing an accurate basis for subsequent analysis.

[0034] First, from multiple data sources such as the network interface, firewall, and intrusion detection system of the edge server, periodically collect the network traffic data of the edge server within a preset duration. The network traffic data includes: source IP information that can reflect the tracking of traffic sources, destination IP information that can reflect traffic targets, port information that can reflect specific network services, and protocol information with different normal traffic patterns for different protocols.

[0035] For edge servers with complex and diverse traffic loads, an appropriate data collection frequency helps to more accurately depict the normal characteristics of network traffic. At the same time, to reduce resource consumption and improve the detection timeliness, during the process of obtaining the network traffic data of the edge server within a preset duration at intervals, within the preset duration, the network traffic data acquisition frequency is obtained based on the traffic load and evaluation requirements of the edge server.

[0036] The network traffic data acquisition frequency is obtained through the following formula:

[0037]

[0038] Let \(f\) be the network traffic data acquisition frequency, \(\alpha\) be the traffic load factor, which reflects the average traffic load of the edge server. \(U\) is the average value of data packets, \(R\) is the required resolution for evaluation, \(P\) is the data processing capacity of the edge server, which depends on the hardware performance of the server, including factors such as CPU, memory, storage, etc. \(\lambda\) is the load rate of the edge server, that is, the proportion of the currently used processing capacity of the server in the total processing capacity.

[0039] Next, to improve the stability during the detection process, the network traffic data is subjected to data cleaning processing to obtain the network traffic data to be processed. Data cleaning processing includes removing noise and duplicate data in the data, as well as filling in missing values; when network data (source IP information, or destination IP information, or port information or protocol information) is missing, interpolation processing is performed on the network data to fill in the missing values.

[0040] The specific operation of the above interpolation processing is as follows: Use the data in the network data except for the missing values as samples, input the samples into the trained decision tree for traversal. Starting from the root node of the decision tree, select the corresponding branch at each node according to the feature values of the samples until reaching the leaf node; among them, the leaf node stores the predicted value or statistical information for this category of samples, which is used as the interpolation result of the missing value; if the leaf node stores the predicted label for this category of samples, then select the label with the largest predicted value as the interpolation result of the missing value; if the leaf node stores numerical statistical information, such as the mean, median, etc., then use the mean or median as the interpolation result. The operation of training the decision tree in the trained decision tree is as follows: Use the complete data as the sample input, use the data that needs to be interpolated as the target variable, and use the training set data to train the decision tree model. During the training process, the decision tree learns the relationship between different samples and the target variable, and continuously recursively divides the nodes until the depth of the decision tree reaches the upper limit value, and the training ends.

[0041] S2. Based on the network traffic data to be processed, obtain the entropy of the IP address, the distribution characteristics of the port number, the proportion of protocol traffic, and the traffic time series characteristics. After mapping them into a feature matrix, perform normalization processing to obtain the network traffic feature matrix; based on the method of generating a boolean matrix, obtain the initial detection value of the network traffic feature matrix; if the initial detection value is the first value, execute S3; if the initial detection value is greater than the first value, then the network traffic of the edge server is abnormal. Based on the network traffic feature matrix, obtain the traffic anomaly level, and issue corresponding network anomaly warnings according to the traffic anomaly level.

[0042] By comprehensively considering information from multiple dimensions such as the entropy of IP addresses, the distribution characteristics of port numbers, the proportion of protocol traffic, and the characteristics of traffic time series, the characteristics of network traffic can be more comprehensively described, avoiding the one-sidedness of single-characteristic analysis, thereby improving the accuracy of anomaly detection; and mapping these characteristics into a feature matrix and performing normalization processing, then obtaining initial detection values by generating a Boolean matrix, and performing different operations according to the comparison result of the initial detection value and the first value, improving the accuracy and intelligence of network traffic anomaly evaluation.

[0043] First, based on the network traffic data to be processed, obtain the entropy of IP addresses, the distribution characteristics of port numbers, the proportion of protocol traffic, and the characteristics of traffic time series.

[0044] Among them, the entropy of the IP address (including the source IP and destination IP addresses) reflects the degree of diversity of edge server addresses within a certain period of time (within a preset time range), and indirectly reflects the complexity of network activities. The higher the entropy value, the more dispersed the distribution of IP addresses, indicating that the edge server communicates with a large number of different clients or servers, which may involve a wide range of networks and various types of users or services, indicating that the network environment where the edge server is located is more complex, facing diverse sources and destinations of network traffic, and the uncertainty of network traffic security is higher.

[0045] The entropy of the IP address is obtained based on the probability distribution corresponding to the number of occurrences of the IP address in the network traffic data to be processed. The specific calculation formula is as follows:

[0046]

[0047] H is the entropy of the IP address, N i is the number of occurrences of the i-th IP address, and n is the total number of IP addresses.

[0048] Different application programs and services usually use specific port numbers for communication. By analyzing the distribution characteristics of port numbers, the main application types and service modes running on the edge server can be understood. For example, a large amount of traffic is concentrated on port 80 (HTTP) and port 443 (HTTPS), indicating that the server mainly provides services related to web browsing; if there is more traffic on port 22 (SSH), it may mean that the server is often used for remote login management; at the same time, abnormal port number usage may imply potential security problems or abnormal activities. For example, if a large amount of traffic suddenly appears on an uncommon port, it may be that the server is being exploited by malware for data transmission, or there is an unauthorized service running.

[0049] The method for obtaining the port number distribution characteristics is specifically as follows: count the number of occurrences of each port number in the network traffic data, draw a line chart with time as the abscissa and the number of occurrences of the port number as the ordinate, obtain the fluctuation characteristics of the number of occurrences of each port number from the line chart, and obtain the port number distribution characteristics; through the line chart, the fluctuation of the port number usage can be observed, and combined with the corresponding functions of the port numbers, abnormal situations can be discovered.

[0050] The method for obtaining the protocol traffic proportion is specifically as follows: the proportion of the i-th protocol traffic = (the traffic size of the i-th protocol / the total traffic size) × 100%, and the total traffic size is the sum of the traffic sizes of all protocols within the preset time range. Different network protocols are used for different application scenarios and business functions. Therefore, analyzing the protocol traffic proportion can clearly understand the relative importance and distribution of various services carried by the edge server; for example, if the traffic proportion of the TCP protocol is relatively high, it indicates that the server mainly processes connection-oriented reliable data transmission services, such as file downloads and database access; while a large proportion of UDP protocol traffic may mean that the server is more involved in services with high real-time requirements, such as video stream and audio stream transmission. In addition, by observing the protocol traffic proportion, it is possible to evaluate whether the network resources of the server are utilized reasonably. For example, if it is found that the traffic proportion of a certain inefficient protocol is too high, it indicates that there are abnormal phenomena such as traffic waste in the network.

[0051] The traffic time series characteristics can reflect the change law of the business traffic of the edge server over time. Sudden changes, abnormal fluctuations or deviations from the normal pattern in the time series may indicate abnormal traffic events or server failures. For example, if the originally stable traffic curve suddenly shows a large fluctuation, it may be due to a network attack or a server failure, resulting in the interruption of some services and a sharp reduction in traffic.

[0052] The method for obtaining the traffic time series features is specifically as follows: Group the respective traffic data (traffic information) according to different types of source IP, destination IP, port number, and protocol to create a multi-dimensional data structure; in the multi-dimensional data structure, the source IP, destination IP, port number, and protocol serve as keys, and the corresponding traffic data serves as values; arrange the traffic data after grouping in the multi-dimensional data structure in chronological order to form several traffic time series; perform standardization processing on each traffic time series to obtain several traffic time standardization series; based on a preset time series length, decompose each traffic time standardization series into a combination of sine and cosine waves with different frequencies to obtain several traffic complex number arrays; each element in the complex number array corresponds to a frequency component, and its modulus value represents the amplitude of the frequency component, reflecting the energy magnitude of the frequency in the time series; based on several traffic complex number arrays, draw a spectrogram and obtain the statistical features of the spectrogram, including but not limited to amplitude, mean value of the spectrum, variance, skewness, and kurtosis, to describe the overall shape of the spectrum. The changes in these statistical features imply changes in the traffic pattern, and the traffic time series features are obtained.

[0053] Then, map the entropy of the IP address, the distribution characteristics of the port number, the proportion of protocol traffic, and the traffic time series features to a feature matrix. In the feature matrix, each column represents different types of traffic data. For example, the data in the first column is the entropy of the IP address, and the data in the fourth column is the traffic time series features; each row represents different dimensions. The total number of dimensions of the port number distribution characteristics is the total number of port numbers, the total number of dimensions of the protocol traffic proportion is the total number of protocols, and the total number of dimensions of the traffic time series features is the total number of types of traffic data. For example, if the entropy of the IP address has only one dimension, then the data in other rows except the first row in the first column is set to 0.

[0054] Next, since there are significant differences in the dimensions and value ranges of different features, the feature matrix is normalized so that all features are in the same dimension for subsequent comparison, and a network traffic feature matrix is obtained.

[0055] Subsequently, based on the method of generating a boolean matrix, obtain the initial detection value of the network traffic feature matrix. That is, construct a threshold matrix corresponding to the network traffic feature matrix, compare the threshold matrix with the network traffic feature matrix, mark the positions in the network traffic feature matrix that exceed the corresponding thresholds in the corresponding threshold matrix as abnormal, and based on all the marking information and the corresponding position data, obtain a boolean matrix; count the number of abnormalities in the boolean matrix to obtain the initial detection value.

[0056] Finally, if the initial detection value is greater than the first value (preferably 0), it means that the network traffic of the edge server is abnormal. Based on the network traffic feature matrix, obtain the traffic anomaly level, and issue corresponding network anomaly warnings according to the traffic anomaly level.

[0057] The operation of obtaining the traffic anomaly level is specifically as follows: if the initial detection value is greater than the first value but less than the second value, the traffic anomaly level is the low-risk level; if the initial detection value is not less than the second value and not greater than the third value, the traffic anomaly level is the medium-risk level; if the initial detection value is greater than the third value, the traffic anomaly level is the high-risk level. The first value is less than the second value which is less than the third value.

[0058] If the initial detection value is the first value (preferably 0), to improve the detection accuracy, perform the operation of refined detection according to the correlation between traffic data in S3.

[0059] S3. Based on the network traffic data to be processed, construct a network traffic dynamic graph structure with nodes of source IP, destination IP, port number, and protocol; divide the network traffic dynamic graph structure according to a preset time window, and arrange them in chronological order to obtain several network traffic sub-graph structures; based on all network traffic sub-graph structures, obtain the shortest path changes between the source IP node and the destination IP node, the shortest path changes between the source IP node and the protocol node, and the shortest path changes between the port number node and the protocol node. After path change deviation detection, obtain their respective path deviation amounts; if there is a path deviation amount greater than the corresponding path deviation amount threshold, the edge server network traffic is abnormal. Based on all path deviation amounts, obtain the traffic anomaly level, and perform corresponding network anomaly warnings according to the traffic anomaly level.

[0060] Constructing the network traffic dynamic graph structure, dividing it, and analyzing it can deeply explore the internal relationships and change laws of network traffic from the perspective of the graph structure. And by obtaining the shortest path changes between different nodes and performing path change deviation detection, it is possible to more accurately discover subtle abnormal behaviors in network traffic, obtain the path deviation amount, judge whether the edge server network traffic is abnormal according to the path deviation amount, and obtain the traffic anomaly level, improving the accuracy of traffic anomaly evaluation.

[0061] First, based on the network traffic data to be processed, construct a network traffic dynamic graph structure with nodes of source IP, destination IP, port number, and protocol. The graph structure changes dynamically over time and can intuitively reflect the relationships between different network data at different times.

[0062] In the network traffic dynamic graph structure, if the source IP communicates with the destination IP through a specific port number, then create an edge between the source IP node and the corresponding port node and between the destination IP node and the corresponding port node. When the port number uses a specific protocol for data transmission, create an edge between the port number node and the corresponding protocol node. If the IP address uses a specific protocol for communication, then create an edge between the IP node and the corresponding protocol node.

[0063] Next, the network traffic dynamic graph structure is segmented according to a preset time window into several sub-graph structures for different times, arranged in chronological order to obtain several network traffic sub-graph structures.

[0064] Then, based on all the network traffic sub-graph structures, obtain the shortest path changes between the source IP node and the destination IP node, the shortest path changes between the source IP node and the protocol node, and the shortest path changes between the port number node and the corresponding protocol node when the port number node uses a specific protocol for data transmission.

[0065] Specifically, based on each network traffic sub-graph structure, obtain the shortest path between the source IP node and the destination IP node, the shortest path between the source IP node and the protocol node, and the shortest path between the port number node and the protocol node, to obtain several shortest paths between the source IP node and the destination IP node, several shortest paths between the source IP node and the protocol node, and several shortest paths between the port number node and the protocol node; count the node changes and path length changes in the shortest paths between the source IP node and the destination IP node at adjacent times, the shortest paths between the source IP node and the protocol node at adjacent times, and the shortest paths between the port number node and the protocol node at adjacent times, to obtain the shortest path changes between the source IP node and the destination IP node, the shortest path changes between the source IP node and the protocol node, and the shortest path changes between the port number node and the protocol node.

[0066] Among them, the method for obtaining the shortest path is specifically as follows: Create a distance array dist[] to store the shortest distance from the source node to each other node. Initially, set the distances of all nodes to infinity, and set the distance of the source node to 0, that is, dist[src]=0, where src is the source node. Create a set S to store the nodes for which the shortest paths have been determined. Initially, S is empty. Find the node u with the smallest distance that is not in S, and add the node u to the set S, indicating that the shortest path from the source node to u has been found; for all outgoing edges (u,v) of the node u, if dist[v]>dist[u]+w(u,v), then update the value of dist[v] to dist[u]+w(u,v), which means that a shorter path from the source node to v has been found.

[0067] When obtaining the shortest path between a source IP node and a destination IP node, using the source IP node as the source node src, execute the shortest path obtaining method. After the execution ends, the shortest distance from the source IP node to the destination IP node is stored in dist[dst], where dst is the destination IP node. To obtain the shortest path, starting from the destination IP node, the path can be constructed by backtracking the predecessor nodes. For example, record the predecessor node prev[v] of each node. When updating dist[v], if a shorter path is found, then update prev[v]=u simultaneously. In this way, starting from dst, by continuously accessing the prev array, the shortest path from the source IP node to the destination IP node can be obtained.

[0068] When obtaining the shortest path between a source IP node and a protocol node, using the source IP node as the source node src, execute the shortest path obtaining method. After the execution ends, for each protocol node p, the shortest distance from the source IP node to this protocol node is stored in dist[p]. Similar to calculating the shortest path from the source IP node to the destination IP node, the specific shortest path can be obtained by backtracking the predecessor node prev.

[0069] When obtaining the shortest path between a port number node and a protocol node, using the source IP node as the source node src, execute the shortest path obtaining method. After the execution ends, for each protocol node p, dist[p] gives the shortest distance from the port number node to the protocol node. Use the predecessor node prev array to backtrack the shortest path from the port number node to the protocol node.

[0070] Next, perform path change deviation detection on the shortest path changes between the source IP node and the destination IP node, the shortest path changes between the source IP node and the protocol node, and the shortest path changes between the port number node and the protocol node to obtain the path deviation result.

[0071] Specifically, based on the shortest path changes between the source IP node and the destination IP node, the shortest path changes between the source IP node and the protocol node, and the shortest path changes between the port number node and the protocol node, obtain their respective path offsets.

[0072] The path offset is obtained through the following formula:

[0073]

[0074] Q is the path deviation amount, β is the path node deviation weight, P1 is the set of nodes of path 1, P2 is the set of nodes of path 2, γ is the path length deviation weight, L1 is the length of path 1, L2 is the length of path 2, and J is the total number of paths.

[0075] Finally, if there is a path deviation amount greater than the corresponding path deviation threshold, the edge server network traffic is abnormal. Based on all path deviation amounts, obtain the traffic anomaly level, and perform corresponding network anomaly warnings according to the traffic anomaly level.

[0076] If the number of paths with path deviation amounts greater than the corresponding path deviation thresholds is 1, the traffic anomaly level is a low-risk level; if the number of paths with path deviation amounts greater than the corresponding path deviation thresholds is 2, the traffic anomaly level is a medium-risk level; if the number of paths with path deviation amounts greater than the corresponding path deviation thresholds is 3, the traffic anomaly level is a high-risk level.

[0077] If the traffic anomaly level is a high-risk level, take network blocking measures for the edge server, such as cutting off the connection between the source IP and the destination IP, or closing the relevant port numbers, closing the relevant network connections, etc.; if the traffic anomaly level is a medium-risk level, limit the edge server network traffic rate to the traffic rate threshold; if the traffic anomaly level is a low-risk level, delete the source IP, or destination IP, or port number or protocol with a reputation value less than the reputation value threshold. The reputation values of the source IP, or destination IP, or port number or protocol are obtained based on the respective numbers of abnormal behaviors within the historical period.

[0078] This embodiment also provides a network traffic anomaly evaluation and warning system based on artificial intelligence for implementing the above-mentioned network traffic anomaly evaluation and warning method based on artificial intelligence, including:

[0079] A network traffic data generation module to be processed, which is used to periodically obtain the network traffic data of the edge server within a preset time range. The network traffic data includes source IP information, destination IP information, port number information, and protocol information; the network traffic data is processed by data cleaning to obtain the network traffic data to be processed.

[0080] An initial detection and network anomaly warning module, which is used to obtain the entropy of the IP address, the port number distribution characteristics, the protocol traffic ratio, and the traffic time series characteristics based on the network traffic data to be processed. After mapping them into a feature matrix, perform normalization processing to obtain a network traffic feature matrix; based on the method of generating a Boolean matrix, obtain the initial detection value of the network traffic feature matrix; if the initial detection value is a first value, execute the refined detection and network anomaly warning module; if the initial detection value is greater than the first value, the edge server network traffic is abnormal. Based on the network traffic feature matrix, obtain the traffic anomaly level, and perform corresponding network anomaly warnings according to the traffic anomaly level.

[0081] The refined detection and network anomaly warning module constructs a dynamic network traffic graph structure with several nodes as source IP, destination IP, port number, and protocol based on the network traffic data to be processed; divides the dynamic network traffic graph structure according to a preset time window, arranges them in chronological order, and obtains several network traffic sub-graph structures; based on all network traffic sub-graph structures, obtains the shortest path changes between the source IP node and the destination IP node, the shortest path changes between the source IP node and the protocol node, and the shortest path changes between the port number node and the protocol node. After path change deviation detection, obtains their respective path deviation amounts; if there is a path deviation amount greater than the corresponding path deviation amount threshold, the edge server network traffic is abnormal. Based on all path deviation amounts, obtains the traffic anomaly level, and issues corresponding network anomaly warnings according to the traffic anomaly level.

[0082] A network traffic anomaly evaluation and warning method based on artificial intelligence provided in this embodiment first obtains source IP information, destination IP information, port number information, and protocol information within a preset duration range at intervals to comprehensively collect key network traffic data. After data cleaning and processing, the data quality is further improved; then, by comprehensively considering information from multiple dimensions such as the entropy of IP addresses, port number distribution characteristics, protocol traffic ratios, and traffic time series characteristics, it can more comprehensively describe the characteristics of network traffic, avoid the one-sidedness of single-characteristic analysis, and thus improve the accuracy of anomaly detection; maps these characteristics to a feature matrix and performs normalization processing, then obtains the initial detection value by generating a boolean matrix, and performs different operations according to the comparison result of the initial detection value and the first value, improving the accuracy and intelligence of network traffic anomaly evaluation; then, when it is determined based on the initial detection value that there is no network traffic anomaly, constructs a dynamic network traffic graph structure and performs segmentation and analysis, which can deeply explore the internal relationships and changing rules of network traffic from the perspective of the graph structure. By obtaining the shortest path changes between different nodes and performing path change deviation detection, it can more accurately discover subtle abnormal behaviors in network traffic, obtain the path deviation amount, judge whether the edge server network traffic is abnormal according to the path deviation amount, and obtain the traffic anomaly level, improving the accuracy of traffic anomaly evaluation; finally, taking different measures according to different traffic anomaly levels can more accurately respond to different degrees of network traffic anomaly situations, avoid overreaction or underreaction, and improve the intelligence and accuracy of network traffic anomaly evaluation, as well as the timeliness and flexibility of network traffic anomaly warning.

[0083] Although the exemplary embodiments of the present invention have been shown and described in detail herein, many other variations or modifications consistent with the principles of the present invention can still be directly determined or derived from the content disclosed in the present invention without departing from the spirit and scope of the present invention. Therefore, the scope of the present invention should be understood and recognized as covering all these other variations or modifications.

Claims

1. A network traffic anomaly evaluation and early warning method based on artificial intelligence, characterized in that: The following operations are included: S1. Periodically obtain network traffic data of edge servers within a preset time range, where the network traffic data includes source IP information, destination IP information, port number information, and protocol information; the network traffic data is cleaned to obtain network traffic data to be processed; S2. Based on the network traffic data to be processed, the entropy of the IP address, the distribution characteristics of the port number, the proportion of the protocol traffic, and the characteristics of the traffic time series are obtained, and after being mapped into a feature matrix, normalization processing is performed to obtain a network traffic feature matrix; based on the method of generating a Boolean matrix, an initial detection value of the network traffic feature matrix is ​​obtained; If the initial detection value is the first value, execute S3; If the initial detection value is greater than the first value, the network traffic of the edge server is abnormal, and the traffic anomaly level is obtained based on the network traffic feature matrix, and a corresponding network anomaly warning is issued according to the traffic anomaly level; S3. Based on the network traffic data to be processed, a network traffic dynamic graph structure with several nodes being the source IP, destination IP, port number and protocol is constructed; the network traffic dynamic graph structure is divided according to a preset time window, and arranged in chronological order to obtain several network traffic subgraph structures; Based on the structure of all network traffic subgraphs, the shortest path changes between the source IP node and the destination IP node, the shortest path changes between the source IP node and the protocol node, and the shortest path changes between the port number node and the protocol node are obtained. After path change deviation detection, the respective path deviations are obtained. If there is a path deviation greater than the corresponding path deviation threshold, the edge server network traffic is abnormal. Based on all path deviations, the traffic anomaly level is obtained, and the corresponding network anomaly warning is issued according to the traffic anomaly level.

2. The method for network traffic anomaly evaluation and early warning based on artificial intelligence according to claim 1 is characterized in that: In S1, the frequency of obtaining network traffic data within the preset time range is obtained based on the traffic load and evaluation requirements of the edge server; the specific calculation formula is as follows: f is the frequency of acquiring network traffic data, α is the traffic load factor, U is the average value of data packets, R is the resolution required for evaluation, P is the data processing capacity of the edge server, and λ is the load rate of the edge server.

3. The network traffic anomaly evaluation and early warning method based on artificial intelligence according to claim 1 is characterized in that: In S2, the method for obtaining the traffic time series characteristics is specifically as follows: After grouping the respective traffic data according to the source IP, destination IP, port number and protocol, a multidimensional data structure is created; in the multidimensional data structure, each grouped traffic data is arranged in chronological order to form several traffic time series; each traffic time series is standardized to obtain several traffic time standardized series; based on the preset time series length, each traffic time standardized series is decomposed into a combination of sine and cosine waves of different frequencies to obtain several traffic complex number arrays; based on several traffic complex number arrays, a spectrum diagram is drawn to obtain the statistical characteristics of the spectrum diagram and obtain the traffic time series characteristics.

4. The method for network traffic anomaly evaluation and early warning based on artificial intelligence according to claim 1 is characterized in that: In S2, the entropy of the IP address is obtained based on the probability distribution corresponding to the number of occurrences of the IP address in the network traffic data to be processed. The specific calculation formula is as follows: H is the entropy of the IP address, N i is the number of occurrences of the i-th IP address, and n is the total number of IP addresses.

5. The method for network traffic anomaly evaluation and early warning based on artificial intelligence according to claim 1 is characterized in that: In S1, the data cleaning process includes removing noise and duplicate data from the data, and supplementing missing values; When network data is missing, the network data is interpolated. The interpolation operation is as follows: the data in the network data except the missing values ​​is taken as samples, and the samples are input into the training decision tree for traversal, starting from the root node of the decision tree, and the corresponding branch is selected at each node according to the characteristic value of the sample until the leaf node is reached; if the leaf node stores the predicted label of the sample, the predicted label is used as the interpolation result of the missing value; if the leaf node stores numerical statistical information, the median is used as the interpolation result.

6. The method for network traffic anomaly evaluation and early warning based on artificial intelligence according to claim 1 is characterized in that: In S3, the path offset is obtained by the following formula: Q is the path deviation, β is the path node deviation weight, P t is the node set of path 1, P2 is the node set of path 2, γ is the path length deviation weight, L t is the length of path 1, L2 is the length of path 2, and J is the total number of paths.

7. The network traffic anomaly evaluation and early warning method based on artificial intelligence according to claim 1 is characterized in that: The specific operations for issuing corresponding network anomaly warnings based on the traffic anomaly level are as follows: If the traffic anomaly level is a high risk level, network blocking measures are taken for the edge server; if the traffic anomaly level is a medium risk level, the edge server network traffic rate is limited to the traffic rate threshold; If the traffic anomaly level is a low risk level, the source IP, destination IP, port number or protocol with a reputation value less than the reputation value threshold will be deleted.

8. An artificial intelligence-based network traffic anomaly evaluation and early warning system, used to implement the artificial intelligence-based network traffic anomaly evaluation and early warning method according to claim 1, characterized in that: include: The module for generating network traffic data to be processed is used to periodically obtain network traffic data of edge servers within a preset time range, the network traffic data including source IP information, destination IP information, port number information and protocol information; the network traffic data is cleaned and processed to obtain the network traffic data to be processed; The initial detection and network anomaly warning module is used to obtain the entropy of the IP address, the distribution characteristics of the port number, the proportion of the protocol traffic, and the characteristics of the traffic time series based on the network traffic data to be processed, and then normalize them after mapping them into a feature matrix to obtain the network traffic feature matrix; based on the method of generating a Boolean matrix, the initial detection value of the network traffic feature matrix is ​​obtained; If the initial detection value is the first value, executing the refined detection and network anomaly warning modules; If the initial detection value is greater than the first value, the network traffic of the edge server is abnormal, and the traffic anomaly level is obtained based on the network traffic feature matrix, and a corresponding network anomaly warning is issued according to the traffic anomaly level; The refined detection and network anomaly warning module builds a network traffic dynamic graph structure with several nodes as source IP, destination IP, port number and protocol based on the network traffic data to be processed; the network traffic dynamic graph structure is divided according to the preset time window, and arranged in chronological order to obtain several network traffic subgraph structures; Based on the structure of all network traffic subgraphs, the shortest path changes between the source IP node and the destination IP node, the shortest path changes between the source IP node and the protocol node, and the shortest path changes between the port number node and the protocol node are obtained. After path change deviation detection, the respective path deviations are obtained. If there is a path deviation greater than the corresponding path deviation threshold, the edge server network traffic is abnormal. Based on all path deviations, the traffic anomaly level is obtained, and the corresponding network anomaly warning is issued according to the traffic anomaly level.

Images