Method, device and system for monitoring abnormal network traffic based on large model

Abstract

The invention relates to the technical field of network abnormal traffic monitoring, in particular to a network abnormal traffic monitoring method, device and system based on a large model, and the method comprises the steps: collecting the network traffic data of each period at a network interface, and building a periodic network data set; determining a flow characteristic value of the same period, determining a periodic sequence of the same network protocol, and calculating a time sequence fluctuation coefficient of a period determined by each period adjacent to the period before the period; marking a target period, determining the influence weight of the characteristics of the network flow data in the target period, and performing dimensionality reduction on the network flow data of all the periods according to the influence weights of all the periods; and constructing an isolated forest model according to the dimensionality-reduced network traffic data, performing dimensionality reduction on the network traffic data for network abnormal traffic monitoring, inputting the network traffic data into the isolated forest model, and obtaining a network abnormal traffic monitoring result according to an anomaly detection result. According to the invention, the interpretability of the data after dimension reduction of the network traffic can be ensured, and the abnormal network traffic can be accurately identified.

Description

Technical Field

[0001] The present application relates to the technical field of abnormal network traffic monitoring, and in particular to a method, device and system for abnormal network traffic monitoring based on a large model. Background Art

[0002] To improve network security capabilities, ensure network service quality, and respond to emerging threats and attack methods, it is necessary to monitor abnormal network traffic. The Isolation Forest algorithm is an unsupervised anomaly detection algorithm capable of processing high-dimensional and complex network traffic data. Its training and detection processes are relatively efficient, allowing it to keep pace with changes in network traffic and meet the needs of real-time network monitoring. Because network traffic contains multiple features such as IP addresses, port numbers, protocol types, packet sizes, and time intervals, data sparsity significantly impacts the detection effectiveness of the Isolation Forest algorithm. Therefore, it is necessary to reduce the dimensionality of network traffic to obtain principal component data, and then build an Isolation Forest model based on this principal component data.

[0003] However, the dimensionality reduction process for network traffic often loses key information while reducing the data dimension, resulting in poor interpretability of the principal component data. This makes it difficult to ensure that the principal component data can reflect or distinguish abnormal network traffic. Therefore, how to reduce the dimensionality of network traffic data while retaining key information and effectively capturing abnormal behavior patterns in network traffic from the principal component data is a core issue in ensuring the accuracy of network traffic anomaly detection. Summary of the Invention

[0004] In order to solve the above technical problems, the purpose of this application is to provide a method, device and system for monitoring abnormal network traffic based on a large model. The technical solutions adopted are as follows: In a first aspect, an embodiment of the present application provides a method for monitoring abnormal network traffic based on a large model, the method comprising the following steps: Collecting network traffic data of each period at the network interface, counting the types of features included in the network traffic data, and establishing a network data set of the same period based on the network traffic data of the same period; Determine the traffic characteristic value of the same period based on the difference in the cardinality of all network data sets in the same period, the difference in the number of data packets of different target IPs, and the difference in the inbound and outbound volumes of different target ports; determine the periodic sequence of the same network protocol based on the number of data packets collected in each period; calculate the timing fluctuation coefficient of the period determined by each adjacent period before the period based on the periodic sequence and the number of data packets of all types of network protocols in the period, as well as the difference between the traffic characteristic values ​​of each adjacent period before the period; Mark any cycle as the target cycle. Based on the correlation between the traffic characteristic values ​​of the cycles adjacent to the target cycle and the characteristics of the network traffic data, as well as the time series fluctuation coefficient of the target cycle determined by the cycle immediately before the target cycle, the influence weight of the characteristics of the network traffic data in the target cycle is determined. Based on the influence weights and network traffic data of all collected cycles, the network traffic data of all cycles is reduced in dimension. An isolation forest model is constructed based on the network traffic data of all cycles of dimensionality reduction, the network traffic data of network abnormal traffic monitoring is dimensionality reduced and input into the isolation forest model, and the results of network abnormal traffic monitoring are obtained based on the anomaly detection results.

[0005] Furthermore, the process of determining the flow characteristic value of the same period is as follows: The ratio of the number of data packets of the target IP with the largest number of visits in the same period to the total number of data packets in the same period is recorded as the first ratio of the same period; Obtaining a segmentation threshold value based on the cardinality of all network data sets in the same period; recording the ratio of the total number of elements contained in all network data sets with cardinality greater than the segmentation threshold value in the same period to the total number of data packets in the same period as the second ratio of the same period; The absolute value of the difference between the inbound volume and the outbound volume of the same target port in the same period is recorded as the inbound-outbound difference of the same target port in the same period; the sum of the inbound volume and the outbound volume of the same target port in the same period is recorded as the inbound-outbound total of the same target port in the same period; and the cumulative sum of the ratios of the inbound-outbound totals of all target ports in the same period to the inbound-outbound difference is recorded as the third ratio of the same period; The cumulative sum of the first ratio, the second ratio, and the third ratio in the same period is recorded as the flow characteristic value of the same period.

[0006] Furthermore, the process of constructing the periodic sequence of the same network protocol is as follows: Arrange the number of data packets collected by the same network protocol in each period in the order of the periods to obtain a data packet number sequence of the same network protocol, perform time series decomposition on the data packet number sequence to obtain a seasonal sequence, draw an autocorrelation graph based on the seasonal sequence, determine the significant peak in the autocorrelation graph, arrange the lags corresponding to the significant peaks in descending order of the significant peaks, and obtain a periodic sequence of the same network protocol.

[0007] Furthermore, the specific process of the time series fluctuation coefficient includes: The variance of the first-order difference sequence of the periodic sequence of the network protocol is recorded as the first variance of the network protocol, and the time series fluctuation coefficient is calculated as follows: in, Indicates that according to cycles ago The time series fluctuation coefficient determined by the period; Indicates the preset adjacent cycle span parameter; Indicates the The cumulative sum of the first squared deviations of all types of network protocols within a period; Indicates the The number of network protocol types per cycle; Indicates the cycles ago In the cycle, the The number of packets of each type of network protocol; Indicates the cycles ago The total number of packets in a cycle; represents a natural constant; Indicates that according to cycles ago The volatility of data determined by the period; Indicates the cycles ago cycles to The flow characteristic values ​​of all cycles in a cycle; Indicates the cycles ago cycles to In the cycle, The flow characteristic value of a cycle; Indicates the cycles ago cycles to In the cycle, The flow characteristic value of a cycle.

[0008] Furthermore, the method of determining the influence weight of the characteristics of the network traffic data in the target period based on the correlation between the traffic characteristic values ​​of the period adjacent to the target period and the characteristics of the network traffic data, and the time series fluctuation coefficient of the target period determined by the period adjacent to the target period, includes the following specific steps: The target period and the adjacent period before the target period The traffic characteristic values ​​of the target period are arranged in the order of the periods to obtain the traffic characteristic value sequence of the target period; the mean of the characteristics of all network traffic data collected in the target period is recorded as the characteristic mean of the target period, and the target period and the adjacent period before the target period are recorded as the characteristic mean of the target period. The characteristic means of the periods are arranged in the order of the periods to obtain the characteristic mean sequence of the target period; the correlation between the characteristics of each network traffic data and the traffic characteristic value is obtained; According to the correlation between the characteristics of all network traffic data and traffic characteristic values, the target period The time series fluctuation coefficient determined by the period is used to determine the influence weight of the characteristics of network traffic data in the target period.

[0009] Furthermore, the correlation between the characteristics of all network traffic data and the traffic characteristic value, the front of the target period The time series fluctuation coefficient determined in each period is used to determine the influence weight of the characteristics of network traffic data in the target period. The specific steps include: The sum of the correlations between the characteristics of all network traffic data and the traffic characteristic values ​​is recorded as the sum of correlations. The ratio of the correlation between the characteristics of network traffic data and the traffic characteristic values ​​to the sum of correlations is used as the weight of the characteristics of network traffic data. The product of the normalized value of the time series fluctuation coefficient determined in a period and the weight of the feature of the network traffic data is recorded as the influence weight of the feature of the network traffic data in the target period.

[0010] Furthermore, the dimension reduction of the network traffic data of all periods according to the collected impact weights and network traffic data of all periods includes the following specific steps: A data matrix is ​​constructed based on the impact weights of all collected periods. The weight matrix is ​​a diagonal matrix, and the values ​​on the diagonal of the weight matrix are the impact weights of each feature of the network traffic data in the target period. A data matrix is ​​constructed based on the network traffic data. The data matrix and the weight matrix have the same number of columns. The corresponding columns of the data matrix and the weight matrix correspond to the characteristics of the same network traffic data. The network traffic data of each period are filled in the same row of the data matrix according to the type of network traffic data. The data matrix and the weight matrix of all cycles are processed using principal component analysis to obtain the network traffic data of all cycles after dimensionality reduction.

[0011] Furthermore, the network traffic data of the network abnormal traffic monitoring is reduced in dimension and input into the isolation forest model, and the results of the network abnormal traffic monitoring are obtained according to the abnormality detection results, including the specific steps of: Collect network traffic data for network abnormal traffic monitoring and reduce its dimension to obtain the reduced-dimensional network traffic data to be monitored, input the reduced-dimensional network traffic data to be monitored into the isolation forest model, and obtain the anomaly score of the reduced-dimensional network traffic data to be monitored; When the anomaly score is greater than the preset anomaly threshold, it is determined that the network traffic data of the network anomaly traffic monitoring system has network anomaly traffic; when the anomaly score is less than or equal to the anomaly threshold, it is determined that the network traffic data of the network anomaly traffic monitoring system has no network anomaly traffic.

[0012] In the second aspect, an embodiment of the present application also provides a network abnormal traffic monitoring device based on a large model, wherein the displacement monitoring device includes a memory, a processor, and a computer program stored in the memory and running on the processor, and when the processor executes the computer program, the steps of any one of the above methods are implemented.

[0013] On the third aspect, an embodiment of the present application provides a network abnormal traffic monitoring system based on a large model, and the network abnormal traffic monitoring system includes: a network traffic data acquisition module, a network traffic data feature analysis module, a network traffic data dimensionality reduction module, and a network abnormal traffic monitoring module.

[0014] A network traffic data collection module is used to collect network traffic data of each period at the network interface, count the types of features included in the network traffic data, and establish a network data set of the same period based on the network traffic data of the same period; A network traffic data characteristic analysis module is configured to determine the traffic characteristic value of the same period based on the difference in the cardinality of all network data sets in the same period, the difference in the number of data packets of different target IP addresses, and the difference in the inbound and outbound volumes of different target ports; determine the periodic sequence of the same network protocol based on the number of data packets collected in each period; and calculate the timing fluctuation coefficient of the period determined by each adjacent period before the period based on the periodic sequence of all types of network protocols in the period, the number of data packets, and the difference between the traffic characteristic values ​​of each adjacent period before the period; The network traffic data dimensionality reduction module is used to record any cycle as the target cycle, determine the influence weight of the network traffic data characteristics in the target cycle based on the correlation between the traffic characteristic values ​​of the cycles adjacent to the target cycle and the characteristics of the network traffic data, and the time series fluctuation coefficient of the target cycle determined by the cycles adjacent to the target cycle, and reduce the dimensionality of the network traffic data of all cycles based on the influence weights and network traffic data collected from all cycles; The network abnormal traffic monitoring module is used to build an isolation forest model based on the network traffic data of all cycles of dimensionality reduction, reduce the dimensionality of the network traffic data for network abnormal traffic monitoring and input it into the isolation forest model, and obtain the results of network abnormal traffic monitoring based on the anomaly detection results.

[0015] As can be seen from the above embodiments, the large model-based network abnormal traffic monitoring method, device, and system provided in the embodiments of the present application have at least the following beneficial effects: In order to improve the interpretability of the principal component data obtained by dimensionality reduction and enable the principal component data after dimensionality reduction to accurately reflect the abnormal situation of network traffic, the present application evaluates the abnormal situation of network traffic in a cycle according to the interactivity of network access requests and the pertinence of network attacks on a target IP, obtains the traffic characteristic value of the cycle, and analyzes the fluctuation of network traffic data of all the same network protocols over time according to the different requirements of different services on network protocols in the network environment, and obtains the time series fluctuation coefficient determined by the first preset number of cycles of each cycle; further, the characteristics of network traffic data are determined respectively through the correlation between the access characteristics of network traffic in the cycle and the network traffic of adjacent cycles and the abnormal situation of network traffic in the cycle. The influence weight of each period is characterized, and based on the influence weights and network traffic data of all periods collected, the network traffic data of all periods is reduced in dimension to improve the accuracy of network abnormal traffic monitoring and ensure that the features obtained after dimensionality reduction can reflect the abnormal situation of network traffic data; finally, an isolation forest model is constructed based on the network traffic data of all periods with reduced dimension, and the network traffic data of network abnormal traffic monitoring is reduced in dimension and input into the isolation forest model. The results of network abnormal traffic monitoring are obtained according to the anomaly detection results, which solves the problem that the data interpretability is poor and key information is lost due to network traffic dimensionality reduction, and the accurate identification of network abnormal traffic cannot be guaranteed. It can ensure the interpretability of the data after network traffic dimensionality reduction and accurately identify network abnormal traffic. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions and advantages of the embodiments of the present application or the prior art, the following is a brief introduction to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0017] Figure 1 A flowchart of a method for monitoring abnormal network traffic based on a large model according to an embodiment of the present application; Figure 2 A schematic diagram of the structure of a large model-based network abnormal traffic monitoring system provided in one embodiment of the present application. DETAILED DESCRIPTION

[0018] In order to further illustrate the technical means and effects adopted by this application to achieve the predetermined invention purpose, the following, in conjunction with the accompanying drawings and preferred embodiments, describes in detail the network abnormal traffic monitoring method, device and system based on the large model proposed in this application, its specific implementation method, structure, features and effects. In the following description, different "one embodiment" or "another embodiment" does not necessarily refer to the same embodiment. In addition, specific features, structures or characteristics in one or more embodiments may be combined in any suitable form.

[0019] The specific scheme of the network abnormal traffic monitoring method, device and system based on large model provided by this application is described in detail below with reference to the accompanying drawings.

[0020] See also Figure 1 , which shows a flowchart of a method for monitoring abnormal network traffic based on a large model according to an embodiment of the present application, the method comprising the following steps: S001: collecting network traffic data of each period at a network interface, counting the types of features included in the network traffic data, and establishing a network data set of the same period based on the network traffic data of the same period.

[0021] Use Wireshark to monitor the network interface and collect network traffic data. This data contains both numerical and non-numerical data. Sequential encoding is used to convert non-numerical data into numerical data. Combine the network traffic data collected within a cycle into a network data set. Specifically, group network traffic data within the same cycle with identical source IP addresses, destination IP addresses, source ports, destination ports, and transport protocol types into the same network data set. This results in a network data set corresponding to the network traffic data within the same cycle.

[0022] Preferably, in one embodiment of the present application, when collecting network traffic data, 10 seconds is used as a cycle of network traffic data, and a cycle of network traffic data is collected every 10 minutes, and all network traffic data within a week is collected as a sample. In actual application, as other implementation methods, the implementer can determine the length of a cycle of network traffic data and the time interval between collecting network traffic data of different cycles based on actual conditions, and this application does not impose any special restrictions.

[0023] The types of features included in all network traffic data collected. The network traffic data features selected in this embodiment include source IP address, destination IP address, source port, destination port, and transport protocol type, totaling five types. In actual application, as other implementation methods, implementers can determine the types and number of types of features included in all network traffic data based on actual circumstances, and this application does not impose any special restrictions.

[0024] At this point, the network traffic data and network data collection of each cycle are obtained.

[0025] S002: Determine the traffic characteristic value of the same period based on the difference in the cardinality of all network data sets of the same period, the difference in the number of data packets of different target IPs, and the difference in the inbound and outbound volumes of different target ports; determine the periodic sequence of the same network protocol based on the number of data packets collected in each period; calculate the timing fluctuation coefficient of the period determined by the adjacent periods before the period based on the periodic sequence, the number of data packets of all types of network protocols in the period, and the difference between the traffic characteristic values ​​of the adjacent periods before the period.

[0026] In order to improve the interpretability of the principal component data obtained by dimensionality reduction and enable the principal component data after dimensionality reduction to accurately reflect the abnormal situation of network traffic, it is necessary to conduct statistical analysis on the network traffic data in combination with the access characteristics of the network traffic.

[0027] First, in a normal network environment, because network access requests are interactive, network traffic is often bidirectional, meaning that the outbound and inbound traffic volumes are relatively close. However, when network traffic becomes abnormal, attackers create a large amount of abnormal one-way traffic to disrupt normal network use and prevent access. Therefore, they create a large amount of abnormal one-way traffic to clog the network, causing a sharp increase in one-way traffic. Second, attackers control zombie machines to send a large number of data packets in a short period of time, resulting in a large number of data packets with the same source IP address, destination IP address, source port, destination port, and transport protocol type. Finally, network attacks are launched against a single target IP, resulting in a large number of access requests to that target IP during the abnormal traffic period.

[0028] The traffic characteristic value of the same period is determined based on the difference in the cardinality of all network data sets in the same period, the difference in the number of data packets of different target IPs, and the difference in the inbound and outbound volumes of different target ports.

[0029] The cardinality of all network data sets in the same period is divided using the OTSU maximum inter-class variance method to obtain the segmentation threshold.

[0030] Among them, using the OTSU maximum inter-class variance method to obtain the segmentation threshold is a well-known technology and will not be described in detail; the cardinality of a set is a basic concept in set theory, which represents the number of elements in a set and will not be described in detail.

[0031] The ratio of the number of data packets of the target IP with the largest number of visits in the same period to the total number of data packets in the same period is recorded as the first ratio of the same period; the ratio of the total number of elements contained in all network data sets with a cardinality greater than the segmentation threshold in the same period to the total number of data packets in the same period is recorded as the second ratio of the same period; the absolute value of the difference between the inbound and outbound volumes of the same target port in the same period is recorded as the inbound and outbound difference of the same target port in the same period, the sum of the inbound and outbound volumes of the same target port in the same period is recorded as the inbound and outbound sum of the same target port in the same period, the cumulative sum of the ratios of the inbound and outbound sums of all target ports in the same period to the inbound and outbound differences is recorded as the third ratio of the same period; the cumulative sum of the first ratio, the second ratio and the third ratio of the same period is recorded as the traffic characteristic value of the same period.

[0032] The traffic characteristic value of a period is used to evaluate the network traffic anomaly within the period. It is an evaluation obtained based on the instantaneous characteristics of the network traffic data. When there is a network traffic anomaly, the bidirectionality of the network traffic will be damaged to a certain extent, the unidirectional traffic will increase sharply, and the outbound and inbound traffic of the target IP will differ greatly, resulting in a larger third ratio of the period; at the same time, a large number of data packets with the same source IP address, target IP address, source port, target port, and transmission protocol type will appear, and a large number of access request data packets will appear for a target IP, making the traffic characteristic value of the period larger.

[0033] Network traffic data changes continuously over time, showing certain time series characteristics and periodicity. By analyzing the periodic characteristics in time series, the evaluation accuracy of abnormal changes in network traffic can be further improved.

[0034] Different services within a network environment have varying requirements, often requiring the use of different network protocols. For example, to ensure reliable and secure file transfer, file transfer protocols such as FTP and SFTP are required. To meet the low-latency, real-time requirements of video access, streaming protocols such as RTSP, HLS, and MPEG-DASH are required. Web browsing often uses protocols such as HTTP and HTTPS. Because different service requirements exhibit a certain periodicity, the timing of requests for the same protocol also follows a certain regularity. If a large number of unusual protocol requests occur at unusual times, there is a greater likelihood of significant abnormal traffic. Secondly, under normal circumstances, the packet sizes corresponding to different protocols often exhibit certain regularities. For example, web browsing typically generates small HTTP requests and responses, while video streaming typically has a fixed bitrate. If packets are significantly larger or smaller than normal, there is a greater likelihood of network traffic anomalies. Finally, when attackers control zombie machines to launch network attacks, they often launch sustained attacks over a certain period of time, resulting in a high degree of consistency in abnormal traffic data over several consecutive cycles.

[0035] According to the number of data packets collected by the same network protocol in each period, the periodic sequence of the same network protocol is determined.

[0036] The number of data packets collected by the same network protocol in each cycle is arranged in the order of the cycles to obtain a data packet number sequence of the same network protocol, the data packet number sequence is processed using the STL time series decomposition algorithm to obtain a seasonal sequence corresponding to the data packet number sequence, the autocorrelation coefficient of the seasonal sequence is calculated, and an ACF graph is drawn. The ACF graph is also an autocorrelation graph. The confidence interval method is used to draw a 99% confidence interval based on the autocorrelation graph. The peak value of the autocorrelation coefficient that exceeds the confidence interval is recorded as a significant peak. The lag amount corresponding to the significant peak is arranged in descending order of the significant peak to obtain the periodic sequence of the same network protocol.

[0037] Among them, using the STL time series decomposition algorithm to obtain the seasonal series corresponding to the sequence, calculating the autocorrelation coefficient of the seasonal series, drawing the ACF graph, and using the confidence interval method to determine the significant peak are all well-known technologies and will not be repeated here.

[0038] At this point, the periodic sequence of each network protocol can be obtained.

[0039] The timing fluctuation coefficient of the period determined by each adjacent period before the period is calculated according to the periodic sequences of all types of network protocols in the period, the number of data packets, and the difference between the traffic characteristic values ​​of each adjacent period before the period.

[0040] The variance of the first-order difference sequence of the periodic sequence of the network protocol is recorded as the first variance of the network protocol.

[0041] Furthermore, the time series fluctuation coefficient determined by each cycle before the cycle is calculated, and the calculation formula is: in, Indicates that according to cycles ago The time series fluctuation coefficient determined by the period; Indicates the The cumulative sum of the first squared deviations of all types of network protocols within a period; Indicates the The number of network protocol types per cycle; Indicates the cycles ago In the cycle, the The number of packets of each type of network protocol; Indicates the cycles ago The total number of packets in a cycle; represents a natural constant; Indicates that according to cycles ago The volatility of data determined by the period; Indicates the cycles ago cycles to The flow characteristic values ​​of all cycles in a cycle; Indicates the cycles ago cycles to In the cycle, The flow characteristic value of a cycle; Indicates the cycles ago cycles to In the cycle, The flow characteristic value of a cycle.

[0042] The time series fluctuation coefficient is used to measure the fluctuation of network traffic data over time. In the process of calculating the time series fluctuation coefficient, the data volatility is placed at the exponential position of an exponential function with a natural constant as the base. The purpose is to ensure that when the network traffic fluctuation between adjacent cycles is not obvious, the network traffic anomaly can still be measured through other features, thereby improving the accuracy of the network traffic anomaly measurement.

[0043] When network traffic anomalies occur within a period, the periodicity of some network protocols will be destroyed, resulting in a larger cumulative sum of the first variances of all types of network protocols within the period; at the same time, the size of the data packet for network access requests will be abnormal, resulting in smaller data volatility. At this time, the determined timing fluctuation coefficient is larger.

[0044] in, Indicates the adjacent cycle span parameter. In this embodiment, The value of is 10. In actual application, as other implementation methods, the implementer can decide the value of the adjacent cycle span parameter according to the actual situation. This application does not impose any special restrictions. It should be noted that when the other adjacent cycles before a certain cycle are insufficient, When a cycle is not analyzed, no analysis is performed on that cycle.

[0045] At this point, the timing fluctuation coefficient determined by the adjacent period before any period is obtained.

[0046] S003: Record any cycle as the target cycle, and determine the influence weight of the characteristics of the network traffic data in the target cycle based on the correlation between the traffic characteristic values ​​of the cycles adjacent to the target cycle and the characteristics of the network traffic data, as well as the timing fluctuation coefficient of the target cycle determined by the cycle adjacent to the target cycle. Based on the influence weights and network traffic data of all collected cycles, reduce the dimension of the network traffic data of all cycles.

[0047] When using isolation forests for outlier detection, network traffic data requires dimensionality reduction because it has too many features. To ensure that the features after dimensionality reduction can accurately extract the characteristics of abnormal network traffic, dimensionality reduction cannot be performed based solely on the size changes of the data. This application improves the accuracy of abnormal network traffic monitoring by combining the access characteristics of network traffic within a cycle and the time series characteristics of network traffic between adjacent cycles, ensuring that the features obtained after dimensionality reduction can reflect the abnormalities of network traffic data.

[0048] The traffic characteristic value of a period is used to evaluate network traffic anomalies within that period. This evaluation is based on the instantaneous characteristics of network traffic data. The time-series fluctuations of the traffic characteristic values ​​of each period can, to a certain extent, reflect the state of the network traffic data. Therefore, when the time-series changes of a feature of the network traffic data are highly similar to the time-series changes of the traffic characteristic value, it indicates that the feature is more likely to have influenced the fluctuations of the network traffic data. Furthermore, the time-series fluctuation coefficient is used to measure the fluctuations of network traffic data over time. The greater the correlation between a feature of the network traffic data and the time-series changes of the traffic characteristic value, the more accurately the feature measures the network traffic situation, and therefore, it should be given a higher weight.

[0049] Any period is marked as the target period. The influence weight of the characteristics of the network traffic data in the target period is determined based on the correlation between the traffic characteristic values ​​of the periods adjacent to the target period and the characteristics of the network traffic data, as well as the time series fluctuation coefficient of the target period determined by the period adjacent to the target period.

[0050] The target period and the adjacent period before the target period Arrange the traffic characteristic values ​​of the target period in the order of the periods to obtain the traffic characteristic value sequence of the target period; record the mean of the characteristics of all network traffic data collected in the target period as the characteristic mean of the target period, and record the target period and the adjacent period before the target period as the characteristic mean. The characteristic means of each cycle are arranged in the order of the cycles to obtain the characteristic mean sequence of the target cycle; the grey correlation analysis method is used to obtain the correlation between the characteristics of each network traffic data and the traffic characteristic value.

[0051] The sum of the correlations between the characteristics of all network traffic data and the traffic characteristic values ​​is recorded as the sum of correlations. The ratio of the correlation between the characteristics of network traffic data and the traffic characteristic values ​​to the sum of correlations is used as the weight of the characteristics of network traffic data. The product of the normalized value of the time series fluctuation coefficient determined in a period and the weight of the feature of the network traffic data is recorded as the influence weight of the feature of the network traffic data in the target period.

[0052] It should be noted that this embodiment uses the Z-Score standard normalization method to calculate the normalized value. In actual application, the implementer can use other existing methods such as maximum and minimum value normalization method, sigmoid function, etc. to calculate the normalized value, which is not limited here.

[0053] The greater the influence weight of the characteristics of the network traffic data in the target period, the more important the characteristics of the network traffic data are in the process of network traffic data dimensionality reduction, and the more accurately the characteristics of the network traffic data can identify abnormal network traffic.

[0054] The data matrix and the weight matrix of each period are constructed according to the impact weights and network traffic data of all collected periods.

[0055] Specifically, as an embodiment of the present application, the weight matrix is ​​a diagonal matrix, and the values ​​on the diagonal of the weight matrix are the values ​​of the influence weights of each feature of the network traffic data in the target period; the data matrix and the weight matrix have the same number of columns, and the corresponding columns of the data matrix and the weight matrix correspond to the features of the same network traffic data. The network traffic data of each period is filled in the same row of the data matrix according to the type of network traffic data.

[0056] Principal component analysis is used to process the data matrix and the weight matrix of all cycles, reduce the dimension of the network traffic data of all cycles, and obtain the principal components of all cycles, that is, the network traffic data of all cycles after dimension reduction.

[0057] Among them, using principal component analysis to reduce the dimension of data is a well-known technology and will not be described in detail.

[0058] At this point, the network traffic data of all cycles after dimensionality reduction is obtained.

[0059] S004: Construct an isolation forest model based on the network traffic data of all cycles of dimension reduction, reduce the dimension of the network traffic data of network abnormal traffic monitoring and input it into the isolation forest model, and obtain the results of network abnormal traffic monitoring based on the anomaly detection results.

[0060] The isolation forest algorithm is used to process the network traffic data of all cycles after dimensionality reduction. samples, the number of isolated trees is set to , construct an isolation forest model.

[0061] in, represents the first preset threshold, represents the second preset threshold. In this embodiment, the value of the first preset threshold is 50, and the value of the second preset threshold is 100.

[0062] For the network interfaces that need to monitor network abnormal traffic, the network traffic data of each period is collected. The network traffic data of all periods that need to be monitored for network abnormal traffic after dimensionality reduction is obtained in the same way as the network traffic data of all periods after dimensionality reduction when building the isolation forest model. The network traffic data of all periods that need to be monitored for network abnormal traffic after dimensionality reduction is obtained and recorded as the reduced-dimensional network traffic data to be monitored. The reduced-dimensional network traffic data to be monitored is input into the isolation forest model to obtain the anomaly score corresponding to the reduced-dimensional network traffic data to be monitored.

[0063] When the anomaly score is greater than the preset anomaly threshold, it is determined that the network traffic data that needs to be monitored for network anomaly traffic has network anomaly traffic; when the anomaly score is less than or equal to the preset anomaly threshold, it is determined that the network traffic data that needs to be monitored for network anomaly traffic does not have network anomaly traffic.

[0064] In this embodiment, the abnormal threshold value is 0.5, and the abnormal threshold value should be greater than or equal to 0.3 and less than or equal to 0.7.

[0065] At this point, abnormal network traffic monitoring is achieved.

[0066] Based on the same inventive concept as the above method, an embodiment of the present application also provides a large model-based network abnormal traffic monitoring device, including a memory, a processor, and a computer program stored in the memory and running on the processor. When the processor executes the computer program, the steps of any one of the above-mentioned large model-based network abnormal traffic monitoring methods are implemented.

[0067] See also Figure 2 , Figure 2 This is a schematic diagram of the structure of a network abnormal traffic monitoring system based on a large model provided by an embodiment of the present application. In this embodiment, the network abnormal traffic monitoring system includes various units for executing the various steps in the corresponding embodiment of the network abnormal traffic monitoring method based on a large model. Figure 2 ,The network abnormal traffic monitoring system includes : a network traffic data acquisition module, a ,network traffic data feature analysis module, a network traffic data ,dimensionality reduction module, and a network abnormal traffic monitoring module.

[0068] A network traffic data collection module is used to collect network traffic data of each period at the network interface, count the types of features included in the network traffic data, and establish a network data set of the same period based on the network traffic data of the same period; A network traffic data characteristic analysis module is configured to determine the traffic characteristic value of the same period based on the difference in the cardinality of all network data sets in the same period, the difference in the number of data packets of different target IP addresses, and the difference in the inbound and outbound volumes of different target ports; determine the periodic sequence of the same network protocol based on the number of data packets collected in each period; and calculate the timing fluctuation coefficient of the period determined by each adjacent period before the period based on the periodic sequence of all types of network protocols in the period, the number of data packets, and the difference between the traffic characteristic values ​​of each adjacent period before the period; The network traffic data dimensionality reduction module is used to record any cycle as the target cycle, determine the influence weight of the network traffic data characteristics in the target cycle based on the correlation between the traffic characteristic values ​​of the cycles adjacent to the target cycle and the characteristics of the network traffic data, and the time series fluctuation coefficient of the target cycle determined by the cycles adjacent to the target cycle, and reduce the dimensionality of the network traffic data of all cycles based on the influence weights and network traffic data collected from all cycles; The network abnormal traffic monitoring module is used to build an isolation forest model based on the network traffic data of all cycles of dimensionality reduction, reduce the dimensionality of the network traffic data for network abnormal traffic monitoring and input it into the isolation forest model, and obtain the results of network abnormal traffic monitoring based on the anomaly detection results.

[0069] The above content is only an implementation method of the present application and is not intended to limit the scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the scope of protection of the present application.

Claims

1. A network abnormal traffic monitoring method based on a large model, characterized in that: The method comprises the following steps: Collecting network traffic data of each period at the network interface, counting the types of features included in the network traffic data, and establishing a network data set of the same period based on the network traffic data of the same period; Determine the traffic characteristic value of the same period based on the difference in the cardinality of all network data sets in the same period, the difference in the number of data packets of different target IPs, and the difference in the inbound and outbound volumes of different target ports; determine the periodic sequence of the same network protocol based on the number of data packets collected in each period; calculate the timing fluctuation coefficient of the period determined by each adjacent period before the period based on the periodic sequence and the number of data packets of all types of network protocols in the period, as well as the difference between the traffic characteristic values ​​of each adjacent period before the period; Mark any cycle as the target cycle. Based on the correlation between the traffic characteristic values ​​of the cycles adjacent to the target cycle and the characteristics of the network traffic data, as well as the time series fluctuation coefficient of the target cycle determined by the cycle immediately before the target cycle, the influence weight of the characteristics of the network traffic data in the target cycle is determined. Based on the influence weights and network traffic data of all collected cycles, the network traffic data of all cycles is reduced in dimension. An isolation forest model is constructed based on the network traffic data of all cycles of dimensionality reduction, the network traffic data of network abnormal traffic monitoring is dimensionality reduced and input into the isolation forest model, and the results of network abnormal traffic monitoring are obtained based on the anomaly detection results.

2. The method for monitoring abnormal network traffic based on a large model according to claim 1, wherein: The process of determining the flow characteristic value of the same period is as follows: The ratio of the number of data packets of the target IP with the largest number of visits in the same period to the total number of data packets in the same period is recorded as the first ratio of the same period; Obtaining a segmentation threshold value based on the cardinality of all network data sets in the same period; recording the ratio of the total number of elements contained in all network data sets with cardinality greater than the segmentation threshold value in the same period to the total number of data packets in the same period as the second ratio of the same period; The absolute value of the difference between the inbound volume and the outbound volume of the same target port in the same period is recorded as the inbound-outbound difference of the same target port in the same period; the sum of the inbound volume and the outbound volume of the same target port in the same period is recorded as the inbound-outbound total of the same target port in the same period; and the cumulative sum of the ratios of the inbound-outbound totals of all target ports in the same period to the inbound-outbound difference is recorded as the third ratio of the same period; The cumulative sum of the first ratio, the second ratio, and the third ratio in the same period is recorded as the flow characteristic value of the same period.

3. The method for monitoring abnormal network traffic based on a large model according to claim 1, wherein: The process of constructing the periodic sequence of the same network protocol is as follows: Arrange the number of data packets collected by the same network protocol in each period in the order of the periods to obtain a data packet number sequence of the same network protocol, perform time series decomposition on the data packet number sequence to obtain a seasonal sequence, draw an autocorrelation graph based on the seasonal sequence, determine the significant peak in the autocorrelation graph, arrange the lags corresponding to the significant peaks in descending order of the significant peaks, and obtain a periodic sequence of the same network protocol.

4. The method for monitoring abnormal network traffic based on a large model according to claim 1, wherein: The specific process of the time series fluctuation coefficient includes: The variance of the first-order difference sequence of the periodic sequence of the network protocol is recorded as the first variance of the network protocol, and the time series fluctuation coefficient is calculated as follows: in, Indicates that according to cycles ago The time series fluctuation coefficient determined by the period; Indicates the preset adjacent cycle span parameter; Indicates the The cumulative sum of the first squared deviations of all types of network protocols within a period; Indicates the The number of network protocol types per cycle; Indicates the cycles ago In the cycle, the The number of packets of each type of network protocol; Indicates the cycles ago The total number of packets in a cycle; represents a natural constant; Indicates that according to cycles ago The volatility of data determined by the period; Indicates the cycles ago cycles to The flow characteristic values ​​of all cycles in a cycle; Indicates the cycles ago cycles to In the cycle, The flow characteristic value of a cycle; Indicates the cycles ago cycles to In the cycle, The flow characteristic value of a cycle.

5. The method for monitoring abnormal network traffic based on a large model according to claim 4, wherein: The method of determining the influence weight of the characteristics of the network traffic data in the target period based on the correlation between the traffic characteristic values ​​of the period adjacent to the target period and the characteristics of the network traffic data, and the time series fluctuation coefficient of the target period determined by the period adjacent to the target period, includes the following specific steps: The target period and the adjacent period before the target period The traffic characteristic values ​​of the target period are arranged in the order of the periods to obtain the traffic characteristic value sequence of the target period; the mean of the characteristics of all network traffic data collected in the target period is recorded as the characteristic mean of the target period, and the target period and the adjacent period before the target period are recorded as the characteristic mean of the target period. The characteristic means of the periods are arranged in the order of the periods to obtain the characteristic mean sequence of the target period; the correlation between the characteristics of each network traffic data and the traffic characteristic value is obtained; According to the correlation between the characteristics of all network traffic data and traffic characteristic values, the target period The time series fluctuation coefficient determined by the period is used to determine the influence weight of the characteristics of network traffic data in the target period.

6. The method for monitoring abnormal network traffic based on a large model according to claim 5, characterized in that: The correlation between the characteristics of all network traffic data and traffic characteristic values, the target period The time series fluctuation coefficient determined in each period is used to determine the influence weight of the characteristics of network traffic data in the target period. The specific steps include: The sum of the correlations between the characteristics of all network traffic data and the traffic characteristic values ​​is recorded as the sum of correlations. The ratio of the correlation between the characteristics of network traffic data and the traffic characteristic values ​​to the sum of correlations is used as the weight of the characteristics of network traffic data. The product of the normalized value of the time series fluctuation coefficient determined in a period and the weight of the feature of the network traffic data is recorded as the influence weight of the feature of the network traffic data in the target period.

7. The method for monitoring abnormal network traffic based on a large model according to claim 1, wherein: The specific steps of reducing the dimension of the network traffic data of all periods according to the collected impact weights and network traffic data of all periods are as follows: A data matrix is ​​constructed based on the impact weights of all collected periods. The weight matrix is ​​a diagonal matrix, and the values ​​on the diagonal of the weight matrix are the impact weights of each feature of the network traffic data in the target period. A data matrix is ​​constructed based on the network traffic data. The data matrix and the weight matrix have the same number of columns. The corresponding columns of the data matrix and the weight matrix correspond to the characteristics of the same network traffic data. The network traffic data of each period are filled in the same row of the data matrix according to the type of network traffic data. The data matrix and the weight matrix of all cycles are processed using principal component analysis to obtain the network traffic data of all cycles after dimensionality reduction.

8. The method for monitoring abnormal network traffic based on a large model according to claim 1, wherein: The network traffic data of the network abnormal traffic monitoring is reduced in dimension and input into the isolation forest model, and the network abnormal traffic monitoring result is obtained according to the abnormality detection result, including the specific steps of: Collect network traffic data for network abnormal traffic monitoring and reduce its dimension to obtain the reduced-dimensional network traffic data to be monitored, input the reduced-dimensional network traffic data to be monitored into the isolation forest model, and obtain the anomaly score of the reduced-dimensional network traffic data to be monitored; When the anomaly score is greater than the preset anomaly threshold, it is determined that the network traffic data of the network anomaly traffic monitoring system has network anomaly traffic; when the anomaly score is less than or equal to the anomaly threshold, it is determined that the network traffic data of the network anomaly traffic monitoring system has no network anomaly traffic.

9. A network abnormal traffic monitoring device based on a large model, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 8 are implemented.

10. A network abnormal traffic monitoring system based on a large model, implementing the method according to claim 1, characterized in that: The network abnormal traffic monitoring system includes: A network traffic data collection module is used to collect network traffic data of each period at the network interface, count the types of features included in the network traffic data, and establish a network data set of the same period based on the network traffic data of the same period; A network traffic data characteristic analysis module is configured to determine the traffic characteristic value of the same period based on the difference in the cardinality of all network data sets in the same period, the difference in the number of data packets of different target IP addresses, and the difference in the inbound and outbound volumes of different target ports; determine the periodic sequence of the same network protocol based on the number of data packets collected in each period; and calculate the timing fluctuation coefficient of the period determined by each adjacent period before the period based on the periodic sequence of all types of network protocols in the period, the number of data packets, and the difference between the traffic characteristic values ​​of each adjacent period before the period; The network traffic data dimensionality reduction module is used to record any cycle as the target cycle, determine the influence weight of the network traffic data characteristics in the target cycle based on the correlation between the traffic characteristic values ​​of the cycles adjacent to the target cycle and the characteristics of the network traffic data, and the time series fluctuation coefficient of the target cycle determined by the cycles adjacent to the target cycle, and reduce the dimensionality of the network traffic data of all cycles based on the influence weights and network traffic data collected from all cycles; The network abnormal traffic monitoring module is used to build an isolation forest model based on the network traffic data of all cycles of dimensionality reduction, reduce the dimensionality of the network traffic data for network abnormal traffic monitoring and input it into the isolation forest model, and obtain the results of network abnormal traffic monitoring based on the anomaly detection results.

Images