Big model-based network abnormal traffic monitoring method, device and system

Abstract

The application relates to the technical field of network abnormal traffic monitoring, in particular to a network abnormal traffic monitoring method, device and system based on a large model, which comprises the following steps: collecting network traffic data of each period at a network interface, and establishing a network data set of the period; determining a traffic characteristic value of the same period, determining a periodic sequence of the same network protocol, and calculating a time sequence fluctuation coefficient of each period before the period; marking a target period, determining the influence weight of the characteristics of the network traffic data in the target period, and reducing the dimension of the network traffic data of all periods according to the influence weight; constructing an isolated forest model according to the reduced network traffic data, reducing the dimension of the network abnormal traffic monitoring network traffic data and inputting the isolated forest model, and obtaining the network abnormal traffic monitoring result according to the abnormal detection result. The application can guarantee the interpretability of the network traffic data after dimension reduction, and accurately identify the network abnormal traffic.

Description

Technical Field

[0001] This application relates to the field of network abnormal traffic monitoring technology, specifically to network abnormal traffic monitoring methods, devices and systems based on large models. Background Technology

[0002] To enhance network security capabilities, ensure network service quality, and address emerging threats and attack methods, it is necessary to monitor abnormal network traffic. Isolation Forest is an unsupervised anomaly detection algorithm capable of handling high-dimensional and complex network traffic data. Furthermore, its training and detection processes are relatively efficient, allowing it to keep pace with changes in network traffic and meet the needs of real-time network monitoring. However, network traffic contains various features such as IP addresses, port numbers, protocol types, packet sizes, and time intervals, resulting in significant data sparsity. This sparsity severely impacts the detection performance of the Isolation Forest algorithm. Therefore, it is necessary to reduce the dimensionality of the network traffic data, obtain principal component data, and then build the Isolation Forest model based on this principal component data.

[0003] However, the dimensionality reduction process of network traffic data often loses key information while reducing data dimensionality, resulting in poor interpretability of the acquired principal component data and making it difficult to guarantee that the acquired principal component data can reflect or distinguish abnormal network traffic. Therefore, how to retain key information while reducing the dimensionality of network traffic data and effectively capture abnormal behavior patterns in network traffic from principal component data is the core issue for ensuring the accuracy of network traffic anomaly detection. Summary of the Invention

[0004] To address the aforementioned technical problems, the purpose of this application is to provide a method, device, and system for monitoring abnormal network traffic based on a large model. The specific technical solution adopted is as follows:

[0005] In a first aspect, embodiments of this application provide a method for monitoring abnormal network traffic based on a large model, the method comprising the following steps:

[0006] Collect network traffic data for each period at the network interface, count the types of characteristics included in the network traffic data, and establish a network data set for the same period based on the network traffic data for the same period.

[0007] Based on the differences in the base values ​​of all network data sets in the same period, the differences in the number of data packets for different target IPs, and the differences in the inbound and outbound traffic for different target ports, the traffic characteristic values ​​for the same period are determined. Based on the number of data packets collected by the same network protocol in each period, the periodic sequence of the same network protocol is determined. Based on the periodic sequences of all types of network protocols in the period, the number of data packets, and the differences in the traffic characteristic values ​​between the periods preceding the current period, the time series fluctuation coefficient of the period determined by the periods preceding the current period is calculated.

[0008] Any period is designated as the target period. Based on the correlation between the traffic characteristic values ​​of the periods adjacent to the target period and the characteristics of the network traffic data, as well as the time-series fluctuation coefficient of the target period determined by the periods preceding the target period, the influence weight of the network traffic data characteristics in the target period is determined. Based on the influence weights of all collected periods and the network traffic data, the dimensionality of the network traffic data in all periods is reduced.

[0009] An isolated forest model is constructed based on network traffic data from all periods of dimensionality reduction. The network traffic data from anomaly traffic monitoring is then dimensionality-reduced and input into the isolated forest model. The results of anomaly traffic monitoring are obtained based on the anomaly detection results.

[0010] Furthermore, the process for determining the flow characteristic values ​​of the same period is as follows:

[0011] The ratio of the number of data packets from the target IP with the highest access volume in the same period to the total number of data packets in the same period is denoted as the first ratio of the same period.

[0012] The segmentation threshold is obtained based on the cardinality of all network data sets in the same period; the ratio of the total number of elements contained in all network data sets with a cardinality greater than the segmentation threshold in the same period to the total number of data packets in the same period is denoted as the second ratio of the same period.

[0013] The absolute value of the difference between the inbound and outbound volumes of the same target port within the same period is denoted as the inbound / outbound difference of the same target port within the same period. The sum of the inbound and outbound volumes of the same target port within the same period is denoted as the total inbound / outbound volume of the same target port within the same period. The cumulative sum of the ratios of the total inbound / outbound volume of all target ports within the same period to the inbound / outbound difference is denoted as the third ratio of the same period.

[0014] The sum of the first ratio, the second ratio, and the third ratio for the same period is recorded as the flow characteristic value for the same period.

[0015] Furthermore, the process of constructing the periodic sequence of the same network protocol is as follows:

[0016] The number of data packets collected by the same network protocol in each period is arranged in chronological order to obtain the data packet number sequence of the same network protocol. The data packet number sequence is decomposed into a time series to obtain a seasonal sequence. An autocorrelation plot is drawn based on the seasonal sequence, and significant peaks in the autocorrelation plot are determined. The lags corresponding to the significant peaks are arranged in descending order of the significant peaks to obtain the periodic sequence of the same network protocol.

[0017] Furthermore, the specific process for obtaining the time-series fluctuation coefficient includes:

[0018] The variance of the first-order difference sequence of the periodic sequence of the network protocol is denoted as the first variance of the network protocol. The time series fluctuation coefficient is calculated using the following formula:

[0019]

[0020]

[0021] in, Indicates according to the first The previous cycle The time-series fluctuation coefficient determined by each period; This indicates the preset adjacent period span parameter; Indicates the first The sum of the first variances of all types of network protocols within a given period; Indicates the first The number of network protocol types per cycle; Indicates the first The previous cycle The first cycle is used The number of data packets for each type of network protocol; Indicates the first The previous cycle Total number of data packets in each cycle; Represents the natural constant; Indicates according to the first The previous cycle Data volatility determined over a given period; Indicates the first The previous cycle The cycle to the first The flow characteristic values ​​for all cycles within a given period; Indicates the first The previous cycle The cycle to the first In the cycle, the ... Flow characteristic values ​​for each cycle; Indicates the first The previous cycle The cycle to the first In the cycle, the ... The flow characteristic value of each cycle.

[0022] Furthermore, the specific steps for determining the influence weight of network traffic data characteristics in the target period based on the correlation between the traffic characteristic values ​​of periods adjacent to the target period and the characteristics of network traffic data, and the time-series fluctuation coefficient of the target period determined by the periods preceding the target period, include:

[0023] The target period and the period immediately preceding the target period. The traffic characteristic values ​​of each period are arranged in chronological order to obtain the traffic characteristic value sequence of the target period; the average value of the characteristics of all network traffic data collected within the target period is recorded as the characteristic mean of the target period; the target period and adjacent periods before the target period are then compared. The feature mean values ​​of each period are arranged in chronological order to obtain the feature mean value sequence of the target period; the correlation between the features of each network traffic data and the traffic feature value is obtained.

[0024] Based on the correlation between the characteristics of all network traffic data and traffic characteristic values, and the previous period of the target period The time-series fluctuation coefficients determined for each period determine the influence weight of network traffic data characteristics in the target period.

[0025] Furthermore, the correlation between the characteristics of all network traffic data and traffic characteristic values, and the previous period of the target period... The specific steps involved in determining the time-series fluctuation coefficient for each period and the influence weight of network traffic data characteristics in the target period are as follows:

[0026] The sum of the correlations between all network traffic data features and traffic feature values ​​is recorded as the total correlation score. The ratio of the correlation score between network traffic data features and traffic feature values ​​to the total correlation score is used as the weight of the network traffic data feature. This weight is then calculated based on the previous period of the target timeframe. The product of the normalized value of the time-series fluctuation coefficient determined for each period and the weight of the characteristics of the network traffic data is denoted as the influence weight of the network traffic data characteristics in the target period.

[0027] Furthermore, the specific steps for dimensionality reduction of network traffic data for all periods based on the influence weights and network traffic data collected for all periods include:

[0028] A data matrix is ​​constructed based on the influence weights of all collected periods. The weight matrix is ​​a diagonal matrix, and the values ​​on the diagonal of the weight matrix are the influence weights of each feature of the network traffic data in the target period.

[0029] A data matrix is ​​constructed based on network traffic data. The data matrix and the weight matrix have the same number of columns. The corresponding columns of the data matrix and the weight matrix correspond to the characteristics of the same network traffic data. The network traffic data of each period are filled in the same row of the data matrix according to the type of network traffic data.

[0030] Principal component analysis is used to process the data matrix and the weight matrix for all periods to obtain network traffic data for all periods after dimensionality reduction.

[0031] Furthermore, the specific steps for reducing the dimensionality of network traffic data from abnormal network traffic monitoring and inputting it into the isolated forest model, and obtaining the results of abnormal network traffic monitoring based on the anomaly detection results, include:

[0032] Collect network traffic data for abnormal network traffic monitoring and reduce its dimensionality to obtain the dimensionality-reduced network traffic data to be monitored. Input the dimensionality-reduced network traffic data to be monitored into the isolated forest model to obtain the abnormality score of the dimensionality-reduced network traffic data to be monitored.

[0033] When the anomaly score is greater than the preset anomaly threshold, it is determined that there is abnormal network traffic in the network traffic data monitored by the network anomaly traffic monitoring system; when the anomaly score is less than or equal to the anomaly threshold, it is determined that there is no abnormal network traffic in the network traffic data monitored by the network anomaly traffic monitoring system.

[0034] Secondly, embodiments of this application also provide a network abnormal traffic monitoring device based on a large model. The displacement monitoring device includes a memory, a processor, and a computer program stored in the memory and running on the processor. When the processor executes the computer program, it implements the steps of any of the methods described above.

[0035] Thirdly, embodiments of this application provide a network abnormal traffic monitoring system based on a large model. The network abnormal traffic monitoring system includes: a network traffic data acquisition module, a network traffic data feature analysis module, a network traffic data dimensionality reduction module, and a network abnormal traffic monitoring module.

[0036] The network traffic data acquisition module is used to collect network traffic data for each period at the network interface, count the types of characteristics included in the network traffic data, and establish the network data set for the same period based on the network traffic data for the same period.

[0037] The network traffic data feature analysis module is used to determine the traffic feature value of the same period based on the differences in the base of all network data sets in the same period, the differences in the number of data packets of different target IPs, and the differences in the inbound and outbound volumes of different target ports. Based on the number of data packets collected in each period for the same network protocol, the module determines the periodic sequence of the same network protocol. Based on the periodic sequence of all types of network protocols in the period, the number of data packets, and the differences in the traffic feature values ​​of each period preceding the current period, the module calculates the time series fluctuation coefficient of each period determined by the periods preceding the current period.

[0038] The network traffic data dimensionality reduction module is used to designate any period as the target period, determine the influence weight of the network traffic data features in the target period based on the correlation between the traffic feature values ​​of the periods adjacent to the target period and the features of the network traffic data, as well as the time series fluctuation coefficient of the target period determined by the periods preceding the target period, and reduce the dimensionality of the network traffic data for all periods based on the influence weights of all collected periods and the network traffic data.

[0039] The network anomaly traffic monitoring module is used to construct an isolated forest model based on network traffic data from all periods of dimensionality reduction. It reduces the dimensionality of the network traffic data for anomaly traffic monitoring and inputs it into the isolated forest model, and obtains the results of network anomaly traffic monitoring based on the anomaly detection results.

[0040] As can be seen from the above embodiments, the network anomaly traffic monitoring method, device, and system based on a large model provided in this application have at least the following beneficial effects:

[0041] To improve the interpretability of principal component data obtained through dimensionality reduction and ensure that the reduced principal component data accurately reflects network traffic anomalies, this application evaluates network traffic anomalies within a period based on the interactivity of network access requests and the targeting of network attacks against a specific IP address. It obtains periodic traffic characteristic values ​​and analyzes the fluctuations of network traffic data for the same network protocol over time, considering the different network protocol requirements of various services in the network environment. This yields a time-series fluctuation coefficient determined for the preceding preset number of periods for each period. Furthermore, by analyzing the access characteristics of network traffic within a period and the correlation between network traffic in adjacent periods and network traffic anomalies within the period, the application determines the characteristics of the network traffic data. The system assigns influence weights to features in each period and, based on the influence weights and network traffic data collected for all periods, performs dimensionality reduction on the network traffic data for all periods. This improves the accuracy of network anomaly traffic monitoring and ensures that the features obtained after dimensionality reduction can reflect the anomalies in the network traffic data. Finally, an isolated forest model is constructed based on the network traffic data from all periods after dimensionality reduction. The network traffic data for anomaly traffic monitoring is then dimensionality reduced and input into the isolated forest model. The results of network anomaly traffic monitoring are obtained based on the anomaly detection results. This addresses the problems of poor data interpretability and loss of key information caused by network traffic dimensionality reduction, which cannot guarantee the accurate identification of network anomaly traffic. It ensures the interpretability of the data after network traffic dimensionality reduction and accurately identifies network anomaly traffic. Attached Figure Description

[0042] To more clearly illustrate the technical solutions and advantages in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0043] Figure 1 A flowchart illustrating the steps of a network anomaly traffic monitoring method based on a large model, as provided in one embodiment of this application;

[0044] Figure 2 This is a schematic diagram of the structure of a network anomaly traffic monitoring system based on a large model, provided in one embodiment of this application. Detailed Implementation

[0045] To further illustrate the technical means and effects adopted by this application to achieve the intended inventive purpose, the following, in conjunction with the accompanying drawings and preferred embodiments, details the specific implementation methods, structures, features, and effects of the network anomaly traffic monitoring method, device, and system based on a large model proposed in this application. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. Furthermore, specific features, structures, or characteristics in one or more embodiments can be combined in any suitable form.

[0046] The following description, in conjunction with the accompanying drawings, details the specific solutions for the network anomaly traffic monitoring method, equipment, and system based on a large model provided in this application.

[0047] Please see Figure 1 The diagram illustrates a flowchart of a network anomaly traffic monitoring method based on a large model according to an embodiment of this application. The method includes the following steps:

[0048] S001: Collect network traffic data for each period at the network interface, count the types of characteristics included in the network traffic data, and establish the network data set for the same period based on the network traffic data for the same period.

[0049] The Wireshark tool is used to monitor network interfaces and collect network traffic data. This data includes both numerical and non-numerical data. Sequential encoding is used to convert the non-numerical data into numerical data. Network traffic data collected within a given period is then merged into a network data set. Specifically, network traffic data within the same period that shares identical source IP address, destination IP address, source port, destination port, and transport protocol type is grouped into the same network data set, thus obtaining the corresponding network data set for that period.

[0050] Preferably, in one embodiment of this application, when collecting network traffic data, a 10-second period is used as one cycle of network traffic data, and network traffic data for one cycle is collected every 10 minutes, for a total of one week's worth of network traffic data as a sample. In practical applications, as other implementation methods, implementers can determine the length of one cycle of network traffic data and the time interval between collecting network traffic data for different cycles according to the actual situation; this application does not impose any special restrictions.

[0051] The characteristics of all collected network traffic data are categorized. In this embodiment, the network traffic data characteristics selected include source IP address, destination IP address, source port, destination port, and transport protocol type, totaling five categories. In practical applications, as other implementation methods, implementers can determine the categories and number of characteristics included in all network traffic data according to actual circumstances; this application does not impose any special restrictions.

[0052] At this point, the network traffic data and network data set for each period have been obtained.

[0053] S002: Based on the differences in the base values ​​of all network data sets in the same period, the differences in the number of data packets for different target IPs, and the differences in the inbound and outbound traffic for different target ports, determine the traffic characteristic value for the same period. Based on the number of data packets collected by the same network protocol in each period, determine the periodic sequence of the same network protocol. Based on the periodic sequences of all types of network protocols in the period, the number of data packets, and the differences in the traffic characteristic values ​​between the periods preceding the current period, calculate the time-series fluctuation coefficient of the period determined by the periods preceding the current period.

[0054] To improve the interpretability of principal component data obtained through dimensionality reduction and to ensure that the reduced principal component data accurately reflects network traffic anomalies, it is necessary to perform statistical analysis on network traffic data in conjunction with network traffic access characteristics.

[0055] First, in a normal network environment, network access requests are interactive, and network traffic is typically bidirectional, meaning outbound and inbound traffic are relatively equal. However, when network traffic becomes abnormal, attackers create this abnormal traffic to disrupt normal network use and render the network inaccessible. Therefore, attackers generate a large amount of abnormal one-way traffic to clog the network, causing a sharp increase in one-way traffic. Second, attackers control botnets to send a large number of data packets in a short period, resulting in a large number of data packets with the same source IP address, destination IP address, source port, destination port, and transport protocol type within a short timeframe. Finally, the network attack will target a single IP address, causing a large number of access requests to that target IP address within the abnormal traffic period.

[0056] Based on the differences in the cardinality of all network data sets in the same period, the differences in the number of data packets for different target IPs, and the differences in the inbound and outbound volumes for different target ports, the traffic characteristic values ​​for the same period are determined.

[0057] The cardinality of all network datasets in the same period is divided using the Otsu's maximum inter-class variance method to obtain the segmentation threshold.

[0058] Among them, the Otsu's method of obtaining the segmentation threshold is a well-known technique and will not be elaborated further; the cardinality of a set is a basic concept in set theory, which represents the number of elements in the set and will not be elaborated further.

[0059] The ratio of the number of data packets from the target IP with the highest access volume in the same period to the total number of data packets in the same period is denoted as the first ratio of the same period; the ratio of the total number of elements contained in all network data sets with a base greater than the segmentation threshold in the same period to the total number of data packets in the same period is denoted as the second ratio of the same period; the absolute value of the difference between the inbound and outbound traffic of the same target port in the same period is denoted as the inbound / outbound difference of the same target port in the same period; the sum of the inbound and outbound traffic of the same target port in the same period is denoted as the total inbound / outbound traffic of the same target port in the same period; the sum of the ratios of the total inbound / outbound traffic of all target ports in the same period to the inbound / outbound difference is denoted as the third ratio of the same period; the sum of the first, second, and third ratios of the same period is denoted as the traffic characteristic value of the same period.

[0060] Periodic traffic characteristic values ​​are used to evaluate network traffic anomalies within a period. These values ​​are derived from the instantaneous characteristics of network traffic data. When network traffic anomalies occur, the bidirectional nature of network traffic is disrupted to some extent, unidirectional traffic increases sharply, and there is a significant difference between outbound and inbound traffic to the target IP, resulting in a larger third ratio for the period. At the same time, a large number of data packets with the same source IP address, destination IP address, source port, destination port, and transport protocol type will appear, and a large number of access request data packets will appear for a single target IP, making the periodic traffic characteristic value larger.

[0061] Network traffic data changes continuously over time, exhibiting certain temporal characteristics and periodicity. By analyzing the periodic characteristics in the temporal sequence, the accuracy of evaluating abnormal changes in network traffic can be further improved.

[0062] Different services in a network environment have different needs, often requiring the use of different network protocols. For example, to ensure the reliability and security of file transfer, file transfer protocols such as FTP and SFTP are needed; to meet the low latency and real-time requirements of video access, streaming media protocols such as RTSP, HLS, and MPEG-DASH are needed; and web browsing commonly uses protocols such as HTTP and HTTPS. Different service needs have a certain periodicity, so the timing of the same request protocol also follows certain patterns. If a large number of illogical protocol requests occur at abnormal times, it indicates a greater likelihood of abnormal traffic. Secondly, under normal circumstances, the size of data packets corresponding to different protocols often follows certain patterns. For example, web browsing typically generates small HTTP requests and responses, while video streaming usually has a fixed bitrate. If significantly large or small data packets appear, the likelihood of abnormal network traffic is greater. Finally, when attackers control botnets to launch network attacks, they will launch continuous attacks over a certain period, resulting in a high degree of consistency in abnormal traffic data within several consecutive periods.

[0063] The periodic sequence of the same network protocol is determined by the number of data packets collected in each period.

[0064] The number of data packets collected in each period for the same network protocol is arranged in chronological order to obtain the data packet number sequence of the same network protocol. The data packet number sequence is processed using the STL time series decomposition algorithm to obtain the seasonal sequence corresponding to the data packet number sequence. The autocorrelation coefficient of the seasonal sequence is calculated and an ACF plot is drawn. The ACF plot is the autocorrelation plot. The confidence interval method is used to draw a 99% confidence interval based on the autocorrelation plot. The peak value of the autocorrelation coefficient that exceeds the confidence interval is recorded as the significant peak value. The lag values ​​corresponding to the significant peak values ​​are arranged in descending order of the significant peak values ​​to obtain the periodic sequence of the same network protocol.

[0065] Among them, the use of the STL time series decomposition algorithm to obtain the corresponding seasonal sequence, calculate the autocorrelation coefficient of the seasonal sequence, draw the ACF plot, and use the confidence interval method to determine the significant peak are all well-known techniques and will not be described in detail.

[0066] At this point, the periodic sequence of each network protocol can be obtained.

[0067] Based on the periodic sequence of all types of network protocols within the period, the number of data packets, and the differences in traffic characteristics between adjacent periods before the current period, the temporal fluctuation coefficient of the period determined by each adjacent period before the current period is calculated.

[0068] The variance of the first difference sequence of the periodic sequence of the network protocol is denoted as the first variance of the network protocol.

[0069] Furthermore, the time series fluctuation coefficients determined for each adjacent period before the calculation period are calculated using the following formula:

[0070]

[0071]

[0072] in, Indicates according to the first The previous cycle The time-series fluctuation coefficient determined by each period; Indicates the first The sum of the first variances of all types of network protocols within a given period; Indicates the first The number of network protocol types per cycle; Indicates the first The previous cycle The first cycle is used The number of data packets for each type of network protocol; Indicates the first The previous cycle Total number of data packets in each cycle; Represents the natural constant; Indicates according to the first The previous cycle Data volatility determined over a given period; Indicates the first The previous cycle The cycle to the first The flow characteristic values ​​for all cycles within a given period; Indicates the first The previous cycle The cycle to the first In the cycle, the ... Flow characteristic values ​​for each cycle; Indicates the first The previous cycle The cycle to the first In the cycle, the ... The flow characteristic value of each cycle.

[0073] The time-series fluctuation coefficient is used to measure the fluctuation of network traffic data over time. In the process of calculating the time-series fluctuation coefficient, the data volatility is placed at the exponential position of an exponential function with the natural constant as the base. Its purpose is to ensure that even when the network traffic fluctuation is not significant between adjacent weeks, it is still possible to measure network traffic anomalies through other characteristics, thereby improving the accuracy of network traffic anomaly measurement.

[0074] When abnormal network traffic occurs within a period, it can disrupt the periodicity of some network protocols, resulting in a larger sum of the first variances of all types of network protocols within the period. At the same time, the size of data packets making network access requests may become abnormal, resulting in lower data volatility. In this case, the determined time-series fluctuation coefficient is larger.

[0075] in, This embodiment represents the span parameter between adjacent periods. The value of is 10. In practical applications, as other implementation methods, the implementer can decide the value of the adjacent period span parameter according to the actual situation. This application does not impose any special restrictions. It should be noted that when the other adjacent periods before a certain period are insufficient... At this time, the cycle is not analyzed.

[0076] Thus, the time series fluctuation coefficients of the adjacent periods before any given period are obtained.

[0077] S003: Denote any period as the target period. Based on the correlation between the traffic characteristic values ​​of the periods adjacent to the target period and the characteristics of the network traffic data, as well as the time series fluctuation coefficient of the target period determined by the periods preceding the target period, determine the influence weight of the characteristics of the network traffic data in the target period. Based on the influence weights of all collected periods and the network traffic data, reduce the dimensionality of the network traffic data of all periods.

[0078] When using isolated forests for outlier detection, the network traffic data has too many features, necessitating dimensionality reduction. To ensure that the reduced features accurately extract the characteristics of abnormal network traffic, dimensionality reduction cannot be based solely on changes in data size. This application improves the accuracy of abnormal network traffic monitoring by combining access characteristics of network traffic within a period with temporal characteristics of network traffic in adjacent periods, ensuring that the reduced features reflect the anomalies in the network traffic data.

[0079] Periodic traffic characteristic values ​​are used to evaluate network traffic anomalies within a period. These evaluations are obtained based on the instantaneous characteristics of network traffic data. The temporal fluctuations of these characteristic values ​​across different periods can reflect the state of network traffic data to a certain extent. Therefore, when there is a high similarity between the temporal changes of a network traffic data characteristic and the temporal changes of the traffic characteristic value, it indicates that this characteristic is likely to have a relatively large impact on network traffic data fluctuations. Simultaneously, the temporal fluctuation coefficient measures the volatility of network traffic data over time. The greater the correlation between a network traffic data characteristic and the temporal changes of the traffic characteristic value, the more accurate the characteristic's assessment of network traffic conditions is, and thus, a greater weight should be given to this characteristic.

[0080] Any period is designated as the target period. Based on the correlation between the traffic characteristic values ​​of the periods adjacent to the target period and the characteristics of the network traffic data, as well as the time-series fluctuation coefficient of the target period determined by the periods preceding the target period, the influence weight of the network traffic data characteristics in the target period is determined.

[0081] The target period and the period immediately preceding the target period. The traffic characteristic values ​​of each period are arranged in chronological order to obtain the traffic characteristic value sequence of the target period; the average value of the characteristics of all network traffic data collected within the target period is recorded as the characteristic mean of the target period; the target period and adjacent periods before the target period are then compared. The characteristic mean values ​​of each period are arranged in chronological order to obtain the characteristic mean value sequence of the target period; the grey relational analysis method is used to obtain the correlation between the characteristics of each network traffic data and the traffic characteristic value.

[0082] The sum of the correlations between all network traffic data features and traffic feature values ​​is recorded as the total correlation score. The ratio of the correlation score between network traffic data features and traffic feature values ​​to the total correlation score is used as the weight of the network traffic data feature. This weight is then calculated based on the previous period of the target timeframe. The product of the normalized value of the time-series fluctuation coefficient determined for each period and the weight of the characteristics of the network traffic data is denoted as the influence weight of the network traffic data characteristics in the target period.

[0083] It should be noted that this embodiment uses the Z-Score standard normalization method to calculate the normalized value. In practical applications, implementers may use other methods of existing technology, such as the maximum-minimum normalization method or the sigmoid function, to calculate the normalized value, and no limitation is made here.

[0084] The greater the influence weight of network traffic data features in the target period, the more important these features are in the dimensionality reduction process of network traffic data, and the more accurately they can identify abnormal network traffic.

[0085] Based on the influence weights and network traffic data collected for all periods, a data matrix and a weight matrix for each period are constructed respectively.

[0086] Specifically, as an embodiment of this application, the weight matrix is ​​a diagonal matrix, and the values ​​on the diagonal of the weight matrix are the influence weights of each feature of the network traffic data in the target period; the data matrix and the weight matrix have the same number of columns, and the corresponding columns of the data matrix and the weight matrix correspond to the same feature of the network traffic data; the network traffic data of each period are filled in the same row of the data matrix according to the type of network traffic data.

[0087] Principal component analysis is used to process the data matrix and the weight matrix for all periods, thereby reducing the dimensionality of the network traffic data for all periods and obtaining the principal components for all periods, i.e., the network traffic data for all periods after dimensionality reduction.

[0088] Principal component analysis for dimensionality reduction is a well-known technique and will not be elaborated further.

[0089] At this point, network traffic data for all cycles after dimensionality reduction has been obtained.

[0090] S004: Construct an isolated forest model based on network traffic data from all periods of dimensionality reduction, reduce the dimensionality of network traffic data from abnormal network traffic monitoring and input it into the isolated forest model, and obtain the results of abnormal network traffic monitoring based on the anomaly detection results.

[0091] The Isolation Forest algorithm is used to process network traffic data for all periods after dimensionality reduction, extracting data each time. The number of samples and the number of isolated trees are set to [number]. Construct an isolated forest model.

[0092] in, This represents the first preset threshold. This represents the second preset threshold. In this embodiment, the first preset threshold is set to 50, and the second preset threshold is set to 100.

[0093] For network interfaces that require network anomaly traffic monitoring, collect network traffic data for each period. Using the same method as when constructing the isolated forest model, obtain the dimensionality-reduced network traffic data for all periods that require network anomaly traffic monitoring. Record this data as the dimensionality-reduced network traffic data to be monitored. Input the dimensionality-reduced network traffic data to be monitored into the isolated forest model and obtain the anomaly score corresponding to the dimensionality-reduced network traffic data to be monitored.

[0094] When the anomaly score is greater than the preset anomaly threshold, it is determined that the network traffic data requiring network anomaly traffic monitoring contains network anomaly traffic; when the anomaly score is less than or equal to the preset anomaly threshold, it is determined that the network traffic data requiring network anomaly traffic monitoring does not contain network anomaly traffic.

[0095] In this embodiment, the abnormal threshold is set to 0.5. The abnormal threshold should be greater than or equal to 0.3 and less than or equal to 0.7.

[0096] This completes the monitoring of abnormal network traffic.

[0097] Based on the same inventive concept as the above methods, this application also provides a network anomaly traffic monitoring device based on a large model, including a memory, a processor, and a computer program stored in the memory and running on the processor. When the processor executes the computer program, it implements the steps of any one of the above-described network anomaly traffic monitoring methods based on a large model.

[0098] Please see Figure 2 , Figure 2 This is a schematic diagram of the structure of a network anomaly traffic monitoring system based on a large model, provided in one embodiment of this application. In this embodiment, the network anomaly traffic monitoring system includes units that execute the steps in the corresponding embodiment of the network anomaly traffic monitoring method based on a large model. See also... Figure 2 The network abnormal traffic monitoring system includes: a network traffic data acquisition module, a network traffic data feature analysis module, a network traffic data dimensionality reduction module, and a network abnormal traffic monitoring module.

[0099] The network traffic data acquisition module is used to collect network traffic data for each period at the network interface, count the types of characteristics included in the network traffic data, and establish the network data set for the same period based on the network traffic data for the same period.

[0100] The network traffic data feature analysis module is used to determine the traffic feature value of the same period based on the differences in the base of all network data sets in the same period, the differences in the number of data packets of different target IPs, and the differences in the inbound and outbound volumes of different target ports. Based on the number of data packets collected in each period for the same network protocol, the module determines the periodic sequence of the same network protocol. Based on the periodic sequence of all types of network protocols in the period, the number of data packets, and the differences in the traffic feature values ​​of each period preceding the current period, the module calculates the time series fluctuation coefficient of each period determined by the periods preceding the current period.

[0101] The network traffic data dimensionality reduction module is used to designate any period as the target period, determine the influence weight of the network traffic data features in the target period based on the correlation between the traffic feature values ​​of the periods adjacent to the target period and the features of the network traffic data, as well as the time series fluctuation coefficient of the target period determined by the periods preceding the target period, and reduce the dimensionality of the network traffic data for all periods based on the influence weights of all collected periods and the network traffic data.

[0102] The network anomaly traffic monitoring module is used to construct an isolated forest model based on network traffic data from all periods of dimensionality reduction. It reduces the dimensionality of the network traffic data for anomaly traffic monitoring and inputs it into the isolated forest model, and obtains the results of network anomaly traffic monitoring based on the anomaly detection results.

[0103] The above description is merely an embodiment of this application and is not intended to limit the scope of this application. Any equivalent structural or procedural transformations made based on the description and drawings of this application, or direct or indirect applications in other related technical fields, are similarly included within the protection scope of this application.

Claims

1. A method for monitoring abnormal network traffic based on a large model, characterized in that, The method includes the following steps: Collect network traffic data for each period at the network interface, count the types of characteristics included in the network traffic data, and establish a network data set for the same period based on the network traffic data for the same period. Based on the differences in the base values ​​of all network data sets in the same period, the differences in the number of data packets for different target IPs, and the differences in the inbound and outbound traffic for different target ports, the traffic characteristic values ​​for the same period are determined. Based on the number of data packets collected by the same network protocol in each period, the periodic sequence of the same network protocol is determined. Based on the periodic sequences of all types of network protocols in the period, the number of data packets, and the differences in the traffic characteristic values ​​between the periods preceding the current period, the time series fluctuation coefficient of the period determined by the periods preceding the current period is calculated. Any period is designated as the target period. Based on the correlation between the traffic characteristic values ​​of the periods adjacent to the target period and the characteristics of the network traffic data, as well as the time-series fluctuation coefficient of the target period determined by the periods preceding the target period, the influence weight of the network traffic data characteristics in the target period is determined. Based on the influence weights of all collected periods and the network traffic data, the dimensionality of the network traffic data in all periods is reduced. An isolated forest model is constructed based on network traffic data from all periods of dimensionality reduction. The network traffic data from anomaly traffic monitoring is then dimensionality-reduced and input into the isolated forest model. The results of anomaly traffic monitoring are obtained based on the anomaly detection results.

2. The network anomaly traffic monitoring method based on a large model as described in claim 1, characterized in that, The process for determining the flow characteristic values ​​of the same period is as follows: The ratio of the number of data packets from the target IP with the highest access volume in the same period to the total number of data packets in the same period is denoted as the first ratio of the same period. The segmentation threshold is obtained based on the cardinality of all network data sets in the same period; the ratio of the total number of elements contained in all network data sets with a cardinality greater than the segmentation threshold in the same period to the total number of data packets in the same period is denoted as the second ratio of the same period. The absolute value of the difference between the inbound and outbound volumes of the same target port within the same period is denoted as the inbound / outbound difference of the same target port within the same period. The sum of the inbound and outbound volumes of the same target port within the same period is denoted as the total inbound / outbound volume of the same target port within the same period. The cumulative sum of the ratios of the total inbound / outbound volume of all target ports within the same period to the inbound / outbound difference is denoted as the third ratio of the same period. The sum of the first ratio, the second ratio, and the third ratio for the same period is recorded as the flow characteristic value for the same period.

3. The network anomaly traffic monitoring method based on a large model as described in claim 1, characterized in that, The process of constructing the periodic sequence of the same network protocol is as follows: The number of data packets collected by the same network protocol in each period is arranged in chronological order to obtain the data packet number sequence of the same network protocol. The data packet number sequence is decomposed into a time series to obtain a seasonal sequence. An autocorrelation plot is drawn based on the seasonal sequence, and significant peaks in the autocorrelation plot are determined. The lags corresponding to the significant peaks are arranged in descending order of the significant peaks to obtain the periodic sequence of the same network protocol.

4. The network anomaly traffic monitoring method based on a large model as described in claim 1, characterized in that, The specific process for obtaining the time-series fluctuation coefficient includes: The variance of the first-order difference sequence of the periodic sequence of the network protocol is denoted as the first variance of the network protocol. The time series fluctuation coefficient is calculated using the following formula: in, Indicates according to the first The previous cycle The time-series fluctuation coefficient determined by each period; This indicates the preset adjacent period span parameter; Indicates the first The sum of the first variances of all types of network protocols within a given period; Indicates the first The number of network protocol types per cycle; Indicates the first The previous cycle The first cycle is used The number of data packets for each type of network protocol; Indicates the first The previous cycle Total number of data packets in each cycle; Represents the natural constant; Indicates according to the first The previous cycle Data volatility determined over a given period; Indicates the first The previous cycle The cycle to the first The flow characteristic values ​​for all cycles within a given period; Indicates the first The previous cycle The cycle to the first In the cycle, the ... Flow characteristic values ​​for each cycle; Indicates the first The previous cycle The cycle to the first In the cycle, the ... The flow characteristic value of each cycle.

5. The network anomaly traffic monitoring method based on a large model as described in claim 4, characterized in that, The specific steps for determining the influence weight of network traffic data characteristics in the target period based on the correlation between the traffic characteristic values ​​of periods adjacent to the target period and the characteristics of network traffic data, and the time-series fluctuation coefficient of the target period determined by the periods preceding the target period, are as follows: The target period and the period immediately preceding the target period. The traffic characteristic values ​​of each period are arranged in chronological order to obtain the traffic characteristic value sequence of the target period; the average value of the characteristics of all network traffic data collected within the target period is recorded as the characteristic mean of the target period; the target period and adjacent periods before the target period are then compared. The feature mean values ​​of each period are arranged in chronological order to obtain the feature mean value sequence of the target period; the correlation between the features of each network traffic data and the traffic feature value is obtained. Based on the correlation between the characteristics of all network traffic data and traffic characteristic values, and the previous period of the target period The time-series fluctuation coefficients determined for each period determine the influence weight of network traffic data characteristics in the target period.

6. The network anomaly traffic monitoring method based on a large model as described in claim 5, characterized in that, The correlation between the characteristics of all network traffic data and traffic characteristic values, and the previous period of the target period. The specific steps involved in determining the time-series fluctuation coefficient for each period and the influence weight of network traffic data characteristics in the target period are as follows: The sum of the correlations between all network traffic data features and traffic feature values ​​is denoted as the total correlation score. The ratio of the correlation score between network traffic data features and traffic feature values ​​to the total correlation score is used as the weight of the network traffic data feature. This weight is then calculated based on the previous period of the target timeframe. The product of the normalized value of the time-series fluctuation coefficient determined for each period and the weight of the characteristics of the network traffic data is denoted as the influence weight of the network traffic data characteristics in the target period.

7. The network anomaly traffic monitoring method based on a large model as described in claim 1, characterized in that, The specific steps for dimensionality reduction of network traffic data for all periods based on the influence weights and network traffic data collected for all periods are as follows: A data matrix is ​​constructed based on the influence weights of all collected periods. The weight matrix is ​​a diagonal matrix, and the values ​​on the diagonal of the weight matrix are the influence weights of each feature of the network traffic data in the target period. A data matrix is ​​constructed based on network traffic data. The data matrix and the weight matrix have the same number of columns. The corresponding columns of the data matrix and the weight matrix correspond to the characteristics of the same network traffic data. The network traffic data of each period are filled in the same row of the data matrix according to the type of network traffic data. Principal component analysis is used to process the data matrix and the weight matrix for all periods to obtain network traffic data for all periods after dimensionality reduction.

8. The network anomaly traffic monitoring method based on a large model as described in claim 1, characterized in that, The specific steps involved in reducing the dimensionality of network traffic data from abnormal network traffic monitoring and inputting it into the isolated forest model, and obtaining the results of abnormal network traffic monitoring based on the anomaly detection results, are as follows: Collect network traffic data for abnormal network traffic monitoring and reduce its dimensionality to obtain the dimensionality-reduced network traffic data to be monitored. Input the dimensionality-reduced network traffic data to be monitored into the isolated forest model to obtain the abnormality score of the dimensionality-reduced network traffic data to be monitored. When the anomaly score is greater than the preset anomaly threshold, it is determined that there is abnormal network traffic in the network traffic data monitored by the network anomaly traffic monitoring system; when the anomaly score is less than or equal to the anomaly threshold, it is determined that there is no abnormal network traffic in the network traffic data monitored by the network anomaly traffic monitoring system.

9. A network anomaly traffic monitoring device based on a large model, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method as described in any one of claims 1-8.

10. A network anomaly traffic monitoring system based on a large model, implementing the method as described in claim 1, characterized in that, The network abnormal traffic monitoring system includes: The network traffic data acquisition module is used to collect network traffic data for each period at the network interface, count the types of characteristics included in the network traffic data, and establish the network data set for the same period based on the network traffic data for the same period. The network traffic data feature analysis module is used to determine the traffic feature value of the same period based on the differences in the base of all network data sets in the same period, the differences in the number of data packets of different target IPs, and the differences in the inbound and outbound volumes of different target ports. Based on the number of data packets collected in each period for the same network protocol, the module determines the periodic sequence of the same network protocol. Based on the periodic sequence of all types of network protocols in the period, the number of data packets, and the differences in the traffic feature values ​​of each period preceding the current period, the module calculates the time series fluctuation coefficient of each period determined by the periods preceding the current period. The network traffic data dimensionality reduction module is used to designate any period as the target period, determine the influence weight of the network traffic data features in the target period based on the correlation between the traffic feature values ​​of the periods adjacent to the target period and the features of the network traffic data, as well as the time series fluctuation coefficient of the target period determined by the periods preceding the target period, and reduce the dimensionality of the network traffic data for all periods based on the influence weights of all collected periods and the network traffic data. The network anomaly traffic monitoring module is used to construct an isolated forest model based on network traffic data from all periods of dimensionality reduction. It reduces the dimensionality of the network traffic data for anomaly traffic monitoring and inputs it into the isolated forest model, and obtains the results of network anomaly traffic monitoring based on the anomaly detection results.

Images