A network anomaly detection method and device based on data analysis

Abstract

The application discloses a network anomaly detection method and device based on data analysis, and belongs to the technical field of network security. The method comprises the following steps: obtaining historical data of core network parameters, and dividing the historical data into continuous multiple periods according to the chronological order; the core network parameters are parameters that can reflect the network operation state; for each period, the following steps are performed: using the historical data of the core network parameters in the period to generate autocorrelation features and cross-correlation features in the period; and fusing the autocorrelation features and the cross-correlation features to obtain fusion features in the period; and using the fusion features of the continuous multiple periods to construct a benchmark, so as to detect whether the network is abnormal by using the deviation between the fusion features of the current period and the benchmark. The application can capture the time anomaly of a single parameter and the linkage anomaly between parameters, thereby improving the detection accuracy for the implicit attack and the complex anomaly detection.

Description

Technical Field

[0001] This invention relates to the field of network security technology, and in particular to a network anomaly detection method and apparatus based on data analysis. Background Technology

[0002] As networks expand and services become more complex, network attacks are becoming increasingly covert and diversified. Traditional detection methods based on single thresholds or rules are finding it increasingly difficult to cope. In related technologies, network anomaly analysis often relies on changes in the values ​​of raw network parameters; however, single numerical anomalies are easily affected by service fluctuations, resulting in a high false alarm rate.

[0003] Therefore, there is an urgent need to provide a method that can accurately detect hidden attacks and complex anomalies. Summary of the Invention

[0004] This invention provides a network anomaly detection method and apparatus based on data analysis. The technical solution is as follows:

[0005] On the one hand, a network anomaly method based on data analysis is provided, the method comprising:

[0006] Historical data of core network parameters are obtained, and the historical data is divided into multiple consecutive periods in chronological order; the core network parameters are parameters that can reflect the network operating status.

[0007] For each period, the following steps are performed: using historical data of the core network parameters within that period, generate the autocorrelation and cross-correlation features for that period; and fuse the autocorrelation and cross-correlation features to obtain the fused features for that period.

[0008] A benchmark is constructed using the fusion features of multiple consecutive periods, and the deviation between the fusion features of the current period and the benchmark is used to detect whether the network is abnormal.

[0009] On the other hand, a network anomaly detection device based on data analysis is provided, the device comprising:

[0010] The acquisition unit is used to acquire historical data of core network parameters and divide the historical data into multiple consecutive periods in chronological order; the core network parameters are parameters that can reflect the network operating status.

[0011] The fusion unit is used to perform the following for each cycle: using historical data of the core network parameters in that cycle, generating autocorrelation features and cross-correlation features for that cycle; and fusing the autocorrelation features and cross-correlation features to obtain the fused features for that cycle.

[0012] The detection unit is used to construct a benchmark using fusion features from multiple consecutive periods, and to detect whether the network is abnormal by using the deviation between the fusion features of the current period and the benchmark.

[0013] On the other hand, a computer device is provided, the computer device including a memory and a processor, the memory for storing computer programs, and the processor for executing the computer programs stored in the memory to implement the steps of the network anomaly detection method based on data analysis described above.

[0014] On the other hand, a computer-readable storage medium is provided, wherein a computer program is stored therein, and when the computer program is executed by a processor, it implements the steps of the network anomaly detection method based on data analysis described above.

[0015] On the other hand, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps of the network anomaly detection method based on data analysis described above.

[0016] The technical solution provided by this invention can bring at least the following beneficial effects:

[0017] In this embodiment of the invention, historical data is divided into multiple consecutive periods. For each period, autocorrelation and cross-correlation features are generated. These features are then fused, and a benchmark is constructed using the fused features from multiple consecutive normal periods. Finally, the deviation between the fused features of the current period and the benchmark is used to detect network anomalies. It is evident that the fused features used in this scheme as a network anomaly detection indicator include both autocorrelation and cross-correlation features. When using these fused features for network anomaly detection, it can capture both temporal anomalies of individual parameters and anomalies arising from the interaction between parameters, thereby improving detection accuracy for covert attacks and complex anomalies. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a flowchart of a network anomaly detection method based on data analysis provided in an embodiment of the present invention;

[0020] Figure 2 This is a structural diagram of a network anomaly detection device based on data analysis provided in an embodiment of the present invention;

[0021] Figure 3 This is a hardware architecture diagram of a computer device provided in an embodiment of the present invention. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are some embodiments of the present invention, but not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0023] Please refer to Figure 1 This invention provides a network anomaly detection method based on data analysis, the method comprising:

[0024] Step 100: Obtain historical data of core network parameters and divide the historical data into multiple consecutive periods in chronological order; the core network parameters are parameters that can reflect the network operating status.

[0025] Step 102: For each period, perform the following: Use historical data of the core network parameters within that period to generate autocorrelation features and cross-correlation features for that period; and fuse the autocorrelation features and cross-correlation features to obtain the fused features for that period.

[0026] Step 104: Construct a benchmark using the fusion features of multiple consecutive cycles, and use the deviation between the fusion features of the current cycle and the benchmark to detect whether the network is abnormal.

[0027] In this embodiment of the invention, historical data is divided into multiple consecutive periods. For each period, autocorrelation and cross-correlation features are generated. These features are then fused, and a benchmark is constructed using the fused features from multiple consecutive normal periods. Finally, the deviation between the fused features of the current period and the benchmark is used to detect network anomalies. As can be seen, this scheme incorporates both autocorrelation and cross-correlation features as network anomaly detection indicators. When using these fused features for network anomaly detection, it can capture both temporal anomalies of individual parameters and anomalies arising from interaction between parameters, thereby improving detection accuracy for covert attacks and complex anomalies.

[0028] The following description Figure 1 The execution method of each step is shown.

[0029] First, for step 100, obtain historical data of the core network parameters and divide the historical data into multiple consecutive periods in chronological order.

[0030] In this embodiment of the invention, the object of network anomaly detection is the network system as a whole, which may include the network system itself, the terminals and servers carried by the network, or the services and user behaviors supported by the network.

[0031] For hidden attacks or complex anomalies, the manifestation of their abnormal states is not obvious. In order to detect these hidden attacks with inconspicuous state manifestations, embodiments of the present invention can use periodic statistical features to identify abnormalities in the overall periodic pattern. Specifically, a normal baseline is constructed using the fusion features of historical normal cycles, and anomalies are detected by the deviation between the fusion features of the current period to be detected and the baseline.

[0032] The core network parameters are parameters that reflect the network's operating status. These core network parameters can include at least the following categories:

[0033] Traffic-related metrics: inbound / outbound bandwidth utilization, packets per second (PPS), and average packet size;

[0034] Connection-related metrics: concurrent TCP connections to IP, connection establishment success rate, and number of UDP sessions;

[0035] Performance metrics: average link latency, packet loss rate, DNS query response time.

[0036] After dividing the historical data into multiple consecutive periods in chronological order, with each period T containing m time points, then when the core network has n parameters, the parameter set within each period is p1, p2, ..., p... n The time series data for each core network parameter is p. i1 p i2 , ..., p im ; where p ik Let be the state value of the i-th core network parameter at time k.

[0037] Then, for step 102, for each period, the following steps are performed: using historical data of the core network parameters within that period, generate autocorrelation features and cross-correlation features for that period; and fuse the autocorrelation features and cross-correlation features to obtain the fused features for that period.

[0038] Autocorrelation features cover the time periodicity of a single parameter, such as the daily bandwidth peak pattern. It can identify anomalies such as the break in the parameter's own pattern. For example, if the number of core server connections suddenly loses its daily periodicity, it may indicate a service failure.

[0039] Cross-correlation features cover the linkage and correlation of multiple parameters, such as the strong positive correlation between bandwidth and PPS. It can identify anomalies such as failure of linkage between parameters, such as a sudden increase in bandwidth but PPS not increasing synchronously, which may be due to abnormally large file transfers.

[0040] The generation methods of autocorrelation features and cross-correlation features are explained below.

[0041] First, autocorrelation characteristics

[0042] In this embodiment of the invention, the autocorrelation feature is generated as follows:

[0043] A1: Calculate the autocorrelation characteristics of each core network parameter within this period;

[0044] In this embodiment of the invention, each core network parameter p can be targeted. i Three types of features can be generated by following the steps A11-A14:

[0045] A11: Based on a number of preset lag durations, calculate the autocorrelation coefficient of the core network parameter at each lag duration;

[0046] Considering that the state value at a single point in time cannot reflect the nature of the anomaly, several lag durations τ can be set in advance based on the period T, such as τ=T / 10, τ=T / 5, etc. For each lag duration, the autocorrelation coefficient of the core network parameter can be calculated, thereby uncovering the state change pattern within different lag durations.

[0047] The autocorrelation coefficient can be calculated using the following formula:

[0048]

[0049] in, Let p be the autocorrelation coefficient of the i-th core network parameter under lag time τ. ik Let i be the state value of the i-th core network parameter at time k. Let be the mean value of the i-th core network parameter during this period.

[0050] A12: Determine the maximum autocorrelation lag time based on the autocorrelation coefficient of the core network parameter for each lag time;

[0051] In this embodiment of the invention, the maximum autocorrelation lag is the lag with the most significant time correlation of the parameter within the period. For example, if the period T is one day, m=24, the autocorrelation coefficient for τ=1 hour is 0.7, the autocorrelation coefficient for τ=2 hours is 0.6, and the autocorrelation coefficient for τ=4 hours is 0.8, then the maximum autocorrelation lag duration is 4 hours. In other words, the maximum autocorrelation lag duration is the lag duration corresponding to the maximum autocorrelation coefficient.

[0052] A13: Calculate the standard deviation of the autocorrelation coefficient for different lag durations;

[0053] The standard deviation of the autocorrelation coefficient reflects the volatility of autocorrelation. The smaller the standard deviation of the autocorrelation coefficient, the more stable the parameter is in terms of time regularity.

[0054] A14: The autocorrelation coefficient, the maximum autocorrelation lag time, and the standard deviation at different lag times are determined as the autocorrelation characteristics of the core network parameter.

[0055] A2: Perform feature aggregation on the autocorrelation features of each core network parameter within the period to obtain the autocorrelation features within the period.

[0056] In this embodiment of the invention, the feature aggregation may specifically include:

[0057] A21: Based on the autocorrelation characteristics of each core network parameter, calculate the average maximum autocorrelation coefficient, the proportion of strongly autocorrelation parameters, and the mean autocorrelation fluctuation.

[0058] A22: The autocorrelation features of each core network parameter, the average maximum autocorrelation coefficient, the proportion of strong autocorrelation parameters, and the mean autocorrelation fluctuation are aggregated as features within the period.

[0059] In this embodiment of the invention, the average maximum autocorrelation coefficient, the proportion of strong autocorrelation parameters, and the mean autocorrelation fluctuation are global statistical characteristics.

[0060] Among them, the average maximum autocorrelation coefficient It can be calculated using the following formula:

[0061]

[0062] The calculation method for the proportion of strongly autocorrelation parameters is as follows: determine the first number of parameters whose autocorrelation coefficient is greater than the first threshold, and determine the proportion of strongly autocorrelation parameters by the ratio of the first number to the total number of core network parameters n.

[0063] Autocorrelation mean fluctuation The calculation method is as follows:

[0064]

[0065] in, Let be the mean of the standard deviations of the i-th core network parameter under different lag durations.

[0066] The autocorrelation features obtained after feature aggregation within this period have a total of 3n+3 dimensions, of which the features of a single core network parameter are 3n-dimensional and the global statistical features are 3-dimensional.

[0067] Second, cross-correlation features

[0068] In the embodiments of the present invention, the generation method of the cross-correlation features is as follows:

[0069] B1: Calculate the cross-correlation coefficient of each pair of core network parameters within a period, and use the cross-correlation threshold to screen out the strongly correlated parameter pairs with the absolute value of the cross-correlation coefficient greater than the cross-correlation threshold from multiple pairs of core network parameters;

[0070] B2: Generate the cross-correlation features of each strongly correlated parameter pair; the cross-correlation features include the cross-correlation coefficient of each strongly correlated parameter pair and the deviation of the cross-correlation coefficient from the mean value of historical cross-correlation coefficients;

[0071] B3: Generate cross-correlation global features according to the cross-correlation features of each strongly correlated parameter pair; the cross-correlation global features include: the average cross-correlation coefficient of strongly correlated parameter pairs, the proportion of deviations exceeding the set value, and the cross-correlation entropy value;

[0072] B4: Determine the cross-correlation features of each strongly correlated parameter pair and the cross-correlation global features as the cross-correlation features within a period.

[0073] In step B1, for all parameter pairs (p i , p j ), i < j, the cross-correlation coefficient r within a period can be calculated by the following formula ij , which is used to reflect the linkage strength:

[0074]

[0075] where , are the mean values of the states of the i-th and j-th core network parameters respectively;

[0076] r ij ∈[-1, 1], and the larger |r ij | is, the stronger the linkage between parameter i and parameter j.

[0077] In the embodiments of the present invention, only the features of strongly correlated parameter pairs can be concerned to reduce the complexity of data analysis and improve the detection efficiency. For example, if the cross-correlation threshold is 0.6, then the core network parameter pairs with |r ij | > 0.6 are retained as strongly correlated parameter pairs. Suppose the obtained strongly correlated parameter pairs after screening are: M = {(i, j)1, (i, j)2,..., (i, j) h}, and h is an integer not less than 1.

[0078] After selecting strongly correlated parameter pairs, the cross-correlation features of each strongly correlated parameter pair can be generated, resulting in cross-correlation features with a 2h dimension.

[0079] The deviation between the cross-correlation count and the historical mean cross-correlation count is:

[0080]

[0081] It is the average of the cross-correlation coefficients of the historical normal cycle.

[0082] The global cross-correlation feature has three dimensions: the average cross-correlation number of strongly correlated parameter pairs, the proportion of deviations exceeding a set deviation value, and the cross-correlation entropy value.

[0083] Mean cross-correlation coefficient of strongly correlated parameter pairs for:

[0084]

[0085] The percentage of deviations exceeding the set deviation value for:

[0086]

[0087] in, μ To set the deviation value.

[0088] Cross-correlation entropy H M Calculated using the following formula:

[0089]

[0090] in, f r for r ij The frequency within a certain range is used to reflect the diversity of linkages.

[0091] Therefore, it can be seen that the cross-correlation features within the period have a total of 2h+3 dimensions.

[0092] In this embodiment of the invention, after obtaining the autocorrelation and cross-correlation features within the period, it is necessary to fuse the autocorrelation and cross-correlation features. In one implementation, a weighted concatenation fusion method can be used to perform the fusion of the autocorrelation and cross-correlation features.

[0093] In this way, for multiple consecutive cycles, the fusion characteristics within the corresponding cycle can be obtained.

[0094] Furthermore, current fusion features can capture not only the temporal patterns of individual parameters based on autocorrelation features, but also the linkage patterns between parameters based on cross-correlation features. However, considering that cross-correlation features are for the linkage between two parameters and do not consider the linkage between multiple parameters, one embodiment of the present invention may further include: generating multi-parameter linkage features.

[0095] The multi-parameter linkage feature is generated as follows:

[0096] The target parameter set is determined based on multiple strongly correlated parameter pairs; the target parameter set is the union of the core network parameters in the multiple strongly correlated parameter pairs.

[0097] For each target parameter in the strongly correlated parameter set, the following steps are performed: In this aggregation, determine the state characteristics of the current target parameter in this aggregation, as well as the state characteristics of other target parameters strongly correlated with the current target parameter in this aggregation; aggregate the state characteristics of other target parameters in this aggregation into the state characteristics of the current target parameter in this aggregation to obtain the aggregation characteristics; use the aggregation characteristics of this aggregation as the state characteristics of the current target parameter in the next aggregation; when the number of aggregations reaches a set number, obtain the aggregation characteristics of the current target parameter.

[0098] The aggregated features of each target parameter in the strongly correlated parameter set are used as multi-parameter linkage features.

[0099] For example, the set of strongly correlated parameters includes target parameter 1, target parameter 2, target parameter 3, and target parameter 4. Among them, target parameter 1 and target parameter 2 are strongly correlated parameter pairs, target parameter 1 and target parameter 3 are strongly correlated parameter pairs, and target parameter 3 and target parameter 4 are strongly correlated parameter pairs.

[0100] For target parameter 1, during the first aggregation, the state features of target parameters 2 and 3 are aggregated into the state features of target parameter 1. During the second aggregation, since the state features of target parameter 3 in the second aggregation include the state features of target parameter 4, the state features of target parameter 1 also include the state features of target parameter 4 after the second aggregation. Therefore, the number of aggregations is determined based on the number of interval parameters between two non-strongly correlated parameters in the strongly correlated parameter set. In other words, it involves identifying non-strongly correlated parameter pairs in the strongly correlated parameter set, determining the number of parameter intervals in each non-correlated parameter pair, determining the maximum number of parameter intervals among multiple non-correlated parameter pairs, and using the sum of the maximum number of parameter intervals and 1 as the number of aggregations.

[0101] It should be noted that the state characteristics of the target parameter can be determined based on the state sequence of the target parameter within the period.

[0102] The autocorrelation feature, cross-correlation feature, and multi-parameter linkage feature are fused together to obtain the fused feature.

[0103] Finally, for step 104, a benchmark is constructed using the fusion features of multiple consecutive cycles, and the deviation between the fusion features of the current cycle and the benchmark is used to detect whether the network is abnormal.

[0104] In this embodiment of the invention, the historical period is a normal period, and the fusion features of multiple historical normal periods can be used to construct a benchmark.

[0105] In one implementation, the benchmark may include the mean vector and covariance matrix of the fused features. Assume the fused features for N historical normal cycles are F1, F2, ..., F... N ,So:

[0106] Mean vector for:

[0107]

[0108] Among them, F t Let t be the fusion feature.

[0109] When performing network anomaly detection for the current period, the fusion features for the current period can be generated using the method in step 102, and then the fusion features for the current period and the benchmark can be used to determine whether the network is abnormal.

[0110] Specifically, the Mahalanobis distance between the fusion features of the current period and the benchmark can be calculated based on the mean vector and covariance matrix; it can be determined whether the Mahalanobis distance is greater than the distance threshold. If it is, it is determined that there is a network anomaly in the current period; otherwise, it is determined that there is no network anomaly in the current period.

[0111] Mahalanobis distance is calculated using the following formula:

[0112]

[0113] Where d is the Mahalanobis distance between the fusion feature of the current period and the benchmark; F current The fusion characteristics for the current cycle; The mean vector in the baseline; is the covariance matrix in the baseline.

[0114] In another implementation, the baselines for different periods are different. The baseline for each period is determined by the variation pattern formed by the fusion features of multiple consecutive periods preceding that period. When performing network anomaly detection for the current period, the baseline of the previous period can be updated using the fusion features of the current period to obtain the baseline for the current period. The fusion features of the current period can then be used for detection based on the baseline of the current period.

[0115] This invention integrates two types of patterns: single-parameter temporal anomalies in autocorrelation features and specific parameter linkage anomalies in cross-correlation features. Furthermore, based on continuous periodic baseline dynamic updates, it can adapt to changes in the network environment. This invention enables accurate detection and localization of network anomalies, and is particularly suitable for detecting stealth attacks and complex anomalies.

[0116] Please refer to Figure 2 This invention provides a network anomaly detection device based on data analysis, the device comprising:

[0117] The acquisition unit 200 is used to acquire historical data of core network parameters and divide the historical data into multiple consecutive periods in chronological order; the core network parameters are parameters that can reflect the network operating status.

[0118] The fusion unit 202 is used to perform the following for each period: using historical data of the core network parameters in that period, generating autocorrelation features and cross-correlation features for that period; and fusing the autocorrelation features and cross-correlation features to obtain the fused features for that period.

[0119] The detection unit 204 is used to construct a benchmark using the fusion features of multiple consecutive periods, and to detect whether the network is abnormal by using the deviation between the fusion features of the current period and the benchmark.

[0120] In one embodiment of the present invention, the autocorrelation feature is generated as follows:

[0121] Calculate the autocorrelation characteristics of each core network parameter within this period;

[0122] The autocorrelation features of each core network parameter within the period are aggregated to obtain the autocorrelation features within that period.

[0123] In one embodiment of the present invention, calculating the autocorrelation characteristics of each core network parameter within the period includes:

[0124] For each core network parameter, the following steps are performed: based on a number of preset lag times, calculate the autocorrelation coefficient of the core network parameter at each lag time; determine the maximum autocorrelation lag time based on the autocorrelation coefficient of the core network parameter at each lag time; calculate the standard deviation of the autocorrelation coefficient at different lag times; and determine the autocorrelation characteristics of the core network parameter as the autocorrelation coefficient at each lag time, the maximum autocorrelation lag time, and the standard deviation at different lag times.

[0125] In one embodiment of the present invention, the step of feature aggregation of the autocorrelation features of each core network parameter within the period includes: calculating the average maximum autocorrelation coefficient, the proportion of strongly autocorrelation parameters, and the mean autocorrelation fluctuation based on the autocorrelation features of each core network parameter; and using the autocorrelation features of each core network parameter, the average maximum autocorrelation coefficient, the proportion of strongly autocorrelation parameters, and the mean autocorrelation fluctuation as the autocorrelation features within the period after feature aggregation.

[0126] In one embodiment of the present invention, the cross-correlation feature is generated as follows:

[0127] Calculate the cross-correlation coefficient of each core network parameter pair within a period, and use a cross-correlation threshold to filter out strongly correlated parameter pairs from multiple core network parameter pairs whose absolute value of the cross-correlation coefficient is greater than the cross-correlation threshold;

[0128] Generate the cross-correlation feature for each strongly correlated parameter pair; the cross-correlation feature includes the cross-correlation number of each strongly correlated parameter pair and the deviation of the cross-correlation number from the historical mean cross-correlation number;

[0129] Based on the cross-correlation characteristics of each strongly correlated parameter pair, a global cross-correlation feature is generated; the global cross-correlation feature includes: the average cross-correlation coefficient of the strongly correlated parameter pair, the proportion of deviations exceeding a set deviation value, and the cross-correlation entropy value;

[0130] The cross-correlation features and global cross-correlation features of each strongly correlated parameter pair are determined as the cross-correlation features within the period.

[0131] In one embodiment of the present invention, the benchmark includes the mean vector and covariance matrix of fused features over multiple periods;

[0132] The method of detecting network anomalies by using the deviation between the fusion features of the current period and the benchmark includes: calculating the Mahalanobis distance between the fusion features of the current period and the benchmark based on the mean vector and covariance matrix; determining whether the Mahalanobis distance is greater than a distance threshold; if so, determining that there is a network anomaly in the current period; otherwise, determining that there is no network anomaly in the current period.

[0133] In one embodiment of the present invention, the Mahalanobis distance is calculated using the following formula:

[0134]

[0135] Where d is the Mahalanobis distance between the fusion feature of the current period and the benchmark; F current The fusion characteristics for the current cycle; The mean vector in the baseline; is the covariance matrix in the baseline.

[0136] It should be noted that the network anomaly detection device based on data analysis provided in the above embodiments is only an example of the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the network anomaly detection device based on data analysis provided in the above embodiments and the network anomaly detection method embodiments based on data analysis belong to the same concept, and the specific implementation process can be found in the method embodiments, which will not be repeated here.

[0137] Embodiments of this application also provide a computer device, please refer to... Figure 3 The computer device includes a processor and a memory, the memory storing at least one instruction, at least one program, code set, or instruction set, the at least one instruction, at least one program, code set, or instruction set being loaded and executed by the processor to implement the network anomaly detection method based on data analysis provided in the above-described method embodiments.

[0138] Embodiments of this application also provide a computer-readable storage medium storing at least one instruction, at least one program, code set, or instruction set, wherein the at least one instruction, at least one program, code set, or instruction set is loaded and executed by a processor to implement the network anomaly detection method based on data analysis provided in the above-described method embodiments.

[0139] Embodiments of this application also provide a computer program product, which includes a computer program. A processor of a computer device reads the computer program from a computer-readable storage medium and executes the computer program, causing the computer device to perform any of the data analysis-based network anomaly detection methods described in the above embodiments.

[0140] For ease of description, the above systems or devices are described separately as various modules or units based on their functions. Of course, in implementing this application, the functions of each unit can be implemented in one or more software and / or hardware components.

[0141] As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in various embodiments or some parts of the embodiments of this application.

[0142] Finally, it should be noted that in this document, relational terms such as first, second, third, and fourth are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0143] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A network anomaly detection method based on data analysis, characterized in that, The method includes: Historical data of core network parameters are obtained, and the historical data is divided into multiple consecutive periods in chronological order; the core network parameters are parameters that can reflect the network operating status. For each period, the following steps are performed: using historical data of the core network parameters within that period, generate autocorrelation and cross-correlation features for that period; and then use a weighted concatenation fusion method to fuse the autocorrelation and cross-correlation features to obtain the fused features for that period. A benchmark is constructed using the fusion features of multiple consecutive cycles, and the deviation between the fusion features of the current cycle and the benchmark is used to detect whether the network is abnormal. The method for generating the autocorrelation features is as follows: calculate the autocorrelation features of each core network parameter within the period; aggregate the autocorrelation features of each core network parameter within the period to obtain the autocorrelation features within the period. The calculation of the autocorrelation characteristics of each core network parameter within the period includes: for each core network parameter, performing the following steps: calculating the autocorrelation coefficient of the core network parameter at each preset lag duration; determining the maximum autocorrelation lag duration based on the autocorrelation coefficient of the core network parameter at each lag duration; calculating the standard deviation of the autocorrelation coefficient at different lag durations; and determining the autocorrelation characteristics of the core network parameter as the autocorrelation coefficient at each lag duration, the maximum autocorrelation lag duration, and the standard deviation at different lag durations. The step of feature aggregation of the autocorrelation features of each core network parameter within the period includes: calculating the average maximum autocorrelation coefficient, the proportion of strongly autocorrelation parameters, and the mean autocorrelation fluctuation based on the autocorrelation features of each core network parameter; and using the autocorrelation features of each core network parameter, the average maximum autocorrelation coefficient, the proportion of strongly autocorrelation parameters, and the mean autocorrelation fluctuation as the autocorrelation features within the period after feature aggregation.

2. The method according to claim 1, characterized in that, The cross-correlation feature is generated as follows: Calculate the cross-correlation coefficient of each core network parameter pair within a period, and use a cross-correlation threshold to filter out strongly correlated parameter pairs from multiple core network parameter pairs whose absolute value of the cross-correlation coefficient is greater than the cross-correlation threshold; Generate the cross-correlation feature for each strongly correlated parameter pair; the cross-correlation feature includes the cross-correlation number of each strongly correlated parameter pair and the deviation of the cross-correlation number from the historical mean cross-correlation number; Generate global cross-correlation features based on the cross-correlation characteristics of each strongly correlated parameter pair; The global cross-correlation features include: the average cross-correlation number of strongly correlated parameter pairs, the proportion of deviations exceeding a set deviation value, and the cross-correlation entropy value; The cross-correlation features and global cross-correlation features of each strongly correlated parameter pair are determined as the cross-correlation features within the period.

3. The method according to any one of claims 1-2, characterized in that, The benchmark includes the mean vector and covariance matrix of fused features from multiple periods; The method of detecting network anomalies by using the deviation between the fusion features of the current period and the benchmark includes: calculating the Mahalanobis distance between the fusion features of the current period and the benchmark based on the mean vector and covariance matrix; determining whether the Mahalanobis distance is greater than a distance threshold; if so, determining that there is a network anomaly in the current period; otherwise, determining that there is no network anomaly in the current period.

4. The method according to claim 3, characterized in that, Mahalanobis distance is calculated using the following formula: Where d is the Mahalanobis distance between the fusion feature of the current period and the benchmark; F current The fusion characteristics for the current cycle; The mean vector in the baseline; is the covariance matrix in the baseline.

5. A network anomaly detection device based on data analysis, characterized in that, The apparatus for performing the network anomaly detection method based on data analysis according to any one of claims 1-4, the apparatus comprising: The acquisition unit is used to acquire historical data of core network parameters and divide the historical data into multiple consecutive periods in chronological order; the core network parameters are parameters that can reflect the network operating status. The fusion unit is used to perform the following for each cycle: using historical data of the core network parameters in that cycle, generating autocorrelation features and cross-correlation features for that cycle; and fusing the autocorrelation features and cross-correlation features to obtain the fused features for that cycle. The detection unit is used to construct a benchmark using fusion features from multiple consecutive periods, and to detect whether the network is abnormal by using the deviation between the fusion features of the current period and the benchmark.

6. A computer device, characterized in that, The computer device includes a memory and a processor. The memory is used to store computer programs, and the processor is used to execute the computer programs stored in the memory to implement the steps of the method according to any one of claims 1-4.

7. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, which, when executed by a processor, implements the steps of the method described in any one of claims 1-4.

8. A computer program product, characterized in that, Includes a computer program, which, when executed by a processor, implements the steps of the method according to any one of claims 1-4.

Images