A method and system for intelligently identifying and analyzing abnormalities in network communication data

By performing feature extraction and timing clustering on historical network communication data, combined with discrete wavelet transformation technology, the abnormal characteristics in communication data are identified, and the problem of the impact of noise and error data is solved, efficient abnormal identification and accuracy are achieved, and the Marking costs.

CN118410446BActive Publication Date: 2024-11-15ZHIFANGYUAN TECHNOLOGY (JIANGSU) CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410863847.6
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-06-29
Publication Date
2024-11-15
Estimated Expiration
2044-06-29

AI Technical Summary

Technical Problem

The existing intelligent abnormality identification analysis methods for communication data are affected in the performance and accuracy of processing noise, missing values ​​or wrong data, and the acquisition of marked exception samples requires complex and expensive.

Method used

By collecting historical network communication data, performing feature extraction and classification, using time-series clustering and feature fusion technology, combining discrete wavelet transformation and inverse discrete transformation, abnormal features in network communication data are identified, and data labeling costs are reduced through unsupervised learning.

Benefits of technology

Accurate abnormal identification of communication data is realized, the accuracy and robustness of identification is improved, the cost of data labeling is reduced, and the identification process is simplified.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118410446B_ABST
    Figure CN118410446B_ABST
Patent Text Reader

Abstract

The present invention discloses a method and system for intelligent anomaly identification and analysis of network communication data, which relates to the field of information analysis technology, including: collecting historical network communication data, extracting features from the historical network communication data, obtaining a network communication anomaly data set, using a classifier to perform feature classification on the network communication anomaly data set, obtaining a number of subsequences of different scales, combining them into time series clustering data, and performing feature fusion on the subsequences in the time series clustering data, determining the abnormal features of the network communication data, and judging whether the network communication integrated abnormal features are in an abnormal range based on the network communication integrated abnormal features and the historical network communication data, if not, no output is made, and if so, the output is abnormal data. The advantages of the present invention are: improving the accuracy and robustness of abnormal identification, and at the same time, the identification process is more economical and efficient.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of information analysis, and in particular to a method and system for intelligently identifying and analyzing abnormalities in network communication data. Background Art

[0002] Communication data refers to various data generated during the communication process, including but not limited to phone call records, text message content, network data transmission records, sensor data, communication equipment status information, etc. These data can be used to analyze communication behavior, network traffic, user habits, etc., which is helpful for optimizing communication networks, improving communication experience, and enhancing network security.

[0003] The existing intelligent anomaly recognition and analysis of communication data is to achieve the purpose of anomaly recognition by establishing a recognition model and repeatedly recognizing and training historical anomaly data. Although this method can identify anomalies in communication data, the communication data itself may contain noise, missing values ​​or erroneous data, which may affect the performance and accuracy of the anomaly recognition model. Secondly, recognition usually requires labeled anomaly samples for supervised learning, but obtaining labeled data is relatively complex and expensive. Summary of the invention

[0004] In order to solve the above technical problems, a method and system for intelligent anomaly identification and analysis of network communication data are provided. The technical solution solves the problem that the above-mentioned existing methods can identify anomalies in communication data, but the communication data itself may contain noise, missing values ​​or erroneous data, which may affect the performance and accuracy of the anomaly identification model. Secondly, identification usually requires labeled abnormal samples for supervised learning, but obtaining labeled data is complex and expensive.

[0005] In order to achieve the above purpose, the technical solution adopted by the present invention is:

[0006] A method for intelligently identifying and analyzing abnormalities in network communication data, comprising:

[0007] Collect historical network communication data;

[0008] Based on historical network communication data, feature extraction is performed on the historical network communication data to obtain a network communication anomaly data set;

[0009] Based on the network communication anomaly data set, the classifier is used to classify the features of the network communication anomaly data set, and several subsequences of different scales are obtained and combined into time series clustering data;

[0010] Based on time series clustering data, feature fusion is performed on subsequences in the time series clustering data to determine the abnormal features of network communication data;

[0011] Based on the network communication integration abnormal characteristics and historical network communication data, determine whether the network communication integration abnormal characteristics are within the abnormal range. If not, no output is made. If so, the output is abnormal data.

[0012] Preferably, based on the historical network communication data, feature extraction is performed on the historical network communication data to obtain a network communication anomaly data set, specifically including:

[0013] Based on historical network communication data, each communication data is parsed to extract timestamps, data packet flow size, number of data packets and transmission protocols, and the time characteristics, flow characteristics and transmission protocol characteristics of each communication data are determined;

[0014] Normalize each communication data according to its time characteristics, traffic characteristics and transmission protocol characteristics;

[0015] Based on the time characteristics, traffic characteristics and transmission protocol characteristics of the normalized communication data, abnormal features are annotated and formed into a network communication anomaly dataset.

[0016] Preferably, based on the network communication anomaly data set, a classifier is used to perform feature classification on the network communication anomaly data set to obtain a number of subsequences of different scales, which are combined into time series clustering data, specifically including:

[0017] Based on the network communication anomaly data set, the discrete wavelet transform function is used to select the decomposition layer number for the network communication anomaly data set, and multi-level decomposition is performed to obtain detail coefficients and approximate coefficients at different scales;

[0018] The detail coefficients and approximation coefficients at different scales are constructed as network communication data feature vectors;

[0019] Use a classifier to classify the network communication data feature vector to obtain normal categories and abnormal categories;

[0020] Based on the normal category and the abnormal category, the detail coefficients and approximate coefficients of different scales in the normal category and the abnormal category are restored to the original signal through the inverse discrete transformation function;

[0021] Based on the original signal, a number of subsequences are extracted through a sliding window, and the subsequences are clustered according to time to obtain time series clustering data;

[0022] Among them, the discrete wavelet transform function is:

[0023] ;

[0024] In the formula, is the approximate coefficient, is the low frequency component signal, is the detail coefficient, is the high frequency component signal, is the input signal, h is the low-pass filter, is a high-pass filter, The low-pass filter h is used to sum the abnormal features k in the network communication abnormal data set. For high pass filter Sum the abnormal features k in the network communication abnormal data set, 2n is downsampling, k is the abnormal feature point, and n is the network communication abnormal data set;

[0025] Among them, the inverse discrete transformation function is:

[0026] ;

[0027] In the formula, is the original signal, For the The approximation coefficient of the layer, is the low frequency component signal, For the The detail factor of the layer, It is the high frequency component signal.

[0028] Preferably, based on the time series clustering data, the subsequences in the time series clustering data are subjected to feature fusion, and the abnormal features of the network communication data are determined to specifically include:

[0029] Based on the time series clustering data, the median of all subsequences in the cluster at the same time point of the time series clustering data is selected as the representative subsequence;

[0030] Each representative subsequence is standardized to obtain a number of standardized subsequences;

[0031] Based on several standardized subsequences, feature splicing is performed on each standardized subsequence to obtain abnormal features of network communication data.

[0032] Preferably, based on the network communication integration abnormality feature and the historical network communication data, it is determined whether the network communication integration abnormality feature is within the abnormal range. If not, no output is made. If yes, the output bit abnormal data specifically includes:

[0033] Determine the confidence level of the network communication integration anomaly characteristics;

[0034] Based on the network communication integration abnormal characteristics and historical network communication data, the maximum and minimum values ​​of each network communication integration abnormal characteristic are screened out, and the peak value of each abnormal characteristic is determined through the peak formula;

[0035] Based on the peak value of each abnormal feature and the total number of abnormal features integrated by network communication, the average value of the abnormal features is calculated by the average value formula;

[0036] Based on the peak value of each abnormal feature and the average value of the abnormal feature, the standard deviation of each abnormal feature is calculated by the difference formula;

[0037] Based on the average value of the abnormal features, the standard deviation of each abnormal feature and the confidence of the abnormal features integrated by network communication, the upper limit and the lower limit of the abnormal range are determined by the abnormal range formula;

[0038] According to the upper limit and the lower limit of the abnormal range, the historical network communication data is judged whether it is within the abnormal range. If not, no output is made. If so, the output is abnormal data.

[0039] The peak value formula is:

[0040] ;

[0041] Where G is the peak value of each abnormal feature, The maximum value of the abnormal characteristics integrated for network communication, Minimum value for integrating abnormal features for network communication;

[0042] The average value formula is:

[0043] ;

[0044] In the formula, is the average value of the abnormal features, is the peak value of the ith abnormal feature, and m is the total number of abnormal features;

[0045] The difference formula is:

[0046] ;

[0047] Where C is the standard deviation of each abnormal feature;

[0048] The abnormal range formula is:

[0049] ;

[0050] In the formula, is the upper limit of the abnormal range, is the lower limit of the abnormal range, z is the confidence level, and the confidence level is 95%.

[0051] Furthermore, a network communication data intelligent anomaly identification and analysis system is proposed, which is used to implement the above-mentioned network communication data intelligent anomaly identification and analysis method, including:

[0052] Data collection module, the data collection module is used to collect historical network communication data;

[0053] An abnormality marking module, the abnormality marking module is electrically connected to the data acquisition module, and the abnormality marking module is used to extract features of the historical network communication data based on the historical network communication data to obtain a network communication abnormality data set;

[0054] A time series clustering module, which is electrically connected to the anomaly annotation module, is used to perform feature classification on the network communication anomaly data set using a classifier based on the network communication anomaly data set, obtain a number of subsequences of different scales, and combine them into time series clustering data;

[0055] A feature fusion module, the feature fusion module is electrically connected to the time series clustering module, and the feature fusion module is used to perform feature fusion on subsequences in the time series clustering data based on the time series clustering data to determine abnormal features of the network communication data;

[0056] The anomaly analysis module is electrically connected to the feature fusion module. The anomaly analysis module is used to determine whether the network communication integrated anomaly features are within the abnormal range based on the network communication integrated anomaly features and historical network communication data. If not, no output is made. If so, the output is abnormal data.

[0057] Optionally, the anomaly annotation module includes:

[0058] The parsing unit parses each communication data based on the historical network communication data, extracts the timestamp, data packet flow size, data packet quantity and transmission protocol, and determines the time characteristics, flow characteristics and transmission protocol characteristics of each communication data;

[0059] A normalization unit performs normalization processing according to the time characteristics, flow characteristics and transmission protocol characteristics of each communication data;

[0060] The abnormal data set unit annotates abnormal features based on the time features, flow features and transmission protocol features of the normalized communication data and forms a network communication abnormal data set.

[0061] Optionally, the time series clustering module includes:

[0062] The decomposition unit uses a discrete wavelet transform function based on the network communication anomaly data set to select the number of decomposition layers for the network communication anomaly data set, perform multi-level decomposition, and obtain detail coefficients and approximate coefficients at different scales;

[0063] The feature vector unit constructs the detail coefficients and approximation coefficients at different scales into the network communication data feature vector;

[0064] The classification unit uses a classifier to classify the network communication data feature vector to obtain normal categories and abnormal categories;

[0065] The restoration unit restores the detail coefficients and approximate coefficients of different scales in the normal category and the abnormal category to the original signal through the inverse discrete transformation function based on the normal category and the abnormal category;

[0066] The clustering unit extracts several subsequences based on the original signal through a sliding window, and clusters the subsequences according to time to obtain time series clustering data.

[0067] Optionally, the feature fusion module includes:

[0068] A screening unit, based on the time series clustering data, screens out the medians of all subsequences in the cluster of the time series clustering data at the same time point as representative subsequences;

[0069] A standardization unit, which performs standardization processing on each representative subsequence to obtain a number of standardized subsequences;

[0070] The splicing unit, based on several standardized subsequences, performs feature splicing on each standardized subsequence to obtain abnormal features of network communication data.

[0071] Optionally, the exception analysis module includes:

[0072] A confidence unit, which determines the confidence of the abnormal characteristics of the network communication integration;

[0073] The peak value calculation unit, based on the network communication integration abnormality characteristics and historical network communication data, screens out the maximum and minimum values ​​of each network communication integration abnormality characteristic, and determines the peak value of each abnormality characteristic through the peak value formula;

[0074] The average value calculation unit calculates the average value of the abnormal features through an average value formula based on the peak value of each abnormal feature and the total number of abnormal features integrated by network communication;

[0075] A standard deviation calculation unit calculates the standard deviation of each abnormal feature through a difference formula based on the peak value of each abnormal feature and the average value of the abnormal feature;

[0076] The abnormal range confirmation unit determines the upper limit and the lower limit of the abnormal range through the abnormal range formula based on the average value of the abnormal features, the standard deviation of each abnormal feature and the confidence of the abnormal features integrated by network communication;

[0077] The judgment unit judges whether the historical network communication data is within the abnormal range according to the upper limit and the lower limit of the abnormal range. If not, no output is made. If yes, the abnormal data is output.

[0078] Compared with the prior art, the present invention has the following beneficial effects:

[0079] The present invention proposes an intelligent anomaly recognition and analysis scheme for network communication data. Feature information at different scales can be captured through the feature extraction process, and the classifier can accurately distinguish abnormal data from normal data, thereby achieving accurate anomaly recognition; at the same time, compared with the method that requires a large amount of labeled data for supervised learning, this method based on unsupervised learning can reduce the cost of data labeling, making the recognition process more economical and efficient. Secondly, through time series clustering processing, the network communication data is converted into time series clustering data, and the subsequences are feature fused. In this way, the time series relationship of the data can be comprehensively considered, the time series patterns and trends in the data can be discovered, and the accuracy and robustness of anomaly recognition can be further improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0080] Figure 1 It is a flow chart of a method for intelligent anomaly identification and analysis of network communication data;

[0081] Figure 2 A flow chart of a method for extracting features from historical network communication data;

[0082] Figure 3 A flow chart of a method for feature classification of a network communication anomaly data set;

[0083] Figure 4 A flow chart of a method for fusing features of subsequences in time series clustering data;

[0084] Figure 5 A flow chart of a method for determining whether the abnormal characteristics of network communication integration are within an abnormal range;

[0085] Figure 6 This is a framework diagram of an intelligent anomaly recognition and analysis system for network communication data. DETAILED DESCRIPTION

[0086] The following description is used to disclose the present invention so that those skilled in the art can implement the present invention. The preferred embodiments described below are only examples, and those skilled in the art may think of other obvious variations.

[0087] Reference Figure 1 As shown, a method for intelligently identifying and analyzing abnormalities in network communication data includes:

[0088] Collect historical network communication data;

[0089] Based on historical network communication data, feature extraction is performed on the historical network communication data to obtain a network communication anomaly data set;

[0090] Based on the network communication anomaly data set, the classifier is used to classify the features of the network communication anomaly data set, and several subsequences of different scales are obtained and combined into time series clustering data;

[0091] Based on time series clustering data, feature fusion is performed on subsequences in the time series clustering data to determine the abnormal features of network communication data;

[0092] Based on the network communication integration abnormal characteristics and historical network communication data, determine whether the network communication integration abnormal characteristics are within the abnormal range. If not, no output is made. If so, the output is abnormal data.

[0093] This solution compares the integrated abnormal features of network communication with historical network communication data to determine whether the integrated abnormal features are within the abnormal range. If not, the system will not make any output; otherwise, it will be marked as abnormal data. The technical means used include: through the combination of machine learning classifiers, feature fusion technology and abnormal judgment algorithms, the system can efficiently and accurately identify abnormal behaviors in network communications.

[0094] Reference Figure 2 As shown, based on the historical network communication data, feature extraction is performed on the historical network communication data to obtain the network communication anomaly data set, which specifically includes:

[0095] Based on historical network communication data, each communication data is parsed to extract timestamps, data packet flow size, number of data packets and transmission protocols, and the time characteristics, flow characteristics and transmission protocol characteristics of each communication data are determined;

[0096] Normalize each communication data according to its time characteristics, traffic characteristics and transmission protocol characteristics;

[0097] Based on the time characteristics, traffic characteristics and transmission protocol characteristics of the normalized communication data, abnormal features are annotated and formed into a network communication anomaly dataset.

[0098] This solution involves steps such as data preprocessing, feature extraction, normalization, and anomaly annotation. By extracting features such as time, traffic, and transmission protocol of communication data and normalizing them, the characteristics of communication data can be better characterized, and a reliable data foundation can be provided for subsequent anomaly detection and analysis.

[0099] Reference Figure 3As shown in the figure, based on the network communication anomaly dataset, a classifier is used to perform feature classification on the network communication anomaly dataset to obtain several subsequences of different scales, which are combined into time series clustering data, including:

[0100] Based on the network communication anomaly data set, the discrete wavelet transform function is used to select the decomposition layer number for the network communication anomaly data set, and multi-level decomposition is performed to obtain detail coefficients and approximate coefficients at different scales;

[0101] The detail coefficients and approximation coefficients at different scales are constructed as network communication data feature vectors;

[0102] Use a classifier to classify the network communication data feature vector to obtain normal categories and abnormal categories;

[0103] Based on the normal category and the abnormal category, the detail coefficients and approximate coefficients of different scales in the normal category and the abnormal category are restored to the original signal through the inverse discrete transformation function;

[0104] Based on the original signal, a number of subsequences are extracted through a sliding window, and the subsequences are clustered according to time to obtain time series clustering data;

[0105] Among them, the discrete wavelet transform function is:

[0106] ;

[0107] In the formula, is the approximate coefficient, is the low frequency component signal, is the detail coefficient, is the high frequency component signal, is the input signal, h is the low-pass filter, is a high-pass filter, The low-pass filter h is used to sum the abnormal features k in the network communication abnormal data set. For high pass filter Sum the abnormal features k in the network communication abnormal data set, 2n is downsampling, k is the abnormal feature point, and n is the network communication abnormal data set;

[0108] Among them, the inverse discrete transformation function is:

[0109] ;

[0110] In the formula, is the original signal, For the The approximation coefficient of the layer, is the low frequency component signal, For the The detail factor of the layer, It is the high frequency component signal.

[0111] In this scheme, the detailed coefficients and approximate coefficients at different scales are obtained through multi-level decomposition of discrete wavelet transform and constructed as feature vectors. These feature vectors represent the characteristic information of the data at different scales. Then, the feature vectors are classified using a classifier to distinguish between normal and abnormal categories. Next, the coefficients of different scales are restored to the original signal through the inverse discrete transformation function for subsequent analysis. Finally, based on the original signal, subsequences are extracted through a sliding window and time series clustering is performed to obtain time series clustering data, which helps to discover time series patterns and trends in the data.

[0112] It can be understood that the operating principle of sliding window extraction of subsequences and time clustering processing is to intercept continuous subsequences through a moving window, and then group them according to their temporal similarities, and finally obtain time series clustering data, thereby revealing the local features and time correlations in the original signal, and providing a deeper analysis perspective and information for network communication anomaly detection.

[0113] Reference Figure 4 As shown, based on the time series clustering data, the subsequences in the time series clustering data are fused to determine the abnormal features of the network communication data, including:

[0114] Based on the time series clustering data, the median of all subsequences in the cluster at the same time point of the time series clustering data is selected as the representative subsequence;

[0115] Each representative subsequence is standardized to obtain a number of standardized subsequences;

[0116] Based on several standardized subsequences, feature splicing is performed on each standardized subsequence to obtain abnormal features of network communication data.

[0117] This scheme fuses the features of the representative subsequences of each cluster. First, the fused features are determined by feature selection, considering the problem requirements, feature importance and relevance. Subsequently, the selected features are standardized or normalized to ensure that they have the same scale. Then, the selected features are integrated into a representative feature vector by selecting a fusion method, including but not limited to: simple average, weighted average, feature concatenation or feature combination. Finally, the obtained feature vector represents the overall characteristics of the cluster, providing strong support for subsequent data analysis and application.

[0118] Reference Figure 5As shown, based on the network communication integration abnormal characteristics and historical network communication data, it is determined whether the network communication integration abnormal characteristics are within the abnormal range. If not, no output is made. If yes, the output abnormal data specifically includes:

[0119] Determine the confidence level of the network communication integration anomaly characteristics;

[0120] Based on the network communication integration abnormal characteristics and historical network communication data, the maximum and minimum values ​​of each network communication integration abnormal characteristic are screened out, and the peak value of each abnormal characteristic is determined through the peak formula;

[0121] Based on the peak value of each abnormal feature and the total number of abnormal features integrated by network communication, the average value of the abnormal features is calculated by the average value formula;

[0122] Based on the peak value of each abnormal feature and the average value of the abnormal feature, the standard deviation of each abnormal feature is calculated by the difference formula;

[0123] Based on the average value of the abnormal features, the standard deviation of each abnormal feature and the confidence of the abnormal features integrated by network communication, the upper limit and the lower limit of the abnormal range are determined by the abnormal range formula;

[0124] According to the upper limit and the lower limit of the abnormal range, the historical network communication data is judged whether it is within the abnormal range. If not, no output is made. If so, the output is abnormal data.

[0125] The peak value formula is:

[0126] ;

[0127] Where G is the peak value of each abnormal feature, The maximum value of the abnormal characteristics integrated for network communication, Minimum value for integrating abnormal features for network communication;

[0128] The average value formula is:

[0129] ;

[0130] In the formula, is the average value of the abnormal features, is the peak value of the ith abnormal feature, and m is the total number of abnormal features;

[0131] The difference formula is:

[0132] ;

[0133] Where C is the standard deviation of each abnormal feature;

[0134] The abnormal range formula is:

[0135] ;

[0136] In the formula, is the upper limit of the abnormal range, is the lower limit of the abnormal range, z is the confidence level, and the confidence level is 95%.

[0137] This solution uses historical network communication data to calculate the peak, average and standard deviation of abnormal features and combines them with the confidence level to establish an anomaly detection framework that can effectively identify and output abnormal data in network communications, helping to discover potential problems in advance.

[0138] It is understandable that the confidence level refers to the reliability of the estimate of the population parameter (such as mean, proportion, etc.). A 95% confidence level is a common choice, which means that in the case of repeated sampling, 95% of the sampling results will contain the true population parameter. Choosing a 95% confidence level is a common practice to balance reliability and precision. A higher confidence level can provide greater certainty, but will result in a wider confidence interval and reduce the precision of the estimate. Conversely, a lower confidence level can result in a narrower confidence interval, which improves the precision of the estimate, but has lower certainty about the result. Therefore, a 95% confidence level is generally considered to be a reasonable choice that takes into account the reliability of the results while maintaining the precision of the estimate to a certain extent.

[0139] Further, see Figure 6 As shown, based on the same inventive concept of a method for intelligently identifying and analyzing anomalies of network communication data, a system for intelligently identifying and analyzing anomalies of network communication data is proposed, comprising:

[0140] Data collection module, the data collection module is used to collect historical network communication data;

[0141] An abnormality marking module, the abnormality marking module is electrically connected to the data acquisition module, and the abnormality marking module is used to extract features of the historical network communication data based on the historical network communication data to obtain a network communication abnormality data set;

[0142] A time series clustering module, which is electrically connected to the anomaly annotation module, is used to perform feature classification on the network communication anomaly data set using a classifier based on the network communication anomaly data set, obtain a number of subsequences of different scales, and combine them into time series clustering data;

[0143] A feature fusion module, the feature fusion module is electrically connected to the time series clustering module, and the feature fusion module is used to perform feature fusion on subsequences in the time series clustering data based on the time series clustering data to determine abnormal features of the network communication data;

[0144] The anomaly analysis module is electrically connected to the feature fusion module. The anomaly analysis module is used to determine whether the network communication integrated anomaly features are within the abnormal range based on the network communication integrated anomaly features and historical network communication data. If not, no output is made. If so, the output is abnormal data.

[0145] The exception annotation module includes:

[0146] The parsing unit parses each communication data based on the historical network communication data, extracts the timestamp, data packet flow size, data packet quantity and transmission protocol, and determines the time characteristics, flow characteristics and transmission protocol characteristics of each communication data;

[0147] A normalization unit performs normalization processing according to the time characteristics, flow characteristics and transmission protocol characteristics of each communication data;

[0148] The abnormal data set unit annotates abnormal features based on the time features, flow features and transmission protocol features of the normalized communication data and forms a network communication abnormal data set.

[0149] The time series clustering module includes:

[0150] The decomposition unit uses a discrete wavelet transform function based on the network communication anomaly data set to select the number of decomposition layers for the network communication anomaly data set, perform multi-level decomposition, and obtain detail coefficients and approximate coefficients at different scales;

[0151] The feature vector unit constructs the detail coefficients and approximation coefficients at different scales into the network communication data feature vector;

[0152] The classification unit uses a classifier to classify the network communication data feature vector to obtain normal categories and abnormal categories;

[0153] The restoration unit restores the detail coefficients and approximate coefficients of different scales in the normal category and the abnormal category to the original signal through the inverse discrete transformation function based on the normal category and the abnormal category;

[0154] The clustering unit extracts several subsequences based on the original signal through a sliding window, and clusters the subsequences according to time to obtain time series clustering data.

[0155] The feature fusion module includes:

[0156] A screening unit, based on the time series clustering data, screens out the medians of all subsequences in the cluster of the time series clustering data at the same time point as representative subsequences;

[0157] A standardization unit, which performs standardization processing on each representative subsequence to obtain a number of standardized subsequences;

[0158] The splicing unit, based on several standardized subsequences, performs feature splicing on each standardized subsequence to obtain abnormal features of network communication data.

[0159] The exception analysis module includes:

[0160] A confidence unit, which determines the confidence of the abnormal characteristics of the network communication integration;

[0161] The peak value calculation unit, based on the network communication integration abnormality characteristics and historical network communication data, screens out the maximum and minimum values ​​of each network communication integration abnormality characteristic, and determines the peak value of each abnormality characteristic through the peak value formula;

[0162] The average value calculation unit calculates the average value of the abnormal features through an average value formula based on the peak value of each abnormal feature and the total number of abnormal features integrated by network communication;

[0163] A standard deviation calculation unit calculates the standard deviation of each abnormal feature through a difference formula based on the peak value of each abnormal feature and the average value of the abnormal feature;

[0164] The abnormal range confirmation unit determines the upper limit and the lower limit of the abnormal range through the abnormal range formula based on the average value of the abnormal features, the standard deviation of each abnormal feature and the confidence of the abnormal features integrated by network communication;

[0165] The judgment unit judges whether the historical network communication data is within the abnormal range according to the upper limit and the lower limit of the abnormal range. If not, no output is made. If yes, the abnormal data is output.

[0166] The use process of a network communication data intelligent anomaly identification and analysis system is as follows:

[0167] Step 1: Collect historical network communication data;

[0168] Step 2: Based on the historical network communication data, parse each communication data, extract the timestamp, data packet flow size, data packet quantity and transmission protocol, and determine the time characteristics, flow characteristics and transmission protocol characteristics of each communication data;

[0169] Step 3: Perform normalization processing based on the time characteristics, traffic characteristics, and transmission protocol characteristics of each communication data;

[0170] Step 4: Based on the time characteristics, traffic characteristics, and transmission protocol characteristics of the normalized communication data, anomaly features are annotated and formed into a network communication anomaly dataset;

[0171] Step 5: Based on the network communication anomaly data set, use the discrete wavelet transform function to select the number of decomposition layers for the network communication anomaly data set, perform multi-level decomposition, and obtain detail coefficients and approximate coefficients at different scales;

[0172] Step 6: Construct the detail coefficients and approximation coefficients at different scales into network communication data feature vectors;

[0173] Step 7: Use the classifier to classify the network communication data feature vector to obtain normal categories and abnormal categories;

[0174] Step 8: Based on the normal category and the abnormal category, the detail coefficients and approximate coefficients of different scales in the normal category and the abnormal category are restored to the original signal through the inverse discrete transformation function;

[0175] Step 9: Based on the original signal, extract several subsequences through a sliding window, and cluster the subsequences according to time to obtain time series clustering data;

[0176] Step 10: Based on the time series clustering data, the median of all subsequences in the cluster at the same time point of the time series clustering data is selected as the representative subsequence;

[0177] Step 11: Standardize each representative subsequence to obtain several standardized subsequences;

[0178] Step 12: Based on several standardized subsequences, feature concatenation is performed on each standardized subsequence to obtain abnormal features of network communication data;

[0179] Step 13: Determine the confidence level of the network communication integration anomaly features;

[0180] Step 14: Based on the network communication integration abnormality characteristics and historical network communication data, the maximum value and the minimum value of each network communication integration abnormality characteristic are screened out, and the peak value of each abnormality characteristic is determined by the peak value formula;

[0181] Step 15: Based on the peak value of each abnormal feature and the total number of abnormal features integrated by network communication, the average value of the abnormal features is calculated by using the average value formula;

[0182] Step 16: Based on the peak value of each abnormal feature and the average value of the abnormal feature, the standard deviation of each abnormal feature is calculated by using a difference formula;

[0183] Step 17: Based on the average value of the abnormal features, the standard deviation of each abnormal feature and the confidence of the network communication integrated abnormal features, the upper limit and the lower limit of the abnormal range are determined by the abnormal range formula;

[0184] Step 18: According to the upper limit of the abnormal range and the lower limit of the abnormal range, determine whether the historical network communication data is within the abnormal range. If not, do not do any output. If so, output abnormal data.

[0185] In summary, the advantages of the present invention are: the feature information at different scales can be captured through the feature extraction process, and the classifier can accurately distinguish abnormal data from normal data, thereby achieving accurate abnormality recognition; at the same time, compared with the method that requires a large amount of labeled data for supervised learning, this method based on unsupervised learning can reduce the cost of data labeling, making the recognition process more economical and efficient, and secondly, through time series clustering processing, network communication data is converted into time series clustering data, and subsequences are feature fused. In this way, the time series relationship of the data can be comprehensively considered, the time series patterns and trends in the data can be discovered, and the accuracy and robustness of abnormality recognition can be further improved.

[0186] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The above embodiments and descriptions only describe the principles of the present invention. The present invention may be subject to various changes and improvements without departing from the spirit and scope of the present invention. These changes and improvements fall within the scope of the present invention. The scope of protection claimed by the present invention is defined by the attached claims and their equivalents.

Claims

1. A method for intelligently identifying and analyzing abnormalities in network communication data, characterized in that: include: Collect historical network communication data; Based on historical network communication data, feature extraction is performed on the historical network communication data to obtain a network communication anomaly data set; Based on the network communication anomaly data set, the classifier is used to classify the features of the network communication anomaly data set, and several subsequences of different scales are obtained and combined into time series clustering data; Based on time series clustering data, feature fusion is performed on subsequences in the time series clustering data to determine the abnormal features of network communication data; Based on the network communication integration abnormal characteristics and historical network communication data, determine whether the network communication integration abnormal characteristics are within the abnormal range. If not, do not make any output. If so, output abnormal data. Based on the network communication anomaly data set, the classifier is used to classify the network communication anomaly data set to obtain several subsequences of different scales, which are combined into time series clustering data, including: Based on the network communication anomaly data set, the discrete wavelet transform function is used to select the decomposition layer number for the network communication anomaly data set, and multi-level decomposition is performed to obtain detail coefficients and approximate coefficients at different scales; The detail coefficients and approximation coefficients at different scales are constructed as network communication data feature vectors; Use a classifier to classify the network communication data feature vector to obtain normal categories and abnormal categories; Based on the normal category and the abnormal category, the detail coefficients and approximate coefficients of different scales in the normal category and the abnormal category are restored to the original signal through the inverse discrete transformation function; Based on the original signal, a number of subsequences are extracted through a sliding window, and the subsequences are clustered according to time to obtain time series clustering data; Among them, the discrete wavelet transform function is: ; In the formula, is the approximate coefficient, is the low frequency component signal, is the detail coefficient, is the high frequency component signal, is the input signal, h is the low-pass filter, is a high-pass filter, The low-pass filter h is used to sum the abnormal features k in the network communication abnormal data set. For high pass filter Sum the abnormal features k in the network communication abnormal data set, 2n is downsampling, k is the abnormal feature point, and n is the network communication abnormal data set; Among them, the inverse discrete transformation function is: ; In the formula, is the original signal, For the The approximation coefficient of the layer, is the low frequency component signal, For the The detail factor of the layer, is the high frequency component signal; Based on the network communication integration abnormal characteristics and historical network communication data, it is determined whether the network communication integration abnormal characteristics are within the abnormal range. If not, no output is made. If yes, the output abnormal data specifically includes: Determine the confidence level of the network communication integration anomaly characteristics; Based on the network communication integration abnormal characteristics and historical network communication data, the maximum and minimum values ​​of each network communication integration abnormal characteristic are screened out, and the peak value of each abnormal characteristic is determined through the peak formula; Based on the peak value of each abnormal feature and the total number of abnormal features integrated by network communication, the average value of the abnormal features is calculated by the average value formula; Based on the peak value of each abnormal feature and the average value of the abnormal feature, the standard deviation of each abnormal feature is calculated by the difference formula; Based on the average value of the abnormal features, the standard deviation of each abnormal feature and the confidence of the abnormal features integrated by network communication, the upper limit and the lower limit of the abnormal range are determined by the abnormal range formula; According to the upper limit and the lower limit of the abnormal range, the historical network communication data is judged whether it is within the abnormal range. If not, no output is made. If so, the output is abnormal data. The peak value formula is: ; Where G is the peak value of each abnormal feature, The maximum value of the abnormal characteristics integrated for network communication, Minimum value for integrating abnormal features for network communication; The average value formula is: ; In the formula, is the average value of the abnormal features, is the peak value of the ith abnormal feature, and m is the total number of abnormal features; The difference formula is: ; Where C is the standard deviation of each abnormal feature; The abnormal range formula is: ; In the formula, is the upper limit of the abnormal range, is the lower limit of the abnormal range, z is the confidence level, and the confidence level is 95%.

2. A method for intelligently identifying and analyzing abnormalities in network communication data according to claim 1, characterized in that: Based on historical network communication data, feature extraction is performed on historical network communication data to obtain network communication anomaly data sets, including: Based on historical network communication data, each communication data is parsed to extract timestamps, data packet flow size, number of data packets and transmission protocols, and the time characteristics, flow characteristics and transmission protocol characteristics of each communication data are determined; Normalize each communication data according to its time characteristics, traffic characteristics and transmission protocol characteristics; Based on the time characteristics, traffic characteristics and transmission protocol characteristics of the normalized communication data, abnormal features are annotated and formed into a network communication anomaly dataset.

3. A method for intelligently identifying and analyzing abnormalities in network communication data according to claim 2, characterized in that: Based on the time series clustering data, the subsequences in the time series clustering data are fused to determine the abnormal features of the network communication data, including: Based on the time series clustering data, the median of all subsequences in the cluster at the same time point of the time series clustering data is selected as the representative subsequence; Each representative subsequence is standardized to obtain a number of standardized subsequences; Based on several standardized subsequences, feature splicing is performed on each standardized subsequence to obtain abnormal features of network communication data.

4. A network communication data intelligent anomaly identification and analysis system, characterized in that: A method for intelligently identifying and analyzing abnormalities in network communication data according to any one of claims 1 to 3, comprising: Data collection module, the data collection module is used to collect historical network communication data; An abnormality marking module, the abnormality marking module is electrically connected to the data acquisition module, and the abnormality marking module is used to extract features of the historical network communication data based on the historical network communication data to obtain a network communication abnormality data set; A time series clustering module, which is electrically connected to the anomaly annotation module, is used to perform feature classification on the network communication anomaly data set using a classifier based on the network communication anomaly data set, obtain a number of subsequences of different scales, and combine them into time series clustering data; A feature fusion module, the feature fusion module is electrically connected to the time series clustering module, and the feature fusion module is used to perform feature fusion on subsequences in the time series clustering data based on the time series clustering data to determine abnormal features of the network communication data; The anomaly analysis module is electrically connected to the feature fusion module. The anomaly analysis module is used to determine whether the network communication integrated anomaly features are within the abnormal range based on the network communication integrated anomaly features and historical network communication data. If not, no output is made. If so, the output is abnormal data.

5. A network communication data intelligent anomaly identification and analysis system according to claim 4, characterized in that: The exception annotation module includes: The parsing unit parses each communication data based on the historical network communication data, extracts the timestamp, data packet flow size, data packet quantity and transmission protocol, and determines the time characteristics, flow characteristics and transmission protocol characteristics of each communication data; A normalization unit performs normalization processing according to the time characteristics, flow characteristics and transmission protocol characteristics of each communication data; The abnormal data set unit annotates abnormal features based on the time features, flow features and transmission protocol features of the normalized communication data and forms a network communication abnormal data set.

6. A network communication data intelligent anomaly identification and analysis system according to claim 4, characterized in that: The time series clustering module includes: The decomposition unit uses a discrete wavelet transform function based on the network communication anomaly data set to select the number of decomposition layers for the network communication anomaly data set, perform multi-level decomposition, and obtain detail coefficients and approximate coefficients at different scales; The feature vector unit constructs the detail coefficients and approximation coefficients at different scales into the network communication data feature vector; The classification unit uses a classifier to classify the network communication data feature vector to obtain normal categories and abnormal categories; The restoration unit restores the detail coefficients and approximate coefficients of different scales in the normal category and the abnormal category to the original signal through the inverse discrete transformation function based on the normal category and the abnormal category; The clustering unit extracts several subsequences based on the original signal through a sliding window, and clusters the subsequences according to time to obtain time series clustering data.

7. The intelligent abnormality identification and analysis system for network communication data according to claim 4 is characterized in that: The feature fusion module includes: A screening unit, based on the time series clustering data, screens out the medians of all subsequences in the cluster of the time series clustering data at the same time point as representative subsequences; A standardization unit, which performs standardization processing on each representative subsequence to obtain a number of standardized subsequences; The splicing unit, based on several standardized subsequences, performs feature splicing on each standardized subsequence to obtain abnormal features of network communication data.

8. The intelligent abnormality identification and analysis system for network communication data according to claim 4 is characterized in that: The exception analysis module includes: A confidence unit, which determines the confidence of the abnormal characteristics of the network communication integration; The peak value calculation unit, based on the network communication integration abnormality characteristics and historical network communication data, screens out the maximum and minimum values ​​of each network communication integration abnormality characteristic, and determines the peak value of each abnormality characteristic through the peak value formula; The average value calculation unit calculates the average value of the abnormal features through an average value formula based on the peak value of each abnormal feature and the total number of abnormal features integrated by network communication; A standard deviation calculation unit calculates the standard deviation of each abnormal feature through a difference formula based on the peak value of each abnormal feature and the average value of the abnormal feature; The abnormal range confirmation unit determines the upper limit and the lower limit of the abnormal range through the abnormal range formula based on the average value of the abnormal features, the standard deviation of each abnormal feature and the confidence of the abnormal features integrated by network communication; The judgment unit judges whether the historical network communication data is within the abnormal range according to the upper limit and the lower limit of the abnormal range. If not, no output is made. If yes, the abnormal data is output.