Method and system for detecting abnormal behavior of application software based on network communication similarity

Abstract

The present invention discloses a method and system for detecting abnormal behavior of application software based on network communication similarity, belonging to the field of network security technology. The method comprises: grouping data packets according to data packet characteristics, and constructing each group of data packets into a software behavior chain; generating a feature vector describing the statistical characteristics of the behavior chain for each software behavior chain; obtaining the abnormal behavior detection result of the software behavior chain based on the similarity between the feature vector describing the statistical characteristics of the behavior chain and the center vector of each cluster in the baseline model; wherein the baseline model is constructed based on normal application software communication flow data. The present invention can more accurately observe the abnormality of the network communication behavior of the application software.

Description

Technical Field

[0001] The present invention relates to the technical field of network security, and in particular to a method and system for detecting abnormal behavior of application software based on network communication similarity. Background Art

[0002] Software communication behavior anomaly detection is an important technology in the field of network security. With the rapid development of information technology, network attack methods are constantly evolving, security threats continue to generalize, and traditional network security protection methods can no longer meet the increasingly urgent needs of network security protection. Security protection systems based on access authorization are easily bypassed by new attacks, and security protection systems based on feature detection require continuous iteration and upgrading of rules. Traditional network security protection methods are difficult to effectively protect against new attacks in a timely manner, and lack the perception and expectation of unknown attacks. Therefore, studying how to introduce automated and intelligent methods to detect abnormal behaviors in the network and promptly discover new or unknown attacks has become one of the hot issues in the current research field of network security.

[0003] The innovation of artificial intelligence and machine learning technology has led to some new research trends in the field of software communication behavior anomaly detection. For example, the use of deep learning algorithms to analyze and model large-scale software traffic data can greatly shorten the detection time due to the multiplication of the computing space that can be covered, and the timeliness and effectiveness can reach the practical scope; baseline behavior modeling based on machine learning improves the accuracy of anomaly detection; the behavior anomaly analysis system that combines behavior analysis with pattern recognition greatly improves the accuracy and timeliness of detection while improving the intelligence and applicability of anomaly detection. However, software communication behavior anomaly detection still faces challenges such as large data volume, complex features, and low algorithm efficiency. Summary of the invention

[0004] In order to observe the abnormality of application software network communication behavior, the present invention proposes an application software abnormal behavior detection method and system based on network communication similarity, which obtains the application software network communication behavior according to network data packet characteristics to calculate its behavior similarity.

[0005] In order to achieve the above-mentioned object of the invention, the technical solution of the present invention includes the following contents.

[0006] A method for detecting abnormal behavior of application software based on network communication similarity, the method comprising:

[0007] Group data packets according to their characteristics, and construct each group of data packets into a software behavior chain;

[0008] Generate a feature vector describing the statistical characteristics of each software behavior chain;

[0009] Based on the similarity between the feature vector describing the statistical characteristics of the behavior chain and the central vector of each cluster in the baseline model, the abnormal behavior detection result of the software behavior chain is obtained; wherein the baseline model is constructed based on normal application software communication traffic data.

[0010] Furthermore, the data packets are grouped according to the data packet characteristics, and each group of data packets is constructed into a software behavior chain, including:

[0011] Receive packets of network traffic using a sliding window;

[0012] Extracting data packet features of each data packet, the data packet features including: protocol type, departure time, arrival time, source address, destination address, source port, destination port, payload, software signature, data packet length, TCP handshake metadata, total number of bytes, maximum / minimum / average / standard deviation of packet length, maximum / minimum / average / standard deviation of packet interval time, and information entropy;

[0013] Group data packets based on software signature, protocol type, source address, source port, destination address, destination port, and payload;

[0014] Sort the grouped data packets by departure time to obtain the corresponding software behavior chain.

[0015] Further, the data packet grouping according to the software signature, protocol type, source address, source port, destination address, destination port and payload includes:

[0016] Obtain a data packet grouping strategy, the data packet grouping strategy comprising:

[0017] Data packets with the same software signature are grouped together;

[0018] and,

[0019] For packets with protocol type TCP, packets from the same TCP connection are grouped together;

[0020] and,

[0021] Data packets with the same protocol type, source address, source port, destination address, and destination port are grouped together;

[0022] and,

[0023] Grouping packets with the same specific file type or keyword in the payload;

[0024] In the case that the data packet characteristics of at least two data packets satisfy the grouping strategy, the data packets are grouped together.

[0025] Furthermore, the step of generating a feature vector describing the statistical characteristics of each software behavior chain includes:

[0026] Get the vector representation x of all data packets in the software behavior chain i ;

[0027] By analyzing the self-attention calculation of the data packet feature correlation, the vector representation x is obtained i The corresponding vector represents s i ;

[0028] The vector representation s corresponding to the software behavior chain is combined i After batch normalization, the combined result is calculated by analyzing the self-attention of the correlation between the data packets to obtain the vector representation corresponding to the combined result;

[0029] The vector representation corresponding to the combination result is passed through the fully connected layer and the softmax layer in sequence to obtain a feature vector that describes the statistical characteristics of the behavior chain.

[0030] Furthermore, the behavior chain statistical features include: vector dimension, mean, standard deviation, median, median absolute deviation, skewness, kurtosis and Mahalanobis distance.

[0031] Furthermore, the baseline model is constructed based on normal application software communication traffic data, including:

[0032] Based on normal application software communication traffic data, build several normal software behavior chains;

[0033] After generating a feature vector describing the statistical characteristics of each normal software behavior chain, clustering processing is performed to obtain a baseline model.

[0034] Furthermore, the generating of the feature vector describing the statistical characteristics of each normal software behavior chain and then performing clustering processing to obtain a baseline model includes:

[0035] Each feature vector describing the statistical characteristics of the behavior chain is considered as an independent cluster;

[0036] By calculating the similarity between feature vectors, several clusters V are generated. i (0) , i is a positive integer;

[0037] Cluster V i (t-1) The variance of the similarity between each eigenvector and other eigenvectors in the cluster V is generated. i (t-1) The center vector of ; where t is the iteration round;

[0038] According to each cluster V i (t-1) The center vector and other clusters V j (t-1) Generate several clusters V based on the similarity of the center vector i (t) Then, let t = t + 1, and return to the cluster V based i (t-1) The variance of the similarity between each eigenvector and other eigenvectors in ; where j is a positive integer and j≠i;

[0039] Until the number of clusters in iteration round t and iteration round t+1 does not change, or the number of clusters V in iteration round t i (t) The total number of is set to k, and the baseline model is obtained; where k is the number of types of basic network behaviors.

[0040] Furthermore, the similarity between the feature vectors Among them, w r is the learnable weight matrix, Cov(β i , β j ) is the eigenvector β i With the eigenvector β j The covariance between i ) is the eigenvector β i Standard deviation, σ(β j ) is the eigenvector β j The standard deviation of .

[0041] Furthermore, after obtaining the abnormal behavior detection result of the software behavior chain based on the similarity between the feature vector describing the statistical characteristics of the behavior chain and the central vector of each cluster in the baseline model, the method further includes:

[0042] Manually analyze and judge the abnormal behavior detection results of the software behavior chain;

[0043] In the case of misjudgment of the abnormal behavior detection result of the software behavior chain, the similarity of the misjudged behavior is recalculated and the learnable weight matrix w is adjusted. r .

[0044] A system for detecting abnormal behavior of application software based on network communication similarity, the system comprising:

[0045] A data packet grouping module is used to group data packets according to data packet characteristics and construct each group of data packets into a software behavior chain;

[0046] A vector generation module, used to generate a feature vector describing the statistical characteristics of each software behavior chain;

[0047] The anomaly detection module is used to obtain the abnormal behavior detection result of the software behavior chain based on the similarity between the feature vector describing the statistical characteristics of the behavior chain and the central vector of each cluster in the baseline model; wherein the baseline model is constructed based on normal application software communication traffic data.

[0048] Compared with the prior art, the present invention combines deep learning models with behavior analysis and designs a method for analyzing the similarity of network communication behaviors using deep learning models. The present invention uses a self-attention mechanism to capture long-distance dependencies between data, thereby more accurately characterizing the characteristics of the software behavior chain; extracts baselines of similar behaviors based on clustering algorithms to obtain a baseline model of the behavior; based on the output behavior baseline of the baseline model, real-time traffic can be processed and analyzed, and the communication behavior of the application software can be judged to be abnormal by comparing the similarity between the newly generated behavior and the normal behavior in the model, providing scientific data support for the detection of abnormal behavior of the application software. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 It is a flow chart of a method for detecting abnormal behavior of application software based on network communication similarity.

[0050] Figure 2 Schematic diagram of the contextual relationship of the self-attention behavior chain. DETAILED DESCRIPTION

[0051] In order to make the objectives, technical solutions and advantages of the present invention more clear, the embodiments of the present invention will be described in further detail below.

[0052] The abnormal behavior detection method of application software based on network communication similarity of the present invention is as follows: Figure 1 As shown, the following steps are included.

[0053] Step 1: Group data packets according to their characteristics and construct each group of data packets into a software behavior chain.

[0054] This embodiment uses a 256-bit sliding window to receive network data packets. When the sliding window reaches the upper limit, in order to obtain a series of network behaviors of the software, the data packets are grouped according to their characteristics, and each group of data packets is constructed into a software behavior chain. In the behavior chain, each data packet represents a node of the chain.

[0055] Specifically, the present invention parses the collected data packets one by one, extracts the protocol fields and effective feature information, and classifies the data packets accordingly. The features used include but are not limited to protocol type (such as TCP, UDP, IP, DNS, etc.), departure time, arrival time, source IP, destination IP, source port, destination port, payload, software signature, data packet length, TCP handshake metadata, total number of bytes, maximum / minimum / average / standard deviation of packet length, maximum / minimum / average / standard deviation of packet interval time, and information entropy, etc. Based on the extracted features, predefined application signatures or rules are used for matching to determine the specific application used in the data packet.

[0056] The specific grouping methods are divided into the following methods, which can be used in combination:

[0057] (1) Software with the same signature are grouped together;

[0058] (2) For TCP connections, data packets from the same TCP connection are grouped together;

[0059] (3) Grouping the packets with the same protocol, source address, source port, destination address, and destination port;

[0060] (4) Files with the same specific file type or keyword in the payload are grouped together.

[0061] The grouped data packets are sorted by departure time, and the features of the data packets are normalized. A representation vector is generated for each data packet. A group of representation vectors is a behavior chain, and each data packet represents a node in the behavior chain.

[0062] Step 2: Generate a feature vector describing the statistical characteristics of each software behavior chain.

[0063] The present invention uses the self-attention mechanism to calculate the contextual relationship between nodes in the behavior chain. Based on the output of the self-attention, its dimension is counted, and statistical characteristics such as the average value and standard deviation are calculated to obtain a feature vector describing the statistical characteristics of the behavior chain.

[0064] In one embodiment, for a behavior chain, first, each of its nodes passes through a self-attention component, which analyzes the correlation of the node's internal features and outputs a fragment vector; then the fragment vectors are combined and input into another self-attention layer after batch normalization. This self-attention component analyzes the correlation between the nodes of the entire behavior chain; then it passes through batch normalization and enters the fully connected layer, which uses ReLU as the activation function; finally, it passes through the softmax layer to obtain the final output result. The specific process is as follows: Figure 2 shown.

[0065] Among them, the calculation formula of self-attention is:

[0066] Let the input vector X = (x1, x2, ..., x n ), the output vector is Z = (z1, z2, ..., z n ),but:

[0067]

[0068] Among them, W Q , W K , Wv is a learnable weight matrix, and d is the feature dimension.

[0069] Therefore, the final output vector of the self-attention component is α = (s1, s2, ..., s n ), calculate the eigenvector β = (y1, y2, ..., y n ), the meaning of the value of β is:

[0070] β = (vector dimension, mean, standard deviation, median, median absolute deviation, skewness, kurtosis, Mahalanobis distance) where the median absolute deviation can indicate the deviation of the concentrated position of the behavioral characteristics of the behavior chain relative to the mean; the skewness indicates the bias of the behavior chain, left or right; the kurtosis indicates the peak value of the behavior chain; and the Mahalanobis distance indicates the similarity between the node and the mean.

[0071] The calculation formula is as follows:

[0072] Vector dimension: n

[0073] Average value: μ=(x1+x2+...+x n ) / n

[0074] Standard Deviation:

[0075] Median: m

[0076] Median absolute deviation: MAD = median(|x i -median(α)|)

[0077] Skewness:

[0078] Kurtosis:

[0079] Mahalanobis distance: Where S is the covariance matrix, let γ = (x1-μ,...,x n -μ),

[0080] Step 3: Based on the similarity between the feature vector describing the statistical characteristics of the behavior chain and the central vector of each cluster in the baseline model, the abnormal behavior detection result of the software behavior chain is obtained.

[0081] The baseline model of the present invention is constructed based on normal application software communication traffic data. In the training set, the collected data are all behavioral data of normal network communication of the software, without abnormal data. After training, the baseline of normal behavior of the software is obtained. For the data in the training set, invalid traffic, such as repetition, TCP retransmission, HTTP handshake data packet, etc., needs to be removed; and repeated traffic needs to be merged. The traffic data is pre-processed as subsequent input data.

[0082] After that, after all the data has been processed, the feature vectors are clustered using a clustering algorithm. In the clustering process, the similarity between the feature vectors is used as the classification standard. The similarity distance between the feature vector and the center vector of each cluster needs to be recorded, and the one with the largest similarity distance is selected as the baseline standard for judging abnormal behavior. The model obtained after training will become the baseline model for detecting abnormal behavior in the present invention.

[0083] This embodiment constructs a baseline model by classifying feature vectors based on a clustering algorithm. m ,The hierarchical clustering method is adopted and the number of clusters after ,classification is set as k, and the k value is determined by the ,number of basic network behavior types of the software.

[0084] First, each vector is considered as an independent cluster, and the similarity between each feature vector is calculated. The vectors with closer similarity are merged into the same cluster. Then the central vector of each cluster is calculated by calculating the variance of the similarity between each vector and other vectors in the cluster. The vector with the smallest variance is used as the central vector. If the variances are equal, one of the vectors is randomly selected as the central vector of the cluster. Finally, the similarity between the central vectors of each cluster is calculated, and clusters with higher similarity are merged. Repeat this process until there is no change from the previous classification or the number of clusters is k. After the classification is completed, each category has a central vector. The similarity between the vectors in the cluster and the central vector is calculated, and the vector with the lowest similarity is selected as the baseline value. The generated model is the baseline model.

[0085] Among them, the formula for calculating the similarity of feature vectors is: Among them, w r is a learnable weight matrix that adjusts the similarity between vectors; Cov(β i ,β j )=E[β i β j ]-E[β i ]E[β j], is β i , β j The covariance between σ(β i ),σ(β j ) is the vector β i , β j The standard deviation of .

[0086] During the test, the similarity between each feature vector and the central vector of each cluster in the baseline model is calculated. If a feature vector cannot be classified into any cluster, the behavior chain is regarded as an abnormal communication behavior of the software.

[0087] Step 4: Optimize the baseline model through manual analysis.

[0088] Abnormal communication behaviors detected by the model need to be manually judged. If there is a misjudgment, the similarity of the misjudged behavior is recalculated, and the learnable weight matrix w used to calculate the similarity in the adjustment step is adjusted. r , to optimize the baseline model

[0089] In summary, in order to obtain a series of network behaviors of the software, the present invention groups the collected network data packets according to the data packet features, distinguishes the data packets sent by each software and the different behaviors represented by the data packets. The types of features used include but are not limited to statistical features, protocol-based features, timing features, manually annotated features, features extracted by deep learning models, etc., and these features and feature extraction methods are combined to extract effective features. According to the features, the data packets are classified based on predefined rules, and finally the features are normalized.

[0090] In order to obtain a series of network behaviors of the software, the present invention groups the collected network data packets according to the data packet features, distinguishes the data packets sent by each software and the different behaviors represented by the data packets. The types of features used include but are not limited to statistical features, protocol-based features, timing features, manually annotated features, features extracted by deep learning models, etc., and these features and feature extraction methods are combined to extract effective features. According to the features, the data packets are classified based on predefined rules, and finally the features are normalized.

[0091] The present invention obtains the contextual relationship of network communication behavior based on self-attention. Self-attention is an attention mechanism for processing sequence data. Self-attention allows the model to assign different attention weights to different positions in the sequence when processing the sequence, and more flexibly captures the dependencies in the sequence. Therefore, the present invention uses a self-attention component to process software communication behavior, first obtains the internal relationship of a single node in the behavior, obtains the fragment vector of the single node, and then combines the fragment vectors to obtain the contextual relationship of the entire behavior chain.

[0092] The statistical features of the present invention can understand the behavioral trends and discreteness of the software behavior chain. For the vector output by self-attention, its dimension is counted, and statistical features such as the mean, standard deviation, variance, deviation, median absolute deviation, and kurtosis are calculated to obtain a feature vector that characterizes the statistical characteristics of the communication behavior.

[0093] The specific number of categories of the present invention is determined according to the type of software behavior. Different software has similar basic behaviors, so these basic behaviors from different software can be classified into the same category through clustering, and the special behaviors of the software itself can be classified into a separate category. After the classification is completed, the similarity baseline of each category is recorded.

[0094] The above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit the same. A person skilled in the art may modify or replace the technical solutions of the present invention with equivalents. The protection scope of the present invention shall be subject to the claims.

Claims

1. A method for detecting abnormal behavior of application software based on network communication similarity, characterized in that: The method comprises: Group data packets according to their characteristics, and construct each group of data packets into a software behavior chain; Generate a feature vector describing the statistical characteristics of the behavior chain for each software behavior chain; wherein, generating a feature vector describing the statistical characteristics of the behavior chain for each software behavior chain includes: Get the vector representation x of all data packets in the software behavior chain i ; By analyzing the self-attention calculation of the data packet feature correlation, the vector representation x is obtained i The corresponding vector represents s i ; The vector representation s corresponding to the software behavior chain is combined i After batch normalization, the combined result is calculated by analyzing the self-attention of the correlation between the data packets to obtain the vector representation corresponding to the combined result; The vector representation corresponding to the combined result is passed through the fully connected layer and the softmax layer in sequence to obtain a feature vector describing the statistical characteristics of the behavior chain; Based on the similarity between the feature vector describing the statistical characteristics of the behavior chain and the central vector of each cluster in the baseline model, the abnormal behavior detection result of the software behavior chain is obtained; wherein the baseline model is constructed based on normal application software communication traffic data, and the construction process of the baseline model includes: Based on normal application software communication traffic data, build several normal software behavior chains; Generate a feature vector describing the statistical characteristics of each normal software behavior chain, and then perform clustering processing to obtain a baseline model; wherein the generating a feature vector describing the statistical characteristics of each normal software behavior chain, and then perform clustering processing to obtain a baseline model, includes: Each feature vector describing the statistical characteristics of the behavior chain is considered as an independent cluster; By calculating the similarity between feature vectors, several clusters V are generated. i (0) , i is a positive integer; Based on Cluster V i (t-1) The variance of the similarity between each eigenvector and other eigenvectors in the cluster V is generated. i (t-1) The center vector of ; where t is the iteration round; According to each cluster V i (t-1) The center vector and other clusters V j (t-1) Generate several clusters V based on the similarity of the center vector i (t) Then, let t = t + 1, and return to the cluster V based i (t-1) The variance of the similarity between each eigenvector and other eigenvectors in ; where j is a positive integer and j≠i; Until the number of clusters in iteration round t and iteration round t+1 does not change, or the number of clusters V in iteration round t i (t) The total number of is set to k, and the baseline model is obtained; where k is the number of types of basic network behaviors.

2. The method according to claim 1, characterized in that The step of grouping data packets according to data packet characteristics and constructing each group of data packets into a software behavior chain includes: Receive packets of network traffic using a sliding window; Extracting data packet features of each data packet, the data packet features including: protocol type, departure time, arrival time, source address, destination address, source port, destination port, payload, software signature, data packet length, TCP handshake metadata, total number of bytes, maximum / minimum / average / standard deviation of packet length, maximum / minimum / average / standard deviation of packet interval time, and information entropy; Group data packets based on software signature, protocol type, source address, source port, destination address, destination port, and payload; Sort the grouped data packets by departure time to obtain the corresponding software behavior chain.

3. The method according to claim 2, characterized in that The data packet grouping according to the software signature, protocol type, source address, source port, destination address, destination port and payload includes: Obtain a data packet grouping strategy, the data packet grouping strategy comprising: Data packets with the same software signature are grouped together; and, For packets with protocol type TCP, packets from the same TCP connection are grouped together; and, Data packets with the same protocol type, source address, source port, destination address, and destination port are grouped together; and, Grouping packets with the same specific file type or keyword in the payload; In the case that the data packet characteristics of at least two data packets satisfy the grouping strategy, the data packets are grouped together.

4. The method according to claim 1, characterized in that: The behavior chain statistical features include: vector dimension, mean, standard deviation, median, median absolute deviation, skewness, kurtosis and Mahalanobis distance.

5. The method according to claim 1, characterized in that The similarity between the feature vectors Among them, w r is the learnable weight matrix, Cov(β i ,β j ) is the eigenvector β i With the eigenvector β j The covariance between i ) is the eigenvector β i Standard deviation, σ(β j ) is the eigenvector β j The standard deviation of .

6. The method according to claim 5, characterized in that After obtaining the abnormal behavior detection result of the software behavior chain based on the similarity between the feature vector describing the statistical characteristics of the behavior chain and the central vector of each cluster in the baseline model, the method further includes: Manually analyze and judge the abnormal behavior detection results of the software behavior chain; In the case of misjudgment of the abnormal behavior detection result of the software behavior chain, the similarity of the misjudged behavior is recalculated and the learnable weight matrix w is adjusted. r .

7. A system for detecting abnormal behavior of application software based on network communication similarity, characterized in that: The system comprises: A data packet grouping module is used to group data packets according to data packet characteristics and construct each group of data packets into a software behavior chain; A vector generation module is used to generate a feature vector describing the statistical characteristics of the behavior chain of each software behavior chain, wherein the generating of the feature vector describing the statistical characteristics of the behavior chain of each software behavior chain includes: Get the vector representation x of all data packets in the software behavior chain i ; By analyzing the self-attention calculation of the data packet feature correlation, the vector representation x is obtained i The corresponding vector represents s i ; The vector representation s corresponding to the software behavior chain is combined i After batch normalization, the combined result is calculated by analyzing the self-attention of the correlation between the data packets to obtain the vector representation corresponding to the combined result; The vector representation corresponding to the combined result is passed through the fully connected layer and the softmax layer in sequence to obtain a feature vector describing the statistical characteristics of the behavior chain; The anomaly detection module is used to obtain the abnormal behavior detection result of the software behavior chain based on the similarity between the feature vector describing the statistical characteristics of the behavior chain and the central vector of each cluster in the baseline model; wherein the baseline model is constructed based on normal application software communication traffic data, and the construction process of the baseline model includes: Based on normal application software communication traffic data, build several normal software behavior chains; Generate a feature vector describing the statistical characteristics of each normal software behavior chain, and then perform clustering processing to obtain a baseline model; wherein the generating a feature vector describing the statistical characteristics of each normal software behavior chain, and then perform clustering processing to obtain a baseline model, includes: Each feature vector describing the statistical characteristics of the behavior chain is considered as an independent cluster; By calculating the similarity between feature vectors, several clusters V are generated. i (0) ,i is a positive integer; Based on Cluster V i (t-1) The variance of the similarity between each eigenvector and other eigenvectors in the cluster V is generated. i (t-1) The center vector of ; where t is the iteration round; According to each cluster V i (t-1) The center vector and other clusters V j (t-1) Generate several clusters V based on the similarity of the center vector i (t) Then, let t = t + 1, and return to the cluster V based i (t-1) The variance of the similarity between each eigenvector and other eigenvectors in ; where j is a positive integer and j≠i; Until the number of clusters in iteration round t and iteration round t+1 does not change, or the number of clusters V in iteration round t i (t) The total number of is set to k, and the baseline model is obtained; where k is the number of types of basic network behaviors.

Images