An APT attack detection method based on threat intelligence and traffic features

Abstract

The application discloses an APT attack detection method based on threat intelligence and traffic characteristics, and comprises the following steps: A, obtaining an original attack sequence set and marking; B, obtaining an attack sequence set after data cleaning and a corresponding network traffic characteristic set; C, constructing a local threat intelligence library and obtaining threat intelligence characteristics of each original traffic; D, constructing a sample message characteristic library and obtaining message characteristics of the sequence set; E, using the obtained network traffic characteristic set, threat intelligence characteristics and message characteristics to train a multi-fusion detection model based on an LSTM recurrent neural network and obtain a trained multi-fusion detection model; F, using the trained multi-fusion detection model to detect APT attacks on unknown traffic sequences, and determining whether APT attacks exist. The application can accurately and effectively detect possible APT attack behaviors in network traffic, so that the safety and stability of information systems and networks are realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of Internet data security, and in particular to an APT attack detection method based on threat intelligence and traffic characteristics. BACKGROUND

[0002] APT attack is a complex and advanced persistent threat attack, mainly referring to network attacks against important enterprises, governments, research institutions and other high-value targets. Compared with general network attacks, APT attacks often cause very serious damage, including important information theft and key infrastructure destruction.

[0003] APT attacks are usually long-term and persistent network attacks against specific targets using advanced composite attack methods. Compared with other attack methods, APT attack methods are more covert, have difficulty in detection, long duration, and clear attack targets. Therefore, the traditional intrusion detection and defense method based on attack characteristics is not ideal in detecting and defending APT. With the continuous development of deep learning technology, more and more researchers use deep learning technology to detect and reconstruct APT attacks. However, due to the complexity, concealment and targeting of APT attacks, attackers often take various behaviors to evade system detection, so the traditional deep learning technology does not achieve the desired effect.

[0004] The Chinese invention patent with application number 202210948265.9 and application date August 8, 2022, "APT attack identification method, device, electronic equipment and medium", discloses an APT attack identification method, including: obtaining network attack related information for a to-be-detected object; extracting APT attack features from the network attack related information; according to the APT attack features, performing multi-dimensional threat point detection through feature detection, behavior detection and machine learning to obtain a first APT attack identification result, a second APT attack identification result and a third APT attack identification result of the to-be-detected object; and determining a target APT attack identification result of the to-be-detected object according to the first APT attack identification result, the second APT attack identification result and the third APT attack identification result. The application can apply different detection methods according to the characteristics of each detection point by feature detection, behavior detection and machine learning, which can effectively improve the detection efficiency and reduce false positives. However, when performing multi-dimensional threat point detection, the extracted threat feature dimension is limited, which affects the overall APT attack identification effect. SUMMARY

[0005] The application aims to provide an APT attack detection method based on threat intelligence and traffic characteristics, which can accurately and effectively detect possible APT attack behaviors in network traffic, thereby guaranteeing the security and stability of information systems and networks.

[0006] The application adopts the following technical solutions:

[0007] An APT attack detection method based on threat intelligence and traffic characteristics, sequentially comprising the following steps:

[0008] A: Obtain the original traffic sequence of the APT attack as an original attack sequence set D1, and set a label for each original traffic in the original attack sequence set D1 according to the attributes of the network request, marked as an attack request or a normal request;

[0009] B: Extract the network traffic characteristics of the original traffic in the original attack sequence set D1, and perform data cleaning on the original traffic data to obtain a data-cleaning attack sequence set D2 and a corresponding network traffic characteristic set F;

[0010] C: Construct a local threat intelligence library using an existing APT attack threat intelligence library, and compare and calculate each original traffic in the attack sequence set D2 with the IP address, domain name, intelligence source authority, publication time and update time of the threat intelligence in the local threat intelligence library, to obtain the threat intelligence characteristics T of each original traffic, including the intelligence source authority characteristics T confidence , time effectiveness characteristics T time , IP matching characteristics T IP and domain name matching characteristics T Domain ;

[0011] D: Construct a sample message feature library, and compare and calculate each original traffic in the attack sequence set D2 with the request domain name value, request IP address, request URL value and request parameter of the APT attack sample in the sample message feature library, to obtain the message characteristics P of the sequence set D2, including the domain name characteristics P Domain , IP characteristics P IP , URL characteristics P URL and request parameter characteristics P Parameter ;

[0012] E: Use the network traffic characteristic set F obtained in step B, the threat intelligence characteristics T obtained in step C and the message characteristics P obtained in step D to train a multi-fusion detection model based on the LSTM recurrent neural network, and finally obtain the trained multi-fusion detection model;

[0013] F: Using the trained multi-fusion detection model to detect APT attacks on unknown traffic sequences, determine whether there is an APT attack.

[0014] The step C comprises the following specific steps:

[0015] C1: Collect and organize threat intelligence, and construct a local threat intelligence library;

[0016] C2: Unpack each raw traffic in the attack sequence set D2 respectively, and obtain the raw traffic d i All domain names and IP addresses corresponding to the request;

[0017] C3: Extract the corresponding threat intelligence features T for each raw traffic in the attack sequence set D2; the threat intelligence features T include intelligence source authority features T confidence , time effectiveness features T time , IP matching features T IP and domain name matching features T Domain .

[0018] In the step C3: for the intelligence source authority T confidence , different intelligence source authorities T confidence are set according to different sources of APT threat intelligence;

[0019] For the time effectiveness feature T time , let the current time be t now , and the last update time of the threat intelligence be t update , then the time effectiveness feature of the threat intelligence data is Wherein, V is the time difference threshold.

[0020] In the step C3, the extraction process of the IP matching feature T IP is as follows:

[0021] c31-1: According to the threat intelligence library, calculate the IP address segment I that may be controlled by the attacker;

[0022] Let the integer type of the request IP address in the jth intelligence R j in the threat intelligence library be r IP , then the IP address segment I that may be controlled by the attacker is {r IP +l m}, wherein, l m is an integer sequence of {-m,…,m}, and m is a preset value;

[0023] c31-2: Calculate the IP matching feature T IP ;

[0024] If the raw traffic di the request IP address integer type q IP In the threat intelligence R j , if the control end IP address segment I is in the original traffic d i , the request IP address corresponds to the IP matching feature The IP matching feature of the original traffic d i ; Where j is the number of intelligence in the threat intelligence library.

[0025] In step C3, the domain name matching feature T Domain is extracted as follows:

[0026] c32-1: Extract the domain name valid string E in each original traffic d i , and the domain name valid string sequence G j in each APT threat intelligence in the threat intelligence library, respectively.

[0027] c32-2: Compare the domain name valid string E with the string in the domain name valid string sequence G j in the threat intelligence library one by one, and calculate the domain name matching feature T Domain ; The specific method is as follows:

[0028] For the domain name valid string E and the string G in the domain name valid string sequence G j in the threat intelligence library, let g len be the length of the string G, e len be the length of the domain name valid string E; Ar g is a 0 array with length g len , used to record the matching record of the string G; Ar e is a 0 array with length e len , used to record the matching record of the string E; min len represents the minimum length allowed for matching, if the matching length is less than min len , skip; max len represents the longest possible match of the substring E r starting from r and the substring G s starting from s in the comparison string, the substring E r is a substring of the valid string E, and the substring G s is a substring of the string G; traverse the characters of the substrings E r and G s one by one to obtain the maximum length k len of the continuous similar string in the current positioning; traverse all substrings according to the above method to obtain all continuous similar maximum length sets K lenThe similarity of the two strings is:

[0029]

[0030] Domain name matching feature of traffic sequence

[0031] The step D comprises the following specific steps:

[0032] D1: Collecting APT attack samples to construct a sample packet feature library, extracting network request features of each APT attack sample in the sample packet feature library through static and / or dynamic analysis, wherein the network request features include request domain name value, request IP address, request URL value and request parameter;

[0033] D2: Unpacking each original traffic in the attack sequence set D2 respectively to obtain the original traffic d i , and the corresponding request packet, splitting the request domain name value d Domain , URL request value d URL , request IP value d IP , and request parameter value d Parameter from the request packet;

[0034] D3: Extracting domain name features P Domain ;

[0035] For the request domain name valid string U of the original traffic d i , a certain string H in the domain name valid string sequence H k in the packet feature library, let h len be the length of the string H, u len be the length of the domain name valid string U; Ar h is a 0 array with a length of h len , used to record the matching record of the string H; Ar u is a 0 array with a length of u len , used to record the matching record of the string U; min p_len represents the minimum length allowed for matching, if the matching length is less than min p_len , it is skipped; max p_len represents the longest possible matching of the substring U r starting from r and the substring H s starting from s in the matching process, the substring U r is a substring of the valid string U, and the substring H s is a substring of the string H; the characters of the substrings U r and H s are traversed one by one to obtain the maximum length k of the continuous similar strings in the current positioning; p_len; traverse all substrings by the above method to obtain all continuous similar maximum length sets K p_len , the similarity of the two strings is:

[0036]

[0037] Domain name matching feature of traffic sequence

[0038] D4: extract IP feature P IP ;

[0039] Let the request IP address in the original traffic d i correspond to an integer type q IP , and the request IP value of the kth APT attack sample in the message feature library is Then the IP address similarity i between the original traffic dand the kth APT attack sample is:

[0040]

[0041] IP feature

[0042] D5: extract URL feature P URL ;

[0043] Let the URL request value of the original traffic d i be d URL , and the URL request value of the kth APT attack sample in the message feature library be Calculate the corresponding URL similarity p url using the simhash algorithm to obtain the URL feature

[0044] D6: extract request parameter feature P Parameter ;

[0045] Let the request parameter value of the original traffic d i be d parameter , and the request parameter value of the kth APT attack sample in the message feature library be Calculate the corresponding request parameter similarity P j using the simhash algorithm to obtain the request parameter feature

[0046] In step E, the multi-fusion detection model based on the LSTM recurrent neural network adds a Drop Layer to the Forward Layer and Backward Layer, and an Optimizing Layer to the Backward Layer and Output Layer, respectively, on the basis of the four-layer structure of the LSTM model.

[0047] Step E includes the following specific steps:

[0048] E1: Input data is D input and for D input Increase the weight W1, where W1 is the source authority T obtained in step C3. confidence The formula for calculating the Forward Layer is as follows:

[0049] I t =σ(W i ·[Η t-1 ,x t ]+B i );

[0050] Among them, I t σ is the input gate, W is the activation function. i H is the weight matrix of the input gate. t-1 This is the final output of the Multi-LSTM at the previous time step, B. i It is the bias term of the current input cell state, x t For the input data, namely D input The subsequent x t This is the calculation result of the data from the previous layer;

[0051] E2: Increase weight W in the Forward Layer f and weight W2; where weight W f The weights are the previous layer weights, initially an identity matrix, which is continuously adjusted according to the previous time step. The weights W2 are random 0 and 1 matrices.

[0052] E3: Increase weight W in the Backward Layer b W3 and W4; where the weight W b For the subsequent layer weights, W3 and W4 are the domain name matching features T obtained in step C3, respectively. Domain IP matching feature T IP ;

[0053] The formula for calculating the Backward Layer is as follows:

[0054] F t =σ(W b·[H t-1 ,x t ]+B b );

[0055] wherein F t is a forget gate, B b is a bias term of the current forget gate;

[0056] E4: adding weight W5 in the Optimizing Layer to optimize the result of the Backward Layer, using least square gradient descent to make the result converge quickly;

[0057] The least square gradient descent calculation formula is as follows:

[0058]

[0059] wherein f(x) is a least square gradient descent function, x is input data at the last moment, and a m is a learning rate; and are bias parameters and weight parameters of the mth group of data;

[0060] E5: adding weight W6 in the Output Layer; W6 is domain name feature P Domain , URL feature P URL , IP feature P IP and request parameter feature P Parameter of the request message;

[0061] The calculation method of the Output Layer is as follows:

[0062] O t =σ(W O ·[H t-1 ,x t ]+B O );

[0063] wherein O t is an output gate, W o is a weight matrix of the output gate, and B o is a bias term of the output gate.

[0064] In the step B, for each original flow in the original attack sequence set D1, the network flow features of the original flow are extracted by using the CICFlowMeter method respectively; then the original flow data after the network flow features are extracted are cleaned, the original flow data containing single values and null values are eliminated, and finally the cleaned attack sequence set D2={d1,…,d n} and the corresponding network flow feature set F={f1,…,f n + are obtained; wherein d1 to dn respectively represent the 1st original flow to the nth original flow in the cleaned attack sequence set D2, f1 to fn respectively represent the 1st original flow to the nth original flow in the cleaned attack sequence set D2. n respectively represent the network traffic features corresponding to the 1st original flow to the nth original flow.

[0065] In the step A, the DAPT2020 public data set is used as the original attack sequence set D1; the APT attacks in the DAPT2020 public data set are divided into five stages of investigation, foothold, horizontal, stealing and destruction, and elimination; the five stages of investigation, foothold, horizontal, stealing and destruction, and elimination are defined as attack requests, and the remaining stages are defined as normal requests.

[0066] The application is based on threat intelligence and traffic features, and the local threat intelligence library is constructed and the intelligence source authority degree features T confidence , the time validity features T time , the IP matching features T IP , and the domain name matching features T Domain are obtained; then the sample message feature library is constructed and the domain name features P Domain , the IP features P IP , the URL features P URL , and the request parameter features P Parameter are obtained; finally, the network traffic feature set F, the threat intelligence features T, and the message features P are used to train and use the multi-fusion detection model based on the LSTM recurrent neural network to detect the APT attack of the unknown traffic sequence, and to determine whether there is an APT attack. The application can accurately and effectively detect the possible APT attack behavior in the network traffic, so as to protect the safety and stability of the information system and the network. BRIEF DESCRIPTION OF DRAWINGS

[0067] Figure 1 is a flowchart of the application. DETAILED DESCRIPTION

[0068] The application will be described in detail below in combination with the drawings and examples:

[0069] As shown in the figure, the APT attack detection method based on threat intelligence and traffic features comprises the following steps in sequence: Figure 1 A: obtaining the original traffic sequence of the APT attack as the original attack sequence set D1, and setting a label for each original traffic in the original attack sequence set D1 according to the attributes of the network request, marked as an attack request or a normal request;

[0070]

[0071] ​In the present application, the DAPT2020 public data set can be used as the original attack sequence set D1; the DAPT2020 public data set contains a large amount of original traffic data of APT attacks, and the APT attacks in the DAPT2020 public data set are divided into five stages: investigation, foothold, horizontal, theft and destruction, and removal; in the present application, the five stages of investigation, foothold, horizontal, theft and destruction, and removal are defined as attack requests, and the remaining stages are defined as normal requests;

[0072] B: extracting network traffic features of the original traffic in the original attack sequence set D1, and performing data cleaning on the original traffic data to obtain the attack sequence set D2 after data cleaning and the corresponding network traffic feature set F;

[0073] In the present application, for each original traffic in the original attack sequence set D1, the network traffic features of the original traffic are extracted using the CICFlowMeter method respectively; wherein the network traffic features include a total of 76 features such as flow duration, total number of forward transmission data packets and total number of reverse transmission data packets;

[0074] Then, the original traffic data after extracting the network traffic features is cleaned, and the original traffic data containing single value and null value is removed, and finally the cleaned attack sequence set D2={d1,…,d n} and the corresponding network traffic feature set F={f1,…,f n} are obtained;

[0075] Wherein d1 to d n represent the 1st original traffic to the nth original traffic in the cleaned attack sequence set D2, and f1 to f n represent the network traffic features corresponding to the 1st original traffic to the nth original traffic;

[0076] C: constructing a local threat intelligence library using an existing APT attack threat intelligence library, and comparing and calculating each original traffic in the attack sequence set D2 with the IP address, domain name, intelligence source authority, publication time and update time of the threat intelligence in the local threat intelligence library to obtain the threat intelligence feature T of each original traffic, wherein the threat intelligence feature T includes intelligence source authority feature T confidence , time effectiveness feature T time , IP matching feature T IP and domain name matching feature T Domain ;

[0077] The step C includes the following specific steps:

[0078] C1: collecting and arranging threat intelligence to construct a local threat intelligence library;

[0079] When analyzing and judging the APT attack, the threat intelligence is an important reference. In the present application, the collection of threat intelligence can be carried out through the existing APT attack threat intelligence library. The existing APT attack threat intelligence library includes the databases such as Microstep Online, 360 Threat Intelligence Center and VIRUSTOTAL.

[0080] For each APT threat intelligence obtained, six attribute information of the threat intelligence, i.e. IP address, domain name, intelligence source authority degree, publishing time and updating time, are extracted respectively;

[0081] C2: each original flow in the attack sequence set D2 is unpacked to obtain the original flow d i corresponding to all the requests of the domain name and IP address;

[0082] C3: for each original flow in the attack sequence set D2, the corresponding threat intelligence feature T is extracted; the threat intelligence feature T includes intelligence source authority degree feature T confidence , time validity feature T time , IP matching feature T IP and domain name matching feature T Domain ; wherein, the subscript IP refers to the IP address, Domain refers to the domain name, confidence refers to the intelligence source authority degree, and time refers to the intelligence time validity;

[0083] In the present application, for the intelligence source authority degree T confidence , since the sources of the intelligence are different, the credibility is different, therefore, the present application sets different intelligence source authority degrees T confidence for different sources of APT threat intelligence through research; for example, as shown in Table 1:

[0084] Table 1: Intelligence source authority degree and ranking

[0085]

[0086] The timeliness requirement of the threat intelligence itself is one of the important indexes for judging the value of the threat intelligence. Therefore, according to the time difference between the last updating time t update of the threat intelligence and the time when the user searches the threat intelligence, the validity of the threat intelligence is judged through research. The smaller the value of the time difference is, the stronger the timeliness of the threat intelligence is, and the higher the credibility is. When the value of the time difference is greater than the set time difference threshold V, it represents that the threat intelligence has no utilization value.

[0087] Let the current time be t now , and the last updating time of the threat intelligence be t update , then the time validity feature of the threat intelligence data is

[0088] In this embodiment, the time difference threshold V can be 30, that is, threat intelligence exceeding 30 days is judged as having no utilization value.

[0089] Since the IP address in the threat intelligence is mostly a control terminal used by an APT attacker for downloading attack payloads, accepting control instructions and traffic forwarding, etc. When the IP address in the threat intelligence is a control terminal, the attacker can possibly move horizontally to the adjacent IP address to obtain more control terminals. Therefore, the present application determines that the IP address segment within a certain range can be controlled by the attacker, and thus can be used as a control terminal IP range.

[0090] According to the above analysis, in this embodiment, the IP matching feature T IP is extracted as follows:

[0091] c31-1: According to the threat intelligence library, the IP address segment I possibly controlled by the attacker is calculated;

[0092] Suppose that the integer type corresponding to the request IP address in the jth intelligence R j in the threat intelligence library is r IP , then the IP address segment I possibly controlled by the attacker is {r IP +l m}, wherein l m is an integer sequence of {-m, …, m}; and m is a preset value;

[0093] In step c31-1, the request IP address in the jth intelligence R j in the threat intelligence library is converted into an integer from 0 to 255, then each control terminal IP address converted into an integer is converted into a binary form and the corresponding binary number is obtained, finally the binary number is converted into a long integer, and the integer type corresponding to the request IP address in the jth intelligence R j in the threat intelligence library is obtained, that is, r IP ;

[0094] c31-2: The IP matching feature T IP is calculated;

[0095] If the integer type q i of the request IP address of the original traffic d IP is within the control terminal IP address segment I of the threat intelligence R j , then the IP matching feature corresponding to the request IP address of the original traffic d i is The IP matching feature of the original traffic d i is

[0096] Where j is the number of threats in the threat intelligence database; m takes the value of 10.

[0097] Since APT attackers often use domain names as control terminals for network access—for downloading attack payloads, receiving control commands, and forwarding traffic—they can easily switch servers by changing the domain's DNS resolution address to use different servers providing control services. This invention, through research, has found that APT attackers often use domains with similar names when registering them, such as "morjgs01.top, morjgs02.top", "iw.gamegame.info, iw.gamegames.top", etc. Therefore, this invention calculates the original traffic d... i The similarity between the domain name and the domain name of each APT threat intelligence is used to achieve domain name matching feature T. Domain The extraction method is as follows:

[0098] c32-1: Since most domain names have highly repetitive prefixes and suffixes, the prefix "www" and the suffix "com" are discarded to extract the domain's valid information, i.e., each original traffic d is extracted separately. i The valid string E of the domain name in the threat intelligence database, and the valid string sequence G of the domain name for each APT threat intelligence entry in the threat intelligence database. j ;

[0099] c32-2: Compare the valid domain string E with the valid domain string sequence G in the threat intelligence database. j The strings in the table are compared one by one to calculate the domain name matching feature T. Domain The specific method is as follows:

[0100] For the valid string E of the domain name, and the valid string sequence G of the domain name in the threat intelligence database. j Let g be a string G in the string. len e is the length of string G len Ar is the length of the valid string E in the domain name; g It is of length g len An array of all zeros, used to record matching records of string G; Ar e It is of length e len A zero-based array is used to record matching records of string E; min len This represents the minimum allowed length for a match. If the match length is less than min... len Then skip; max len This indicates that during the matching process, the substring E starting from r... r Compare the substring G of the string starting with s s The longest possible match, substring E rFor a valid substring E, substring G s Given a substring of string G; iterate through substring E one by one. r and G s Given the given character, obtain the maximum length k of consecutive similar strings in the current location context. len By traversing all substrings using the method described above, we obtain the set K of all consecutive similar maximum lengths. len Then the similarity between the two strings is:

[0101]

[0102] Therefore, the domain name matching characteristics of traffic sequences

[0103] D: Construct a sample packet feature library, and compare the request domain name value, request IP address, request URL value, and request parameters of the APT attack samples in the sample packet feature library with each original traffic in the attack sequence set D2 to obtain the packet feature P of sequence set D2. The packet feature P includes the domain name feature P. Domain IP characteristics P IP URL features P URL and request parameter feature P Parameter ;

[0104] Since the downloads and attack payloads used in APT attacks are usually related, and the control modules and communication modules have a certain inheritance relationship, obtaining the message features of sequence set D2 is of great significance for detecting APT attack behavior; in this invention, the specific method for extracting message feature P is as follows:

[0105] D1: Collect APT attack samples to build a sample packet feature library. Extract the network request features of each APT attack sample in the sample packet feature library through static and / or dynamic analysis. The network request features include the request domain name value, request IP address, request URL value, and request parameters.

[0106] D2: Unpack each piece of raw traffic in the attack sequence set D2 to obtain the raw traffic d. i The corresponding request message, from which the request domain name value d is extracted. Domain URL request value d URL Request IP value d IP Request parameter value d Parameter ;

[0107] D3: Extracting Domain Name Features P Domain ;

[0108] Extracting domain name features P DomainThe method and step c32-2 extract domain name matching feature T Domain The idea is consistent, as follows:

[0109] For the original traffic d i Request domain name valid string U, packet feature library domain name valid string sequence H k Some string H, h len Length of string H, u len Length of domain name valid string U; Ar h Is a 0 array of length h len , Used to record the matching record of string H; Ar u Is a 0 array of length u len , Used to record the matching record of string U; min p_len Represent the minimum length allowed by matching, if the matching length is less than min p_len Skip; max p_len Indicates the longest possible match of the substring U r Starting from r and the comparison string starting from s, the substring H s , Substring U r Is a substring of valid string U, and substring H s Is a substring of string H; traverse the characters of substring U r And H s , Get the maximum length k p_len Of continuous similar string in the current positioning; traverse all substrings according to the above method, get all continuous similar maximum length set K p_len ,

[0110] The similarity of two strings is:

[0111]

[0112] Therefore, the domain name matching feature of traffic sequence

[0113] D4: extract IP feature P IP ;

[0114] Let the integer type of the request IP address in the original traffic d i Q IP , The request IP value integer type of the kth APT attack sample in the packet feature library is The similarity of IP address between original traffic d i And the kth APT attack sample is:

[0115] ​

[0116] IP features

[0117] In this embodiment, m is 10;

[0118] D5: extract URL feature P URL ;

[0119] Let the URL request value of the original traffic d i d URL , the URL request value of the kth APT attack sample in the message feature library is The corresponding URL similarity p url is calculated by using the simhash algorithm, and the URL feature P

[0120] The simhash algorithm is a conventional technology in the art, which will not be repeated here.

[0121] D6: extract request parameter feature P Parameter ;

[0122] Let the request parameter value of the original traffic d i d parameter , the request parameter value of the kth APT attack sample in the message feature library is The corresponding request parameter similarity P j is calculated by using the simhash algorithm, and the request parameter feature P

[0123] E: using the network traffic feature set F obtained in step B, the threat intelligence feature T obtained in step C and the message feature P obtained in step D, using a multi-fusion detection model (Multi-LSTM) based on LSTM recurrent neural network for training, and finally obtaining a trained multi-fusion detection model;

[0124] In the present application, the multi-fusion detection model (Multi-LSTM) based on LSTM recurrent neural network is improved on the original LSTM model, and Drop Layer layer is added in Forward Layer layer and Backward Layer layer based on the original four-layer mechanism, and Optimizing Layer layer is added in Backward Layer layer and Output Layer layer; The addition of Drop Layer layer can effectively improve the applicability of the algorithm in the present application and avoid overfitting; The addition of Optimizing Layer layer can realize further optimization of the results of Backward Layer layer.

[0125] The step E includes the following specific steps:

[0126] E1: Input data is D input and for D input Increase the weight W1, where W1 is the source authority T obtained in step C3. confidence The formula for calculating the Forward Layer is as follows:

[0127] I t =σ(W i ·[Η t-1 ,x t ]+B i )

[0128] Where I t σ is the input gate, W is the activation function. i H is the weight matrix of the input gate. t-1 This is the final output of the Multi-LSTM at the previous time step, B. i It is the bias term of the current input cell state, x t For the input data, namely D input The subsequent x t This is the calculation result of the data from the previous layer;

[0129] E2: Increase weight W in the Forward Layer f And weight W2, used to randomly disconnect a certain proportion of network connections in the Drop Layer to avoid overfitting;

[0130] Wherein, weight W f The weights are the previous layer weights, initially an identity matrix, which is continuously adjusted according to the previous time step. The weights W2 are random 0 and 1 matrices.

[0131] E3: Increase weight W in the Backward Layer b W3 and W4; where the weight W b For the subsequent layer weights, W3 and W4 are the domain name matching features T obtained in step C3, respectively. Domain IP matching feature T IP ;

[0132] The formula for calculating the Backward Layer is as follows:

[0133] F t =σ(W b ·[Η t-1 ,x t ]+B b )

[0134] Among them, F t It's the Gate of Oblivion, B b It is the bias term of the current forget gate;

[0135] E4: Increase the weight W5 in the Optimizing Layer, optimize the Backward Layer result, use the least square gradient descent to make the result converge quickly;

[0136] The least square gradient descent calculation formula is as follows:

[0137]

[0138] Wherein f(x) is the least square gradient descent function, x is the input data at the last time, and a is the learning rate. m And is the bias parameter and the weight parameter of the mth group of data.

[0139] E5: Since the sample packet features have high reliability, the weight W6 is increased in the Output Layer; wherein W6 is the domain name feature P Domain , the URL feature P URL , the IP feature P IP and the request parameter feature P Parameter of the request packet obtained in step D;

[0140] The calculation method of the Output Layer is as follows:

[0141] O t = sigma (W O ·[H t-1 , x t ]+B O )

[0142] Wherein O t is the output gate, W o is the weight matrix of the output gate, and B o is the bias term of the output gate.

[0143] In the data set DAPT2020, 80% of the data are randomly extracted as a training set for model training, and the trained multi-fusion detection model is obtained through multiple iterative training to achieve convergence.

[0144] F: Use the trained multi-fusion detection model to detect APT attacks on unknown traffic sequences to determine whether there is an APT attack.

Claims

1.A method for detecting APT attack based on threat intelligence and traffic features, characterized in that, Comprise the following steps in sequence: A: Obtain the original traffic sequence of APT attack as an original attack sequence set D1, and set a label for each original traffic in the original attack sequence set D1 according to the attributes of network request, marked as attack request or normal request; B: Extract the network traffic features of the original traffic in the original attack sequence set D1, and perform data cleaning on the original traffic data to obtain an attack sequence set D2 after data cleaning and a corresponding network traffic feature set F; C: constructing a local threat intelligence library by using an existing APT attack threat intelligence library, and comparing and calculating each original traffic in the attack sequence set D2 through the IP address, domain name, intelligence source authority degree, publishing time and updating time of the threat intelligence in the local threat intelligence library to obtain the threat intelligence feature T of each original traffic, the threat intelligence feature T including the intelligence source authority degree feature T confidence , time validity feature T time , IP matching feature T IP and domain name matching feature T Domain ; D: Construct a sample packet feature library, and compare and calculate the request domain name value, request IP address, request URL value and request parameter of the APT attack sample in the sample packet feature library with each original flow in the attack sequence set D2 to obtain the packet feature P of the sequence set D2. The packet feature P includes the domain name feature P Domain , IP feature P IP , URL feature P URL and request parameter feature P Parameter ; E: Use the network traffic feature set F obtained in step B, the threat intelligence feature T obtained in step C and the packet feature P obtained in step D to train a multi-fusion detection model based on LSTM recurrent neural network, and finally obtain a trained multi-fusion detection model; F: Use the trained multi-fusion detection model to detect APT attack on unknown traffic sequence, and determine whether there is APT attack; In step E, the multi-fusion detection model based on LSTM recurrent neural network increases the Drop Layer layer in the Forward Layer layer and the Backward Layer layer, and increases the Optimizing Layer layer in the Backward Layer layer and the Output Layer layer based on the four-layer mechanism of the LSTM model. 2.The APT attack detection method based on threat intelligence and traffic features according to claim 1, characterized in that, The step C comprises the following specific steps: C1: Collect and organize threat intelligence to construct a local threat intelligence library; C2: unpack each raw traffic in the attack sequence set D2 respectively to obtain raw traffic d i domain name and IP address corresponding to all requests; C3: for each original traffic in the attack sequence set D2, extract the corresponding threat intelligence feature T respectively; the threat intelligence feature T includes intelligence source authority degree feature T confidence , time validity feature T time , IP matching feature T IP , and domain name matching feature T Domain . 3.The APT attack detection method based on threat intelligence and traffic features according to claim 2, characterized in that, In step C3: For the intelligence source authority degree T confidence , different intelligence source authority degrees T confidence are set according to different APT threat intelligence sources. For the time validity feature T time , let the current time be t now , the threat intelligence last update time be t update , then the time validity feature of the threat intelligence data is wherein V is a time difference threshold. 4.The method for APT attack detection based on threat intelligence and traffic features according to claim 2, characterized in that, In the step C3, the IP matching feature T IP The extraction process is as follows: c31-1: According to the threat intelligence library, calculate the IP address segment I that may be used as the control end by the attacker; Set the integer type of the request IP address corresponding to the jth piece of intelligence R in the threat intelligence library as r j IO Then the IP address segment I={r IP +l m} that may be controlled by the attacker, wherein l m is an integer sequence of {-m,…,m}, and m is a preset value​ c31-2: Calculate IP matching feature T IP ; If the request IP address integer type q i of the original flow d IP is within the control end IP address segment I of the threat intelligence R j , the request IP address of the original flow d i corresponds to the IP matching feature of the original flow d i The IP matching feature of the original flow d i wherein j is the intelligence quantity in the threat intelligence library. 5.The APT attack detection method based on threat intelligence and traffic features according to claim 2, characterized in that, In the step C3, the domain name matching feature T Domain The extraction process is as follows: c32-1: extract domain name valid string E in each original flow d i respectively, and domain name valid string sequence G of each APT threat intelligence in the threat intelligence library j ; c32-2: compare the domain name valid string E with the strings in the sequence G of domain name valid strings in the threat intelligence library one by one, and calculate the domain name matching feature T j Domain ; the specific method is as follows:​ For a domain name valid string E, and a sequence of domain name valid strings G in threat intelligence library j , let g len be the length of string G, e len be the length of domain name valid string E; Ar g be a all-0 array with length g len , used to record the matching record of string G; Ar e be a all-0 array with length e len , used to record the matching record of string E; min len represents the minimum length allowed for a match, if the match length is less than min len then skip; max len represents the longest possible match of the substring E r starting from r in the match process, with the substring G s starting from s in the comparison string, the substring E r is a substring of the effective string E, the substring G s is a substring of the string G; the characters of the substrings E r and G s are traversed one by one to obtain the maximum length k len of the continuous similar string in the current positioning; all the continuous similar maximum length sets K len are obtained by traversing all the substrings in the above manner, and the similarity of the two strings is: Domain name matching feature for traffic sequences 6.The APT attack detection method based on threat intelligence and traffic features according to claim 1, characterized in that, The step D comprises the following specific steps: D1: Collect APT attack samples to construct a sample packet feature library, extract network request features of each APT attack sample in the sample packet feature library through static and / or dynamic analysis, and the network request features include request domain name value, request IP address, request URL value and request parameter; D2: respectively unpack each original traffic in the attack sequence set D2 to obtain the original traffic d i The corresponding request message, the request domain name value d is split from the request message Domain , URL request value d URL , request IP value d IP , request parameter value d Parameter ; D3: Extract domain name feature P Domain ; For the original flow d i Request domain name valid string U, the message characteristics library domain name valid string sequence H k Some string H, set h len As the length of the string H, u len As the length of the domain name valid string U; Ar h Is a 0 array with a length of h len , used to record the matching record of string H; Ar u Is a 0 array with a length of u len , used to record the matching record of string U; min p_len Represents the minimum length allowed for matching, if the matching length is less than min p_len Skip; max p_len Indicates the longest possible match of the substring U r Starting from r and the substring H s Starting from s in the comparison string, the substring U r Is a substring of the valid string U, and the substring H s Is a substring of the string H; traverse the characters of the substrings U r And H s One by one, get the maximum length k p_len Of the continuous similar string in the current positioning; traverse all substrings according to the above method to get all the continuous similar maximum length set K p_len , then the similarity of the two strings is: Domain name matching feature for traffic sequences D4: Extract IP features P IP ; Let the original flow d i Request IP address corresponding to the integer type for q IP , the request IP value of the kth APT attack sample in the message feature library is an integer type Then the original flow d i The similarity of the IP address of the kth APT attack sample is : IP features D5: Extract URL feature P UrL ; Let the original flow d i The URL request value of the message feature library is d URL The URL request value of the kth APT attack sample in the message feature library is The corresponding URL similarity p url is calculated by using the simhash algorithm, and the URL feature is obtained D6: Extract request parameter feature P Parameter ; Let the original flow d i The request parameter value of the kth APT attack sample in the message feature library is d parameter The request parameter value of the kth APT attack sample in the message feature library is d The corresponding request parameter similarity P j The request parameter feature is obtained 7.The APT attack detection method based on threat intelligence and traffic features according to claim 1, characterized in that, The step E comprises the following specific steps: E1: input data is D input and D input weight W1 is added, and the weight W1 is the authority T of the information source obtained in step C3 confidence The calculation formula of Forward Layer is as follows: I t = σ(W i · [H t-1 , x t ]+ B i ); where I t is the input gate, σ is the activation function, W i is the weight matrix of the input gate, H t-1 is the final output of the previous time of Multi-LSTM, B i is the bias term of the current input unit state, x t is the input data, i.e. D input , the subsequent x t is the calculation result of the previous layer data; E2: add weight W to Forward Layer f and weight W2; wherein, weight W f is the forward layer weight, initially as an identity matrix, constantly adjusted according to the previous moment, and weight W2 is a random 0, 1 matrix; E3: Adding weight W in Backward Layer b , W3 and W4; wherein, weight W b is the weight of the backward layer, W3 and W4 are the domain name matching feature T Domain and IP matching feature T IP obtained in step C3, respectively. The calculation formula of Backward Layer is as follows F t = σ(W b · [H t-1 , x t ]+ B b ); wherein F t is a forget gate, B b is a bias term for the current forget gate; E4: Increase the weight W5 in the Optimizing Layer to optimize the result of Backward Layer, and use the least square gradient descent to make the result converge quickly; The calculation formula of the least square gradient descent is as follows: Wherein, f(x) is a least square gradient descent function, x is the input data at the last time, and a m With is the bias parameter and the weight parameter of the mth group of data; E5: add weight W6 in Output Layer; W6 is domain name feature P of request message Domain , URL feature P URL , IP feature P IP , and request parameter feature P Parameter ; The calculation method of Output Layer is as follows: O t = σ(W O · [H t-1 , x t ] + B O ); where O t is the output gate, W o is the weight matrix of the output gate, and B o is the bias term of the output gate. 8.The APT attack detection method based on threat intelligence and traffic features according to claim 1, characterized in that: In step B, the network traffic features of each original traffic in the original attack sequence set D1 are extracted using the CICFlowMeter method respectively; Then, the raw traffic data after extracting network traffic features is cleaned to remove raw traffic data containing single values ​​and null values, finally obtaining the cleaned attack sequence set D2 = {d1,…,d...} n } and the corresponding set of network traffic features F = {f1, ..., f n }; where d1 to d n Let f1 and f2 represent the first to the nth original traffic in the cleaned attack sequence set D2, respectively. n These represent the network traffic characteristics corresponding to the first to the nth raw traffic, respectively. 9.The APT attack detection method based on threat intelligence and traffic features of claim 1, wherein: In step A, the DAPT2020 public data set is used as the original attack sequence set D1; the APT attacks in the DAPT2020 public data set are divided into five stages of investigation, foothold, horizontal, stealing and destruction, and cleaning; the five stages of investigation, foothold, horizontal, stealing and destruction are defined as attack requests, and the remaining stages are defined as normal requests.

Images