Attack detection method based on multi-source heterogeneous data feature extraction

Abstract

The invention discloses an attack detection method based on multi-source heterogeneous data feature extraction, which belongs to the technical field of network security, and comprises a data acquisition and cyclic shift expansion processing method, a data packet iterative compression and data abstract generation method and format unification processing of multi-source heterogeneous data. And carrying out category division on the semantics of the data, and comparing the data with known alarm information in an alarm information base so as to realize attack detection. According to the attack detection method based on multi-source heterogeneous data feature extraction, the problems that multi-source data are heterogeneous in format, diversified in semantics and poor in relevance application can be solved, and after data semantics are subjected to category division, the data semantics are compared with known alarm information in an alarm information base, so that attack detection is achieved.

Description

Technical Field

[0001] The present invention belongs to the technical field of network security, and in particular, relates to an attack detection method based on multi-source heterogeneous data feature extraction. Background Art

[0002] With the large-scale development of new technologies and new applications, the network has brought many conveniences to people's work and life, but the security risks and security threats faced by cyberspace have also gradually increased. Security vulnerabilities in network applications and information systems are difficult to discover and repair in a timely manner, and network attacks are becoming increasingly rampant. At the same time, network attacks are gradually developing in the direction of dynamic, advanced, hidden, multi-step, collaborative, distributed, and directional. Traditional network security protection methods, such as firewalls, intrusion detection, antivirus, vulnerability scanning and other systems are obviously insufficient in dealing with dynamically customized, highly hidden and directional network attacks. Traditional static detection methods have long been ineffective. Even in a network environment where security equipment is deployed, it is difficult to resist the occurrence of various attack activities. The complex network environment brings many problems and challenges to attack detection:

[0003] (1) Since attacks are highly concealed, they often disguise themselves as normal operations of legitimate users, making it more difficult to detect attack activities from raw log data. In addition, during the detection process, various information systems generate massive amounts of detection data every day, and the attacker's operation behavior is also hidden in the audit logs and network traffic data recorded by the system itself. These basic data sources are scattered, semantically diverse, heterogeneous in format, and interrelated. There is no perfect solution to coordinate their processing, and there are challenges such as data explosion, semantic gap, and concept drift.

[0004] (2) Multi-source heterogeneous detection data is complex and diverse. There are various attack detection methods for different data sources. The combination relationship between research contents is not clear enough and there is a lack of systematic discussion in the research. In addition, the existing detection methods cannot quickly and effectively screen and correlate various types of data related to network attacks, and the problem of data correlation has not been well solved.

[0005] (3) The detection results of existing attack detection methods and intrusion detection systems contain a large number of false alarms and redundant alarms. The real alarms are drowned in a large number of useless alarms. Various scattered detection methods do not play their due role. Various types of alarm data have problems such as rapid growth in quantity, low alarm quality, high false alarm rate, high duplication rate, and heterogeneous format. This makes it more difficult to discover truly threatening attack events through correlation analysis of multi-source alarm data. Summary of the invention

[0006] The purpose of the present invention is to solve the problems of heterogeneous formats, diverse semantics and weak correlation application of multi-source data, and to propose an attack detection method based on feature extraction of multi-source heterogeneous data. The multi-source heterogeneous data is formatted in a unified manner, and the semantics of the data are classified and compared with the known alarm information in the alarm information library to achieve attack detection.

[0007] The objective of the present invention is achieved by the following technical solution: an attack detection method based on multi-source heterogeneous data feature extraction, comprising:

[0008] Step S1, collecting data from routers, system logs, server logs, application logs, databases, firewalls, and vulnerability libraries, and dividing the data into non-semantic security data NS, security semantic data SS, and other data MS according to the content of the data, and using a cyclic shift method to expand each non-semantic security data NS and each security semantic data SS to generate n data with a length of 132 words;

[0009] The classification criteria in step S1 are as follows: if the collected data contains detailed descriptions of the network communication process and behavioral operation information caused by attack activities, it is defined as non-semantic security data NS; if the collected data is the alarm information of the original log record data, it is defined as security semantic data SS; if the collected data is daily traffic data and there is no data related to system security, it is defined as other data MS.

[0010] In step S1, the process of using the cyclic shift method to expand each non-semantic security data NS and each security semantic data SS to generate n data with a length of 132 words is as follows:

[0011] The non-semantic security data NS and the security semantic data SS are divided into n groups with a size of 512 bits per group and stored in data group A and a In the formula, a=1,2,···,n, for the nth group of data with less than 512 bits, a "1" and then k "0"s need to be filled after the last bit of the data, where the value of k satisfies:

[0012] (m+1+k)mod512=448 (1)

[0013] Where m is the total length of the original data, mod is the remainder operation, and 64 bits of original data length information are added after the last "0" filled, that is, the total length of the original data is represented by a 64-bit binary number;

[0014] Group each data after filling A a , a=1,2,···,n, are divided into 16 words, namely A a,0 ,Aa,1 ,···,A a,15 According to the calculation method of formula (2), each data group is expanded to generate 52 words A a,b , a=1,2,···,n,b=16,17,···,67, a total of 68 characters A are generated a,0 ,A a,1 ,···,A a,67 ;

[0015]

[0016] Among them, ← represents the left-hand assignment operator, represents a 32-bit XOR operator, <<< represents a 32-bit circular left shift operator, the number of bits of the circular left shift is determined by the number after <<<, P1() is a permutation function, and its implementation function is shown in formula (3), X is the input value in the permutation function P1(),

[0017]

[0018] A a,0 ,A a,1 ,···,A a,67 The data in W are stored in the corresponding order. a,h1 In the equation, a=1,2,···,n,h1=0,1,···,67,W a,0 ,W a,1 ,···,W a,63 The value is stored in W a,u In the formula (4), u = 0, 1, ..., 63, and then W is used according to the method shown in formula (4). a,u , generate 64 words W a,h1′ , where h1′=0,1,···,63,

[0019]

[0020] Each non-semantic security data NS and each security semantic data SS are expanded by using the cyclic shift method through equations (2), (3) and (4) to generate n data with a length of 132 words.

[0021] Step S2: compress each non-semantic security data NS and each security semantic data SS based on the 132 characters generated by the expansion to obtain a summary value of each non-semantic security data NS and a summary value of each security semantic data SS;

[0022] The compression steps in step S2 are as follows:

[0023] ① Use a random number generator to randomly generate a 64-bit random hexadecimal number, and assign this 64-bit random hexadecimal number to V0;

[0024] ② Apply for 8 word registers X1, X2, ···, X8 with a length of 32 bits to store V0. The storage rule is to store the 1st to 8th bits of V0 in word register X1, store the 9th to 16th bits of V0 in word register X2, store the 17th to 24th bits of V0 in word register X3, store the 25th to 32nd bits of V0 in word register X4, store the 33rd to 40th bits of V0 in word register X5, store the 41st to 48th bits of V0 in word register X6, store the 49th to 56th bits of V0 in word register X7, and store the 57th to 64th bits of V0 in word register X8;

[0025] ③ Obtain V2 by the calculation method of formula (5), where CF() is the compression function, u=0, 1, ···, 63, h1′=0, 1, ···, 63, and the compression process is shown in formulas (6) to (21).

[0026] V2=CF(V0,W a,u ,W a，h1′ ) (5)

[0027] ④ Obtain the value of the intermediate variable UU1 according to the method shown in formula (6), where + represents modulo 2 32 The addition operator, its carry rule is to add the carry digit to 2 32 Modulo, so that the result of the operation is always a 32-bit binary number:

[0028] UU1←((X1<<<12)+X5+(T u <<<umod32))<<<7 (6)

[0029] Among them, T u is the algorithm constant, u=0, 1, ···, 63, T u The initial value of is determined by u, as shown in formula (7), T uThe value range consists of 50 groups of 8-bit hexadecimal data as shown below: 79CC4519,F3988A32,E7311465,CE6228CB,9CC45197,3988A32F,7311465E,E6228CBC,CC451979,988A32F3,311465E7,6228CBCE,C451979C,88A32F39, 11465E73,228CBCE6,9D8A7A87,3B14F50F,7629EA1E,EC53D43C,D8A7A879,B14F50F3,629 EA1E7,C53D43CE,8A7A879D,14F50F3B,29EA1E76,53D43CEC,A7A879D8,4F50F3B1,9EA1E7 62,3D43CEC5,7A879D8A,F50F3B14,EA1E7629,D43CEC53,A879D8A7,50F3B14F,A1E7629E, 43CEC53D,879D8A7A,0F3B14F5,1E7629EA,3CEC53D4,79D8A7A8,F3B14F50,E7629EA1,CEC 53D43,9D8A7A87,3B14F50F,7629EA1E,EC53D43C,D8A7A879,B14F50F3,629EA1E7,C53D43 CE,8A7A879D,14F50F3B,29EA1E76,53D43CEC,A7A879D8,4F50F3B1,9EA1E762,3D43CEC5,

[0030]

[0031] Wherein, A, B, C, D, E and F are hexadecimal numbers in a computer, which are not described in detail here for the sake of prior art;

[0032] According to the method shown in formula (8), the value of the intermediate variable UU2 is obtained:

[0033]

[0034] The value of the intermediate variable TT1 is obtained according to the method shown in formula (9), where FF u () is a Boolean function as shown in formula (10), where ∩ is the symbol of the bitwise AND operator, and ∪ is the symbol of the bitwise OR operator;

[0035] TT1←FF u (X1,X2,X3)+X4+UU2+W a,h1' (9)

[0036]

[0037] The value of the intermediate variable TT2 is obtained according to the method shown in formula (11), where GG u () is a Boolean function as shown in formula (12), where ∩ is the symbol of the bitwise AND operator, ∪ is the symbol of the bitwise OR operator, It is the symbol of the negation operator, i.e. bitwise inversion;

[0038] TT2←GG u (X5,X6,X7)+X8+UU1+W a,u (11)

[0039]

[0040] ⑤ Update the stored data in word registers X1, X2, ···, X8 according to the method of formula (13) to formula (20), where P0() is a permutation function, and the implementation function is shown in formula (21):

[0041] X4←X3 (13)

[0042] X3←X2<<<9 (14)

[0043] X2←X1 (15)

[0044] X1←TT1 (16)

[0045] X8←X7 (17)

[0046] X7←X6<<<19 (18)

[0047] X6←X5 (19)

[0048] X5←P0(TT2) (20)

[0049]

[0050] ⑥ Repeat the calculation process from step ④ to step ⑤ 63 times to obtain the new storage data in word registers X1, X2, ···, X8, store the data in V1 in ascending order of subscripts, and calculate the compression value according to the method shown in formula (22),

[0051]

[0052] ⑦ Repeat the above steps ② to ⑥ for n-1 times, and then obtain the final 256-bit compressed value V2, which is the summary value M.

[0053] Step S3: Unify the formats of the collected non-semantic security data NS and security semantic data SS:

[0054] ① Apply for a storage array C with a length of 336 bits for each non-semantic security data NS and security semantic data SS received i , i = 1, 2, ···, z, z is the sum of the number of non-semantic security data NS and security semantic data SS received by the event processing layer; storage array C i Bits 0-31 store the source IP address of the data information, and the storage array C i The 32-63 bits store the destination IP address of the data information, and the storage array C i The 64th bit stores the information type, 0 indicates security semantic data SS, 1 indicates non-semantic security data NS, and the storage array C i Bits 65-320 of the storage array store the summary value M, and the storage array C i The 321-336 bits of the security semantic data SS store the attack type H. The corresponding code of the attack type H is shown in formula (23):

[0055]

[0056] ② In order to solve the problem that multi-source data are interrelated and have data redundancy, the storage array C is calculated by formulas (24), (25) and (26): d and C e The similarity between d ,C e ), d=1,2,···,z,e=1,2,···,z, and determine the relationship between the storage arrays according to formula (27);

[0057] F=C d ∩C e (twenty four)

[0058] G=C d ∪C e (25)

[0059] J(C d ,C e )=|F| / |G|×100% (26)

[0060] Among them, |F| is the storage array C d and C e Take the number of elements in the intersection, |G| is the storage array C d and C e The number of elements in the union, ∩ is the intersection, which means the storage array C d and storage array C eThe corresponding positions of the digits are all 1, ∪ is the union, indicating that the storage array C d and storage array C e The corresponding position of at least one element is 1, through the similarity J(C d ,C e ) can get two storage arrays C d and C e The correlation R between

[0061]

[0062] When the information is judged to be redundant according to the relevance R, the storage array C storing the information is released. e , apply for a two-dimensional associative array S i,j , i=1,2,···,z,j=1,2,···,8×z,used to store and store array C i There is an associated storage array C o The group number and association degree, where o = 1, 2, ···, z, are stored in the row of a two-dimensional array and in the storage array C i There is an associated storage array C o The group number o is stored in the column of the two-dimensional array and the storage array C i There is an associated storage array C o The binary representation of the correlation degree Q is shown in formula (28):

[0063] Q=R×100 (28)

[0064] Filter out redundant information from the storage array C i , i=1,2,···,z is analyzed and detected to determine whether there is an attack behavior, and the storage array C of the attack behavior is i The data information of the 321st to 336th bits is extracted, and the attack type is determined according to formula (23), and compared with the known alarm information in the alarm information database. If it is a known attack type, the corresponding interception measures are taken to respond to the alarm information, and the two-dimensional associative array S is generated. i,jSearch for other storage arrays that are associated with the storage array, and perform attack detection on them in order of similarity R from high to low; if it is an unknown alarm information, add it to the alarm information library, define the attack type, determine the possible attack path and attack method of the alarm information according to the highly associated information and the low associated information, and take corresponding measures to intercept and process the alarm information; finally, for the attack fragment or attack step of the unknown destination IP address, find its highly associated information and the low associated information, analyze the destination IP address of these information, and then infer the destination IP address of the attack fragment or attack step of the unknown destination IP address, so as to construct the attack scenario.

[0065] Through the above-mentioned design scheme, the present invention can bring the following beneficial effects: the attack detection method based on multi-source heterogeneous data feature extraction proposed in the present invention unifies the format of multi-source heterogeneous data, and classifies the semantics of the data into categories, and then compares them with the known alarm information in the alarm information library to achieve attack detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of the present invention. The exemplary embodiments of the present invention and their descriptions are used to understand the present invention and do not constitute improper limitations of the present invention. In the drawings:

[0067] Figure 1 The figure is a flow chart of the attack detection method based on feature extraction of multi-source heterogeneous data. DETAILED DESCRIPTION

[0068] In order to make the purpose, features and advantages of the present invention more obvious and easy to understand, the technical scheme in the present invention is clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the present invention is not limited to the following embodiments, and the specific implementation method can be determined according to the technical scheme of the present invention and the actual situation. In order to avoid confusing the essence of the present invention, the known methods, processes and flows are not described in detail.

[0069] Reference Figure 1 ,The attack detection method based on multi-source heterogeneous data feature extraction is carried out in the following steps:

[0070] 1) Data acquisition and cyclic shift expansion processing method;

[0071] By collecting data from routers, system logs, server logs, application logs, databases, firewalls, and vulnerability libraries, and dividing them into non-semantic security data NS, security semantic data SS, and other data MS according to the content of the collected data, the classification criteria are as follows: if the collected data contains detailed descriptions of the network communication process and behavioral operation information brought about by attack activities, it is defined as non-semantic security data NS; if the collected data is the alarm information of the original log record data, it is defined as security semantic data SS; if the collected data is daily traffic data and there is no data related to system security, it is defined as other data MS;

[0072] The non-semantic security data NS and the security semantic data SS are divided into n groups with a size of 512 bits per group and stored in data group A and a In the formula, a=1,2,···,n, for the nth group of data with less than 512 bits, a "1" and then k "0"s need to be filled after the last bit of the data, where the value of k satisfies:

[0073] (m+1+k)mod512=448 (1)

[0074] Where m is the total length of the original data, mod is the remainder operation, and 64 bits of original data length information are added after the last "0" filled, that is, the total length of the original data is represented by a 64-bit binary number;

[0075] Group each data after filling A a , a=1,2,···,n, are divided into 16 words, namely A a,0 ,A a,1 ,···,A a,15 According to the calculation method of formula (2), each data group is expanded to generate 52 words A a,b , a=1,2,···,n,b=16,17,···,67, a total of 68 characters A are generated a,0 ,A a,1 ,···,A a,67 ;

[0076]

[0077] Among them, ← represents the left-hand assignment operator, represents a 32-bit XOR operator, <<< represents a 32-bit circular left shift operator, the number of bits of circular left shift is determined by the number after <<<, P1() is a permutation function, and its implementation function is shown in formula (3), X is the input value in the permutation function P1(),

[0078]

[0079] A a,0 ,A a,1 ,···,A a,67 The data in W are stored in the corresponding order. a,h1 In the equation, a=1,2,···,n,h1=0,1,···,67,W a,0 ,W a,1 ,···,W a,63 The value is stored in W a,u In the formula (4), u = 0, 1, ..., 63. It should be noted that, because 132 words of data are needed in the end, and 68 words are already available, only 64 words need to be expanded. Then, according to the method shown in formula (4), use W a,u , generate 64 words W a,h1′ , where h1′=0,1,···,63,

[0080]

[0081] Each non-semantic security data NS and each security semantic data SS are expanded by using the cyclic shift method through equations (2), (3) and (4) to generate n data with a length of 132 words.

[0082] Step S2: compress each non-semantic security data NS and each security semantic data SS based on the 132 characters generated by the expansion to obtain a summary value of each non-semantic security data NS and a summary value of each security semantic data SS;

[0083] Further, the compression step in step S2 is as follows:

[0084] ① Use a random number generator to randomly generate a 64-bit random hexadecimal number, and assign this 64-bit random hexadecimal number to V0;

[0085] ② Apply for 8 word registers X1, X2, ···, X8 with a length of 32 bits to store V0. The storage rule is to store the 1st to 8th bits of V0 in word register X1, store the 9th to 16th bits of V0 in word register X2, store the 17th to 24th bits of V0 in word register X3, store the 25th to 32nd bits of V0 in word register X4, store the 33rd to 40th bits of V0 in word register X5, store the 41st to 48th bits of V0 in word register X6, store the 49th to 56th bits of V0 in word register X7, and store the 57th to 64th bits of V0 in word register X8;

[0086] ③ Obtain V2 by the calculation method of formula (5), where CF() is the compression function, u=0, 1, ···, 63, h1′=0, 1, ···, 63, and the compression process is shown in formulas (6) to (21).

[0087] V2=CF(V0,W a,u ,W a，h1′ ) (5)

[0088] ④ Obtain the value of the intermediate variable UU1 according to the method shown in formula (6), where + represents modulo 2 32 The addition operator, its carry rule is to add the carry digit to 2 32 Modulo, so that the result of the operation is always a 32-bit binary number:

[0089] UU1←((X1<<<12)+X5+(T u << <umod32))<<<7 (6)

[0090] Among them, T u is the algorithm constant, u=0, 1, ···, 63, T u The initial value of is determined by u, as shown in formula (7), T u The value range consists of 50 groups of 8-bit hexadecimal data as shown below: 79CC4519,F3988A32,E7311465,CE6228CB,9CC45197,3988A32F,7311465E,E6228CBC,CC451979,988A32F3,311465E7,6228CBCE,C451979C,88A32F39, 11465E73,228CBCE6,9D8A7A87,3B14F50F,7629EA1E,EC53D43C,D8A7A879,B14F50F3,629 EA1E7,C53D43CE,8A7A879D,14F50F3B,29EA1E76,53D43CEC,A7A879D8,4F50F3B1,9EA1E7 62,3D43CEC5,7A879D8A,F50F3B14,EA1E7629,D43CEC53,A879D8A7,50F3B14F,A1E7629E, 43CEC53D,879D8A7A,0F3B14F5,1E7629EA,3CEC53D4,79D8A7A8,F3B14F50,E7629EA1,CEC 53D43,9D8A7A87,3B14F50F,7629EA1E,EC53D43C,D8A7A879,B14F50F3,629EA1E7,C53D43 CE,8A7A879D,14F50F3B,29EA1E76,53D43CEC,A7A879D8,4F50F3B1,9EA1E762,3D43CEC5,

[0091]

[0092] Wherein, A, B, C, D, E and F are hexadecimal numbers in a computer, which are not described in detail here for the sake of prior art;

[0093] According to the method shown in formula (8), the value of the intermediate variable UU2 is obtained:

[0094]

[0095] The value of the intermediate variable TT1 is obtained according to the method shown in formula (9), where FF u () is a Boolean function as shown in formula (10), where ∩ is the symbol of the bitwise AND operator, ∪ is the symbol of the bitwise OR operator,

[0096] TT1←FF u (X1,X2,X3)+X4+UU2+W a,h1' (9)

[0097]

[0098] The value of the intermediate variable TT2 is obtained according to the method shown in formula (11), where GG u () is a Boolean function as shown in formula (12), where ∩ is the symbol of the bitwise AND operator, ∪ is the symbol of the bitwise OR operator, is the symbol of the negation operator, which is bitwise inversion.

[0099] TT2←GG u (X5,X6,X7)+X8+UU1+W a,u (11)

[0100]

[0101] Step 5: Update the stored data in word registers X1, X2, . . . , X8 according to the method of formula (13) to formula (20), where P0() is a permutation function, and the implementation function is shown in formula (21):

[0102] X4←X3 (13)

[0103] X3←X2<<<9 (14)

[0104] X2←X1 (15)

[0105] X1←TT1 (16)

[0106] X8←X7 (17)

[0107] X7←X6<<<19 (18)

[0108] X6←X5 (19)

[0109] X5←P0(TT2) (20)

[0110]

[0111] Step 6: Repeat the calculation process of step 4 to step 5 63 times to obtain the new storage data in word registers X1, X2, . . . , X8, store the data in V1 in ascending order of subscripts, and calculate the compression value according to the method shown in formula (22),

[0112]

[0113] Step 7: Repeat the above steps 2 to 6 for n-1 times, and then obtain the final 256-bit compressed value V2, which is the digest value M;

[0114] 3) Unified processing method for multi-source heterogeneous data formats

[0115] In order to solve the problem of heterogeneous formats of multi-source data, the collected non-semantic security data NS and security semantic data SS are processed in a unified format:

[0116] The first step is to apply for a storage array C with a length of 336 bits for each non-semantic security data NS and security semantic data SS received. i , i = 1, 2, ···, z, z is the sum of the number of non-semantic security data NS and security semantic data SS received by the event processing layer; storage array C i Bits 0-31 of the data store the source IP address of the data information, and the storage array C i Bits 32-63 of the storage array C store the destination IP address of the data information. i The 64th bit stores the information type, 0 indicates security semantic data SS, 1 indicates non-semantic security data NS, and the storage array C i Bits 65-320 of the storage array store the summary value M, and the storage array C i The 321-336 bits of the security semantic data SS store the attack type H. The code corresponding to the attack type H is shown in formula (23);

[0117]

[0118] In the second step, in order to solve the problem that multi-source data are interrelated and have data redundancy, the storage array C is calculated using formulas (24), (25) and (26). d and C e The similarity between d ,C e), d=1,2,···,z,e=1,2,···,z,and determine the relationship between the storage arrays according to formula (27);

[0119] F=C d ∩C e (twenty four)

[0120] G=C d ∪C e (25)

[0121] J(C d ,C e )=|F| / |G|×100% (26)

[0122] Among them, |F| is the storage array C d and C e Take the number of elements in the intersection, |G| is the storage array C d and C e The number of elements in the union, ∩ is the intersection, which means the storage array C d and storage array C e The corresponding positions of the digits are all 1, ∪ is the union, indicating that the storage array C d and storage array C e The corresponding position of at least one element is 1, through the similarity J(C d ,C e ) can get two storage arrays C d and C e The correlation R between

[0123]

[0124] When the information is judged to be redundant according to the relevance R, the storage array C storing the information is released. e , apply for a two-dimensional associative array S i,j , i=1,2,···,z,j=1,2,···,8×z,used to store and store array C i There is an associated storage array C o The group number and association degree, where o = 1, 2, ···, z, are stored in a two-dimensional array row and the storage array C i There is an associated storage array C o The group number o is stored in the columns of the two-dimensional array and the storage array C i There is an associated storage array C o The binary representation of the correlation degree Q is shown in formula (28):

[0125] Q=R×100 (28)

[0126] Filter out redundant information from the storage array C i , i=1,2,···,z is analyzed and detected to determine whether there is an attack behavior, and the storage array C of the attack behavior is i The data information of the 321st to 336th bits is extracted, and the attack type is determined according to formula (23), and compared with the known alarm information in the alarm information database. If it is a known attack type, the corresponding interception measures are taken to respond to the alarm information, and the two-dimensional associative array S i,j Search for other storage arrays that are associated with the storage array, and perform attack detection on them in order of similarity R from high to low; if it is an unknown alarm information, add it to the alarm information library, define the attack type, determine the possible attack path and attack method of the alarm information according to the highly associated information and the low associated information, and take corresponding measures to intercept and process the alarm information; finally, for the attack fragment or attack step of the unknown destination IP address, find its highly associated information and the low associated information, analyze the destination IP address of these information, and then infer the destination IP address of the attack fragment or attack step of the unknown destination IP address, so as to construct the attack scenario.

[0127] The software program involved in the present invention is compiled based on mobile communication, network and computer processing technologies, which are technologies familiar to those skilled in the art.

[0128] The specific embodiments of the present invention are merely examples for clear explanation and are not limitations on the implementation methods. For ordinary technicians in the field, other different forms of changes or modifications can be made based on the above description. There is no need to list all the implementation methods here, and the obvious changes or modifications derived therefrom are still within the scope of protection of the present invention.

Claims

1. An attack detection method based on feature extraction of multi-source heterogeneous data, characterized in that: include: Step S1, collecting data from routers, system logs, server logs, application logs, databases, firewalls, and vulnerability libraries, and dividing the data into non-semantic security data NS, security semantic data SS, and other data MS according to the content of the data, and using a cyclic shift method to expand each non-semantic security data NS and each security semantic data SS to generate n data with a length of 132 words; Step S2: compress each non-semantic security data NS and each security semantic data SS based on the 132 characters generated by the expansion to obtain a summary value of each non-semantic security data NS and a summary value of each security semantic data SS; Step S3: Unify the formats of the collected non-semantic security data NS and security semantic data SS: ① Apply for a storage array C with a length of 336 bits for each non-semantic security data NS and security semantic data SS received i , i = 1, 2, ···, z, z is the sum of the number of non-semantic security data NS and security semantic data SS received by the event processing layer; storage array C i Bits 0-31 store the source IP address of the data information, and the storage array C i The 32-63 bits store the destination IP address of the data information, and the storage array C i The 64th bit stores the information type, 0 indicates security semantic data SS, 1 indicates non-semantic security data NS, and the storage array C i Bits 65-320 of the storage array store the summary value M, and the storage array C i The 321-336 bits of the security semantic data SS store the attack type H. The corresponding code of the attack type H is shown in formula (23): ② In order to solve the problem that multi-source data are interrelated and have data redundancy, the storage array C is calculated by formulas (24), (25) and (26): d and C e The similarity between d ,C e ), d=1,2,···,z,e=1,2,···,z, and determine the relationship between the storage arrays according to formula (27); F=C d ∩C e (24) G=C d ∪C e (25) J(C d ,C e )=|F| / |G|×100% (26) Among them, |F| is the storage array C d and C e Take the number of elements in the intersection, |G| is the storage array C d and C e The number of elements in the union, ∩ is the intersection, which means the storage array C d and storage array C e The corresponding positions of the digits are all 1, ∪ is the union, indicating that the storage array C d and storage array C e The corresponding position of at least one element is 1, through the similarity J(C d ,C e ) can get two storage arrays C d and C e The correlation R between When the information is judged to be redundant according to the relevance R, the storage array C storing the information is released. e , apply for a two-dimensional associative array S i,j , i=1,2,···,z,j=1,2,···,8×z,used to store and store array C i There is an associated storage array C o The group number and association degree, where o = 1, 2, ···, z, is stored in a two-dimensional array row and stored in array C i There is an associated storage array C o The group number o is stored in the column of the two-dimensional array and the storage array C i There is an associated storage array C o The binary representation of the correlation degree Q is shown in formula (28): Q=R×100 (28) Filter out redundant information from the storage array C i , i=1,2,···,z is analyzed and detected to determine whether there is an attack behavior, and the storage array C of the attack behavior is i The data information of the 321st to 336th bits is extracted, and the attack type is determined according to formula (23), and compared with the known alarm information in the alarm information database. If it is a known attack type, the corresponding interception measures are taken to respond to the alarm information, and the two-dimensional associative array S is generated. i,j Search for other storage arrays that are associated with the storage array, and perform attack detection on them in order of similarity R from high to low; if it is an unknown alarm information, add it to the alarm information library, define the attack type, determine the possible attack path and attack method of the alarm information according to the highly associated information and the low associated information, and take corresponding measures to intercept and process the alarm information; finally, for the attack fragment or attack step of the unknown destination IP address, find its highly associated information and the low associated information, analyze the destination IP address of these information, and then infer the destination IP address of the attack fragment or attack step of the unknown destination IP address, so as to construct the attack scenario.

2. The attack detection method based on multi-source heterogeneous data feature extraction according to claim 1 is characterized in that: The classification criteria in step S1 are as follows: if the collected data contains detailed descriptions of the network communication process and behavioral operation information caused by attack activities, it is defined as non-semantic security data NS; if the collected data is the alarm information of the original log record data, it is defined as security semantic data SS; if the collected data is daily traffic data and there is no data related to system security, it is defined as other data MS.

3. The attack detection method based on multi-source heterogeneous data feature extraction according to claim 2 is characterized in that: In step S1, the process of using the cyclic shift method to expand each non-semantic security data NS and each security semantic data SS to generate n data with a length of 132 words is as follows: The non-semantic security data NS and the security semantic data SS are divided into n groups with a size of 512 bits per group and stored in data group A and a In the formula, a=1,2,···,n, for the nth group of data with less than 512 bits, a "1" and then k "0" need to be filled after the last bit of the data, where the value of k satisfies: (m+1+k)mod512=448 (1) Where m is the total length of the original data, mod is the remainder operation, and 64 bits of original data length information are added after the last "0" filled, that is, the total length of the original data is represented by a 64-bit binary number; Group each data after filling A a , a=1,2,···,n, are divided into 16 words, namely A a,0 ,A a,1 ,···,A a,15 According to the calculation method of formula (2), each data group is expanded to generate 52 words A a,b , a=1,2,···,n,b=16,17,···,67, a total of 68 characters A are generated a,0 ,A a,1 ,···,A a,67 ; Among them, ← represents the left-hand assignment operator, represents a 32-bit XOR operator, <<< represents a 32-bit circular left shift operator, the number of bits of circular left shift is determined by the number after <<<, P1() is a permutation function, and its implementation function is shown in formula (3), X is the input value in the permutation function P1(), A a,0 ,A a,1 ,···,A a,67 The data in W are stored in the corresponding order. a,h1 In the equation, a=1,2,···,n,h1=0,1,···,67,W a,0 ,W a,1 ,···,W a,63 The value is stored in W a,u In the formula (4), u = 0, 1, ..., 63, and then W is used according to the method shown in formula (4). a,u , generate 64 words W a,h1′ , where h1′=0,1,···,63, Each non-semantic security data NS and each security semantic data SS are expanded by using the cyclic shift method through equations (2), (3) and (4) to generate n data with a length of 132 words.

4. The attack detection method based on multi-source heterogeneous data feature extraction according to claim 3 is characterized in that: The compression steps in step S2 are as follows: ① Use a random number generator to randomly generate a 64-bit random hexadecimal number, and assign this 64-bit random hexadecimal number to V0; ② Apply for 8 word registers X1, X2, ···, X8 with a length of 32 bits to store V0. The storage rule is to store the 1st to 8th bits of V0 in word register X1, store the 9th to 16th bits of V0 in word register X2, store the 17th to 24th bits of V0 in word register X3, store the 25th to 32nd bits of V0 in word register X4, store the 33rd to 40th bits of V0 in word register X5, store the 41st to 48th bits of V0 in word register X6, store the 49th to 56th bits of V0 in word register X7, and store the 57th to 64th bits of V0 in word register X8; ③ Obtain V2 by the calculation method of formula (5), where CF() is the compression function, u=0, 1, ···, 63, h1′=0, 1, ···, 63, and the compression process is shown in formulas (6) to (21). <h2 style=";text-align:left;direction:ltr">V2 = CF(V0,W)<h2 style=";text-align:left;direction:ltr"> a,u <h2 style=";text-align:left;direction:ltr"> ,W<h2 style=";text-align:left;direction:ltr"> a，h1′ <h2 style=";text-align:left;direction:ltr"> ) (5) ④ Obtain the value of the intermediate variable UU1 according to the method shown in formula (6), where + represents modulo 2 32 The addition operator, its carry rule is to add the carry digit to 2 32 Modulo, so that the result of the operation is always a 32-bit binary number: UU1←((X1<<<12)+X5+(T u <<<umod32))<<<7 (6) Among them, T u is the algorithm constant, u=0, 1, ···, 63, T u The initial value of is determined by u, as shown in formula (7), T u The value range consists of 50 groups of 8-bit hexadecimal data as shown below: 79CC4519,F3988A32,E7311465,CE6228CB,9CC45197,3988A32F,7311465E,E6228CBC,CC451979,988A32F3,311465E7,6228CBCE,C451979C,88A32F39, 11465E73,228CBCE6,9D8A7A87,3B14F50F,7629EA1E,EC53D43C,D8A7A879,B14F50F3,629 EA1E7,C53D43CE,8A7A879D,14F50F3B,29EA1E76,53D43CEC,A7A879D8,4F50F3B1,9EA1E7 62,3D43CEC5,7A879D8A,F50F3B14,EA1E7629,D43CEC53,A879D8A7,50F3B14F,A1E7629E, 43CEC53D,879D8A7A,0F3B14F5,1E7629EA,3CEC53D4,79D8A7A8,F3B14F50,E7629EA1,CEC 53D43,9D8A7A87,3B14F50F,7629EA1E,EC53D43C,D8A7A879,B14F50F3,629EA1E7,C53D43 CE,8A7A879D,14F50F3B,29EA1E76,53D43CEC,A7A879D8,4F50F3B1,9EA1E762,3D43CEC5, Wherein, A, B, C, D, E and F are hexadecimal numbers in a computer, which are not described in detail here for the prior art; The value of the intermediate variable UU2 is obtained according to the method shown in formula (8): The value of the intermediate variable TT1 is obtained according to the method shown in formula (9), where FF u () is a Boolean function as shown in formula (10), where ∩ is the symbol of the bitwise AND operator, and ∪ is the symbol of the bitwise OR operator; TT1←FF u (X1,X2,X3)+X4+UU2+W a,h1' (9) The value of the intermediate variable TT2 is obtained according to the method shown in formula (11), where GG u () is a Boolean function as shown in formula (12), where ∩ is the symbol of the bitwise AND operator, ∪ is the symbol of the bitwise OR operator, It is the symbol of the negation operator, i.e. bitwise inversion; TT2←GG u (X5,X6,X7)+X8+UU1+W a,u (11) ⑤ Update the stored data in word registers X1, X2, ···, X8 according to the method of formula (13) to formula (20), where P0() is a permutation function, and the implementation function is shown in formula (21): X4←X3 (13) X3←X2<<<9 (14) X2←X1 (15) X1←TT1 (16) X8←X7 (17) X7←X6<<<19 (18) X6←X5 (19) X5←P0(TT2) (20) ⑥ Repeat the calculation process from step ④ to step ⑤ 63 times to obtain the new storage data in word registers X1, X2, ···, X8, store the data in V1 in ascending order of subscripts, and calculate the compression value according to the method shown in formula (22), ⑦ Repeat the above steps ② to ⑥ for n-1 times, and then obtain the final 256-bit compressed value V2, which is the summary value M.

Images