APT attack discovery method based on feature enhanced community detection

Abstract

The invention belongs to the technical field of network security, and discloses a feature enhanced community detection-based APT attack discovery method, which comprises the following steps of: collecting log data and constructing an origin graph; the source graph is segmented into a plurality of sub-graphs, a feature association vector set and a weight set are generated for each sub-graph, the sub-graphs with feature association vectors and weights are reconstructed, each reconstructed sub-graph comprises a plurality of triads, and the feature association vectors and the weights exist among the triads; performing feature enhanced community detection on each reconstructed sub-graph; and performing rule matching on the communities in each reconstructed sub-graph to obtain attack communities, and according to the timestamp attribute of each reconstructed sub-graph, connecting the attack communities in each reconstructed sub-graph to complete APT attack discovery. According to the method, the origin graph is constructed by using the collected logs, and accurate division of attack communities is performed by performing feature enhanced community detection on the origin graph.

Description

Technical Field

[0001] The present invention belongs to the technical field of network security, and in particular relates to a method for constructing a community detection algorithm based on feature enhancement and performing attack community discovery. Background Art

[0002] APT attacks (Advanced Persistent Threat) refer to persistent and effective attacks launched by certain organizations against specific targets. Such attacks are highly concealed and targeted, and usually use various infected media, supply chains, social engineering and other means to implement advanced, persistent and effective threats and attacks.

[0003] APT attacks are considered a highly dangerous network security threat because they have a long latency period and are difficult to detect by existing security defense systems. They are also difficult to trace even after the attack. APT attacks have become the main security threat faced by all types of networks, so various APT detection methods have emerged one after another. Currently, a very popular approach is to collect host logs and build a provenance graph to reflect the interactions between entities in the system. Logs build a provenance graph through <subject, operation, object> triples, while the data structure type of ordinary graphs can only consider the behavior of neighboring points but cannot consider the correlation of distant points.

[0004] Community detection is also often used to aggregate and classify attack behaviors. It relies on the influence of nodes on neighboring nodes and aggregates communities through modularity optimization. However, such unsupervised methods are prone to cause the correlation and coupling between normal behaviors and attack behaviors. Aggregated communities are prone to aggregate a large number of benign behaviors and cause dependency explosion problems, making attack behaviors in attack communities unclear and increasing false positives. Summary of the invention

[0005] The purpose of the present invention is to provide an attack discovery method based on feature-enhanced community detection, which uses the collected logs to construct an origin graph and accurately divides the attack community by performing feature-enhanced community detection on the origin graph.

[0006] To achieve the above object, the technical solution adopted by the present invention is:

[0007] A method for discovering APT attacks based on feature-enhanced community detection, the method comprising:

[0008] Step 1: Collect log data and build a provenance graph;

[0009] Step 2: Divide the original graph into several subgraphs, generate a feature association vector set and a weight set for each subgraph, and reconstruct the subgraph with the feature association vector and weight. Each reconstructed subgraph contains multiple triplets, and there are feature association vectors and weights between the triplets.

[0010] Step 3: Perform feature-enhanced community detection on each reconstructed subgraph, including:

[0011] Step 3-1: Treat each triple in the subgraph as an independent community;

[0012] Step 3-2: For each triple, the triple is sequentially allocated to the community where the adjacent triple is located, and the modularity change of the subgraph before and after the proposed allocation is calculated. If the modularity change is greater than zero, the non-overlapping part of the triple to be allocated is actually allocated to the community where the adjacent triple with the largest modularity change is located, otherwise the proposed triple is not actually allocated;

[0013] Step 3-3, repeat step 3-2 until the communities to which all triples belong do not change;

[0014] Step 3-4: Take all triples belonging to the same community as a new triple, and update the feature association vectors and weights between the new triples;

[0015] Step 3-5: Repeat steps 3-1 to 3-4 until the modularity of the entire subgraph does not change, and the community division of the subgraph is completed;

[0016] Step 4: Match the rules of the communities in each reconstructed subgraph to obtain the attack community. According to the timestamp attribute of each reconstructed subgraph, connect the attack communities in each reconstructed subgraph to complete the APT attack discovery.

[0017] Several optional methods are also provided below, but they are not intended to be additional limitations on the above-mentioned overall solution, but are merely further supplements or preferences. Under the premise that there are no technical or logical contradictions, each optional method can be combined with the above-mentioned overall solution separately, and multiple optional methods can also be combined.

[0018] Preferably, the step 2 comprises:

[0019] Step 2-1, split the origin graph according to the timestamp attributes of the edges in multiple triples, and after the splitting is completed, multiple subgraphs are obtained, each subgraph contains multiple triples;

[0020] Step 2-2: for the segmented subgraph, extract the event features of each triple in the subgraph, and compare them with the event features of other triples in the subgraph to obtain the feature association vector between the two triples, that is, obtain the feature association vector set of the subgraph;

[0021] Step 2-3: Use the logistic regression algorithm to optimize the weight of the feature association vector to obtain the weight set corresponding to the subgraph. Each element in the weight set is the weight between the two corresponding triples in the subgraph, and the initial self-loop weight of each triple is reset to 0.

[0022] Step 2-4: Use the weight set to weight the feature association vector set, specifically, splice the weight to the tail of its corresponding feature association vector to complete the reconstruction of the subgraph.

[0023] Preferably, the attributes included in the feature association vector are: Subject type, Object type, Operator type, timestamp, S_pid, S_pname, S_domain, S_ip, S_socket, S_type, S_path, O_pid, O_pname, O_domain, O_ip, O_socket, O_type and O_path;

[0024] If the event features extracted from the two triples are recorded as event u and event v respectively, the specific meanings of the attributes contained in the feature association vector are as follows:

[0025] Subject type means: whether the initiator types of event u and event v are the same. If they are the same, the attribute value is 1, otherwise the attribute value is 0;

[0026] Object type means: whether the operation object types of event u and event v are the same. If they are the same, the attribute value is 1, otherwise the attribute value is 0;

[0027] Operator type means: whether the operation behaviors of event u and event v are the same. If they are the same, the attribute value is 1, otherwise the attribute value is 0;

[0028] The meaning of timestamp is: whether the time between event u and event v is less than the time threshold. If so, the attribute value of the timestamp attribute between event u and event v is 1, otherwise the attribute value is 0;

[0029] S_pid means: if the initiator types of event u and event v are both processes, whether their process id numbers are the same, if they are the same, the attribute value is 1, otherwise the attribute value is 0;

[0030] S_pname means: if the initiator types of event u and event v are both processes, whether the process names of the processes are the same, if they are the same, the attribute value is 1, otherwise the attribute value is 0;

[0031] S_domain means: if the initiator types of event u and event v are both sockets, whether the domains to which the sockets belong are the same, if they are the same, the attribute value is 1, otherwise the attribute value is 0;

[0032] S_ip means: if the initiator types of event u and event v are both sockets, whether the IP addresses of the sockets are the same, if they are the same, the attribute value is 1, otherwise the attribute value is 0;

[0033] S_socket means: if the initiator types of event u and event v are both sockets, whether the port numbers of their sockets are the same, if they are the same, the attribute value is 1, otherwise the attribute value is 0;

[0034] S_type means: if the initiator types of event u and event v are both sockets, whether the attachment types of their operations are the same, if they are the same, the attribute value is 1, otherwise the attribute value is 0;

[0035] S_path means: if the initiator types of event u and event v are both files, whether their file paths are the same. If they are the same, the attribute value is 1, otherwise the attribute value is 0;

[0036] O_pid means: if the operation object types of event u and event v are both processes, then whether their process id numbers are the same, if they are the same, the attribute value is 1, otherwise the attribute value is 0;

[0037] O_pname means: if the operation object types of event u and event v are both processes, whether the process names of the processes are the same, if they are the same, the attribute value is 1, otherwise the attribute value is 0;

[0038] O_domain means: if the operation object types of event u and event v are both sockets, whether the domains to which the sockets belong are the same, if they are the same, the attribute value is 1, otherwise the attribute value is 0;

[0039] O_ip means: if the operation object types of event u and event v are both sockets, whether the IP addresses of the sockets are the same, if they are the same, the attribute value is 1, otherwise the attribute value is 0;

[0040] O_socket means: if the operation object types of event u and event v are both sockets, whether the port numbers of the sockets are the same, if they are the same, the attribute value is 1, otherwise the attribute value is 0;

[0041] O_type means: if the operation object type of event u and event v are both sockets, whether the attachment types of their operations are the same, if they are the same, the attribute value is 1, otherwise the attribute value is 0;

[0042] O_path means: if the operation object types of event u and event v are both files, whether their file paths are the same. If they are the same, the attribute value is 1, otherwise the attribute value is 0.

[0043] Preferably, in step 2-3, weight optimization is performed on the feature association vector by a logistic regression algorithm to obtain a weight set corresponding to the subgraph, including:

[0044] A weight vector is obtained through the logistic regression algorithm Using feature association vector and weight vector The dot product is used as the weight between every two triplets. The weight calculation formula is as follows:

[0045]

[0046] Among them, ω is the weight, α i is the weight vector The value of the i-th position, e i is the value of the i-th bit of the feature association vector, and k is the total number of bits of the weight vector or feature association vector.

[0047] Preferably, in step 3-2, the change in modularity of the subgraph before and after the proposed allocation is calculated as follows:

[0048]

[0049] Where Δ is the change in modularity of the subgraph, m is the total number of feature association vectors in the subgraph, i,j∈[1,m], k i represents the sum of the weights of all feature association vectors related to triple i and twice the self-loop weight of triple i, k j A represents the sum of the weights of all feature association vectors related to triple j and twice the self-loop weight of triple j. ij Represents the weight corresponding to the feature association vector between triple i and triple j, a binary parameter c i represents the community to which triple i belongs, c j represents the community to which triple j belongs, c i = = c j It means that triple i and triple j belong to the same community.

[0050] Preferably, in step 3-4, updating the feature association vectors and weights between the new triples includes:

[0051] Step 3-4-1: Perform bitwise AND operation on the feature association vectors of all triples in each community to form an enhanced self-loop feature association vector for each community.

[0052] Step 3-4-2, eliminate all feature association vectors between different triples in the same community;

[0053] Step 3-4-3: For feature association vectors between triples in different communities, if there are q feature association vectors between two communities A and B, calculate a feature association vector As the unique characteristic association vector between communities A and B, the calculation formula is: Among them, β i Represents the feature association vector The value of the i-th position, Δ i It represents the difference between the number of vectors with 1 and 0 at the i-th position in the q feature association vectors;

[0054] Step 3-4-4: Take the weight vector obtained by the logistic regression algorithm in step 2-3 According to the feature association vector and the weight vector The weight is calculated as the enhanced weight between communities A and B, according to the self-loop feature association vector and the weight vector The calculated weight is used as the self-loop weight after community enhancement, and the unique feature association vector between communities A and B is The enhanced weights between communities A and B are used as the feature association vectors and weights between the new triples, and the self-loop feature association vectors of the communities are And the enhanced self-loop weight is used as the self-loop feature association vector and self-loop weight of the new triplet.

[0055] The attack discovery method based on feature-enhanced community detection provided by the present invention has the following beneficial effects compared with the prior art: 1. The Louvain algorithm with excellent performance in processing large networks is improved by enhancing the features so that it can be applied to triples, detect overlapping communities, optimize the detection results and retain its powerful performance in processing large networks. 2. A method for constructing feature association vectors and calculating weights between triples is provided to fully explore the semantic associations between events and has strong scalability. 3. By using community detection to divide attack communities, multiple sensitive operations can be matched within a community, reducing the difficulty of matching and improving the matching accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 A flowchart of an attack discovery method based on feature-enhanced community detection of the present invention;

[0057] Figure 2 This is an example diagram of a triplet of the present invention;

[0058] Figure 3 An example diagram of the invention for dividing the origin graph into subgraphs;

[0059] Figure 4 An example diagram of community detection with feature enhancement for the present invention. DETAILED DESCRIPTION

[0060] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0061] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art of the present invention. The terms used in the specification of the present invention herein are only for the purpose of describing specific embodiments and are not intended to limit the present invention.

[0062] like Figure 1 As shown, this embodiment provides an APT attack detection method based on feature-enhanced community detection, comprising the following steps:

[0063] Step 1. Collect log data and build a provenance graph: Use a published log collection tool (such as the log collection tool disclosed in Patent No. 2022110610516) to collect system logs, and build a provenance graph based on the collected log data. The nodes represent process, file or socket entities, and the edges represent the operation relationship between two nodes, such as read, write, download, send, etc. The timestamp is used as the attribute of the edge to indicate the time when the event occurred. The provenance graph is composed of multiple triples and stored. Figure 2 As shown, a triple (subject, operator, object) represents an event, where subject represents the main node that initiates the operation (initiator), object represents the object node that is operated (operation object), and operator represents the operation behavior, that is, the edge, subject and object may overlap.

[0064] Step 2: Divide the original graph into subgraphs, generate a feature association vector set and a weight set for each subgraph, and reconstruct the subgraph with the feature association vector and weight. Each reconstructed subgraph contains multiple triplets, and there are feature association vectors and weights between the triplets.

[0065] Step 2-1, split the provenance graph according to the timestamp attribute of the edges in multiple triples. If there are multiple triplets, the time interval between their occurrence does not exceed 10ms, that is, Δtimestamp<10ms, then these triplets will be divided into a subgraph. After the division is completed, multiple subgraphs are obtained, each subgraph contains multiple triplets, and each subgraph is used as a unit for subsequent operations. Figure 3 An example of segmenting a provenance graph based on timestamps.

[0066] Step 2-2: For the segmented subgraph, extract the event features of each triplet in the subgraph, and compare them with the event features of other triples in the subgraph to obtain the feature association vector between the two triplets, that is, obtain the feature association vector set of the subgraph.

[0067] This embodiment extracts event features from each triple and compares them with event features of other triples to form a feature association vector The values ​​of all attributes in this vector (eq) are 0 or 1, and the feature association vector formed is in the form of the following Table 1 (including the continuation table):

[0068] Table 1 Feature association vector attributes

[0069] property Subject type Object type Operator type timestamp eq 0 or 1 0 or 1 0 or 1 0 or 1

[0070] Table 1 Feature association vector attributes

[0071] property S_pid S_pname S_domain S_ip S_socket S_type S_path eq 0 or 1 0 or 1 0 or 1 0 or 1 0 or 1 0 or 1 0 or 1

[0072] Table 1 Feature association vector attributes

[0073] property O_pid O_pname O_domain O_ip O_socket O_type O_path eq 0 or 1 0 or 1 0 or 1 0 or 1 0 or 1 0 or 1 0 or 1

[0074] If the event features extracted from the two triples are recorded as event u and event v respectively, the specific meanings of the attributes contained in the feature association vector are as follows:

[0075] Subject type means: whether the initiator types of event y and event v are the same, such as both are processes or both are network sockets. If they are the same, eq=1, otherwise eq=0;

[0076] Object type means: whether the operation object types of event u and event v are the same, such as both are processes or both are files. If they are the same, eq=1, otherwise eq=0;

[0077] Operator means: whether the operations performed by event u and event v are the same, such as both are read or both are write. If they are the same, eq = 1, otherwise eq = 0;

[0078] The meaning of timestamp is: whether the time when event u and event v occur is close. If u.timestamp-v.timestamp<5ms, then the timestamp feature attribute between events u and v = 1, where 5ms is the time interval between events. It can be changed according to actual needs, but be careful not to exceed the time interval used to split the origin graph in step 2-1.

[0079] S_pid (O_pid) means: if the initiator (operation object) type of event u and event v are both processes, then whether their process id numbers are the same, if they are the same, eq = 1, otherwise eq = 0;

[0080] S_pname (O_pname) means: if the initiator (operation object) type of event u and event v are both processes, whether the process names of the processes are the same, if they are the same, eq=1, otherwise eq=0;

[0081] S_domain (O_domain) means: if the initiator (operation object) type of event u and event v are both sockets, whether the domains to which the sockets belong are the same, if they are the same, eq = 1, otherwise eq = 0;

[0082] S_ip (O_ip) means: if the initiator (operation object) type of event u and event v are both sockets, whether the IP addresses of the sockets are the same, if they are the same, eq = 1, otherwise eq = 0;

[0083] S_socket (O_socket) means: if the initiator (operation object) type of event u and event v are both sockets, whether the port numbers of their sockets are the same, if they are the same, eq = 1, otherwise eq = 0;

[0084] S_type (O_type) means: if the initiator (operation object) type of event u and event v are both sockets, whether the attachment types of their operations are the same, if they are the same, eq = 1, otherwise eq = 0;

[0085] S_path (O_path) means: if the initiator (operation object) type of event u and event v are both files, whether their file paths are the same, if they are the same, eq=1, otherwise eq=0;

[0086] like Figure 2 As shown in , there are two events in the provenance graph, namely event e1 and event e2. Assuming that the time interval between the two events is less than 10ms, there is a feature association vector between e1 and e2. The value of each item in the feature association vector represents the following meaning:

[0087] The initiator (Subject) type of event e1 and event e2 is the same, both are processes, and they are the same process (Subject type, S_pid, S_pname=1);

[0088] The operation (Operator) types of event e1 and event e2 are different, event e1 is downloading, and event e2 is uploading (Operator type=0);

[0089] The operation objects (Object) of event e1 and event e2 are of the same type, both are sockets, and their domains, IP addresses, and port numbers are the same (Object type, O_domain, O_ip, O_socket=1);

[0090] The download type of event e1 (e.g., executable file) is different from the upload type of event e2 (e.g., PDF file) (O_type=0);

[0091] The time between event e1 and event e2 is less than 10ms (timestamp=1); then the feature association vector between event e1 and event e2 is 110111000000011100.

[0092] After generating the feature association vector between each triple, each subgraph will obtain a feature vector set, each element of which is the feature association vector between two triples in the subgraph.

[0093] Step 2-3: Use the logistic regression algorithm to optimize the weight of the feature association vector. Since there are complex relationships between events, and there are a large number of connections between attack events and benign events, there will be a dependency explosion problem between events. A weight vector is found through the logistic regression algorithm. Use the feature association vector and the vector to perform dot multiplication as the weight between every two triplets, and calculate the weight formula:

[0094]

[0095] Among them, α i is the weight vector The value of the i-th position, e i is the feature association vector The value of the i-th position, weight ω is the weight vector Associated with feature vector The dot product result of .

[0096] After weight optimization of the feature association vector, each subgraph will obtain a weight set, in which each element is the weight between two triples in the subgraph, and the initial self-loop weight of each triple is reset to 0.

[0097] Step 2-4: Use the weight set to weight the feature vector set, splice each calculated weight to the end of its corresponding feature association vector, and perform weighted reconstruction of the subgraph. Each triple in the reconstructed subgraph has a feature association vector and weight.

[0098] Step 3: Perform feature-enhanced community detection on each reconstructed subgraph. Use the Louvain algorithm, which can efficiently process large networks, to perform community detection and improve it so that it can perform feature enhancement in each iteration and make it suitable for triple division. Specifically, it includes: enabling the Louvain algorithm originally used for general graphs to be used for triples that have overlaps, and the detected communities can also have overlapping parts, and perform feature enhancement after each iteration to obtain a better division. The specific steps are:

[0099] Step 3-1: Treat each triple in the subgraph as an independent community.

[0100] Step 3-2: For each triple, the triple is sequentially allocated to the community where the adjacent triple is located, and the modularity change of the subgraph before and after the proposed allocation is calculated. If the modularity change is greater than zero, the non-overlapping part of the triple to be allocated is actually allocated to the community where the adjacent triple with the largest modularity change is located, otherwise the actual allocation operation is not performed on the triple to be allocated. The modularity change calculation formula is as follows:

[0101]

[0102] Where Δ is the change in modularity of the subgraph, m is the total number of feature association vectors in the subgraph, i,j∈[1,m], k i represents the sum of the weights of all feature association vectors related to triple i and twice the self-loop weight of triple i, k j represents the sum of the weights corresponding to all feature association vectors related to triple j and twice the self-loop weight of triple j. When the self-loop weight of the triple is 0, k i represents the sum of the weights corresponding to all feature association vectors related to triple i, k j A represents the sum of the weights corresponding to all feature association vectors related to triple j. ij Represents the weight corresponding to the feature association vector between triple i and triple j, a binary parameter c i represents the community to which triple i belongs, c j represents the community to which triple j belongs, c i = = c j It means that triple i and triple j belong to the same community.

[0103] Step 3-3: Repeat step 3-2 until the communities to which all triples belong do not change.

[0104] Step 3-4: All triples belonging to the same community are taken as a new triple (that is, all triples in the same community are compressed into a new "triplet"), and the feature association vectors and weights between the new triples are updated. The update method is as follows:

[0105] Step 3-4-1: Perform bitwise AND operation on the feature association vectors of all triples in each community to form an enhanced self-loop feature association vector for each community. Among them, the feature association vectors in all communities are used to enhance the features of the community self-loop edges through bitwise AND operations. If the values ​​of the same position of all feature association vectors in a community are 1, the attribute corresponding to the position is regarded as an important dependent attribute for dividing these triples into the same community. Then all the feature association vectors in a community are bitwise ANDed to obtain a new feature association vector as the self-loop feature association vector of the community. This attribute is enhanced in the vector so that all important dependent attribute values ​​in the vector are 1, which further strengthens the characteristics of the community and can help obtain a better division.

[0106] It should be noted that the self-loop feature association vector and self-loop weight refer to the feature association vector and weight of the community (or triplet) itself, while those described only as feature association vector and weight refer to the feature association vector and weight between two communities (or between two triples).

[0107] Step 3-4-2: Eliminate all feature association vectors between different triplets in the same community.

[0108] Step 3-4-3: For the feature association vectors existing between different communities, assuming that there are q feature association vectors between two communities A and B, calculate a feature association vector As the unique characteristic association vector between communities A and B, the calculation formula is: Among them, β i Represents the feature association vector The value of the i-th position, Δ i It represents the difference between the number of vectors with the value of 1 and 0 at the i-th position in the q feature association vectors; according to the feature association vector The weight is calculated as the enhanced weight between communities A and B. The feature association vector between communities is used to enhance the features between communities through the above formula. If the number of vectors with a value of 0 and a value of 1 at the same position of all feature association vectors between two communities is significantly different (the difference is greater than half of the total number of vectors), the attribute corresponding to the position is regarded as an important dependent attribute of the triplets of the two communities not being classified in the same community. Then a new feature association vector is calculated according to the above formula. This property is strengthened in the vector so that all important dependent attribute values ​​in the vector are 1, which further strengthens the characteristics between communities and can help obtain a better division.

[0109] Step 3-4-4: Weight vector calculated based on step 2-3 The self-loop feature correlation vector calculated in steps 3-4-1 and 3-4-3 and feature association vector The weight calculation formula in step 2-3 is used to calculate the community's enhanced self-loop weight and the enhanced weight between communities. The next iteration will directly use the community's self-loop and the enhanced weight between communities to participate in the division.

[0110] Step 3-5: Repeat steps 3-1 to 3-4 until the modularity of the entire subgraph no longer changes. At this time, events related to attack features have been divided into a community.

[0111] Figure 4 Schematic diagram of community detection enhancement features, where w in P4 is the feature association vector corresponding to w1, w2, w4, w5, w7, and w9 in P3, obtained through the calculation formula of the feature association vector between communities, w1 in P4 is the feature association vector corresponding to w3, w6, and w8 in P3, and w2 is the feature association vector corresponding to w3, w6, and w8 in P3. 10 The corresponding feature association vector is bitwise ANDed (i.e. w 10 itself).

[0112] The communities formed after community aggregation are divided according to the degree of attack characteristics shown by their operations. The community with very prominent attack characteristics is a community, the community with a small number of attack characteristics is a community (this part is usually a false alarm), the community with almost no attack characteristics is a community, and so on. There may be overlapping parts between each community, and the overlapping parts are generally some processes that are needed in both benign and malicious operations, such as cmd, powershell, etc.

[0113] Step 4: Perform rule matching on the communities in each segmented subgraph to find the attack community, and connect the attack communities in each subgraph according to the timestamp attribute of each subgraph.

[0114] Step 4-1: Match the rules of the community and specify the rules according to the characteristics of the attack behavior in the underlying log, as shown in Table 2:

[0115] Table 2 Rules table

[0116]

[0117]

[0118] It should be noted that the technologies shown in the table are only part of them. In actual implementation, they can be added as needed.

[0119] Step 4-2, rule matching: After the communities are divided, check the subject and operator parts in each community, take the subjects of all triples in each community to form a subject set, take the operators of all triples to form an operator set, and match each subject in the subject set with the subject of the specific operation in Table 2, and match each operator in the operator set with the log representation in Table 2. If the match is successful, it is an attack triple, and the community with the number of attack triples greater than the attack threshold is regarded as an attack community. For example, if the number of attack triples in a community is greater than one-fifth of the total number of triples in the community, the community is determined to be an attack community.

[0120] If a community has a triplet with subject as command line process and operator as netshadvfirewall set allprofiles state off, it means that a firewall-off operation was performed in the file operation, which is likely to be malicious behavior. If this community establishes a connection to download a file and executes it after turning off the firewall (subject is socket, operator is download), the behavior of the entire community is superimposed. If the more rules matched in the table, the more likely it is that this community is an attack community.

[0121] This has several advantages over using the origin graph directly. The first is that most of the logs are benign, with only a few malicious ones. Figure 1Matching one by one is not efficient. After the detection, the area will be directly divided into communities. The rule matching is not aimless, but has a defined range (that is, operations in the same community), avoiding needle-in-a-haystack matching, but a directional matching. Secondly, communities with very prominent attack features are very easy to match. Therefore, once a community is found to have a large number of operations matched, other communities in this subgraph can no longer be matched, thus reducing the number of matching operations.

[0122] The technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0123] The above-mentioned embodiments only express several implementation modes of the present invention, and the description thereof is relatively specific and detailed, but it cannot be understood as limiting the scope of the invention. It should be pointed out that, for a person of ordinary skill in the art, several modifications and improvements can be made without departing from the concept of the present invention, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the attached claims.

Claims

1. A method for discovering APT attacks based on feature-enhanced community detection, characterized in that: The APT attack detection method based on feature-enhanced community detection includes: Step 1: Collect log data and build a provenance graph; Step 2: Divide the original graph into several subgraphs, generate a feature association vector set and a weight set for each subgraph, and reconstruct the subgraph with the feature association vector and weight. Each reconstructed subgraph contains multiple triplets, and there are feature association vectors and weights between the triplets. Step 3: Perform feature-enhanced community detection on each reconstructed subgraph, including: Step 3-1: Treat each triple in the subgraph as an independent community; Step 3-2: For each triple, the triple is sequentially allocated to the community where the adjacent triple is located, and the modularity change of the subgraph before and after the proposed allocation is calculated. If the modularity change is greater than zero, the non-overlapping part of the triple to be allocated is actually allocated to the community where the adjacent triple with the largest modularity change is located, otherwise the proposed triple is not actually allocated; Step 3-3, repeat step 3-2 until the communities to which all triples belong do not change; Step 3-4: Take all triples belonging to the same community as a new triple, and update the feature association vectors and weights between the new triples; Step 3-5: Repeat steps 3-1 to 3-4 until the modularity of the entire subgraph does not change, and the community division of the subgraph is completed; Step 4: Match the rules of the communities in each reconstructed subgraph to obtain the attack community. According to the timestamp attribute of each reconstructed subgraph, connect the attack communities in each reconstructed subgraph to complete the APT attack discovery.

2. The APT attack detection method based on feature-enhanced community detection according to claim 1 is characterized in that: The step 2 comprises: Step 2-1, split the origin graph according to the timestamp attributes of the edges in multiple triples, and after the splitting is completed, multiple subgraphs are obtained, each subgraph contains multiple triples; Step 2-2: for the segmented subgraph, extract the event features of each triple in the subgraph, and compare them with the event features of other triples in the subgraph to obtain the feature association vector between the two triples, that is, obtain the feature association vector set of the subgraph; Step 2-3: Use the logistic regression algorithm to optimize the weight of the feature association vector to obtain the weight set corresponding to the subgraph. Each element in the weight set is the weight between the two corresponding triples in the subgraph, and the initial self-loop weight of each triple is reset to 0. Step 2-4: Use the weight set to weight the feature association vector set, specifically, splice the weight to the tail of its corresponding feature association vector to complete the reconstruction of the subgraph.

3. The APT attack detection method based on feature-enhanced community detection according to claim 2 is characterized in that: The attributes included in the feature association vector are: Subject type, Object type, Operator type, timestamp, S_pid, S_pname, S_domain, S_ip, S_socket, S_type, S_path, O_pid, O_pname, O_domain, O_ip, O_socket, O_type and O_path; If the event features extracted from the two triples are recorded as event u and event v respectively, the specific meanings of the attributes contained in the feature association vector are as follows: Subject type means: whether the initiator types of event u and event v are the same. If they are the same, the attribute value is 1, otherwise the attribute value is 0; Object type means: whether the operation object types of event u and event v are the same. If they are the same, the attribute value is 1, otherwise the attribute value is 0; Operator type means: whether the operation behaviors of event u and event v are the same. If they are the same, the attribute value is 1, otherwise the attribute value is 0; The meaning of timestamp is: whether the time between event u and event v is less than the time threshold. If so, the attribute value of the timestamp attribute between event u and event v is 1, otherwise the attribute value is 0; S_pid means: if the initiator types of event u and event v are both processes, whether their process id numbers are the same, if they are the same, the attribute value is 1, otherwise the attribute value is 0; S_pname means: if the initiator types of event u and event v are both processes, whether the process names of the processes are the same, if they are the same, the attribute value is 1, otherwise the attribute value is 0; S_domain means: if the initiator types of event u and event v are both sockets, whether the domains to which the sockets belong are the same, if they are the same, the attribute value is 1, otherwise the attribute value is 0; S_ip means: if the initiator types of event u and event v are both sockets, whether the IP addresses of the sockets are the same, if they are the same, the attribute value is 1, otherwise the attribute value is 0; S_socket means: if the initiator types of event u and event v are both sockets, whether the port numbers of their sockets are the same, if they are the same, the attribute value is 1, otherwise the attribute value is 0; S_type means: if the initiator types of event u and event v are both sockets, whether the attachment types of their operations are the same, if they are the same, the attribute value is 1, otherwise the attribute value is 0; S_path means: if the initiator types of event u and event v are both files, whether their file paths are the same. If they are the same, the attribute value is 1, otherwise the attribute value is 0; O_pid means: if the operation object types of event u and event v are both processes, then whether their process id numbers are the same, if they are the same, the attribute value is 1, otherwise the attribute value is 0; O_pname means: if the operation object types of event u and event v are both processes, whether the process names of the processes are the same, if they are the same, the attribute value is 1, otherwise the attribute value is 0; O_domain means: if the operation object types of event u and event v are both sockets, whether the domains to which the sockets belong are the same, if they are the same, the attribute value is 1, otherwise the attribute value is 0; O_ip means: if the operation object types of event u and event v are both sockets, whether the IP addresses of the sockets are the same, if they are the same, the attribute value is 1, otherwise the attribute value is 0; O_socket means: if the operation object types of event u and event v are both sockets, whether the port numbers of the sockets are the same, if they are the same, the attribute value is 1, otherwise the attribute value is 0; O_type means: if the operation object type of event u and event v are both sockets, whether the attachment types of their operations are the same, if they are the same, the attribute value is 1, otherwise the attribute value is 0; O_path means: if the operation object types of event u and event v are both files, whether their file paths are the same. If they are the same, the attribute value is 1, otherwise the attribute value is 0.

4. The APT attack detection method based on feature-enhanced community detection according to claim 2 is characterized in that: In step 2-3, the feature association vector is weight optimized by a logistic regression algorithm to obtain a weight set corresponding to the subgraph, including: A weight vector is obtained through the logistic regression algorithm Using feature association vector and weight vector The dot product is used as the weight between every two triplets. The weight calculation formula is as follows: Among them, ω is the weight, α i is the weight vector The value of the i-th position, e i is the value of the i-th bit of the feature association vector, and k is the total number of bits of the weight vector or feature association vector.

5. The APT attack detection method based on feature-enhanced community detection according to claim 2 is characterized in that: In step 3-2, the change in the modularity of the subgraph before and after the proposed allocation is calculated as follows: Where Δ is the change in modularity of the subgraph, m is the total number of feature association vectors in the subgraph, i,j∈[1,m], k i represents the sum of the weights of all feature association vectors related to triple i and twice the self-loop weight of triple i, k j A represents the sum of the weights of all feature association vectors related to triple j and twice the self-loop weight of triple j. ij Represents the weight corresponding to the feature association vector between triple i and triple j, a binary parameter c i represents the community to which triple i belongs, c j represents the community to which triple j belongs, c i = = c j It means that triple i and triple j belong to the same community.

6. The APT attack detection method based on feature-enhanced community detection according to claim 2 is characterized in that: In step 3-4, the feature association vectors and weights between the new triplets are updated, including: Step 3-4-1: Perform bitwise AND operation on the feature association vectors of all triples in each community to form an enhanced self-loop feature association vector for each community. Step 3-4-2, eliminate all feature association vectors between different triples in the same community; Step 3-4-3: For feature association vectors between triples in different communities, if there are q feature association vectors between two communities A and B, calculate a feature association vector As the unique characteristic association vector between communities A and B, the calculation formula is: Among them, β i Represents the feature association vector The value of the i-th position, Δ i It represents the difference between the number of vectors with 1 and 0 at the i-th position in the q feature association vectors; Step 3-4-4: Take the weight vector obtained by the logistic regression algorithm in step 2-3 According to the feature association vector and the weight vector The weight is calculated as the enhanced weight between communities A and B, according to the self-loop feature association vector and the weight vector The calculated weight is used as the self-loop weight after community enhancement, and the unique feature association vector between communities A and B is The enhanced weights between communities A and B are used as the feature association vectors and weights between the new triples, and the self-loop feature association vectors of the communities are And the enhanced self-loop weight is used as the self-loop feature association vector and self-loop weight of the new triplet.

Images