Threat hunting method and device of network data, storage medium and computer equipment

Abstract

The invention discloses a threat hunting method and device for network data, a storage medium and computer equipment, and the method comprises the steps: obtaining a first initial clue and first hunting demand information in response to a clue extension task execution instruction, and obtaining a first hunting request information through a target search algorithm according to the first hunting demand information; starting to search from a first starting node indicated by the first starting clue in the preset graph structure, and counting the number of current search objects in the search; if the first hunting demand information is greater than the first preset number threshold value, classifying the current search object based on a preset classification rule, allocating a processing resource for each classification, and searching in a preset graph structure through a target search algorithm by taking the current search object under the corresponding classification as a starting point through the processing resources according to the first hunting demand information, continuously counting the number of the current search objects in the search and continuously judging until a preset ending condition is met; and obtaining a threat hunting result according to the search object meeting the first hunting demand information.

Description

Technical Field

[0001] The present application relates to the field of network security technology, and in particular to a method and apparatus, storage medium, and computer equipment for threat hunting of network data. Background Art

[0002] With the rapid development of information technology, the network environment is becoming increasingly complex, and the scale of network data is growing explosively, which has brought unprecedented challenges to network security. At present, the means of network attacks are constantly being updated, and the concealment of attack behaviors has increased. Traditional defense systems based on signatures or rules can no longer effectively deal with all potential threats. Therefore, threat hunting, as an active security protection strategy, has gradually become an important research direction in the field of network security. Threat hunting aims to proactively discover and identify potential security threats through in-depth analysis of network data. The key lies in how to efficiently and accurately mine abnormal or malicious behaviors from massive data. Currently, threat hunting is gaining more and more attention.

[0003] Existing threat hunting methods mostly rely on manual analysis or simple automated tools, which often have problems such as low efficiency and high false positive rate. Especially in the clue expansion task, due to the huge amount of network data, if the relevant clues of a clue are directly found through manual analysis in the massive network data and the clue is expanded, then this process often takes a lot of time and has poor accuracy; if the relevant clues of a clue are directly found from the massive network data through simple automated tools, then this process often faces system resource exhaustion and performance bottlenecks, which seriously affects the search efficiency and may even cause a jam. Summary of the invention

[0004] In view of this, the present application provides a threat hunting method and device, storage medium, and computer equipment for network data, which constructs a preset graph structure in advance based on massive network data, performs threat hunting through the graph structure, optimizes the query performance of multi-level and multi-relationship network data, and can return query results faster than relational databases; combines graph structure search, classification processing, parallel computing and other technologies to achieve efficient and intelligent identification and tracking of potential threats in network data, which can not only process large-scale network data, but also flexibly adjust the search strategy according to the specific number of search objects, thereby helping to improve the accuracy and efficiency of threat hunting.

[0005] According to one aspect of the present application, a threat hunting method for network data is provided, comprising:

[0006] In response to a clue expansion task execution instruction, a first starting clue and first hunting demand information are obtained, and according to the first hunting demand information, a target search algorithm is used to start searching from a first starting node indicated by the first starting clue in a preset graph structure, and during the search process, the number of current search objects is counted, and the size relationship between the number of current search objects and a first preset number threshold is determined, wherein the preset graph structure includes a plurality of nodes, and associated edges between different nodes, the nodes are constructed based on network data, the associated edges are used to indicate an associated relationship between network data, and the current search object includes a node object and / or an associated edge object;

[0007] If the number of current search objects is greater than a first preset number threshold, the current search objects are classified based on a preset classification rule, a processing resource is allocated to each classification, and the current search objects under the corresponding classification are used as a starting point through the processing resource, and the search is continued in the preset graph structure through a target search algorithm according to the first hunting demand information, and the number of the corresponding current search objects is continued to be counted and judgment is continued during the search process until the preset end condition is met;

[0008] When all the processing resources meet the preset end condition, the threat hunting result corresponding to the first starting clue is obtained according to the search object that meets the first hunting requirement information.

[0009] According to another aspect of the present application, a threat hunting device for network data is provided, comprising:

[0010] An information acquisition module, for obtaining a first starting clue and first hunting demand information in response to a clue expansion task execution instruction, and starting a search from a first starting node indicated by the first starting clue in a preset graph structure through a target search algorithm according to the first hunting demand information, and counting the number of current search objects during the search process, and determining the size relationship between the number of current search objects and a first preset number threshold, wherein the preset graph structure includes a plurality of nodes, and associated edges between different nodes, the nodes are constructed based on network data, the associated edges are used to indicate an associated relationship between network data, and the current search object includes a node object and / or an associated edge object;

[0011] A search module, configured to classify the current search objects based on a preset classification rule if the number of the current search objects is greater than a first preset number threshold, allocate a processing resource to each classification, and continue searching in the preset graph structure through a target search algorithm using the processing resource and the current search objects under the corresponding classification as a starting point according to the first hunting demand information, and continue to count the number of the corresponding current search objects and continue to make judgments during the search process until the preset end condition is met;

[0012] A result acquisition module is used to obtain a threat hunting result corresponding to the first starting clue according to a search object that meets the first hunting requirement information when each of the processing resources meets a preset end condition.

[0013] According to another aspect of the present application, a storage medium is provided, on which a computer program is stored, and when the program is executed by a processor, the threat hunting method for network data described above is implemented.

[0014] According to another aspect of the present application, a computer device is provided, including a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, wherein the processor implements the above-mentioned threat hunting method for network data when executing the program.

[0015] By means of the above technical solution, the present application provides a threat hunting method and device for network data, a storage medium, and a computer device. First, when a clue extension task execution instruction is received, the clue extension task execution instruction can be responded to, and a first starting clue and a first hunting requirement information can be obtained. Then, according to the first starting clue, the first starting node is located in the preset graph structure, and starting from the node, according to the first hunting requirement information, a target search algorithm is used to search, and nodes and associated edges that meet the first hunting requirement information are found in the preset graph structure. During the search process, the number of node objects and / or associated edge objects currently searched can also be counted, and the number of current search objects can be compared with the first preset number threshold. If it is found through comparison that the number of current search objects exceeds the first preset number threshold, the current search object can be classified according to the preset classification rule. After that, a processing resource is allocated to each classification. Next, each processing resource takes the current search object under its corresponding classification as the starting point, and according to the first hunting requirement information, continues to use the target search algorithm to search in the preset graph structure, and continues to count and judge. The above search process is carried out until the preset end condition is met. When all processing resources meet the preset end conditions, the threat hunting results corresponding to the first starting clue can be generated according to the search object that meets the first hunting demand information. The embodiment of the present application pre-builds a preset graph structure based on massive network data, performs threat hunting through a graph structure, optimizes the query performance of multi-level and multi-relational network data, and can return query results faster than relational databases; combined with graph structure search, classification processing, parallel computing and other technologies, it realizes efficient and intelligent identification and tracking of potential threats in network data, which can not only process large-scale network data, but also flexibly adjust the search strategy according to the specific number of search objects, thereby helping to improve the accuracy and efficiency of threat hunting.

[0016] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0018] Figure 1 A schematic diagram of a process flow of a threat hunting method for network data provided by an embodiment of the present application is shown;

[0019] Figure 2A schematic diagram of the structure of a threat hunting device for network data provided by an embodiment of the present application is shown;

[0020] Figure 3 A schematic diagram of the device structure of a computer device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0021] The present application will be described in detail below with reference to the accompanying drawings and in combination with embodiments. It should be noted that the embodiments and features in the embodiments of the present application can be combined with each other without conflict.

[0022] In this embodiment, a threat hunting method for network data is provided. Figure 1 As shown, the method includes:

[0023] Step 101, in response to a clue expansion task execution instruction, obtain a first starting clue and a first hunting demand information, and according to the first hunting demand information, start searching from the first starting node indicated by the first starting clue in a preset graph structure through a target search algorithm, and count the number of current search objects during the search process, and determine the size relationship between the number of current search objects and a first preset number threshold, wherein the preset graph structure includes multiple nodes and associated edges between different nodes, the nodes are constructed based on network data, the associated edges are used to indicate the association relationship between network data, and the current search object includes node objects and / or associated edge objects.

[0024] Step 102, if the number of current search objects is greater than a first preset number threshold, the current search objects are classified based on preset classification rules, a processing resource is allocated to each classification, and the processing resource is used to take the current search object under the corresponding classification as the starting point, and according to the first hunting demand information, the search is continued in the preset graph structure through the target search algorithm, and the corresponding number of current search objects is continued to be counted and judgment is continued during the search process until the preset end condition is met.

[0025] Step 103: when all the processing resources satisfy a preset termination condition, a threat hunting result corresponding to the first starting clue is obtained according to a search object satisfying the first hunting requirement information.

[0026] A threat hunting method for network data provided by an embodiment of the present application can quickly and accurately find relevant clues of a certain clue from massive network data. First, when a clue expansion task execution instruction is received, the clue expansion task execution instruction can be responded to, and a first starting clue and a first hunting requirement information can be obtained. Among them, the first starting clue can be a specific network event, an abnormal behavior, or an identifier of a known threat, etc., and the first hunting requirement information defines the conditions that need to be met in the search, for example, it defines which network data with which association relationship meets the search requirement during the search process.

[0027] Then, according to the first starting clue, the first starting node is located in the preset graph structure, and starting from the node, according to the first hunting demand information, a target search algorithm (such as depth-first search, etc.) is used to search, and nodes and associated edges that meet the first hunting demand information are found in the preset graph structure. Among them, the preset graph structure is composed of multiple nodes and associated edges between them. The node represents the type of network data (such as IP address, domain name, user account, etc.), and the associated edge indicates the association relationship between these network data (such as communication, access, shared resources, etc.). In addition, the node is also provided with attribute data, such as a specific IP address, a specific domain name, etc., so the first starting node can be found in the preset graph structure according to the first starting clue. During the search process, the number of node objects and / or associated edge objects currently searched can also be counted, and the number of currently searched objects can be compared with the first preset number threshold.

[0028] If it is found after comparison that the number of current search objects exceeds the first preset number threshold, it means that the current search range is already very large, and the number of nodes and / or edges to be searched has increased rapidly. If the processing resource is continued to be used alone for searching, it is likely to cause a performance bottleneck. At this time, the current search object can be classified according to the preset classification rules (such as based on threat type, behavior pattern, impact range, etc.). After that, a processing resource (such as CPU core, memory block, thread, etc.) is allocated to each classification to achieve parallel processing and improve search efficiency. Next, each processing resource takes the current search object under its corresponding classification as the starting point, and continues to use the target search algorithm to search in the preset graph structure according to the first hunting demand information, and continues to count and judge. For example, for a certain processing resource, during the process of searching the processing resource, the number of its current search objects can continue to be counted, and when the number of current search objects exceeds the first preset number threshold, the current search object corresponding to the processing resource can continue to be classified, and a new processing resource can continue to be allocated to each classification... The above search process continues until the preset end condition (such as reaching the maximum search depth, finding a specific number of threats, time limit, etc.) is met. It should be noted that the "satisfying the preset end condition" mentioned here means that each processing resource satisfies the above preset end condition, and only then is the entire search process considered to be finished.

[0029] When all processing resources meet the preset end condition, the threat hunting result corresponding to the first starting clue can be generated according to the search object (ie, the found potential threat) that meets the first hunting requirement information.

[0030] By applying the technical solution of this embodiment, first, when a clue expansion task execution instruction is received, the clue expansion task execution instruction can be responded to, and the first starting clue and the first hunting demand information can be obtained. Then, according to the first starting clue, the first starting node is located in the preset graph structure, and starting from the node, according to the first hunting demand information, the target search algorithm is used to search, and the nodes and associated edges that meet the first hunting demand information are found in the preset graph structure. During the search process, the number of node objects and / or associated edge objects currently searched can also be counted, and the number of current search objects can be compared with the first preset number threshold. If it is found through comparison that the number of current search objects exceeds the first preset number threshold, the current search object can be classified according to the preset classification rule. After that, a processing resource is allocated to each classification. Next, each processing resource takes the current search object under its corresponding classification as the starting point, and according to the first hunting demand information, continues to use the target search algorithm to search in the preset graph structure, and continues to count and judge. The above search process is carried out until the preset end condition is met. When all processing resources meet the preset end condition, the threat hunting result corresponding to the first starting clue can be generated according to the search object that meets the first hunting demand information. The embodiment of the present application constructs a preset graph structure based on massive network data in advance, performs threat hunting through the graph structure, optimizes the query performance of multi-level and multi-relationship network data, and can return query results faster than relational databases; combines graph structure search, classification processing, parallel computing and other technologies to achieve efficient and intelligent identification and tracking of potential threats in network data, which can not only process large-scale network data, but also flexibly adjust the search strategy according to the specific number of search objects, thereby helping to improve the accuracy and efficiency of threat hunting.

[0031] In an embodiment of the present application, optionally, when the preset classification rule is a similarity classification rule, the "classifying the current search object based on the preset classification rule" in step 102 includes: for each current search object, determining the target-associated object within the preset level corresponding to the current search object, and based on the current search object and the target-associated object, determining the vector representation corresponding to the current search object, wherein the target-associated object includes a target-associated node and a target-associated edge; based on the vector representation corresponding to each current search object, clustering each current search object to obtain multiple clustering clusters, and treating the current search object contained in each clustering cluster as a classification.

[0032] In this embodiment, the preset classification rule may include a similarity classification rule. When the preset classification rule is a similarity classification rule, for each current search object, the following operation may be performed: determine the target associated object within the preset level corresponding to the current search object, that is, the target associated node and the target associated edge. Here, the preset level may be determined according to actual needs. For example, the preset level is four levels. When the current search object is a node, then the node directly connected to the current search object (first level node), the node directly connected to the first level node (second level node), the node directly connected to the second level node (third level node), and the node directly connected to the third level node (fourth level node) may be used as the target associated node; the associated edge between the current search object and the directly connected node (first level node), the associated edge between the first level node and the directly connected node (second level node), the associated edge between the second level node and the directly connected node (third level node), and the associated edge between the third level node and the directly connected node (fourth level node) may be used as the target associated edge. Next, the vector representation of the current search object can be determined based on these target associated nodes and target associated edges. When the current search object is an associated edge, the node directly connected to the current search object (first-level node), the node directly connected to the first-level node (second-level node), the node directly connected to the second-level node (third-level node), and the node directly connected to the third-level node (fourth-level node) can be used as the target associated nodes; the associated edge between the first-level node and the second-level node, the associated edge between the second-level node and the third-level node, and the associated edge between the third-level node and the fourth-level node are used as the target associated edges. Next, the vector representation of the current search object can be determined based on these target associated nodes and target associated edges.

[0033] In one embodiment, when the target-associated object is used to calculate the vector representation of the current search object, the calculation can be performed according to the first hunting demand information. Specifically, weights can be assigned to the target-associated nodes and target-associated edges according to the first hunting demand information. For example, the target-associated nodes and target-associated edges that can meet the first hunting demand information are assigned a first weight. For the remaining target-associated nodes and target-associated edges, a second weight can be assigned to the remaining target-associated nodes and target-associated edges in the manner of a larger level and a smaller weight. Here, the first weight can be greater than the second weight. Afterwards, the initial vector representation of each target-associated node and target-associated edge can be multiplied by the corresponding weight, and the results can be added. The result of the addition is then added to the initial vector representation corresponding to the current search object to obtain the vector representation of the current search object. In this way, when calculating the vector representation of the current search object, the user's threat hunting needs can be fully considered, so that the calculation of the vector representation pays more attention to clues related to threat hunting, and gives higher weights to target-related nodes and target-related edges that meet the hunting needs, while giving smaller weights to the remaining target-related nodes and target-related edges. This can ensure that the calculation of the vector representation fully considers the influence of different surrounding nodes and edges, making the vector representation of the current search object more accurate. In the clustering process, the all-round information of the current search object can be fully considered, thereby improving the accuracy of subsequent clustering.

[0034] After obtaining the vector representation of each current search object, these vector representations can be used for clustering. Clustering is an unsupervised learning method used to group similar objects into the same cluster. There are many specific clustering algorithms, such as K-means, hierarchical clustering, DBSCAN, etc. You can choose the appropriate algorithm according to the characteristics and needs of the data. After clustering is completed, multiple clusters can be obtained, each of which contains a group of similar current search objects. These clusters can be regarded as different classifications, that is, the current search objects contained in each cluster are divided into one classification.

[0035] The embodiment of the present application uses the target-related objects within the preset level of the current search object to determine the vector representation of the current search object, so that the vector representation of the current search object can comprehensively consider its directly and indirectly related nodes and related edges. In addition, different weights are given to different target-related objects, so that the vector representations of different target-related objects can have different proportions in the vector representation of the current search object, thereby meeting the actual situation that the degree of association between different target-related objects and the current search object is different, which is conducive to improving the accuracy of the vector representation of the current search object.

[0036] In an embodiment of the present application, optionally, the method further includes: in response to a threat path identification task execution instruction, obtaining a second starting clue, an ending clue and a second hunting demand information, and according to the second hunting demand information, starting a two-way search in the preset graph structure from the second starting node indicated by the second starting clue and the first ending node indicated by the ending clue through a two-way search algorithm, and counting the number of current search objects during the search process; if the number of current search objects is greater than a second preset number threshold, determining a third starting node and a second ending node based on the current search results of the two-way search algorithm, and determining the threat path between the third starting node and the second ending node through an A-star search algorithm; determining the target threat path based on the threat path between the second starting node and the third starting node, the threat path between the first ending node and the second ending node, and the threat path between the third starting node and the second ending node.

[0037] In this embodiment, a threat path identification task can also be executed. First, when a threat path identification task execution instruction is received, the instruction can be responded to, and the second starting clue, the ending clue and the second hunting requirement information can be obtained. Among them, the second starting clue and the ending clue can both be a specific network event, abnormal behavior or a known threat identifier, etc. The second hunting requirement information also defines the conditions that need to be met in the search, for example, it defines which network data with which association relationship meets the search requirement during the search process, and is used to guide the search process.

[0038] Afterwards, according to the second hunting demand information, a search is performed in the preset graph structure through a bidirectional search algorithm. Bidirectional search is a search strategy that starts from both the starting point and the ending point and approaches the middle. Here, the bidirectional search algorithm searches simultaneously from the second starting node indicated by the second starting clue and the first ending node indicated by the ending clue. During the search process, the number of node objects and / or associated edge objects currently searched can also be counted to evaluate the efficiency of the search and whether the search strategy needs to be adjusted.

[0039] If the number of current search objects is greater than the second preset number threshold, it indicates that the search has become too large or complex. If the processing resources are continued to be used alone for the search, it is likely to cause a performance bottleneck. At this time, measures can be taken to optimize the search process. Specifically, two intermediate nodes can be selected based on the current search results of the bidirectional search algorithm, respectively as new starting nodes (third starting nodes) and ending nodes (second ending nodes) to narrow the search scope. Afterwards, the threat path between the third starting node and the second ending node is determined by the A star search algorithm. The A star search algorithm is a heuristic search algorithm, which is particularly suitable for finding the shortest path between two points. In an embodiment of the present application, the A star search algorithm is used to determine the optimal threat path between two newly determined intermediate nodes.

[0040] In fact, it is not clear how large the search range is before searching. The bidirectional search algorithm is usually suitable for finding solutions quickly within a smaller range. It can narrow the search space more quickly by searching from the starting point and the end point at the same time, especially when the solution path is relatively short or the search space is relatively limited. This method can quickly lock the possible solution set in the early stage and reduce unnecessary search overhead. However, as the search range gradually expands, the efficiency of the bidirectional search algorithm will drop significantly because it needs to maintain the search status in two directions at the same time. At this time, switching to the A-star search algorithm can better handle large-scale search spaces, thereby obtaining the desired target threat path.

[0041] After the A-star search algorithm completes the search, the threat path between the second starting node and the third starting node, the threat path between the first ending node and the second ending node, and the threat path between the third starting node and the second ending node can be combined to form a complete threat path from the original starting node (i.e., the second starting node) to the original ending node (i.e., the first ending node), thereby obtaining the target threat path desired by this application.

[0042] The embodiments of the present application can allocate computing resources more efficiently by using a bidirectional search algorithm to quickly narrow the search scope in the early stage, and then switching to the A-star search algorithm for a deeper search when necessary. This method avoids the waste of resources that may result from using a computationally intensive algorithm (such as the A-star algorithm) at the beginning, especially when the solution space is unknown or may be very small. The embodiments of the present application combine the bidirectional search algorithm and the A-star search algorithm, as well as the dynamic adjustment of the search algorithm during the search process, and can efficiently identify threat paths in complex preset graph structures.

[0043] In an embodiment of the present application, optionally, the current search result includes a forward local optimal path and a reverse local optimal path; the "determining a third starting node and a second terminating node based on the current search result of the bidirectional search algorithm" includes: determining the last node of the forward local optimal path, and using the last node of the forward local optimal path as the third starting node, and determining the last node of the reverse local optimal path, and using the last node of the reverse local optimal path as the second terminating node; or, determining the last node of the forward local optimal path, and using the last node of the forward local optimal path as the third starting node, and using the first terminating node as the second terminating node.

[0044] In this embodiment, "forward local optimal path" and "reverse local optimal path" refer to the current optimal path (forward) obtained in the search process from the second starting node to the first terminating node, and the current optimal path (reverse) obtained in the search process from the first terminating node to the second starting node, respectively. After determining the forward local optimal path and the reverse local optimal path, the difference between the length of the forward local optimal path and the length of the reverse local optimal path can be calculated. This difference can be the forward length minus the reverse length, or the reverse length minus the forward length. Then, the absolute value of the length difference is divided by the length of the forward local optimal path to obtain a ratio. The ratio reflects the ratio of the difference in length between the reverse local optimal path and the forward local optimal path relative to the length of the forward path. In addition, a preset ratio is set in advance to determine whether the difference in length between the reverse local optimal path and the forward local optimal path is significant.

[0045] If the calculated ratio is less than or equal to the preset ratio, it is considered that the reverse local optimal path is not much different from the forward local optimal path in length, or the reverse search progresses relatively quickly, so the search can continue from both ends to try to connect the two local optimal paths. Specifically, the last node of the forward local optimal path is determined and used as the third starting node. This node represents the position closest to the first termination node found so far by forward search starting from the original starting node. At the same time, the last node of the reverse local optimal path is determined and used as the second termination node. This node represents the position closest to the second starting node found so far by reverse search starting from the first termination node. In this case, the A-star search algorithm chooses the ends of these two local optimal paths as the new search starting point and end point, and continues to search in the hope of finding the optimal path connecting them between these two points, thereby combining a complete path from the second starting node to the first termination node, that is, the final target threat path.

[0046] If the calculated ratio is greater than the preset ratio, it is considered that the reverse local optimal path differs greatly from the forward local optimal path in length, or the progress of the reverse search is relatively slow, and continuing the reverse search may not be the most efficient strategy at this time. Therefore, the search can be continued only from the end of the forward local optimal path until the first termination node is reached. Specifically, the last node of the forward local optimal path is determined and used as the third starting node. At the same time, the original first termination node is directly used as the second termination node. In this case, the A star search algorithm chooses to continue searching from the end of the forward local optimal path (i.e., the third starting node) to the original first termination node.

[0047] The embodiment of the present application uses the first method, and subsequently uses the third starting node and the second ending node as the new starting node and ending node of the A-star search algorithm, which can effectively narrow the search space and reduce the number of nodes that need to be searched, so that the optimal path can be found more quickly. Through the second method, when the reverse search progresses slowly, this method can better find the threat path from the third starting node to the first ending node. The embodiment of the present application automatically adjusts the subsequent search starting point and end point by judging whether the reverse search is slow, which can better cope with different situations, and is simple and convenient.

[0048] In an embodiment of the present application, optionally, the method also includes: in response to a threat community identification task execution instruction, performing community division on the preset graph structure through a community discovery algorithm to obtain multiple community division results; inputting each community division result into a preset graph neural network respectively, calculating the graph-level vector, node-level vector and edge-level vector corresponding to the community division result through the preset graph neural network, splicing the graph-level vector, the node-level vector and the edge-level vector to obtain a target fusion vector corresponding to the community division result, and outputting the community threat type corresponding to the community division result according to the target fusion vector.

[0049] In this embodiment, threat communities can also be identified. When an execution instruction for a threat community identification task is received, the instruction can be responded to, and a community discovery algorithm can be used to identify and divide the preset graph structure into communities, thereby obtaining multiple community division results. A community discovery algorithm is an algorithm for identifying tightly connected groups of nodes in a graph. Common community discovery algorithms include Louvain algorithm, Label Propagation algorithm, spectral clustering, etc. These algorithms divide the graph into multiple communities by analyzing the connection patterns between nodes. The nodes within each community are tightly connected, while the connections between communities are relatively sparse.

[0050] Next, each community division result can be input into a preset graph neural network. Here, the preset graph neural network (GNN) is a trained or predefined graph neural network model that can process graph structure data and output community threat types. After the community division result is input into the preset graph neural network, the preset graph neural network can generate vector representations of the graph level (representing the characteristics of the entire community), node level (representing the characteristics of each node in the community), and edge level (representing the characteristics of each edge in the community) corresponding to the community division result. These vectors contain rich community structure and node relationship information.

[0051] In one embodiment, the preset graph neural network can calculate the graph-level vector, node-level vector and edge-level vector in the following manner. Specifically, each node in the preset graph structure represents a type of network data, and the node also has node attributes, which represent the specific value of the node. For example, the node type can be an IP address, a user account, etc. When the node type is an IP address, the node attribute can be a specific IP address; when the node type is a user account, the node attribute can be a specific account identifier. It should be noted that different initial vector representations can be prepared in advance for different node types, and different initial vector representations can be prepared for different numbers and different texts. In this way, for each node in each community division result, the preset graph neural network can obtain the initial vector representation corresponding to the node through the node type and node attributes. Then, for each node in the community division result, the preset graph neural network can randomly select N nodes from multiple nodes connected to the current node in combination with a random algorithm, and fuse the initial vector representations of the current node and the N nodes through the attention mechanism, so as to obtain the final vector representation of the node. Afterwards, the attention mechanism is used again to set a weight for each node in the community division result, and the final vector representations corresponding to all nodes in the community division result are weighted summed to obtain the node-level vector of the community division result. It should be noted that the above attention mechanism can be predetermined according to actual needs and is not limited here.

[0052] For edge-level vectors, each associated edge also corresponds to edge attributes, such as communication, access, shared resources, etc. Similarly, initial vector representations can be compiled for these edge attributes in advance. Then, for each associated edge, the preset graph neural network can determine the initial vector representation corresponding to the two connected nodes based on the associated edge. After that, the initial vector representation corresponding to the associated edge and the initial vector representation corresponding to the two nodes are added together and the average value is calculated, and the average value is used as the final vector representation of the associated edge. Similarly, through the attention mechanism, a weight is set for each associated edge in the community division result, and all associated edges in the community division result are weighted summed to obtain the edge-level vector of the community division result.

[0053] For graph-level vectors, the preset graph neural network can assign different weights to each node in the community division result based on the position of each node in the community division result. Similarly, different weights are assigned to each associated edge in the community division result based on the position of each associated edge in the community division result. Specifically, the weights of nodes and associated edges close to the center of the community division result are higher, and the weights of nodes and associated edges far from the center of the community division result are lower. Subsequently, the preset graph neural network can perform weighted summation of the initial vector representation of each node in the community division result according to the weight of each node to obtain a first vector representation, and can perform weighted summation of the initial vector representation of each associated edge in the community division result according to the weight of each associated edge to obtain a second vector representation, add the first vector representation and the second vector representation, and finally obtain the graph-level vector of the community division result.

[0054] Afterwards, the graph-level vector, node-level vector, and edge-level vector corresponding to each community division result are spliced ​​to form a longer vector, namely the target fusion vector. This vector combines the overall characteristics of the community, the individual characteristics of the nodes, and the relationship characteristics of the edges. After obtaining the target fusion vector, the preset graph neural network can judge the community threat type corresponding to the community division result according to the target fusion vector. Threat types may include malicious behavior, fraudulent activities, security vulnerabilities, etc. The embodiment of the present application combines the capabilities of the community discovery algorithm and the graph neural network to realize the automatic identification and analysis of potential threat communities in the graph structure. It can not only discover the community structure, but also extract and fuse a variety of feature information to accurately judge the threat type of the community.

[0055] In an embodiment of the present application, optionally, the method also includes: according to a preset time period, based on the current preset graph structure and the preset graph structure corresponding to the last time the task was executed, determining the incremental substructure through a graph incremental query algorithm, and continuing to execute the corresponding task in the incremental substructure until the task end instruction is received.

[0056] In this embodiment, for real-time updated data scenarios, an incremental graph query algorithm can also be used to periodically query the incremental substructure through the incremental graph query algorithm to avoid repeated tasks on the entire graph structure. In this scenario, the incremental substructure can be determined by the graph incremental query algorithm according to the preset period, based on the current preset graph structure and the preset graph structure corresponding to the last task execution. The incremental substructure refers to the part of the graph structure that has changed between the current period and the previous period (that is, between the current time and the response time). The graph incremental query algorithm is an algorithm specifically used to find new or changed parts (that is, increments) in the graph structure. This algorithm is typically used to process dynamically changing graph data to efficiently identify which parts have changed. In this embodiment, the preset period can be set as required.

[0057] After the incremental substructure is determined, the corresponding tasks can be continued to be executed in this incremental substructure. For example, the clue expansion task, threat path identification task, threat community identification task, etc. can be continued to be executed in the incremental substructure until the task end instruction is received.

[0058] The embodiment of the present application uses a graph incremental query algorithm to dynamically search for incremental substructures, and subsequently performs the same task on the incremental substructures, which can greatly improve the execution efficiency and accuracy of tasks in dynamic scenarios.

[0059] In an embodiment of the present application, optionally, the method further includes: after any task is executed, generating a result display page based on the task execution result, and color-marking the task execution result on the display page based on preset marking rules, wherein different colors are used to indicate different threat levels and / or attributes.

[0060] In this embodiment, when any task is completed, the results of the task can be automatically collected and sorted. The tasks here can be the above-mentioned clue expansion tasks, threat path identification tasks, threat community identification tasks, etc. Then, based on the task execution results, a result display page can be automatically generated. The page can be a web page, a report or any visual interface. On the display page, the task execution results can be intuitively displayed. In addition, the color corresponding to each part of the task execution results can be automatically matched according to the preset marking rules. In this way, the user can quickly understand the threat level and / or attributes of the task execution results by viewing the color markings on the result display page. Here, the preset marking rules can define the corresponding relationship between different colors and threat levels and attributes. The threat level can include low, medium, high, urgent, etc., and each level corresponds to a color. For example, red may indicate an urgent threat, yellow indicates a medium threat, and green indicates a low threat or no threat. This marking method facilitates security analysts to quickly judge the risk characteristics and importance of nodes and edges in a visual interface.

[0061] For example, for clue expansion tasks, the threat hunting results can be visualized, and the associations between nodes and threat expansion paths can be displayed through a visual interface. At the same time, since each node and edge contains attribute information (such as timestamp, IP address, alarm level, vulnerability severity, etc.), this information can be distinguished by visual elements such as different colors, sizes, shapes, and line thicknesses to help quickly identify and understand key threat clues.

[0062] For example, for the threat community identification task, different community division results can be marked with different colors according to the corresponding community threat types, which makes it easier for analysts to quickly identify and focus on areas with greater threats, thereby conducting in-depth analysis and formulating response strategies more effectively.

[0063] Further, as Figure 1 The specific implementation of the method, the embodiment of the present application provides a threat hunting device for network data, such as Figure 2 As shown, the device comprises:

[0064] An information acquisition module, for obtaining a first starting clue and first hunting demand information in response to a clue expansion task execution instruction, and starting a search from a first starting node indicated by the first starting clue in a preset graph structure through a target search algorithm according to the first hunting demand information, and counting the number of current search objects during the search process, and determining the size relationship between the number of current search objects and a first preset number threshold, wherein the preset graph structure includes a plurality of nodes, and associated edges between different nodes, the nodes are constructed based on network data, the associated edges are used to indicate an associated relationship between network data, and the current search object includes a node object and / or an associated edge object;

[0065] A search module, configured to classify the current search objects based on a preset classification rule if the number of the current search objects is greater than a first preset number threshold, allocate a processing resource to each classification, and continue searching in the preset graph structure through a target search algorithm using the processing resource and the current search objects under the corresponding classification as a starting point according to the first hunting demand information, and continue to count the number of the corresponding current search objects and continue to make judgments during the search process until the preset end condition is met;

[0066] A result acquisition module is used to obtain a threat hunting result corresponding to the first starting clue according to a search object that meets the first hunting requirement information when each of the processing resources meets a preset end condition.

[0067] Optionally, when the preset classification rule is a similarity classification rule, the search module is used to:

[0068] For each current search object, determine a target-associated object within a preset level corresponding to the current search object, and determine a vector representation corresponding to the current search object based on the current search object and the target-associated object, wherein the target-associated object includes a target-associated node and a target-associated edge;

[0069] Based on the vector representation corresponding to each current search object, each current search object is clustered to obtain a plurality of clusters, and the current search object contained in each cluster is regarded as a category.

[0070] Optionally, the information acquisition module is further used to obtain a second starting clue, an ending clue and a second hunting demand information in response to a threat path identification task execution instruction, and according to the second hunting demand information, perform a bidirectional search in the preset graph structure from a second starting node indicated by the second starting clue and a first ending node indicated by the ending clue through a bidirectional search algorithm, and count the number of current search objects during the search process;

[0071] The search module is further configured to determine a third starting node and a second ending node based on the current search results of the bidirectional search algorithm if the number of the current search objects is greater than a second preset number threshold, and determine a threat path between the third starting node and the second ending node through an A-star search algorithm;

[0072] The result acquisition module is also used to determine the target threat path based on the threat path between the second starting node and the third starting node, the threat path between the first terminating node and the second terminating node, and the threat path between the third starting node and the second terminating node.

[0073] Optionally, the current search result includes a forward local optimal path and a reverse local optimal path; and the search module is further used to:

[0074] Determine a length difference between a reverse local optimal path length and a forward local optimal path length, and calculate a ratio between an absolute value of the length difference and the forward local optimal path length;

[0075] When the ratio is less than or equal to the preset ratio, determining the last node of the forward local optimal path, and using the last node of the forward local optimal path as the third starting node, and determining the last node of the reverse local optimal path, and using the last node of the reverse local optimal path as the second ending node; or,

[0076] When the ratio is greater than a preset ratio, the last node of the forward local optimal path is determined, and the last node of the forward local optimal path is used as the third starting node, and the first terminating node is used as the second terminating node.

[0077] Optionally, the device further comprises:

[0078] A community division module, configured to respond to a threat community identification task execution instruction, divide the preset graph structure into communities by using a community discovery algorithm, and obtain multiple community division results;

[0079] The community threat type output module is used to input each community division result into a preset graph neural network respectively, calculate the graph-level vector, node-level vector and edge-level vector corresponding to the community division result through the preset graph neural network, concatenate the graph-level vector, the node-level vector and the edge-level vector to obtain the target fusion vector corresponding to the community division result, and output the community threat type corresponding to the community division result according to the target fusion vector.

[0080] Optionally, the device further comprises:

[0081] The graph incremental query module is used to determine the incremental substructure according to the preset time period, based on the current preset graph structure and the preset graph structure corresponding to the last task execution, through the graph incremental query algorithm, and continue to execute the corresponding task in the incremental substructure until the task end instruction is received.

[0082] Optionally, the device further comprises:

[0083] The display module is used to generate a result display page based on the task execution results after any task is completed, and to color-mark the task execution results on the display page based on preset marking rules, wherein different colors are used to indicate different threat levels and / or attributes.

[0084] It should be noted that for other corresponding descriptions of the functional units involved in the threat hunting device for network data provided in the embodiment of the present application, reference can be made to Figure 1 The corresponding description in the method will not be repeated here.

[0085] The present application also provides a computer device, which may be a personal computer, a server, a network device, etc. Figure 3As shown, the computer device includes a bus, a processor, a memory and a communication interface, and may also include an input and output interface and a display device. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store location information. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, the steps in each method embodiment are implemented.

[0086] Those skilled in the art will understand that Figure 3 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0087] In one embodiment, a computer-readable storage medium is provided. The computer-readable storage medium may be non-volatile or volatile, and stores a computer program thereon. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0088] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0089] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0090] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0091] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0092] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A threat hunting method for network data, characterized in that: include: In response to a clue expansion task execution instruction, a first starting clue and first hunting demand information are obtained, and according to the first hunting demand information, a target search algorithm is used to start searching from a first starting node indicated by the first starting clue in a preset graph structure, and during the search process, the number of current search objects is counted, and the size relationship between the number of current search objects and a first preset number threshold is determined, wherein the preset graph structure includes a plurality of nodes, and associated edges between different nodes, the nodes are constructed based on network data, the associated edges are used to indicate an associated relationship between network data, and the current search object includes a node object and / or an associated edge object; If the number of current search objects is greater than a first preset number threshold, the current search objects are classified based on a preset classification rule, a processing resource is allocated to each classification, and the current search objects under the corresponding classification are used as a starting point through the processing resource, and the search is continued in the preset graph structure through a target search algorithm according to the first hunting demand information, and the number of the corresponding current search objects is continued to be counted and judgment is continued during the search process until the preset end condition is met; When all the processing resources meet the preset end condition, the threat hunting result corresponding to the first starting clue is obtained according to the search object that meets the first hunting requirement information.

2. The method according to claim 1, characterized in that When the preset classification rule is a similarity classification rule, classifying the current search object based on the preset classification rule includes: For each current search object, determine a target-associated object within a preset level corresponding to the current search object, and determine a vector representation corresponding to the current search object based on the current search object and the target-associated object, wherein the target-associated object includes a target-associated node and a target-associated edge; Based on the vector representation corresponding to each current search object, each current search object is clustered to obtain a plurality of clusters, and the current search object contained in each cluster is regarded as a category.

3. The method according to claim 1, characterized in that The method further comprises: In response to the threat path identification task execution instruction, a second starting clue, an ending clue and second hunting demand information are obtained, and according to the second hunting demand information, a bidirectional search is performed in the preset graph structure from the second starting node indicated by the second starting clue and the first ending node indicated by the ending clue by a bidirectional search algorithm, and the number of current search objects is counted during the search process; If the number of current search objects is greater than a second preset number threshold, a third starting node and a second ending node are determined based on the current search results of the bidirectional search algorithm, and a threat path between the third starting node and the second ending node is determined through the A-star search algorithm; A target threat path is determined based on the threat path between the second starting node and the third starting node, the threat path between the first terminating node and the second terminating node, and the threat path between the third starting node and the second terminating node.

4. The method according to claim 3, characterized in that: The current search result includes a forward local optimal path and a reverse local optimal path; the current search result based on the bidirectional search algorithm, determining the third starting node and the second ending node, includes: Determine a length difference between a reverse local optimal path length and a forward local optimal path length, and calculate a ratio between an absolute value of the length difference and the forward local optimal path length; When the ratio is less than or equal to the preset ratio, determining the last node of the forward local optimal path, and using the last node of the forward local optimal path as the third starting node, and determining the last node of the reverse local optimal path, and using the last node of the reverse local optimal path as the second ending node; or, When the ratio is greater than a preset ratio, the last node of the forward local optimal path is determined, and the last node of the forward local optimal path is used as the third starting node, and the first terminating node is used as the second terminating node.

5. The method according to claim 1, characterized in that The method further comprises: In response to a threat community identification task execution instruction, the preset graph structure is divided into communities by a community discovery algorithm to obtain a plurality of community division results; Each community division result is input into a preset graph neural network respectively, and the graph-level vector, node-level vector and edge-level vector corresponding to the community division result are calculated by the preset graph neural network. The graph-level vector, the node-level vector and the edge-level vector are concatenated to obtain a target fusion vector corresponding to the community division result, and according to the target fusion vector, the community threat type corresponding to the community division result is output.

6. The method according to any one of claims 1 to 5, characterized in that The method further comprises: According to the preset time period, based on the current preset graph structure and the preset graph structure corresponding to the last task execution, the incremental substructure is determined through the graph incremental query algorithm, and the corresponding task is continued to be executed in the incremental substructure until the task end instruction is received.

7. The method according to any one of claims 1 to 5, characterized in that The method further comprises: When any task is completed, a result display page is generated based on the task execution result, and the task execution result on the display page is color-marked based on preset marking rules, wherein different colors are used to indicate different threat levels and / or attributes.

8. A threat hunting device for network data, characterized in that: include: An information acquisition module, for obtaining a first starting clue and first hunting demand information in response to a clue expansion task execution instruction, and starting a search from a first starting node indicated by the first starting clue in a preset graph structure through a target search algorithm according to the first hunting demand information, and counting the number of current search objects during the search process, and determining the size relationship between the number of current search objects and a first preset number threshold, wherein the preset graph structure includes a plurality of nodes, and associated edges between different nodes, the nodes are constructed based on network data, the associated edges are used to indicate an associated relationship between network data, and the current search object includes a node object and / or an associated edge object; A search module, configured to classify the current search objects based on a preset classification rule if the number of the current search objects is greater than a first preset number threshold, allocate a processing resource to each classification, and continue searching in the preset graph structure through a target search algorithm using the processing resource and the current search objects under the corresponding classification as a starting point according to the first hunting demand information, and continue to count the number of the corresponding current search objects and continue to make judgments during the search process until the preset end condition is met; A result acquisition module is used to obtain a threat hunting result corresponding to the first starting clue according to a search object that meets the first hunting requirement information when each of the processing resources meets a preset end condition.

9. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

10. A computer device comprising a storage medium, a processor, and a computer program stored in the storage medium and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 7 is implemented.

Images