Threat hunting method and device of network data, storage medium and computer equipment

Abstract

The application discloses a threat hunting method and device of network data, a storage medium and a computer device. The method comprises the following steps: in response to a clue extension task execution instruction, acquiring a first starting clue and first hunting requirement information; according to the first hunting requirement information, searching in a preset graph structure from a first starting node indicated by the first starting clue by using a target search algorithm, and counting the number of current search objects in the searching; if the number of the current search objects is greater than a first preset number threshold, classifying the current search objects based on a preset classification rule, allocating a processing resource to each classification, and taking the current search objects under the corresponding classification as a starting point to search in the preset graph structure according to the first hunting requirement information by using the target search algorithm; in the searching, the number of the current search objects is continuously counted and a judgment is continuously made until a preset ending condition is met; and according to the search objects meeting the first hunting requirement information, a threat hunting result is obtained.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of network security, in particular to a network data threat hunting method and device, a storage medium and a computer device. BACKGROUND

[0002] With the rapid development of information technology, the network environment is becoming increasingly complex, and the scale of network data is growing explosively, which brings unprecedented challenges to network security. Current network attack means is constantly renewed, and the attack behavior is enhanced. The traditional defense system based on signature or rule has been difficult to effectively cope with all potential threats. Therefore, as an active security protection strategy, threat hunting has gradually become an important research direction in the field of network security. Threat hunting aims to actively discover and identify potential security threats by in-depth analysis of network data, and the key is how to efficiently and accurately mine abnormal or malicious behavior from massive data. Current threat hunting is gaining more and more attention.

[0003] Existing threat hunting methods mostly rely on manual analysis or simple automation tools, and these methods often have problems such as low efficiency, high false positive rate, etc. Especially in the online clue expansion task, due to the huge amount of network data, if the relevant clues of a certain clue are found directly in the massive network data through manual analysis, the clue is expanded, then this process often needs to consume a lot of time and has poor accuracy; if the relevant clues of a certain clue are found directly from the massive network data through simple automation tools, then this process often faces system resource exhaustion and performance bottleneck, seriously affecting search efficiency, and even causing a dead lock situation. SUMMARY

[0004] Therefore, the present application provides a network data threat hunting method and device, a storage medium and a computer device, which pre-constructs a preset graph structure according to massive network data, and performs threat hunting through the graph structure, optimizing the query performance of multi-level and multi-relation network data. Compared with a relational database, the query result can be returned faster; combined with graph structure search, classification processing, parallel computing and other technologies, efficient and intelligent identification and tracking of potential threats in network data are realized, which not only can process large-scale network data, but also can flexibly adjust the search strategy according to the specific number of search objects, thereby helping to improve the accuracy and efficiency of threat hunting.

[0005] According to one aspect of the present application, a network data threat hunting method is provided, comprising:

[0006] In response to the clue expansion task execution instruction, the first starting clue and first hunting requirement information are acquired, and according to the first hunting requirement information, a target search algorithm is used to search from a first starting node indicated by the first starting clue in a preset graph structure, and the number of current search objects is counted in the search process, and a size relationship between the number of current search objects and a first preset number threshold is judged, wherein the preset graph structure includes a plurality of nodes and associated edges between different nodes, the nodes are constructed based on network data, the associated edges are used to indicate the association relationship between the network data, and the current search objects include node objects and / or associated edge objects.

[0007] If the number of current search objects is greater than the first preset number threshold, the current search objects are classified based on a preset classification rule, a processing resource is allocated for each classification, and the current search objects under the corresponding classification are taken as the starting point by the processing resource, and the search in the preset graph structure is continued according to the first hunting requirement information by using the target search algorithm, the number of corresponding current search objects is counted and the judgment is continued in the search process, and the process is ended when a preset end condition is met.

[0008] When each of the processing resources meets the preset end condition, the threat hunting result corresponding to the first starting clue is obtained according to the search objects meeting the first hunting requirement information.

[0009] According to another aspect of the present application, a network data threat hunting device is provided, comprising:

[0010] The information acquisition module is configured to acquire a first starting clue and first hunting requirement information in response to a clue expansion task execution instruction, and to search from a first starting node indicated by the first starting clue in a preset graph structure according to the first hunting requirement information by using a target search algorithm, and to count the number of current search objects in the search process, and to judge the size relationship between the number of current search objects and a first preset number threshold, wherein the preset graph structure includes a plurality of nodes and associated edges between different nodes, the nodes are constructed based on network data, the associated edges are used to indicate the association relationship between the network data, and the current search objects include node objects and / or associated edge objects.

[0011] The search module is configured to, if the number of current search objects is greater than a first preset number threshold, classify the current search objects based on a preset classification rule, assign a processing resource to each classification, and start searching in the preset graph structure based on the first hunting demand information and a target search algorithm, with the current search objects in the corresponding classification as the starting point. The number of current search objects is continuously counted and the judgment is continuously performed during the searching process, until a preset ending condition is met.

[0012] The result obtaining module is configured to, when each of the processing resources meets the preset ending condition, obtain a threat hunting result corresponding to the first starting clue based on the search objects meeting the first hunting demand information.

[0013] According to yet another aspect of the present application, a storage medium having a computer program stored thereon is provided. The program, when executed by a processor, implements the threat hunting method for network data.

[0014] According to still another aspect of the present application, a computer device is provided. The computer device includes a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor. The processor, when executing the program, implements the threat hunting method for network data.

[0015] By the technical scheme, the method and device for threat hunting of network data, the storage medium and the computer device provided by the application first acquire a first starting clue and first hunting requirement information in response to a clue extension task execution instruction when the clue extension task execution instruction is received. Then, according to the first starting clue, a first starting node is located in a preset graph structure, and starting from the node, a target search algorithm is used to search in the preset graph structure to find nodes and associated edges that meet the first hunting requirement information according to the first hunting requirement information. In the searching process, the number of currently searched node objects and / or associated edge objects can also be counted, and the number of the current search objects is compared with a first preset number threshold. If it is found through the comparison that the number of the current search objects exceeds the first preset number threshold, the current search objects can be classified according to a preset classification rule. Then, a processing resource is allocated to each classification. Next, each processing resource takes the current search objects under the corresponding classification as a starting point, continues to search in the preset graph structure using the target search algorithm according to the first hunting requirement information, and continues to count and judge. The above searching process continues until a preset ending condition is met. When all the processing resources meet the preset ending condition, a threat hunting result corresponding to the first starting clue can be generated according to the search objects that meet the first hunting requirement information. The embodiment of the application preconstructs a preset graph structure according to massive network data, and performs threat hunting in the form of the graph structure, thereby optimizing the query performance of network data of multiple levels and multiple relationships, and returning query results faster than a relational database. In combination with the technologies of graph structure search, classification processing and parallel computing, efficient and intelligent identification and tracking of potential threats in network data are realized, which not only can process large-scale network data, but also can flexibly adjust the search strategy according to the specific number of search objects, thereby being conducive to improving the accuracy and efficiency of threat hunting.

[0016] The above description is only a summary of the technical scheme of the application. In order to more clearly understand the technical means of the application, the application can be implemented in accordance with the content of the specification, and in order to make the above and other purposes, features and advantages of the application more obvious and easy to understand, the following specific embodiments of the application are described. BRIEF DESCRIPTION OF DRAWINGS

[0017] The drawings described herein are used to provide further understanding of the application, and form a part of the application. The schematic embodiments of the application and the description thereof are used to explain the application, and do not constitute an improper limitation on the application. In the drawings:

[0018] Figure 1 A flowchart of a method for threat hunting of network data provided by an embodiment of the application is shown;

[0019] Figure 2A structural schematic diagram of a network data threat hunting device is shown.

[0020] Figure 3 A device structure schematic diagram of a computer device is shown. DETAILED DESCRIPTION

[0021] The present application will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments. It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other without conflict.

[0022] A network data threat hunting method is provided in the present embodiment, as shown in the figure, the method comprises: Figure 1

[0023] Step 101, in response to a clue extension task execution instruction, a first starting clue and first hunting requirement information are obtained, according to the first hunting requirement information, a target search algorithm is used to search in a preset graph structure from a first starting node indicated by the first starting clue, and the number of current search objects is counted in the search process, and the size relationship between the number of current search objects and a first preset number threshold is judged, wherein the preset graph structure includes multiple nodes and associated edges between different nodes, the nodes are constructed based on network data, the associated edges are used to indicate the association relationship between network data, and the current search objects include node objects and / or associated edge objects.

[0024] Step 102, if the number of current search objects is greater than the first preset number threshold, the current search objects are classified based on a preset classification rule, a processing resource is allocated for each classification, the current search objects under the corresponding classification are taken as the starting point by the processing resource, the first hunting requirement information is used to continue searching in the preset graph structure by the target search algorithm, the number of corresponding current search objects is counted and the judgment is continued in the search process, and the process is ended when a preset ending condition is met.

[0025] Step 103, when each of the processing resources meets the preset ending condition, the threat hunting result corresponding to the first starting clue is obtained according to the search objects meeting the first hunting requirement information.

[0026] ​The network data threat hunting method provided by the embodiments of the present application can quickly and accurately find relevant clues of a certain clue from massive network data. First, when a clue extension task execution instruction is received, the clue extension task execution instruction can be responded to, and a first starting clue and first hunting requirement information can be obtained. The first starting clue can be an identifier of a specific network event, abnormal behavior or known threat, etc., and the first hunting requirement information defines conditions that need to be met in the search, for example, defines network data having which association relationship in the search process meets the search requirement.

[0027] Then, according to the first starting clue, a first starting node is located in a preset graph structure, and starting from the node, a target search algorithm (such as a depth-first search, etc.) is used to search according to the first hunting requirement information, and nodes and associated edges that meet the first hunting requirement information are found in the preset graph structure. The preset graph structure is composed of multiple nodes and associated edges between the nodes. The nodes represent categories of network data (such as IP addresses, domain names, user accounts, etc.), and the associated edges indicate association relationships (such as communication, access, sharing of resources, etc.) between the network data. In addition, the nodes are also provided with attribute data, such as specific IP addresses, specific domain names, etc., so that the first starting node can be found in the preset graph structure according to the first starting clue. In the search process, the number of currently searched node objects and / or associated edge objects can also be counted, and the number of current search objects is compared with a first preset number threshold.

[0028] If it is found through comparison that the number of current search objects exceeds the first preset number threshold, it indicates that the current search range has been very large, and the number of nodes and / or edges to be searched has grown rapidly. If the search continues to be performed using the processing resource alone, it is likely to cause a performance bottleneck. At this time, the current search objects can be classified according to a preset classification rule (such as based on threat type, behavior pattern, impact range, etc.). Then, a processing resource (such as a CPU core, a memory block, a thread, etc.) is allocated to each classification to implement parallel processing and improve search efficiency. Next, each processing resource takes the current search objects under its corresponding classification as the starting point, continues to use the target search algorithm to search in the preset graph structure according to the first hunting demand information, and continues to count and judge. For example, for a processing resource, the number of current search objects can be continuously counted during the search performed by the processing resource, and when the number of current search objects exceeds the first preset number threshold, the current search objects corresponding to the processing resource can be continuously classified, and a new processing resource can be continuously allocated to each classification. The above search process continues until a preset end condition (such as reaching a maximum search depth, finding a certain number of threats, time limit, etc.) is met, and then the search process ends. It should be noted that the preset end condition mentioned here means that each processing resource meets the above preset end condition, and only in this way can the entire search process be considered to end.

[0029] When all processing resources meet the preset end condition, a threat hunting result corresponding to the first starting clue can be generated according to the search objects (i.e., the found potential threats) that meet the first hunting demand information.

[0030] By applying the technical solutions of the embodiment, first, when the clue extension task execution instruction is received, the first starting clue and the first hunting requirement information can be acquired in response to the clue extension task execution instruction. Then, according to the first starting clue, the first starting node is located in the preset graph structure, and starting from the node, the target search algorithm is used to search according to the first hunting requirement information, and the nodes and associated edges that meet the first hunting requirement information are found in the preset graph structure. In the search process, the number of currently searched node objects and / or associated edge objects can also be counted, and the number of current search objects is compared with the first preset number threshold. If it is found through comparison that the number of current search objects exceeds the first preset number threshold, the current search objects can be classified according to the preset classification rule. Then, a processing resource is allocated to each classification. Next, each processing resource takes the current search object under its corresponding classification as the starting point, continues to use the target search algorithm to search in the preset graph structure according to the first hunting requirement information, and continues to count and judge. The above search process continues until the preset ending condition is met. When all processing resources meet the preset ending condition, the threat hunting result corresponding to the first starting clue can be generated according to the search objects that meet the first hunting requirement information. In the embodiment of the application, the preset graph structure is constructed in advance according to massive network data, and the threat hunting is performed in the form of a graph structure, which optimizes the query performance of network data with multiple levels and multiple relationships, and can return query results faster than a relational database. In combination with the technologies of graph structure search, classification processing, parallel computing and the like, efficient and intelligent identification and tracking of potential threats in network data are realized, which not only can process large-scale network data, but also can flexibly adjust the search strategy according to the specific number of search objects, thereby being conducive to improving the accuracy and efficiency of threat hunting.

[0031] In the embodiment of the application, optionally, when the preset classification rule is a similarity classification rule, the "classifying the current search objects based on the preset classification rule" in step 102 includes: for each current search object, determining a target associated object in a preset level corresponding to the current search object, and determining a vector representation corresponding to the current search object based on the current search object and the target associated object, wherein the target associated object includes a target associated node and a target associated edge; clustering each current search object based on the vector representation corresponding to the current search object to obtain a plurality of clustering clusters, and taking the current search objects included in each clustering cluster as a classification.

[0032] In this embodiment, the preset classification rule can include a similarity classification rule. When the preset classification rule is the similarity classification rule, for each current search object, the following operations can be performed: determining target associated objects, i.e., target associated nodes and target associated edges, in a preset hierarchy corresponding to the current search object. Here, the preset hierarchy can be determined according to actual requirements. For example, the preset hierarchy is four levels, when the current search object is a node, then the nodes directly connected to the current search object (first level nodes), the nodes directly connected to the first level nodes (second level nodes), the nodes directly connected to the second level nodes (third level nodes), and the nodes directly connected to the third level nodes (fourth level nodes) can be taken as target associated nodes; the association edges between the current search object and the directly connected nodes (first level nodes), the association edges between the first level nodes and the directly connected nodes (second level nodes), the association edges between the second level nodes and the directly connected nodes (third level nodes), and the association edges between the third level nodes and the directly connected nodes (fourth level nodes) can be taken as target associated edges. Next, the vector representation of the current search object can be determined based on the target associated nodes and the target associated edges. When the current search object is an association edge, then the nodes directly connected to the current search object (first level nodes), the nodes directly connected to the first level nodes (second level nodes), the nodes directly connected to the second level nodes (third level nodes), and the nodes directly connected to the third level nodes (fourth level nodes) can be taken as target associated nodes; the association edges between the first level nodes and the second level nodes, the association edges between the second level nodes and the third level nodes, and the association edges between the third level nodes and the fourth level nodes can be taken as target associated edges. Next, the vector representation of the current search object can be determined based on the target associated nodes and the target associated edges.

[0033] In an embodiment, when calculating the vector representation of the current search object by using the target associated objects, the calculation can be performed according to the first hunting demand information. Specifically, the target associated nodes and the target associated edges can be assigned weights according to the first hunting demand information, for example, the target associated nodes and the target associated edges that can meet the first hunting demand information are assigned a first weight, and for the remaining target associated nodes and the target associated edges, a second weight can be assigned to the remaining target associated nodes and the target associated edges in a manner that the larger the level, the smaller the weight, where the first weight can be greater than the second weight. Then, the initial vector representation of each target associated node and target associated edge is multiplied by the corresponding weight, and the results are subsequently added, and the added result is added to the initial vector representation corresponding to the current search object, and the vector representation of the current search object is obtained. In this way, when calculating the vector representation of the current search object, the threat hunting demand of the user can be fully considered, so that the calculation of the vector representation pays more attention to the clues related to threat hunting, and a higher weight is given to the target associated nodes and the target associated edges that meet the hunting demand, while a smaller weight is given to the remaining target associated nodes and the target associated edges. This can ensure that the calculation of the vector representation fully considers the influence of different nodes and edges around, so that the vector representation of the current search object is more accurate, and the all-around information of the current search object can be fully considered in the clustering process, thereby improving the accuracy of subsequent clustering.

[0034] After obtaining the vector representation of each current search object, the vector representation can be used to perform clustering processing. Clustering is an unsupervised learning method for grouping similar objects into the same cluster. There are many specific algorithms for clustering, such as K-means, hierarchical clustering, DBSCAN, etc. A suitable algorithm can be selected according to the characteristics and needs of the data. After clustering is completed, a plurality of cluster clusters can be obtained, and each cluster cluster contains a group of similar current search objects. These cluster clusters can be regarded as different classifications, i.e. the current search objects contained in each cluster cluster are divided into a classification.

[0035] The embodiments of the present application use the target associated objects within the preset level of the current search object to determine the vector representation of the current search object, which can make the vector representation of the current search object consider the nodes and associated edges of direct association and indirect association comprehensively. In addition, different weights are assigned to different target associated objects, which can make the proportion of the vector representation of different target associated objects in the vector representation of the current search object different, so as to conform to the actual situation that the association degrees between different target associated objects and the current search object are different, and is beneficial to improving the accuracy of the vector representation of the current search object.

[0036] In the embodiments of the present application, optionally, the method further comprises: in response to the threat path identification task execution instruction, acquiring a second starting clue, a termination clue, and second hunting requirement information; according to the second hunting requirement information, starting a bidirectional search from a second starting node indicated by the second starting clue and a first termination node indicated by the termination clue in the preset graph structure through a bidirectional search algorithm, and counting the number of current search objects in the search process; if the number of current search objects is greater than a second preset number threshold, determining a third starting node and a second termination node based on the current search result of the bidirectional search algorithm, and determining a threat path between the third starting node and the second termination node through an A-star search algorithm; and determining a target threat path based on the threat path between the second starting node and the third starting node, the threat path between the first termination node and the second termination node, and the threat path between the third starting node and the second termination node.

[0037] In this embodiment, a threat path identification task can also be performed. First, when a threat path identification task execution instruction is received, a second starting clue, a termination clue, and second hunting requirement information can be acquired in response to the instruction. The second starting clue and the termination clue can each be an identifier of a specific network event, abnormal behavior, or known threat, and the second hunting requirement information also defines conditions that need to be met in the search, for example, defines network data having which association relationship in the search process meets the search requirement, for guiding the search process.

[0038] Then, according to the second hunting requirement information, a search is performed in the preset graph structure through a bidirectional search algorithm. Bidirectional search is a search strategy that starts from the starting point and the termination point at the same time and approaches the middle. Here, the bidirectional search algorithm searches from the second starting node indicated by the second starting clue and the first termination node indicated by the termination clue at the same time. In the search process, the number of node objects and / or associated edge objects searched currently can also be counted to evaluate the efficiency of the search and whether the search strategy needs to be adjusted.

[0039] If the number of current search objects is greater than the second preset number threshold, it indicates that the search becomes too large or complex, and if the search continues to be performed using the processing resource alone, it is likely to cause a performance bottleneck. At this time, measures can be taken to optimize the search process. Specifically, based on the current search result of the bidirectional search algorithm, two intermediate nodes can be selected as a new starting node (third starting node) and a new ending node (second ending node) respectively, so as to narrow the search range. Then, the threat path between the third starting node and the second ending node is determined by using the A-star search algorithm. The A-star search algorithm is a heuristic search algorithm, which is particularly suitable for finding the shortest path between two points. In the embodiments of the present application, the A-star search algorithm is used to determine the optimal threat path between the two newly determined intermediate nodes.

[0040] In fact, it is not clear how large the search range is before the search. The bidirectional search algorithm is generally suitable for quickly finding solutions in a small range. It can quickly narrow the search space by searching from both the starting point and the ending point at the same time, especially when the path of the solution is relatively short or the search space is limited. This method can quickly lock the possible solution set at the initial stage, reducing unnecessary search overhead. However, when the search range gradually expands, the efficiency of the bidirectional search algorithm will decrease significantly, because it needs to maintain the search state of both directions at the same time. At this time, switching to the A-star search algorithm can better handle large-scale search space, so as to obtain the final desired target threat path.

[0041] After the A-star search algorithm finishes searching, the threat path between the second starting node and the third starting node, the threat path between the first ending node and the second ending node, and the threat path between the third starting node and the second ending node can be combined to form a complete threat path from the original starting node (i.e. the second starting node) to the original ending node (i.e. the first ending node), thereby obtaining the target threat path desired by the present application.

[0042] The embodiments of the present application can more effectively allocate computing resources by using the bidirectional search algorithm to quickly narrow the search range at the initial stage, and then switching to the A-star search algorithm for further search when necessary. This method avoids the waste of resources that may be caused by using a computationally intensive algorithm (such as the A-star algorithm) at the beginning, especially when the solution space is unknown or may be small. The embodiments of the present application combine the bidirectional search algorithm and the A-star search algorithm, and dynamically adjust the search algorithm during the search process, so as to efficiently identify the threat path in a complex preset graph structure.

[0043] In the embodiments of the present application, the current search result can include a forward local optimal path and a reverse local optimal path. The determining of the third start node and the second end node based on the current search result of the bidirectional search algorithm can include: determining a last node of the forward local optimal path, taking the last node of the forward local optimal path as the third start node, determining a last node of the reverse local optimal path, and taking the last node of the reverse local optimal path as the second end node; or determining a last node of the forward local optimal path, taking the last node of the forward local optimal path as the third start node, and taking the first end node as the second end node.

[0044] In the embodiments, the forward local optimal path and the reverse local optimal path respectively refer to a current optimal path (forward) obtained in a search process from the second start node to the first end node, and a current optimal path (reverse) obtained in a search process from the first end node to the second start node. After the forward local optimal path and the reverse local optimal path are determined, a difference between a length of the forward local optimal path and a length of the reverse local optimal path can be calculated. The difference can be the length of the forward path minus the length of the reverse path, or the length of the reverse path minus the length of the forward path. Then, an absolute value of the length difference is divided by the length of the forward local optimal path to obtain a ratio. The ratio reflects a proportion of a difference between the length of the reverse local optimal path and the length of the forward local optimal path relative to the length of the forward path. In addition, a preset ratio is set in advance to determine whether the difference between the length of the reverse local optimal path and the length of the forward local optimal path is significant.

[0045] If the calculated ratio is less than or equal to the preset ratio, it is considered that the difference between the length of the reverse local optimal path and the length of the forward local optimal path is not large, or the progress of the reverse search is relatively fast, so that the search from the two ends can be continued to attempt to connect the two local optimal paths. Specifically, a last node of the forward local optimal path is determined and taken as the third start node. The node represents a position closest to the first end node found by the forward search so far from the original start node. Meanwhile, a last node of the reverse local optimal path is determined and taken as the second end node. The node represents a position closest to the second start node found by the reverse search so far from the first end node. In this case, the A-star search algorithm selects the ends of the two local optimal paths as new search starting points and ending points, and continues to search in order to find an optimal path connecting them, so as to combine into a complete path from the second start node to the first end node, that is, the final target threat path.

[0046] If the calculated ratio is greater than the preset ratio, it is considered that the reverse local optimal path and the forward local optimal path differ greatly in length, or the progress of the reverse search is relatively slow, at this time, it is not the most efficient strategy to continue from the reverse search. Therefore, the search can be continued only from the end of the forward local optimal path until the first termination node is reached. Specifically, the last node of the forward local optimal path is determined and taken as a third starting node. At the same time, the original first termination node is directly taken as a second termination node. In this case, the A-star search algorithm selects to continue searching from the end of the forward local optimal path (i.e., the third starting node) to the original first termination node.

[0047] Through the first method, and subsequently taking the third starting node and the second termination node as new starting node and termination node of the A-star search algorithm, the embodiments of the present application can effectively reduce the search space and reduce the number of nodes that need to be searched, so that the optimal path can be found faster. Through the second method, when the reverse search progresses slowly, this method can better find the threat path between the third starting node and the first termination node. The embodiments of the present application automatically adjust the starting point and the end point of the subsequent search by judging whether the reverse search is slow, which can better cope with different situations and is simple and convenient.

[0048] In the embodiments of the present application, optionally, the method further comprises: in response to an execution instruction of a threat community identification task, performing community division on the preset graph structure through a community discovery algorithm to obtain a plurality of community division results; inputting each community division result into a preset graph neural network respectively, calculating a graph-level vector, a node-level vector and an edge-level vector corresponding to the community division result through the preset graph neural network, splicing the graph-level vector, the node-level vector and the edge-level vector to obtain a target fusion vector corresponding to the community division result, and outputting a community threat type corresponding to the community division result according to the target fusion vector.

[0049] In this embodiment, the threat community can also be identified. When an execution instruction of a threat community identification task is received, the instruction can be responded to, and a community discovery algorithm can be used to identify and divide the preset graph structure, so as to obtain a plurality of community division results. The community discovery algorithm is an algorithm for identifying closely connected node groups in a graph. Common community discovery algorithms include Louvain algorithm, Label Propagation algorithm, spectral clustering, etc. These algorithms divide the graph into multiple communities by analyzing the connection mode between nodes, and the nodes within each community are closely connected, while the connection between communities is relatively sparse.

[0050] Then, each community division result can be input into a preset graph neural network. Here, the preset graph neural network (GNN) is a graph neural network model that has been trained or predefined, which can process graph structure data and output community threat types. After inputting the community division result into the preset graph neural network, the preset graph neural network can generate vector representations of the graph level (representing the features of the entire community), the node level (representing the features of each node in the community), and the edge level (representing the features of each edge in the community) corresponding to the community division result. These vectors contain rich community structure and node relationship information.

[0051] In one embodiment, the preset graph neural network can calculate the graph level vector, the node level vector, and the edge level vector in the following manner. Specifically, each node in the preset graph structure represents a type of network data, and the node also has node attributes representing specific numerical values of the node. For example, the node type can be an IP address, a user account, etc. When the node type is an IP address, the node attribute can be a specific IP address; when the node type is a user account, the node attribute can be a specific account identifier. It should be noted that different initial vector representations can be prepared in advance for different node types, and different initial vector representations can be prepared for different numbers and different letters, so that for each node in each community division result, the preset graph neural network can obtain the initial vector representation corresponding to the node through the node type and the node attribute. Then, for each node in the community division result, the preset graph neural network can randomly select N nodes from the multiple nodes connected to the current node through a random algorithm, and fuse the initial vector representations of the current node and the N nodes through an attention mechanism, thereby obtaining the final vector representation of the node. Then, again through the attention mechanism, a weight is set for each node in the community division result, and the final vector representations corresponding to all nodes in the community division result are weighted and summed, thereby obtaining the node level vector of the community division result. It should be noted that the above attention mechanism can be predetermined according to actual needs, which is not limited here.

[0052] For the edge level vector, each associated edge also has edge attributes, such as communication, access, sharing of resources, etc. Similarly, initial vector representations can be prepared in advance for these edge attributes, and then for each associated edge, the preset graph neural network can determine the initial vector representations corresponding to the two nodes connected based on the associated edge, and then average the sum of the initial vector representation corresponding to the associated edge and the initial vector representations corresponding to the two nodes, taking the average value as the final vector representation of the associated edge. Similarly, through the attention mechanism, a weight is set for each associated edge in the community division result, and all associated edges in the community division result are weighted and summed, thereby obtaining the edge level vector of the community division result.

[0053] For the graph-level vector, the preset graph neural network can assign different weights to each node in the community partition result based on the position of each node in the community partition result, and similarly, assign different weights to each associated edge in the community partition result based on the position of each associated edge in the community partition result. Specifically, the weights of the nodes and associated edges close to the central position of the community partition result are higher, and the weights of the nodes and associated edges far from the central position of the community partition result are lower. Subsequently, the preset graph neural network can perform weighted summation on the initial vector representation of each node in the community partition result according to the weight of each node, to obtain a first vector representation, and can perform weighted summation on the initial vector representation of each associated edge in the community partition result according to the weight of each associated edge, to obtain a second vector representation. The first vector representation and the second vector representation are added to obtain a graph-level vector of the community partition result.

[0054] After that, the graph-level vector, the node-level vector and the edge-level vector corresponding to each community partition result are spliced to form a longer vector, i.e., a target fusion vector. This vector integrates the overall features of the community, the individual features of the nodes and the relationship features of the edges. After obtaining the target fusion vector, the preset graph neural network can determine the community threat type corresponding to the community partition result according to the target fusion vector. The threat type can include malicious behavior, fraudulent activity, security vulnerability, etc. The embodiment of the present application realizes the automatic identification and analysis of potential threat communities in the graph structure by combining the community discovery algorithm and the ability of the graph neural network. Not only can the community structure be found, but also various feature information can be extracted and fused to accurately determine the threat type of the community.

[0055] In the embodiment of the present application, optionally, the method further comprises: according to a preset time period, determining an incremental substructure by a graph incremental query algorithm according to the current preset graph structure and the preset graph structure corresponding to the last time when the task is executed, and continuing to execute the corresponding task in the incremental substructure until a task end instruction is received.

[0056] In this embodiment, for the real-time updated data scenario, an incremental graph query algorithm can also be used. The incremental graph query algorithm is used to periodically query the incremental sub-structure, avoiding repeated execution of tasks on the entire graph structure. In this scenario, according to the preset period, the incremental sub-structure can be determined by the graph incremental query algorithm based on the current preset graph structure and the preset graph structure corresponding to the last execution of the task. The incremental sub-structure refers to the part of the graph structure that has changed between the current period and the last period (i.e., between the current time and the response time). The graph incremental query algorithm is an algorithm specially used to find the newly added or changed part (i.e., the increment) in the graph structure. This algorithm is usually used to process dynamically changing graph data to efficiently identify which part has changed. In this embodiment, the preset period can be set according to the requirements.

[0057] After the incremental sub-structure is determined, the corresponding task can be continued to be executed in this incremental sub-structure, for example, the thread expansion task, the threat path identification task, the threat community identification task, and the like are continued to be executed in the incremental sub-structure until the end instruction of the task is received.

[0058] In this embodiment, the graph incremental query algorithm is used to dynamically find the incremental sub-structure, and the same task is executed on the incremental sub-structure, which can greatly improve the execution efficiency and accuracy of the task in a dynamic scenario.

[0059] In this embodiment, optionally, the method further includes: when any task execution ends, generating a result display page based on the task execution result, and color marking the task execution result on the display page based on a preset marking rule, wherein different colors are used to indicate different threat levels and / or attributes.

[0060] In this embodiment, after any one of the tasks ends, the results of the task can be automatically collected and arranged. Here, the task can be the thread expansion task, the threat path identification task, the threat community identification task, and the like. Then, based on the task execution result, a result display page can be automatically generated. The page can be a web page, a report, or any visual interface. On the display page, the task execution result can be intuitively displayed. In addition, the color corresponding to each part of the task execution result can be automatically matched according to the preset marking rule. In this way, the user can quickly understand the threat level and / or attribute of the task execution result by viewing the color marking on the result display page. Here, the preset marking rule can define the correspondence between different colors and threat levels, attributes. The threat level can include low, medium, high, urgent, and the like, and each level corresponds to a color. For example, red can represent an urgent threat, yellow can represent a medium threat, and green can represent a low threat or no threat. This marking method facilitates security analysts to quickly judge the risk characteristics and importance of nodes and edges in the visual interface.

[0061] For example, in clue expansion tasks, threat hunting results can be visualized, displaying the relationships between nodes and the threat expansion path through a visual interface. Furthermore, since each node and edge contains attribute information (such as timestamps, IP addresses, alert levels, vulnerability severity, etc.), this information can be distinguished using visual elements such as different colors, sizes, shapes, and line thicknesses to help quickly identify and understand key threat clues.

[0062] For example, in the task of threat community identification, different community segmentation results can be marked with different colors according to the corresponding community threat type, which makes it easier for analysts to quickly identify and focus on areas with greater threats, thereby enabling more effective in-depth analysis and response strategy formulation.

[0063] Furthermore, as Figure 1 In its specific implementation, this application provides a network data threat hunting device, such as... Figure 2 As shown, the device includes:

[0064] The information acquisition module is used to respond to the clue expansion task execution instruction, acquire the first starting clue and the first hunting requirement information, and, based on the first hunting requirement information, search in a preset graph structure starting from the first starting node indicated by the first starting clue through a target search algorithm. During the search process, the number of current search objects is counted, and the relationship between the number of current search objects and a first preset number threshold is determined. The preset graph structure includes multiple nodes and the associated edges between different nodes. The nodes are constructed based on network data, and the associated edges are used to indicate the relationship between network data. The current search objects include node objects and / or associated edge objects.

[0065] The search module is used to classify the current search objects according to a preset classification rule if the number of current search objects is greater than a first preset number threshold. It allocates a processing resource to each category and uses the processing resource to start from the current search objects under the corresponding category. Based on the first hunting demand information, it continues to search in the preset graph structure through the target search algorithm. During the search process, it continues to count the number of the corresponding current search objects and makes judgments until the preset termination condition is met.

[0066] The result acquisition module is used to obtain the threat hunting result corresponding to the first starting clue based on the search object that satisfies the first hunting requirement information when all the processing resources meet the preset termination conditions.

[0067] Optionally, when the preset classification rule is a similarity classification rule, the search module is used to:

[0068] For each current search object, a target associated object in a preset hierarchy corresponding to the current search object is determined, and a vector representation corresponding to the current search object is determined based on the current search object and the target associated object, wherein the target associated object includes a target associated node and a target associated edge;

[0069] Based on the vector representation corresponding to each current search object, clustering processing is performed on each current search object to obtain a plurality of clustering clusters, and each current search object included in each clustering cluster is taken as a classification.

[0070] Optionally, the information acquisition module is further configured to, in response to a threat path identification task execution instruction, acquire a second starting clue, a terminal clue, and second hunting requirement information, perform bidirectional search in the preset graph structure from a second starting node indicated by the second starting clue and a first terminal node indicated by the terminal clue based on the second hunting requirement information through a bidirectional search algorithm, and count the number of current search objects in the search process;

[0071] The search module is further configured to, if the number of current search objects is greater than a second preset number threshold, determine a third starting node and a second terminal node based on a current search result of the bidirectional search algorithm, and determine a threat path between the third starting node and the second terminal node through an A-star search algorithm.

[0072] The result acquisition module is further configured to determine a target threat path based on the threat path between the second starting node and the third starting node, the threat path between the first terminal node and the second terminal node, and the threat path between the third starting node and the second terminal node.

[0073] Optionally, the current search result includes a forward local optimal path and a reverse local optimal path; and the search module is further configured to:

[0074] determine a length difference value between the length of the reverse local optimal path and the length of the forward local optimal path, and calculate a ratio of an absolute value of the length difference value to the length of the forward local optimal path;

[0075] when the ratio is less than or equal to a preset ratio, determine a last node of the forward local optimal path, take the last node of the forward local optimal path as the third starting node, determine a last node of the reverse local optimal path, and take the last node of the reverse local optimal path as the second terminal node; or

[0076] When the ratio is greater than a preset ratio, a last node of the forward local optimal path is determined, the last node of the forward local optimal path is taken as the third starting node, and the first terminal node is taken as the second terminal node.

[0077] Optionally, the apparatus further comprises:

[0078] The community division module is configured to perform community division on the preset graph structure by a community discovery algorithm to obtain a plurality of community division results in response to a threat community identification task execution instruction.

[0079] The community threat type output module is configured to input each community division result into a preset graph neural network respectively, calculate a graph-level vector, a node-level vector and an edge-level vector corresponding to the community division result by the preset graph neural network, splice the graph-level vector, the node-level vector and the edge-level vector to obtain a target fusion vector corresponding to the community division result, and output a community threat type corresponding to the community division result according to the target fusion vector.

[0080] Optionally, the apparatus further comprises:

[0081] The graph increment query module is configured to determine an incremental substructure by a graph increment query algorithm according to the current preset graph structure and a preset graph structure corresponding to a last time when a task is executed according to a preset time period, and continue to execute a corresponding task in the incremental substructure until a task end instruction is received.

[0082] Optionally, the apparatus further comprises:

[0083] The display module is configured to generate a result display page based on a task execution result after any task execution ends, and color mark the task execution result on the display page based on a preset marking rule, wherein different colors are used to indicate different threat levels and / or attributes.

[0084] It should be noted that other corresponding descriptions of the various functional units involved in the network data threat hunting apparatus provided by the embodiments of the present application can be referred to the corresponding descriptions in the method. Figure 1 The corresponding descriptions in the method will not be repeated here.

[0085] The embodiments of the present application also provide a computer device, which can be a personal computer, a server, a network device, etc. Figure 3As shown, the computer device includes a bus, a processor, a memory and a communication interface, and can further include an input / output interface and a display device. The processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for running the operating system and the computer program in the non-volatile storage medium. The database of the computer device is configured to store location information. The network interface of the computer device is configured to communicate with an external terminal through a network connection. The computer program is executed by the processor to implement the steps in the method embodiments.

[0086] Those skilled in the art can understand that, Figure 3 The structure shown in the figure is only a block diagram of part of the structure related to the scheme of the present application, and does not constitute a limitation on the computer device to which the scheme of the present application is applied. The specific computer device can include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0087] In one embodiment, a computer readable storage medium is provided, which can be non-volatile or volatile, and has stored thereon a computer program. The computer program is executed by a processor to implement the steps in the method embodiments described above.

[0088] In one embodiment, a computer program product is provided, which includes a computer program. The computer program is executed by a processor to implement the steps in the method embodiments described above.

[0089] It should be noted that the user information (including but not limited to user equipment information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data authorized by the user or authorized by all parties.

[0090] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer readable storage medium, and when the computer program is executed, the processes of the above-mentioned embodiments of the methods can be included. Any reference to memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (Read-Only Memory, ROM), magnetic tape, floppy disk, flash memory, optical storage, high-density embedded non-volatile memory, resistive memory (ReRAM), magnetoresistive random access memory (Magnetoresistive Random Access Memory, MRAM), ferroelectric memory (Ferroelectric Random Access Memory, FRAM), phase change memory (Phase Change Memory, PCM), graphene memory, etc. Volatile memory can include random access memory (Random Access Memory, RAM) or external cache memory, etc. As an illustration but not limitation, RAM can be in various forms, such as static random access memory (Static Random Access Memory, SRAM) or dynamic random access memory (Dynamic Random Access Memory, DRAM), etc. The database involved in the embodiments provided in the present application can include at least one of a relational database and a non-relational database. The non-relational database can include a distributed database based on a block chain, etc., without being limited thereto. The processor involved in the embodiments provided in the present application can be a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., without being limited thereto.

[0091] Any combination of the technical features of the above embodiments can be made. In order to make the description simple, all possible combinations of the technical features in the above embodiments are not described, however, as long as the combination of the technical features does not exist contradictory, it should be considered as the scope of the present application.

[0092] The above embodiments only express several implementation manners of the present application, and the description is more specific and detailed, but it should not be understood as a limitation on the scope of the patent of the present application. It should be pointed out that for ordinary skilled in the art, without departing from the concept of the present application, a number of modifications and improvements can be made, which are within the scope of protection of the present application. Therefore, the protection scope of the present application should be subject to the appended claims.

Claims

1. A threat hunting method of network data, characterized by, The method comprises: in response to a clue expansion task execution instruction, obtaining a first starting clue and first hunting requirement information, according to the first hunting requirement information, searching in a preset graph structure from a first starting node indicated by the first starting clue by a target search algorithm, and in the searching process, counting the number of current search objects, and judging the size relationship between the number of current search objects and a first preset number threshold, wherein the preset graph structure comprises a plurality of nodes and associated edges between different nodes, the nodes are constructed based on network data, and the associated edges are used to indicate the association relationship between the network data, and the current search object comprises a node object and / or an associated edge object; if the number of current search objects is greater than the first preset number threshold, classifying the current search objects based on a preset classification rule, assigning a processing resource to each classification, and taking the current search object under the corresponding classification as the starting point, according to the first hunting requirement information, continuing to search in the preset graph structure by the target search algorithm, continuing to count the number of corresponding current search objects in the searching process and continuing to judge until the preset end condition is met; when each of the processing resources meets the preset end condition, obtaining the threat hunting result corresponding to the first starting clue according to the search object meeting the first hunting requirement information.

2. The method of claim 1, wherein, When the preset classification rule is a similarity classification rule, the classification of the current search objects based on the preset classification rule comprises: for each current search object, determining a target associated object in a preset hierarchy corresponding to the current search object, and determining a vector representation corresponding to the current search object based on the current search object and the target associated object, wherein the target associated object comprises a target associated node and a target associated edge; based on the vector representation corresponding to each current search object, clustering processing is performed on each current search object to obtain a plurality of clustering clusters, and each current search object included in each clustering cluster is taken as a classification.

3. The method of claim 1, wherein, The method further comprises: in response to a threat path identification task execution instruction, obtaining a second starting clue, a terminal clue and second hunting requirement information, according to the second hunting requirement information, starting from a second starting node indicated by the second starting clue and a first terminal node indicated by the terminal clue in the preset graph structure by a bidirectional search algorithm, and in the searching process, counting the number of current search objects; if the number of current search objects is greater than a second preset number threshold, determining a third starting node and a second terminal node based on the current search result of the bidirectional search algorithm, and determining a threat path between the third starting node and the second terminal node by an A-star search algorithm; based on the threat path between the second starting node and the third starting node, the threat path between the first terminal node and the second terminal node, and the threat path between the third starting node and the second terminal node, a target threat path is determined.

4. The method of claim 3, wherein, The current search result includes a forward local optimal path and a reverse local optimal path; and the third starting node and the second ending node are determined based on the current search result of the bidirectional search algorithm, including: determining a length difference between the length of the reverse local optimal path and the length of the forward local optimal path, and calculating a ratio between the absolute value of the length difference and the length of the forward local optimal path; when the ratio is less than or equal to a preset ratio, determining a last node of the forward local optimal path as the third starting node, and determining a last node of the reverse local optimal path as the second ending node; or when the ratio is greater than the preset ratio, determining a last node of the forward local optimal path as the third starting node, and determining the first ending node as the second ending node.

5. The method of claim 1, wherein, The method further includes: in response to a threat community identification task execution instruction, performing community division on the preset graph structure through a community discovery algorithm to obtain a plurality of community division results; inputting each community division result into a preset graph neural network respectively, calculating a graph-level vector, a node-level vector and an edge-level vector corresponding to the community division result through the preset graph neural network, splicing the graph-level vector, the node-level vector and the edge-level vector to obtain a target fusion vector corresponding to the community division result, and outputting a community threat type corresponding to the community division result according to the target fusion vector.

6. The method according to any one of claims 1 to 5, characterized in that, The method further includes: According to a preset time period, according to the current preset graph structure and the corresponding preset graph structure when the last time the task is executed, determine the incremental substructure through the graph incremental query algorithm, and continue to execute the corresponding task in the incremental substructure until the task end instruction is received.

7. The method according to any one of claims 1 to 5, characterized in that, The method further includes: When any task execution ends, a result display page is generated based on the task execution result, and the task execution result on the display page is color marked based on a preset marking rule, wherein different colors are used to indicate different threat levels and / or attributes.

8. A threat hunting apparatus of network data, characterized by, including: An information acquisition module is configured to, in response to a clue expansion task execution instruction, acquire a first starting clue and first hunting demand information, and based on the first hunting demand information, perform a search in a preset graph structure starting from a first starting node indicated by the first starting clue through a target search algorithm, and in the search process, count the number of current search objects and determine the size relationship between the number of current search objects and a first preset number threshold, wherein the preset graph structure includes a plurality of nodes and associated edges between different nodes, the nodes are constructed based on network data, and the associated edges are used to indicate the association relationship between the network data, and the current search objects include node objects and / or associated edge objects; The search module is configured to, if the number of current search objects is greater than a first preset number threshold, classify the current search objects based on a preset classification rule, assign a processing resource to each classification, and start searching in the preset graph structure based on the first hunting demand information and the current search objects in the corresponding classification by using the target search algorithm. The number of current search objects is continuously counted and the judgment is continuously performed during the searching process until the preset ending condition is met. The result obtaining module is configured to, when each processing resource meets the preset ending condition, obtain a threat hunting result corresponding to the first starting clue based on the search objects meeting the first hunting demand information.

9. A storage medium having stored thereon a computer program, characterized in that The computer program is executed by the processor to implement the method in any one of claims 1 to 7.

10. A computer device comprising a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, characterized in that, The processor executes the computer program to implement the method in any one of claims 1 to 7.

Images