A terminal side alarm tracing graph filtering method based on a graph neural network clustering

Abstract

This application relates to a terminal-side alarm source tracing graph filtering method based on graph neural network clustering. The method includes: constructing an initial encoder by training a graph node autoencoder and a graph autoencoder for the terminal-side alarm source tracing graph; optimizing the initial encoder through an alarm metric prediction task; training a clustering model by obtaining the optimal number of clusters using an adaptive clustering algorithm; labeling cluster categories using the clustering model and training an alarm classification model; filtering alarms using the alarm classification model and monitoring the proportion of "new alarms" to guide subsequent model iterations. The terminal-side alarm filtering method based on graph neural network clustering proposed in this application can improve the automation and accuracy of alarm analysis, solving the problem of difficulty for network security personnel in identifying key alarms due to a large number of redundant alarms.

Description

Technical Field

[0001] This invention relates to the field of alarm filtering in computer security, and in particular to a terminal-side alarm source tracing graph filtering technology based on graph neural networks and clustering models. Background Technology

[0002] With the development of technology, network service systems have become increasingly large-scale and complex in design, leading to a continuous increase in the number and complexity of network security incidents. Many of the numerous security alerts generated daily may be false alarms caused by improper system configuration, compatibility issues, or other non-malicious factors, causing truly critical alerts to be overlooked. This increases the potential risks to the system, making the filtering of redundant alerts a meaningful task.

[0003] Endpoint Detection and Response (EDR) is a next-generation attack defense tool designed to detect anomalous behavior on endpoint systems and provide a means of monitoring advanced threats. Its main features include: 1. Utilizing existing blacklists and virus signature identification technologies to provide static protection against known threats; 2. Proactively detecting various security risks by combining cloud-based threat intelligence, machine learning, anomalous behavior analysis, and attack indicators; 3. Conducting in-depth reviews of the endpoint's data background, malware activity, and the entire lifecycle of advanced threats, while also possessing automated interception, forensics, remediation, and tracing capabilities, thereby comprehensively strengthening endpoint security. Furthermore, EDR can clearly demonstrate the dependencies between processes during an attack, such as how processes affect other processes through creation, memory sharing, and communication; the potential associations formed when different processes operate on the same file or object; and the read / write relationships between processes and files, generating alert source maps.

[0004] Traditional alert handling methods often classify and filter alerts based on explicitly defined rules. While effective against common and predictable threats, these methods struggle with novel, complex, or unknown alerts. Manual alert review, though allowing for deeper analysis and judgment, is insufficient to handle thousands of alerts effectively with only human resources. In recent years, deep learning has offered solutions for alert filtering. Unlike traditional methods, deep learning learns from data rather than relying on predefined rules. Combined with natural language processing and other technologies, deep learning can encode and classify alerts more meticulously, significantly reducing false positives and false negatives. Subsequent review by the security team after filtering saves considerable manpower. These alert filtering techniques largely share a similar approach: first, deep encoding of alerts, then efficient filtering using clustering techniques. Given that malicious behavior often achieves its malicious goals through a series of processes, alert source graphs effectively represent these processes. Representing alert source graphs allows the encoder to represent more alert information.

[0005] Compared to related technologies, this method utilizes autoencoder technology and specific task training to obtain a superior alarm encoder module for encoding alarm source maps generated by EDR; it uses an adaptive clustering selection algorithm to select an optimal number of clusters to train a clustering model; it trains a classification model to define rules for "new alarm" categories; and it establishes a model update scheme to guide iterative updates. This approach is an advancement and optimization of deep learning filtering schemes, which will better solve alarm filtering problems and improve the work efficiency of cybersecurity personnel. Summary of the Invention

[0006] This application proposes a terminal-side alarm source graph filtering method based on graph neural network clustering. The purpose is to more efficiently cluster and filter alarm source graphs generated by EDR tools to solve the problem of network security personnel having difficulty identifying key alarms due to a large number of redundant alarms.

[0007] This application provides a terminal-side alarm source tracing graph filtering method based on graph neural network clustering, the steps of which include:

[0008] Step S101: Train the node autoencoder and graph autoencoder of the alarm source graph using node information and graph structure information to complete the initial construction of the encoder.

[0009] Step S102: Use alarm indicator information to train node autoencoders and graph autoencoders to obtain alarm graph encoders, thereby improving alarm feature extraction capabilities.

[0010] Step S103: Cluster the alarm dataset using a clustering algorithm. When different numbers of clusters are selected, the optimal number of clusters is chosen based on clustering metrics to train the clustering model.

[0011] Step S104: Set the clustering results as "classification labels" and train the alarm classification model.

[0012] Step S105: Classify the actual alarms within the specified window time using a classification model, define the "new alarm" category. If an alarm differs significantly from the classified category, it is placed in the "new alarm" category. Finally, the alarms displayed after filtering by the system are the custom number of alarms under each "classification label" category and all alarms in the "new alarm" category.

[0013] Step S106: Determine the "most recent update time of the model". When the proportion of alarms in the "new category" exceeds the threshold and the current time exceeds the "most recent update time of the model", repeat S101-S105 to build a new classification model.

[0014] The invention mainly proposes solutions to the following problems:

[0015] To enhance the feature extraction capability of the alarm encoder, this invention optimizes the encoder construction scheme. In step S101, node autoencoders and graph autoencoders are used to deeply mine various characteristics in the alarm data. In S102, a prediction task is designed using alarm indicator information to train the alarm encoder, ensuring that the encoder has a more comprehensive and in-depth understanding of the alarm information. Furthermore, the node autoencoder can also encode and compress alarm nodes, solving the problem of excessive computational complexity during the training process of graph neural networks.

[0016] To address the issue of unclear cluster numbers in alarm clustering, this invention employs an adaptive clustering algorithm to determine the optimal number of clusters. In step S103, the clustering effect of alarm data is evaluated under different cluster numbers, and the optimal cluster number is selected, thus avoiding poor clustering results caused by a fixed cluster number.

[0017] To address the difficulty in defining the category of "new alarms," ​​this invention, based on clustering results, assigns classification labels to the data in each cluster in step S104, and then trains a classification model using these labels. Whenever a new alarm is generated, this model can accurately classify it into a specific category based on the alarm's characteristics. Although clustering itself has a classification function, using clustering methods for classification requires determining whether an alarm is a "new alarm" based on the "distance" from the alarm vector to the center of its cluster, and determining the threshold for this "distance" is difficult. In contrast, the classification model can provide a probability output of the classification results, making it easier to determine the similarity between an alarm and its classification label, and classifying alarms with low similarity as new. This scheme supports the "new alarm" definition strategy in step S105 and the model update strategy in step S106. Attached Figure Description

[0018] The following sections will describe some specific embodiments of the invention in detail by way of example and not limitation, with reference to the accompanying drawings. The drawings illustrate the steps and main algorithms of the method in the form of flowcharts and schematic diagrams. Those skilled in the art should understand that these drawings are not necessarily directly applicable. The objectives and features of the invention will become more apparent from the following description taken in conjunction with the accompanying drawings, in which:

[0019] Figure 1 This is a flowchart illustrating the terminal-side alarm source tracing graph filtering method based on graph neural network clustering.

[0020] Figure 2 This is a schematic diagram of the construction of the terminal-side alarm tracing diagram.

[0021] Figure 3 Diagram of the autoencoder structure

[0022] Figure 4 Alarm source tracing diagram coding illustration

[0023] Figure 5 A schematic diagram of labeling after clustering.

[0024] Figure 6 Classification model structure diagram

[0025] Figure 7 Alarm filtering diagram Detailed Implementation

[0026] To make the key points of this invention more apparent and understandable, the invention will be further described below with reference to the accompanying drawings and examples. Many details and specific examples are set forth in the following description. These examples are provided to enable a more thorough understanding of the invention and to fully and vividly convey its contents to those skilled in the art. Although the invention can be implemented in many other ways different from those described herein, those skilled in the art can make corresponding extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific examples and drawings disclosed below.

[0027] Figure 1 This is a flowchart illustrating a terminal-side alarm source tracing graph filtering method based on graph neural network clustering. Figure 1 This method includes the following steps:

[0028] Step S101: Train the node autoencoder and graph autoencoder of the alarm tracing graph on the terminal side using node information and graph structure information respectively.

[0029] Specifically, a terminal-side alarm source graph refers to the causal dependencies between the process that triggered the alarm and other processes, files, etc., at the time the alarm is triggered. The source graph provides system administrators and security experts with an intuitive way to identify potential root causes of problems and assess the scope of an alarm's impact. For example, if the relevant system logs for the alarm process are: time0: process A creates process B; time1: process B writes file 1; time2: process B writes file 2; time3: process A reads file 0; time4: process A creates process C; time5: process C reads file 1; time6: process C writes file X; time7: process C reads file 2; time8: process A creates process X. The writing to file X is identified as a sensitive behavior, and EDR or other terminal-side tools will use the log information to generate a source graph such as... Figure 2 Alarm source tracing diagram.

[0030] Furthermore, security tools such as EDR typically establish rules for alert levels and alert scores when detecting attacks. These rules help security personnel quickly locate important alerts, and these metrics are crucial for training the encoder in this solution. Moreover, the strategies for establishing these scores and alert levels are generally based on, but not limited to, the following: 1. Behavioral analysis: Certain sensitive behaviors or events are important evidence that the system is under threat; 2. Asset value: Scoring based on the importance of affected files or systems; 3. Attack chain analysis: Assessment based on the existence of a certain stage in the attack chain; 4. AI scoring: Scoring activities or behaviors based on trained machine learning models.

[0031] The goal of a node autoencoder is to learn embedding representations for nodes and to accelerate the training speed of graph neural networks by reducing data dimensionality, thereby reducing the computational complexity of subsequent graph neural network operations. First, all node information is extracted from the dataset. Textual attributes of nodes, such as path, pName, cmdLine, and user, are converted into vector representations using a pre-trained BERT model; other attributes, such as category attributes, are encoded using one-hot encoding. The autoencoder consists of an encoder and a decoder. The encoder compresses the original features of the nodes to a smaller dimension, while the decoder attempts to reconstruct the original node features from this compressed representation. During training, the model is optimized by minimizing the mean squared error between the input features and the decoder output. For node autoencoder training, the encoder part consists of fully connected layers and non-linear activation functions; the decoder structure is the opposite of the encoder, its input is the encoded result, and its output is a reconstruction of the original data. Figure 3 As shown, for the graph autoencoder, the encoder consists of two layers of graph convolutional neural networks (GCNs) to capture the topological structure and attribute information of nodes in the graph. The first GCN layer receives the original features of the nodes, i.e., the vectors a, b, c, etc. (in_dim dimension) encoded by the node autoencoder, and outputs an intermediate representation of the hidden_dim dimension. The second GCN layer continues to process the output of the first layer and generates the final graph embedding of the embedding_dim dimension. The decoder's task is similar to that of the node autoencoder, aiming to reconstruct the original features of the nodes from the graph embedding. The decoder consists of two fully connected layers. The first layer extends the graph embedding from the embedding_dim dimension to the hidden_dim dimension, and the second layer further extends the embedding from the hidden_dim dimension to the in_dim dimension, i.e., the original encoded vectors a, b, c, etc. In the training of the graph autoencoder, the mean squared loss is used to measure the reconstruction effect of the nodes, and the Adam method is used to train the model.

[0032] Step S102: Use alarm indicators to collaboratively train the node encoder and graph encoder to improve the overall feature extraction capability of the encoder.

[0033] Specifically, the alarm data includes alarm evaluation metrics set by security experts, such as alarm level and alarm score. In this step, we use the alarm score prediction task as the encoder training task to further optimize the alarm encoder's capabilities, giving it superior alarm feature extraction abilities. It uses the node autoencoder and graph autoencoder trained by S101 as the initial encoder, inputs the encoding results into a fully connected layer, then uses the ReLU activation function to input them into another fully connected layer, outputting the predicted value. The mean squared loss MSE = 1 / n*∑(yi-yi′)^2 is used as the loss function, and the backpropagation algorithm is used for model training.

[0034] Step S103: Cluster the alarm dataset using a clustering algorithm, and select the optimal number of clusters by observing clustering indicators.

[0035] Specifically: This is the process of selecting an optimal number of clusters before training the clustering task. In this step, the K-Means clustering method is first used to cluster the encoded alarm data under different numbers of clusters. Then, based on the silhouette coefficient, the clustering effect under each number of clusters is calculated, and the clusters with higher silhouette coefficients are selected as the optimal number of clusters. The clustering model is then trained using the selected optimal number of clusters. The clustering process is as follows:

[0036] First, K data points are randomly selected as the initial cluster centers. Then, each data point is assigned to the nearest cluster center, where "nearest" is calculated based on a chosen distance metric (such as Euclidean distance). After this step, each data point is assigned to a cluster. Then, for each defined cluster, the average of all its data points is calculated, and this average is set as the new cluster center. The new center is compared with the center from the previous iteration. If there is no change, the algorithm is considered to have converged, and the iteration is terminated. If the center has changed, the previous steps are repeated.

[0037] Step S104: Label the clustering results as “classification labels” and train the alarm classification model.

[0038] Specifically: Figure 4The diagram illustrates the encoding process of an alarm using a trained graph encoder. Mark 1 in the diagram represents the encoding vector of a node. After encoding a node, for each node in the graph, the graph neural network collects information from all its neighbors, aggregating the information of surrounding nodes into that node. Figures 2 and 3 show the information aggregation process of the graph neural network for node e, with arrows indicating the aggregation process towards node e. First, after passing through a graph convolutional network, node e is updated to a weighted sum of its original information and the information of its neighboring nodes, thus obtaining the information of its neighboring nodes. Simultaneously, nodes b and c also obtain their neighboring node information in the same way. At this point, the information of node d is passed to b and c. Then, through a second graph neural network aggregation process, the information of d is indirectly passed to e. Through this process, node e obtains the node information of its surrounding field of view. Process 4 in the diagram represents the concatenation of the information from each graph node, ultimately forming a one-dimensional embedding of the alarm source graph. Figure 5 This demonstrates the process of clustering alarms using the optimal clustering algorithm across various alarm source maps, and then labeling the data within each cluster based on the clustering results. Next, a classifier is trained using this labeled data; the structure of the classifier model is as follows. Figure 6 As shown, the encoded result is transformed into a probability output through two fully connected layers and one softmax layer. Activation functions are included between layers. Cross-entropy loss is used as the loss function during training. The final output is the classification probability of each category, and the category with the highest probability is selected as the classification result. If the classification probability is less than a threshold x, it is placed in the "new alarm" category. This scheme does not directly use a clustering model for classification because direct classification usually requires determining the definition strategy of "new alarm" by calculating the "distance" between the alarm and the cluster center. This distance threshold is often difficult to measure. The classifier can more intuitively evaluate the matching degree between the alarm data and each category through the output category probability, thus making it easier to formulate a strategy for defining "new alarms".

[0039] Step S105: Filter alarms using the trained classification model.

[0040] Specifically: The newly arrived alarms are classified using the trained alarm classification model, as follows: Figure 7 As shown, newly generated alarms will be assigned to the best-matching known category. Alarms whose predicted probability by the classification model is less than a threshold will be assigned to the "New Alarm" category. The filtering strategy is as follows: for each known category, a portion of the alarms within a specified window period will be retained, with the alarm display ratio being a custom value n. Within each category, a (1-n) proportion of alarms will be filtered. However, categories marked as "New Alarms" will not be filtered due to their special characteristics.

[0041] Step S106: Formulate a model update strategy. When the model update conditions are met, update the alarm filter.

[0042] Specifically: Establish a "model latest update time" rule to prevent the model from being updated too frequently. When the proportion of alarms in the "new category" in S105 exceeds the specified threshold x, a large number of new alarms may have been generated within the window time. At this time, it is considered that the classification model can no longer meet the requirements in filtering alarms. When the current time exceeds the "model latest update time", repeat S101-S105 to build a new classification model.

[0043] While the present invention has been described with reference to specific illustrative embodiments, it is not limited to these embodiments but only to the appended claims. Those skilled in the art will understand that modifications and alterations can be made to the embodiments of the invention without departing from the scope and spirit of the invention.

Claims

1. A terminal-side alarm traceability graph filtering method based on graph neural network clustering, characterized by Includes the following steps: Step S101: Initial construction of the alarm source graph encoder. Use BERT to encode the Linux command text attributes in the nodes and use one-hot encoding to encode the file category attributes. Then, train the node autoencoder and graph autoencoder of the terminal-side alarm source graph using node and graph structure information respectively. Step S102: Use alarm metrics to collaboratively train node autoencoders and graph autoencoders to improve the overall feature extraction capability of the alarm encoder. Step S103: Cluster the alarm dataset using a clustering algorithm. When different numbers of clusters are selected, the optimal number of clusters is selected by comparing clustering metrics to train the clustering model. Step S104: Set the clustering results as "classification labels" and train the alarm classification model; Step S105: The actual alarms within the specified window time are classified and filtered by the classification model. When the alarm is significantly different from the classified category, it is placed in the "new alarm" category. Finally, the alarms output after system filtering are the custom proportion of alarms under each "classification label" category and all alarms in the "new alarm" category. Step S106: Define the "most recent update time of the model". When the proportion of alarms in the "new category" exceeds the threshold and the current time exceeds the "most recent update time of the model", repeat S101-S105 to build a new classification model.

2. The terminal-side alarm source filtering method based on graph neural network clustering according to claim 1, characterized in that... The specific method for collaboratively training the node autoencoder and graph autoencoder using alarm indicator information in step S102 is as follows: Design a prediction task using the alarm score and alarm level indicators in the alarm dataset, and collaboratively train the node autoencoder and graph autoencoder. After this step is completed, the encoder part of the alarm is obtained.

3. The terminal-side alarm source tracing graph filtering method based on graph neural network clustering according to claim 1, characterized in that... The specific method for selecting the optimal number of clusters in step S103 is as follows: by using the K-Means clustering method, different numbers of clusters are selected to cluster the encoded alarm data, the clustering effect under each number of clusters is calculated based on the silhouette coefficient, and the number of clusters with the best index is selected as the final number of clusters to train the clustering model.

4. The terminal-side alarm source filtering method based on graph neural network clustering according to claim 1, characterized in that... The specific method for training the alarm classification model in step S104 is as follows: based on the clustering results, the data of each cluster is labeled, and a classification model is trained using this labeled dataset. When there is a new alarm input, the model assigns it to the corresponding category according to the characteristics of the alarm.

5. The terminal-side alarm source tracing graph filtering method based on graph neural network clustering according to claim 1, characterized in that... The specific method of the alarm filtering scheme in step S105 is as follows: the newly entered alarms are classified using the trained alarm classification model. The alarms will be assigned to the most matching known category. For alarms that the classification model identifies as "new alarms", that is, the classification result probability is less than the threshold, they will be assigned to the "new alarm" category. The filtering strategy is as follows: for known categories, a portion of the alarms within a specified window time are retained, and too many redundant alarms will be filtered out. For categories marked as "new alarms", no filtering will be performed.

Images