Real-time attack detection method and system based on label transfer and event baseline learning

By adopting a method based on tag delivery and event baseline learning in the traceability map, combining event outlier prediction model and tag propagation, the real-time and accuracy of abnormal detection in the traceability map is solved, and efficient abnormal path detection and alarm are achieved. Rich info.

CN118827248BActive Publication Date: 2024-11-19ZHEJIANG UNIV
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
CN202411313350.3
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-09-20
Publication Date
2024-11-19
Estimated Expiration
2044-09-20

AI Technical Summary

Technical Problem

When the prior art performs abnormal detection in the traceability map, it is difficult to meet the requirements of processing massive data in real time, and it is difficult to build a baseline for normal behavior, which makes it difficult to compatible with the accuracy and effectiveness of the detection.

Method used

Using real-time attack chain detection method based on label delivery and event baseline learning, the event anomalies are judged through the event outlier prediction model, and real-time exception path mining and detection are realized in combination with label propagation, which reduces the construction and query overhead of the baseline model and realizes the implementation of Better generalization and detection effects.

Benefits of technology

Real-time and accurate abnormal path detection in the traceability map is realized, the alarm volume is reduced, the alarm information is enriched, and the detection efficiency and accuracy are improved.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118827248B_ABST
    Figure CN118827248B_ABST
Patent Text Reader

Abstract

This solution provides a real-time attack chain detection method and system based on label transfer and event baseline learning. Labels are designed, existing labels are propagated to the traceability graph nodes corresponding to the current event and the labels are updated, and the updated labels are cached in the abnormal path key-value index structure, wherein the label update rule is as follows: the event vector in the label is updated according to the event vector of the current event, the abnormal path score of the current event is calculated, the abnormal path is updated according to the propagation path between the traceability graph nodes of the last propagation and the traceability graph nodes corresponding to the current event, an event baseline learning model is designed to replace the traditional event-based probability security baseline judgment, and real-time and accurate abnormal path mining and attack detection are realized based on stream processing and label transfer.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of network attacks, and in particular to a real-time attack chain detection method and system based on label transfer and event baseline learning. Background Art

[0002] Digitalization, especially the continuous advancement of cloud computing technology, continues to inject new impetus into social and economic development, but it also introduces new risks. In recent years, the rapid evolution of cyber attacks has made intrusion detection increasingly challenging. In response to the difficulty of complex attack detection, host threat analysis technology based on the concept of provenance has received attention and application in academia and industry at home and abroad for many years. Causal analysis / provenance analysis based on coarse-grained information flow, also known as causality, can effectively characterize the operations between entities in the system, and widely associate events that occur in the system (including: behavior subject s, operation v, behavior object o, time t quadruple), construct a global system behavior model, and show good detection and analysis effects on the above complex attacks.

[0003] Anomaly detection and rule-based detection are the two main methods of attack detection in the field of network security. Among them, anomaly detection has the advantages of no need for manual rule configuration, support for unknown threat detection, and high degree of automation, and has been widely used. However, anomaly detection in the traceability graph requires real-time processing of massive data, which places high demands on the overhead of the algorithm model. The current anomaly detection system is difficult to meet this requirement. In addition, anomaly detection requires the pre-construction of a baseline behavior model representing normal behavior. The current main normal behavior baseline construction methods include graph neural network models based on local behavior subgraphs and baseline event databases that collect normal behavior events. The former solution artificially divides the traceability graph into multiple subgraphs, resulting in the constructed graph neural network model being easily bypassed by long-term and spatial attack chains, and the graph neural network analysis overhead is high, making it difficult to apply in real time; the latter solution requires recording a large number of normal behavior event quadruplets, which takes up a large amount of space, and it is difficult to design a general generalization strategy to be compatible with newly emerging events (mainly subjects and objects). Otherwise, processing untrained data is prone to false alarms.

[0004] In summary, there are currently problems with anomaly detection in traceability graphs, such as the difficulty in constructing a normal behavior baseline behavior model, and the difficulty in reconciling the effectiveness and accuracy of anomaly detection. Summary of the invention

[0005] The embodiments of the present application provide a real-time attack chain detection method and system based on label transfer and event baseline learning, design an event baseline learning model to replace the traditional event probability-based security baseline judgment, and realize real-time and accurate abnormal path mining and attack detection based on stream processing and label transfer.

[0006] In the first aspect, the embodiment of the present application provides a real-time attack chain detection method based on label transfer and event baseline learning, including: real-time event collection and event embedding encoding to obtain event vectors, inputting the event vectors into a pre-trained event anomaly value prediction model to output event anomaly values, wherein the event anomaly value prediction model is trained through events and corresponding event anomaly values; if the event anomaly value exceeds a threshold, initializing and setting a label on the traceability graph node corresponding to the event and establishing an abnormal path key value index structure, wherein the abnormal path key value index structure caches key-value pairs with the traceability graph node as the key and the label as the value, and each The key-value pair corresponds to a unique label propagation index, in which the label ID, event vector, abnormal path score and abnormal path are set in the label; if the label is found to exist in the abnormal path key-value index structure, the existing label will be propagated to the traceability graph node corresponding to the current event and the label will be updated, and the updated label will be cached in the abnormal path key-value index structure, where the label update rule is: update the event vector in the label according to the event vector of the current event, calculate the abnormal path score of the current event, and update the abnormal path according to the propagation path between the traceability graph node of the last propagation and the traceability graph node corresponding to the current event.

[0007] In the second aspect, the embodiment of the present application provides a real-time attack chain detection system based on label transfer and event baseline learning, including a data acquisition system for real-time event acquisition; an event anomaly value prediction system for embedding and encoding events to obtain event vectors, and inputting the event vectors into a pre-trained event anomaly value prediction model to output event anomaly values, wherein the event anomaly value prediction model is trained through events and corresponding event anomaly values; an attack judgment system for initializing and setting labels on the traceability graph nodes corresponding to the events when the event anomaly value exceeds a threshold value and establishing an abnormal path key value index structure, wherein the abnormal path key value index structure caches the traceability graph nodes A key-value pair with the tag as the key and the label as the value, and each key-value pair corresponds to a unique label propagation index, in which the label ID, event vector, abnormal path score and abnormal path are set in the label; if it is found that the label already exists in the abnormal path key-value index structure, the existing label will be propagated to the traceability graph node corresponding to the current event and the label will be updated, and the updated label will be cached in the abnormal path key-value index structure, where the label update rule is: update the event vector in the label according to the event vector of the current event, calculate the abnormal path score of the current event, and update the abnormal path according to the propagation path between the traceability graph node of the last propagation and the traceability graph node corresponding to the current event.

[0008] In a third aspect, an embodiment of the present application provides an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the real-time attack chain detection method based on label transfer and event baseline learning.

[0009] In a fourth aspect, an embodiment of the present application provides a readable storage medium, in which a computer program is stored. The computer program includes a program code for controlling a process to execute a process, and the process includes the real-time attack chain detection method based on label transfer and event baseline learning.

[0010] The main contributions and innovations of the present invention are as follows:

[0011] The embodiment of the present application provides a real-time attack chain detection method and system based on label transfer and event baseline learning, which uses an event outlier prediction model to judge event outliers, and implements real-time mining and detection of abnormal paths based on event outliers combined with label propagation. This solution uses an event outlier prediction model to replace the traditional database to achieve better generalization and detection effects, and designs labels for abnormal paths and abnormal path scores, where the abnormal path score changes according to the change of the path to realize the propagation of labels and realize abnormal path detection of real-time attack chains; at the same time, this solution can merge paths with repeated alarms to reduce the amount of alarms and enrich alarm information.

[0012] Details of one or more embodiments of the present application are set forth in the following drawings and description to make other features, objects, and advantages of the present application more readily apparent. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0014] Figure 1 is a schematic diagram of label propagation of a real-time attack chain detection method based on label transfer and event baseline learning according to an embodiment of the present application;

[0015] Figure 2 is a flow chart of a real-time attack chain detection method based on label transfer and event baseline learning according to an embodiment of the present application;

[0016] Figure 3 is a structural diagram of a real-time attack chain detection system based on label transfer and event baseline learning according to an embodiment of the present application;

[0017] Figure 4This is a schematic diagram of the framework of a real-time attack chain detection system based on label transfer and event baseline learning;

[0018] Figure 5 It is a schematic diagram of the hardware structure of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0019] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with one or more embodiments of this specification. Instead, they are merely examples of devices and methods consistent with some aspects of one or more embodiments of this specification as detailed in the appended claims.

[0020] It should be noted that: in other embodiments, the steps of the corresponding method are not necessarily performed in the order shown and described in this specification. In some other embodiments, the steps included in the method may be more or less than those described in this specification. In addition, a single step described in this specification may be decomposed into multiple steps for description in other embodiments; and multiple steps described in this specification may be combined into a single step for description in other embodiments.

[0021] Before introducing the real-time attack chain detection method and system based on label transfer and event baseline learning provided by this solution, the key words for understanding this solution are explained first:

[0022] Provenance Analysis: Understanding the context and impact of a security incident (such as a malware attack or data breach) by tracing the origin and history of data, files, processes, and operations within a single computer system or network.

[0023] Causality Analysis: In the field of cyberspace security, by identifying and understanding the causal relationship between security events (such as network attacks or system anomalies), analyzing the root causes of the events, how other events are triggered, and how these events and behaviors interact with each other, thereby affecting the security and stability of the entire system.

[0024] Security Baseline: A security baseline is a standardized definition of normal operations and behavior patterns in a specific environment, used to identify and detect abnormal activities or potential security threats.

[0025] Behavioral Learning: In the field of cybersecurity, behavioral learning refers to the use of machine learning and artificial intelligence technology to identify and classify normal and abnormal behavior patterns within the system. This learning method relies on a large amount of historical data and real-time interaction data to establish a behavioral baseline to effectively identify potential malicious activities or strategic attacks.

[0026] Embodiment 1

[0027] This solution provides a real-time attack chain detection method based on label transfer and event baseline learning, including the following steps:

[0028] Events are collected in real time and embedded into codes to obtain event vectors, and the event vectors are input into a pre-trained event anomaly value prediction model to output event anomaly values, wherein the event anomaly value prediction model is trained through events and corresponding event anomaly values;

[0029] If the event abnormal value exceeds the threshold, the label is initialized and set on the traceability graph node corresponding to the event and an abnormal path key value index structure is established, where the abnormal path key value index structure caches key-value pairs with the traceability graph node as the key and the label as the value, and each key-value pair corresponds to a unique label propagation index, where the label ID, event vector, abnormal path score and abnormal path are set in the label;

[0030] If it is found that there is already a label in the abnormal path key-value index structure, the existing label will be propagated to the traceability graph node corresponding to the current event and the label will be updated, and the updated label will be cached in the abnormal path key-value index structure. The label update rule is as follows: update the event vector in the label according to the event vector of the current event, calculate the abnormal path score of the current event, and update the abnormal path according to the propagation path between the traceability graph node of the last propagation and the traceability graph node corresponding to the current event.

[0031] This solution caches labels and traceability graph nodes through the abnormal path key-value index structure, without the need to cache the traceability graph corresponding to the event. At the same time, since the abnormal path score and abnormal path are recorded in the label, the abnormal path mining can be realized through the label. In other words, this solution implements a secondary index structure through the design of labels. A secondary index structure is set in the abnormal path key-value index structure, where the secondary index structure is represented as: traceability graph node → label and abnormal path score → abnormal path, where the traceability graph node → label caches the propagated label in the form of key-value mapping, and records the abnormal path score of the current label, label → abnormal path records the abnormal detection result of the abnormal path, and the transmitted abnormal path is updated and cached each time it is propagated. This solution aims to achieve real-time, efficient and accurate abnormal path detection of the traceability graph through the accumulation of abnormal path scores and the label streaming propagation architecture.

[0032] In the "real-time event collection" step, the event is a triple data consisting of subject, operation and object, or a quadruple data consisting of subject, operation, object and time, wherein the quadruple data or triple data constitutes a traceability graph in chronological order. It should be noted that this solution is suitable for detecting and analyzing abnormal paths of real-time attack chains for systems that generate massive logs in real time. The system's log stream can be represented as a quadruple data stream in the form of a quadruple of subject, operation, object and time, or as a triple data stream in the form of a triple of subject, operation and object. The system analyzes and judges each collected event.

[0033] Preferably, an event is a triplet of data consisting of a subject, an operation, and an object. The reason why time data is ignored is that whether an event is abnormal is often irrelevant to time data.

[0034] In some embodiments, the underlying system log collector is used to collect log streams from the protected object (such as PC, server, etc.) in real time. The log collector can be built based on underlying data collection tools such as eBPF, LTTng, Kernel Module, Event Tracing for Windows, etc.

[0035] It should be noted that this solution can support real-time attack detection of multiple events and can be expanded according to actual needs. The following Table 1 shows the expandable event types:

[0036] Table 1 Extensible event types

[0037]

[0038] In the actual operation of the system, traditional anomaly detection for traceability graphs requires real-time processing of massive data, which places high demands on the algorithm model overhead. Current anomaly detection systems are difficult to meet this requirement. This solution replaces the traditional baseline behavior model of normal behavior with a designed event anomaly prediction model, and combines the event anomaly values ​​obtained based on the event anomaly prediction model with label propagation to achieve real-time abnormal path mining and attack detection. It should be emphasized that this solution changes the event baseline model from the traditional table lookup (event->frequency) to an event anomaly prediction model trained with a table, which simultaneously reduces query overhead, reduces baseline model cache overhead, and achieves necessary event generalization.

[0039] Specifically, in the step of "embedding the event to obtain the event vector", the event consisting of the triple data of subject, operation and object is embedded and encoded to obtain the event vector. The reason why this scheme embeds and encodes the event to obtain the event vector is to enable the event anomaly prediction model to predict the anomaly for the event vector. It should be noted that this scheme can select a variety of embedding encoding methods, as long as the embedding encoding method that can accurately represent the event is acceptable.

[0040] In order to reflect the semantic information of the words in the event and the semantic information of the context structure in the event vector, for the semantic information of the words, due to the existence of special characters in the file name and other issues, it will cause OOV (Out-of-vocabulary) problems. Therefore, in some embodiments, the n-gram method is used to process the file names with special symbols in the event.

[0041] Specifically, this solution can use the n-gram method to convert the file name in the event into a long sequence and decompose the long sequence into multiple subsequences. For file names containing special symbols or unknown words, the n-gram method is used to split them into multiple n-gram units, where n can be 1 (unigram), 2 (bigram), or 3 (trigram). For example, when processing a file name of "the network IP address is: 127.0.0.1" or a process name of "XIM", it is necessary to split it into multiple n-gram units using the n-gram method.

[0042] For example, if the file name is my_file@2023.pdf, the n-gram method is used to process the string of the file name to obtain:

[0043] Unigrams(n=1):m,y,_,f,i,l,e,@,2,0,2,3,.,p,d,f;

[0044] Bigrams(n=2):my,y_,_f,fi,il,le,e@,@2,20,02,23,3.,.p,pd,df;

[0045] Trigrams(n=3):my_,y_f,_fi,fil,ile,le@,e@2,@20,202,023,23.,3.p,.pd,pdf.

[0046] In view of the semantic information of the context structure, this solution introduces position coding to obtain the contextual relationship between words. In some embodiments, the words in the event are embedded with vector coding to obtain word vectors and position coding, and the position coding and word vector of each word are concatenated to obtain a first vector, and the first vector is reduced in dimension to obtain an event vector. It should be noted that since the position coding is introduced for each word, the concatenated vectors will be non-uniform, so this solution reduces the dimension of the first vector to obtain an event vector, and in some cases, PCA dimensionality reduction can be used for dimensionality reduction.

[0047] In the step of "inputting the event vector into the pre-trained event anomaly prediction model to output the event anomaly value, wherein the event anomaly prediction model is trained by the event and the corresponding event anomaly value", the event anomaly value is used to indicate the degree of abnormality of the event, and the event anomaly value prediction model is trained by the event anomaly value and the event to obtain an event anomaly value prediction model that can preliminarily determine the abnormality of the event. This solution uses the event anomaly value prediction model to replace the traditional event database, which can achieve better generalization and detection effects.

[0048] In some embodiments, the abnormal event frequency is used as the event abnormal value, and the event abnormal value is used as the value, and the event vector is used as the key to form a key-value pair to train the event abnormal value prediction model. When the event abnormal value is the abnormal event frequency, the abnormal event frequency is represented by the quotient of the frequency of the abnormal event in the event frequency database and the frequency of the same type of event with the same subject and operation as the current abnormal event but the object is arbitrary, which can be represented by the formula Freq (s, v, o) / Freq (s, v, *), where Freq (s, v, o) represents the abnormal event, and Freq (s, v, *) represents the same type of event with the same subject and operation as the current abnormal event but the object is arbitrary.

[0049] For example, for the abnormal event of (chrome, connect, 255.255.255.255), other events of the same type are expressed as (chrome, connect, *) (where * represents any IP address). It can be understood that the intuitive meaning of (chrome, connect, 255.255.255.255) relative to (chrome, connect, *) is the proportion of IP address 255.255.255.255 in the process of process chrome connecting to the network.

[0050] In some embodiments, the trained event outlier prediction model includes an input layer, a hidden layer, and an output layer. The event vector is input into the input layer and features are extracted in the hidden layer. The output layer outputs the corresponding event outlier based on the features extracted in the hidden layer. In order to enable the event outlier prediction model to better understand the relationship between events and event outliers, this solution introduces negative samples when training the event outlier prediction model, where negative samples refer to event samples with low event outliers. In an embodiment of this solution, when the event outlier is an event frequency, the negative samples are mainly divided into two categories: one is event samples that are not much different from those in the event frequency database, and the other is event samples that are quite different from those in the event frequency database.

[0051] After obtaining the event anomaly prediction model, this solution uses the event anomaly prediction model to predict the event anomaly values ​​collected in real time. However, the occurrence of event anomaly values ​​does not necessarily mean that the current event is abnormal. For example, the event anomaly value (chrome, connect, 255.255.255.255) mentioned in the above example is very small, indicating that the IP address does not appear often, and the corresponding abnormal event frequency should be very low. This low frequency of abnormal events means that the Chrome process is connected to a rare IP address, and it is reasonable for this event to be identified as a suspicious event (here "suspicious" is used instead of "abnormal"), but this alone will indeed result in low accuracy. Therefore, this solution combines event anomaly values ​​based on label propagation to solve the accuracy problem of subsequent detection.

[0052] In other words, it should be noted that the event anomaly value of the initial event when the label is initialized at the beginning is relatively low. As the propagation proceeds, new abnormal events will be continuously introduced in the propagation path, thereby making the abnormal path score higher and higher as the propagation process progresses. When the abnormal path score reaches the set threshold, an alarm will occur and the propagation will no longer occur. When the abnormal path score does not reach the set threshold, the propagation will continue.

[0053] This solution sets the label initialization setting when the event outlier value exceeds the threshold. That is to say, a preliminary judgment is made in advance through the event outlier value prediction model, and then an accurate judgment of the abnormal event is made in combination with label propagation.

[0054] It should be noted that labels play a key role in the streaming calculation of abnormal paths in this solution. This solution caches necessary data through labels to support the calculation process. The labels designed in this solution include label ID, event vector, abnormal path score and abnormal path, where label ID marks the ID of the current label and uses label ID to identify the label; event vector records the event code of the current event, which is used to predict the abnormal value of the event; abnormal path score is used to record the abnormal path score of the current path, and the abnormal path records the propagation path between the traceability graph nodes of the last propagation and the traceability graph nodes corresponding to the current event, which is used to record the abnormal path.

[0055] For example, the content of a tag is: {tag ID: 1; event vector: (43.254.3.67→browser); abnormal path score: (1-nu); abnormal path: (43.254.3.67→browser)}

[0056] In the step of "initializing the label setting on the traceability graph node corresponding to the event", the label ID is set, the event vector of the current event is obtained as the event vector of the label, the path of the traceability graph node corresponding to the event is set as the abnormal path, and the label ID, event vector, abnormal path score and abnormal path are cached in the label to complete the label initialization.

[0057] It should be noted that a unique identifier is designed for each key-value pair in the abnormal path key-value index structure as a label propagation index. The unique identifier is similar to the md5 code of a file to distinguish different key-value pairs. In other words, the purpose of designing the abnormal path key-value index structure in this solution is to quickly locate the existing label status on the event subject and object when processing an event.

[0058] In some embodiments, this solution calculates the rule score corresponding to each event based on the event anomaly value, and then multiplies the rule scores on the abnormal path to obtain the abnormal path score. Specifically:

[0059] The rule score of the current event is calculated according to the event anomaly value of the current event, and the difference between 1 and the rule score is taken as the abnormal path score. Every time a label undergoes label propagation, the rule score is updated by multiplying the rule score by the decay value.

[0060] It should be noted that since the streaming traceability graph is not cached, but labels are used to cache the calculation results and the calculation process, this solution will cache the labels in the abnormal path key-value index structure after completing the label initialization. A secondary index structure is set in the abnormal path key-value index structure, where the secondary index structure is represented as: traceability graph node → label and abnormal path score → abnormal path, where the traceability graph node → label and abnormal path score caches the propagated label in the form of a key-value mapping, and the label → abnormal path records the anomaly detection results of the abnormal path, thereby realizing the recording of the calculation results and the calculation process.

[0061] The label can only be propagated after the initialization setting is completed. The label propagation referred to here is to copy the label after updating the abnormal path and cache it in the abnormal path key value index structure, and then quickly index the abnormal path through the abnormal path key value index structure. In other words, there needs to be a label at the starting point of the traceability graph before the label is propagated, so this solution performs label propagation when it is found that the abnormal path key value index structure already has a label. It should be noted that the traceability graph is a directed graph, but its direction is not necessarily from subject to object, but is determined according to the cause and effect relationship.

[0062] In addition, this solution also needs to determine whether the location to be propagated forms a closed loop. If it does, it will not be propagated. Specifically, in the step of "if the abnormal path key value index structure is found to have a label", this solution determines whether the location to be propagated is a closed loop structure. If it does, the current label propagation is stopped. If it does not form a closed loop, the label is propagated. This solution is designed in this way because some nodes will have frequent interactions. In order to avoid false alarms caused by the accumulation of abnormal scores back and forth in the closed loop, this solution does not propagate the labels of nodes with closed loop structures.

[0063] In some embodiments, if a certain provenance graph node is visited twice by a label of the same label propagation index, it means that the current propagation position forms a closed loop.

[0064] When it is found that the abnormal path key value index structure already has a label, and the propagation position does not form a closed loop, the existing label will be propagated to the traceability graph node corresponding to the current event and the label will be updated.

[0065] In some embodiments, in order to further reduce the cache pressure caused by the label, the real-time attack chain detection method based on label transfer and event baseline learning additionally includes the steps of: setting a label removal condition, and removing the label when the label removal condition is met. The label removal condition is an upper limit on the number of labels or an upper limit on the number of propagation times. If the number of labels is greater than the upper limit on the number of labels, the label is removed, or if the number of times the label is transmitted is greater than the upper limit on the number of propagation times, the label is removed.

[0066] In addition, in some embodiments, the real-time attack chain detection method based on label transfer and event baseline learning additionally includes the step of: triggering an alarm when the abnormal path score of the label reaches a security threshold. Moreover, since the same malicious process or file may form multiple abnormal paths, the real-time attack chain detection method based on label transfer and event baseline learning proposed in this scheme additionally includes the steps of: defining an alarm label when the abnormal path score of the label reaches a set threshold, merging repeated alarm labels in different alarm labels, and forming an attack graph based on the traceability graph nodes where the remaining alarm labels are located. In other words, when the abnormal path score reaches the security threshold, the paths of repeated alarms are merged, thereby reducing the amount of alarms and enriching the alarm information.

[0067] In addition, in some embodiments, when the abnormal path score of the tag decays to a decay threshold during the propagation process, the propagation of the tag is stopped.

[0068] In order to more clearly show the label propagation process on the streaming traceability graph, the following will combine Figure 1 An exemplary description is given. Figure 1 The directed graph connected by arrows in the figure is the streaming traceability graph corresponding to the event. At this time, the browser event is the traceability graph node whose event anomaly value exceeds the threshold. Therefore, the label is initialized and set at the browser traceability graph node. The corresponding label content is: {label ID: 1; event vector: (43.254.3.67→browser); abnormal path score: (1-nu); abnormal path: (43.254.3.67→browser)}; then the new event browser→d.sh is obtained. At this time, it is detected that the abnormal path score of the current event does not exceed the threshold and the abnormal path key value If the index structure already has a label, the label is propagated to the traceability graph node corresponding to the event, and the updated label content is {label ID: 2; event vector: (browser→d.sh); abnormal path score: (1-(nu*α)*nu); abnormal path: (43.254.3.67→browser), (browser→d.sh)}; Similarly, when the event is cron→malware, the updated label content is {label ID: m; event vector: (cron→malware); abnormal path score: (1-(nu*α)*nu)}. m *nu); abnormal path: (43.254.3.67→browser), (browser→d.sh)...(cron→malware)}, at this time, the abnormal path score reaches the alarm threshold and an alarm is triggered; and corresponding to the acquisition of the new event browser→**.doc→MS Office, since **.doc→MS Office are all normal events, the abnormal path score continues to decay, and when the abnormal path score is less than the set value, the current label is no longer propagated.

[0069] The implementation logic of the real-time attack chain detection method based on label transfer and event baseline learning is as follows: Figure 2 As shown. First, input the event (which can be triple data or quadruple data), embed the event to get the event vector, input the event vector into the event anomaly prediction model to output the event anomaly value, where the event anomaly prediction model is trained with the event frequency database; if the event anomaly value exceeds the threshold, initialize the label and establish the abnormal path key index structure; if the abnormal path key index structure is queried to see if there is a label, if so, perform label propagation, and write the propagation result into the abnormal path key index structure, perform label removal for redundant labels, and determine whether the abnormal path score of the label exceeds the security threshold. If so, determine whether a merge alarm is needed in combination with the abnormal path alarm structure, if not, directly generate an alarm, and finally end the processing of the real-time attack chain of an event.

[0070] Embodiment 2

[0071] Based on the same idea, refer to Figure 3 ,This application also proposes a real-time attack chain detection system based on label transfer and event baseline learning, including:

[0072] Data collection system for real-time collection of events:

[0073] An event outlier prediction system is used to embed and encode an event to obtain an event vector, and input the event vector into a pre-trained event outlier prediction model to output an event outlier value, wherein the event outlier prediction model is trained through events and corresponding event outlier values;

[0074] An attack judgment system is used to initialize and set a label on the traceability graph node corresponding to the event and establish an abnormal path key value index structure when the event abnormal value exceeds a threshold, wherein the abnormal path key value index structure caches key-value pairs with the traceability graph node as the key and the label as the value, and each key value pair corresponds to a unique label propagation index, wherein the label ID, event vector, abnormal path score and abnormal path are set in the label; if it is found that the abnormal path key value index structure already has a label, the existing label is propagated to the traceability graph node corresponding to the current event and the label is updated, and the updated label is cached in the abnormal path key value index structure, wherein the label update rule is: update the event vector in the label according to the event vector of the current event, calculate the abnormal path score of the current event, and update the abnormal path according to the propagation path between the traceability graph node of the last propagation and the traceability graph node corresponding to the current event.

[0075] In some embodiments, the real-time attack chain detection system based on label transfer and event baseline learning includes an alarm system for issuing an alarm when the abnormal path score of the label exceeds a safety threshold.

[0076] It should be noted that Figure 4 This is the logical structure of the real-time attack chain detection system based on label transfer and event baseline learning in this solution. The events collected by the data acquisition system in real time are stored in the message queue and exist in the form of a streaming traceability graph. The events are input into the event anomaly prediction system for embedding to generate event vectors, and the event codes are input into the event anomaly prediction model for prediction to obtain event anomalies. Event anomalies are input into the attack judgment system for attack. Figure 4 The "abnormal path mining algorithm" corresponds to the implementation algorithm of the attack judgment system of this solution, the "alarm aggregation" corresponds to the implementation algorithm of the alarm system of this solution, the "stream computing engine" corresponds to the attack judgment system and the alarm system, and the "regression predictor" corresponds to the "event outlier prediction model".

[0077] Embodiment 3

[0078] This embodiment also provides an electronic device, referring to Figure 5 , including a memory 404 and a processor 402, wherein the memory 404 stores a computer program, and the processor 402 is configured to run the computer program to execute the steps in any of the above-mentioned embodiments of the real-time attack chain detection method based on label transfer and event baseline learning.

[0079] Specifically, the processor 402 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.

[0080] Among them, the memory 404 may include a large capacity memory 404 for data or instructions. For example, but not limitation, the memory 404 may include a hard disk drive (HDD), a floppy disk drive, a solid state drive (SSD), a flash memory, an optical disk, a magneto-optical disk, a tape, or a universal serial bus (USB) drive, or a combination of two or more of these. In appropriate cases, the memory 404 may include a removable or non-removable (or fixed) medium. In appropriate cases, the memory 404 may be inside or outside the data processing device. In a specific embodiment, the memory 404 is a non-volatile memory. In a specific embodiment, the memory 404 includes a read-only memory (ROM) and a random access memory (RAM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable ROM (Programmable Read-Only Memory, PROM for short), an erasable PROM (Erasable Programmable Read-Only Memory, EPROM for short), an electrically erasable PROM (Electrically Erasable Programmable Read-Only Memory, EEPROM for short), an electrically alterable ROM (Electrically Alterable Read-Only Memory, EAROM for short) or a flash memory (FLASH) or a combination of two or more of these. In appropriate circumstances, the RAM may be a static random access memory (SRAM) or a dynamic random access memory (DRAM), wherein the DRAM may be a fast page mode dynamic random access memory 404 (FPMDRAM), an extended data output dynamic random access memory (EDODRAM), a synchronous dynamic random access memory (SDRAM), etc.

[0081] The memory 404 may be used to store or cache various data files required for processing and / or communication, as well as possible computer program instructions executed by the processor 402 .

[0082] The processor 402 reads and executes the computer program instructions stored in the memory 404 to implement any one of the real-time attack chain detection methods based on label transfer and event baseline learning in the above embodiments.

[0083] Optionally, the electronic device may further include a transmission device 406 and an input / output device 408 , wherein the transmission device 406 is connected to the processor 402 , and the input / output device 408 is connected to the processor 402 .

[0084] The transmission device 406 can be used to receive or send data via a network. Specific examples of the above-mentioned network may include a wired or wireless network provided by a communication provider of the electronic device. In one example, the transmission device includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 406 can be a radio frequency (Radio Frequency, referred to as RF) module, which is used to communicate with the Internet wirelessly.

[0085] The input and output devices 408 are used to input or output information. In this embodiment, the input information may be events collected in real time, and the output information may be abnormal paths and alarms.

[0086] Optionally, in this embodiment, the processor 402 may be configured to perform the following steps through a computer program:

[0087] Events are collected in real time and embedded into codes to obtain event vectors, and the event vectors are input into a pre-trained event anomaly value prediction model to output event anomaly values, wherein the event anomaly value prediction model is trained through events and corresponding event anomaly values;

[0088] If the event abnormal value exceeds the threshold, the label is initialized and set on the traceability graph node corresponding to the event and an abnormal path key value index structure is established, where the abnormal path key value index structure caches key-value pairs with the traceability graph node as the key and the label as the value, and each key-value pair corresponds to a unique label propagation index, where the label ID, event vector, abnormal path score and abnormal path are set in the label;

[0089] If it is found that there is already a label in the abnormal path key-value index structure, the existing label will be propagated to the traceability graph node corresponding to the current event and the label will be updated, and the updated label will be cached in the abnormal path key-value index structure. The label update rule is as follows: update the event vector in the label according to the event vector of the current event, calculate the abnormal path score of the current event, and update the abnormal path according to the propagation path between the traceability graph node of the last propagation and the traceability graph node corresponding to the current event.

[0090] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementation modes, and this embodiment will not be described in detail here.

[0091] In general, various embodiments may be implemented in hardware or dedicated circuits, software, logic, or any combination thereof. Some aspects of the invention may be implemented in hardware, while other aspects may be implemented in firmware or software that may be executed by a controller, microprocessor, or other computing device, but the invention is not limited thereto. Although various aspects of the invention may be shown and described as block diagrams, flow charts, or using some other graphical representation, it should be understood that, as non-limiting examples, the boxes, devices, systems, techniques, or methods described herein may be implemented in hardware, software, firmware, dedicated circuits or logic, general-purpose hardware or controllers or other computing devices, or some combination thereof.

[0092] Embodiments of the present invention can be implemented by computer software, which is executable by a data processor of a mobile device, such as in a processor entity, or implemented by hardware, or implemented by a combination of software and hardware. Computer software or programs (also referred to as program products) including software routines, applets and / or macros can be stored in any device readable data storage medium, and they include program instructions for performing specific tasks. Computer program products can include one or more computer executable components configured to perform embodiments when the program is running. One or more computer executable components can be at least one software code or a part thereof. In addition, at this point, it should be noted that any box of the logic flow in the figure can represent a program step, or interconnected logic circuits, boxes and functions, or a combination of program steps and logic circuits, boxes and functions. Software can be stored in physical media such as memory chips or storage blocks implemented in processors, magnetic media such as hard disks or floppy disks, and optical media such as, for example, DVDs and data variants thereof, CDs. Physical media are non-transient media.

[0093] Those skilled in the art should understand that the technical features of the above embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0094] The above embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A real-time attack chain detection method based on label transfer and event baseline learning, characterized in that: include: Events are collected in real time and embedded into codes to obtain event vectors, and the event vectors are input into a pre-trained event anomaly value prediction model to output event anomaly values, wherein the event anomaly value prediction model is trained through events and corresponding event anomaly values; If the event abnormal value exceeds the threshold, the label is initialized and set on the traceability graph node corresponding to the event and an abnormal path key value index structure is established, where the abnormal path key value index structure caches key-value pairs with the traceability graph node as the key and the label as the value, and each key-value pair corresponds to a unique label propagation index, where the label ID, event vector, abnormal path score and abnormal path are set in the label; If it is found that there is already a label in the abnormal path key-value index structure, the existing label will be propagated to the traceability graph node corresponding to the current event and the label will be updated, and the updated label will be cached in the abnormal path key-value index structure. The label update rule is as follows: update the event vector in the label according to the event vector of the current event, calculate the abnormal path score of the current event, and update the abnormal path according to the propagation path between the traceability graph node of the last propagation and the traceability graph node corresponding to the current event.

2. The real-time attack chain detection method based on label transfer and event baseline learning according to claim 1 is characterized in that: The frequency of abnormal events is used as the event abnormal value, and the event abnormal value is used as the value, and the event is used as the key to form a key-value pair to train the event abnormal value prediction model.

3. The real-time attack chain detection method based on label transfer and event baseline learning according to claim 2 is characterized in that: When the event abnormal value is the abnormal event frequency, it is expressed by the quotient of the frequency of the abnormal event in the event frequency database and the frequency of the same type of event with the same subject and operation as the current abnormal event but with an arbitrary object.

4. The real-time attack chain detection method based on label transfer and event baseline learning according to claim 1 is characterized in that: A secondary index structure is set within the abnormal path key-value index structure, where the secondary index structure is represented as: tracing graph node → label and abnormal path score → abnormal path, where tracing graph node → label and abnormal path score caches the propagated label in the form of key-value mapping, and label → abnormal path records the anomaly detection results of the abnormal path.

5. The real-time attack chain detection method based on label transfer and event baseline learning according to claim 1 is characterized in that: In the step of "initializing the setting of labels on the traceability graph node corresponding to the event", set the label ID, obtain the event vector of the current event as the event vector of the label, set the path where the traceability graph node corresponding to the event is located as the abnormal path, and cache the label ID, event vector, abnormal path score and abnormal path in the label to complete the initialization of the label.

6. The real-time attack chain detection method based on label transfer and event baseline learning according to claim 1 is characterized in that: The rule score of the current event is calculated according to the event anomaly value of the current event, and the difference between 1 and the rule score is taken as the abnormal path score. Every time a label undergoes label propagation, the rule score is updated by multiplying the rule score by the decay value.

7. The real-time attack chain detection method based on label transfer and event baseline learning according to claim 1 is characterized in that: When the abnormal path score of a label reaches the set threshold, it is defined as an alarm label. The repeated labels in different alarm labels are merged, and an attack graph is formed based on the traceability graph nodes where the remaining alarm labels are located.

8. A real-time attack chain detection system based on label transfer and event baseline learning, characterized in that: include, Data acquisition system for real-time acquisition of events: An event outlier prediction system is used to embed and encode an event to obtain an event vector, and input the event vector into a pre-trained event outlier prediction model to output an event outlier value, wherein the event outlier prediction model is trained through events and corresponding event outlier values; An attack judgment system is used to initialize and set a label on the traceability graph node corresponding to the event and establish an abnormal path key value index structure when the event abnormal value exceeds a threshold, wherein the abnormal path key value index structure caches key-value pairs with the traceability graph node as the key and the label as the value, and each key value pair corresponds to a unique label propagation index, wherein the label ID, event vector, abnormal path score and abnormal path are set in the label; if it is found that the abnormal path key value index structure already has a label, the existing label is propagated to the traceability graph node corresponding to the current event and the label is updated, and the updated label is cached in the abnormal path key value index structure, wherein the label update rule is: update the event vector in the label according to the event vector of the current event, calculate the abnormal path score of the current event, and update the abnormal path according to the propagation path between the traceability graph node of the last propagation and the traceability graph node corresponding to the current event.

9. An electronic device comprising a memory and a processor, characterized in that: The memory stores a computer program, and the processor is configured to run the computer program to execute the real-time attack chain detection method based on label transfer and event baseline learning according to any one of claims 1 to 7.

10. A readable storage medium, characterized in that: The readable storage medium stores a computer program, wherein the computer program includes a program code for controlling a process to execute a process, wherein the process includes the real-time attack chain detection method based on label transfer and event baseline learning according to any one of claims 1 to 7.

Citation Information

Patent Citations

  • Attack tracing method and device, equipment and storage medium

    CN115622802A

  • APT attack detection method and system based on auto-encoder model

    CN117478354A

  • Streaming traceability graph real-time attack detection method and system based on label and graph alignment

    CN117560228A

  • Network abnormal behavior event detection method based on hypergraph

    CN118573412A

  • Significant events identifier for outlier root cause investigation

    US20190228296A1