A real-time streaming data attack detection method based on label transfer

Abstract

The present invention belongs to the technical field of network security, and discloses a real-time stream data attack detection method based on label transfer, including real-time kernel cache-level data compression of kernel log data, and using the Kellect tool to collect system call events in the compressed kernel cache data; converting the system call function in the system call event into the corresponding system call event type; the stream cube platform initializes the labels of processes and files in the system call event and caches them, screens valid system call events according to the system call event type, transfers labels to valid system call events based on label transfer rules, and updates the cached labels in real time during the label transfer process, and identifies attack nodes according to threat judgment rules, and stores and displays the attributes of the identified attack nodes. The present invention performs high-quality attack detection on real-time data streams.

Description

Technical Field

[0001] The invention belongs to the technical field of network security, and in particular relates to a real-time stream data attack detection method based on label transmission. Background Art

[0002] Advanced Persistent Threat (APT) often attacks core infrastructure (such as energy, transportation, and communications) and key industries (such as finance and healthcare). In recent years, APT attacks have occurred frequently, and APT attack detection has become an important research topic that has attracted widespread attention from academia and industry. Timely attack detection can help organizations discover and block malicious activities in a timely manner, and prevent security incidents such as data leakage, asset loss, or system paralysis.

[0003] On the other hand, rule-based detection technology is a common security detection method that identifies and responds to specific security threats or abnormal behaviors through a predefined set of rules. These rules can be defined based on specific attack patterns, malware features, abnormal network traffic, or system behavior. For example, rules can include specific string matching, file hash values, abnormal port usage, specific command execution sequences, etc. Rule-based detection technology is usually fast and efficient, can provide a low false alarm rate, and is suitable for detecting known attack patterns and known malicious behaviors. However, its limitation is that a large number of rules need to be accurately defined and maintained, and it shows poor adaptability to unknown and new attack patterns. Therefore, when facing complex and changing threats, it may be necessary to combine other detection methods to improve detection coverage and accuracy. In view of the above problems, how to correctly define the rules of malicious attacks to achieve rule-based attack detection is an urgent problem to be solved. Summary of the invention

[0004] The purpose of the present invention is to provide a real-time stream data attack detection method based on label transmission, which performs high-quality attack detection on real-time data streams.

[0005] To achieve the above object, the technical solution adopted by the present invention is:

[0006] A real-time streaming data attack detection method based on label transmission, the real-time streaming data attack detection method based on label transmission comprising:

[0007] Perform kernel cache-level data compression on kernel log data in real time, and use the Kellect tool to collect system call events in the compressed kernel cache data;

[0008] Convert the system call function in the system call event to the corresponding system call event type, and pass the converted system call event to the Kafka message queue;

[0009] Based on the Stream Cube platform, system call events are retrieved from the Kafka message queue, labels of processes and files in the system call events are initialized and cached, valid system call events are screened according to the system call event type, labels are transferred to valid system call events based on label transfer rules, and cached labels are updated in real time during the label transfer process. At the same time, attack nodes are identified according to threat judgment rules, and the attributes of the identified attack nodes are transferred to the Kafka message queue;

[0010] Read the properties of the attacking node in the Kafka message queue for storage and display.

[0011] Several optional methods are also provided below, but they are not intended to be additional limitations on the above-mentioned overall solution, but are merely further supplements or preferences. Under the premise that there are no technical or logical contradictions, each optional method can be combined with the above-mentioned overall solution separately, and multiple optional methods can also be combined.

[0012] Preferably, the real-time kernel cache level data compression of the kernel log data comprises the following operations:

[0013] Establishing a process dependency cache, wherein the process dependency cache is used to record data dependencies between different execution units within a single process;

[0014] Establishing an event cache, wherein the event cache is used to store system call events related to the system object;

[0015] When a system call event occurs, determine whether the data dependency of the process involved in the current system call event already exists in the process dependency cache, and if so, discard the current system call event; otherwise, insert the data dependency into the process dependency cache, and insert the current system call event into the event cache;

[0016] Add an automaton for each file in the system object, the automaton is used to detect whether the file is a temporary file, if it is a temporary file, delete the file and the corresponding system call event in the event cache;

[0017] Use the data in the event buffer as compressed kernel buffer data.

[0018] Preferably, the system call function in the system call event is represented by an EventType field, and the system call event type is represented by an event type descriptor WDM field, and the converting of the system call function in the system call event into a corresponding system call event type includes:

[0019] If the EventType field value is ProcessStart, it is converted to the WDM field value EVENT_EXECUTE;

[0020] If the EventType field value is ThreadStart, it is converted to the WDM field value EVENT_CREATE_THREAD;

[0021] If the EventType field value is ProcessEnd or ThreadEnd, it is converted to the WDM field value EVENT_EXIT;

[0022] If the EventType field value is ImageDCStart or ImageLoad, it is converted to the WDM field value EVENT_LOADLIBARY;

[0023] If the EventType field value is TcpIpSendIPV4, TcpIpSendIPV6, TcpIpRetransmitIPV4, or TcpIpRetransmitIPV6, it is converted to the WDM field value EVENT_SENDMSG;

[0024] If the EventType field value is TcpIpRecvIPV4 or TcpIpRecvIPV6, it is converted to the WDM field value EVENT_RECVMSG;

[0025] If the EventType field value is TcpIpConnectIPV4, TcpIpReconnectIPV6, TcpIpConnectIPV6, or TcpIpReconnectIPV4, it is converted to the WDM field value EVENT_CONNECT;

[0026] If the EventType field value is TcpIpAcceptIPV4 or TcpIpAcceptIPV6, it is converted to the WDM field value EVENT_ACCEPT;

[0027] If the EventType field value is FileIOWrite, DiskIOWriteInit, or DiskIOWrite, it is converted to the WDM field value EVENT_WRITE;

[0028] If the EventType field value is DiskIOReadInit, DiskIORead, or FileIORead, it is converted to the WDM field value EVENT_READ;

[0029] If the EventType field value is FileIOFileCreate, RegisterCreate, or RegistryKCBCreate, it is converted to the WDM field value EVENT_CREATE_OBJECT;

[0030] If the EventType field value is FileIORename, it is converted to the WDM field value EVENT_RENAME;

[0031] If the EventType field value is RegistrySetValue or RegistryDeleteValue, it is converted to the WDM field value EVENT_MODIFY_FILE_ATTRIBUTES;

[0032] If the EventType field value is FileIOCreate or RegisterOpen, it is converted to the WDM field value EVENT_OPEN;

[0033] If the EventType field value is FileIOCleanup, FileIOFlush, FileIOClose, RegistryFlush, RegistryClose, TcpIpDisconnectIPV4, or TcpIpDisconnectIPV6, it is converted to the WDM field value EVENT_CLOSE;

[0034] If the EventType field value is file and registry, it is converted to the WDM field value FILE_OBJECT_FILE;

[0035] If the EventType field value is RegistrySetInformation, it is converted to the WDM field value EVENT_UPDATE;

[0036] If the EventType field value is FileIODirEnum, RegistryQuery, RegistryQueryValue, RegistryEnumerateKey, RegistryEnumerateValueKey, or RegistryQueryMultipleValue, it is converted to the WDM field value EVENT_ITERATE;

[0037] If the EventType field value is FileIOFileRundown, ProcessDCStart, ThreadDCStart, or RegistryKCBRundownBegin, it is converted to the WDM field value EVENT_DC_STAET;

[0038] If the EventType field value is ProcessDCEnd, ThreadDCEnd, or RegistryKCBRundownEnd, it is converted to the WDM field value EVENT_DC_END;

[0039] If the EventType field value is FileIODelete, FileIOFileDelete, RegistryDelete, or RegistryKCBDelete, it is converted to the WDM field value EVENT_DELETE.

[0040] Preferably, screening valid system call events according to the system call event type includes:

[0041] System call events of type EVENT_READ, EVENT_WRITE, EVENT_CREATE_THREAD, EVENT_EXECUTE, EVENT_LOADLIBARY, EVENT_DELETE, EVENT_RENAME, EVENT_CREATE_OBJECT, EVENT_UPDATE, EVENT_EXIT, EVENT_LOADELF, EVENT_OPEN, EVENT_CLOSE, and EVENT_CREATE_THREAD are considered valid system call events.

[0042] Preferably, the labels of the processes and files in the initialization system call event are cached, and the caching process is as follows:

[0043] If the process is initially assigned a label, the process ID, process attributes, and process label are cached;

[0044] If the file is initially tagged, the file ID, file attributes, and file tags are cached.

[0045] Preferably, the labels of the process are divided into state labels and behavior labels;

[0046] The state labels include PSL1, PSL2, PSL3, PSL4, PSL5, PSL6 and PSL7, wherein PSL1 is described as the process having a network connection, PSL2 is described as the process having accessed a high-value data flow node, PSL3 is described as the process containing network data, PSL4 is described as the process having read or loaded a file from the network, PSL5 is described as the process interacting with a non-existent file, PSL6 is described as the process having read a user information file, and PSL7 is described as the process having read a log file;

[0047] The behavior labels include PBL1, PBL2, PBL3, PBL4, PBL5, PBL6 and PBL7, wherein PBL1 is described as the process executing a file from the network, PBL2 is described as the process executing a sensitive file, PBL3 is described as the process executing a command, PBL4 is described as the process executing a shell command, PBL5 is described as the process modifying a timing file, PBL6 is described as the process modifying a permission file, and PBL7 is described as the process reading a high-value information file.

[0048] Preferably, the threat judgment rule includes:

[0049] If the process is assigned the label PBL1, the process is identified as an attack node, and the threat type is download and execute;

[0050] If a process is assigned labels PSL4 and PBL3 at the same time, the process is identified as an attack node and the threat type is Websehll;

[0051] If a process is assigned labels PBL1 and PBL4 at the same time, the process is identified as an attack node and the threat type is RAT;

[0052] If a process is assigned labels PSL5 and PBL4 at the same time, the process is identified as an attack node and the threat type is fileless attack;

[0053] If the process is assigned the label PBL5, PBL6, PSL6, or PSL7, the process is identified as an attack node, and the threat type is suspicious behavior;

[0054] If the process is assigned the label PBL7, the process is identified as an attack node, and the threat type is data leakage;

[0055] If a process is assigned labels PSL4, PBL3, and PBL7 at the same time, the process is identified as an attack node and the threat type is APT;

[0056] If a process is assigned labels PBL1, PBL4, and PBL7 at the same time, the process is identified as an attack node and the threat type is APT;

[0057] If a process is assigned labels PSL5, PBL4, and PBL7 at the same time, the process is identified as an attack node and the threat type is APT.

[0058] The present invention provides a real-time streaming data attack detection method based on label transmission, which has the following beneficial effects compared with the prior art:

[0059] (1) Through label caching, the problem of the inability to retain the long-term dependencies between entities in the short-term window data of the Stream Cube platform during the label transmission process is overcome, and the good integration of the Stream Cube platform and APT detection is achieved. The integration into Stream Cube can perform real-time attack detection and show good real-time performance in real-world scenarios; (2) In terms of data collection, the present invention performs log compression at the kernel level to avoid generating a large number of redundant logs, reduce the overhead of data transmission, processing and writing, and ensure the integrity of log dependency information. In terms of attack detection, the attacks in the ATT&CK knowledge base and network threat intelligence are analyzed, the rules of malicious attacks are correctly and completely defined, and the response and alarm of attacks are realized through the transmission and aggregation of labels; (3) Through the display of warning information, that is, icons, dashboards, etc., the frequency, type, impact range and other information of attack events can be intuitively presented to help the security team quickly understand and evaluate the current security situation. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] Figure 1 A flowchart of a real-time streaming data attack detection method based on label transmission according to the present invention;

[0061] Figure 2 The present invention is a data transmission schematic diagram of a real-time streaming data attack detection method based on label transmission. DETAILED DESCRIPTION

[0062] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0063] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art of the present invention. The terms used in the specification of the present invention herein are only for the purpose of describing specific embodiments and are not intended to limit the present invention.

[0064] like Figure 1-Figure 2As shown, this embodiment provides a real-time stream data attack detection method based on label transmission, comprising the following steps:

[0065] (1) Data collection: Real-time kernel cache-level data compression is performed to detect and prune redundant events at the kernel cache level, reduce runtime space overhead and speed up detection. The kernel log collection tool Kellect is used to collect system call events of the Windows operating system. As a kernel-based audit tool, Kellect can provide rich system logs with low overhead. The collected system call events record detailed information about system objects (such as processes, files, sockets) and the called system services or system resources.

[0066] (1-1) Kernel cache-level data compression: First, a process dependency cache is established. The process dependency cache is used to record the data dependencies between different execution units in a single process through IPC methods such as shared memory, pipes, and signals. Then, an event cache is established to store system call events related to system objects. When a system call event occurs, it is determined whether the data dependency of the process involved in the current system call event already exists in the process dependency cache. If it does, the current system call event is discarded; otherwise, the data dependency is inserted into the process dependency cache, and the current system call event is inserted into the event cache. And an automaton Automata (such as a finite automaton) is added to each file in the system object. The automaton is used to detect whether the file is a temporary file. If it is a temporary file, the file and the corresponding system call event in the event cache are deleted, because temporary files usually have a small impact on attacks. Finally, the data in the process dependency cache and the event cache are used as compressed kernel cache data. The compressed kernel cache data can be cleared after being read by the Kellect tool.

[0067] Data dependency refers to the dependency formed between different execution units due to data sharing during program execution. Simply put, if the result of an execution unit's operation depends on the data generated by another operation, then there is a data dependency between the two operations. For example, reading and writing files: if a process first reads a file and then writes to it based on the read content, then the write operation depends on the data generated by the read operation. Different execution units within a single process refer to smaller execution fragments that are divided during the execution of the process to complete specific tasks.

[0068] (1-2) System log collection: After kernel cache-level data compression, the Kellect tool is used to collect system call events in the event cache. The compression method of this embodiment performs log compression at the kernel level, avoiding the generation of a large number of redundant logs, reducing the overhead of data transmission, processing and writing, and ensuring the integrity of log dependency information.

[0069] (2) Format conversion: Convert the system call function in the system call event into the corresponding system call event type, and pass the converted system call event into the Kafka message queue.

[0070] The system call function in the system call event is represented by the EventType field, and the system call event type is represented by the event type descriptor WDM field. During the conversion, according to the system log data definition table in Table 1, the EventType field in the collected original system call event is mapped and converted to the corresponding unified event type descriptor WDM field to standardize and unify the semantics of log events under different operating system platforms. The system call event after format conversion is then transmitted to the Kafka message queue.

[0071] Table 1 System log data definition table

[0072] Serial number WDM Field EventType Field describe 1 EVENT_EXECUTE ProcessStart Start the program 2 EVENT_CREATE_THREAD ThreadStart Starting a Thread 3 EVENT_EXIT ProcessEnd Program Exit EVENT_EXIT ThreadEnd Program Exit 4 EVENT_LOADLIBARY ImageDCStart Image loading EVENT_LOADLIBARY ImageLoad Image loading 5 EVENT_SENDMSG TcpIpSendIPV4 Send Message EVENT_SENDMSG TcpIpSendIPV6 Send Message EVENT_SENDMSG TcpIpRetransmitIPV4 Send Message EVENT_SENDMSG TcpIpRetransmitIPV6 Send Message 6 EVENT_RECVMSG TcpIpRecvIPV4 Receiving information EVENT_RECVMSG TcpIpRecvIPV6 Receiving information 7 EVENT_CONNECT TcpIpConnectIPV4 Sending a connection EVENT_CONNECT TcpIpReconnectIPV6 Send a connection EVENT_CONNECT TcpIpConnectIPV6 Sending a connection EVENT_CONNECT TcpIpReconnectIPV4 Sending a connection 8 EVENT_ACCEPT TcpIpAcceptIPV4 Accepting Connections EVENT_ACCEPT TcpIpAcceptIPV6 Accepting Connections 9 EVENT_WRITE FileIOWrite Writing Data EVENT_WRITE DiskIOWriteInit Writing Data EVENT_WRITE DiskIOWrite Writing Data 10 EVENT_READ DiskIOReadInit Read Data EVENT_READ DiskIORead Read Data EVENT_READ FileIORead Read Data 11 EVENT_CREATE_OBJECT FileIOFileCreate Creating an object EVENT_CREATE_OBJECT RegisterCreate Creating an object EVENT_CREATE_OBJECT RegistryKCBCreate Creating an object 12 EVENT_RENAME FileIORename Rename 13 EVENT_MODIFY_FILE_ATTRIBUTES RegistrySetValue Editing File Properties EVENT_MODIFY_FILE_ATTRIBUTES RegistryDeleteValue Editing File Properties 14 EVENT_OPEN FileIOCreate Open EVENT_OPEN RegisterOpen Open 15 EVENT_CLOSE FileIOCleanup closure EVENT_CLOSE FileIOFlush closure EVENT_CLOSE FileIOClose closure EVENT_CLOSE RegistryFlush closure EVENT_CLOSE RegistryClose closure EVENT_CLOSE TcpIpDisconnectIPV4 closure EVENT_CLOSE TcpIpDisconnectIPV6 closure 16 FILE_OBJECT_FILE Files + Registry Declaration File 17 EVENT_UPDATE RegistrySetInformation Update Event 18 EVENT_ITERATE FileIODirEnum Enumeration information EVENT_ITERATE RegistryQuery Enumeration information EVENT_ITERATE RegistryQueryValue Enumeration information EVENT_ITERATE RegistryEnumerateKey Enumeration information EVENT_ITERATE RegistryEnumerateValueKey Enumeration information EVENT_ITERATE RegistryQueryMultipleValue Enumeration information 19 EVENT_DC_STAET FileIOFileRundown Enumeration information at the beginning of a session EVENT_DC_STAET ProcessDCStart Enumeration information at the beginning of a session EVENT_DC_STAET ThreadDCStart Enumeration information at the beginning of a session EVENT_DC_STAET RegistryKCBRundownBegin Enumeration information at the beginning of a session 20 EVENT_DC_END ProcessDCEnd Enumeration information when the session ends EVENT_DC_END ThreadDCEnd Enumeration information when the session ends EVENT_DC_END RegistryKCBRundownEnd Enumeration information when the session ends 21 EVENT_DELETE FileIODelete delete EVENT_DELETE FileIOFileDelete delete EVENT_DELETE RegistryDelete delete EVENT_DELETE RegistryKCBDelete delete

[0073] (3) Real-time stream data processing and detection: The Stream Cube platform retrieves system call events from the Kafka message queue based on its processing capacity, initializes and caches the labels of processes and files in the system call events, filters valid system call events based on the system call event type, and transfers labels to valid system call events based on label transfer rules. During the label transfer process, the cached labels are updated in real time. At the same time, the attack nodes are identified based on the threat judgment rules, and the attributes of the identified attack nodes are transferred to the Kafka message queue.

[0074] (3-1) Real-time streaming data processing:

[0075] (3-1-1) Entity extraction: The system call events after format conversion are passed to the Kafka message queue. The Stream Cube platform consumes the data in the Kafka message queue in real time. The operators of the Stream Cube platform initialize labels according to the various defined system object labels, and dynamically match and transfer labels according to the defined label transfer rules through the subject objects and events recorded in the system log.

[0076] The log data collected by Kellect is in json format. Each log data corresponds to a system call event, that is, a json data. Each system call event corresponds to two entities, that is, the operation of the subject on the object. The subject is generally a process, and the object can be a process, a file, or a socket. For example, a system call event can be: a process creates a new child process, a process loads a file, a process receives data through a socket, etc. This embodiment extracts these three types of entities from the log data in json format according to the relevant key. The entity types are shown in Table 2.

[0077] Table 2 Entity type table

[0078] main body object Entity attributes process process Name, Command (cmd) process document File pathname process Sockets IP address (IP), port (port)

[0079] (3-1-2) Process label initialization: Process labels are divided into two categories, status labels and behavior labels, as shown in the process label definition table in Table 3. Status labels represent labels that the semantics of a process is passed to a child process through the system EVENT_CREATE_THREAD event. This type of label usually refers to reading or loading certain files. Because most attacks occur in child processes, it is necessary to pass the semantics of reading or loading files from the parent process to the child process; behavior labels represent the operations performed by the process and are used to accurately locate threats, and behavior labels will not be passed to child processes through the EVENT_CREATE_THREAD event. For example: If a process has a network connection, it is marked with label PSL1 because data from the network cannot be trusted.

[0080] Table 3 Process label definition table

[0081] Label describe category PSL1 The process has a network connection Status Tags PSL2 The process accessed too many high-value data flow nodes Status Tags PSL3 The process contains network data Status Tags PSL4 The process reads or loads files from the network Status Tags PSL5 The process interacts with a nonexistent file Status Tags PSL6 The process reads the user information file Status Tags PSL7 The process reads the log file Status Tags PBL1 The process executed a file from the network Behavioral Tags PBL2 The process executed a sensitive file Behavioral Tags PBL3 The process executed a command (this tag is only used to detect Webshell) Behavioral Tags PBL4 The process executes the shell command Behavioral Tags PBL5 The process modified the timer file Behavioral Tags PBL6 The process modified the permissions file Behavioral Tags PBL7 The process reads a high-value information file Behavioral Tags

[0082] (3-1-3) File label initialization: File labels are divided into two categories, untrusted labels and high-value labels, as shown in the file label definition table in Table 4. Untrusted files refer to files that contain untrusted data from the network. Such files are generally generated during the attack process and are not files that exist in the system at the beginning; high-value files refer to files that contain sensitive data. Such files are usually the most vulnerable to theft and access by attackers in APT attacks, and the data contained in them is usually special and important. For example, if a file contains data from the network, it is marked as label FUL2, because untrusted data may lead to subsequent attacks such as loading and executing attack code.

[0083] Table 4 File tag definition table

[0084] Label describe category FUL1 Files are created by uploading Untrusted Tags FUL2 The file contains data from the network Untrusted Tags FUL3 File does not exist Untrusted Tags FUL4 The file is created in a Webshell attack Untrusted Tags FUL5 Files created during RAT attacks Untrusted Tags FUL6 The file was created in a fileless attack Untrusted Tags FHL1 Files can control scheduled tasks High value label FHL2 Files can control user permissions High value label FHL3 The file stores sensitive user information High value label FHL4 The file saves the history of executed commands High value label FHL5 File created by a process that reads a high value file High value label

[0085] (3-1-4) Event selection: After the process and file tags are initialized, the tags are cached to cope with the characteristics of short-term window data on the streaming platform. If the process is initialized with tags, the process ID, process attributes, and process tags are cached; if the file is initialized with tags, the file ID, file attributes, and file tags are cached.

[0086] In addition, it is necessary to clarify the valid system call events between entities in order to facilitate the transfer and aggregation of context information. The system call event types involved in the valid system call events are shown in the system event table of Table 5.

[0087] Table 5 System event table

[0088] serial number Event Type describe ES0 EVENT_READ (abbreviated as Read) Read file event ES1 EVENT_WRITE (abbreviated as Write) Write file event ES2 EVENT_CREATE_THREAD Create child process event ES3 EVENT_EXECUTE (abbreviated as Execute) Process execution events ES4 EVENT_LOADLIBARY Process loading event ES5 EVENT_DELETE File deletion events ES6 EVENT_RENAME File rename event ES7 EVENT_CREATE_OBJECT File creation event ES8 EVENT_UPDATE File attribute modification events ES9 EVENT_EXIT Process exit event ES10 EVENT_LOADELF (abbreviated as LoadElf) Loading Event ES11 EVENT_OPEN File open event ES12 EVENT_CLOSE File close event ES13 EVENT_CREATE_THREAD Shared file creation subprocess event

[0089] (3-1-5) Label transfer: According to the label transfer rule table in Table 6, the label is transferred according to the system call event type in the system call event. In Table 6, label 1 and label 2 represent the semantic labels carried by the system entity, and the direction represents the flow direction of the system call event, which is also the direction of label transfer.

[0090] Table 6 Label transfer rules

[0091] Tag 1 Event Type Tag 2 direction describe PSL1 Write FUL2 just A process with a network connection (PSL1) writes a file that is labeled FUL2 PSL3 Read FUL2 inverse The process reads a file (FUL2) containing network data. The process is labeled PSL3. PSL3 Write FUL2 just The process (PSL3) that read the network data file wrote a file that was labeled FUL2. PSL4 Read FUL1 inverse The uploaded file (FUL1) is read or loaded by the process, which is marked with the PSL4 tag PSL5 Read or LoadElf FUL3 inverse The process interacts with a non-existent file (FUL3) and is labeled PSL5 PBL1 Execute or LoadElf FUL2 inverse The process executes or loads a code file (FUL2) containing network data. The process is labeled PBL1. PBL1 Write FUL2 just The process (PB1) that executed the code from the network wrote a file, which was labeled FUL2 PBL3 Write FUL4 just The process that caused the Webshell threat (PBL3) wrote a file that was tagged with FUL4 PBL3 Read FUL4 inverse The process reads or loads a file created by the webshell (FU4). The process is labeled PBL3. PBL1 Write FUL5 just The process that caused the RAT threat (PBL1) wrote a file that was tagged with FUL5. PBL1 Read FUL5 inverse The process reads a file (FUL5) created by the RAT. The process is labeled PBL1. PSL5 Write FUL6 just The process that caused the fileless attack (PSL5) wrote a file that was tagged with FUL6. PSL5 Read FUL6 inverse The process reads or loads a file created by a fileless attack (FUL6). The process is marked with PSL5. PBL5 Write FHL1 inverse The process has modified the file that controls the scheduled tasks (FHL1), and the process is labeled PBL5 (persistent base) PBL6 Write FHL2 inverse The process has modified a file that can control user privileges (FHL2), and the process is labeled PBL6 (privilege escalation) PSL6 Read FHL3 inverse The process has read a file containing user credential information (FHL3). The process is tagged with PSL6 (obtaining credentials) PSL7 Read FHL4 inverse The process has read the file (FHL4) that saves the user's historical commands. The process is marked with the PSL7 tag (information collection) PBL5 / PBL 6 / PSL6 / PSL 7 Write FHL5 just The process that caused the suspicious behavior (PBL5 or PBL6 or PSL6 or PSL7) wrote a file that was tagged with FHL5 PBL7 Read FHL5 inverse The process reads a file containing high-value data (FHL5), and the process is tagged with PBL7

[0092] APT attacks are persistent and last for a long time, so there are long-term dependencies between system entities. As a real-time stream data processing platform, the Stream Cube platform usually processes data in short time windows. In order to retain the long-term dependencies between system entities as much as possible, a database is designed on the Stream Cube platform to cache label information to alleviate the problems caused by the persistence of APT attacks. Because the attack behavior may span a long period of time, the label cache records the entity labels of a long time window, which can retain more label information for each entity. When it comes to long-term dependencies, the label information will not be lost when the label is transferred. Therefore, this embodiment caches the corresponding label information when the process and file labels are initialized, and during the label transfer process, the entity label data in the database is updated in real time according to the entity ID. The Stream Cube platform in this application can record entity labels for a longer time window, retain more label information for each entity, and enable better aggregation of label information.

[0093] (3-2) Attack pattern identification: Through the transmission and aggregation of tags, the relevant information of the attack behavior is associated and integrated to form a complete attack chain. According to the defined multiple judgment rules, the corresponding attack pattern is matched, and the warning information of the attack is transmitted to the Kafa message queue.

[0094] During the label transfer process, the attack node will be determined synchronously according to the defined threat judgment rule table. When the attack node is determined, the attack-related information will be transmitted to the Kafka message queue. The threat type in the first column of Table 7 indicates the attack, the satisfied label in the second column indicates the label contained in a certain process, and the prerequisite in the third column explains the label transfer situation and the cause of the attack through a formal method. The labels of different entities in the attack chain will eventually reach a process through transfer and aggregation. When the label contained in the process matches the threat judgment rule, the corresponding attack alarm information will be reported.

[0095] Table 7 Threat judgment rules

[0096] Threat Types Meet the label Prerequisites Download and execute PBL1 When a process executes a command containing scp or wget, it indicates that the process may contain a network connection, and the corresponding process is initially labeled PSL1; if the process contains the PSL1 label and executes the Write event, the generated file will be labeled FUL2; if the file contains the FUL2 label and a process executes the Execute event on it, the process will be labeled PBL1; through threat judgment rules, all processes containing the PBL1 label are reported as having the threat of downloading and executing. Websehll PSL4&PBL3 When the file path name contains upload, it means that the file is created by uploading, and the corresponding file is initially labeled FUL1; if the file contains the FUL1 tag and a process executes a Read or Mmap event on it, the process will be labeled PSL4; when a command executed by a process contains *, it means that the process has executed the command, and the corresponding process is initially labeled PBL3; through threat judgment rules, Webshell threats are reported for all processes containing PSL4&PBL3 tags. RAT (RemoteAdministrationTool or RemoteAccessTrojan) PBL1 & PBL4 When a process executes a command containing sh or bash, it means that the process has executed a shell command, and the corresponding process is initialized with the label PBL4; if a process causes the threat-download and execution, the process will be labeled PBL1; through the threat judgment rules, all processes containing the PBL1&PBL4 labels are reported as RAT threats. FilelessAttack PSL5&PBL4 When the file path name contains null, it means that the file does not exist, and the corresponding file is initialized with the label FUL3; if the file contains the FUL3 label and a process executes a Read or Load event on it, the process will be labeled with the PSL5 label; when a process executes a command containing sh or bash, it means that the process executes a shell command, and the corresponding process is initialized with the label PBL4; through threat judgment rules, all processes containing PSL5&PBL4 are reported as fileless attack threats. Suspicious behavior PBL5|PBL6|PSL6|PSL7 When a file path name contains etc / crontab, it means that the file can control the scheduled task, and the corresponding file is initialized with the tag FHL1; if the file contains the FHL1 tag and a process executes the Write event on it, the process will be labeled with the PBL5 tag. When a file path name contains etc / sudoers, it means that the file can control user permissions, and the corresponding file is initialized with the tag FHL2; if the file contains the FHL2 tag and a process executes the Write event on it, the process will be labeled with the PBL6 tag. When a file path name contains etc / passwd, it means that the file stores user sensitive information, and the corresponding file is initialized with the tag FHL3; if the file contains the FHL3 tag and a process executes the Write event on it, the process will be labeled with the PSL6 tag. When a file path name contains bash_history, it means that the file stores the executed historical commands, and the corresponding file is initialized with the tag FHL4; if the file contains the FHL4 tag and a process executes the Read event on it, the process will be labeled with the PSL7 tag. Through threat judgment rules, all processes containing PBL5, PBL6, PSL6 or PSL7 labels are reported as suspicious behavior threats. Data Leakage or Data Exfiltration PBL7 If a process causes suspicious behavior, the process is labeled PBL5|PBL6|PSL6|PSL7; if a process contains PBL5|PBL6|PSL6|PSL7 tags and executes a Write event on a file, the file is labeled FHL5; if a file contains an FHL5 tag and a process executes a Read event on it, the process is labeled PBL7; through threat judgment rules, data infiltration threats are reported for all processes containing the PBL7 tag. APT (AdvancedPersistentThreat) (PSL4&PBL3)|(PBL1&PBL4)|(PSL5&PBL4)&PBL7 If a process causes a Webshell|RAT|fileless attack and the process contains the PBL8 tag, an APT attack is reported for the process containing (PSL4&PBL3)|(PBL1&PBL4)|(PSL5&PBL4)&PBL7 through threat judgment rules.

[0097] (4) Information display: Take the warning information from the Kafka message queue, store it in the database, and display the warning information.

[0098] (4-1) Data storage: Read the attributes of the attack node (such as process name and process ID) in the Kafka message queue and store them in the database.

[0099] (4-2) Attack display: Read relevant data from the database and use front-end technology to display the attack information on the large screen interface.

[0100] The technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0101] The above-mentioned embodiments only express several implementation modes of the present invention, and the description thereof is relatively specific and detailed, but it cannot be understood as limiting the scope of the invention. It should be pointed out that, for a person of ordinary skill in the art, several modifications and improvements can be made without departing from the concept of the present invention, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the attached claims.

Claims

1. A real-time streaming data attack detection method based on label transmission, characterized in that: The real-time stream data attack detection method based on label transmission includes: Perform kernel cache-level data compression on kernel log data in real time, and use the Kellect tool to collect system call events in the compressed kernel cache data; Convert the system call function in the system call event to the corresponding system call event type, and pass the converted system call event to the Kafka message queue; Based on the Stream Cube platform, system call events are retrieved from the Kafka message queue, labels of processes and files in the system call events are initialized and cached, valid system call events are screened according to the system call event type, labels are transferred to valid system call events based on label transfer rules, and cached labels are updated in real time during the label transfer process. At the same time, attack nodes are identified according to threat judgment rules, and the attributes of the identified attack nodes are transferred to the Kafka message queue; Read the properties of the attacking node in the Kafka message queue for storage and display; The real-time kernel cache level data compression of the kernel log data includes the following operations: Establishing a process dependency cache, wherein the process dependency cache is used to record data dependencies between different execution units within a single process; Establishing an event cache, wherein the event cache is used to store system call events related to the system object; When a system call event occurs, determine whether the data dependency of the process involved in the current system call event already exists in the process dependency cache, and if so, discard the current system call event; otherwise, insert the data dependency into the process dependency cache, and insert the current system call event into the event cache; Add an automaton for each file in the system object, the automaton is used to detect whether the file is a temporary file, if it is a temporary file, delete the file and the corresponding system call event in the event cache; Use the data in the event buffer as compressed kernel buffer data; The process labels are divided into status labels and behavior labels; The state labels include PSL1, PSL2, PSL3, PSL4, PSL5, PSL6 and PSL7, wherein PSL1 is described as the process having a network connection, PSL2 is described as the process having accessed a high-value data flow node, PSL3 is described as the process containing network data, PSL4 is described as the process having read or loaded a file from the network, PSL5 is described as the process interacting with a non-existent file, PSL6 is described as the process having read a user information file, and PSL7 is described as the process having read a log file; The behavior tags include PBL1, PBL2, PBL3, PBL4, PBL5, PBL6 and PBL7, wherein PBL1 is described as the process executing a file from the network, PBL2 is described as the process executing a sensitive file, PBL3 is described as the process executing a command, PBL4 is described as the process executing a shell command, PBL5 is described as the process modifying a timed file, PBL6 is described as the process modifying a permission file, and PBL7 is described as the process reading a high-value information file; The threat judgment rules include: If the process is assigned the label PBL1, the process is identified as an attack node, and the threat type is download and execute; If a process is assigned labels PSL4 and PBL3 at the same time, the process is identified as an attack node and the threat type is Websehll; If a process is assigned labels PBL1 and PBL4 at the same time, the process is identified as an attack node and the threat type is RAT; If a process is assigned labels PSL5 and PBL4 at the same time, the process is identified as an attack node and the threat type is fileless attack; If the process is assigned the label PBL5, PBL6, PSL6, or PSL7, the process is identified as an attack node, and the threat type is suspicious behavior; If the process is assigned the label PBL7, the process is identified as an attack node, and the threat type is data leakage; If a process is assigned labels PSL4, PBL3, and PBL7 at the same time, the process is identified as an attack node and the threat type is APT; If a process is assigned labels PBL1, PBL4, and PBL7 at the same time, the process is identified as an attack node and the threat type is APT; If a process is assigned labels PSL5, PBL4, and PBL7 at the same time, the process is identified as an attack node and the threat type is APT.

2. The real-time streaming data attack detection method based on label transmission according to claim 1 is characterized in that: The system call function in the system call event is represented by the EventType field, and the system call event type is represented by the event type descriptor WDM field. Then, converting the system call function in the system call event into the corresponding system call event type includes: If the EventType field value is ProcessStart, it is converted to the WDM field value EVENT_EXECUTE; If the EventType field value is ThreadStart, it is converted to the WDM field value EVENT_CREATE_THREAD; If the EventType field value is ProcessEnd or ThreadEnd, it is converted to the WDM field value EVENT_EXIT; If the EventType field value is ImageDCStart or ImageLoad, it is converted to the WDM field value EVENT_LOADLIBARY; If the EventType field value is TcpIpSendIPV4, TcpIpSendIPV6, TcpIpRetransmitIPV4, or TcpIpRetransmitIPV6, it is converted to the WDM field value EVENT_SENDMSG; If the EventType field value is TcpIpRecvIPV4 or TcpIpRecvIPV6, it is converted to the WDM field value EVENT_RECVMSG; If the EventType field value is TcpIpConnectIPV4, TcpIpReconnectIPV6, TcpIpConnectIPV6, or TcpIpReconnectIPV4, it is converted to the WDM field value EVENT_CONNECT; If the EventType field value is TcpIpAcceptIPV4 or TcpIpAcceptIPV6, it is converted to the WDM field value EVENT_ACCEPT; If the EventType field value is FileIOWrite, DiskIOWriteInit, or DiskIOWrite, it is converted to the WDM field value EVENT_WRITE; If the EventType field value is DiskIOReadInit, DiskIORead, or FileIORead, it is converted to the WDM field value EVENT_READ; If the EventType field value is FileIOFileCreate, RegisterCreate, or RegistryKCBCreate, it is converted to the WDM field value EVENT_CREATE_OBJECT; If the EventType field value is FileIORename, it is converted to the WDM field value EVENT_RENAME; If the EventType field value is RegistrySetValue or RegistryDeleteValue, it is converted to the WDM field value EVENT_MODIFY_FILE_ATTRIBUTES; If the EventType field value is FileIOCreate or RegisterOpen, it is converted to the WDM field value EVENT_OPEN; If the EventType field value is FileIOCleanup, FileIOFlush, FileIOClose, RegistryFlush, RegistryClose, TcpIpDisconnectIPV4, or TcpIpDisconnectIPV6, it is converted to the WDM field value EVENT_CLOSE; If the EventType field value is file and registry, it is converted to the WDM field value FILE_OBJECT_FILE; If the EventType field value is RegistrySetInformation, it is converted to the WDM field value EVENT_UPDATE; If the EventType field value is FileIODirEnum, RegistryQuery, RegistryQueryValue, RegistryEnumerateKey, RegistryEnumerateValueKey, or RegistryQueryMultipleValue, it is converted to the WDM field value EVENT_ITERATE; If the EventType field value is FileIOFileRundown, ProcessDCStart, ThreadDCStart, or RegistryKCBRundownBegin, it is converted to the WDM field value EVENT_DC_STAET; If the EventType field value is ProcessDCEnd, ThreadDCEnd, or RegistryKCBRundownEnd, it is converted to the WDM field value EVENT_DC_END; If the EventType field value is FileIODelete, FileIOFileDelete, RegistryDelete, or RegistryKCBDelete, it is converted to the WDM field value EVENT_DELETE.

3. The real-time streaming data attack detection method based on label transmission according to claim 2 is characterized in that: The filtering of valid system call events according to the system call event type includes: System call events of type EVENT_READ, EVENT_WRITE, EVENT_CREATE_THREAD, EVENT_EXECUTE, EVENT_LOADLIBARY, EVENT_DELETE, EVENT_RENAME, EVENT_CREATE_OBJECT, EVENT_UPDATE, EVENT_EXIT, EVENT_LOADELF, EVENT_OPEN, EVENT_CLOSE, and EVENT_CREATE_THREAD are considered valid system call events.

4. The real-time streaming data attack detection method based on label transmission according to claim 1 is characterized in that: The process and file labels in the initialization system call event are cached, and the caching process is as follows: If the process is initially assigned a label, the process ID, process attributes, and process label are cached; If the file is initially tagged, the file ID, file attributes, and file tags are cached.

Images