Container Threat Monitoring System and Method Based on Complex Event Processing

Abstract

The present invention discloses a container threat monitoring system and method based on complex event processing. The container information collection agent module is used to obtain container information and transmit the container information to the container information collection server; the container information collection server is used to preprocess the container information and forward the preprocessed container information to the message queue cluster module; the message queue cluster module is used to send the preprocessed container information to the information consumption service; the information consumption service module is used to package the preprocessed container information into container events and send them to the complex event processing engine module; the complex event processing engine module is used to determine the container event type, perform correlation analysis based on the container event and the container event type, and obtain container threat detection information. The system and method simplify the complexity of container threat detection based on the client / server mode, monitor the security situation of the internal operation of the container, and realize real-time early warning of container threat detection information.

Description

Technical Field

[0001] The present invention belongs to the technical field of network information security, and particularly relates to a container threat monitoring system and method based on complex event processing. Background Art

[0002] With the gradual deepening of the global digital wave, cloud computing has become an important infrastructure for the development of informatization. More and more traditional application developments are gradually transforming towards cloud-native applications. Through continuous development, enrichment, and practical implementation, the concept of cloud-native has become an important driving force for the innovative development of digital businesses. With the gradual popularization of cloud-native, more and more security problems are faced by container technology and cloud-native, especially the security risks during container runtime. Intruders will use diverse intrusion means for attacks, such as virus and malicious program attacks, intrusion behaviors inside containers, high-risk operations, etc. Traditional security technical means, such as vulnerability scanning tools, security baseline scanning, firewalls, etc., often perform security detections for cloud-native environments and cannot monitor the security conditions of the operations inside containers. Summary of the Invention

[0003] Object of the Invention: The technical problem to be solved by the present invention is to provide a container threat monitoring system and method based on complex event processing in view of the deficiencies of the prior art.

[0004] To solve the above technical problem, in the first aspect, a container threat monitoring system based on complex event processing is disclosed, which includes a container information collection proxy module, a container information collection server, a message queue cluster module, an information consumption service module, and a complex event processing engine module.

[0005] The container information collection proxy module is used to obtain container information and transmit the container information to the container information collection server; the container information includes container log information, container process information, container inter-call information, and container traffic information.

[0006] The container information collection server is used to preprocess the container information and forward the preprocessed container information to the message queue cluster module.

[0007] The message queue cluster module is used to receive the preprocessed container information and send the preprocessed container information to the information consumption service.

[0008] The information consumption service module is used to obtain the preprocessed container information from the message queue cluster module, package the preprocessed container information into container events, and send them to the complex event processing engine module.

[0009] The complex event processing engine module is used to define event types and event rules for container information. After receiving container events from the information consumption service module, it performs event rule matching on the container events to obtain container event results, and conducts correlation analysis based on the container event results to obtain container threat detection information.

[0010] Further, the container information collection agent module includes a container log collection service agent unit, a container process information collection service agent unit, an inter-container call information collection service agent unit, and a container traffic information collection service agent unit.

[0011] The container traffic information collection service agent is used to obtain container log information and transmit the log information to the container information collection server.

[0012] The container process information collection service agent unit is used to obtain container process information and transmit the process information to the container information collection server.

[0013] The inter-container call information collection service agent unit is used to obtain inter-container call information and transmit the call information to the container information collection server.

[0014] The container traffic information collection service agent unit is used to obtain container traffic information and transmit the traffic information to the container information collection server.

[0015] Further, the container information collection server includes an information collection server, an information preprocessing server, and an information forwarding server.

[0016] The information collection server is used to collect container log information, container process information, inter-container call information, and container traffic information transmitted by the container information collection agent module.

[0017] The information preprocessing server is used to preprocess the container information collected by the information collection server. The preprocessing includes information data cleaning, information data integration, information data transformation, and information data reduction.

[0018] The information forwarding server is used to forward the preprocessed container information to the message queue cluster module, and send information to the message queue cluster module using the interface msgSend. msgSend is composed of a five-tuple <msgSId, msgSContent, msgSSize, msgSType, msgSFlag>, where msgSId represents the identification number of the sending message queue; msgSContent represents the content of the message to be stored; msgSSize represents the length of the recorded data.

[0019] msgSType represents the type of the message; msgSFlag represents the sending method of the message, which has two methods: blocking and non-blocking.

[0020] Furthermore, the message queue cluster module includes a container log information queue cluster, a container process information queue cluster, an inter-container call information queue cluster, and a container traffic information queue cluster.

[0021] The container log information queue cluster is used to receive the container log information transmitted by the container information collection server and send it to the information consumption service module.

[0022] The container process information queue cluster is used to receive the container process information transmitted by the container information collection server and send it to the information consumption service module.

[0023] The inter-container call information queue cluster is used to receive the inter-container call information transmitted by the container information collection server and send it to the information consumption service module.

[0024] The container traffic information queue cluster is used to receive the container traffic information transmitted by the container information collection server and send it to the information consumption service module.

[0025] Furthermore, the information consumption service module uses the interface msgReceive to receive information from the message queue cluster module. msgReceive consists of a five-tuple <msgRId, msgRContent, msgRSize, msgRType, msgRFlag>. Among them, msgRId represents the identification number of the receiving message queue; msgRContent represents the content of the message to be received and stored; msgRSize represents the length of the received message data; mgsRType represents the type of the received message; msgRFlag represents the receiving method of the message, which has two methods: blocking and non-blocking.

[0026] The information consumption service module packages the preprocessed container information into container events, including:

[0027] An event is composed of a seven - tuple <eventId, eventName, superType, eventType, eventContent, eventLength, timeStamp> object, where eventId represents the unique identifier id of the event; eventName represents the name of the event; superType represents the parent type of the event, eventType represents the specific type of the event; eventContent represents the body content of the event; eventLength represents the length of the event body; timeStamp represents the time stamp when the event is recorded. When the information consumption service module receives container information, it packages the information into an event of the corresponding event type in the complex event engine module.

[0028] Furthermore, the definition of event types and event rules for container information by the complex event processing engine module includes:

[0029] The event types and event rules in the container threat monitoring system are divided into two categories. Category one: event types and event rules for correlation analysis rules; Category two: event types and event rules for matching container information.

[0030] Create event types and event rules based on Category one and Category two, and register the event types and event rules into the complex event processing engine. The event types and event rules have a one - to - one correspondence, and events need to be processed through the event rules corresponding to the event types.

[0031] To create an event type, a base class Event needs to be created first, with the following format:

[0032]

[0033]

[0034] Among them, superType represents the parent type of the event, 1 represents the correlation analysis category, and 2 represents the container information category;

[0035] The subclass inherits the Event base class to define childEvent, and initializes eventName, superType, and eventTyp to implement various event types;

[0036] Event rules are defined using EPL (Event Processing Language) statements.

[0037] Furthermore, the complex event processing engine module performs event rule matching processing on container events to obtain container event results, including:

[0038] After receiving the container event from the information consumption module, the complex event engine module determines the event type of this event according to the eventType attribute in the container event, looks up the corresponding event rule based on the event type, puts the container event into the event rule, and obtains the container event result after being processed by the event rule;

[0039] Among them, the container event result logItem of the container log information event after being processed by the container log information processing rule, the container event result courseItem of the container process information event after being processed by the container process information processing rule, the container event result containerItem of the inter-container call information after being processed by the inter-container information processing rule, and the container event result flowItem of the container traffic information after being processed by the container traffic information processing rule;

[0040] The complex event processing engine module performs correlation analysis based on the container event result to obtain container threat detection information, including:

[0041] The form of the correlation analysis event rule for Category 1 is as follows: Antecedent -> Consequent, where both the antecedent and the consequent are item sets, that is, event sets, -> represents the correlation relationship, the antecedent represents the condition of the correlation rule, and the consequent represents the conclusion derived from the correlation rule.

[0042] Take the container event results logItem, courseItem, containerItem, and flowItem after processing by the container event rule as the antecedent into the correlation analysis event rule, and analyze to obtain the consequent conclusion, that is, container threat detection information;

[0043] The container threat detection information has the following structure:

[0044]

[0045] Among them, the threat level menaceLevel is divided into three levels (1 represents low-level threat, 2 represents medium-level threat, 3 represents high-level threat); the threat type menaceType includes seven categories (1 represents container escape, 2 represents isolation failure, 3 represents system kernel vulnerability exploitation, 4 represents container image vulnerability, 5 represents insecure configuration, 6 represents malicious image injection, 7 represents others).

[0046] In a second aspect, a container threat monitoring method based on complex event processing is disclosed. Using the above-mentioned container threat monitoring system based on complex event processing, it includes the following steps:

[0047] Step 1: When the container is running, the container information collection agent module collects container information in real time and forwards it to the container information collection server; the container information includes container log information, container process information, container inter - call information, and container traffic information;

[0048] Step 2: The container information collection server pre - processes the obtained container information and forwards it to the message queue cluster module;

[0049] Step 3: The information consumption service obtains the corresponding information from the message queue cluster module and wraps the information into a container event and forwards it to the complex event processing engine;

[0050] Step 4: The complex event processing engine performs rule matching processing on the received container events to obtain container event results;

[0051] Step 5: Perform correlation analysis based on the container event results to obtain container threat detection information.

[0052] Further, step 4 includes:

[0053] After the complex event engine module receives the container event from the information consumption module, it determines the event type of this event according to the eventType attribute in the event object, looks up the corresponding event rule based on the event type, puts the container event into the event rule, and obtains the container event result after the event rule processing;

[0054] Among them, the container event result logItem of the container log information event through the container log information processing rule, the container event result courseItem of the container process information event through the container process information processing rule, the container event result containerItem of the container inter - call information through the container inter - information processing rule, and the container event result flowItem of the container traffic information through the container traffic information processing rule.

[0055] Further, step 5 includes:

[0056] The form of the correlation analysis event rule for Category 1 is as follows: Antecedent —> Consequent, where both the antecedent and the consequent are item sets, that is, event sets, —> represents the correlation relationship, the antecedent represents the condition of the correlation rule, and the consequent represents the conclusion derived from the correlation rule.

[0057] Take the container event results logItem, courseItem, containerItem, and flowItem after the container event rule processing as the antecedent into the correlation analysis event rule, and analyze to obtain the consequent conclusion, that is, the container threat detection information.

[0058] The structure of the container threat detection information is as follows:

[0059] {

[0060] "id": "Threat number id";

[0061] "containerId": "Threatened container id";

[0062] "containerName": "Threatened container name";

[0063] "containerIp": "Threatened container address";

[0064] "menaceLevel": "Threat level";

[0065] "menaceType": "Threat type";

[0066] "menaceDetails": "Threat details";

[0067] "solution": "Solution";

[0068] "timeStamp": "Timestamp"

[0069] }

[0070] Among them, the threat level menaceLevel is divided into three levels (1 indicates low-level threat, 2 indicates medium-level threat, and 3 indicates high-level threat); the threat type menaceType includes seven categories (1 indicates container escape, 2 indicates isolation failure, 3 indicates system kernel vulnerability exploitation, 4 indicates container image vulnerability, 5 indicates insecure configuration, 6 indicates malicious image injection, and 7 indicates others).

[0071] Beneficial effects:

[0072] 1. Simplify complexity: Adopting the client / server mode can decompose the complex container threat detection process into two parts, the client and the server, making the entire detection process simpler and more manageable.

[0073] 2. Introduce a complex event processing engine: It can flexibly process and analyze the events generated during the container information processing process. It is convenient to monitor the security situation inside the container and give real-time warnings of potential threats.

[0074] 3. Real-time warning: Compared with traditional security technical means, the present invention can achieve real-time warning of container threat detection information. The system can timely discover and respond to potential threats inside the container, thereby reducing potential risks and improving security.

[0075] In summary, the present invention simplifies the complexity of container threat detection based on the client / server mode, and introduces a complex event processing engine in the process of container information processing. Compared with traditional security technical means, it monitors the security situation inside the container and realizes real-time early warning of container threat detection information. Description of the Drawings

[0076] The following will further specifically describe the present invention in conjunction with the drawings and specific embodiments, and the above and / or other advantages of the present invention will become clearer.

[0077] Figure 1 It is a schematic structural diagram of a container threat monitoring system based on complex event processing provided by an embodiment of the present application. Specific Embodiments

[0078] The embodiments of the present invention will be described below in conjunction with the drawings.

[0079] The container threat monitoring system and method based on complex event processing provided by the present application can be applied to a system with a microservices architecture. The microservices architecture divides an application into multiple small, independently deployable services, and each service runs in an independent container. When a security threat occurs to a container at a certain node, it will affect the normal operation of the entire system.

[0080] As Figure 1 shown, the first embodiment of the present application discloses a container threat monitoring system based on complex event processing, including a container information collection proxy module, a container information collection server, a message queue cluster module, an information consumption service module, and a complex event processing engine module.

[0081] The container information collection proxy module is used to obtain container information and transmit the container information to the container information collection server; the container information includes container log information, container process information, container inter-call information, and container traffic information; in a specific implementation process, the container information collection proxy module can be deployed at the node where the container is located, or the container information collection proxy module can be deployed in each container.

[0082] The container information collection server is used to preprocess the container information and forward the preprocessed container information to the message queue information collection server for the column cluster module;

[0083] The message queue cluster module is used to receive the preprocessed container information and send the preprocessed container information to the information consumption service;

[0084] The information consumption service module is used to obtain the preprocessed container information from the message queue cluster module, package the preprocessed container information into container events, and send them to the complex event processing engine module;

[0085] The complex event processing engine module is used to define event types and event rules for container information. When receiving container events from the information consumption service module, it performs event rule matching processing on the container events to determine the container event types, and conducts correlation analysis based on the container events and container event types to obtain container threat detection information.

[0086] The container information collection agent module is used to obtain container information and send the container information to the container information collection server; the container information includes container log information, container process information, container inter - call information, and container traffic information;

[0087] The container information collection server is used to preprocess the container information and forward the preprocessed container information to the message queue cluster module;

[0088] The message queue cluster module is used to receive the preprocessed container information and send the preprocessed container information to the information consumption service;

[0089] The information consumption service module is used to obtain the preprocessed container information from the message queue cluster module, package the preprocessed container information into container events, and send them to the complex event processing engine module;

[0090] The complex event processing engine module is used to define event types and event rules for container information. When receiving container events from the information consumption service module, it performs event rule matching processing on the container events to obtain container event results, and conducts correlation analysis based on the container event results to obtain container threat detection information.

[0091] In this embodiment, the container information collection agent module includes a container log collection service agent unit, a container process information collection service agent unit, a container inter - call information collection service agent unit, and a container traffic information collection service agent unit.

[0092] The container log collection service proxy unit is used to obtain container log information for container devices providing standard protocol interfaces and transmit the log information to the container information collection server. Container logs refer to the log data generated during the operation of containers. These logs can help managers and developers ensure the smooth operation of containers, troubleshoot problems, and perform fault diagnosis. Filebeat is used to collect log files. Filebeat consists of three parts: inputs, harvesters, and spooler. Among them, inputs are responsible for configuring the path to read log files, harvesters are responsible for reading the content of log files, and spooler is used to temporarily cache log data and send log information. After Filebeat is started, it matches log files according to the input configuration. After finding a log file, it starts a harvester for each file. The harvester reads the log information and passes it to the spooler, and the spooler sends the log information to the information collection server.

[0093] The container process information collection service proxy unit is used to obtain container process information for container devices providing standard protocol interfaces and transmit the process information to the container information collection server. The collection of process information is responsible by a dedicated process detection class. The collection of process data is divided into the collection of process port data and other data. The process port timing task reads the variable port list at regular intervals to obtain the port numbers to be collected. Then, for the Windows environment, jpcap is used to obtain the network card object, and a TCP filter is set on the network card to count the port traffic within a time period. For the Linux environment, a Python script is called to open a socket and analyze the passing data packets to obtain the information. When the process detection task starts, it creates a data container of the map structure to store the collected process information. Each process is distinguished by the pid, which serves as the key of the map. The task will scan all processes to obtain the pid and ports. For each process, it will collect the connection count, CPU, memory occupancy, disk read data, and network port traffic data by running system commands. Finally, the task will send the process information to the information collection server. The collection of process data comes from the following data sources respectively:

[0094] 1) System commands: including CPU, memory, connection count, etc. (such as the top command)

[0095] 2) Subdirectories of each process under the / proc directory: including information such as CPU, memory, disk read and write, etc.

[0096] 3) Execution scripts: including the collection of port traffic data in the Linux environment

[0097] 4) Third-party toolkits: including port traffic data collection (jpcap) in the Windows environment

[0098] The container inter-call information collection service proxy unit is used to obtain container inter-call information for container devices providing standard protocol interfaces and transmit the call information to the container information collection server.

[0099] The main container communication methods mainly include the following:

[0100] 1) Through the container IP: Since the IP will change after the container restarts.

[0101] 2) Through the host's IP:port: Rely on listening to the exposed port process for limited communication.

[0102] 3) Establish a connection through link: When running a container, specify the parameter link so that the source container and the linked container can communicate with each other, and the receiving container can obtain some data of the source container. The linked container can ping the source container to achieve communication, and vice versa.

[0103] 4) Through user-defined networds: Use docker network to create a bridge network, and specify the container to the created bridge network when running docker run, so that the containers in the same bridge network can access each other.

[0104] The container traffic information collection service proxy unit is used to obtain container traffic information for container devices providing standard protocol interfaces and transmit the traffic information to the container information collection server. Use ebpf (Extended Berkeley Packet Filter) to collect container traffic. Ebpf counts network traffic at different levels by tracking kernel functions. The main differences in traffic at each level are mainly in packet headers, retransmissions, control messages, etc. Containers are created based on cgroups. When a process in a container is generated, its task_struct—>cgroups will record the relevant information of the cgroup it belongs to. The ebpf program runs in the kernel state and can directly access the task_struct structure of the process, and then obtain its cgroup ID. When counting traffic, it is convenient to distinguish which cgroup the current network traffic belongs to. Ebpf can collect traffic data based on the tcp and udp network protocols at the transport layer; it can collect traffic data based on the ip network protocol at the network layer and all network packet traffic data at the data link layer. Ebpf forwards the collected container traffic information to the information collection server according to the kernel function.

[0105] In this embodiment, the container information collection server includes an information collection server, an information preprocessing server, and an information forwarding server.

[0106] The information collection server is used to collect container log information, container process information, container inter - call information, and container traffic information transmitted by the container information collection agent module.

[0107] The information preprocessing server is used to preprocess the container information collected by the information collection server. The following preprocessing operations are performed on the collected container information:

[0108] Information data cleaning: Complete the missing values in the data, eliminate noise data, identify or delete outliers, and resolve inconsistencies.

[0109] Information data integration: Data from different sources, formats, and characteristics are organically centralized logically or physically, and the data in multiple data sources are integrated and stored uniformly.

[0110] Information data transformation: Transform the data into a form suitable for data mining through methods such as smoothing aggregation, data generalization, and normalization.

[0111] Information data reduction: Since the data information is very large, by reducing or simplifying the data set, the integrity of the metadata can be maintained, and the result after data reduction is almost the same as the result before reduction.

[0112] The information forwarding server is used as a message queue producer to forward the preprocessed container log information, container process information, container inter - call information, and container traffic information to the corresponding container log information queue cluster, container process information queue cluster, container inter - call information queue cluster, and container traffic information queue cluster. The information forwarding service uses the interface msgSend to send information to the message queue. msgSend is composed of a five - tuple <msgSId, msgSContent, msgSSize, msgSType, msgSFlag>. Among them, msgSId represents the identification number of the sending message queue; msgSContent represents the content of the message to be stored, and the message structure can be designed according to actual requirements; msgSType represents the type of the message; msgSSize represents the length of the recorded data, excluding the part of the message type of the sending message, and must be greater than 0; msgSFlag represents the sending method of the message, and there are two methods: blocking (0) and non - blocking (IPC_NOWAIT).

[0113] In this embodiment, the message queue cluster module includes a container log information queue cluster, a container process information queue cluster, a container inter - call information queue cluster, and a container traffic information queue cluster.

[0114] A message queue is a first-in, first-out queue data structure, which is actually an internal linked list in the system kernel. Messages are inserted into the queue sequentially, where the sending process adds information to the tail of the queue, and the receiving process reads messages from the head of the queue. Once a message is read, it is deleted from the queue.

[0115] The message format in the message queue is as follows:

[0116] class Message

[0117] {

[0118] Int mType,

[0119] Object mContent

[0120] }

[0121] Among them, mtype represents the message type (number 1: log information, number 2: process information, number 3: inter-container call information, number 4: traffic information), and mContent represents the message content in json format. After introducing the message type, the message queue is logically transformed from a single message linked list into multiple message linked lists. The sending process still unconditionally writes messages to the tail of the queue, and the receiving process can choose to read the one closest to the head of the queue among messages of a specific type (the message queue is a linked list in the kernel). Once a message is read, it is deleted from the queue, and other messages remain unchanged.

[0122] The container log information message queue cluster is used to receive the container log information passed by the information collection server and send it to the information consumption service.

[0123] The container process information message queue cluster is used to receive the container process information passed by the information collection server and send it to the information consumption service.

[0124] The container call information message queue cluster is used to receive the inter-container call information passed by the information collection server and send it to the information consumption service.

[0125] The container traffic information message queue cluster is used to receive the container traffic information passed by the information collection server and send it to the information consumption service.

[0126] In this embodiment, the information consumption service is used to obtain container log information, container process information, inter-container call information, and container traffic information from the message queue cluster.

[0127] The information consumption service uses the interface msgReceive to receive information from the message queue. msgReceive consists of a five-tuple <msgRId, msgRContent, msgRSize, msgRType, msgRFlag>. Among them, msgRId represents the identification number of the received message queue; msgRContent represents the received message content, which is the JSON format message body; msgRSize represents the length of the received message data, excluding the part of the message type; mgsRType represents the type of the received message; msgRFlag represents the receiving method of the message, and there are two methods: blocking (0) and non-blocking (IPC_NOWAIT).

[0128] An event generally refers to something happening in a system. Events can occur at various levels of the system. It can be an action, such as a user accessing a database, downloading a file, or inter-process calls, or a change in a certain state, such as a change in CPU utilization, disk read / write rate, or request response speed. The information consumption service module wraps the preprocessed container information into container events, including:

[0129] The event Event consists of a seven-tuple <eventId, eventName, superType, eventType, eventContent, eventLength, timeStamp> object. Among them, eventId represents the unique identification id of the event; eventName represents the name of the event; superType represents the parent type of the event, eventType represents the specific type of the event; eventContent represents the body content of the event; eventLength represents the length of the event body; timeStamp represents the timestamp of the event record. When the information consumption service module receives the container information, it wraps the information into an event corresponding to the event type in the complex event engine module.

[0130] In this embodiment, complex event processing (CEP) is an emerging event stream-based technology. It regards system data as different types of events, analyzes the relationships between events, establishes different event relationship sequence libraries, and uses technologies such as filtering, correlation, and aggregation to finally generate high-level events or business processes from simple events. Suitable scenarios for CEP include real-time risk management, real-time transaction analysis, network fraud, network attacks, market trend analysis, and so on.

[0131] The definition of event types and event rules for the container information by the complex event processing engine module includes:

[0132] The event types and event rules of the container threat monitoring system are divided into two categories. Category 1: Event types and event rules for correlation analysis rules; Category 2: Event types and event rules for processing container flow information (container log information, container process information, container inter-call information, container traffic information).

[0133] Create a complex event processing engine module based on the CEP (complex event processing) complex event processing engine. Create event types and event rules based on Category 1 and Category 2, and register the event types and event rules into the complex event processing engine.

[0134] To create an event type, first create an Event base class with the following format:

[0135]

[0136] Among them, eventId represents the Id of the event, eventName represents the event name, superType represents the type (number 1: correlation analysis category, number 2: container flow information category), eventType represents the specific event type, eventContent represents the event body content, eventLength represents the body length, timeStamp represents the timestamp, and "…" represents the getter, setter, and contructor methods.

[0137] The subclass inherits the Event base class to define childEvent and initializes eventName and eventTyp to implement various event types.

[0138] For example: Container resource usage event type

[0139]

[0140] The event rule eventRule is defined using EPL statements. For example, select hostIp, hostUrl, count(*) as requestNum from containerInterviewEvernt:time_batch(1hour) group by hostIp, hostUrl order by requestNum desc. This EPL statement is used to statistically analyze the number of times the host (hostIp) and address (url) are accessed (requestNum) within 1 hour.

[0141] There is a one-to-one correspondence between event types and event rules, and events need to be processed through the event rules corresponding to the event types.

[0142] The complex event processing engine module performs event rule matching on container events, and the obtained container event results include:

[0143] After receiving the container event from the information consumption module, the complex event engine module determines the event type (eventType) of this event according to the eventType attribute in the event object, looks up the corresponding event rule (eventRule) by eventType, puts the event object into eventRule, and obtains the container event result eventItem after being processed by eventRule.

[0144] Among them, the event result logItem of the container log information event is processed by the container log information processing rule, the event result courseItem of the container process information event is processed by the container process information processing rule, the event result containerItem of the container - to - container call information is processed by the container - to - container information processing rule, and the event result flowItem of the container traffic information is processed by the container traffic information processing rule.

[0145] Define the container event type and the event generation rules. When the container log information, container process information, container - to - container call information, and container - to - container traffic information match the container information log processing rule, container process information processing rule, container - to - container call information processing rule, and container traffic information processing rule, they conform to the corresponding event types.

[0146] The complex event processing engine module performs correlation analysis based on the container event results, and the obtained container threat detection information includes:

[0147] The form of the correlation analysis event rule (category one) is as follows: Antecedent —> Consequent, where both the antecedent and the consequent are item sets (event sets), and “—>” represents the correlation relationship. The antecedent represents the condition of the correlation rule, and the consequent represents the conclusion deduced by the correlation rule.

[0148] Take the container event results logItem, courseItem, containerItem, and flowItem after being processed by the event rule as the antecedent to enter the correlation rule analysis to obtain the consequent conclusion, that is, the container threat detection information.

[0149] The structure of the container threat detection information is as follows:

[0150]

[0151] Among them, the threat level menaceLevel is divided into three levels (1 represents low-level threat, 2 represents medium-level threat, and 3 represents high-level threat); the threat type menaceType includes seven categories (1 represents container escape, 2 represents isolation failure, 3 represents system kernel vulnerability exploitation, 4 represents container image vulnerability, 5 represents insecure configuration, 6 represents malicious image injection, and 7 represents others).

[0152] When it is detected that the container security is threatened, threat alerts can be triggered according to the above container threat detection information.

[0153] The second embodiment of this application discloses a container threat monitoring method based on complex event processing. Using the above container threat monitoring system based on complex event processing, it includes the following steps:

[0154] Step 1, when the container is running, the container information collection agent module collects container log information, container process information, container inter-call information, and container traffic information in real time and forwards them to the container information collection server.

[0155] Step 1-1, when the container is running, the container information collection agent classifies and collects container log information, container process information, container inter-call information, and container traffic information.

[0156] Step 1-2, the information collection agent forwards the collected container log information, container process information, container inter-call information, and container traffic information to the container information collection server in categories.

[0157] Step 2, the container information collection server collects and preprocesses the collected information and forwards it to the corresponding message queue cluster.

[0158] Step 2-1, the container information collection server collects the container information forwarded by the container information collection agent.

[0159] Step 2-2, the container information collection server classifies and preprocesses the container log information, container process information, container inter-call information, and container traffic information according to the configured data preprocessing strategy.

[0160] Step 2-3, the container collection server, as a producer, sends the container log information, container process information, container inter-call information, and container traffic information to the container log information consumption queue, container process information consumption queue, container inter-call information consumption queue, and container traffic information consumption queue respectively.

[0161] Step 3, the information consumption service obtains the corresponding information from the message queue cluster module and wraps the information into events and forwards them to the complex event processing engine.

[0162] Step 3-1: The information consumption service obtains container log information, container process information, container inter-call information, and container traffic information from the container log information queue, container process information queue, container inter-call information queue, and container traffic information queue respectively.

[0163] Step 3-2: The information consumer service packages the information into events and forwards them to the complex event processing engine. Among them, packaging the information into events specifically includes:

[0164] An event generally refers to something happening in a system. Events can occur at various levels of the system. It can be an action, such as a user accessing a database, downloading a file, or processes calling each other, or a change in a certain state, such as a change in CPU utilization, disk read / write rate, or request response speed. An event Event consists of a seven-tuple <eventId, eventName, superType, eventType, eventContent, eventLength, timeStamp> object, where eventId represents the unique identifier id of the event; eventName represents the name of the event; superType represents the parent type of the event, eventType represents the specific type of the event; eventContent represents the body content of the event; eventLength represents the length of the event body; timeStamp represents the timestamp of the event record. When the information consumption service module receives container information, it packages the information into an event corresponding to the event type in the complex event engine.

[0165] Step 4: The complex event processing engine performs rule matching processing on the received container events to obtain container event results.

[0166] Step 4-1: The complex event processing engine receives container event information.

[0167] Step 4-2: Respectively perform matching processing on the received container log events, container inter-process events, container inter-call events, and container traffic events with the container log information processing rules, container inter-process information processing rules, container inter-call information processing rules, and container traffic information processing rules. After event rule matching processing, the corresponding container event results are obtained, specifically including:

[0168] The event types and event rules of the container threat monitoring system are divided into two major categories. Category 1: Event types and event rules for correlation analysis rules; Category 2: Event types and event rules for processing container flow information (container log information, container process information, container inter-call information, container traffic information).

[0169] Create a complex event processing engine module based on the CEP (complex event processing) complex event processing engine. Create event types and event rules based on Category 1 and Category 2, and register the event types and event rules into the complex event processing engine.

[0170] When creating the event type, first create an Event base class in the following format:

[0171]

[0172] Among them, eventId represents the Id of the event, eventName represents the event name, superType represents the type (number 1: association analysis category, number 2: container flow information category), eventType represents the specific event type, eventContent represents the event body content, eventLength represents the body length, timeStamp represents the timestamp, and "…" represents the getter, setter, and contructor methods.

[0173] The subclass inherits the Event base class to define childEvent, and initializes eventName and eventTyp to implement various event types.

[0174] For example: the event type of container resource usage

[0175]

[0176] Among them, the event rule is defined using an EPL statement. For example, select hostIp, hostUrl, count(*) as requestNum from containerInterviewEvernt:time_batch(1hour) group by hostIp, hostUrl order by requestNum desc. This EPL statement is used to statistically analyze the number of times the host (hostIp) and address (url) are accessed (requestNum) within 1 hour.

[0177] Among them, there is a one-to-one correspondence between the event type and the event rule, and the event needs to be processed through the event rule corresponding to the event type.

[0178] After the complex event engine receiving module receives the events of the information consumption module, it determines the event type (eventType) of this event according to the eventType attribute in the event object, looks up the corresponding event rule (eventRule) by eventType, puts the event object into eventRule, and obtains the container event result eventItem after being processed by eventRule.

[0179] Among them, the container log information event is the container event result logItem of the container log information processing rule, the container process information event is the container event result courseItem of the container process information processing rule, the container inter-call information is the container event result containerItem of the container inter-information processing rule, and the container traffic information is the container event result flowItem of the container traffic information processing rule.

[0180] Step 5: Conduct correlation analysis based on the container event results to obtain container threat detection information.

[0181] Step 5-1: Perform correlation analysis rule processing on the events passing through the processing rules.

[0182] Association reflects the interdependent relationship between a certain thing and other things. Association analysis refers to finding the association patterns existing between item sets in events, that is, if there is a certain association between two or more things, then one thing can be predicted through other things.

[0183] Perform correlation analysis on the events captured by the container log information processing rule, the container process information processing rule, the container inter-call information processing rule, and the container inter-traffic information processing rule, specifically including:

[0184] The form of the correlation analysis event rule (category one) is as follows: Antecedent —> Consequent, where both the antecedent and the consequent are item sets (event sets), and "—>" represents the association relationship. The antecedent represents the condition of the association rule, and the consequent represents the conclusion derived from the association rule. After processing the event rule, logItem, courseItem, containerItem, and flowItem are used as the antecedent to enter the association rule analysis;

[0185] Step 5-2: Output the container threat check information obtained after correlation analysis, specifically including:

[0186] The structure of the container threat check information is as follows:

[0187]

[0188] Among them, the threat level menaceLevel is divided into three levels (1 represents low-level threat, 2 represents medium-level threat, and 3 represents high-level threat); the threat type menaceType includes seven categories (1 represents container escape, 2 represents isolation failure, 3 represents system kernel vulnerability exploitation, 4 represents container image vulnerability, 5 represents insecure configuration, 6 represents malicious image injection, and 7 represents others).

[0189] When it is detected that the container security is threatened, threat alerts can be triggered according to the above container threat check information.

[0190] In a specific implementation, the present application provides a computer storage medium and a corresponding data processing unit. Among them, the computer storage medium can store a computer program, and when the computer program is executed by the data processing unit, it can run the content of the invention of the container threat monitoring method based on complex event processing and some or all of the steps in each embodiment. The storage medium can be a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM), etc.

[0191] Those skilled in the art can clearly understand that the technical solutions in the embodiments of the present invention can be implemented by means of a computer program and its corresponding general hardware platform. Based on such an understanding, the technical solutions in the embodiments of the present invention, in essence, or the part that contributes to the prior art can be embodied in the form of a computer program, that is, a software product. The computer program software product can be stored in the storage medium and includes several instructions for causing a device (which can be a personal computer, a server, a single-chip microcomputer, a MUU, or a network device, etc.) including a data processing unit to execute the methods described in each embodiment or some parts of the embodiments of the present invention.

[0192] The present invention provides a container threat monitoring method based on complex event processing. There are many methods and ways to specifically implement this technical solution. The above are only the specific implementation manners of the present invention. It should be noted that for those of ordinary skill in the art, without departing from the principle of the present invention, several improvements and refinements can be made, and these improvements and refinements should also be regarded as the protection scope of the present invention. Each component not clearly defined in this embodiment can be implemented by the prior art.

Claims

1. A container threat monitoring system based on complex event processing, characterized in that It includes a container information collection agent module, a container information collection server, a message queue cluster module, an information consumption service module, and a complex event processing engine module. The container information collection agent module is used to obtain container information and transmit the container information to the container information collection server; the container information includes container log information, container process information, container inter - call information, and container traffic information. The container information collection server is used to pre - process the container information and forward the pre - processed container information to the message queue cluster module. The message queue cluster module is used to receive the pre - processed container information and send the pre - processed container information to the information consumption service. The information consumption service module is used to obtain the pre - processed container information from the message queue cluster module, package the pre - processed container information into container events, and send them to the complex event processing engine module. The complex event processing engine module is used to define event types and event rules for container information. When receiving container events from the information consumption service module, it performs event rule matching processing on the container events to obtain container event results, and conducts correlation analysis based on the container event results to obtain container threat detection information.

2. The container threat monitoring system based on complex event processing according to claim 1, characterized in that, The container information collection agent module includes a container log collection service agent unit, a container process information collection service agent unit, a container inter - call information collection service agent unit, and a container traffic information collection service agent unit. The container traffic information collection service agent is used to obtain container log information and transmit the log information to the container information collection server. The container process information collection service agent unit is used to obtain container process information and transmit the process information to the container information collection server. The container inter - call information collection service agent unit is used to obtain container inter - call information and transmit the call information to the container information collection server. The container traffic information collection service agent unit is used to obtain container traffic information and transmit the traffic information to the container information collection server.

3. The container threat monitoring system based on complex event processing according to claim 2, characterized in that, The container information collection server includes an information collection server, an information pre - processing server, and an information forwarding server. The information collection server is used to collect the container log information, container process information, container inter - call information, and container traffic information transmitted by the container information collection agent module. The information pre - processing server is used to pre - process the container information collected by the information collection server. The pre - processing includes information data cleaning, information data integration, information data transformation, and information data reduction. The information forwarding server is used to forward the preprocessed container information to the message queue cluster module, and send information to the message queue cluster module using the interface msgSend. msgSend consists of a five-tuple <msgSId, msgSContent, msgSSize, msgSType, msgSFlag>. Among them, msgSId represents the identification number of the sending message queue; msgSContent represents the content of the message to be stored for sending; msgSSize represents the length of the recorded data; msgSType represents the type of the message; msgSFlag represents the sending method of the message, and there are two methods: blocking and non-blocking.

4. A container threat monitoring system based on complex event processing according to claim 3, characterized in that, The message queue cluster module includes a container log information queue cluster, a container process information queue cluster, an inter-container call information queue cluster, and a container traffic information queue cluster. The container log information queue cluster is used to receive the container log information transmitted by the container information collection server and send it to the information consumption service module. The container process information queue cluster is used to receive the container process information transmitted by the container information collection server and send it to the information consumption service module. The inter-container call information queue cluster is used to receive the inter-container call information transmitted by the container information collection server and send it to the information consumption service module. The container traffic information queue cluster is used to receive the container traffic information transmitted by the container information collection server and send it to the information consumption service module.

5. The container threat monitoring system based on complex event processing according to claim 4, characterized in that The information consumption service module receives information from the message queue cluster module using the interface msgReceive. msgReceive consists of a five-tuple <msgRId, msgRContent, msgRSize, msgRType, msgRFlag>. Among them, msgRId represents the identification number of the receiving message queue; msgRContent represents the content of the message to be stored for receiving; msgRSize represents the length of the received message data; mgsRType represents the type of the received message; msgRFlag represents the receiving method of the message, and there are two methods: blocking and non-blocking. The information consumption service module packages the preprocessed container information into container events, including: An event is composed of a seven-tuple <eventId, eventName, superType, eventType, eventContent, eventLength, timeStamp> object, where eventId represents the unique identification id of the event; eventName represents the name of the event; superType represents the parent type of the event, and eventType represents the specific type of the event; eventContent represents the body content of the event; eventLength represents the length of the event body; timeStamp represents the time stamp of the event record. When the information consumption service module receives the container information, it packages the information into an event of the corresponding event type in the complex event engine module.

6. The container threat monitoring system based on complex event processing according to claim 5, wherein, The definition of event types and event rules for container information by the complex event processing engine module includes: The event types and event rules in the container threat monitoring system are divided into two categories. Category 1: Event types and event rules for correlation analysis rules; Category 2: Event types and event rules for matching container information. Create event types and event rules based on Category 1 and Category 2, and register the event types and event rules into the complex event processing engine. The event types and event rules have a one-to-one correspondence, and events need to be processed through the event rules corresponding to the event types. To create an event type, a base class Event needs to be created first, and the format is as follows: class Event{ Long eventId; String eventName; String superType; String eventType; Object eventContent; Long eventLength; Date timeStamp; … } Among them, superType represents the parent type of the event. 1 represents the correlation analysis category, and 2 represents the container information category. The subclass inherits the Event base class to define childEvent and initializes eventName, superType, and eventTyp to implement various event types. The event rules are defined using EPL statements.

7. An in-container threat monitoring system based on complex event processing according to claim 6, wherein, The complex event processing engine module performs event rule matching processing on container events to obtain container event results, including: After the complex event engine module receives the container event from the information consumption module, it determines the event type of this event according to the eventType attribute in the event object, looks up the corresponding event rule by the event type, puts the container event into the event rule, and obtains the container event result after being processed by the event rule. Among them, the container event result logItem of the container log information event through the container log information processing rule, the container event result courseItem of the container process information event through the container process information processing rule, the container event result containerItem of the container - to - container call information through the container - to - container information processing rule, and the container event result flowItem of the container traffic information through the container traffic information processing rule; The complex event processing engine module performs correlation analysis based on the container event results to obtain container threat detection information, including: The form of the correlation analysis event rule for Category 1 is as follows: Antecedent -> Consequent, where both the antecedent and the consequent are item sets, that is, event sets, -> represents the correlation relationship, the antecedent represents the condition of the correlation rule, and the consequent represents the conclusion deduced by the correlation rule; Taking the container event results logItem, courseItem, containerItem, and flowItem after processing by the container event rule as the antecedent into the correlation analysis event rule to analyze and obtain the consequent conclusion, that is, the container threat detection information; The container threat detection information has the following structure: { "id": "threat number id"; "containerId": "threatened container id"; "containerName": "threatened container name"; "containerIp": "threatened container address"; "menaceLevel": "threat level"; "menaceType": "threat type"; "menaceDetails": "threat details"; "solution": "solution"; "timeStamp": "timestamp" } Among them, the threat level menaceLevel includes three levels: 1 represents low - level threat, 2 represents medium - level threat, and 3 represents high - level threat; the threat type menaceType includes seven categories: 1 represents container escape, 2 represents isolation failure, 3 represents system kernel vulnerability exploitation, 4 represents container image vulnerability, 5 represents insecure configuration, 6 represents malicious image injection, and 7 represents others.

8. A container threat monitoring method based on complex event processing, using the container threat monitoring system based on complex event processing described in any one of claims 1-7, characterized in that, It includes the following steps: Step 1, when the container is running, the container information collection proxy module collects container information in real - time and forwards it to the container information collection server; the container information includes container log information, container process information, container - to - container call information, and container traffic information; Step 2, the container information collection server pre - processes the obtained container information and forwards it to the message queue cluster module; Step 3, the information consumption service obtains the corresponding information from the message queue cluster module and wraps the information into a container event and forwards it to the complex event processing engine; Step 4, the complex event processing engine performs rule matching processing on the received container events to obtain container event results; Step 5, perform correlation analysis based on the container event results to obtain container threat detection information.

9. The container threat monitoring method based on complex event processing according to claim 8, wherein The said Step 4 includes: After receiving the container event from the information consumption module, the complex event engine module determines the event type of this event according to the eventType attribute in the event object, looks up the corresponding event rule based on the event type, puts the container event into the event rule, and obtains the container event result after being processed by the event rule; Among them, the container event result logItem of the container log information event is processed by the container log information processing rule, the container event result courseItem of the container process information event is processed by the container process information processing rule, the container event result containerItem of the inter-container call information is processed by the inter-container information processing rule, and the container event result flowItem of the container traffic information is processed by the container traffic information processing rule.

10. A container threat monitoring method based on complex event processing according to claim 9, characterized in that, The said step 5 includes: The form of the association analysis event rule of category one is as follows: Antecedent -> Consequent, where both the antecedent and the consequent are item sets, that is, event sets, -> represents the association relationship, the antecedent represents the condition of the association rule, and the consequent represents the conclusion deduced by the association rule. Taking the event results logItem, courseItem, containerItem, and flowItem after being processed by the event rule as the antecedent and entering the association analysis event rule to analyze and obtain the consequent conclusion, that is, the container threat detection information; the structure of the container threat detection information is as follows: { "id": "threat number id"; "containerId": "threatened container id"; "containerName": "threatened container name"; "containerIp": "threatened container address"; "menaceLevel": "threat level"; "menaceType": "threat type"; "menaceDetails": "threat details"; "solution": "solution"; "timeStamp": "timestamp" } Among them, the threat level menaceLevel includes three levels: 1 represents low-level threat, 2 represents medium-level threat, and 3 represents high-level threat; the threat type menaceType includes seven categories: 1 represents container escape, 2 represents isolation failure, 3 represents system kernel vulnerability exploitation, 4 represents container image vulnerability, 5 represents insecure configuration, 6 represents malicious image injection, and 7 represents others.

Images