A kubernetes-based pod terminal behavior auditing method and system

Abstract

This invention relates to the field of software development technology, and in particular to a method and system for auditing Pod terminal behavior based on Kubernetes. The method includes: S1. Establishing a login terminal on a local client and requesting a connection to the target Pod; S2. Obtaining the node running the target Pod, as well as the address and port of the Kubelet, through the Kube-Apiserver service; S3. Forwarding the request from the Kube-Apiserver service to the container runtime via the Kubelet; S4. Establishing a WebSocket connection between the target Pod and the client, and creating an audit-controller service; S5. Creating an audit-record-sync service. The audit-record-sync service consumes messages from the Kafka message queue and messages pushed by the audit-controller service, and synchronously writes the messages to the database. Through the collaborative work of the audit-controller and audit-record-sync services, the interaction information between the client and the target Pod can be completely recorded, enabling auditing of operations performed by users logging into the Pod terminal in a container cloud cluster.

Description

Technical Field

[0001] This invention relates to the field of software development technology, and in particular to a method and system for auditing Pod terminal behavior based on Kubernetes. Background Technology

[0002] Kubernetes is an open-source container orchestration engine for automating the deployment, scaling, and management of containerized applications. A Pod is the smallest deployable and manageable unit in Kubernetes. A Pod can contain one or more closely related containers that share resources. These containers typically run on the same host and share resources such as network namespaces and storage volumes. Pod endpoint behavior auditing refers to the process of monitoring, recording, and analyzing the operational behavior of endpoints within Pods running in Kubernetes. By auditing Pod endpoint behavior, we can understand the user's actions within the Pod, including executed commands and accessed resources, enabling security monitoring, troubleshooting, and compliance checks.

[0003] In today's rapidly evolving digital age, cloud platform cluster environments have become the mainstream choice for many enterprises and organizations to deploy business services. With the continuous maturation and widespread adoption of cloud computing technology, an increasing number of critical business services are running in container form on cloud platforms. This containerized deployment approach brings numerous advantages, such as efficient resource utilization, rapid deployment, and scalability. However, with the rapid development of information technology and the increasing sophistication of cyberattack methods, information security issues have become increasingly severe. Against this backdrop, the security requirements for business services within cloud platforms have reached unprecedented levels. Especially in cloud platform cluster environments, the security and traceability of Pod operations, as the carriers of containers, are of paramount importance.

[0004] In the daily operations of enterprises and organizations, user actions on Pod terminals involve all aspects of business services, including data processing, configuration changes, service startup and shutdown, etc. If these operations lack effective supervision and auditing, they may lead to a series of serious security problems. For example, unauthorized users may accidentally or maliciously tamper with critical configurations, thereby affecting the normal operation of the business; sensitive data may be leaked due to improper operations, causing huge economic losses and reputational damage to the enterprise; in addition, untraceable operations make it difficult to troubleshoot and determine responsibility when problems occur.

[0005] Therefore, to meet increasingly stringent security requirements, the need for auditing Pod operations has become increasingly urgent. Comprehensive and accurate recording and auditing of user actions performed upon logging into Pod terminals not only enables the timely detection of potential security risks but also provides administrators with necessary information for effective security management and decision-making, ensuring the stable and secure operation of business services on the cloud platform. Summary of the Invention

[0006] Auditing user actions performed in Pods within a Kubernetes cluster suffers from two main challenges. Firstly, Kubernetes itself lacks native support for auditing Pod terminal user operations. Secondly, while some third-party tools exist, they primarily analyze Kubernetes audit logs, focusing mainly on cluster-level auditing and offering insufficient support for auditing Pod terminal behavior. To address this issue of recording user operations and providing them to administrators, this invention provides a Kubernetes-based method for auditing Pod terminal behavior.

[0007] Firstly, the present invention provides a Pod terminal behavior auditing method based on Kubernetes, which adopts the following technical solution:

[0008] A method for auditing Pod endpoint behavior based on Kubernetes includes:

[0009] S1. Establish a login terminal on the local client and request a connection to the target Pod. The login terminal request is sent to the Kube-Apiserver service via the HTTP protocol.

[0010] S2. Obtain the node running the target Pod, as well as the address and port of the Kubelet, through the Kube-Apiserver service;

[0011] S3. Requests from the Kube-Apiserver service are forwarded to the container runtime via Kubelet, and processes are created and executed in the container namespace and cgroup of the target Pod;

[0012] S4. The target Pod establishes a WebSocket connection with the client, creates an audit-controller service, and pushes the client's input to Kafka.

[0013] S5. Create an audit-record-sync service. The audit-record-sync service consumes messages from the Kafka message queue and messages pushed by the audit-controller service, and synchronously writes the messages to the database to achieve persistent storage of client operation records.

[0014] Furthermore, the URL format of the Kube-Apiserver service is as follows:

[0015] http[s]: / / ${api_server} / ns / ${pod_namespace} / pods / ${pod_name} / exec?stdin=true&stdout=true&stderr=true&tty=true&container=${container_name}&command=sh.

[0016] Furthermore, the URL format between the Kube-Apiserver service and the Kubelet is as follows:

[0017] http[s]: / / ${node_ip}:${kubelet_port} / exec / ${pod_namespace} / ${pod_name} / ${container_name``.

[0018] Furthermore, obtaining the address and port of the node running the target Pod and the Kubelet includes establishing a streaming proxy connection between the address and port of the node running the target Pod and the Kubelet.

[0019] Furthermore, after establishing the streaming proxy connection, Kubelet receives requests from the Kube-Apiserver service and forwards them to the container runtime. Kubelet continues to proxy I / O streams between the API service and the container runtime for data transmission and interaction.

[0020] Furthermore, the specific steps for the audit-controller service to push client input to Kafka are as follows: the controller of the audit-controller service intercepts the response of the Pod establishing a WebSocket connection with the client; the target Pod directly connects to the audit-controller service; the audit-controller service creates a new WebSocket connection with the client; the audit-controller service obtains the client input and pushes the client input to Kafka through the audit-controller service.

[0021] Furthermore, the Kafka is configured with a threshold time. If the threshold time is exceeded, the push fails, and the input content is stored in the controller. At the same time, Kafka starts a separate process to listen for push failure information and restarts the push until the push is successful.

[0022] Secondly, a Kubernetes-based Pod endpoint behavior auditing system includes:

[0023] The request connection module is configured to establish a login terminal on the local client to request a connection to the target Pod. The login terminal request is sent to the Kube-Apiserver service via the HTTP protocol.

[0024] The port acquisition module is configured to obtain the node running the target Pod and the address and port of the Kubelet through the Kube-Apiserver service;

[0025] The resource management module is configured such that requests to the Kube-Apiserver service are forwarded to the container runtime via Kubelet, and processes are created and executed in the container namespace and cgroup of the target Pod.

[0026] The data processing module is configured to establish a WebSocket connection between the target Pod and the client, create an audit-controller service, and push the client's input to Kafka.

[0027] The data synchronization module is configured to create an audit-record-sync service, which consumes messages from the Kafka message queue and messages pushed by the audit-controller service, and synchronously writes the messages to the database to achieve persistent storage of client operation records.

[0028] Thirdly, the present invention provides a computer-readable storage medium storing a plurality of instructions adapted to be loaded and executed by a processor of a terminal device as described in the Kubernetes-based Pod terminal behavior auditing method.

[0029] Fourthly, the present invention provides a terminal device, including a processor and a computer-readable storage medium, wherein the processor is used to implement various instructions; the computer-readable storage medium is used to store multiple instructions, the instructions being adapted to be loaded and executed by the processor to provide the Kubernetes-based Pod terminal behavior auditing method.

[0030] In summary, the present invention has the following beneficial technical effects:

[0031] 1. A Kubernetes-based method for auditing Pod terminal behavior, which creates audit-controller and audit-record-sync services. Through the collaborative work of audit-controller and audit-record-sync services, the interaction information between the client and the target Pod can be completely recorded, including the client's input and the target Pod's response. This helps enterprises trace and analyze user operations, promptly identify potential security risks and issues, and achieve auditing of operations performed by users logging into Pod terminals in container cloud clusters.

[0032] 2. In this invention, the audit-controller is responsible for establishing a WebSocket connection between the target Pod and the client, obtaining the client's input, sending the information to the Kafka message middleware, and having the audit-record-sync service consume and write it to the database. This avoids the blocking and performance issues caused by directly writing large amounts of data to the database. Simultaneously, setting a default timeout and a separate process to listen for unsuccessfully pushed messages and attempt to restart the push ensures data reliability and integrity, preventing data loss due to push failures. Through this intermediary proxy, the client and server are unaware of the audit-controller's existence, ensuring uninterrupted communication. Users can enjoy a smooth operating experience when using the Pod terminal, without experiencing noticeable delays or stuttering due to the audit mechanism.

[0033] 3. Audit-record-sync is responsible for consuming Kafka messages. By pushing client input to Kafka and synchronously writing it to the database, it achieves persistent storage of client operation records, facilitating post-event traceability. The overall solution is reasonably designed, realizing comprehensive auditing and recording of Pod end-user operation behavior, improving system security and reliability. Enterprises can deploy business applications to container cloud platforms and enjoy the advantages and convenience brought by container cloud technology. Attached Figure Description

[0034] Figure 1 This is a schematic diagram illustrating access via a virtual IP and port provided by a Service resource in the prior art of Embodiment 1 of the present invention;

[0035] Figure 2 This is a schematic diagram of the interaction between the user terminal and the Pod in the prior art of Embodiment 1 of the present invention;

[0036] Figure 3 This is the difference between the virtual machine method and the Pod method in Embodiment 1 of the present invention;

[0037] Figure 4 This is a schematic diagram of the method flow of Embodiment 1 of the present invention;

[0038] Figure 5 This is a schematic diagram illustrating the operation of the audio-controller service and the audio-record-sync service in Embodiment 1 of the present invention. Detailed Implementation

[0039] The present invention will be further described in detail below with reference to the accompanying drawings.

[0040] 1. Kubernetes: An open-source container orchestration engine used to automate the deployment, scaling, and management of containerized applications;

[0041] 2. Service: Used to define a logical group of Pods and the strategies for accessing them;

[0042] 3. Pod: In Kubernetes, a Pod is the smallest deployable unit. It is a collection of closely related containers that share network and storage resources and are scheduled and managed together.

[0043] 4. WebSocket: A protocol that enables full-duplex communication over a single TCP connection, allowing real-time, bidirectional data transmission between browsers and servers without frequent HTTP requests and responses;

[0044] 5. HTTP: Hypertext Transfer Protocol, is the most widely used network protocol on the Internet;

[0045] 6. Kube-Apiserver: It is one of the core components of the Kubernetes control plane. It is responsible for providing the Kubernetes API and serves as the hub for communication between various components within the cluster.

[0046] 7. URL: Uniform Resource Locator, is an address used to identify resources on the Internet. It provides a standard way to locate and access various resources on the network, such as web pages, images, files, databases, etc.

[0047] 8. Kubelet: This is the worker node agent in a Kubernetes cluster. It runs on each node and is responsible for managing the Pods and containers on that node.

[0048] 9. API: Application Programming Interface, is a set of rules and interfaces that define the interaction between software components;

[0049] 10. Audit Controller: A custom service defined in this invention;

[0050] 11. Kafka: Kafka is a distributed stream processing platform, primarily used for processing real-time and streaming data.

[0051] 12. audit-record-sync: A custom service defined in this invention;

[0052] 13. Client: The client application itself;

[0053] 14. CRI: Container Runtime Interface, a standard interface defined by Kubernetes for interacting with different container runtimes;

[0054] 15. MySQL: A widely used relational database management system.

[0055] Example 1

[0056] In the prior art, referring to Figure 1 In a cloud platform cluster environment, specifically a Kubernetes-based container cloud cluster, accessing a service requires using the virtual IP address and port provided by the Service resource. The service is then forwarded to the corresponding Pod according to specified forwarding rules within the cluster. The relationship between a Service and a Pod is maintained based on the label specified in the Service. Therefore, in a Kubernetes cluster, the Pod is the one truly responsible for providing the work.

[0057] However, in daily work, it is often necessary to log in to the service environment to check logs or debug programs. When checking logs or debugging programs, users may accidentally perform dangerous operations, causing service crashes or even damage to company assets. Therefore, an operation audit mechanism is needed to ensure that all user operations are recorded for easy traceability afterward.

[0058] Reference Figure 2 Currently, in virtual machine mode, the operation auditing capabilities of the operating system itself are very mature. However, in container cloud cluster scenarios, since Pods cannot be directly seen on the nodes, logging into the Pod terminal mainly involves establishing a WebSocket-like connection between the container runtime and the Pod, and then all interactions between the user and the Pod are transmitted through this connection.

[0059] Reference Figure 3 The difference between the virtual machine approach and the Pod approach is that user input cannot be detected at the outer layer.

[0060] For the problems in the existing technology, refer to Figure 4 This embodiment of a Kubernetes-based Pod terminal behavior auditing method adopts the following technical solution, including:

[0061] S1. Establish a login terminal on the local client and request a connection to the target Pod. The login terminal request is sent to the Kube-Apiserver service via the HTTP protocol.

[0062] The URL format of the Kube-Apiserver service is:

[0063] http[s]: / / ${api_server} / ns / ${pod_namespace} / pods / ${pod_name} / exec?stdin=true&stdout=true&stderr=true&tty=true&container=${container_name}&command=sh.

[0064] The URL format between the Kube-Apiserver service and the Kubelet is as follows:

[0065] http[s]: / / ${node_ip}:${kubelet_port} / exec / ${pod_namespace} / ${pod_name} / ${container_name``.

[0066] S2. Obtain the node running the target Pod, as well as the address and port of the Kubelet, through the Kube-Apiserver service;

[0067] The process of obtaining the address and port of the node running the target Pod and the Kubelet includes establishing a streaming proxy connection between the address and port of the node running the target Pod and the Kubelet.

[0068] After establishing the streaming proxy connection, Kubelet receives requests from the Kube-Apiserver service and forwards them to the container runtime. Kubelet continues to proxy I / O streams between the API service and the container runtime for data transmission and interaction.

[0069] S3. Requests from the Kube-Apiserver service are forwarded to the container runtime via Kubelet, and processes are created and executed in the container namespace and cgroup of the target Pod;

[0070] S4. The target Pod establishes a WebSocket connection with the client, creates an audit-controller service, and pushes the client's input to Kafka.

[0071] The specific steps are as follows: The controller of the audit-controller service intercepts the response of the Pod establishing a WebSocket connection with the client. The target Pod directly connects to the audit-controller service. The audit-controller service creates a new WebSocket connection with the client. The audit-controller service obtains the client's input content and pushes the client's input content to Kafka through the audit-controller service. This can avoid the blocking and performance problems caused by directly writing a large amount of data to the database.

[0072] The Kafka is configured with a threshold time. If the threshold time is exceeded, the push will fail, and the input content will be stored in the controller. At the same time, Kafka starts a separate process to listen for push failure information and restarts the push until the push is successful.

[0073] By setting a default timeout and a separate process to listen for messages that failed to be pushed and attempt to restart the push, the reliability and integrity of the data are ensured, and no data loss will occur due to push failure. Through the intermediate proxy, the client and server are unaware of the existence of the audit-controller, and it does not affect the communication between the client and the server. Users can enjoy a smooth operating experience when using the Pod terminal, and will not feel any noticeable delay or lag due to the introduction of the audit mechanism.

[0074] S5. Create an audit-record-sync service. The audit-record-sync service consumes messages from the Kafka message queue and messages pushed by the audit-controller service, and synchronously writes the messages to the database to achieve persistent storage of client operation records.

[0075] audit-record-sync is responsible for consuming Kafka messages. By pushing client input to Kafka and synchronously writing it to the database, it achieves persistent storage of client operation records, facilitating post-event traceability. The overall solution is reasonably designed, realizing comprehensive auditing and recording of Pod end-user operation behavior, and improving the security and reliability of the system.

[0076] Reference Figure 5In a Kubernetes cluster, the audit-controller service and the audit-record-sync service are implemented. The audit-controller service is responsible for establishing a WebSocket connection between the target Pod and the client. The audit-controller service first establishes the WebSocket connection with the client, and then the audit-controller service establishes the WebSocket connection with the client. Although there is an extra layer in the middle, the client and the server are unaware of this intermediate proxy and it does not affect the communication between the client and the server. The intermediate proxy can be understood as a pass-through module. Thus, the audit-controller can obtain the client's input. After obtaining the input, it can send the information to the Kafka message middleware. This is to prevent a large number of connected Pods from directly writing to the database, which would cause blocking and affect the client experience. After the audit-controller is responsible for writing to Kafka, another service, audit-record-sync, is responsible for consuming the Kafka messages and finally synchronously writing them to the database. Through the collaborative work of the audit-controller and audit-record-sync services, the interaction information between the client and the target Pod can be completely recorded, including the client's input and the target Pod's response. This helps enterprises trace and analyze user operations, promptly discover potential security risks and problems, and achieve auditing of the operations performed by users logging into the Pod terminal in the container cloud cluster.

[0077] Example 2

[0078] This embodiment provides a Pod endpoint behavior auditing system based on Kubernetes, including:

[0079] The request connection module is configured to establish a login terminal on the local client to request a connection to the target Pod. The login terminal request is sent to the Kube-Apiserver service via the HTTP protocol.

[0080] The port acquisition module is configured to obtain the node running the target Pod and the address and port of the Kubelet through the Kube-Apiserver service;

[0081] The resource management module is configured such that requests to the Kube-Apiserver service are forwarded to the container runtime via Kubelet, and processes are created and executed in the container namespace and cgroup of the target Pod.

[0082] The data processing module is configured to establish a WebSocket connection between the target Pod and the client, create an audit-controller service, and push the client's input to Kafka.

[0083] The data synchronization module is configured to create an audit-record-sync service, which consumes messages from the Kafka message queue and messages pushed by the audit-controller service, and synchronously writes the messages to the database to achieve persistent storage of client operation records.

[0084] A computer-readable storage medium storing a plurality of instructions adapted for loading and execution by a processor of a terminal device, as described in the Kubernetes-based Pod terminal behavior auditing method.

[0085] A terminal device includes a processor and a computer-readable storage medium, the processor being configured to implement various instructions; the computer-readable storage medium being configured to store multiple instructions adapted for loading and execution by the processor, as described in the Kubernetes-based Pod terminal behavior auditing method.

[0086] The above are all preferred embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Therefore, all equivalent changes made in accordance with the structure, shape and principle of the present invention should be covered within the scope of protection of the present invention.

Claims

1. A method for auditing Pod endpoint behavior based on Kubernetes, characterized in that, Includes the following steps: S1. Establish a login terminal on the local client and request a connection to the target Pod. The login terminal request is sent to the Kube-Apiserver service via the HTTP protocol. S2. Obtain the node running the target Pod, as well as the address and port of the Kubelet, through the Kube-Apiserver service; S3. Requests from the Kube-Apiserver service are forwarded to the container runtime via Kubelet, and processes are created and executed in the container namespace and cgroup of the target Pod; S4. The target Pod establishes a WebSocket connection with the client, creates an audit-controller service, and pushes the client's input to Kafka. S5. Create an audit-record-sync service, which consumes messages from the Kafka message queue and messages pushed by the audit-controller service, and synchronously writes the messages to the database to achieve persistent storage of client operation records; The specific steps for the audit-controller service to push client input to Kafka are as follows: the controller of the audit-controller service intercepts the response of the Pod establishing a WebSocket connection with the client; the target Pod directly connects to the audit-controller service; the audit-controller service creates a new WebSocket connection with the client; the audit-controller service obtains the client input and pushes the client input to Kafka through the audit-controller service.

2. The Pod terminal behavior auditing method based on Kubernetes according to claim 1, characterized in that, Obtaining the address and port of the node running the target Pod and the Kubelet includes establishing a streaming proxy connection between the node running the target Pod and the address and port of the Kubelet.

3. The Pod terminal behavior auditing method based on Kubernetes according to claim 2, characterized in that, After establishing the streaming proxy connection, Kubelet receives requests from the Kube-Apiserver service and forwards them to the container runtime. Kubelet continues to proxy I / O streams between the Kube-Apiserver service and the container runtime for data transmission and interaction.

4. The Pod terminal behavior auditing method based on Kubernetes according to claim 1, characterized in that, The Kafka is configured with a threshold time. If the threshold time is exceeded, the push will fail, and the input content will be stored in the controller. At the same time, Kafka starts a separate process to listen for push failure information and restarts the push until the push is successful.

5. A Pod endpoint behavior auditing system based on Kubernetes, characterized in that, include: The request connection module is configured to establish a login terminal on the local client to request a connection to the target Pod. The login terminal request is sent to the Kube-Apiserver service via the HTTP protocol. The port acquisition module is configured to obtain the node running the target Pod and the address and port of the Kubelet through the Kube-Apiserver service; The resource management module is configured such that requests to the Kube-Apiserver service are forwarded to the container runtime via Kubelet, and processes are created and executed in the container namespace and cgroup of the target Pod. The data processing module is configured to establish a WebSocket connection between the target Pod and the client, create an audit-controller service, and push the client's input to Kafka. The data synchronization module is configured to create an audit-record-sync service, which consumes messages from the Kafka message queue and messages pushed by the audit-controller service, and synchronously writes the messages to the database to achieve persistent storage of client operation records.

6. A computer-readable storage medium storing a plurality of instructions, characterized in that, The instructions are adapted to be loaded and executed by the processor of the terminal device as described in claim 1, which is a Kubernetes-based Pod terminal behavior auditing method.

7. A terminal device, comprising a processor and a computer-readable storage medium, wherein the processor implements instructions; and the computer-readable storage medium stores multiple instructions, characterized in that, The instructions are adapted to be loaded and executed by a processor as described in claim 1, which is a Kubernetes-based Pod endpoint behavior auditing method.

Images