Container nested running method and system based on cloud native to realize security isolation

Abstract

The application provides a container nesting running method and system based on cloud native and realizing security isolation, and belongs to the technical field of cloud computing and container virtualization. The system comprises: a dynamic injection component configured to intercept a request of deploying an application container group and inject configuration information of an auxiliary component into the application container group; the configuration information of the auxiliary component comprises: a definition of an initialization container, a definition of a sidecar container and a definition of a shared storage volume; the initialization container is configured to deploy a container management tool to the shared storage volume; a business container is configured to mount the shared storage volume and send a container operation request through the container management tool; and the sidecar container is configured to run a daemon process of a container management service end, receive and execute the container operation request. By isolating the daemon process into an independent sidecar container, the business container serves as a client and does not need to run in a privileged mode, realizing security isolation of the container nesting running and zero invasion to a business application.

Description

Technical Field

[0001] This application relates to the fields of cloud computing and container virtualization technology, and in particular to a method and system for nested container operation based on cloud-native security isolation. Background Technology

[0002] In CI / CD pipelines (such as GitLab CI, Jenkins, Tekton, etc.) for AI model training / inference, it is often necessary to perform container building or nested container tasks within containers; this is called container nesting technology. Currently, the mainstream implementation methods include: (1) Docker outside Docker (DoD): In the DoD scheme, the outer container does not directly run an independent container management server daemon (such as Docker Daemon). Instead, it communicates with the container management server daemon on the host machine through a shared communication interface of the host machine (also called a node, Host) (such as Docker Socket, usually mounted under the / var / run / docker.sock path on the host machine). This enables remote control of the container management server daemon on the host machine, and then the container management server daemon on the host machine executes the actual container building or running tasks. The advantage of this scheme is that it is simple to implement and has low performance overhead. However, since the outer container has complete control over the host machine through the host machine's daemon, there are obvious security risks. Moreover, for cloud platform vendors, multiple tenants sharing the host machine's daemon can cause mutual interference between the build tasks of different tenants, which can easily lead to problems such as image naming conflicts and resource contention. Therefore, although the DoD scheme is easy to implement, it is often not recommended in cloud platforms or production environments with high security and isolation requirements.

[0003] (2) Docker in Docker (DinD): An independent container management server daemon runs inside the outer container, obtaining a nested container building and runtime environment. This container environment is decoupled from the host machine's container environment, avoiding the security risks caused by directly manipulating the host machine's daemon process. Although this solution is better than the DoD solution in terms of isolation, its implementation depends on the outer container having privileged permissions, that is, it needs to run in privileged mode. There is still a problem of container security boundary being violated. In addition, the outer container in privileged mode needs to be configured with additional mount paths, permissions and network settings. Currently, most of these configurations are performed by the business application development team, which is complex for the business team to deploy and maintain. For container cloud platform vendors (providing cloud platforms), it is also not conducive to the vendor's cluster administrators to perform large-scale automated management. In addition, since the container environment of the outer container is decoupled from the host machine, when the host machine enables heterogeneous GPU devices (such as NVIDIA or Metax, etc.), the inner container often cannot correctly identify or access these heterogeneous GPU resources, resulting in build failure or training tasks not obtaining GPU acceleration effects.

[0004] (3) Dedicated build tools: such as Kaniko and Buildah, which build and run new images by simulating the configuration and code of container images in user space, thereby achieving image management without daemons and privileged mode. However, this approach cannot fully simulate the standard container management (such as Docker) environment, has compatibility issues, and is difficult to handle complex container orchestration tasks.

[0005] In summary, existing solutions all have varying degrees of security, isolation, and compatibility issues, making it difficult to simultaneously meet the requirements of security, efficiency, and ease of use in cloud-native environments. Therefore, there is an urgent need to provide an improved technical solution. Summary of the Invention

[0006] The purpose of this application is to provide a method and system for secure container nesting based on cloud-native architecture. By running the container management daemon within the sidecar container, the business application and the daemon are isolated in two different containers within the same container group (Pod). The business container does not need to run in privileged mode to perform container management operations, achieving high security isolation and non-privileged container nesting. This solution is completely transparent to the business container, supports large-scale automated deployment, and can support the execution of complex container orchestration tasks.

[0007] To achieve the above objectives, this application provides the following technical solution: Firstly, this application provides a cloud-native-based container nesting system for secure isolation, comprising: A dynamic injection component is configured to intercept requests for application container group deployment and inject configuration information of auxiliary components into the application container group; the configuration information of the auxiliary components includes: the definition of the initialization container, the definition of the sidecar container, and the definition of the shared storage volume; Initialize the container and configure it to deploy the container management tool to the shared storage volume before the business container starts; The business container is configured to mount the shared storage volume and send container operation requests through the container management tool. The sidecar container is configured as a daemon process running the container management server, receiving and executing the container operation requests.

[0008] In the above solution, the high-privilege (privileged) daemon process is isolated into a separate sidecar container. The business container acts only as a client, sending container operation requests through the container management tool, fundamentally eliminating the need for privileged mode for the business container. Auxiliary components are injected via dynamic injection, and the container management tool is deployed using the initialization container. The business container does not need to modify its image or code; all auxiliary components are automatically configured through dynamic injection, and the container management tool becomes automatically available through shared storage volumes, achieving complete transparency to the business application. The dynamic injection component intercepts Pod deployment requests and automatically injects auxiliary component configurations, eliminating the need for any manual configuration of the Pod's YAML file, which facilitates large-scale automated deployment for container cloud platform vendors. Furthermore, the sidecar container provides a complete container management server (Docker Daemon), supporting orchestration tools such as docker-compose.

[0009] As a further improvement, the sidecar container is further configured to: create the service endpoint of the daemon process of the container management server in the shared storage volume; The service container is configured to use the container management tool to access the service endpoint in the shared storage volume and send a container operation request to the daemon process of the container management server.

[0010] In the above scheme, by creating the daemon's service endpoint (i.e., Unix socket file) on a shared storage volume, the business container and the sidecar container can operate in completely isolated process spaces. The business container only accesses the endpoint file as an unprivileged client; the sidecar container runs a high-privilege daemon independently as a server, which strengthens the security boundary, avoids the exposure of network ports and complex network policy configuration, and provides an access point for the subsequent introduction of security proxy components for request interception and security processing, as well as resource control.

[0011] As a further improvement, the auxiliary components also include a security proxy component. The security proxy component is configured to intercept container operation requests from the business container, perform security processing and resource access control on the container operation requests, and then forward them to the daemon process of the container management server. The sidecar container is further configured to receive and execute container operation requests forwarded by the security proxy component.

[0012] In the above solution, an independent security proxy component is introduced between the business container (request initiator) and the sidecar container (request executor) as the execution layer for security policies and resource control. This component performs centralized and real-time security checks and resource access control on all container operation requests, and enforces the security policies in the nested container operation layer. This achieves both physical isolation and runtime management, ensuring that even if the business container is compromised, its malicious intent can be blocked at the proxy layer. This significantly improves the system's proactive security capabilities and the strength of resource isolation in a multi-tenant environment.

[0013] As a further improvement, the security proxy component is further configured as follows: Parse the parameters in the container creation request or container run request; When runtime parameters that do not comply with security checks are detected, security processing is performed, including at least one of the following operations: removing the privileged runtime parameters; rewriting the resource access request; or rejecting the execution of a request containing the privileged runtime parameters. When access parameters of a hardware accelerator are detected, resource access control is performed, including the following steps: determining the authorized hardware accelerator resource information of the business container; rewriting the access parameters of the hardware accelerator based on the authorized hardware accelerator resource information; and forwarding the rewritten request to the daemon process of the container management server.

[0014] The above solution concretizes the two core functions of the security proxy component: security processing and resource access control, achieving fine-grained proactive security control capabilities for nested container operations. On the one hand, by parsing the parameters of container operation requests, it achieves precise identification and handling (removal, rewriting, or denial) of risks such as privileged parameters and device mounting. On the other hand, a dynamic resource rewriting mechanism is specifically designed for hardware accelerators (such as GPUs). By querying the authorization information of the business container, broad access requests (such as "--gpus all") are rewritten into a specific list of authorized devices, thus realizing the implementation of security policies.

[0015] As a further improvement, the dynamic injection component is further configured as follows: Based on the access control mechanism of the container cloud platform, the system intercepts requests for application container group deployment and parses the configuration information of the requests. Based on the parsing results, determine whether auxiliary component injection is required; in response to the need to inject auxiliary components, dynamically add the configuration information of the auxiliary components to the configuration information of the application container group.

[0016] In the above solution, the working mechanism of the dynamic injection component is concretized into automated decision-making and configuration modification based on the admission control process of the container cloud platform (such as Kubernetes). By intercepting and parsing deployment requests, the dynamic injection component can intelligently determine whether a specific application container group (Pod) needs to inject container nesting capabilities (such as based on tags or annotations), and precisely modify its configuration only when needed. This ensures that the container nesting operation capability is provided on demand and accurately, avoiding unnecessary resource overhead. At the same time, it fully complies with the standard extension mode of cloud-native platforms, so that this secure container nesting capability can be seamlessly and standardizedly integrated and enabled like a native function, simplifying the deployment and management of cluster-level functions.

[0017] As a further improvement, the container operation request includes a container creation request, a container run request, and container orchestration instructions.

[0018] By specifically listing container creation requests, container run requests, and container orchestration instructions, the complete scope of supported container operations is clarified. In other words, business containers can not only safely start (docker run) or build (docker build) a single nested container in an isolated environment, but also execute orchestration instructions such as docker-compose to deploy and manage applications with multiple nested containers, thus fully supporting the ability to deploy complex cloud-native applications and improving the applicability to practical production environments.

[0019] As a further improvement, the shared storage volume is a temporary storage volume of type emptyDir, which has the same lifecycle as the application container group.

[0020] By defining the shared storage volume as the emptyDir type, the volume's lifecycle is fully tied to the application container group (Pod): it is automatically created when the Pod is created and automatically cleaned up when the Pod is deleted. Users do not need to manually configure persistent storage or clean up residual data. This automatic lifecycle management of storage resources simplifies operations and maintenance. Furthermore, emptyDir volumes are typically set up in the host machine's memory or local disk, providing a high-performance inter-container communication channel that meets the low-latency requirements of tool sharing and socket communication.

[0021] Secondly, this embodiment provides a method for secure container nesting operation based on cloud-native technology, wherein the method is executed by the cloud-native secure container nesting operation system provided in any of the above embodiments, and includes: The dynamic injection component intercepts the application container group deployment request and injects the configuration information of the auxiliary component into the application container group; the configuration information of the auxiliary component includes: initialization container definition, sidecar container definition, and shared storage volume definition; Before the business container starts, initialize the container and deploy the container management tools to the shared storage volume; The business container mounts the shared storage volume and sends a container operation request through the container management tool; The sidecar container runs a daemon process for the container management server, which receives and executes the container operation requests.

[0022] Thirdly, this embodiment provides a computer device, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of a cloud-native method for securely isolated nested container operation.

[0023] Fourthly, this embodiment provides a computer-readable storage medium storing a computer program / instructions thereon, which, when executed by a processor, implements the steps of a cloud-native method for securely isolating nested container execution.

[0024] The technical effects of the second, third, and fourth aspects of this application can be referred to the description of the first aspect, and will not be repeated here. Attached Figure Description

[0025] Figure 1 This is a logical schematic diagram of a dynamic injection component provided according to some embodiments of this application.

[0026] Figure 2 This is a technical logic diagram of the initialization container, sidecar container, and service container (user main container) provided according to some embodiments of this application.

[0027] Figure 3 A schematic diagram of the structure of the computer device provided in this application. Detailed Implementation

[0028] To facilitate understanding of the technical solution of this application, the application scenarios are explained.

[0029] In the development, training / inference scenarios of large AI models (such as Llama, GPT, etc.), it is necessary to frequently release new model versions or launch new inference engines. Model developers / researchers want to quickly verify the effect, performance and compatibility of the new version. At this time, it is necessary to isolate a test environment so that it can perform test tasks without affecting the main development environment and other model services, and thoroughly clean up after the test is completed, leaving no residual files or environmental impact.

[0030] As a business application, large AI model testing is typically performed by model developers renting container resources from cloud container platforms. Cloud container platforms (such as Kubernetes, an open-source container orchestration platform used for automated deployment, management, and scaling of containerized applications) schedule and allocate resources in units of container groups (Pods). Running large AI model container groups is called application container groups. A Pod can contain one or more containers and can be assigned to run on any node (host machine).

[0031] When a new model configuration or version update is detected, the model developer releases a new image, triggering the CI / CD pipeline and sending a deployment request for the containerized application to the container cloud platform. The container cloud platform creates a Pod resource object, schedules the Pod to any host machine that meets the conditions, and starts the Pod. For ease of description, this Pod is referred to as the test pipeline Pod.

[0032] After the aforementioned Pod starts, the inner container is restarted via the CI / CD pipeline code to test the performance of the new model version or the new inference engine. Therefore, in this scenario, the AI ​​large model test Pod, acting as the outer container, is a lightweight CI / CD pipeline executor, responsible only for process control (such as preparing data, executing test scripts, and collecting results). The inner container, on the other hand, is the large model's testing environment, a heavy-duty container containing all the complex software stack related to the AI ​​model. The outer container uses container management technologies (such as Docker) to create and start the model test container (inner container) within itself. The test code runs in the inner container, which also runs the new AI model and the newly released engine. The testing process generates and collects performance data. Throughout the entire testing process, the inner container should be confined to the sandbox defined by the outer container, completely isolated from the host machine's real environment or other business containers. After the test, the outer container automatically destroys the inner container it created. Then, the outer container itself completes its task and is destroyed by the container cloud platform.

[0033] Based on the background technology, existing DoD, DinD, and dedicated build tool solutions all have varying degrees of security, isolation, and compatibility issues. Therefore, this application improves upon the DinD solution by injecting auxiliary components into application container groups (Pods) to separate the main business container (hereinafter referred to as the business container) from the container running the daemon process of the container management server (such as Docker Daemon, containerd, etc.). The daemon process is run by the sidecar container, and the business container does not need to run in privileged mode to perform nested container build and run tasks through the sidecar container. This provides a technical solution that is securely isolated, highly compatible, and easy to deploy and maintain.

[0034] The embodiments of this application will now be described with reference to the accompanying drawings.

[0035] This embodiment provides a cloud-native container nesting system for secure isolation. The system includes: a dynamic injection component configured to intercept application container group deployment requests and inject auxiliary component configuration information into the application container group's configuration information; the auxiliary component configuration information includes: an initialization container definition, a sidecar container definition, and a shared storage volume definition; an initialization container configured to deploy a container management tool to the shared storage volume before the business container starts; a business container, serving as the main business container of the application container group, configured to mount the shared storage volume and send container operation requests through the container management tool; and a sidecar container configured to run a daemon process of the container management server, receiving and executing container operation requests.

[0036] Furthermore, container operation requests include: container creation requests (such as the docker build command) and container running requests (such as the docker run command), and container orchestration instructions (such as the docker-compose command).

[0037] It should be noted that in container cloud platforms represented by Kubernetes (K8s), container groups (Pods) are usually used as the basic unit for resource allocation and scheduling. A Pod includes one or more containers. For example, in the embodiment of this application, since the configuration information of the application container group (Pod) is injected with the initialization container definition and the sidecar container definition, the Pod may include the initialization container and the sidecar container at different stages of its life cycle, in addition to the main business container.

[0038] To make it easier to understand, the container management process of a Kubernetes cluster is described below.

[0039] As a container orchestration and management platform, a Kubernetes cluster consists of a control plane and a data plane. The control plane runs on the Kubernetes master node and includes core control plane components such as the API Server, ETCD (Enterprise State Store), Controller Manager, and Scheduler. The data plane refers to the cluster's worker nodes, which are responsible for running containerized applications. When business application developers / testers (as users or tenants of the container cloud platform) rent and use resources from Kubernetes, the typical process involves the following two stages: Phase 1: Submission and Coordination (primarily handled in the control plane).

[0040] 1) Users submit Pod declarations (including annotations).

[0041] Business application developers / testers (hereinafter referred to as users) write configuration information (also known as configuration manifest) for application container groups Pods based on the resource requirements of business applications, and declare the expected resources (such as CPU, memory) and status of business operation to the K8s system. A typical configuration manifest is usually organized into a Pod Spec file and stored in YAML format (such as pod.yaml). After the user performs basic syntax checking and formatting on the YAML file using a command-line tool (such as kubectl apply -f pod.yaml), it is converted into an HTTP POST / PUT request and then sent to the API Server of the Kubernetes cluster control plane.

[0042] 2) Kubernetes API Server reception and verification.

[0043] Upon receiving the request, the API Server sequentially performs authentication (such as certificates and tokens), authorization checks (e.g., checking if the user has permission to create a Pod in a specified namespace), and admission control. Finally, it converts the request into a Pod object definition, which is then persistently saved as a record in the cluster state storage center ETCD. At this point, the Pod's configuration information has been transformed into its expected state and is officially recorded in Kubernetes.

[0044] The technical solution of this application is an improvement on the access control mechanism. By adding a Webhook component, the configuration information of the auxiliary component is dynamically injected on the basis of the existing configuration information of the Pod. The detailed process will be described later.

[0045] 3) The controller detects changes in a loop.

[0046] The Controller Manager in the Kubernetes cluster continuously monitors (via the List-Watch mechanism) changes to Pod resources on the API Server. When it detects a new Pod record in ETCD with a desired status of "Running" but a current status of "Pending Scheduling", the controller marks this Pod as pending scheduling.

[0047] 4) Scheduler decision.

[0048] The scheduler also continuously monitors the API Server through the List-Watch mechanism. When it finds a Pod to be scheduled in the API Server, it filters the worker nodes based on the expected resources declared in pod.yaml and selects the most suitable worker node as the worker node to run the Pod (hereinafter referred to as the target worker node).

[0049] The scheduler initiates a binding call to the API Server, writing the name of the target worker node back to the corresponding field of the Pod object in ETCD, thus binding the Pod to the target worker node.

[0050] Phase 2: Pod creation and execution (primarily executed in the data plane).

[0051] 5) Node agent (Kubelet) takeover.

[0052] The node agent Kubelet is a standard component of the data plane, running on every node (including control and worker nodes). When a Pod is bound to a target worker node, the node agent on the target worker node monitors the Pod scheduling on the API Server through a List-Watch mechanism. When it finds a Pod bound to itself (i.e., the field stored in ETCD of the Pod matches the name of this node), it retrieves the Pod's configuration information and initiates the Pod creation process.

[0053] 6) Pull the image and start the program.

[0054] Kubelet first creates a sandbox for the Pod through the Container Runtime Interface (CRI, such as containerd, CRI-O), and then executes the Pod startup process according to the definition of the Pod object.

[0055] This application improves the initialization container, business container, and sidecar container by having the sidecar container execute the daemon process. The business container no longer executes the container management server requiring privileged permissions; instead, it acts as a client, sending container management requests to the daemon process. The container management server is run by the sidecar container. Since the sidecar container is provided by a container cloud platform, this prevents the business container from gaining privileged permissions, thus improving overall security. The specific implementation process of the above technologies will be described in detail later.

[0056] 7) Status reporting and updates.

[0057] Kubelet continuously monitors the status of all containers on the local node. Once it confirms that all containers within the Pod have successfully started and entered the running state, it reports the Pod's current status (including IP address, container status, readiness status, etc.) to the API Server. The API Server then persists this status update to ETCD. At this point, the entire Pod startup process is complete.

[0058] The following is a detailed description of one of the technical improvements in this application, namely the dynamic injection scheme for configuration information.

[0059] This application employs an admission control mechanism to dynamically inject auxiliary components. In the aforementioned API Server verification process, admission control refers to a mechanism that verifies, modifies, or rejects requests after the API Server receives a resource operation request (such as a Pod deployment request) and completes authentication and authorization, but before the object is persisted to ETCD (a native distributed database deployed on Kubernetes). Admission control is divided into two main types: built-in admission controllers and custom admission controllers. Built-in admission controllers are provided by Kubernetes as plugins with fixed functionality. Currently provided built-in plugins include: NamespaceLifecycle (for managing namespace lifecycles), LimitRanger (for limiting resource usage), and PodSecurity (for enabling default Pod security policies), etc.

[0060] Custom admission controller rules are typically written and published by the cluster administrator as an admission control service (Admission Webhook). After authenticating and authorizing resource operation requests, the API Server converts the request into an AdmissionReview request and sends it to the custom admission controller. The custom admission controller processes the request and returns a result. The API Server then either continues processing or terminates processing based on the result. Because custom admission control dynamically injects external control logic into the API Server's request processing flow during Kubernetes runtime via Webhook, it is fundamentally different from the static, fixed rules within the API Server. Therefore, custom admission control is also known as "dynamic injection."

[0061] Webhook is a Kubernetes extension mechanism that can trigger the execution logic of custom admission rules when the API Server processes requests for resource objects (such as Pods).

[0062] In this embodiment, the dynamic injection component is a self-developed custom admission controller (also known as a Webhook component). Its core is a change controller built on a Mutating Admission Webhook mechanism, used to intercept the creation requests of application container groups (such as the aforementioned test pipeline Pods) and modify their configuration information (i.e., the Pod's Spec file). In other words, the dynamic injection component embeds an automated configuration modification layer (implemented by the Webhook component) into the deployment process of containerized applications. This layer can detect the resource operation requests of application deployment in real time and add the auxiliary components required for the containerized application to run accordingly. This process is completely transparent to application developers, and the business development team can obtain container nesting capabilities without any configuration. At the same time, due to the zero-intrusion processing method, cluster administrators can easily perform standardized and large-scale management, improving the management efficiency and security of platform vendors.

[0063] Specifically, the dynamically injected component is deployed on the resource request processing chain of the container orchestration platform (such as Kubernetes) (after the API Server completes the authentication and authorization of the resource operation request and before it is stored in ETCD). It is configured to: intercept the application container group deployment request based on the access control mechanism of the container cloud platform, parse the configuration information of the request; determine whether auxiliary component injection is required based on the parsing result; and dynamically add the configuration information of the auxiliary component to the configuration information of the application container group in response to the need to inject the auxiliary component.

[0064] In other words, when a user submits a Pod deployment (creation or modification) request to Kubernetes (usually submitting a Pod's YAML configuration file to K8s), the Webhook component developed in this embodiment intercepts the request and checks whether it contains a specified annotation. If the specified annotation exists, auxiliary component injection is triggered; otherwise, auxiliary component injection is not triggered. After auxiliary component injection is triggered, the Webhook component automatically injects the following configuration information into the Pod configuration: the definition of the initialization container, the definition of the sidecar container, and the definition of the shared storage volume. This injection process does not require the user to modify the image or business code, achieving zero-intrusion injection.

[0065] In detail, such as Figure 1 As shown, after intercepting the deployment request of the Pod, the Webhook component parses the configuration information of the request (i.e., the YAML definition of the Pod), obtains information such as the Pod's namespace, label, and annotation, and determines whether the current request requires auxiliary component injection. For example, if the YAML definition of the Pod contains a specified annotation (example: inject: "true"), it is determined that the Pod needs auxiliary component injection.

[0066] Subsequently, the dynamic injection component generates configuration information (called a Patch) for the auxiliary components to be injected based on a predefined template, including the definition of the initialization container (InitContainer), the definition of the sidecar container (Sidecar), and the definition of the shared storage volume.

[0067] The generation process of the auxiliary component configuration information (Patch) is dynamic (rather than statically fixed in the API Server). It can be adaptively adjusted based on the context of the original AdmissionReview request (the request body of which is the YAML definition of the Pod) (such as resource requests, node selectors, etc.). Then, the generated auxiliary component configuration information is merged into the Pod's original configuration information to form a new, complete deployment definition, which is returned to the API Server so that the API Server can continue to execute the subsequent processing flow after applying the Patch. On the other hand, if the Pod does not require auxiliary component injection (i.e., does not contain specified annotations), the dynamically injected component does not perform any operation, and the API Server directly continues to execute the subsequent processing flow.

[0068] The following reference Figure 2 The document provides a detailed explanation of the operational flow and interrelationships of the three components: the initialization container (initContainer), the sidebar container, and the business container.

[0069] In this embodiment, the initialization container (initContainer), the sidebar container, and the business container are different containers within the same application container group (Pod). The business container is used to execute the workload of the business application and implement the business logic, so it is also referred to as the main container. The initialization container (initContainer, or init container for short) and the sidebar container are special types of containers.

[0070] The init container is primarily used to prepare for the initialization of the business container. It is a dedicated container that must be successfully executed before the business container starts. It is important to note that the init container only runs during Pod startup and is typically used for pre-configuring the environment and waiting for dependent services to be ready. The lifecycle of the init container is strictly sequential with that of the business container. The sequential logic is: the business container will only start after the init container has successfully run and terminated (exit code is 0).

[0071] In this embodiment, in order to achieve the purpose of secure isolation of nested containers, the init container is configured to deploy the container management tool to the shared storage volume before the business container starts. That is, the container management tool is deployed to a file storage location (shared storage volume) that can be accessed by the business container, rather than performing operations directly related to the application business logic.

[0072] The definition of the initialization container is dynamically written into the configuration information of the Pod object by a dynamic injection component before the Pod object is created. This definition may include: the tool image used by the container, the startup command, and the shared storage volume it needs to mount. The startup command is used to deploy the container management tools to the shared storage volume. In practice, the initialization container typically corresponds to a container instance defined in the `initContainers` field of the Pod's specification (i.e., the Pod's Spec description). During Pod startup, the initialization container starts before the business container and performs initialization preparation work according to the above definition for the business container.

[0073] Here, container management tools refer to a set of client command-line tools and related dependencies used to interact with the container management server daemon to perform container lifecycle management operations. Core tools typically include a container management command-line interface (e.g., Docker CLI) for executing commands such as `docker run` and `docker build`; they may also include container orchestration tools (e.g., `docker-compose`). In this solution, these tools are pre-installed within the init container image, not within the business container image. Deployment is an operation performed by the init container to make the container management tools available and executable for the business containers. This process can be achieved by copying or extracting the corresponding files; that is, the init container copies the tool binaries and necessary library files pre-stored in its image file system to a specific directory on a shared storage volume, possibly setting appropriate file execution permissions.

[0074] In this context, a shared storage volume refers to a storage resource within the same Pod (application container group) that can be accessed simultaneously by multiple containers (at least including the init container, application container, and sidecar container). Preferably, the shared storage volume is a temporary storage volume of type emptyDir, with the same lifecycle as the application container group. This shared storage volume provides a shared, consistent file system namespace among the containers within the Pod.

[0075] In other words, in this embodiment, the shared storage volume acts as a data exchange bus: the init container deploys container management tools on the shared storage volume, the sidecar container creates the service endpoints (such as Unix socket files) of its daemon process on the shared storage volume, and the application container accesses the container management tools and connects to the service from the shared storage volume. In practice, the shared storage volume can be of various types, such as a temporary volume or a persistent volume. As a preferred implementation of this application, the shared storage volume is an emptyDir volume in Kubernetes, whose lifecycle is bound to the Pod.

[0076] The definition of a shared storage volume is dynamically written into the configuration information of a Pod object by a dynamic injection component before the Pod object is created. This can include: the name of the storage volume, its type (e.g., an emptyDir type temporary volume, which is a temporary volume synchronized with the Pod's lifecycle), and its mount point path within the container. This volume serves as a channel for file exchange (such as tool binaries and daemon sockets) between the initialization container, the sidecar container, and the business container.

[0077] After the InitContainer starts, it copies binary tools such as docker and docker-compose (collectively referred to as container management tools) to the shared emptyDir path, such as / opt / dind-tools / . The main business container can use them directly without installation.

[0078] In this embodiment, the service container is configured to mount a shared storage volume and send a container operation request through a container management tool (for example).

[0079] The business container is the main container in the Pod that runs the core business logic. Its image is configured and packaged by the business development team. It is an application image focused on business functions and is designed not to contain any container management-related tools (such as Docker CLI) or daemons (such as Docker Daemon). This setting allows the business container to run without privileged mode, fundamentally reducing the possibility of being attacked.

[0080] Sidecar containers are a special type of container that runs alongside the main container (i.e., the application container) within the same Pod. They enhance or extend the functionality of the main application container by providing additional services or features without directly modifying the main application code.

[0081] In practical implementation, taking Kubernetes as an example, the Kubernetes system treats the sidecar container as a special case of the Init container. The difference is that the Init container's lifecycle terminates after the Pod starts, while the sidecar container continues to run after the Pod starts. In other words, the sidecar container starts and runs in parallel with the application container, and has the same lifecycle as the Pod.

[0082] In this embodiment, the sidecar container is configured to run a daemon process (such as DockerDaemon) of the container management server, receiving and executing container operation requests. The daemon process of the container management server is the engine that actually performs container lifecycle operations and management (such as creating, starting, and stopping containers) and image management operations. By placing the daemon process in a separate sidecar container, physical isolation between container management capabilities and business application logic is achieved. The business container only acts as a client sending requests, while all privileged or risky underlying operations are completed within the controlled execution sandbox of the sidecar container.

[0083] The definition of the sidecar container is also dynamically written into the configuration information of the application container group Pod by the dynamic injection component. As an example, the above definition may include: specifying the image of the daemon (such as Docker Daemon) that contains the container management server, startup parameters (such as binding the server endpoint to a socket file in the shared storage volume), resource limits, and necessary security context configuration.

[0084] Furthermore, the sidecar container is configured to: create the service endpoint of the container management server daemon in the shared storage volume; and the business container is configured to use the container management tool to access the service endpoint in the shared storage volume and send container operation requests to the container management server daemon.

[0085] In this embodiment, the sidecar container still needs to start the daemon process in privileged mode. It has a built-in complete container management server (such as Docker Daemon) runtime environment. When Docker Server starts, it creates and binds the daemon process's service listening endpoint (service endpoint, a Unix domain socket file) to a specific path on the shared storage volume through preset running parameters (such as dockerd -H unix: / / / var / run / dind-docker.sock). This enables Docker Server to start on the shared path, allowing business containers in the same application container group (Pod) to access the socket by mounting this volume, thereby establishing communication.

[0086] Because the sidecar container is primarily used to run Docker Server, it is also called a "Docker serversidecar". Figure 2 As shown.

[0087] In this embodiment, the service container itself does not have the ability to directly operate and manage containers. It securely obtains container operation and management capabilities from the sidecar container through the following configuration: First, mount the shared storage volume: When the business container starts, it maps a specified path in its file system (such as / opt / dind-tools) to a shared storage volume defined in the Pod. This makes all container management tools (such as the docker executable) that the init container pre-deploys in this volume visible and accessible to the business container.

[0088] Next, integrate container management tools into the runtime environment: This can be achieved through environment variables (such as adding the tool's path ` / opt / dind-tools / bin` to the PATH) or by using absolute paths, allowing processes within the business containers to invoke these container management tools from the shared storage volume as if using native system commands. Simultaneously, the container management tools are configured with access paths for service listening endpoints. For example, the Docker CLI can access the Docker Daemon running in the sidecar container via `DOCKER_HOST=unix: / / / var / run / dind-docker.sock`, enabling nested Docker build and run capabilities.

[0089] When an application within a business container needs to perform nested container operations (e.g., starting a data processing container), the typical process is as follows: (1) Application logic (such as AI test scripts, program code) calls standard container management commands (such as executing the docker run command).

[0090] (2) Based on the preset configuration (usually the environment variable DOCKER_HOST), the request is sent to the service endpoint (such as the Unix domain socket unix: / / / var / run / dind-docker.sock) created by the sidecar container on the shared storage volume. It should be noted that this service endpoint does not point to the host machine or the business container's own daemon process, but to the independent container runtime daemon process running in the same Pod.

[0091] (3) The container management server running inside the side vehicle container actually executes the operation specified in the container management request (such as executing the docker run command).

[0092] In this embodiment, through the above settings, the daemon process runs on the side vehicle container (privileged mode required). The business container does not need to make any changes to obtain the container nesting function (no changes to the image or code). The container cloud platform provides the tools required for dynamic injection and deployment and performs connection configuration, thereby realizing the integration of container management capabilities for business containers with zero intrusion. This is beneficial for the cluster administrator of the container cloud platform to perform large-scale automated management.

[0093] By using sidecar containers, the actual execution of high-privilege (privileged mode) container management operations is isolated within a dedicated container. Even if the container management server's daemon process is attacked, its impact is limited to the sidecar container, making it difficult to directly affect business containers or the host machine. This achieves secure isolation and boundary strengthening of business containers. Secondly, business applications do not need to, and do not have, the ability to directly manage or operate the underlying containers. All actual management operations are executed by the sidecar containers, achieving complete decoupling between business logic and infrastructure operations. In addition, the sidecar container images are uniformly provided and managed by the container cloud platform, ensuring the consistency of the container management server version and configuration on all container group instances, unaffected by differences in business container images. This is conducive to building a unified container nesting runtime environment.

[0094] To further enhance the security of nested container operation, in some optional embodiments, the auxiliary components also include: a security proxy component (also known as Docker Proxy), configured to intercept container operation requests (such as Docker API requests) from business containers, perform security processing and resource access control on the container operation requests, and then forward them to the daemon process of the container management server; correspondingly, the sidecar container is configured to receive and execute the container operation requests forwarded by the security proxy component.

[0095] In this embodiment, the security proxy component is an independent proxy layer that works in conjunction with the sidecar container to implement security policies. The Docker Proxy is deployed on the communication path between the business container (the initiator of container operation requests) and the sidecar container (the executor of container operation requests). Its core function is to act as a policy checkpoint and cleaning station for all container operation requests, ensuring that only operation instructions that comply with the security policy can be sent to the execution layer (i.e., the container management server).

[0096] Furthermore, the security proxy component is configured to: parse parameters in container creation requests or container run requests; when insecure run parameters are detected, perform security processing, including at least one of the following operations: remove privileged run parameters; review and rewrite resource access requests; refuse to execute requests containing privileged run parameters; when access parameters of hardware accelerators (such as NVIDIA GPUs, Huawei Ascend NPUs, Metax, etc.) are detected, perform resource access control, including the following steps: determine the authorized hardware accelerator resource information of the business container; rewrite the access parameters of the hardware accelerator based on the authorized hardware accelerator resource information; and forward the rewritten request to the daemon process of the container management server.

[0097] Specifically, the security proxy component intercepts all container operation requests (such as `docker run`, `docker create`, etc.) sent from the business container to the sidecar container daemon process via the container management tool in real time. Then, it performs syntax and semantic parsing on the requests, deconstructing them into request elements such as operation type (e.g., creating a container) and configuration parameters (e.g., permissions, mounted volumes, device mappings). Next, it compares and evaluates the parsed request elements against predefined security policies and resource scope control policies, including: (1) Permission restriction check: Forcefully remove or refuse parameters that grant excessive privileges to the inner container (i.e. privileged running parameters), and do not allow privileged mode to be enabled for the business container. For example, detect and filter the --privileged flag (prohibit "--privileged=true").

[0098] (2) File system isolation check: Filter or restrict mount requests that may break the isolation, such as preventing sensitive directories on the host machine from being mounted into the business container, such as filtering parameters such as "--device / dev / *".

[0099] (3) Network security policy check: restrict business containers from using dangerous network modes, such as restricting hostPath mounting and not allowing users to mount hostPath paths.

[0100] In AI model training and inference scenarios, issues related to GPU environment adaptation, isolation, and compatibility are frequently encountered. First, regarding compatibility: for a container to use the GPU, the host machine must mount device files (such as / dev / nvidia0, / dev / nvidiactl, / dev / nvidia-uvm, etc.) into the container. In a standard single-level, non-nested container, the container can access the host machine's GPU resources by mounting them using `docker run --gpus all` or `--device`. However, in the context of container nesting technology, access to GPU resources by inner containers requires a continuous and complete access chain that runs from the host physical device, through the outer container, to the innermost nested container. Currently, when Docker Daemon is running in the outer container of DinD, even if the outer container has GPU device files, Docker Daemon does not automatically pass these device files to the nested container it starts by default. This often causes inner containers to fail to correctly identify or access GPU resources. Especially under Kubernetes' device plugin mechanism, this can easily lead to GPU devices becoming invisible or conflicting, which in turn prevents AI applications running in inner containers from using GPU resources properly. Secondly, there are adaptation and isolation issues: In scenarios using heterogeneous GPU devices such as NVIDIA or Metax, the traditional DinD solution cannot correctly identify or use GPU resources. Especially under Kubernetes' device plugin mechanism, Kubernetes allocates specific GPU devices (such as GPU 0, GPU 2) to Pods through device plugins (such as GPU Device Plugins). However, the container management layer (such as Docker Daemon) lacks the ability to perceive cluster scheduling decisions. When a business container initiates a GPU access request in a nested environment, the container management server cannot automatically map broad requests (such as accessing all GPUs) and restrict them to the specific device that the business container is actually authorized to. This poses resource abuse and security risks and can easily lead to GPU devices becoming invisible or conflicting.

[0101] In addition, as customers of the container cloud platform, business containers (main containers) typically only rent a portion of the GPU resources, rather than renting all of them, in order to save costs. Traditional DinD solutions directly allocate all GPU resources, causing cluster-level resource scheduling and isolation mechanisms to fail. Therefore, to address the issue of correctly exposing and using GPU resources in nested container environments, this embodiment configures a resource access control process in the security proxy component. Specifically, it reviews and rewrites resource access requests. For example, when a container creation request contains "--gpus all", Docker Proxy queries the hardware acceleration device allocation status of the current business container (main container) through API interfaces provided by the K8s system (such as the kubelet pod resource api), determines the list of GPU devices actually authorized for the business container, and then compares whether the hardware acceleration devices in the current request are within the visible range. If not, it dynamically modifies the Docker request parameters and rewrites them to "--gpus device=X,Y" (where X and Y are the authorized GPU device IDs). The rewritten request is then forwarded to the daemon process (Docker Server) of the sidecar container, thereby preventing users from accessing unauthorized GPU resources and achieving precise secondary allocation and isolation of GPU resources.

[0102] In summary, compared with existing Docker outside Docker (DoD) and traditional Docker in Docker (DinD) solutions, the solution provided in this application, by introducing a Sidecar container mechanism based on Webhook automatic injection on a container cloud platform (such as Kubernetes), achieves automation, security, and scalability of nested container operation, and has the following significant advantages: 1. Achieve secure nested execution of user containers (i.e., business containers) in non-privileged mode. This application achieves the ability to securely run Docker without enabling privileged mode in the main container by running the container management server (such as the Docker Daemon) in a sidecar container (privileged mode), while the business container communicates with it only with normal permissions via Socket. This design significantly reduces the security risks of containers to the host machine while ensuring full functionality, and meets the security isolation requirements of multi-tenant cloud environments.

[0103] 2. Supports automated injection and on-demand activation: Leveraging the Webhook mechanism of the Kubernetes container cloud platform, the configuration of auxiliary components such as Sidecar, InitContainer, and shared storage volumes is automatically injected based on annotations or tags of the application container group (Pod), eliminating the need for users to manually modify the YAML files or images of the application container group (Pod). In practice, cluster administrators can package the above content into HelmChart for unified installation and deployment, enabling the container nesting capability throughout the entire cluster. This greatly simplifies the deployment and usage process and improves the system's maintainability and scalability.

[0104] 3. Provides a complete GPU adaptation and isolation mechanism: This application is compatible with a variety of hardware accelerator architectures, such as Nvidia GPU and Metax GPU. Thanks to the architecture design, it can adapt to all GPU devices that register resources through Device-plugin. Simply adapting the Docker proxy can achieve isolation and fine-grained allocation management of GPU resources.

[0105] 4. Enhanced System Usability and Scalability: The InitContainer mechanism automatically injects container management tools (such as the Docker CLI) into the main container. Users can directly execute container management commands (such as Docker commands, issued as container operation requests) in any business container environment, maintaining a consistent user experience with the host machine. Simultaneously, the injection rules for the Webhook component and the Sidecar image type can be flexibly customized through configuration files, supporting extensions for different GPU types, namespaces, and business scenarios, thereby improving the system's usability and scalability.

[0106] In summary, this application achieves secure, easy-to-use, and highly compatible Docker-in-Docker capabilities on the Kubernetes platform by introducing Webhook automatic injection, Sidecar container isolation, non-privileged mode operation, and GPU security adaptation mechanisms, significantly improving the scalability and security of cloud-native container environments.

[0107] Based on the same inventive concept, this embodiment provides a method for secure container nesting operation based on cloud-native technology. This method is executed by the cloud-native secure container nesting operation system provided in any of the above embodiments, and includes: The dynamically injected component intercepts the application container group deployment request and injects the configuration information of the auxiliary component into the application container group; the configuration information of the auxiliary component includes: initialization container definition, sidecar container definition, and shared storage volume definition; Before the business container starts, initialize the container and deploy the container management tools to the shared storage volume; The business container mounts a shared storage volume and sends container operation requests through the container management tool; The sidecar container runs a daemon process for the container management server, which receives and executes container operation requests.

[0108] The container nesting operation method based on cloud-native security isolation provided in this embodiment can achieve the same functions and technical effects as the container nesting operation system based on cloud-native security isolation provided in any of the above embodiments, and will not be described in detail here.

[0109] The embodiments of this application can be applied to Figure 3 The computer devices (electronic devices) shown may be, but are not limited to, mobile terminals such as mobile phones, tablets, handheld computers, and personal digital assistants (PDAs), smart home devices such as smart TVs and smart cameras, wearable devices such as smart bracelets, smartwatches, and smart glasses, or other desktop, laptop, notebook, ultra-mobile personal computer (UMPC), netbook, and smart screen computer devices.

[0110] like Figure 3 As shown, the electronic device 200 may include one or more of the following components: a processor 201, a memory 203, a communication interface 202, and a communication bus 204. The memory 203 can be connected to the processor 201 via the bus 204. The bus can transfer data between the processor 201 and the memory 203. The bus can be divided into an address bus, a data bus, a control bus, etc.

[0111] Processor 201 may include one or more processing cores. Processor 201 can connect to various parts within the electronic device 200 using various interfaces and lines. It performs various functions and processes data by running or executing instructions, programs, code sets, or instruction sets stored in memory 203, and by calling data stored in memory 203. For example, processor 201 may include an application processor (AP), a modem processor, a CPU, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a field-programmable gate array (FPGA), a programmable logic array (PLA), and / or a neural network processing unit (NPU). The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the content to be displayed; the NPU implements artificial intelligence (AI) functions; and the modem handles wireless communication. Different processing units can be independent devices or integrated into one or more processors. For example, the multiple processing units shown above are all integrated into a single SoC, or the AP is a separate semiconductor chip, while other processing units are integrated into a single SoC. This application does not limit this to any particular type.

[0112] Memory 203 may include random access memory (RAM), read-only memory (ROM), or non-transitory computer-readable storage medium. Memory 203 can be used to store instructions, programs, code, code sets, or instruction sets. Memory 203 may include a program storage area and a data storage area. The program storage area may store instructions for implementing an operating system, instructions for at least one function, such as a cloud-native method for secure container nesting; the data storage area may store data created based on the use of electronic device 200, such as a Pod's YAML file.

[0113] In addition, those skilled in the art will understand that the structure of the electronic device 200 shown in the above figures does not constitute a limitation on the electronic device 200. The electronic device may include more or fewer components than shown, or combine certain components, or have different component arrangements. For example, the electronic device 200 may also include components such as a microphone, speaker, radio frequency circuit, sensor, audio circuit, power supply, and Bluetooth module, which will not be described in detail here.

[0114] This application also provides a computer-readable storage medium storing a computer program / instructions thereon, which, when executed by a processor, implements the steps of a cloud-native method for securely isolating nested container execution.

[0115] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A container nesting system based on cloud-native technology for secure isolation, characterized in that, include: Dynamically inject components, configured to intercept requests for application container group deployment, and inject configuration information of auxiliary components into the application container group; The configuration information of the auxiliary components includes: the definition of the initialization container, the definition of the sidecar container, and the definition of the shared storage volume; Initialize the container and configure it to deploy the container management tool to the shared storage volume before the business container starts; The business container is configured to mount the shared storage volume and send container operation requests through the container management tool. The sidecar container is configured as a daemon process running the container management server, receiving and executing the container operation requests.

2. The system according to claim 1, characterized in that, The sidecar container is further configured to: create the server endpoint of the daemon process of the container management server in the shared storage volume; The service container is configured to use the container management tool to access the service endpoint in the shared storage volume and send a container operation request to the daemon process of the container management server.

3. The system according to claim 1, characterized in that, The auxiliary components also include a security proxy component. The security proxy component is configured to intercept container operation requests from the business container, perform security processing and resource access control on the container operation requests, and then forward them to the daemon process of the container management server. The sidecar container is further configured to receive and execute container operation requests forwarded by the security proxy component.

4. The system according to claim 3, characterized in that, The security proxy component is further configured as follows: Parse the parameters in the container creation request or container run request; When runtime parameters that do not comply with security checks are detected, security processing is performed, including at least one of the following operations: removing the privileged runtime parameters; rewriting the resource access request; or rejecting the execution of a request containing the privileged runtime parameters. When access parameters of a hardware accelerator are detected, resource access control is performed, including the following steps: determining the authorized hardware accelerator resource information of the business container; rewriting the access parameters of the hardware accelerator based on the authorized hardware accelerator resource information; and forwarding the rewritten request to the daemon process of the container management server.

5. The system according to claim 1, characterized in that, The dynamic injection component is further configured as follows: Based on the access control mechanism of the container cloud platform, intercept requests for application container group deployment and parse the configuration information in the requests; Determine whether auxiliary component injection is needed based on the analysis results; In response to the need to inject auxiliary components, the configuration information of the auxiliary components is dynamically added to the configuration information of the application container group.

6. The system according to claim 1, characterized in that, The container operation requests include container creation requests, container run requests, and container orchestration instructions.

7. The system according to claim 1, characterized in that, The shared storage volume is a temporary storage volume of type emptyDir, which has the same lifecycle as the application container group.

8. A method for secure isolation of nested containers based on cloud-native architecture, characterized in that, The method is performed by the system described in any one of claims 1 to 7, and includes: The dynamic injection component intercepts the application container group deployment request and injects the configuration information of the auxiliary component into the application container group; the configuration information of the auxiliary component includes: initialization container definition, sidecar container definition, and shared storage volume definition; Before the business container starts, initialize the container and deploy the container management tools to the shared storage volume; The business container mounts the shared storage volume and sends a container operation request through the container management tool; The sidecar container runs a daemon process for the container management server, which receives and executes the container operation requests.

9. A computer device, comprising a memory, a processor, and a computer program stored in the memory, characterized in that, The processor executes the computer program to implement the steps of the method of claim 8.

10. A computer-readable storage medium having a computer program / instructions stored thereon, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the method of claim 8.

Images