Method and system for packaging development machine in K8S cluster

Abstract

The invention relates to a development machine packaging method and system in a K8S cluster, which is characterized in that development machine resources are defined based on CRD and a controller, configuration information is described, and automatic life cycle management is realized in combination with operators; cRD is deployed in a host cluster, and is synchronized to K8S space of each tenant through an exclusive synchronizer; an SSH access path is opened by means of node port type service resources and an exclusive layer 4 forwarding entry; a public network IP path is automatically allocated by using the load balancer type service resources in combination with a metal load balancer component; remote access and safety control of the Zhu service are realized through a gateway, a signature verification mechanism and a domain name system analysis function of list monitoring; and deploying an exclusive daemon process set at each host node.

Description

Technical Field

[0001] This application relates to the fields of cloud-native computing and container technology, and in particular to a method and system for encapsulating development machines in a K8S cluster. Background Technology

[0002] In AI computing power application scenarios, managing GPU computing resources based on Kubernetes clusters has become a mainstream solution. For multiple tenants, each tenant is provided with an independent Kubernetes control plane entry point. Each tenant can perform cluster resource operations based on the native kubectl commands and flexibly install native Kubernetes extension components.

[0003] In practical AI development, it's typically necessary to launch a development machine based on a Kubernetes (K8S) container. This means using a K8S container group as the main runtime environment for the development machine, where environment preparation is performed and the image saved can be directly used for subsequent task submissions. Because K8S container groups are stateless, automatic persistence upon shutdown is difficult to achieve, necessitating various improvements at higher layers. Existing technical solutions offer two main approaches: The first simple solution is to deploy a development machine. In this case, in order to persist the system disk, the contents of the image are usually copied to persistent storage during the boot phase. This will cause the overlays of the image itself to be discarded. When saving the image of the development machine, all previous overlays need to be discarded, and users cannot clearly distinguish between the deployment of the development machine and the service deployment of the business scenario.

[0004] Another approach is to provide a SaaS solution, which requires additional operations in the system service backend, providing a web-based console interface. This solution typically means abandoning the native Kubernetes (K8S) entry point and implementing all operations through the web console. Replacing all native K8S operations with a web console entry point incurs significant development costs and often degrades the user experience.

[0005] Furthermore, some AI application scenarios require running Docker again on the development machine, or creating images based on dockerbuild. In a regular Kubernetes container group, privileged mode must be enabled to run the container engine. However, in a public cloud multi-tenant scenario, enabling privileged mode can pose security risks. Summary of the Invention

[0006] To address this issue, this invention proposes a method for encapsulating a development machine based on custom resource definitions. This method maintains the native Kubernetes (K8S) operating experience, allowing users to operate the cluster using native K8S command-line tools. It also provides persistent attributes for the development machine, including: automatic image saving upon shutdown, support for configuration modification, support for SSH connections, and support for saving as an image, among others. These include: The development machine resources are defined based on CRD and controller, the development machine configuration information is described, and automated lifecycle management is achieved by combining operators; Deploy the CRD on the host cluster and synchronize it to the K8S space of each tenant through a dedicated synchronizer; SSH access is enabled through service resources of node port type and dedicated Layer 4 forwarding entry; By combining load balancer type service resources with metal load balancer components, public IP access can be automatically allocated and obtained. Remote access and security control of the Jupiter service are achieved through a gateway, signature verification mechanism, and domain name system resolution functionality that includes list listening; and Deploy a dedicated set of daemons on each host node.

[0007] In one possible implementation, when the node port and dedicated Layer 4 forwarding entry are enabled, signature pre-verification is performed, allowing only services derived from the CRD to establish network connections.

[0008] In one possible implementation, when a dedicated set of daemons is deployed on each host node, the daemon set is used to store images when the development machine POD is shut down.

[0009] In one possible implementation, the image stored when the development system is shut down will have its saved image tag added to the CR's description YAML for use on the next boot.

[0010] In one possible implementation, when the development machine saves the image, the current persistent image is re-tagged in a critical state, and the persistent image of each tenant's development machine and the saved custom image are isolated through different harbor projects and usernames.

[0011] In one possible implementation, the development machine configuration information includes the image, network, storage resource specifications, and whether Docker is enabled and the startup command.

[0012] In one possible implementation, automated lifecycle management includes the creation, updating, and deletion of development machines.

[0013] The present invention also includes a system for packaging on a development machine in a K8S cluster, including a main development container, DinDPOD, DinD management device, image storage and isolation module, and network communication components; The main development container runs in the user namespace and connects to DinDPOD via a shared volume. The main development container includes SSH service and Jupiter service development tools. DinDPOD runs in the system's main namespace, communicates with the main development container through a Docker agent, supports GPU resource mounting, and shares storage paths with the main development container. The DinD management device analyzes the status of the main development container in real time. The DinD management device injects the node affinity and GPU type information of the main development container into the DinDPOD. The DinD management device also works with the firewall and the domain name system service provider. The image storage and isolation module builds a multi-tenant image repository based on Harbor. The image storage and isolation module triggers the image saving logic when the main development container is shut down through a pre-stop hook. Network communication components include SSH channels, Jupiter channels, and public IP channels.

[0014] In one possible implementation, the main development container defines a startup command through a command field to perform initialization configuration.

[0015] In one possible implementation, the system also includes a persistent storage module that supports persistent volume mounting and injects socket files via local shared storage when Docker is enabled.

[0016] The beneficial effects of this invention are: Compared to existing technologies, this invention enables the development machine to achieve black-and-white screen alignment, ensuring that a white screen creation is visible on a black screen, and vice versa. The development machine automatically persists data to the system disk, preventing data loss upon shutdown. The persistence process must be low-overhead, avoiding the copying of files from the entire image to optimize startup speed. The development machine can be saved as a Docker image, and a development machine created on a black screen can also save an image; therefore, caching of the base image is necessary. Users can start the development machine / training task on a white screen or a black screen based on their saved Docker image. The development machine supports three connection methods: independent public network, IP / SSH, and P / Juptyer connections, all requiring black-and-white screen alignment. The development machine supports image creation and execution via DIND without requiring privileged mode, avoiding security risks.

[0017] Other features and aspects of this application will become clear from the following detailed description of exemplary embodiments with reference to the accompanying drawings. Attached Figure Description

[0018] The accompanying drawings, which are included in and form part of this specification, illustrate exemplary embodiments, features, and aspects of this application together with the specification and serve to explain the principles of this application.

[0019] Figure 1 An architecture diagram of a Kubernetes-based development machine according to an embodiment of this application is shown; Figure 2 A flowchart illustrating a method for packaging a development machine according to an embodiment of this application is shown; Figure 3 This application illustrates a public network access path architecture diagram according to an embodiment of the present application. Figure 4 The Dind solution architecture diagram of an embodiment of this application is shown; Figure 5 A flowchart of the DinD solution according to an embodiment of this application is shown. Detailed Implementation Various exemplary embodiments, features, and aspects of this application will now be described in detail with reference to the accompanying drawings. The same reference numerals in the drawings denote elements that have the same or similar functions. Although various aspects of the embodiments are shown in the drawings, they are not necessarily drawn to scale unless specifically indicated otherwise.

[0020] It should be understood that the terms "center," "longitudinal," "lateral," "length," "width," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," "outer," "clockwise," "counterclockwise," "axial," "radial," and "circumferential" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing the present invention or simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the present invention.

[0021] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0022] The term “exemplary” as used herein means “serving as an example, embodiment, or illustration.” Any embodiment illustrated herein as “exemplary” is not necessarily to be construed as superior to or better than other embodiments.

[0023] Furthermore, to better illustrate this application, numerous specific details are provided in the following detailed embodiments. Those skilled in the art should understand that this application can be implemented without certain specific details. In some instances, methods, means, components, and circuits well-known to those skilled in the art have not been described in detail in order to highlight the main points of this application.

[0024] Encapsulation method This invention proposes a method for encapsulating a development machine based on custom resource definitions, applicable to the field of cloud-native computing and container technology / equipment. It maintains the native K8S operating experience, allowing users to operate the cluster based on native K8S command-line tools, while also possessing the persistent attributes of a development machine, including: automatic image saving upon shutdown, support for configuration modification, support for SSH connection, and support for saving as an image, etc.

[0025] specifically refer to Figure 1 , Figure 1 This diagram illustrates an architecture of a Kubernetes (K8S)-based development machine according to an embodiment of this application. As shown, this architecture diagram demonstrates a K8S-based containerized application deployment and management architecture. Tenant K8S Space 1 and Tenant K8S Space 2 are independent K8S namespaces used to isolate applications and resources from different tenants. Controllers eb-controller-1 and eb-controller-2 are controllers corresponding to each tenant space, responsible for managing application deployment and resource synchronization under that tenant. Synccrd represents the controller synchronizing with custom resource definitions to ensure consistent resource configurations. watchpod represents the controller monitoring Pod state changes to ensure Pods run as expected. createcr represents the controller creating custom resources. syncpod represents the controller synchronizing Pod state and configuration. CRD (Container Server) is a custom resource definition used to extend the K8S API, defining the configuration and behavior of the container server. Nodes node1, node2, and node3 are worker nodes in the K8S cluster, running containerized applications. A daemon set (daemonSet) runs a daemon on each node to ensure that certain necessary services run on each node. The image repository Harbor is used to store and manage container images, providing pull and push image functionality. The pullimage node pulls the required container images from Harbor. The watchpodandpushimage set, representing a daemon set, monitors the status of pods and pushes images to Harbor when needed. The storage system provides persistent storage support for the entire architecture, ensuring data reliability and durability.

[0026] exist Figure 1 In the architecture diagram shown, each tenant space manages its applications and resources through a corresponding controller. The controller synchronizes resource configurations with the CRD and monitors Pod status changes. Nodes pull images from the image repository Harbor, and daemon sets monitor Pods and push images to Harbor. The entire architecture is supported by persistent storage provided by the storage system.

[0027] This architecture diagram illustrates a complete containerized application management process, from resource isolation in the tenant space to resource management of the controller, application execution and image management on the nodes, and finally to storage system support, ensuring the reliability and scalability of the application.

[0028] specifically refer to Figure 2 , Figure 2 A flowchart illustrating a method for packaging a development machine according to an embodiment of this application is shown. Based on the above architecture, the method of the present invention includes: Step 201: Define development machine resources based on CRD and controllers, describe development machine configuration information, and combine operators to achieve automated lifecycle management. By customizing the development machine resource model through CRDs, configuration information is standardized; by combining controllers and operators, lifecycle operations such as creation, update, and deletion of development machines are automatically handled, preserving the native Kubernetes operating experience while solving the low management efficiency problem caused by the statelessness of traditional containers.

[0029] In one specific embodiment, the CRD field is defined to include configurations such as image version, GPU model, and storage capacity when the user creates a development machine. Upon detecting the creation of the CRD, the controller automatically schedules nodes, mounts resources, and starts the container without manual intervention. When the user modifies the "storage capacity" field in the CRD, the controller automatically expands the storage volume and restarts the development machine, ensuring the configuration takes effect in real time.

[0030] In one specific embodiment, for AI development scenarios, fields such as "Whether to enable Docker" and "Startup command" are added to the CRD. When the development machine starts, the operator automatically installs the CUDA toolkit based on the "Startup command"; when the user deletes the CRD, the controller first saves the current image and then cleans up resources to avoid data loss. Step 202 involves deploying the CRD on the host cluster and synchronizing it to the Kubernetes spaces of each tenant using a dedicated synchronizer. This ensures unified CRD configuration between the host cluster and tenant spaces, guaranteeing consistent resource models for development machines in a multi-tenant environment. The synchronizer synchronizes configurations in real time, avoiding manual CRD maintenance by tenants and reducing the complexity of cross-space management.

[0031] In one specific embodiment, after the host cluster deploys the basic CRD, the synchronizer pushes CRD copies to the Kubernetes spaces of tenant 1 and tenant 2 through the tenant API gateway. When the host cluster CRD is upgraded, the synchronizer automatically triggers the tenant space CRD update to ensure that all tenants use the same resource definition.

[0032] Step 203: Open an SSH access path through the service resources of the node port type and a dedicated Layer 4 forwarding entry. This utilizes the K8S node port to achieve standardized exposure of SSH access, combined with the signature verification mechanism of the Layer 4 forwarding entry, ensuring the flexibility of remote access from the development machine while preventing unauthorized connections from non-CRD derived services, thus improving network security.

[0033] In one specific embodiment, when the development machine starts up, the controller automatically creates a node port service, and a Layer 4 forwarding entry listens on this port. When a user initiates an SSH connection, they must carry a temporary signature generated by the CRD. Access is only allowed after the forwarding entry verifies the signature; connections without the signature are directly blocked.

[0034] Step 204: The public IP address path is automatically allocated and obtained by combining the service resources of the load balancer type with the metal load balancer component. By leveraging the linkage between the load balancer service and the metal load balancer, the automatic allocation and binding of public IP addresses is achieved, resolving network reachability issues during remote debugging of the development machine; it also supports dynamic IP reclamation, improving the utilization rate of public network resources.

[0035] In one specific embodiment, the user configures "Public network access required" in CR. The controller creates a LoadBalance type service. After detecting the service, the metal load balancer allocates one IP from the public IP pool and binds it to the node. When the development machine is shut down, the load balancer automatically unbinds the IP and returns it to the pool for reuse by other development machines.

[0036] In one specific embodiment, the metal load balancer integrates traffic monitoring functionality. When abnormal public network traffic is detected on a development machine, it automatically sends an alarm to the controller. The controller then suspends the public network access of the development machine and notifies the user to prevent bandwidth abuse.

[0037] Step 205: Remote access and security control of the Jupiter service are achieved through a gateway, signature verification mechanism, and DNS list monitoring. The gateway provides a unified entry point for managing access to the Jupiter service, signature verification prevents unauthorized requests, and the DNS list monitoring dynamically updates DNS records. The combination of these three features satisfies both the need for convenient access by remote development tools and ensures service security through multi-layered protection.

[0038] In one specific embodiment, when the Jupiter service starts, the controller generates a signing key containing an expiration date and registers it in a Domain Name System (DNS) record. When a user accesses the service, they must include the signature in the Uniform Resource Locator (URL). The gateway verifies the validity of the signature and the legitimacy of the domain name in the DNS list before forwarding the request to the service. Expired signatures or invalid domain names are directly rejected.

[0039] Step 206: Deploy a dedicated set of daemons on each host node. The daemon set runs resident on all nodes, enabling localized real-time management of development machine Pods. In particular, it solves problems such as image storage and resource cleanup during shutdown, improving fault tolerance in the event of a single machine failure.

[0040] In one specific embodiment, the daemon process on the node monitors the status of the development machine's Pods. When it detects that a Pod has triggered a "shutdown" command, it immediately calls the container snapshot interface to save the current image to the local cache and synchronize it to the Harbor repository. Even if the node temporarily goes offline, the daemon process can still continue to upload images after restarting.

[0041] In one possible implementation, when access is granted via a node port and a dedicated Layer 4 forwarding entry point, a signature pre-verification is performed, allowing only services derived from the CRD to establish network connections. This signature pre-verification mechanism strictly restricts network connections to legitimate services derived from the CRD, blocking unauthorized service access requests at the network entry layer. This effectively prevents malicious programs or illegal services from consuming node port resources, while ensuring the security and compliance of access pathways such as SSH, and strengthening network isolation capabilities in a multi-tenant environment.

[0042] In one possible implementation, when a dedicated set of daemons is deployed on each host node, the daemons are used to store images when the development machine POD is shut down. Leveraging the localized operation of the node-level daemons, the image storage logic is triggered in real time when the development machine POD is shut down, avoiding the remote scheduling delays of the cluster-level controller and ensuring the timeliness and reliability of image saving operations. This also mitigates the risk of image loss when nodes are offline, improving the stability of data persistence on the development machine.

[0043] In one possible implementation, the image stored when the development machine is shut down will have its saved image tag added to the description YAML of the CR (Create View) for use on the next boot. By associating the image tag with the CR's YAML file, a strong binding between the development machine's state and the image version is achieved. On the next boot, the corresponding image can be directly pulled based on the tag, eliminating the need for the user to manually specify the version and simplifying the operation process. At the same time, CR, as a single data source for configuration, ensures the accuracy of image version tracking and avoids version confusion.

[0044] In one possible implementation, when the development machine saves an image, the current persistent image is re-marked under critical conditions. Each tenant's persistent development machine image and saved custom images are isolated through different Harbor projects and usernames. This image re-marking mechanism under critical conditions enables fine-grained management of image versions. Combined with Harbor's multi-project and user isolation strategy, image resources from different tenants are completely isolated at the storage layer, preventing data leakage or accidental operations and meeting the access control and data security requirements in multi-tenant scenarios.

[0045] In one possible implementation, the development machine configuration information includes image, network, storage resource specifications, and whether Docker is enabled and the startup command. By standardizing the core configuration information of the development machine through CRD (Customer Relationship Management), resource requirements can be quantified and configurable, facilitating the controller's automatic parsing and scheduling of corresponding resources. Simultaneously, it supports customization of Docker on / off and startup commands to meet the environmental requirements of different development scenarios, improving the flexibility and adaptability of the development machine.

[0046] In one possible implementation, automated lifecycle management includes the creation, updating, and deletion of development machines. By automating the entire lifecycle of development machines through controllers and operators, manual intervention costs are reduced (e.g., automatic resource scheduling during creation, dynamic configuration adjustment during updates, and cleanup of residual data during deletion), while ensuring consistency and standardization of operations and avoiding human error. Furthermore, automated responses at each stage of the lifecycle improve the operational efficiency of development machines and the utilization rate of cluster resources.

[0047] Packaging System The present invention also includes a system for packaging a development machine in a K8S cluster, including a main development container, DinDPOD, DinD management device, image storage and isolation module, and network communication components; The main development container runs in the user namespace and connects to DinDPOD via a shared volume. The main development container includes SSH service and Jupiter service development tools. DinDPOD runs in the system's main namespace, communicates with the main development container through a Docker agent, supports GPU resource mounting, and shares storage paths with the main development container. The DinD management device analyzes the status of the main development container in real time. The DinD management device injects the node affinity and GPU type information of the main development container into the DinDPOD. The DinD management device also works with the firewall and the domain name system service provider. The image storage and isolation module builds a multi-tenant image repository based on Harbor. The image storage and isolation module triggers the image saving logic when the main development container is shut down through a pre-stop hook. Network communication components include SSH channels, Jupiter channels, and public IP channels.

[0048] In one possible implementation, the main development container defines a startup command through a command field to perform initialization configuration.

[0049] In one possible implementation, the system also includes a persistent storage module that supports persistent volume mounting and injects socket files via local shared storage when Docker is enabled.

[0050] specifically refer to Figure 3 , Figure 3 A public network access path architecture diagram of an embodiment of this application is shown.

[0051] The development machine encapsulated in this application provides three public network access paths. All three access paths use a service as the base for service forwarding within the host cluster. These include the SSHAccess path: `ssh-service` is a node port-type service. In the host cluster, this service undergoes signature verification. If the signature is valid, the firewall's network address translation service is invoked to map the node port to the public network, forming an access path. The JupyterAccess path consists of a service and an entry point. The `jupyter-service` is a cluster IP type, primarily used for serverPod port mapping. The `jupyter-entry point` is an access domain name; it also undergoes signature verification, and only valid entries can be correctly resolved to the public IP provided by the firewall. Access token authentication is performed on this domain name within the entry point to prevent unauthorized access and privilege escalation. The PublicAccess path contains only one `public-service`, a load balancer. The host cluster's container network interface plugin requests a public IP for routing from the firewall via the border gateway protocol. Users can open specific ports by modifying the `port` of the `public-service`.

[0052] The command syntax for the development machine CRD schema in this application is designed as follows: YAML apiVersion: apps.example.com / v1alpha1 kind: ContainerServer metadata: annotations: eks.example.com / description: "" # Remarks / description information for the development machine name: demo1 # The name of the development machine namespace: default # The namespace of the user space to which the development machine belongs spec: command: # Startup command for the development machine - sh - -c - |- if [[ -z "${EBSYS_INITIALIZED}" || "${EBSYS_INITIALIZED}" == "False"]]; then dpkg -i / public / shared-resources / openssh-server / ubuntu_20.04_amd64 / .deb&&\ echo "root:$INIT_ROOT_PASSWORD" | chpasswd&&\ echo 'Initialization paasword complete' \ fi&&\ / usr / sbin / sshd -D enableDocker: false # Whether to enable dind image: registry-cn-beijing2-internal.example-inc.com / tenant-37666535 / demo195 / demo11:v1.1 # Base image for the development machine initRootPassword: MyPaSsW0rd@123 # Initial password for the development machine network: # Network configuration public: true # Enable public IP address tcp: ports: # Ports exposed externally in the network configuration - 80 - 443 power: "ON" # Power on / off configuration resources: # Resource specifications CPU: count: "20" GPU: count: "1" type: H800_NVLINK_80GB # GPU type used to generate affinity policies memory: count: 200Gi sshAccess: # SSH access configuration enable: true targetPort: 22 # SSH access port jupyterAccess: # A MLOPs development kit enable: true port: 8888 volumeMounts: # Storage mount configuration information - mountPath: / data #mount path name: t256g persistentVolumeClaim: claimName: t256g # Name of the persistent volume status: # Status description information of the development machine initialized: true # Whether initialization was successful latestTag: "1748434283" # The latest mirror version tag Phase: Running # Development machine status podStates: serverPod: clusterIP: 10.233.97.148 name: cs-9ef36-39e86-server phase: Running dindPod: clusterIP: 10.233.97.149 name: cs-9ef36-39e86-dind phase: Running sshAccess: domain: ssh-cn-beijing2.example-inc.com format: 'ssh -p 37661 root@ssh-cn-beijing2.example-inc.com' port: 37661 jupyter: url: https: / / example-xxx-jupyter-cn-beijing2.example-inc.com The command is the startup command for the development machine. This parameter allows for the configuration of the development machine's boot process. Resources are the development machine's specification options; users can select appropriate quotas and GPU models through defined fields. `enableDocker` indicates whether Docker is enabled on the development machine. When enabled, an auxiliary DindPod will be created and the relevant socket file will be injected. Volume mounting configures storage mounting, such as mounting the persistent volume t256g to the / data directory on the development machine. `network` and `sshAccess` configure the development machine's network. `network` allows quick configuration of a public port or public IP address, while `sshAccess` allows quick configuration of a public SSH port. `jupyterAccess` is an optional built-in plugin. When enabled, it automatically injects a specified valid version of the Jupyter IDE into the development machine and enables remote entry domain access.

[0053] In this application, the development machine network setup involves creating the development machine, and then the CR controller, based on the definitions in the schama, calling the apiserver to create specific service / entry resources.

[0054] On the control plane's wenhook side, the created service / entry point will be validated for legality, such as whether it is a subordinate resource of the development machine, whether the expectation is consistent with the boot schama definition, whether the signature is valid, etc. If the validation passes, it will be redirected to listwatch. On the ListWatch client, when a corresponding service is detected, signature verification is performed on the service's annotations and creation timestamp. If the service passes the verification, the firewall interface is invoked to open the network address translation port and provide public network access service.

[0055] If Jupyter is enabled, it will additionally call the Domain Name System (DNS) service provider to configure DNS resolution.

[0056] According to the definition of a development machine, the storage settings of the development machine in this application will perform persistent volume verification in the control plane, including name validity verification and existence verification. If the verification passes, a mount point will be rendered for the serverPod.

[0057] If Docker is enabled, local shared storage will be created for sock injection.

[0058] specifically refer to Figure 4 and Figure 5 , Figure 4 The Dind solution architecture diagram of an embodiment of this application is shown. Figure 5 A flowchart of the DinD solution according to an embodiment of this application is shown. This solution, through a separate deployment architecture of the main Pod and the DindPod, supports users running Docker build tasks on the development machine while avoiding the main Pod from enabling privileged mode, and simultaneously achieves secure mounting and access control of GPU resources.

[0059] Its architecture includes the main development container ServerPod, which runs the user's development environment in a non-privileged mode and includes the docker-client tool, communicating with the DindPod via a shared volume.

[0060] It also includes DindPod, which runs in the system namespace, enables privileged mode to support the Docker engine, and includes the docker-proxy component, which is responsible for resource forwarding and permission verification.

[0061] It also includes shared volumes, which establish Docker communication links between the two through sock files, and mount persistent storage to save Docker images and container data.

[0062] The process includes step 501: When the user sets `enableDocker:true` in CR, the CS-operator (controller) detects the configuration change and sends a rendering request for the DindPod YAML to the Dind management device. Step 502: The Dind management device monitors the ServerPod's status in real time, obtaining its node information, GPU mount type, and resource quota to ensure that the DindPod matches the main Pod's resources. Step 503: The Dind management device injects the parsed ServerPod information into the DindPod's YAML, including: Node affinity ensures that DindPod and ServerPod are scheduled to the same node, reducing cross-node communication overhead; GPU resource mapping synchronously allocates the GPU resources mounted on the ServerPod to the DindPod, supporting GPU-accelerated tasks within containers; Permission injection injects only the minimum permissions required to run Docker into the DindPod and restricts user access to sensitive system calls.

[0063] Step 504: The docker-client in ServerPod communicates with the docker-proxy in DindPod through the sock file of the shared volume. All Docker commands must be verified by docker-proxy: high-risk parameters are prohibited to prevent unauthorized user operations; access to the host node's file system is restricted, and operations on resources within the shared volume are only allowed.

[0064] This technical solution addresses the risk of permission leaks in multi-tenant scenarios by disabling privileged mode in the main Pod, centrally managing privileged operations in the DindPod, and intercepting high-risk commands using docker-proxy. Furthermore, it ensures resource consistency between the DindPod and the main Pod through node affinity and GPU synchronization mechanisms, meeting the GPU acceleration requirements of Docker containers in AI development. This solution also allows users to operate using native Docker commands without needing to adapt to new tools, preserving the native Kubernetes operating experience.

[0065] This solution strikes a balance between security and functionality, supporting Docker tasks within the development machine while avoiding the risks associated with privileged modes through a separate architecture, making it suitable for multi-tenant AI development scenarios.

[0066] The system persistence and image storage mechanism of this invention achieves automatic persistence and secure storage of development machine images by deploying a daemon set on physical machine nodes, combined with the Harbor multi-tenant image repository and Kubernetes lifecycle hooks. Specifically, it allocates an independent Harbor image repository project to each tenant and controls access through username permissions. Images from different tenants are stored in their respective dedicated projects, and cross-tenant access is restricted through Harbor's role-based access control system to prevent image data leakage or accidental operation. Each image is automatically tagged with a tenant ID, development machine ID, and timestamp for easy traceability of image source and version.

[0067] The core process of the shutdown signal triggering mechanism (preStopHook and superimposed write) is as follows: when the ServerPod (main development container) shuts down, the shutdown signal is written to the superimposed file system through the preStopHook, triggering the image saving process.

[0068] The core logic of image saving and pushing (daemonSet and finalizer collaboration) is as follows: The container server daemonSet intercepts the ServerPod termination process through the finalizer mechanism, executes image commit, pushes it to Harbor, and updates the CR configuration. Specifically, it includes finalizer injection; that is, when the ServerPod starts, the daemonSet automatically adds a dedicated finalizer to it, preventing the Pod from being directly deleted until the image saving is complete.

[0069] Image commit and push: The daemonSet listens to the `shutdown-signal` file in its stack. Upon detecting a shutdown signal, it uses the `docker commit` command to commit the current state of the ServerPod container (including user development environment configuration and data) as a new image. After the image commit is complete, the daemonSet uses the tenant-specific Harbor account to push the image to that tenant's dedicated Project. After a successful push, the daemonSet writes the image tag to the YAML field of the corresponding CR on the development machine and removes the finalizer, allowing the Pod to terminate normally. The image is automatically reused on the next startup. Based on the above technical solutions, users do not need to manually save images. The entire process is automated through the pre-stop hook (preStopHook) and the daemon set (daemonSet). Furthermore, image submission is based on an overlay layer, avoiding copying the entire image file and reducing resource consumption. The finalizer mechanism ensures that Pods are not deleted before the image is saved, resolving image loss issues caused by node offline or network interruptions. CR is strongly bound to image tags, supporting "snapshot-style" recovery of development machine states, meeting the environment consistency requirements in AI development.

[0070] This mechanism achieves secure isolation, automatic persistence, and efficient reuse of development machine images in multi-tenant scenarios, balancing security and user experience.

[0071] The various embodiments of this application have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or improvement of the technology in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A method for encapsulating a development machine in a K8S cluster, characterized in that, The application comprises: Developing machine resources based on CRD plus controller definition, describing the developing machine configuration information, and realizing automatic life cycle management in combination with operators; Deploying CRD in the host cluster and synchronizing to the K8S space of each tenant through a dedicated synchronizer; Opening an SSH access path through a node port type service resource and a dedicated 4-layer forwarding entry; Automatically allocating a public network IP path through a load balancer type service resource combined with a metal load balancer component; Realizing remote access and security control of the Jupiter service through a gateway plus signature verification mechanism and a domain name system resolution function of list listening; and Deploying a dedicated daemon set on each host node.

2. The method of claim 1, wherein, When opened through a node port and a dedicated 4-layer forwarding entry, pre-signature verification is performed, and only the CRD derived service is allowed to establish a network connection.

3. The method of claim 2, wherein, When a dedicated daemon set is deployed on each host node, the daemon set is used to store images when the developing machine POD is shut down.

4. The method of claim 3, wherein, Including the images stored when the developing machine is shut down, the saved image tag is tagged in the description YAML of the CR, which is used for next time startup.

5. The method of claim 4, wherein, Including the re-labeling of the current persistent image in the key state when the developing machine saves the image, and the persistent image and the saved custom image of the developing machine of each tenant are isolated through different harbor projects and usernames.

6. The method of claim 5, wherein, The developing machine configuration information includes image, network, storage resource specification, and whether to start docker and start command.

7. The method of claim 6, wherein, The automatic life cycle management includes the creation, update, and deletion of the developing machine.

8. A system for encapsulation by a development machine in a K8S cluster, comprising the method of claims 1-7, characterized in that, Including a main development container, a DinD POD, a DinD management device, an image storage and isolation module, and a network communication component; The main development container runs in a user namespace and is connected to the DinD POD through a shared volume, and the main development container includes an SSH service and a Jupiter service development tool; The DinD POD runs in a system main namespace, communicates with the main development container through a docker agent, supports GPU resource mounting, and shares a storage path with the main development container; The DinD management device analyzes the status of the main development container in real time, injects node affinity and GPU type information of the main development container into the DinD POD, and links the firewall and the domain name system service provider; The image storage and isolation module constructs a multi-tenant image warehouse based on Harbor, and triggers the image saving logic when the main development container is shut down through a pre-stop hook; The network communication component includes an SSH path, a Jupiter path, and a public network IP path.

9. The system of claim 8, wherein, The main development container defines a start command through a command field to realize initialization configuration.

10. The system of claim 9, wherein, The system further comprises a persistent storage module, the persistent storage module supports persistent volume mounting, and the persistent storage module injects a socket file through local shared storage when Docker is enabled.

Images