Method and device for creating secure container

Abstract

The invention provides a method and a device for creating a security container, and relates to the field of cloud computing, and the method comprises the following steps: a management node creates pod information according to an obtained pod definition, and allocates a worker node for operating the security container in a kubernetes cloud native platform according to a scheduling strategy, resources corresponding to the pod information and a resource quantity. According to the technical scheme, a worker node obtains pod information which corresponds to a host and the worker node and is in a to-be-created state from a management node of a kubernetes cloud native platform through kubelet, device resources are distributed to a security container according to the pod information, the security container is created according to the device resources, and the high-performance storage capacity is provided for the security container.

Description

Technical Field

[0001] The present application relates to the field of cloud computing technology, and in particular to a method and device for creating a secure container. Background Art

[0002] With the development of cloud computing, cloud native technologies represented by containers and microservices have attracted widespread attention. However, in cloud native scenarios, containers are multi-tenant resource isolation, and ordinary containers run directly on the kernel of physical machines, which poses a security risk to physical machines. Based on this, secure containers (pods) came into being. Compared with ordinary containers, secure containers run in a lightweight virtual machine, have an independent operating system kernel and secure isolation of the virtualization layer. The virtualization-level isolation achieved by secure containers not only ensures the isolation of user resources in cloud native scenarios, but also ensures the security of the platform itself.

[0003] At present, in the cloud-native environment, the creation of secure containers is mainly based on the unified management of physical machine resources by the kubernetes (abbreviated as: K8s) platform. According to user-defined requirements, the secure containers are dynamically scheduled and provided. The computing resources used by the secure containers are provided based on hardware virtualization technology, while the storage resources use virtualization technologies such as virtio-9p, virtio-fs, virtio-blk, and virtio-scsi. kubernetes needs to first map the storage system accessed by the container on the worker node physical machine, and then the virtual machine monitor (virtual machine monitor, VMM) uses virtualization technology to virtualize the storage device to the virtual machine of the secure container.

[0004] However, the existing security container cannot achieve the performance of running directly on a physical machine, resulting in the security container being unable to obtain high-performance storage capabilities. Summary of the invention

[0005] The present application provides a method and device for creating a secure container to solve the problem that the secure container cannot obtain high-performance storage capabilities.

[0006] In a first aspect, the present application provides a method for creating a secure container, which is applied to a management node of a kubernetes cloud native platform, including:

[0007] Get a pod definition, which is used to describe the resources and resource quantities required to create a secure container;

[0008] Obtain a pod creation request through the API server, where the pod creation request is generated according to the pod definition;

[0009] According to the pod creation request, create pod information through the API server;

[0010] Allocate, through a scheduler, a worker node for running the secure container in the Kubernetes cloud native platform according to a scheduling strategy, resources corresponding to the pod information, and the number of resources;

[0011] Update the host of the pod information, and update the status of the pod information to to be created, where the host corresponds to the worker node.

[0012] In a second aspect, the present application provides a method for creating a secure container, which is applied to a worker node of a kubernetes cloud native platform, including:

[0013] Obtaining, through kubelet, pod information corresponding to the host and the worker node and in a state of being created from the management node of the kubernetes cloud native platform;

[0014] According to the pod information, call the grpc service provided by the device plugin to allocate device resources for the secure container;

[0015] The secure container is created according to the device resources.

[0016] In a possible implementation manner, creating the secure container according to the device resources includes:

[0017] Call the secure container runtime component to generate a RunPodSandbox request, where the RunPodSandbox request is used to indicate the device resources required to create the secure container and configuration information for configuring the guest kernel and image;

[0018] According to the RunPodSandbox request, the secure container runtime component is called to create a secure container.

[0019] In a possible implementation, calling the secure container runtime component according to the RunPodSandbox request to create a secure container includes:

[0020] Pull the container image of the secure container according to the RunPodSandbox request;

[0021] According to the container image and the device resources, a virtual machine running the secure container is configured;

[0022] Configure the guest kernel and image according to the configuration information requested by the RunPodSandbox;

[0023] Start the configured virtual machine;

[0024] Through the secure container runtime component, notify the Agent in the virtual machine to configure resources for the secure container, where the resource configuration includes at least one of the following: loading a kernel module, loading a device driver, configuring a network, mounting a file system, and configuring a security policy;

[0025] The component is run through the secure container, and a creation result indicating successful creation is returned.

[0026] In a possible implementation, before obtaining, from the management node of the kubernetes cloud native platform through kubelet, pod information corresponding to the host and the worker node and in a state of being to be created, the method further includes:

[0027] The type and quantity of device resources available to the worker node are registered to the Kubernetes cloud native platform through the device plugin service.

[0028] In a possible implementation, registering the type and quantity of device resources available to the worker node with the Kubernetes cloud native platform through the device plugin service includes:

[0029] Read the service configuration file to obtain the device ID to be processed;

[0030] Scan the device of the worker node to determine the type and quantity of device resources of the target device corresponding to the device identifier to be processed;

[0031] Update the driver of each target device of the worker node to vfio / vfio-pci;

[0032] Determining the type and quantity of device resources available to the worker node through the device plugin service;

[0033] Start the grpc service through the device plugin service;

[0034] Register the device plugin service to the kubelet service;

[0035] Calling the grpc service through the kubelet service to obtain the type and quantity of the available device resources;

[0036] Register the types and quantities of the available device resources to the Kubernetes cloud native platform.

[0037] In a possible implementation, the method further includes:

[0038] Calling the secure container runtime component to run the secure container;

[0039] According to the creation result, the state of the secure container is updated.

[0040] In a third aspect, the present application provides a secure container creation device, which is applied to a management node of a kubernetes cloud native platform, including:

[0041] A first acquisition module is used to acquire a pod definition, where the pod definition is used to describe the resources and resource quantities required to create a secure container;

[0042] A second acquisition module is used to obtain a pod creation request through an API server, where the pod creation request is generated according to the pod definition;

[0043] A creation module, used to create pod information through the API server according to the pod creation request;

[0044] An allocation module, configured to allocate a worker node for running the secure container in the Kubernetes cloud native platform according to a scheduling strategy, resources corresponding to the pod information, and the number of resources through a scheduler;

[0045] An update module is used to update the host of the pod information and update the status of the pod information to be created, wherein the host corresponds to the worker node.

[0046] In a fourth aspect, the present application provides a secure container creation device, which is applied to a worker node of a kubernetes cloud native platform, including:

[0047] An acquisition module, used to obtain pod information corresponding to the host and the worker node and in a state of being created from the management node of the kubernetes cloud native platform through kubelet;

[0048] A calling module is used to call the grpc service provided by the device plugin to allocate device resources for the secure container according to the pod information;

[0049] A creation module is used to create the security container according to the device resources.

[0050] In a possible implementation, the creation module is specifically used to:

[0051] Call the secure container runtime component to generate a RunPodSandbox request, where the RunPodSandbox request is used to indicate the device resources required to create the secure container and configuration information for configuring the guest kernel and image;

[0052] According to the RunPodSandbox request, the secure container runtime component is called to create a secure container.

[0053] In a possible implementation, the creation module is specifically used to:

[0054] Pull the container image of the secure container according to the RunPodSandbox request;

[0055] According to the container image and the device resources, a virtual machine running the secure container is configured;

[0056] Configure the guest kernel and image according to the configuration information requested by the RunPodSandbox;

[0057] Start the configured virtual machine;

[0058] Through the secure container runtime component, notify the Agent in the virtual machine to configure resources for the secure container, where the resource configuration includes at least one of the following: loading a kernel module, loading a device driver, configuring a network, mounting a file system, and configuring a security policy;

[0059] The component is run through the secure container, and a creation result indicating successful creation is returned.

[0060] In a possible implementation, before obtaining the pod information corresponding to the host and the worker node and in a state of being created from the management node of the Kubernetes cloud native platform through kubelet, the creation device of the secure container also includes a registration module for registering the type and quantity of device resources available to the worker node to the Kubernetes cloud native platform through the device plugin service.

[0061] In a possible implementation, the registration module is specifically used to:

[0062] Read the service configuration file to obtain the device ID to be processed;

[0063] Scan the device of the worker node to determine the type and quantity of device resources of the target device corresponding to the device identifier to be processed;

[0064] Update the driver of each target device of the worker node to vfio / vfio-pci;

[0065] Determining the type and quantity of device resources available to the worker node through the device plugin service;

[0066] Start the grpc service through the device plugin service;

[0067] Register the device plugin service to the kubelet service;

[0068] Calling the grpc service through the kubelet service to obtain the type and quantity of the available device resources;

[0069] Register the types and quantities of the available device resources to the Kubernetes cloud native platform.

[0070] In a possible implementation, the calling module is used to call the secure container runtime component to run the secure container;

[0071] The secure container creation device also includes an update module, which is used to update the state of the secure container according to the creation result.

[0072] In a fifth aspect, the present application provides a management node, comprising: a processor, a memory, and computer program instructions stored in the memory and executable on the processor, wherein the processor is used to implement the first aspect and the methods provided by various possible designs when executing the computer program instructions.

[0073] In a sixth aspect, the present application provides a worker node, comprising: a processor, a memory, and computer program instructions stored in the memory and executable on the processor, wherein the processor is used to implement the second aspect and the methods provided by various possible designs when executing the computer program instructions.

[0074] In the seventh aspect, the present application provides a Kubernetes cloud native platform, including: a management node and a worker node, the management node is used to implement the first aspect and the method provided by each possible design, and the worker node is used to implement the second aspect and the method provided by each possible design.

[0075] In an eighth aspect, the present application may provide a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, they are used to implement the methods provided by the first aspect, the second aspect, and various possible designs.

[0076] The present application provides a method and device for creating a secure container. In this method, the management node obtains the pod definition, obtains the pod creation request through the API server, creates the pod information through the API server according to the pod creation request, and allocates the worker node for running the secure container in the Kubernetes cloud native platform through the scheduler according to the scheduling strategy, the resources and resource quantity corresponding to the pod information, updates the host of the pod information, and updates the status of the pod information to be created. The worker node obtains the pod information corresponding to the host and the worker node and the status is to be created from the management node of the Kubernetes cloud native platform through kubelet, and calls the grpc service provided by the device plugin to allocate device resources for the secure container according to the pod information, and creates the secure container according to the device resources. In this technical solution, the resources and resource quantity required to create the secure container are determined through the pod definition, so that the worker node used to create the secure container is determined according to the resources and resource quantity, and the grpc service provided by the device plugin is called to allocate device resources for the secure container, which can meet the security needs in the cloud native scenario, and enable the secure container to obtain high-performance storage capabilities to meet the user scenarios of high-performance distributed applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0077] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0078] Figure 1 A schematic diagram of the structure of the Kubernetes cloud native platform provided in the embodiment of the present application;

[0079] Figure 2 A flowchart of a first embodiment of a method for creating a secure container provided in an embodiment of the present application;

[0080] Figure 3 A schematic diagram of a flow chart of a second embodiment of a method for creating a secure container provided in an embodiment of the present application;

[0081] Figure 4 A schematic diagram of the device plugin workflow provided in the embodiment of the present application;

[0082] Figure 5 A schematic diagram of a secure container creation process provided in an embodiment of the present application;

[0083] Figure 6 The processing flow of the secure container runtime component for the RunPodSandbox request provided in the embodiment of the present application;

[0084] Figure 7 A schematic diagram of the structure of a first embodiment of a device for creating a secure container provided in an embodiment of the present application;

[0085] Figure 8 A schematic diagram of the structure of a second embodiment of a device for creating a secure container provided in an embodiment of the present application;

[0086] Fig. 9 A schematic diagram of the structure of the management node provided in the embodiment of the present application;

[0087] Fig.10 A schematic diagram of the structure of a worker node provided in an embodiment of the present application.

[0088] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0089] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0090] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0091] Before introducing the embodiments of the present application, the application background of the embodiments of the present application is first explained:

[0092] In a cloud-native environment, ordinary containers are likely to be invaded into physical machines through permissions, kernel vulnerabilities, etc., which will cause security threats to the entire cloud-native platform. Based on this, secure containers came into being. The virtualization-level isolation achieved by secure containers not only ensures the isolation of user resources in cloud-native scenarios, but also ensures the security of the platform itself.

[0093] In the prior art, secure containers all use virtualization solutions for security isolation. However, in order to pursue better security, secure containers introduce virtualization overhead, which results in the inability of applications in containers to obtain the performance of running directly on physical machines, especially for high-performance distributed application scenarios such as Tensorflow, Spark, etc. The existing solution is to map the storage system to the physical node where the secure container is located, using virtualization technologies such as virtio-9p, virtio-fs, virtio-blk, and virtio-scsi to present it to the secure container, resulting in the inability of the secure container to obtain high-performance storage capabilities.

[0094] Take Tensorflow as an example. Tensorflow is currently the most popular deep learning training service. It has high performance requirements for both computing and storage. Providing Tensorflow services based on cloud native platforms must ensure the security of cloud platforms and user data while also meeting the performance requirements of the service itself. Tensorflow computing power relies on the computing power of physical graphics processing units (GPUs). Secure containers use device pass-through technology to meet computing power requirements, but for high-performance storage systems, Lustre and RDMA are the mainstream high-performance storage systems for data-intensive distributed applications. The introduction of virtualization overhead in secure containers makes it impossible to meet the performance requirements of such application scenarios, and secure containers cannot obtain high-performance storage capabilities.

[0095] Based on this, the present application provides a method for creating a secure container, which can meet the security needs in cloud-native scenarios and enable the secure container to obtain high-performance storage capabilities to meet the user scenarios of high-performance distributed applications.

[0096] Exemplarily, the method for creating a secure container provided in the embodiment of the present application can be applied to Figure 1 In the Kubernetes cloud native platform shown. Figure 1 This is a schematic diagram of the structure of the Kubernetes cloud native platform provided in the embodiment of the present application. Figure 1As shown, the kubernetes cloud native platform 100 includes a management node 101 and a worker node 102. The kubernetes cloud native platform 100 may include one worker node or multiple worker nodes. The number of worker nodes may be determined according to actual conditions, and the embodiment of the present application does not impose any specific restrictions on this.

[0097] The embodiment of the present application uses a secure container runtime component (runtime) based on the Kubernetes cloud native platform 100 to provide an operating environment for cloud native applications. The secure container runtime component uses virtualization technology to use guest kernel / image to securely isolate cloud users from physical infrastructure and host kernel. The Kubernetes cloud native platform 100 dynamically detects and centrally manages the physical resources on the worker node 102. The physical machine resources on the node that provides high-energy storage capabilities of the worker node will be marked.

[0098] Among them, the management node 101 can obtain the pod definition input by the user and generate pod information according to the pod definition. The pod information is used to indicate the worker node used to create the secure container, the state of the secure container is to be created, and the resources and resource quantity required for the secure container. The worker node 102 retrieves the pod information corresponding to the worker node 102 and the state of the secure container is to be created from the management node 101, and creates the secure container according to the pod information.

[0099] It should be understood that the specific process and principle of the method for creating a secure container executed by the Kubernetes cloud native platform 100 can be referred to the following embodiments and will not be repeated here.

[0100] The technical solution of the present application is described in detail below through specific embodiments.

[0101] It should be noted that the following specific embodiments may be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0102] Figure 2 This is a flow chart of a first embodiment of a method for creating a secure container provided in an embodiment of the present application. Figure 2 As shown, the method for creating the secure container may include the following steps:

[0103] S201. The management node obtains the pod definition.

[0104] The pod definition is used to describe the resources and resource quantities required to create a secure container.

[0105] In a possible implementation, a user may declare a pod definition in advance and input the pod definition into a management node. The management node obtains the pod definition input by the user in response to the user's operation.

[0106] Optionally, the pod definition can also include configuration information for configuring the guest kernel and image.

[0107] Among them, the lightweight virtual machine running the secure container uses a lightweight guest kernel and image to start, and the guest kernel / image needs to be customized in advance. In order to ensure the performance and resource overhead of the secure container, the guest kernel / image needs to be customized according to the container resource requirements. For example, for a container service that requires the Lustre high-performance concurrent distributed file system, the guest kernel / image needs to provide an Infiniband (abbreviated as: IB) driver and a Lustre client program so that the secure container runtime component can analyze the resource type required by the container through the definition in the OCI spec, and dynamically match the guest kernel / image according to the resource type to ensure the performance and resource overhead of the secure container. In other words, the above configuration information is used to customize the guest kernel / image according to the container resource requirements.

[0108] S202. The management node obtains a pod creation request through the API server.

[0109] Among them, the pod creation request is generated according to the pod definition.

[0110] In a possible implementation, the user can operate the management node based on the pod definition. The management node responds to the user's operation and obtains the pod creation request input by the user to the API server.

[0111] S203. The management node creates pod information through the API server according to the pod creation request.

[0112] The pod information includes resources and resource quantities required to create a secure container. Optionally, the pod information may also include configuration information for configuring the guest kernel and image.

[0113] S204. The management node allocates worker nodes for running secure containers in the Kubernetes cloud native platform through the scheduler according to the scheduling strategy, resources corresponding to the pod information, and the number of resources.

[0114] In one possible implementation, the scheduler uses the ListWatch mechanism to determine the pod information that is in a pending call state based on the pod information, and allocates a worker node for running the secure container in the Kubernetes cloud native platform based on the scheduling strategy and the resources and resource quantities corresponding to the pod information.

[0115] S205: The management node updates the host of the pod information and updates the status of the pod information to to be created.

[0116] In one possible implementation, after the management node determines the worker node for running the secure container, the host of the pod information is modified to the host corresponding to the worker node, and the status of the pod information is updated from pending call to pending creation (English: unbound).

[0117] The host of the updated pod information corresponds to the worker node, so that the subsequent worker node can determine whether the pod information is the pod information that the worker node needs to process according to the host of the pod information.

[0118] S206. The worker node obtains the pod information corresponding to the host and the worker node and in the state of being created from the management node of the kubernetes cloud native platform through kubelet.

[0119] In one possible implementation, kubelet obtains pod information corresponding to the host and worker nodes and in the state of being created from the management node through the ListWatch mechanism.

[0120] S207. The worker node calls the grpc service provided by the device plugin according to the pod information to allocate device resources for the secure container.

[0121] The device plugin registers the available device resource types and quantities of the worker nodes to the Kubernetes cloud native cloud platform in advance. For the specific process, please refer to Figure 4 The embodiments shown will not be described in detail here.

[0122] S208. The worker node creates a secure container based on device resources.

[0123] Among them, the worker node calls the secure container runtime component, generates a RunPodSandbox request, and calls the secure container runtime component according to the RunPodSandbox request to create a secure container.

[0124] It should be understood that the specific implementation principle and implementation process of this step can refer to Figure 3 The embodiments shown will not be described in detail here.

[0125] The embodiment of the present application provides a method for creating a secure container, wherein the management node obtains a pod definition, obtains a pod creation request through an apiserver, creates pod information through an apiserver according to the pod creation request, and allocates a worker node for running the secure container in the kubernetes cloud native platform through a scheduler according to the scheduling strategy, the resources corresponding to the pod information, and the number of resources, updates the host of the pod information, and updates the status of the pod information to be created. The worker node obtains the pod information corresponding to the host and the worker node and whose status is to be created from the management node of the kubernetes cloud native platform through kubelet, and calls the grpc service provided by the device plugin to allocate device resources for the secure container according to the pod information, and creates a secure container according to the device resources. In this technical solution, the resources and the number of resources required to create a secure container are determined through the pod definition, so that the worker node for creating the secure container is determined according to the resources and the number of resources, and the grpc service provided by the device plugin is called to allocate device resources for the secure container, which can meet the security needs in the cloud native scenario, and enable the secure container to obtain high-performance storage capabilities to meet the user scenarios of high-performance distributed applications.

[0126] Optionally, in some embodiments, after S208, the worker node may also call the secure container runtime component to run the secure container and update the state of the secure container according to the creation result.

[0127] based on Figure 2 The embodiment shown, Figure 3 This is a flow chart of a second embodiment of a method for creating a secure container provided in an embodiment of the present application. Figure 3 As shown, S208 may include the following steps:

[0128] S301. The worker node calls the secure container runtime component to generate a RunPodSandbox request.

[0129] The RunPodSandbox request is used to indicate the device resources required to create a secure container and the configuration information used to configure the guest kernel and image.

[0130] S302. The worker node calls the secure container runtime component according to the RunPodSandbox request and creates a secure container.

[0131] In a possible implementation, S302 may be implemented by the following steps (1) to (6):

[0132] Step (1), pull the container image of the secure container according to the RunPodSandbox request.

[0133] Step (2): configure the virtual machine running the secure container according to the container image and device resources.

[0134] Optionally, the central processing unit (CPU), memory, network, device resources, etc. of the virtual machine may be configured.

[0135] Step (3): Configure the guest kernel and image according to the configuration information requested by RunPodSandbox.

[0136] For example, for container services that require the Lustre high-performance concurrent distributed file system, the guest kernel / image needs to provide an Infiniband driver and a Lustre client program so that the secure container runtime component can analyze the resource types required by the container through the definition in OCIspec and dynamically match the guest kernel / image based on the resource types to ensure the performance and resource overhead of the secure container.

[0137] Step (4), start the configured virtual machine.

[0138] For example, for secure containers that require high-performance storage services, such as a Lustre file system or RDMA, the secure container runtime component obtains the IB card device on the physical machine based on the IB card allocated in the OCI spec, and uses device pass-through technology to hot-plug the IB card into a lightweight virtual machine through virtualization hot-plug capabilities.

[0139] Step (5): through the secure container runtime component, notify the Agent in the virtual machine to configure resources for the secure container.

[0140] The resource configuration includes at least one of the following: loading kernel modules, loading device drivers, configuring networks, mounting file systems, and configuring security policies.

[0141] The secure container runtime component notifies the agent in the virtual machine to configure resources for the secure container through hooks according to the resource configuration in the OCI spec, including: loading kernel modules, loading device drivers, configuring networks, mounting file systems, configuring security policies, etc. After the configuration is completed, the container creation request is returned to the secure container runtime component.

[0142] It should be understood that a container creation request and a pod creation request are not the same request.

[0143] Step (6), running the component through the secure container, and returning a creation result indicating successful creation.

[0144] After receiving the container creation request, the secure container runtime component notifies the agent to create a secure container. After the agent configures the container resources, the secure container runtime component starts the secure container and returns the sandbox to indicate the successful creation result.

[0145] In the above embodiment, a customizable guest kernel / image can be implemented through pod definition, and a customized guest kernel / image can be dynamically selected according to the secure container definition, thereby ensuring container performance and resource overhead. At the same time, the secure container runtime component can pass high-performance storage devices directly to the virtual machine through device pass-through, and the agent configures the high-performance storage resources required by the container, so that the secure container can obtain the high-performance storage system access capability of the ordinary container.

[0146] Based on any of the above embodiments, after obtaining the pod information corresponding to the host and the worker node and in the state of being created from the management node of the kubernetes cloud native platform through kubelet, the worker node can also register the type and quantity of device resources available to the worker node to the kubernetes cloud native platform through the deviceplugin service.

[0147] Specifically, the implementation process can refer to the following steps:

[0148] Step 1: Read the service configuration file to obtain the device ID to be processed.

[0149] Step 2: Scan the devices of the worker node to determine the type and quantity of device resources of the target device corresponding to the device identifier to be processed.

[0150] Step 3: Update the driver of each target device of the worker node to vfio / vfio-pci.

[0151] Step 4: Determine the type and quantity of device resources available to the worker node through the device plugin service.

[0152] Step 5. Start the grpc service through the device plugin service.

[0153] Step 6. Register the device plugin service to the kubelet service.

[0154] Step 7. Call the grpc service through the kubelet service to obtain the type and quantity of available device resources.

[0155] The kubelet service calls the grpc service to obtain the types and quantities of device resources of all target devices. The types and quantities of device resources of all target devices are the types and quantities of available device resources.

[0156] Step 8. Register the types and quantities of available device resources to the Kubernetes cloud native platform.

[0157] Based on the method for creating a secure container shown in any of the above embodiments, the method is explained below through several specific examples.

[0158] The worker node runs the high-performance storage device plugin service. The device plugin workflow of the device plugin service is as follows: Figure 4 shown.

[0159] Figure 4 The following is a schematic diagram of the workflow of the device plugin provided in the embodiment of the present application. Figure 4 As shown, the workflow includes the following steps:

[0160] S41. Initialize the device type and quantity of the worker node.

[0161] It should be understood that the device type and quantity of the initialized worker node are 0.

[0162] S42. Read the configuration file of the device plugin to obtain the vendorID and DeviceID that need to be managed.

[0163] S43. Scan the local devices on the worker node.

[0164] S44. According to the vendorID and DeviceID to be managed, it is determined whether there is a device to be managed on the worker node.

[0165] If yes, execute step S45, if no, end.

[0166] S45. Update the device driver of the device to be managed to vfio / vfio-pci.

[0167] Among them, check whether the device driver of the device to be managed is vfio / vfio-pci. If so, there is no need to update the device driver. If not, the device driver needs to be updated to vfio / vfio-pci. Specifically, configure the physical machine resources provided to the kuberneteS4 cloud native platform, remove the default device driver, and use vfio / vfio-pci as the device driver.

[0168] S46. The device plugin counts the available device resource types and available resource quantities, and updates the device types and quantities.

[0169] S47. Device Plugin starts the grpc server.

[0170] S48. The device plugin registers resources to the kubelet service.

[0171] Furthermore, kubelet obtains the type and quantity of resources to be registered through grpc server calls, and registers the resources to the kubernetes cloud native platform.

[0172] Figure 5 A schematic diagram of the secure container creation process provided in the embodiment of the present application. Figure 4 As shown, the secure container creation process includes the following steps:

[0173] S51. User declares pod definition.

[0174] The pod definition includes all the resources and quantities required by the secure container.

[0175] S52. The user sends a pod creation request to the API server according to the pod definition.

[0176] Correspondingly, the API server receives the pod creation request sent by the user.

[0177] S53. The API server creates pod information according to the pod creation request.

[0178] S54. The scheduler obtains the pod information to be called from the API server through the ListWatch mechanism, allocates the worker node to run the secure container according to the resource definition and scheduling strategy of the pod information, and updates the host and status in the pod information (the status is updated to unbound).

[0179] S55. The work service kubelet on the worker node obtains the pod information in the ubound state through the ListWatch mechanism, and prepares the resources required for the secure container based on the pod information:

[0180] a. Call the grpc service provided by the device plugin to allocate device resources to the secure container;

[0181] b. Call RunPodSandbox of the secure container runtime component;

[0182] c. Call the secure container runtime component to create a container;

[0183] d. Call the secure container runtime component to run the container;

[0184] S56. Kubelet updates the container status of the pod information in the API server based on the request return result.

[0185] The container status of the pod information is the status of the pod information mentioned above, that is, the status of the pod information is updated from ubound to creation completed.

[0186] Figure 6 The processing flow of the secure container runtime component for the RunPodSandbox request provided in the embodiment of the present application. Figure 6 As shown, the processing flow includes:

[0187] S61. Select the guest kernel / image of the virtual machine.

[0188] Among them, you can pull the container image and select the guest kernel and image of the virtual machine according to the user-defined guest kernel and image carried in the RunPodSandbox request.

[0189] Among them, the lightweight virtual machine running the secure container uses a lightweight guest kernel and image to start, and the guest kernel / image needs to be customized in advance. In order to ensure the performance and resource overhead of the secure container, the guest kernel / image needs to be customized according to the container resource requirements. For example, for a container service that requires the Lustre high-performance concurrent distributed file system, the guest kernel / image needs to provide an Infiniband driver and a Lustre client program so that the secure container runtime component can analyze the resource type required by the container through the definition in the OCI spec, and dynamically match the guest kernel / image according to the resource type to ensure the performance and resource overhead of the secure container.

[0190] S62: Allocate physical resources to the virtual machine.

[0191] The virtual machine running the secure container is configured, and the physical resources may include: CPU, memory, network, device resources, etc., and may also include other resources, which can be limited according to actual conditions. The embodiments of the present application do not impose specific restrictions on this.

[0192] S63. Run the virtual machine.

[0193] S64. Obtain the container storage type.

[0194] S65. Obtain a high-performance storage device.

[0195] S66. Hot-insert storage devices.

[0196] For secure containers that require high-performance storage services, such as the Lustre file system or RDMA, the secure container runtime component obtains the IB card device on the physical machine based on the IB card allocated in the OCI spec, and uses device pass-through technology to hot-plug the IB card into a lightweight virtual machine through virtualization hot-plug capabilities.

[0197] S67. Configure the agent.

[0198] The secure container runtime component notifies the agent in the virtual machine. The agent configures resources for the secure container through hooks according to the resource configuration in the OCI spec, loads kernel modules, loads device drivers, configures networks, mounts file systems, configures security policies, etc. After the configuration is completed, the container creation request is returned to the secure container runtime component.

[0199] After receiving the container creation request, the secure container runtime component notifies the agent to create a secure container. After the agent configures the container resources, the secure container runtime component starts the secure container to provide container services.

[0200] The following is an embodiment of the device of the present application, which can be used to execute the embodiment of the method of the present application. For details not disclosed in the embodiment of the device of the present application, please refer to the embodiment of the method of the present application.

[0201] Figure 7 This is a schematic diagram of the structure of the first embodiment of the device for creating a secure container provided in the present application. Figure 7 As shown, the secure container creation device 700 includes:

[0202] The first acquisition module 701 is used to obtain a pod definition, where the pod definition is used to describe the resources and resource quantities required to create a secure container.

[0203] The second acquisition module 702 is used to obtain a pod creation request through the API server, where the pod creation request is generated according to the pod definition.

[0204] The creation module 703 is used to create pod information through the API server according to the pod creation request.

[0205] The allocation module 704 is used to allocate worker nodes for running secure containers in the Kubernetes cloud native platform through the scheduler according to the scheduling strategy, resources and resource quantities corresponding to the pod information.

[0206] The update module 705 is used to update the host of the pod information and update the status of the pod information to be created, where the host corresponds to the worker node.

[0207] The secure container creation device provided in the embodiments of the present application can be used to execute the secure container creation method on the management node side in any of the above embodiments. The implementation principle and technical effects are similar and will not be repeated here.

[0208] Figure 8 This is a schematic diagram of the structure of the second embodiment of the device for creating a secure container provided in the embodiment of the present application. Figure 8 As shown, the secure container creation device 800 includes:

[0209] The acquisition module 801 is used to obtain the pod information corresponding to the host and worker nodes and in the state of being created from the management node of the kubernetes cloud native platform through kubelet.

[0210] The calling module 802 is used to call the grpc service provided by the device plugin to allocate device resources for the secure container according to the pod information.

[0211] The creation module 803 is used to create a secure container according to device resources.

[0212] In a possible implementation, the creation module 803 is specifically configured to:

[0213] The secure container runtime component is called to generate a RunPodSandbox request, which is used to indicate the device resources required to create a secure container and the configuration information used to configure the guest kernel and image.

[0214] Based on the RunPodSandbox request, the secure container runtime component is called to create a secure container.

[0215] In a possible implementation, the creation module 803 is specifically configured to:

[0216] Pull the container image of the secure container according to the RunPodSandbox request.

[0217] Configure the virtual machine running the secure container based on the container image and device resources.

[0218] Configure the guest kernel and image according to the configuration information requested by RunPodSandbox.

[0219] Start the configured virtual machine.

[0220] Through the secure container runtime component, the Agent in the virtual machine is notified to configure resources for the secure container. The resource configuration includes at least one of the following: loading kernel modules, loading device drivers, configuring networks, mounting file systems, and configuring security policies.

[0221] Runs the component through a secure container and returns a creation result indicating that the creation was successful.

[0222] In a possible implementation, before obtaining the pod definition, the secure container creation device 800 further includes:

[0223] The registration module is used to register the types and quantities of device resources available to worker nodes to the Kubernetes cloud native platform through the device plugin service.

[0224] In a possible implementation, the registration module is specifically used to:

[0225] Read the service configuration file to obtain the device ID to be processed.

[0226] Scan the devices of the worker node to determine the type and quantity of device resources of the target device corresponding to the device identifier to be processed.

[0227] Update the driver of each target device of the worker node to vfio / vfio-pci.

[0228] The device plugin service determines the type and quantity of device resources available to the worker node.

[0229] Start the grpc service through the device plugin service.

[0230] Register the device plugin service to the kubelet service.

[0231] The kubelet service calls the grpc service to obtain the type and quantity of available device resources.

[0232] Register the types and quantities of available device resources to the Kubernetes cloud native platform.

[0233] In one possible implementation, the calling module 802 is used to call the secure container runtime component to run the secure container.

[0234] The secure container creation device 800 also includes an update module for updating the state of the secure container according to the creation result.

[0235] The secure container creation device provided in the embodiment of the present application can be used to execute the secure container creation method on the worker node side in any of the above embodiments. Its implementation principle and technical effects are similar and will not be repeated here.

[0236] It should be noted that it should be understood that the division of the various modules of the above device is only a division of logical functions. In actual implementation, they can be fully or partially integrated into one physical entity, or they can be physically separated. And these modules can all be implemented in the form of software calling through processing elements; they can also be all implemented in the form of hardware; some modules can also be implemented in the form of software calling through processing elements, and some modules can be implemented in the form of hardware. In addition, all or part of these modules can be integrated together, or they can be implemented independently. The processing element described here can be an integrated circuit with signal processing capabilities. In the implementation process, each step of the above method or each module above can be completed by an integrated logic circuit of hardware in the processor element or instructions in the form of software.

[0237] Fig. 9This is a schematic diagram of the structure of the management node provided in the embodiment of the present application. Fig. 9 As shown, the management node 101 may include: a processor 1011, a memory 1012, and computer program instructions stored in the memory 1012 and executable on the processor 1011. When the processor 1011 executes the computer program instructions, it implements the method for creating a secure container on the management node side provided by any of the aforementioned embodiments.

[0238] Optionally, the above-mentioned components of the management node 101 may be connected via a system bus.

[0239] The memory 1012 may be a separate storage unit or a storage unit integrated in the processor. The number of processors may be one or more.

[0240] Optionally, the management node 101 may also include a communication interface for interacting with other nodes.

[0241] The management node provided in the embodiment of the present application can be used to execute the method for creating a secure container on the management node side provided in any of the above-mentioned method embodiments. Its implementation principle and technical effects are similar and will not be repeated here.

[0242] Fig.10 This is a schematic diagram of the structure of a worker node provided in an embodiment of the present application. Fig.10 As shown, the worker node 102 may include: a processor 1021, a memory 1022, and computer program instructions stored in the memory 1022 and executable on the processor 1021. When the processor 1021 executes the computer program instructions, it implements the method for creating a secure container on the worker node side provided in any of the aforementioned embodiments.

[0243] Optionally, the above-mentioned components of the worker node 102 may be connected via a system bus.

[0244] The memory 1022 may be a separate storage unit or a storage unit integrated in the processor. The number of processors may be one or more.

[0245] Optionally, the worker node 102 may also include a communication interface for interacting with other nodes.

[0246] The worker node provided in the embodiment of the present application can be used to execute the method for creating a secure container on the worker node side provided in any of the above method embodiments. The implementation principle and technical effects are similar and will not be repeated here.

[0247] It should be understood that the processor may be a CPU, or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), etc. A general-purpose processor may be a microprocessor or the processor may be any conventional processor, etc. The steps of the method disclosed in the present application may be directly embodied as being executed by a hardware processor, or may be executed by a combination of hardware and software modules in the processor.

[0248] The system bus may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The system bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus. The memory may include random access memory (RAM) and may also include non-volatile memory (NVM), such as at least one disk storage.

[0249] All or part of the steps of the above-mentioned method embodiments can be completed by hardware related to program instructions. The above-mentioned program can be stored in a readable memory. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the above-mentioned memory (storage medium) includes: read-only memory (ROM), RAM, flash memory, hard disk, solid state drive, magnetic tape, floppy disk, optical disc and any combination thereof.

[0250] An embodiment of the present application provides a computer-readable storage medium, in which computer execution instructions are stored. When the computer execution instructions are executed on a computer, the computer executes the above-mentioned method for creating a secure container.

[0251] The computer-readable storage medium mentioned above can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory, electrically erasable programmable read-only memory, erasable programmable read-only memory, programmable read-only memory, read-only memory, magnetic memory, flash memory, magnetic disk or optical disk. The readable storage medium can be any available medium that can be accessed by a general or special-purpose computer.

[0252] Optionally, a readable storage medium is coupled to a processor so that the processor can read information from the readable storage medium and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can be located in an application specific integrated circuit (ASIC). Of course, the processor and the readable storage medium can also exist in the device as discrete components.

[0253] An embodiment of the present application also provides a computer program product, which includes a computer program. The computer program is stored in a computer-readable storage medium. At least one processor can read the computer program from the computer-readable storage medium. When the at least one processor executes the computer program, the above-mentioned method for creating a secure container can be implemented.

[0254] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A method for creating a secure container, characterized in that: Management nodes applied to the Kubernetes cloud native platform, including: Get a pod definition, which is used to describe the resources and resource quantities required to create a secure container; Obtain a pod creation request through the API server, where the pod creation request is generated according to the pod definition; According to the pod creation request, create pod information through the API server; Allocate, through a scheduler, a worker node for running the secure container in the Kubernetes cloud native platform according to a scheduling strategy, resources corresponding to the pod information, and the number of resources; Update the host of the pod information, and update the status of the pod information to to be created, where the host corresponds to the worker node.

2. A method for creating a secure container, characterized in that: Worker nodes applied to the Kubernetes cloud native platform include: Obtaining, through kubelet, pod information corresponding to the host and the worker node and in a state of being created from the management node of the kubernetes cloud native platform; According to the pod information, call the grpc service provided by the device plugin to allocate device resources for the secure container; The secure container is created according to the device resources.

3. The method according to claim 2, characterized in that The step of creating the secure container according to the device resources includes: Call the secure container runtime component to generate a RunPodSandbox request, where the RunPodSandbox request is used to indicate the device resources required to create the secure container and configuration information for configuring the guest kernel and image; According to the RunPodSandbox request, the secure container runtime component is called to create a secure container.

4. The method according to claim 3, characterized in that The step of calling the secure container runtime component according to the RunPodSandbox request to create a secure container includes: Pull the container image of the secure container according to the RunPodSandbox request; According to the container image and the device resources, a virtual machine running the secure container is configured; Configure the guest kernel and image according to the configuration information requested by the RunPodSandbox; Start the configured virtual machine; Through the secure container runtime component, notify the Agent in the virtual machine to configure resources for the secure container, where the resource configuration includes at least one of the following: loading a kernel module, loading a device driver, configuring a network, mounting a file system, and configuring a security policy; The component is run through the secure container, and a creation result indicating successful creation is returned.

5. The method according to any one of claims 2 to 4, characterized in that: Before obtaining, by kubelet, pod information corresponding to the host and the worker node and in a state of being to be created from the management node of the kubernetes cloud native platform, the method further includes: The type and quantity of device resources available to the worker node are registered to the Kubernetes cloud native platform through the device plugin service.

6. The method according to claim 5, characterized in that The registering the type and quantity of device resources available to the worker node to the kubernetes cloud native platform through the device plugin service includes: Read the service configuration file to obtain the device ID to be processed; Scan the device of the worker node to determine the type and quantity of device resources of the target device corresponding to the device identifier to be processed; Update the driver of each target device of the worker node to vfio / vfio-pci; Determining the type and quantity of device resources available to the worker node through the device plugin service; Start the grpc service through the device plugin service; Register the device plugin service to the kubelet service; Calling the grpc service through the kubelet service to obtain the type and quantity of the available device resources; Register the types and quantities of the available device resources to the Kubernetes cloud native platform.

7. The method according to claim 4, characterized in that The method further comprises: Calling the secure container runtime component to run the secure container; According to the creation result, the state of the secure container is updated.

8. A device for creating a secure container, characterized in that: Management nodes applied to the Kubernetes cloud native platform, including: A first acquisition module is used to acquire a pod definition, where the pod definition is used to describe the resources and resource quantities required to create a secure container; A second acquisition module is used to obtain a pod creation request through an API server, where the pod creation request is generated according to the pod definition; A creation module, used to create pod information through the API server according to the pod creation request; An allocation module, configured to allocate a worker node for running the secure container in the Kubernetes cloud native platform according to a scheduling strategy, resources corresponding to the pod information, and the number of resources through a scheduler; An update module is used to update the host of the pod information and update the status of the pod information to be created, wherein the host corresponds to the worker node.

9. A device for creating a secure container, characterized in that: Worker nodes applied to the Kubernetes cloud native platform include: An acquisition module, used to obtain pod information corresponding to the host and the worker node and in a state of being created from the management node of the kubernetes cloud native platform through kubelet; A calling module is used to call the grpc service provided by the device plugin to allocate device resources for the secure container according to the pod information; A creation module is used to create the security container according to the device resources.

10. A management node, comprising: A processor, a memory, and computer program instructions stored in the memory and executable on the processor, wherein the processor is used to implement the method according to claim 1 when executing the computer program instructions.

11. A worker node, comprising: A processor, a memory, and computer program instructions stored in the memory and executable on the processor, wherein the processor is used to implement the method according to any one of claims 2 to 7 when executing the computer program instructions.

12. A kubernetes cloud native platform, comprising: A management node and a worker node, wherein the management node is used to implement the method according to claim 1, and the worker node is used to implement the method according to any one of claims 2 to 7.

13. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 7 when executed by a processor.

Images