Secure container migration

Abstract

Provided in the embodiments of the present disclosure are a secure container migration method, a device, a storage medium and a program product. A first secure container on a source node comprises a target resource object and a non-target resource object. When the target resource object is implemented as a pass-through device of the secure container, for different characteristics of the target resource object and the non-target resource object, state data of the target resource object and the non-target resource object in the first secure container is acquired by means of the cooperation of different levels of components, i.e., by means of a driver of the target resource object and a container runtime component, so that the state data of the target resource object and the non-target resource object can be relatively accurately migrated to a second secure container on a target node. By means of the cooperation of different levels of components, the rapid migration of workloads in a secure container can be better achieved, thereby solving the problem of state migration for a secure container including a target resource object that cannot be directly managed by the secure container or by its affiliated cloud provider.

Description

Security container migration TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of cloud computing, and particularly relates to security container migration. BACKGROUND

[0002] In a cloud computing environment, a multi-tenant architecture is usually adopted on a single server node, and in order to achieve security isolation between different tenants, containerization technology can be used to package an application and its dependencies into an independent container, so as to ensure the security and consistency of the application. As a branch of container technology, a secure container aims to make the isolation between containers more thorough by combining the security of virtualization and the convenience of containers, and can provide a higher level of isolation and protection for the application in the container, and therefore is widely used.

[0003] AI (Artificial Intelligence) intelligent computing as an important application scenario of cloud computing, more and more AI applications and AI frameworks for developing and deploying AI applications are deployed in secure containers, which leads to higher demand for computing resources in secure containers. As a high-performance computing resource provided by cloud vendors, GPU (Graphics Processing Unit) has shown strong performance in the field of AI and deep learning, and therefore some cloud vendors have proposed to apply GPU to secure containers as a computing power supply for applications in secure containers.

[0004] Among them, the combination of GPU and secure containers can solve the computing power demand problem of secure containers on the basis of realizing security isolation, and at the same time, new technical challenges are also faced. For example, in the case of application upgrade, resource optimization or application failure, the secure container needs to be migrated, and how to migrate the secure container with GPU resources is a technical problem to be solved. SUMMARY

[0005] The aspects of the present disclosure provide a security container migration method, device, storage medium and program product to solve the migration problem of secure containers with GPU resources.

[0006] The embodiment of the present disclosure provides a secure container migration method, which is used for migrating a workload in a first secure container on a source node, the first secure container comprising a target resource object and a non-target resource object, the target resource object being a pass-through device of the first secure container, and the method comprising the following steps: during running of the first secure container, suspending running of the target resource object of the first secure container through a driver of the target resource object, and obtaining state data of the target resource object of the first secure container; migrating the state data of the target resource object of the first secure container into a target resource object of a second secure container, the second secure container being a secure container on a target node; suspending running of the first secure container through a container runtime component corresponding to the first secure container, and obtaining state data of the non-target resource object of the first secure container; and migrating the state data of the non-target resource object of the first secure container into a non-target resource object of the second secure container, so as to migrate the workload in the first secure container to the second secure container.

[0007] The embodiment of the present disclosure also provides a secure container migration method, which is used for migrating a workload of a first secure container on a source node, and a second secure container is created on a target node, the second secure container comprising a target resource object managed by a driver and a non-target resource object not managed by the driver; the method comprising the following steps: receiving state data of the target resource object of the first secure container, and saving the state data of the target resource object of the first secure container in the target resource object of the second secure container; receiving state data of the non-target resource object of the first secure container, and saving the state data of the non-target resource object of the first secure container in the non-target resource object of the second secure container.

[0008] The embodiment of the present disclosure also provides a computer device, which can be implemented as a source node, and the source node comprising: a host, the host being deployed with a first secure container and a container runtime component, the first secure container comprising a target resource object and a non-target resource object, the target resource object being a pass-through device of the first secure container, and the container runtime component being used for executing steps in the secure container migration method.

[0009] The embodiment of the present disclosure also provides a computer device, comprising: a memory and a processor; the memory is used for storing a computer program, and the processor is coupled with the memory and is used for executing the computer program in the memory, so as to implement steps in the secure container migration method.

[0010] The embodiments of the present disclosure further provide a computer readable storage medium storing a computer program, which, when executed by a processor, causes the processor to implement the steps in the secure container migration method.

[0011] The embodiments of the present disclosure further provide a computer program product comprising computer programs / instructions, which, when executed by a processor, perform the steps in the secure container migration method.

[0012] The embodiments of the present disclosure further provide a cloud computing system, comprising: a source node configured to implement the first set of secure container migration methods, and a target node configured to implement the second set of secure container migration methods; the source node and the target node cooperate with each other to migrate the workloads in the first secure container on the source node to the second secure container on the target node.

[0013] In the present embodiment, the first secure container on the source node comprises target resource objects and non-target resource objects, and in the case where the target resource objects are implemented as pass-through devices of the first secure container, different characteristics of the target resource objects and the non-target resource objects can be obtained by cooperation of different levels of components, i.e., the state data of the target resource objects and the non-target resource objects in the first secure container can be obtained by the driver of the target resource objects and the container runtime component, and then the state data of the target resource objects and the non-target resource objects can be more accurately migrated to the second secure container on the target node. In this way, the rapid migration of the workloads in the secure container can be better achieved by cooperation of different levels of components, and the state migration of the secure container with the target resource objects that cannot be directly managed by the cloud vendor of the secure container can be solved. BRIEF DESCRIPTION OF DRAWINGS

[0014] The accompanying drawings, which are included to provide a further understanding of the present disclosure and constitute a part of this disclosure, illustrate certain illustrative embodiments of the present disclosure and are used to explain the present disclosure, but do not limit the present disclosure. In the drawings:

[0015] FIG. 1a is a system architecture diagram of a computer device carrying a secure container according to an illustrative embodiment of the present disclosure;

[0016] FIG. 1b is another system architecture diagram of a computer device carrying a secure container according to an illustrative embodiment of the present disclosure;

[0017] FIG. 2a is an interaction flow diagram for state migration between a source node and a target node according to an illustrative embodiment of the present disclosure;

[0018] FIG. 2b is another interaction flow diagram for state migration between a source node and a target node according to an example embodiment of the present disclosure;

[0019] FIG. 3 is a flow diagram of a method for secure container migration according to an example embodiment of the present disclosure;

[0020] FIG. 4 is a flow diagram of another method for secure container migration according to an example embodiment of the present disclosure;

[0021] FIGS. 5-6 are diagrams of a secure container migration device according to an example embodiment of the present disclosure;

[0022] FIG. 7 is a diagram of a computer device according to an example embodiment of the present disclosure. DETAILED DESCRIPTION

[0023] For the purposes of the present disclosure, technical solutions and advantages, the following will combine specific embodiments of the present disclosure and corresponding drawings to clearly and completely describe the technical solutions of the present disclosure. Obviously, the described embodiments are only a part of the embodiments of the present disclosure, not all embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by a person of ordinary skill in the art without creative labor fall within the scope of the present disclosure.

[0024] It should be noted that in the case of user information involved in the embodiments of the present disclosure, the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the embodiments of the present disclosure are all information and data authorized by the user or authorized by all parties, and the collection, use and processing of related data need to comply with relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation portal for user to choose authorization or refusal. In addition, the various models (including but not limited to language models or large models) involved in the present disclosure are in compliance with relevant laws and standards.

[0025] With the deployment of AI applications and AI frameworks in secure containers, the requirements of secure containers for computing resources are becoming higher and higher. Among them, the AI framework refers to a framework for developing and deploying AI applications, such as TensorFlow (an open source machine learning platform), PyTorch (an open source machine learning library), or Caffe (a deep learning framework with expressiveness, speed, and modular thinking). AI applications refer to applications developed and deployed based on AI frameworks, such as AI-based image processing programs, video processing programs, or game programs, etc.

[0026] The secure container is a secure isolated container running environment provided by a virtualization technology. Different from a traditional container running environment, the secure container is a container running environment with a complete operating system (i.e., a guest operating system). In the traditional container running environment, no guest operating system is included, and the operating system of the host is relied on. Based on this difference, compared with the traditional container, the secure container can have higher security and isolation.

[0027] Exemplarily, FIG. 1a is a system architecture diagram of a computer device 1 carrying a secure container. The computer device 1 can be implemented as any electronic device, such as a mobile phone, a tablet computer, a computer, or any type of server, and the present embodiment is not limited thereto. As shown in FIG. 1a, the computer device 1 can include a host, which includes a user space 11 of the host (which can also be referred to as a user mode of the host), an operating system 12 of the host (which can also be referred to as a kernel mode of the host), and a hardware layer 13 including various hardware resources of the host, such as, but not limited to, a GPU 131, a CPU (Central Processing Unit) 132, and a network card device, etc. In FIG. 1a, the network card device is implemented as an RDMA (Remote Direct Memory Access) network card 133 as an example, but is not limited thereto, and the network card device can also be implemented as a normal network card. The operating system 12 of the host refers to an operating system running on the host, which is the basis of the secure container and the virtualization technology, and can include drivers of various hardware resources on the host, as well as a virtualization manager. The virtualization manager can be, but is not limited to, KVM (Kernel-based Virtual Machine), and in FIG. 1a, the virtualization manager is implemented as KVM 121 as an example, wherein the KVM can make the operating system 12 of the host become a virtual machine monitor, and can allow a user to run multiple virtualization instances on the host operating system. The user space 11 of the host is used to carry various virtualization instances running on the host, such as a secure container 111 and a container runtime component 112.

[0028] The secure container 111 can implement its core functions and features through a container runtime component 112, which is a series of software components responsible for managing the secure container in a containerized environment. These components work together to provide functions such as creation, startup, stop, monitoring, etc. of the secure container 111, and ensure that the secure container can run safely in an isolated environment. In the embodiments of the present disclosure, the implementation of the container runtime component 112 is not limited, for example, Docker, RunC, RunD, etc. can be used. Among them, Docker is one of the more popular container runtimes, which provides complete container management functions. RunC is a lightweight container runtime that is the basis for Docker and other container tools. RunD is a lightweight secure container solution that provides a secure environment to run containerized applications, aiming to provide enhanced security and isolation to protect sensitive data and applications from potential threats. RunD contains several key components, one of which is RunD VMM (Virtual Machine Manager), which is a lightweight virtual machine manager for managing and running virtual machine instances.

[0029] The secure container 111 can include an application 1111 and a client operating system 1112. The client operating system 1112 in the secure container 111 represents an independent and isolated running environment, which can be a virtual machine, but is not limited thereto. The application 1111 can be implemented as the AI framework or AI application mentioned above, or as any other type of application framework or application, and the embodiments are not limited in this regard. With respect to the client operating system 1112, the application 1111 can be located in the user space of the secure container 111. It should be noted that the user space 11 of the secure container 111 can include other components in addition to the application 1111.

[0030] The client operating system 1112 refers to an operating system running inside the secure container 111, which can provide fine-grained access control functions, encrypted data storage functions, and trusted execution environments to ensure that the applications and data inside the container are protected. The access control function can limit the processes inside the container to access resources based on security policies, the encrypted data storage function can protect the data inside the container based on encryption technology, and the trusted execution environment can be used to provide higher levels of security protection. The client operating system 1112 includes drivers for various hardware resources contained in the secure container, such as drivers for various hardware resources passed through using pass-through technology.

[0031] In the embodiments of the present disclosure, the introduction of GPU resources into the secure container scenario can solve the problem of the demand of secure containers for computing power resources. It is explained herein that, in addition to GPU resources, NPU (Neural Processing Unit), DPU (Data Processing Unit), and the like can also be implemented as computing power resources of secure containers. In the embodiments of the present disclosure, GPU, NPU, DPU, and the like are collectively referred to as target resource objects. First, the target resource objects refer to computing resources of hardware, and second, the target resource objects refer to computing resources that do not support virtualization or for which there is no mature and standardized virtualization solution. For example, TPU (Tensor Processing Unit) does not have built-in scheduling functions and therefore does not support virtualization. For another example, GPU is mainly responsible for processing complex computing tasks and has a large amount of built-in graphics memory. Although some virtualization solutions exist, they mostly face challenges in resource allocation in a multi-tenant environment and security issues between different tenant information, and are not mature or standardized. In comparison, CPU, network card, memory, and the like are resources that have relatively mature and standardized virtualization solutions, and these resources are referred to as non-target resource objects.

[0032] In the embodiments of the present disclosure, the secure container (or the cloud vendor to which the secure container belongs) has certain control requirements for resource objects. These control requirements refer to control related to the secure container, such as the creation, destruction, and migration of the secure container, which all involve control of related resource objects. Depending on whether the resource objects support virtualization technology or whether there is a mature virtualization technology, the implementation of the control of the resource objects from the secure container level will be different. For resource objects that support virtualization technology and have mature virtualization technology, such as CPU, network card, and memory, the cloud vendor to which the secure container belongs can implement control logic for these resource objects in the container runtime component 112 in the secure container, without relying on the drivers of these resource objects. That is, the control logic of the resource objects from the secure container level does not need to be implemented in the drivers of the resource objects. In this way, the cloud vendor to which the secure container belongs can implement the control logic for these resource objects by itself, without relying on the vendors of the resource objects. For resource objects that do not support virtualization technology or do not have mature virtualization technology, such as GPU and TPU, the control of the resource objects from the secure container level needs to rely on the drivers of the resource objects, that is, the relevant API interfaces exposed by the drivers need to be called for implementation. In this way, the cloud vendor to which the secure container belongs cannot implement the control logic for these resource objects by itself and needs to rely on the implementation of the drivers by the vendors of the resource objects.

[0033] Based on the above, various resource objects involved in the security container can be divided into two categories, namely target resource objects and non-target resource objects. Among them, the non-target resource object refers to a resource object that can implement control logic in the container runtime component 112 in the security container based on virtualization technology. Controlling non-target resource objects from the security container level does not depend on the driver of the non-target resource object. The target resource object refers to a resource object that cannot implement control logic in the container runtime component 112 in the security container based on virtualization technology. Controlling target resource objects from the security container level depends on the driver of the target resource object.

[0034] Since the target resource object itself does not support virtualization or the virtualization scheme is not mature, the target resource object cannot be directly applied to the security container 111 based on the virtualization mode like CPU, memory and other resources. This poses the primary problem of how to apply the target resource object to the security container 111. In view of this, the present embodiment provides two implementation modes.

[0035] Implementation mode 1: Function expansion is performed in the driver of the target resource object (such as GPU), and a function module (referred to as a virtualization function module) for virtualizing the target resource object is added. The target resource object (such as GPU) is divided into multiple virtual computing resources (virtual GPU) through the virtualization function module, and the divided virtual computing resources are provided for the security container 111 to use; the driver of the target resource object is located in the operating system 12 of the host computer, and needs to provide an API (Application Programming Interface) to the outside. In this way, when the security container 111 can obtain the related state information of the target resource object by calling the API interface, it provides conditions for the migration of the security container 111. However, since the driver of the target resource object is located in the operating system of the host computer, it involves interaction with the kernel of the host computer, which can increase the consumption of computing power and cause waste of computing power. In addition, since the driver belongs to the private solution of the target resource object manufacturer, it is difficult for cloud manufacturers to control the technical details and difficult to perform additional function expansion on the driver.

[0036] In a manner of implementation 2, the target resource object is passed through to the secure container 111 to form an overall architecture of the secure container 111 + pass-through device. In the manner of implementation 2, the target resource object can be referred to as a pass-through device of the secure container 111. In the manner of implementation 2, the pass-through technology can be used to enable the secure container 111 to directly access the target resource object, instead of indirectly accessing the target resource object through the host operating system 12. Since the pass-through technology enables the secure container 111 to directly access the target resource object, the overhead of the host operating system 12 can be reduced, and the I / O (Input / Output) performance is improved. At the same time, the number of times of copying data between the host operating system 12 and the secure container 111 can be reduced, and the latency is reduced and the throughput is improved.

[0037] The embodiments of the present disclosure do not limit the pass-through technology to be used, such as SR-IOV (Single Root I / O Virtualization, a virtualization technology) or PCI Passthrough (Peripheral Component Interconnect Passthrough, a standard for peripheral components) and the like. The SR-IOV can be used to enable a physical device such as a network adapter and a GPU to directly provide a virtual function (Virtual Function, VF) for multiple virtual machines, each virtual function can be independently configured, and can directly access the hardware resources of the host. The PCI Passthrough can be used to pass a physical device such as a network adapter and a GPU to the secure container, so that the secure container can directly access the physical device, and the physical device can be referred to as a pass-through device.

[0038] In some preferred embodiments, the device pass-through can also be implemented based on the VFIO (Virtual Function I / O) technology. The VFIO is a module for implementing the pass-through of a PCI device on a host computer, where the PCI device refers to an external device interconnected with the CPU of the host computer through a PCI (Peripheral Component Interconnect) bus or a PCIe (Peripheral Component Interconnect express) bus. In the computer device shown in FIG. 1b, the pass-through of the target resource object to the secure container is implemented by using the VFIO as an example. The computer device shown in FIG. 1b is different from the computer device shown in FIG. 1a in that, in addition to the KVM 121, the VFIO 122 and the drivers of the hardware resources on the host computer are included in the operating system 12 of the host computer; the VFIO 122 can directly deliver the target resource object (for example, a GPU) to the secure container for use, which is beneficial to improving the access efficiency of the secure container to the GPU; in addition, in the computer device shown in FIG. 1b, the application programs are taken as an AI framework and an AI application service developed and deployed based on the AI framework as an example for illustration. In the user space 1110 of the guest computer, a runtime library of the target resource object is also included, and in FIG. 1b, the GPU runtime library is taken as an example for illustration.

[0039] The AI framework can be PyTorch or Tensorflow, etc., where PyTorch is an open-source machine learning library for building and training deep learning models, and Tensorflow is an open-source machine learning framework for building and training various types of machine learning models. The AI application service can be an inference service of a large language model, a graphic image recognition service, a natural language processing service, a search advertisement recommendation service, a function calculation service, etc., which are not limited in the present embodiment.

[0040] In the case where the target resource object is implemented as a pass-through device of the secure container 111, the driver of the target resource object is located in the guest operating system 1112, and in the case where the target resource object is a GPU, the GPU driver is included in the guest operating system 1112. Correspondingly, for the non-target resource object which is not implemented as a pass-through device of the secure container 111, the driver thereof is located in the operating system 12 of the host computer, and of course the KVM and the VFIO and other virtualization-related functions are also included in the operating system 12 of the host computer.

[0041] Through the above architecture of the security container + pass-through device, the security container can effectively reduce the overhead of the host operating system when interacting with the target resource object (such as a GPU), thereby improving the I / O (Input / Output) performance. At the same time, the number of data copying between the host operating system and the security container can be reduced, thereby reducing the delay and improving the throughput.

[0042] The method for state migration of the security container will be described in detail below with reference to the interaction process between any two nodes under the above pass-through architecture. For ease of description, the source node and the target node will be used to describe any two nodes. The source node and the target node are not limited in the implementation form of the present disclosure. For example, in the host dimension, the source node can be implemented as a host that needs to perform state migration, and the target node can be implemented as another host. For example, in the GPU dimension, the source node can be implemented as a GPU on a host that needs to perform state migration, and the target node can be implemented as another GPU. The two GPUs can be located on the same host or different hosts, and are not limited in this regard.

[0043] FIG. 2a is a schematic diagram of an interaction process for state migration between a source node and a target node, which is used to migrate the workload in a first security container on the source node. The first security container refers to a security container on the source node that needs to perform state migration. The workload in the first security container mainly refers to the state data of the resource object involved in the first security container. The migration of the workload in the first security container mainly refers to the migration of the state of the resource object included in the first security container. The resource object involved in the first security container can include a target resource object managed by a dependency driver and a non-target resource object not managed by the dependency driver. The migration of the workload in the first security container includes the migration of the state data of the target resource object in the first security container and the migration of the state data of the non-target resource object in the first security container.

[0044] For example, taking the GPU as the target resource object, the workload in the first security container can include state data of the GPU (such as a list of video memories and corresponding video memory contents, other states in the GPU), state data of the CPU (such as CPU register states), state data of the storage and network (such as user temporary storage data, network addresses, disk states, and the like), state data of the memory (such as kernel data and user data of the security container), and metadata of the security container (such as management data of the security container, information of the container engine layer, and the like). Among them, the state data of the GPU belongs to the state data of the target resource object, and the state data of the CPU, the state data of the storage and network, the state data of the memory, and the metadata of the security container belong to the state data of the non-target resource object. Based on this, the state data of the GPU and the state data of the CPU, the state data of the storage and network, the state data of the memory, and the metadata of the security container can be migrated from the first security container to the second security container, thereby completing the migration process of the workload in the first security container.

[0045] As shown in FIG. 2a, the state migration between the source node and the target node can be implemented based on the following steps.

[0046] Step S1, during the running of the first security container, the target resource object of the first security container is suspended through the driver program of the target resource object.

[0047] Step S2, the state data of the target resource object of the first security container is obtained through the driver program of the target resource object.

[0048] Step S3, instructing to create a second security container on the target node; the second security container includes a target resource object managed by a dependent driver program and a non-target resource object not managed by a dependent driver program.

[0049] Step S4, migrating the state data of the target resource object of the first security container to the target resource object of the second security container.

[0050] Step S5, receiving the state data of the target resource object of the first security container, and saving the state data of the target resource object of the first security container in the target resource object of the second security container.

[0051] Step S6, suspending the first security container through the container runtime component corresponding to the first security container.

[0052] Step S7, obtaining the state data of the non-target resource object of the first security container through the container runtime component corresponding to the first security container.

[0053] Step S8, migrating the state data of the non-target resource object of the first secure container into the non-target resource object of the second secure container.

[0054] Step S9, receiving the state data of the non-target resource object of the first secure container, and saving the state data of the non-target resource object of the first secure container in the non-target resource object of the second secure container, to migrate the workload in the first secure container to the second secure container.

[0055] In the present embodiment, the target resource object refers to a resource object that cannot be directly managed or controlled by the cloud vendor or the secure container, but needs to rely on a driver for management / control. It is generally a hardware resource object that is directly connected to the secure container, such as a GPU, a DPU, or an NPU, etc. Correspondingly, the target resource object refers to a resource object that can be directly managed or controlled by the cloud vendor or the secure container, without the need for program control / management. It can be a hardware resource object such as a CPU, a memory, or a network card, etc. It can also be a virtual resource object that is virtualized by a hardware resource object through a virtualization technology, such as a virtual CPU, a virtual memory, or a virtual network card, etc. It can also be a software resource object, such as engine information and node information involved in the Pod metadata of the secure container, etc. The Pod is the smallest scheduling unit in Kubernetes, which can be understood as a specific secure container cluster. The secure container cluster can contain one or more secure containers. Kubernetes, commonly referred to as k8s, is an open-source container orchestration platform for automatically deploying, expanding, and operating containerized applications. The engine information generally refers to configuration information related to the running environment of the Pod, and the node information refers to metadata related to the node to which the Pod is scheduled.

[0056] In the embodiment, the state data of the target resource object is acquired and saved by the driver of the target resource object in the client operating system of the first secure container. The acquisition and saving of the state data of the target resource object by the driver of the target resource object depends on the normal operation of the first secure container, and the state data of the target resource object of the first secure container needs to be in a stable state, i.e., remains unchanged. Based on this, the operation of the target resource object of the first secure container can be suspended by the driver of the target resource object during the operation of the first secure container, and the state data of the target resource object of the first secure container is acquired when the target resource object is in a suspended state. In this way, the consistency of the state data of the target resource object can be maintained. The state data of the target resource object is used to describe the state of the target resource object. When the target resource object has a video memory, the state data can include a video memory list, video memory content, and other state data. The other state data refers to state data other than the video memory list and the corresponding video memory content, for example, can include at least one of register information, working state information, and configuration information in the target resource object. The video memory list refers to a list of video memory regions allocated to different application programs on the target resource object. The register information is used to describe the current state of the register, such as the value, ID (Identity document, identification code), or type to which the register belongs, and the like. The configuration information is used to describe various parameters and setting information of the target resource object, such as the enable / disable of the function of the display card, the performance setting, or the display mode, and the like. The working state information can be used to describe the working state of the target resource object, including at least one of the ID of the target resource object, the power state, the task allocation, and the like.

[0057] In this embodiment, the second secure container can be created on the target node to implement the migration of the workloads of the first secure container, but the timing of creating the second secure container is not limited. For example, the second secure container can be created on the target node after the state data of the target resource objects of the first secure container is obtained, and in the embodiment shown in FIG. 2a, this implementation is taken as an example for illustration, but is not limited thereto. For another example, the second secure container can also be created on the target node before the workloads in the first secure container are migrated, and the like, and the creation time of the second secure container is not limited, as long as the creation of the second secure container is completed before the workloads are migrated to the second secure container. The second secure container also needs to include the target resource objects managed by the dependency driver and the non-target resource objects not managed by the dependency driver. It is explained that the number and type of the target resource objects included in the second secure container need to be adapted to the number and type of the target resource objects included in the first secure container, that is, the types of the target resource objects are the same, and the number of the target resource objects included in the second secure container is greater than or equal to the number of the target resource objects that need to be migrated in the first secure container, so that the state data of the target resource objects in the first secure container can be successfully migrated to the target resource objects in the second secure container. The second secure container refers to the secure container created on the target node, and the use of “first” and “second” to limit the secure container is only used to distinguish the secure container on the source node and the secure container on the target node, and does not limit the configuration or type of the secure container. The embodiments of the present disclosure do not limit the specific implementation of creating the second secure container, and in some exemplary embodiments, the second secure container can be created by using Kata Containers and the like. The Kata Containers is a lightweight virtualization technology that combines the convenience of containers and the security isolation features of virtual machines, and can be used to create the second secure container. In addition, the second secure container can be created by the container runtime component on the source node instructing the target node, or can be created by the management node instructing the target node, and the foregoing is not limited. On the basis that the second secure container is created on the target node, the state data of the target resource objects of the first secure container can be migrated to the target resource objects of the second secure container.

[0058] In the foregoing manner, during the running of the first secure container, the creation of the secure container on the target node can be used to further migrate the state data of the target resource objects. The migration process of the state data of the non-target resource objects will be described below.

[0059] In view of the fact that the management of the non-target resource object by the secure container can not rely on the driver of the non-target resource object, but can be managed by the container runtime component, therefore, unlike the foregoing, the acquisition and storage of the state data of the non-target resource object in the first secure container can be performed by the container runtime component. The acquisition and storage of the state data of the non-target resource object in the first secure container by the container runtime component requires that the state data of the non-target resource object in the first secure container be in a stable state, i.e., remain unchanged. Based on this, the first secure container can be suspended by the container runtime component. Here, the suspension of the first secure container mainly refers to the suspension of the operation of the client operating system in the first secure container. The container runtime component can first acquire the identification information such as the ID or name of the first secure container, and then suspend the operation of the first secure container according to the identification information and the container suspension command provided by the container runtime component.

[0060] After the operation of the first secure container is suspended, the state data of the non-target resource object in the first secure container can be acquired by the container runtime component. The state data of the non-target resource object is used to describe the working state of the non-target resource object, and can include at least one of the currently stored data, the storage space information, the running space information, the internal register information, and the configuration information of the non-target resource object. The register information is used to describe the current state of the register, such as the value, ID, or type of the register. The configuration information is used to describe various parameters and setting information of the non-target resource object, such as the function switch, performance setting, or working mode of the CPU. After the state data of the non-target resource object in the first secure container is acquired, the state data of the non-target resource object in the first secure container can be migrated to the non-target resource object in the second secure container. It is noted that the number of non-target resource objects contained in the second secure container is greater than or equal to the number of non-target resource objects in the first secure container that need to be migrated, so as to successfully migrate the state data of the non-target resource object in the first secure container to the non-target resource object in the second secure container.

[0061] In the embodiment, in the pass-through architecture between the target resource object and the secure container, the target resource object of the first secure container on the source node is managed by the target resource object dependent driver, and the non-target resource object is not managed by the driver. According to different characteristics of the target resource object and the non-target resource object, different levels of components can be cooperated with each other, that is, the state data of the target resource object and the non-target resource object in the first secure container can be obtained by the driver of the target resource object and the container runtime component, and then the state data of the target resource object and the non-target resource object can be migrated to the second secure container on the target node. In this way, the fast migration of the workloads in the secure container can be better completed by the cooperation of different levels of components, and the state migration of the secure container with the target resource object which cannot be directly managed by the secure container or the cloud vendor to which the secure container belongs can be solved.

[0062] The secure container migration method in the embodiment of the present disclosure can be applied to various scenarios. For example, in the scenarios of application upgrade, resource optimization or application failure, the workloads of the secure container of the source node can be efficiently migrated to the secure container of the target node by the above method. Specifically, in the application upgrade scenario, smooth transition between new and old versions is very important. In the resource optimization scenario, by migrating the workloads, more uniform load distribution can be achieved, the formation of hot nodes can be avoided, and the stability and response speed of the overall system can be improved. In the application failure scenario, when a node fails, the service can be restored by migrating the workloads to other nodes that have not failed to ensure data security.

[0063] In some optional embodiments, the first secure container includes a driver of the target resource object in a client operating system, and the driver of the target resource object is used to interact with the target resource object.

[0064] In the first secure container, an application in the client operating system can run in a user space. In other words, the user space refers to an environment in which the application in the secure container runs, and can be understood as a place where the application in the client operating system runs. In the embodiment, the user space of the first secure container can include a runtime library of the target resource object. The runtime library of the target resource object is built on the basis of the driver of the target resource object, and encapsulates the functions provided by the driver. The runtime library of the target resource object can be used as a bridge for the application to interact with the driver of the target resource object, that is, the application can initiate various requests to the driver of the target resource object through the runtime library of the target resource object. When the application initiates a request through the runtime library of the target resource object, the request can be converted into a call to the driver of the target resource object.

[0065] Taking the GPU as an example, the runtime library of the GPU can be CUDA (Compute Unified Device Architecture). The CUDA can encapsulate the functions provided by the GPU. When the application sends a request through the CUDA, the sent request can be converted into a call to the driver program of the GPU. That is, the application can use the CUDA to call the driver program of the GPU to interact with the GPU.

[0066] On this basis, the step S1 in the foregoing embodiment, that is, “suspending the running of the target resource object of the first security container through the driver program of the target resource object”, can be implemented based on the following implementation manner one or implementation manner two.

[0067] Implementation manner one: intercepting an access request initiated by an application in the first security container by using an interception function in the runtime library, to suspend the running of the target resource object of the first security container. The access request is initiated by the application to the driver program of the target resource object through a library function in the runtime library.

[0068] The present embodiment does not limit the acquisition manner of the interception function. In some exemplary embodiments, a function configuration interface can be provided to the user, and the interception function input by the user is acquired in response to a configuration operation of the user on the function configuration interface. Alternatively, a preset interception function can be acquired from the storage space corresponding to the first security container in a preset manner, and the present embodiment does not limit the same.

[0069] The present embodiment can add the interception function in the runtime library in advance, but does not limit the specific implementation of adding the interception function. Alternatively, the hook technology can be used to add the interception function in the code of the runtime library to intercept the function that needs to be intercepted in the runtime library. The hook technology is used to intercept the behavior of the function without modifying the original code.

[0070] Specifically, the target function that needs to be intercepted in the library function in the runtime library can be selected first, where the library function in the runtime library refers to the function set provided in the runtime library. After the target function is selected, a hook method, such as an LD_PRELOAD method or a function replacement method, can be selected, and the selected hook method is used to add the acquired interception function to the code of the runtime library. The LD_PRELOAD method can load a custom dynamic link library by setting the LD_PRELOAD environment variable, and the dynamic link library contains the interception function that replaces the original library function. The function replacement method can replace the original library function with the interception function by modifying the pointer of the library function.

[0071] The application program in the first secure container can initiate an access request to the driver of the target resource object through a library function in the runtime library. Through an interception function in the runtime library, the above-mentioned driver access behavior of the application program in the first secure container can be effectively intercepted, so that the driver of the target resource object cannot be triggered by the application program in the first secure container, and the purpose of suspending the running of the target resource object of the first secure container is achieved.

[0072] In this way, on the one hand, the purpose of suspending the running of the target resource object of the first secure container can be achieved without relying on the driver, by adding an interception function to the runtime library in the prelinking link. Since the interception function can be written according to actual design requirements, the controllability of this way is relatively high.

[0073] In the second implementation, the driver of the target resource object is called to send a suspension command to the target resource object of the first secure container to suspend the running of the target resource object of the first secure container. Specifically, the identification information of the target resource object can be obtained through the driver, and the identification information is used to identify the identity of the target resource object. Then, the current state of the target resource object can be queried through the driver according to the identification information to ensure that it can be suspended. The suspension command is sent to the target resource object through the driver, and the process of sending the suspension command can be implemented as a function call process, such as calling PauseDevice() or similar operations. PauseDevice() is a function for suspending or stopping the running of a device. In this way, the running of the target resource object can be suspended relatively quickly through the calling of the driver.

[0074] Through the above-mentioned first implementation and the second implementation, the running of the target resource object of the first secure container can be suspended relatively efficiently by adding an interception function in the runtime library or by calling the driver.

[0075] After the running of the target resource object of the first secure container is suspended through the driver of the target resource object, step S2 can be continued to obtain the state data of the target resource object of the first secure container.

[0076] The disclosure does not limit the way of obtaining the state data of the target resource object of the first secure container, and in some exemplary embodiments, the following steps S221-S222 can be implemented.

[0077] In step S221, a target application program interface in a driver of the target resource object is invoked to obtain a memory list of the target resource object of the first security container and corresponding memory content thereof. The target application program interface refers to an interface for managing memory usage. The memory list refers to a list of memory regions allocated to different application programs on the target resource object, and can include information of a memory segment used by each application program, such as a start address, size, type, and other related information of the memory segment. The start address of the memory segment refers to a start physical address or a virtual address of the memory segment; the type of the memory segment can be VRAM (a type of memory segment) or WRAM (a type of memory segment), and the like. The memory segment is used to store memory content, which can be different for different application programs running in the first security container. In the case of running the AI framework and the AI application in the first security container, the memory content can be various data related to the machine learning model, and the like. The other related information can include the state of the memory segment, such as idle, in use, and the like, and can also include the priority of the memory segment, which is not limited in the embodiment.

[0078] Optionally, before obtaining the memory list of the target resource object of the first security container, the memory list of the target resource object can also be determined based on the following manner: first, a proxy function can be added in the runtime library to proxy an access request initiated by an application program in the first security container to the driver of the target resource object through a library function in the runtime library. The proxy function can be added in the runtime library based on the hook technology, and the specific implementation manner of this content has been described in detail in the foregoing “add an interception function in the runtime library”. In this way, when the application program in the first security container initiates an access request to the driver of the target resource object, the access request can not be initiated directly through the library function in the runtime library, but the proxy function initiates the access request. The proxy function can determine the request type of the access request, which can include a memory allocation request, an initialization request, and a mode setting request, and the like, wherein the memory allocation request can contain a memory list applied by the application program to the target resource object; in the case of the access request being the memory allocation request, the access request can be initiated and the memory list applied by the application program can be read therefrom as the memory list of the target resource object of the first security container.

[0079] Step S222: Call other application interfaces in the driver of the target resource object besides the target application interface to obtain other state data of the target resource object of the first security container. Other state data refers to state data other than the video memory list and its corresponding video memory content. There may be one or more other application interfaces, depending on the type and quantity of other state data. One application interface can be used to obtain one or more types of other state data; this embodiment does not impose any limitations. Other state data may include: internal register information, working status information, and configuration information.

[0080] The following example uses other application programming interfaces (APIs) implemented using the NVML library, a library for monitoring and managing target resource objects. Specifically, the name of the target resource object can be obtained by calling the API implemented based on the nvmlDeviceGetName() function in the driver of the target resource object; alternatively, the runtime space utilization rate of the target resource object can be obtained by calling the API implemented based on the nvmlDeviceGetUtilizationRates() function in the driver of the target resource object. This embodiment does not impose any limitations on this.

[0081] In this way, the state data of the target resource object in the first security container can be obtained relatively accurately by making interface calls to the application programming interface of the driver of the target resource object. Then, step S3 can be executed to create a second security container on the target node, and step S4 can be executed to migrate the state data of the target resource object in the first security container to the target resource object in the second security container.

[0082] The embodiments of the present disclosure do not limit the specific implementation of the migration manner of the state data of step S4. In some exemplary embodiments, the non-target resource object of the first secure container can include a normal network card, based on which the state data of the target resource object of the first secure container can be copied into the target resource object of the second secure container based on TCP / IP (Transmission Control Protocol / Internet Protocol). In some exemplary embodiments, the non-target resource object of the first secure container can include an RDMA network card, based on which an RDMA operation can be issued through the RDMA network card, and the state data of the target resource object of the first secure container can be copied into the target resource object of the second secure container through the RDMA operation. The RDMA network card can support the RDMA technology for directly accessing the memory between different nodes without the intervention of the CPU, thereby improving the data transmission efficiency and reducing the transmission delay. Therefore, based on the RDMA operation issued by the RDMA network card, the state data of the target resource object of the first secure container can be more efficiently migrated to the target resource object of the second secure container. Further, in the case where the target resource object is implemented as a GPU, the process of migrating the state data of the target resource object through the RDMA network card can be referred to as GDR (GPU Direct RDMA), that is, the state data is directly transmitted between the GPUs of the first secure container and the second secure container through the RDMA network without the memory copying mode of the CPU, which can realize the high-speed transmission of the GPU state data.

[0083] Correspondingly, one embodiment of step S4 for migrating the state data of the target resource object of the first secure container into the target resource object of the second secure container includes: instructing to create a first virtual machine in the second secure container and bind the target resource object of the second secure container to the first virtual machine, and migrating the state data of the target resource object of the first secure container into the target resource object bound to the first virtual machine; and instructing to create a second virtual machine in the second secure container and bind the non-target resource object of the second secure container to the second virtual machine, and bind the target resource object bound to the first virtual machine to the second virtual machine, so as to migrate the state data of the target resource object of the first secure container into the target resource object of the second secure container. Accordingly, step S5 receives the state data of the target resource object of the first secure container and saves the state data of the target resource object of the first secure container in the target resource object of the second secure container. Specifically, step S5 can be performed based on steps S51-S52.

[0084] Step 51, according to the indication information, create the first virtual machine in the second security container, bind the target resource object of the second security container with the first virtual machine, and migrate the state data of the target resource object of the first security container to the target resource object bound with the first virtual machine. The indication information comes from the container runtime component on the source node or the management node, and is used to instruct to create the first virtual machine in the second security container, bind the target resource object of the second security container with the first virtual machine, and migrate the state data of the target resource object of the first security container to the target resource object bound with the first virtual machine.

[0085] Wherein, the first virtual machine can be created by using Kata Containers, QEMU (Quick EMUlator) and the like, which are not limited in the embodiment. QEMU is an open source virtual machine monitor and emulator, which allows multiple virtual machines to run on one host machine at the same time, and provides management and control capabilities for these virtual machines. After creating the first virtual machine, a mapping relationship between the target resource object of the second security container and the first virtual machine can be established, and then the state data of the target resource object of the first security container can be sent to the target resource object bound with the first virtual machine according to the mapping relationship. In this way, the target resource object bound with the first virtual machine in the second security container can have the state data of the target resource object of the first security container.

[0086] Step 52, according to the indication information, create the second virtual machine in the second security container, bind the non-target resource object of the second security container with the second virtual machine, and bind the target resource object bound with the first virtual machine with the second virtual machine, so as to migrate the state data of the target resource object of the first security container to the target resource object of the second security container. The indication information comes from the container runtime component on the source node or the management node, and is used to instruct to create the second virtual machine in the second security container, bind the non-target resource object of the second security container with the second virtual machine, and bind the target resource object bound with the first virtual machine with the second virtual machine, so as to migrate the state data of the target resource object of the first security container to the target resource object of the second security container.

[0087] The second virtual machine can be created by using a virtualization tool such as Kata Containers, QEMU, or Hypervisor, which is not limited in the embodiment. After the second virtual machine is created, a mapping relationship between the non-target resource object of the second secure container and the second virtual machine can be established. Further, on the basis of the mapping relationship, the state data of the non-target resource object of the first secure container can be migrated to the non-target resource object bound to the second virtual machine. In addition, considering that the first virtual machine and the second virtual machine are both running in the second secure container, the target resource object bound to the first virtual machine can be bound to the second virtual machine, and the first virtual machine can be destroyed or paused to run the second virtual machine in the second secure container. Since the target resource object in the second secure container is bound to the second virtual machine, it means that the state data of the target resource object of the first secure container is migrated to the target resource object bound to the second virtual machine in the second secure container.

[0088] In this way, on the one hand, the target resource object and the non-target resource object of the second secure container can both be mapped to the second virtual machine, thereby ensuring the consistency of resources between the second virtual machine and the second secure container. That is, the second virtual machine can not only access the target resource object of the second secure container, but also access the non-target resource object of the second secure container. On the other hand, by establishing and destroying the virtual machine, it can be ensured that only one virtual machine runs in the second secure container, thereby improving the isolation and security of the virtual machine and the correctness of the data.

[0089] In addition to the above embodiment, the embodiment of the present disclosure also provides an implementation manner of saving the state data of the target resource object of the first secure container in the target resource object of the second secure container, which can be specifically referred to steps S23, S24, and S30 in FIG. 2b. The secure container migration method shown in FIG. 2b includes the following steps.

[0090] Step S21, during the running of the first secure container, the target resource object of the first secure container is paused by the driver program of the target resource object.

[0091] Step S22, the state data of the target resource object of the first secure container is obtained by the driver program of the target resource object.

[0092] Step S23, instructing to create a third secure container on a target node; the third secure container includes a target resource object managed by a dependent driver program.

[0093] Step S24, migrating the state data of the target resource object of the first secure container to the target resource object of the third secure container.

[0094] Step S25, receiving the state data of the target resource object of the first security container, and saving the state data of the target resource object of the first security container in the target resource object of the third security container.

[0095] Step S26, suspending the running of the first security container through the container runtime component corresponding to the first security container.

[0096] Step S27, obtaining the state data of the non-target resource object of the first security container through the container runtime component corresponding to the first security container.

[0097] Step S28, instructing to create a second security container on the target node; the second security container includes the non-target resource object which is not managed by the driver.

[0098] Step S29, migrating the state data of the non-target resource object of the first security container to the non-target resource object of the second security container.

[0099] Step S30, transferring the target resource object of the third security container to the second security container to migrate the workload in the first security container to the second security container.

[0100] Wherein, for steps S21-S22, S26-S27, S29, etc., please refer to the description of the corresponding steps in the embodiment shown in FIG. 2a, which will not be repeated here.

[0101] In this embodiment, two security containers are created on the target node, which are the third security container and the second security container respectively, the third security container includes the target resource object, and the second security container includes the non-target resource object; first, the state data of the target resource object of the first security container is saved in the target resource object of the third security container, and the state data of the non-target resource object of the first security container is migrated to the non-target resource object of the second security container; then, the target resource object of the third security container is transferred to the second security container to migrate the workload in the first security container to the second security container. Wherein, transferring the target resource object of the third security container to the second security container means remounting the target resource object mounted on the third security container to the second security container, and releasing the mounting relationship between the target resource object and the third security container.

[0102] In addition, in the embodiments of the present disclosure, the order and time of creating the second secure container and the third secure container are not limited, and the second secure container and the third secure container can be created in advance or in real time when state data migration is needed. The creation of the second secure container and the third secure container is not limited as long as the creation of the secure container is completed before the corresponding secure container is used. Similarly, the third secure container can be created by the container runtime component on the source node or by the management node, and the creation of the third secure container is not limited. Regardless of the object that indicates the creation of the second secure container and the third secure container, the process of creating the second secure container and the third secure container and the process of mounting the related resource object for the second or third secure container are completed by the container runtime component on the target node. The specific processes are not limited in the embodiments and are not focused on. For reference, some related technical implementations can be referred to, and a self-developed scheme can be used.

[0103] In this way, the state data of the target resource object of the first secure container can be saved to the target resource object of the second secure container through the new secure container created on the target node.

[0104] Further optionally, as shown in FIG. 2b, after step S30, the following steps can be further included.

[0105] Step S31, stopping the first secure container from running; specifically, the container runtime component corresponding to the first secure container on the source node can be used to stop the first secure container from running. Here, stopping means closing the first secure container.

[0106] Step S32, stopping the third secure container from running; specifically, the container runtime component corresponding to the third secure container on the target node can be used to stop the third secure container from running. Here, stopping means closing the third secure container.

[0107] Step S33, resuming the second secure container from running by the container runtime component corresponding to the second secure container in the target node, and resuming the work of the target resource object by the driver of the target resource object. The driver of the target resource object is located in the guest operating system of the second secure container. Specifically, the container runtime component can call the driver of the target resource object to resume the work of the target resource object by the driver of the target resource object. The work of the target resource object can be resumed by referring to the description in steps S11 and S12 below, which is not described in detail here.

[0108] After the state data of the target resource object of the first security container is migrated to the target resource object of the second security container based on the various embodiments above, step S6 can be performed to suspend the running of the first security container by the container runtime component. By suspending the running of the first security container, the state data of the non-target resource object of the first security container can be kept unchanged; in other words, suspending the running of the first security container is intended to prevent the state data of the non-target resource object from changing when step S7 is performed. The related content about the state data has been described in detail in the foregoing, and will not be described here again.

[0109] The embodiments of the present disclosure do not limit the specific implementation of step S7 "obtaining the state data of the non-target resource object of the first security container by the container runtime component corresponding to the first security container". In some exemplary embodiments, the container runtime component can obtain and save the state data of the non-target resource object of the first security container based on the state saving tool in the user space of the first security container. Alternatively, in other exemplary embodiments, the user can manually save and restore the state data of the non-target resource object of the first security container, and the container runtime component can obtain and save the state data of the non-target resource object of the first security container in response to the saving operation of the user.

[0110] The state saving tool of the first security container is a security container persistence tool and is implemented based on checkpoint-restore technology. The checkpoint-restore technology is a technology for instant state saving and restoration of a container. This technology enables the container to save the state data of its runtime (referred to as checkpoint) at one location and restore the state data (referred to as restore) at another location. In short, the state saving tool can be used to save the runtime state of the security container and can quickly restore to the runtime state in subsequent processes. The embodiments of the present disclosure do not limit the specific implementation of the state saving tool. In some exemplary embodiments, the state saving tool can be implemented as a CRIU (Checkpoint / Restore In Userspace) tool. The CRIU tool is a tool for implementing process state snapshot and restoration, which can be used to save the state data of the resource object in the user space and restore the state data again when needed.

[0111] Based on the foregoing, the container runtime component can invoke the state saving tool, and the state saving tool can save the state data of the non-target resource object of the first secure container to the memory of the source node. Specifically, the container runtime component can invoke the state saving tool by sending an invocation instruction to the state saving tool, the state saving tool can perform a state snapshot on the non-target resource object of the first secure container to obtain corresponding state data, and save the state data to the memory of the source node.

[0112] In this way, the state data of the non-target resource object of the first secure container can be efficiently obtained based on the state saving tool.

[0113] After obtaining the state data of the non-target resource object of the first secure container through the container runtime component, the state data of the non-target resource object of the first secure container can be migrated to the non-target resource object of the second secure container in step S8 or step S28. In some exemplary embodiments, the non-target resource object of the first secure container can include a normal network card, based on which the state data of the non-target resource object of the first secure container can be copied to the non-target resource object of the second secure container based on TCP / IP through the normal network card. Alternatively, in the case where the non-target resource objects of the first secure container and the second secure container include RDMA network cards, the state data of the non-target resource object of the first secure container can be migrated to the non-target resource object of the second secure container through RDMA operation. Since RDMA operation can directly access memory between different nodes without the intervention of CPU, it is beneficial to improve data transmission efficiency and reduce transmission delay, so that the state data of the non-target resource object of the first secure container can be efficiently and quickly migrated to the non-target resource object of the second secure container.

[0114] In some optional embodiments, after the target node receives the state data of the target resource object and the non-target resource object of the first secure container and saves the state data to the second secure container, i.e., after completing the migration of the work load of the first secure container, the second secure container can further restore the working state of the second secure container based on the state data received by the second secure container.

[0115] The embodiments of the present disclosure do not limit the specific implementation of the restoration of the working state of the second secure container, and in some exemplary embodiments, the working state of the second secure container can be restored based on steps S10-S13.

[0116] Step S10, running the second secure container created in the foregoing step S3 or step S27.

[0117] Step S11: Restore the working state of the non-target resource object based on the state data in the non-target resource object of the second security container.

[0118] Step S12: Restore the working state of the target resource object based on the state data in the target resource object of the second security container.

[0119] In some optional embodiments, the user space of the second secure container may include a state recovery tool, which corresponds to the state saving tool in the user space and is also implemented based on checkpoint-restore technology. Based on the above, the container runtime component can invoke the state recovery tool by sending an invocation command to it. The state recovery tool can provide the state data of the non-target resource object in the second secure container to the non-target resource object, allowing the non-target resource object to restore its working state based on this state data. Alternatively, in other exemplary embodiments, the user can manually restore the state data of the non-target resource object in the second secure container. The container runtime component can respond to the user's restoration operation by adjusting the working state of the non-target resource object based on its state data.

[0120] In some optional embodiments, the second security container may include a driver for the target resource object. The state data of the target resource object may include a video memory list and its corresponding video memory content, as well as other state data, which will not be elaborated here. Based on this, the target application programming interface (API) in the driver of the target resource object can be invoked to provide the video memory list and its corresponding video memory content from the target resource object of the second security container to the target resource object of the second security container. Here, the target application programming interface refers to an interface used to manage video memory usage. The relevant content regarding the video memory list has been described in detail above and will not be repeated here.

[0121] Subsequently, other application programming interfaces (APIs) in the driver of the target resource object can be invoked to provide other state data from the target resource object in the second security container to the target resource object of the second security container, thereby restoring the working state of the target resource object. There can be one or more APIs, depending on the number and types of other state data. One API can be used to obtain one or more other state data types; this embodiment does not impose any limitations.

[0122] In this way, the target node can efficiently restore the working state of the second security container based on the received state data after completing the state data migration.

[0123] On the basis of the various embodiments above, in addition to the source node and the target node, a management and control node can also be provided in the embodiments of the present disclosure to manage and control the source node and the target node. The management and control node can issue a container migration task to the source node, and the container migration task is used to instruct the migration of the workload of the first secure container on the source node to the second secure container on the target node. Alternatively, the management and control node can issue the container migration task to the source node in response to a task issuance event. The task issuance event can be implemented by detecting that the source node has a fault, or by detecting that the first secure container on the source node has a fault, or by detecting that the workload of the first secure container on the source node exceeds a preset range. The preset range can be set to any range according to actual design requirements, and the present embodiment does not make any limitation.

[0124] In the present embodiment, the management and control node can timely issue a container migration task to the source node, and the container migration task is used to instruct the migration of the workload of the first secure container on the source node to the second secure container on the target node.

[0125] Based on the above embodiments, the secure container migration method provided in the embodiments of the present disclosure can be applied to an intelligent computing service scenario. The intelligent computing service can be deployed in the user space of the first secure container as the application in the above embodiments. Among them, the user can deploy the first secure container carrying the intelligent computing service on the source node through the management and control node, and pass the GPU on the source node to the first secure container as the computing power resource of the intelligent computing service. In this scenario, the intelligent computing service in the first secure container runs on the GPU. During the running of the intelligent computing service, the following situations may occur: the user needs to upgrade the intelligent computing service, or the intelligent computing service fails, or the source node fails. In these situations, the intelligent computing service needs to be migrated, that is, the workload of the first secure container needs to be migrated. At this time, the management and control node can determine the target node to which the migration is needed, and issue a state migration instruction to the source node to instruct the migration of the first secure container on the source node to the second secure container on the target node. Alternatively, the management and control node can notify the target node to create the second secure container, or the source node can also notify the target node to create the second secure container; the second secure container can be created in advance or in real time in the case of needing to migrate the state data. The source node executes the migration steps described in the above method embodiments according to the state migration instruction, and migrates the intelligent computing service from the first secure container on the source node to the second secure container on the target node. Then, the states of the GPU, CPU, network, memory, etc. in the second secure container are recovered according to the various state data migrated, so as to ensure that the states of the GPU, CPU, storage and network, etc. before and after the migration are consistent, and the running of the intelligent computing service in the second secure container is restored, so that the intelligent computing service can continue to provide services for customers.

[0126] The intelligent computing service scenario is only one application scenario of the secure container migration method provided by the embodiments of the present disclosure, and is not limited thereto. For example, the secure container migration method provided by the embodiments of the present disclosure can also be applied to a function computing (Function Compute) service scenario. The function computing service can be deployed as an application in the user space of the first secure container in the above embodiments. The function computing provides a way to execute code on demand. A user only needs to write and upload function code, and does not need to care about the maintenance and expansion of the underlying infrastructure. The migration process of the secure container in this function computing application scenario is the same as that in the above embodiments, and will not be described again.

[0127] For another example, the secure container migration method provided by the embodiments of the present disclosure can also be applied to a PAI (Platform for AI, artificial intelligence platform service) scenario. The artificial intelligence platform service can be deployed as an application in the user space of the first secure container in the above embodiments. The artificial intelligence platform service is used for a target user to build, train, deploy and manage artificial intelligence and machine learning models. The migration process of the secure container in this PAI application scenario is the same as that in the above embodiments, and will not be described again.

[0128] Based on the above embodiments, the embodiments of the present disclosure further provide a secure container migration method for migrating a workload in a first secure container on a source node. The first secure container includes a target resource object and a non-target resource object, and the target resource object is a pass-through device of the first secure container. As shown in FIG. 3, the method includes the following steps.

[0129] Step 31, during the running of the first secure container, suspending the running of the target resource object of the first secure container through a driver of the target resource object, and obtaining state data of the target resource object of the first secure container.

[0130] Step 32, migrating the state data of the target resource object of the first secure container to a target resource object of a second secure container, the second secure container being a secure container on a target node.

[0131] Step 33, suspending the running of the first secure container through a container runtime component corresponding to the first secure container, and obtaining state data of the non-target resource object of the first secure container.

[0132] Step 34, migrating the state data of the non-target resource object of the first secure container to a non-target resource object of a second secure container, so as to migrate the workload in the first secure container to the second secure container.

[0133] Further optionally, the first secure container comprises a client operating system, the target resource object comprises a driver in the client operating system, and the first secure container comprises a runtime library in a user space of the first secure container; and the suspending of the target resource object of the first secure container comprises: intercepting, by an intercept function in the runtime library, an access request initiated by an application in the first secure container to suspend the target resource object of the first secure container; the access request is initiated by the application in the first secure container through a library function in the runtime library to the driver of the target resource object; or, sending a suspension command to the target resource object of the first secure container by calling the driver of the target resource object to suspend the target resource object of the first secure container.

[0134] Further optionally, the obtaining of the state data of the target resource object of the first secure container by the driver of the target resource object comprises: calling a target application programming interface in the driver of the target resource object to obtain a video memory list and corresponding video memory content of the target resource object of the first secure container; and calling other application programming interfaces in the driver of the target resource object other than the target application programming interface to obtain other state data of the target resource object of the first secure container, the other state data being state data other than the video memory list and the corresponding video memory content.

[0135] Further optionally, the first secure container comprises a runtime library in a user space of the first secure container; and the method further comprises: proxying, by a proxy function in the runtime library, an access request initiated by an application in the first secure container, and in a case where the access request is a memory allocation request, obtaining a video memory list applied for by the application as a video memory list of the target resource object of the first secure container; the access request is initiated by the application in the first secure container through a library function in the runtime library to the driver of the target resource object.

[0136] Further optionally, the first secure container comprises a state saving tool in a user space of the first secure container; and the suspending of the first secure container by the container runtime component corresponding to the first secure container and the obtaining of the state data of the non-target resource object of the first secure container comprises: suspending, by the container runtime component, the first secure container to keep the state data of the non-target resource object of the first secure container unchanged; and calling, by the container runtime component, the state saving tool to obtain the state data of the non-target resource object of the first secure container and save the state data into a memory of the source node.

[0137] Further optionally, the migrating the state data of the target resource object of the first secure container into the target resource object of the second secure container on the target node comprises: creating a third secure container on the target node, migrating the state data of the target resource object of the first secure container into the target resource object of the third secure container; creating the second secure container on the target node, transferring the target resource object of the third secure container to the second secure container to migrate the state data of the target resource object of the first secure container into the target resource object of the second secure container; or creating a first virtual machine in the second secure container, binding the target resource object of the second secure container with the first virtual machine, migrating the state data of the target resource object of the first secure container into the target resource object bound with the first virtual machine; creating a second virtual machine in the second secure container, binding the non-target resource object of the second secure container with the second virtual machine, and binding the target resource object bound with the first virtual machine with the second virtual machine to migrate the state data of the target resource object of the first secure container into the target resource object of the second secure container.

[0138] Further optionally, the non-target resource object of the first secure container comprises an RDMA network card; the migrating the state data of the target resource object of the first secure container into the target resource object of the second secure container comprises: copying the state data of the target resource object of the first secure container into the target resource object of the second secure container through an RDMA operation; and the migrating the state data of the non-target resource object of the first secure container into the non-target resource object of the second secure container comprises: copying the state data of the non-target resource object of the first secure container into the non-target resource object of the second secure container through an RDMA operation.

[0139] Further optionally, the method further comprises: receiving a container migration task issued by the management node, the container migration task being used to instruct to migrate the work load of the first secure container on the source node into the second secure container on the target node.

[0140] The detailed implementation and beneficial effects of each step in the method of the embodiment have been described in the foregoing embodiments, and will not be described in detail here.

[0141] With the above embodiments, the present disclosure further provides a secure container migration method for migrating the workloads of a first secure container on a source node, a second secure container being created on a target node, the second secure container including target resource objects managed by a dependency driver and non-target resource objects not managed by the dependency driver; as shown in FIG. 4, the method includes: step 42, receiving state data of the target resource objects of the first secure container, and saving the state data of the target resource objects of the first secure container in the target resource objects of the second secure container; step 43, receiving state data of the non-target resource objects of the first secure container, and saving the state data of the non-target resource objects of the first secure container in the non-target resource objects of the second secure container.

[0142] Further optionally, as shown in FIG. 4, before step 42, the method further includes: step 41, creating the second secure container on the target node, the second secure container including the target resource objects managed by the dependency driver and the non-target resource objects not managed by the dependency driver. It is explained herein that the present embodiment does not limit the timing of creating the second secure container, and the example shown in FIG. 4 is merely provided.

[0143] Further optionally, the method further includes: running the second secure container; restoring the working state of the non-target resource objects according to the state data in the non-target resource objects of the second secure container; and restoring the working state of the target resource objects according to the state data in the target resource objects of the second secure container.

[0144] Further optionally, the user space of the second secure container includes: a state restoration tool for restoring the working state of the non-target resource objects according to the state data in the non-target resource objects of the second secure container, including: calling the state restoration tool to provide the state data in the non-target resource objects of the second secure container to the non-target resource objects, so as to restore the working state of the non-target resource objects.

[0145] Further optionally, the second secure container includes a driver of the target resource objects, the state data of the target resource objects including a list of display memories and corresponding display memory contents and other state data; and restoring the working state of the target resource objects according to the state data in the target resource objects of the second secure container, including: calling a target application programming interface in the driver of the target resource objects to provide the list of display memories and the corresponding display memory contents in the target resource objects of the second secure container to the target resource objects of the second secure container; and calling other application programming interfaces in the driver of the target resource objects other than the target application programming interface to provide the other state data in the target resource objects of the second secure container to the target resource objects of the second secure container, so as to restore the working state of the target resource objects, the other state data being state data other than the list of display memories and the corresponding display memory contents.

[0146] Further optionally, the saving the state data of the target resource object of the first security container in the target resource object of the second security container comprises: migrating the state data of the target resource object of the first security container into a target resource object of a third security container, the third security container being a security container on the target node; in a case where the second security container is created, transferring the target resource object of the third security container to the second security container to migrate the state data of the target resource object of the first security container into the target resource object of the second security container.

[0147] Further optionally, before the creating the second security container on the target node, the method further comprises: creating a third security container on the target node.

[0148] Further optionally, the saving the state data of the target resource object of the first security container in the target resource object of the second security container comprises: creating a first virtual machine in the second security container, binding the target resource object of the second security container with the first virtual machine, migrating the state data of the target resource object of the first security container into the target resource object bound with the first virtual machine; creating a second virtual machine in the second security container, binding a non-target resource object of the second security container with the second virtual machine, and binding the target resource object bound with the first virtual machine with the second virtual machine to migrate the state data of the target resource object of the first security container into the target resource object of the second security container.

[0149] The detailed implementation and beneficial effects of each step in the method of the embodiment have been described in the foregoing embodiments, and will not be described in detail here.

[0150] It should be noted that the execution subject of each step of the method provided in the above embodiments can be the same device, or the method can also be executed by different devices as the execution subject. For example, the execution subject of steps 31 to 34 can be the first device; for another example, the execution subject of steps 31 and 32 can be the first device, and the execution subject of steps 33 and 34 can be the second device; and the like.

[0151] In addition, in some of the processes described in the above embodiments and the accompanying drawings, a plurality of operations are included in a specific order, but it should be clear that these operations can be executed in the order in which they appear in this document or in parallel, and the serial numbers of the operations such as 31, 32, etc. are only used to distinguish different operations, and the serial numbers themselves do not represent any execution order. In addition, these processes can include more or fewer operations, and the operations can be executed in sequence or in parallel. It should be noted that the "first", "second" and the like described herein are used to distinguish different messages, devices, modules, etc., and do not represent the order of precedence, nor do "first" and "second" represent different types.

[0152] FIG. 5 is a structural schematic diagram of a security container migration apparatus provided by another exemplary embodiment of the present disclosure, for migrating a workload in a first security container on a source node, the first security container including target resource objects and non-target resource objects, the target resource objects being pass-through devices of the first security container. As shown in FIG. 5, the apparatus includes: a first suspension module 501, configured to suspend running of target resource objects of the first security container through a driver of the target resource objects during running of the first security container, and acquire state data of the target resource objects of the first security container; a first migration module 502, configured to create a second security container on a target node, and migrate the state data of the target resource objects of the first security container to target resource objects of the second security container; a second suspension module 503, configured to suspend running of the first security container through a container runtime component corresponding to the first security container, and acquire state data of non-target resource objects of the first security container; and a second migration module 504, configured to migrate the state data of the non-target resource objects of the first security container to non-target resource objects of the second security container, so as to migrate the workload in the first security container to the second security container.

[0153] Further optionally, the first security container includes a guest operating system, the guest operating system including a driver of the target resource objects, and a runtime library of the target resource objects in a user space of the first security container, and the first suspension module 501 suspending running of the target resource objects of the first security container through the driver of the target resource objects is specifically configured to: intercept an access request initiated by an application program in the first security container through an intercept function in the runtime library, so as to suspend running of the target resource objects of the first security container; or call the driver of the target resource objects to send a suspension command to the target resource objects of the first security container, so as to suspend running of the target resource objects of the first security container.

[0154] Further optionally, when the first suspension module 501 acquires the state data of the target resource object of the first secure container through a driver of the target resource object, the first suspension module 501 is specifically configured to: call a target application programming interface in the driver of the target resource object to acquire a video memory list of the target resource object of the first secure container and corresponding video memory content of the video memory list; and call other application programming interfaces in the driver of the target resource object other than the target application programming interface to acquire other state data of the target resource object of the first secure container, the other state data being state data other than the video memory list and the corresponding video memory content.

[0155] Further optionally, the user space of the first secure container includes a runtime library of the target resource object, and the first suspension module 501 is further configured to: use an agent function in the runtime library to proxy an access request initiated by an application in the first secure container, and when the access request is a memory allocation request, acquire a video memory list applied by the application as the video memory list of the target resource object of the first secure container. The access request is initiated by the application to the driver of the target resource object through a library function in the runtime library.

[0156] Further optionally, the user space of the first secure container includes a state saving tool, and when the second suspension module 503 suspends the running of the first secure container through a container runtime component corresponding to the first secure container and acquires the state data of the non-target resource object of the first secure container, the second suspension module 503 is specifically configured to: suspend the running of the first secure container by the container runtime component to keep the state data of the non-target resource object of the first secure container unchanged; and call the state saving tool by the container runtime component to acquire the state data of the non-target resource object of the first secure container and save the state data into the memory of the source node.

[0157] Further optionally, the first migration module 502 creates a second secure container on the target node, and when migrating the state data of the target resource object of the first secure container into the target resource object of the second secure container, specifically configured to: create a third secure container on the target node, and migrate the state data of the target resource object of the first secure container into the target resource object of the third secure container; create a second secure container on the target node, and transfer the target resource object of the third secure container to the second secure container to migrate the state data of the target resource object of the first secure container into the target resource object of the second secure container; or create a first virtual machine in the second secure container, bind the target resource object of the second secure container with the first virtual machine, and migrate the state data of the target resource object of the first secure container into the target resource object bound with the first virtual machine; create a second virtual machine in the second secure container, bind the non-target resource object of the second secure container with the second virtual machine, and bind the target resource object bound with the first virtual machine with the second virtual machine to migrate the state data of the target resource object of the first secure container into the target resource object of the second secure container.

[0158] Further optionally, the non-target resource object of the first secure container includes an RDMA network card; and when the first migration module 502 migrates the state data of the target resource object of the first secure container into the target resource object of the second secure container, specifically configured to: copy the state data of the target resource object of the first secure container into the target resource object of the second secure container through an RDMA operation; and when the second migration module 504 migrates the state data of the non-target resource object of the first secure container into the non-target resource object of the second secure container, specifically configured to: migrate the state data of the non-target resource object of the first secure container into the non-target resource object of the second secure container through an RDMA operation.

[0159] Further optionally, the second migration module 504 is further configured to: receive a container migration task issued by the management node, the container migration task being used to instruct to migrate a workload of the first secure container on the source node into the second secure container on the target node.

[0160] FIG. 6 is a structural schematic diagram of another security container migration apparatus provided by another exemplary embodiment of the present disclosure, for migrating the workloads of a first security container on a source node. The security container migration apparatus can be implemented in a target node, wherein a second security container is created on the target node, and the second security container includes target resource objects managed by a dependency driver and non-target resource objects not managed by the dependency driver. As shown in FIG. 6, the apparatus includes: a first saving module 602, configured to receive state data of target resource objects of the first security container, and save the state data of the target resource objects of the first security container in target resource objects of the second security container; and a second saving module 603, configured to receive state data of non-target resource objects of the first security container, and save the state data of the non-target resource objects of the first security container in non-target resource objects of the second security container.

[0161] Further optionally, as shown in FIG. 6, the apparatus further includes: a creating module 601, configured to create the second security container on the target node.

[0162] Further optionally, the creating module 601 is further configured to: run the second security container; recover a work state of the non-target resource objects according to the state data in the non-target resource objects of the second security container; and recover a work state of the target resource objects according to the state data in the target resource objects of the second security container.

[0163] Further optionally, the user space of the second security container includes a state recovery tool, and when the creating module 601 recovers the work state of the non-target resource objects according to the state data in the non-target resource objects of the second security container, the state recovery tool is specifically configured to: provide the state data in the non-target resource objects of the second security container to the non-target resource objects by invoking the state recovery tool, so as to recover the work state of the non-target resource objects.

[0164] Further optionally, the second security container includes a driver of the target resource objects, and the state data of the target resource objects includes a list of display memories and corresponding display memory contents and other state data; and when the creating module 601 recovers the work state of the target resource objects according to the state data in the target resource objects of the second security container, the creating module 601 is specifically configured to: provide the list of display memories and the corresponding display memory contents in the target resource objects of the second security container to the target resource objects of the second security container by invoking a target application programming interface in the driver of the target resource objects; and provide other state data in the target resource objects of the second security container to the target resource objects of the second security container by invoking other application programming interfaces in the driver of the target resource objects except the target application programming interface, so as to recover the work state of the target resource objects, wherein the other state data refers to state data other than the list of display memories and the corresponding display memory contents.

[0165] Further optionally, the first saving module 602 saves the state data of the target resource object of the first security container in the target resource object of the second security container, including: migrating the state data of the target resource object of the first security container into the target resource object of the third security container, the third security container being a security container on the target node; in the case that the second security container is created, transferring the target resource object of the third security container to the second security container to migrate the state data of the target resource object of the first security container into the target resource object of the second security container.

[0166] Further optionally, the creating module 601 is further configured to create a third security container on the target node before creating the second security container on the target node.

[0167] Further optionally, when the first saving module 602 saves the state data of the target resource object of the first security container in the target resource object of the second security container, the first saving module 602 is specifically configured to: create a first virtual machine in the second security container, bind the target resource object of the second security container with the first virtual machine, and migrate the state data of the target resource object of the first security container into the target resource object bound with the first virtual machine; create a second virtual machine in the second security container, bind the non-target resource object of the second security container with the second virtual machine, and bind the target resource object bound with the first virtual machine with the second virtual machine again to migrate the state data of the target resource object of the first security container into the target resource object of the second security container.

[0168] The above describes the internal functions and structures of the security container migration apparatus. In practice, the security container migration apparatus can be implemented as a computer device, which can be implemented as the source node in the above-described embodiments, and includes a memory 701, a processor 702, and a communication component 703.

[0169] The memory 701 is configured to store computer programs and can be configured to store other various data to support operations on the computer device. Examples of the data include instructions of any application program or method for operating on the computer device, contact data, phonebook data, messages, pictures, videos, etc. Optionally, the memory 701 stores program codes related to the first security container and program codes related to the container runtime component.

[0170] In some embodiments, the processor 702, coupled with the memory 701, is configured to execute a computer program (here, the computer program mainly refers to program code related to the container runtime component on the source node) in the memory 701, to: suspend running of a target resource object of the first secure container through a driver of the target resource object during running of the first secure container, and obtain state data of the target resource object of the first secure container; instruct to create a second secure container on a target node, and migrate the state data of the target resource object of the first secure container to a target resource object of the second secure container; suspend running of the first secure container through a container runtime component corresponding to the first secure container, and obtain state data of a non-target resource object of the first secure container; and migrate the state data of the non-target resource object of the first secure container to a non-target resource object of the second secure container, to migrate a workload in the first secure container to the second secure container.

[0171] Further optionally, the first secure container includes a client operating system, the client operating system includes a driver of the target resource object, and a runtime library of the target resource object is included in a user space of the first secure container. The processor 702 suspending running of the target resource object of the first secure container through the driver of the target resource object is specifically configured to: add an interception function in the runtime library, to intercept an access request initiated by an application in the first secure container to the driver of the target resource object through a library function in the runtime library, to suspend running of the target resource object of the first secure container; or, call the driver of the target resource object to send a suspension command to the target resource object of the first secure container, to suspend running of the target resource object of the first secure container.

[0172] Further optionally, when the processor 702 obtains the state data of the target resource object of the first secure container through the driver of the target resource object, the processor 702 is specifically configured to: call a target application programming interface in the driver of the target resource object to obtain a list of video memory and corresponding video memory content of the target resource object of the first secure container; and call other application programming interfaces in the driver of the target resource object except the target application programming interface to obtain other state data of the target resource object of the first secure container, the other state data being state data other than the list of video memory and the corresponding video memory content.

[0173] Optionally, the user space of the first secure container includes a runtime library for the target resource object. The processor 702 is further configured to: add proxy functions to the runtime library to proxy access requests initiated by the application in the first secure container to the driver of the target resource object through library functions in the runtime library, and, if the access request is a memory allocation request, obtain the list of video memory requested by the application as the list of video memory of the target resource object of the first secure container.

[0174] Optionally, the user space of the first secure container includes a state saving tool. When the processor 702 pauses the operation of the first secure container through the container runtime component corresponding to the first secure container and obtains the state data of the non-target resource objects of the first secure container, it is specifically used for: the container runtime component pausing the operation of the first secure container to keep the state data of the non-target resource objects of the first secure container unchanged; and the container runtime component calling the state saving tool to save the state data of the non-target resource objects of the first secure container to the memory of the source node.

[0175] Further optionally, when the processor 702 creates a second secure container on the target node and migrates the state data of the target resource object of the first secure container to the target resource object of the second secure container, it is specifically used to: instruct the creation of a third secure container on the target node and migrate the state data of the target resource object of the first secure container to the target resource object of the third secure container; instruct the creation of a second secure container on the target node and transfer the target resource object of the third secure container to the second secure container to migrate the state data of the target resource object of the first secure container to the target resource object of the second secure container; or instruct the creation of a first virtual machine in the second secure container, bind the target resource object of the second secure container to the first virtual machine, and migrate the state data of the target resource object of the first secure container to the target resource object bound to the first virtual machine; instruct the creation of a second virtual machine in the second secure container, bind the non-target resource object of the second secure container to the second virtual machine, and bind the target resource object bound to the first virtual machine to the second virtual machine to migrate the state data of the target resource object of the first secure container to the target resource object of the second secure container.

[0176] Further optionally, the non-target resource object of the first secure container comprises an RDMA network card; when the processor 702 migrates the state data of the target resource object of the first secure container into the target resource object of the second secure container, specifically for: copying the state data of the target resource object of the first secure container into the target resource object of the second secure container through an RDMA operation; when the processor 702 migrates the state data of the non-target resource object of the first secure container into the non-target resource object of the second secure container, specifically for: migrating the state data of the non-target resource object of the first secure container into the non-target resource object of the second secure container through an RDMA operation.

[0177] Further optionally, the processor 702 is further configured to: receive a container migration task issued by the management node, the container migration task being used to indicate that the workload of the first secure container on the source node is migrated into the second secure container on the target node.

[0178] In some embodiments, the processor 702, coupled with the memory 701, is configured to execute a computer program in the memory 701, so as to: receive state data of a target resource object of a first secure container, and save the state data of the target resource object of the first secure container in a target resource object of a second secure container, the second secure container being a secure container on a target node; receive state data of a non-target resource object of the first secure container, and save the state data of the non-target resource object of the first secure container in a non-target resource object of the second secure container.

[0179] Further optionally, the processor 702 is further configured to: create the second secure container on the target node, the second secure container comprising a target resource object managed by a dependency driver and a non-target resource object not managed by the dependency driver.

[0180] Further optionally, the processor 702 is further configured to: run the second secure container; restore a working state of the non-target resource object according to the state data in the non-target resource object of the second secure container; and restore a working state of the target resource object according to the state data in the target resource object of the second secure container.

[0181] Further optionally, the user space of the second secure container comprises a state recovery tool; when the processor 702 restores the working state of the non-target resource object according to the state data in the non-target resource object of the second secure container, specifically for: invoking the state recovery tool to provide the state data in the non-target resource object of the second secure container to the non-target resource object, so as to restore the working state of the non-target resource object.

[0182] Further optionally, the second secure container comprises a driver of the target resource object, and the state data of the target resource object comprises a display list and corresponding display content of the display list and other state data; when the processor 702 recovers the working state of the target resource object according to the state data in the target resource object of the second secure container, the processor 702 is specifically configured to: call a target application programming interface in the driver of the target resource object, and provide the display list and the corresponding display content of the display list in the target resource object of the second secure container to the target resource object of the second secure container; call other application programming interfaces in the driver of the target resource object except the target application programming interface, and provide the other state data in the target resource object of the second secure container to the target resource object of the second secure container, so as to recover the working state of the target resource object, wherein the other state data refers to state data except the display list and the corresponding display content of the display list.

[0183] Further optionally, when the processor 702 saves the state data of the target resource object of the first secure container in the target resource object of the second secure container, the processor 702 is specifically configured to: migrate the state data of the target resource object of the first secure container to a target resource object of a third secure container, the third secure container being a secure container on the target node; and in a case where the second secure container is created, transfer the target resource object of the third secure container to the second secure container, so as to migrate the state data of the target resource object of the first secure container to the target resource object of the second secure container.

[0184] Further optionally, before the processor 702 creates the second secure container on the target node, the processor 702 further creates a third secure container on the target node.

[0185] Further optionally, when the processor 702 saves the state data of the target resource object of the first secure container in the target resource object of the second secure container, the processor 702 is specifically configured to: create a first virtual machine in the second secure container, bind the target resource object of the second secure container to the first virtual machine, and migrate the state data of the target resource object of the first secure container to the target resource object bound to the first virtual machine; create a second virtual machine in the second secure container, bind a non-target resource object of the second secure container to the second virtual machine, and bind the target resource object bound to the first virtual machine to the second virtual machine, so as to migrate the state data of the target resource object of the first secure container to the target resource object of the second secure container.

[0186] Further, as shown in FIG. 7, the computer device further includes a display 704, a power supply component 705, an audio component 706, and other components. The components shown in FIG. 7 are only schematic and the computer device can include more components not shown in FIG. 7. In addition, the components in the dashed box in FIG. 7 are optional components and not mandatory components, and the specific components can be determined according to the product form of the computer device. The computer device in the embodiment can be implemented as a terminal device such as a desktop computer, a notebook computer, a smart phone, or an IOT device, or a server device such as a general server, a cloud server, or a server array. If the computer device in the embodiment is implemented as a terminal device such as a desktop computer, a notebook computer, or a smart phone, the computer device can include the components in the dashed box in FIG. 7. If the computer device in the embodiment is implemented as a server device such as a general server, a cloud server, or a server array, the computer device can not include the components in the dashed box in FIG. 7.

[0187] The embodiment of the present disclosure also provides a cloud computing system, comprising: a source node and a target node; the source node and the target node cooperate to migrate a workload in a first security container on the source node to a second security container on the target node; wherein the cooperation process of the source node and the target node and the working process of each node in the cooperation process can be referred to the foregoing embodiments, and will not be described here.

[0188] Correspondingly, the embodiment of the present disclosure also provides a computer readable storage medium storing a computer program, which can implement each step in the foregoing method embodiments when executed by a processor.

[0189] Correspondingly, the embodiment of the present disclosure also provides a computer program product, comprising a computer program / instruction, which can implement each step in the foregoing method embodiments when executed by a processor.

[0190] The memory can be implemented by any type of volatile or nonvolatile storage devices or a combination thereof, such as a static random access memory (SRAM), an electrically erasable programmable read only memory (EEPROM), an erasable programmable read only memory (EPROM), a programmable read only memory (PROM), a read only memory (ROM), a magnetic memory, a flash memory, a magnetic disk, or an optical disk.

[0191] The communication component is configured to facilitate wired or wireless communication between a device on which the communication component is installed and other devices. The device on which the communication component is installed can access a wireless network based on a communication standard, such as WiFi, a 2G, 3G, 4G / LTE, 5G, or the like mobile communication network, or a combination thereof. In an example embodiment, the communication component receives a broadcast signal or a broadcast related information from an external broadcast management system via a broadcast channel. In an example embodiment, the communication component further includes a Near Field Communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on Radio Frequency Identification (RFID) technology, Infrared Data Association (IrDA) technology, Ultra Wide Band (UWB) technology, BlueTooth (BT) technology, and other technologies.

[0192] The display includes a screen, which can include a Liquid Crystal Display (LCD) and a Touch Panel (TP). If the screen includes a touch panel, the screen can be implemented as a touch screen to receive an input signal from a user. The touch panel includes one or more touch sensors to sense a touch, a slide, and a gesture on the touch panel. The touch sensor can not only sense a boundary of a touch or a slide action, but also detect a duration and a pressure associated with a touch or a slide operation.

[0193] The power supply component provides power to various components of a device on which the power supply component is installed. The power supply component can include a power management system, one or more power sources, and other components associated with generating, managing, and distributing power for a device on which the power supply component is installed.

[0194] The audio component can be configured to output and / or input audio signals. For example, the audio component includes a microphone (MIC) that is configured to receive an external audio signal when the device on which the audio component is installed is in a particular mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signal can be further stored in memory or transmitted via the communication component. In some embodiments, the audio component also includes a speaker to output audio signals.

[0195] Those skilled in the art will appreciate that embodiments of the disclosure can be supplied as a method, a system, or a computer program product. Thus, the disclosure can take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the disclosure can take the form of a computer program product on one or more computer readable storage media (including, but not limited to, disk memory, CD-ROMs, optical storage devices, etc.) embodying computer readable program code.

[0196] The disclosure is described in reference to the flowchart and / or block diagrams of the method, apparatus (system) and computer program product according to embodiments of the disclosure. It should be understood that each flow and / or block in the flowchart and / or block diagram, as well as a combination of flows and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general purpose computer, a special purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions, which are executed via the processor of the computer or other programmable data processing device, generate means for implementing the functions specified in one or more flows in the flowchart and / or one or more blocks in the block diagram.

[0197] These computer program instructions can also be stored in a computer readable memory that can direct the computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce a manufactured product including instruction means, which implements the functions specified in one or more flows in the flowchart and / or one or more blocks in the block diagram.

[0198] These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operational steps are performed on the computer or other programmable device to produce a computer implemented process, so that the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one or more flows in the flowchart and / or one or more blocks in the block diagram.

[0199] In a typical configuration, the computing device includes one or more processors (CPU), input / output interfaces, network interfaces and memory.

[0200] The memory can include non-persistent memory in the computer readable medium, random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash memory. The memory is an example of computer readable media.

[0201] Computer-readable media includes permanent and non-permanent, movable and non-movable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase-change random access memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital video disc (DVD) or other optical storage, magnetic cassette, magnetic tape disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information accessible to a computing device. According to the definition herein, computer-readable media does not include transitory media such as modulated data signals and carriers.

[0202] It should also be noted that the terms "comprising", "containing", or any other variant thereof are intended to cover non-exclusive inclusions, so that a process, method, article or apparatus that includes a list of elements does not only include those elements, but also includes other elements not explicitly listed, or further includes elements inherent in such a process, method, article or apparatus. Without more limitations, the element defined by the statement "comprising a" does not exclude the presence of additional identical elements in the process, method, article or apparatus that includes the element.

[0203] The above only is an embodiment of the disclosure and is not used to limit the disclosure. The disclosure can have various changes and variations for those skilled in the art. Any modification, equivalent replacement, improvement, etc. within the spirit and principle of the disclosure shall be included in the scope of claims of the disclosure.

Claims

1. A secure container migration method for migrating a workload in a first secure container on a source node, the first secure container comprising target resource objects and non-target resource objects, the target resource objects being pass-through devices of the first secure container, the method comprising: suspending running of the target resource objects of the first secure container by a driver of the target resource objects and obtaining state data of the target resource objects of the first secure container during running of the first secure container; migrating the state data of the target resource objects of the first secure container to target resource objects of a second secure container, the second secure container being a secure container on a target node; suspending running of the first secure container by a container runtime component corresponding to the first secure container and obtaining state data of the non-target resource objects of the first secure container; migrating the state data of the non-target resource objects of the first secure container to non-target resource objects of the second secure container to migrate the workload in the first secure container to the second secure container.

2. The method of claim 1, wherein, the driver of the target resource objects is included in a guest operating system of the first secure container, and a runtime library of the target resource objects is included in a user space of the first secure container, suspending running of the target resource objects of the first secure container by the driver of the target resource objects comprises: intercepting, by an intercept function in the runtime library, an access request initiated by an application program in the first secure container to suspend running of the target resource objects of the first secure container, the access request being initiated by the application program to the driver of the target resource objects through a library function in the runtime library; or calling the driver of the target resource objects to send a suspension command to the target resource objects of the first secure container to suspend running of the target resource objects of the first secure container. obtaining the state data of the target resource objects of the first secure container by the driver of the target resource objects comprises:

3. The method of claim 1, wherein, calling a target application program interface in the driver of the target resource objects to obtain a video memory list and corresponding video memory content of the target resource objects of the first secure container; calling other application program interfaces in the driver of the target resource objects other than the target application program interface to obtain other state data of the target resource objects of the first secure container, the other state data being state data other than the video memory list and the corresponding video memory content. the runtime library of the target resource objects is included in the user space of the first secure container, and the method further comprises:

4. The method of claim 1, wherein, ​ The proxy function in the runtime library proxies an access request initiated by an application in the first secure container, and in a case where the access request is a memory allocation request, obtains a list of display memories applied for by the application as a list of display memories of a target resource object of the first secure container; the access request is initiated by the application to a driver of the target resource object through a library function in the runtime library.

5. The method of claim 1, wherein, The state saving tool is included in a user space of the first secure container, and the first secure container is suspended by a container runtime component corresponding to the first secure container, and state data of a non-target resource object of the first secure container is obtained, including: The container runtime component suspends the first secure container to keep the state data of the non-target resource object of the first secure container unchanged; The container runtime component calls the state saving tool to obtain the state data of the non-target resource object of the first secure container and save the state data into the memory of the source node.

6. The method of claim 1, wherein, Indicating to create a second secure container on a target node, and migrating the state data of the target resource object of the first secure container to a target resource object of the second secure container, including: Indicating to create a third secure container on the target node, and migrating the state data of the target resource object of the first secure container to a target resource object of the third secure container; Indicating to create a second secure container on the target node, and migrating the target resource object of the third secure container to the second secure container to migrate the state data of the target resource object of the first secure container to the target resource object of the second secure container; Or Indicating to create a first virtual machine in the second secure container, binding the target resource object of the second secure container with the first virtual machine, and migrating the state data of the target resource object of the first secure container to the target resource object bound with the first virtual machine; Indicating to create a second virtual machine in the second secure container, binding the non-target resource object of the second secure container with the second virtual machine, and binding the target resource object bound with the first virtual machine with the second virtual machine to migrate the state data of the target resource object of the first secure container to the target resource object of the second secure container.

7. The method according to any one of claims 1 to 6, wherein, The non-target resource object of the first secure container includes a remote direct data access network card; Migrating the state data of the target resource object of the first secure container to the target resource object of the second secure container includes copying the state data of the target resource object of the first secure container to the target resource object of the second secure container through a remote direct data access operation; Migrating the state data of the non-target resource object of the first secure container to the non-target resource object of the second secure container includes copying the state data of the non-target resource object of the first secure container to the non-target resource object of the second secure container through a remote direct data access operation.

8. The method of any of claims 1-6, further comprising: receiving a container migration task issued by a management node, the container migration task being used to instruct to migrate a workload of a first secure container on a source node to a second secure container on a target node.

9. A method for migrating a workload of a first secure container on a source node, a second secure container being created on a target node, the second secure container comprising target resource objects managed by a dependency driver and non-target resource objects not managed by the dependency driver; the method comprising: receiving state data of target resource objects of the first secure container and saving the state data of the target resource objects of the first secure container in target resource objects of the second secure container; receiving state data of non-target resource objects of the first secure container and saving the state data of the non-target resource objects of the first secure container in non-target resource objects of the second secure container.

10. The method of claim 9, further comprising: running the second secure container; restoring a working state of the non-target resource objects according to the state data in the non-target resource objects of the second secure container; restoring a working state of the target resource objects according to the state data in the target resource objects of the second secure container.

11. The method of claim 10, wherein, the second secure container comprising a driver of the target resource objects, the state data of the target resource objects comprising a list of video memory and corresponding video memory content and other state data; restoring the working state of the target resource objects according to the state data in the target resource objects of the second secure container comprises: calling a target application programming interface in the driver of the target resource objects to provide the list of video memory and corresponding video memory content in the target resource objects of the second secure container to the target resource objects of the second secure container; calling other application programming interfaces in the driver of the target resource objects other than the target application programming interface to provide other state data in the target resource objects of the second secure container to the target resource objects of the second secure container to restore the working state of the target resource objects, the other state data being state data other than the list of video memory and corresponding video memory content.

12. The method according to any one of claims 9-11, wherein, saving the state data of the target resource objects of the first secure container in the target resource objects of the second secure container comprises: migrating the state data of the target resource objects of the first secure container to target resource objects of a third secure container, the third secure container being a secure container on the target node; in a case where the second secure container is created, transferring the target resource objects of the third secure container to the second secure container to migrate the state data of the target resource objects of the first secure container to the target resource objects of the second secure container.

13. A computer device, implementable as a source node, the source node comprising: a host machine, the host machine deploying a first secure container and a container runtime component, the first secure container comprising target resource objects and non-target resource objects; The target resource object is a pass-through device of the first secure container, wherein the container runtime component is configured to perform the steps of the method of any of claims 1-8.

14. A computer device comprising: a memory and a processor; The memory is configured to store a computer program, and the processor is coupled to the memory and configured to execute the computer program in the memory to implement the steps of the method of any of claims 1-8 and 9-12.

15. A computer readable storage medium storing a computer program, wherein, The computer program, when executed by the processor, causes the processor to implement the steps of the method of any of claims 1-8 and 9-12.

16. A computer program product comprising computer programs / instructions, wherein, The computer program / instructions, when executed by the processor, perform the steps of the method of any of claims 1-8 and 9-12.

17. A cloud computing system comprising: A source node configured to perform the method of any of claims 1-8, and a target node configured to perform the method of any of claims 9-12; wherein the source node and the target node cooperate with each other to migrate a workload in a first secure container on the source node to a second secure container on the target node.

Images