Secure container migration method, device, storage medium, and program product

Abstract

The embodiment of the invention provides a secure container migration method and device, a storage medium and a program product. A first security container on a source node comprises a target resource object and a non-target resource object, and under the condition that the target resource object is realized as a direct connection device of the security container, aiming at different characteristics of the target resource object and the non-target resource object, through mutual cooperation of different hierarchical components, the target resource object and the non-target resource object can be directly connected. That is, the state data of the target resource object and the non-target resource object in the first security container are obtained through the drive program of the target resource object and the container runtime component, so that the state data of the target resource object and the non-target resource object can be migrated to the second security container on the target node more accurately. Through mutual cooperation of different hierarchical components, rapid migration of the workload in the security container can be well completed, and the problem of state migration of the security container with the security container or a target resource object which cannot be directly managed by a cloud manufacturer to which the security container belongs is solved.

Description

(19) State Intellectual Property Office (12) Invention Patent Application (10) Application Publication Number (43) Application Publication Date (21) Application Number 202411259618.X (22) Application Date 2024.09.09 (71) Applicant Alibaba Cloud Computing Co., Ltd. Address 310024, Room 1-2-A06, Yung Valley Park, No. 1008, Dengcai Street, Sandun Town, Xihu District, Hangzhou City, Zhejiang Province (72) Inventor Liu Jiang Bian Zizheng (74) Patent Agency Beijing Taihe Jiusi Intellectual Property Agency Co., Ltd. 11610 Patent Attorney Zhang Ai Liu Ge (51) Int.Cl. G06F 9 / 455 (2018.01) (54) Invention Title Secure Container Migration Method, Device, Storage Medium and Program Product (57) Abstract This application provides a secure container migration method, device, storage medium and program product. The first secure container on the source node includes target resource objects and non-target resource objects. When the target resource object is implemented as a pass-through device for the secure container, different layers of components work together to address the different characteristics of the target and non-target resource objects. Specifically, the state data of the target and non-target resource objects in the first secure container is obtained through the driver of the target resource object and the container runtime components. This allows for relatively accurate migration of the state data of the target and non-target resource objects to the second secure container on the target node. This collaborative approach of different layers of components effectively enables rapid migration of workloads within secure containers, resolving the state migration problem of secure containers containing target resource objects that cannot be directly managed by the secure container or its cloud provider. Claims 3 pages, Description 27 pages, Drawings 7 pages, CN 121636047 A 2026.03.10 CN 1 21 63 60 47 A 1. A secure container migration method, characterized in that it is used to migrate workloads in a first secure container on a source node, the first secure container including a target resource object and non-target resource objects, the target resource object being a pass-through device of the first secure container, the method comprising: during the operation of the first secure container, pausing the operation of the target resource object of the first secure container through the driver of the target resource object, and obtaining the status data of the target resource object of the first secure container; migrating the status data of the target resource object of the first secure container to the target resource object of a second secure container, the second secure container being a secure container on a target node; pausing the operation of the first secure container through the container runtime component corresponding to the first secure container, and obtaining the status data of the non-target resource object of the first secure container; migrating the status data of the non-target resource object of the first secure container to the non-target resource object of the second secure container.In the target resource object, the workload in the first secure container is migrated to the second secure container. 2. The method according to claim 1, wherein the guest operating system of the first secure container includes a driver for the target resource object, the user space of the first secure container includes a runtime library for the target resource object, and pausing the operation of the target resource object of the first secure container through the driver for the target resource object includes: intercepting access requests initiated by applications in the first secure container using interception functions in the runtime library to pause the operation of the target resource object of the first secure container; the access request is initiated by the application to the driver for the target resource object through library functions in the runtime library; or, calling the driver for the target resource object to send a pause command to the target resource object of the first secure container to pause the operation of the target resource object of the first secure container. 3. The method according to claim 1, characterized in that, obtaining the state data of the target resource object of the first secure container through the driver of the target resource object includes: calling the target application interface in the driver of the target resource object to obtain the video memory list and its corresponding video memory content of the target resource object of the first secure container; calling other application interfaces in the driver of the target resource object other than the target application interface to obtain other state data of the target resource object of the first secure container, wherein the other state data refers to state data other than the video memory list and its corresponding video memory content. 4. The method according to claim 1, characterized in that, the user space of the first secure container includes the runtime library of the target resource object, and the method further includes: using the proxy function in the runtime library to proxy the access request initiated by the application in the first secure container, and, when the access request is a memory allocation request, obtaining the video memory list requested by the application as the video memory list of the target resource object of the first secure container; the access request is initiated by the application to the driver of the target resource object through the library function in the runtime library. 5. The method according to claim 1, characterized in that the user space of the first secure container includes a state saving tool, which, through the container runtime component corresponding to the first secure container, pauses the operation of the first secure container and obtains the state data of the non-target resource objects of the first secure container, including: the container runtime component pausing the operation of the first secure container to keep the state data of the non-target resource objects of the first secure container unchanged; The container runtime component calls the state saving tool to obtain the non-target resource objects of the first secure container.The state data of the object is stored in the memory of the source node. 6. The method according to claim 1, characterized in that, instructing the creation of a second secure container on the target node and migrating the state data of the target resource object of the first secure container to the target resource object of the second secure container includes: instructing the creation of a third secure container on the target node and migrating the state data of the target resource object of the first secure container to the target resource object of the third secure container; instructing the creation of a second secure container on the target node and transferring the target resource object of the third secure container to the second secure container, so as to migrate the state data of the target resource object of the first secure container to the target resource object of the second secure container; or instructing the creation of a first virtual machine in the second secure container, binding the target resource object of the second secure container to the first virtual machine, and migrating the state data of the target resource object of the first secure container to the target resource object bound to the first virtual machine; instructing the creation of a second virtual machine in the second secure container, binding the non-target resource object of the second secure container to the second virtual machine, and binding the target resource object bound to the first virtual machine to the second virtual machine, so as to migrate the state data of the target resource object of the first secure container to the target resource object of the second secure container. 7. The method according to any one of claims 1-6, wherein the non-target resource object of the first secure container includes a remote direct data access network interface card (RDBNIC); migrating the state data of the target resource object of the first secure container to the target resource object of the second secure container includes: copying the state data of the target resource object of the first secure container to the target resource object of the second secure container through a remote direct data access operation; migrating the state data of the non-target resource object of the first secure container to the non-target resource object of the second secure container includes: copying the state data of the non-target resource object of the first secure container to the non-target resource object of the second secure container through a remote direct data access operation. 8. The method according to any one of claims 1-6, further comprising: receiving a container migration task issued by a management node, the container migration task being used to instruct the migration of the workload of the first secure container on the source node to the second secure container on the target node. 9. A secure container migration method, characterized in that it is used to migrate the workload of a first secure container on a source node, and a second secure container is created on a target node, the second secure container including a target resource object managed by a driver and a non-target resource object not managed by a driver; the method includes: receiving the status data of the target resource object of the first secure container, and migrating the target resource object of the first secure container...The state data of the source object is stored in the target resource object of the second security container; the state data of the non-target resource object of the first security container is received, and the state data of the non-target resource object of the first security container is stored in the non-target resource object of the second security container. 10. The method according to claim 9, characterized in that it further includes: running the second security container; restoring the working state of the non-target resource object according to the state data in the non-target resource object of the second security container; restoring the working state of the target resource object according to the state data in the target resource object of the second security container. 11. The method according to claim 10, wherein the second secure container includes a driver for the target resource object, and the state data of the target resource object includes a video memory list and its corresponding video memory content and other state data; restoring the working state of the target resource object according to the state data in the target resource object of the second secure container includes: calling a target application programming interface in the driver of the target resource object to provide the video memory list and its corresponding video memory content in the target resource object of the second secure container to the target resource object of the second secure container; calling other application programming interfaces in the driver of the target resource object other than the target application programming interface to provide other state data in the target resource object of the second secure container to the target resource object of the second secure container to restore the working state of the target resource object, wherein the other state data refers to state data other than the video memory list and its corresponding video memory content. 12. The method according to any one of claims 9-11, characterized in that storing the state data of the target resource object of the first secure container in the target resource object of the second secure container comprises: migrating the state data of the target resource object of the first secure container to the target resource object of a third secure container, wherein the third secure container is a secure container on the target node; and, when the second secure container is created, transferring the target resource object of the third secure container to the second secure container to migrate the state data of the target resource object of the first secure container to the target resource object of the second secure container. 13. A computer device, characterized in that it can be implemented as a source node, the source node comprising: a host machine, on which a first secure container and a container runtime component are deployed, the first secure container comprising a target resource object and a non-target resource object; the target resource object being a pass-through device of the first secure container, wherein the container runtime component is used to execute the steps of the method according to any one of claims 1-8.14. A computer device, characterized in that it comprises: a memory and a processor; the memory for storing a computer program, the processor coupled to the memory for executing the computer program in the memory to implement the steps of the method according to any one of claims 1-8 and 9-12. 15. A computer-readable storage medium, characterized in that, when the computer program is executed by a processor, it causes the processor to implement the steps of the method according to any one of claims 1-8 and 9-12. 16. A computer program product, characterized in that it comprises a computer program / instructions, which, when executed by a processor, perform the steps of the method according to any one of claims 1-8 and 9-12. 17. A cloud computing system, characterized in that it comprises: a source node for executing the method according to any one of claims 1-8, and a target node for executing the method according to any one of claims 9-12; the source node and the target node cooperate to migrate workloads in a first secure container on the source node to a second secure container on the target node. Claims 3 / 3 Page 4 CN 121636047 A Secure Container Migration Method, Device, Storage Medium and Program Product Technical Field

[0001] This application relates to the field of cloud computing technology, and in particular to a secure container migration method, device, storage medium and program product. Background Art

[0002] In cloud computing environments, multi-tenant architecture is usually adopted on a single server node. In order to achieve secure isolation between different tenants, containerization technology can be used to package applications and their dependencies into independent containers to ensure the security and consistency of applications. As a branch of container technology, secure containers aim to combine the security of virtualization and the convenience of containers to make the isolation between containers more thorough, and can provide a higher level of isolation and protection for applications in containers, and are therefore widely used.

[0003] AI (Artificial Intelligence) intelligent computing is an important application scenario of cloud computing. More and more AI applications and AI frameworks used to develop and deploy AI applications are deployed in secure containers, which leads to an increasing demand for computing resources for secure containers. GPUs (Graphics Processing Units), as a high-performance computing resource provided by cloud providers, have demonstrated powerful performance in the fields of AI and deep learning. Therefore, some cloud providers have proposed using GPUs in secure containers as the computing power supply for applications within the secure containers.

[0004] Among these, combining GPUs with secure containers can solve the problems of secure containers while achieving secure isolation.While addressing the issue of computing power requirements, new technical challenges also arise. For example, in cases of application upgrades, resource optimization, or application failures, it is necessary to migrate secure containers. How to migrate secure containers with GPU resources is a technical problem that urgently needs to be solved.

[0005] This application provides a secure container migration method, apparatus, storage medium, and program product to solve the problem of migrating secure containers with GPU resources.

[0006] This application provides a secure container migration method for migrating workloads in a first secure container on a source node. The first secure container includes a target resource object and non-target resource objects. The target resource object is a pass-through device of the first secure container. The method includes: during the operation of the first secure container, pausing the operation of the target resource object of the first secure container through the driver of the target resource object, and obtaining the status data of the target resource object of the first secure container; migrating the status data of the target resource object of the first secure container to the target resource object of a second secure container, the second secure container being a secure container on the target node; pausing the operation of the first secure container through the container runtime component corresponding to the first secure container, and obtaining the status data of the non-target resource object of the first secure container; migrating the status data of the non-target resource object of the first secure container to the non-target resource object of the second secure container, so as to migrate the workload in the first secure container to the second secure container.

[0007] This application embodiment also provides a secure container migration method for migrating the workload of a first secure container on a source node, and creating a second secure container on a target node. The second secure container includes a target resource object managed by a driver and a non-target resource object not managed by a driver. The method includes: receiving state data of the target resource object of the first secure container (see page 1 / 27 of specification CN 121636047 A), and storing the state data of the target resource object of the first secure container in the target resource object of the second secure container; receiving state data of the non-target resource object of the first secure container, and storing the state data of the non-target resource object of the first secure container in the non-target resource object of the second secure container.

[0008] This application embodiment also provides a computer device that can be implemented as a source node. The source node includes: a host machine, on which a first secure container and a container runtime component are deployed. The first secure container includes a target resource object and a non-target resource object. The target resource object is a passthrough device of the first secure container, wherein the container runtime component is used to execute the steps in the secure container migration method.

[0009] This application embodiment also provides a computer device, including: a memory and a processor; the memory is used to store a computer program, and the processor is coupled to the memory and used to execute the computer program in the memory to implement the steps in the secure container migration method.

[0010] This application embodiment also provides a computer-readable storage medium, which, when the computer program is executed by the processor, causes the processor to implement the steps in the secure container migration method.

[0011] This application embodiment also provides a computer program product, including a computer program / instruction, which, when the computer program / instruction is executed by the processor, performs the steps in the secure container migration method.

[0012] This application embodiment also provides a cloud computing system, including: a source node for executing the first set of secure container migration methods described above, and a target node for executing the second set of secure container migration methods described above; the source node and the target node cooperate with each other to migrate the workload in the first secure container on the source node to the second secure container on the target node.

[0013] In this embodiment, the first security container on the source node includes a target resource object and non-target resource objects. When the target resource object is implemented as a pass-through device for the first security container, based on the different characteristics of the target resource object and non-target resource objects, different layers of components can cooperate to obtain the state data of the target resource object and non-target resource objects in the first security container through the driver of the target resource object and the container runtime component. In this way, the state data of the target resource object and non-target resource objects can be migrated to the second security container on the target node more accurately. In this way, the rapid migration of workloads in the security container can be completed better through the cooperation of different layers of components, solving the state migration problem of security containers with target resource objects that cannot be directly managed by the security container or its cloud vendor. Brief Description of the Drawings

[0014] The accompanying drawings, which are included to provide a further understanding of this application and constitute a part of this application, illustrate the illustrative embodiments of this application and their descriptions, and are used to explain this application, but do not constitute an improper limitation of this application. In the accompanying drawings:

[0015] FIG1a is a system architecture diagram of a computer device carrying a secure container provided in an exemplary embodiment of this application;

[0016] FIG1b is another system architecture diagram of a computer device carrying a secure container provided in an exemplary embodiment of this application;

[0017] FIG2a is a schematic diagram of the interaction flow for state transition between a source node and a target node provided in an exemplary embodiment of this application;

[0018] FIG2b is another schematic diagram of the interaction flow for state transition between a source node and a target node provided in an exemplary embodiment of this application; Specification 2 / 27 pages 6 CN 121636047 A

[0019] Figure 3 is a flowchart illustrating a secure container migration method provided in an exemplary embodiment of this application;

[0020] Figure 4 is a flowchart illustrating another secure container migration method provided in an exemplary embodiment of this application;

[0021] Figures 5-6 are schematic diagrams illustrating a secure container migration device provided in an exemplary embodiment of this application;

[0022] Figure 7 is a schematic diagram illustrating a computer device provided in an exemplary embodiment of this application. Detailed Description

[0023] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0024] It should be noted that, in the case of user information involved in the embodiments of this application, the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in the embodiments of this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of related data need to comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entry points for users to choose to authorize or refuse. In addition, the various models involved in this application (including but not limited to language models or large models) comply with the relevant laws and standards.

[0025] With the deployment and application of AI applications and AI frameworks in secure containers, the requirements of secure containers for computing resources are getting higher and higher. Among them, AI frameworks refer to frameworks used to develop and deploy AI applications, such as TensorFlow (an open source machine learning platform), PyTorch (an open source machine learning library) or Caffe (a deep learning framework that combines expressiveness, speed and modular thinking); AI applications refer to applications developed and deployed based on AI frameworks, such as AI-based image processing programs, video processing programs or game programs, etc.

[0026] A secure container is a secure and isolated container runtime environment provided by virtualization technology. Unlike traditional container runtime environments, a secure container is a container runtime environment with a complete operating system (i.e., a guest operating system). Traditional container runtime environments do not have a guest operating system and rely on the host machine's operating system. Based on this difference, secure containers offer higher security and isolation compared to traditional containers.

[0027] For example, Figure 1a is a system architecture diagram of a computer device 1 carrying a secure container. This computer device 1 can be implemented as any electronic device, such as a mobile phone, tablet computer, calculator, or any type of server. In this embodiment...No restrictions are imposed. As shown in Figure 1a, the computer device 1 may include a host machine, which includes the host machine's user space 11 (also known as the host machine's user mode), the host machine's operating system 12 (also known as the host machine's kernel mode), and a hardware layer 13. The hardware layer 13 includes various hardware resources of the host machine, such as, but not limited to, a GPU 131, a CPU (Central Processing Unit) 132, and a network interface card (NIC). In Figure 1a, the NIC is implemented as an RDMA (Remote Direct Memory Access) NIC 133, but it is not limited to this; the NIC can also be implemented using a regular NIC. The host machine's operating system 12 refers to the operating system running on the host machine, which is the foundation of secure containers and virtualization technology. It may include drivers for various hardware resources on the host machine, as well as a virtualization manager. The virtualization manager can be, but is not limited to, KVM (Kernel-based Virtual Machine). Figure 1a illustrates this using KVM 121 as an example. KVM allows the host operating system 12 to become a virtual machine monitor, enabling users to run multiple virtualization instances on the host operating system. The host's user space 11 is used to host various virtualization instances running on the host, such as secure containers 111 and container runtime components 112.

[0028] Secure containers 111 can implement their core functions and features through container runtime components 112. Container runtime components 112 are a series of software components responsible for managing secure containers in a containerized environment. These components work together to provide functions such as creating, starting, stopping, and monitoring secure containers 111, and ensure that secure containers can run securely in an isolated environment. In this embodiment, the implementation method of container runtime components 112 is not limited; for example, but not limited to, Docker, RunC, RunD, etc., can be used. Docker is one of the more popular container runtimes, providing complete container management functionality. RunC is a lightweight container runtime that forms the basis for Docker and other container tools. RunD is a lightweight secure container solution that provides a secure environment for running containerized applications, designed to provide enhanced security and isolation to protect sensitive data and applications from potential threats. RunD comprises several key components, one of which is the RunD VMM (Virtual Machine Manager), a lightweight hypervisor used to manage and run virtual machine instances.

[0029] The secure container 111 may include: an application 1111 and a guest operating system 1112; the guest operating system 1112 in the secure container 111 represents an independent and isolated runtime environment, which may be a virtual machine, but is not limited thereto. The application 1111 may be implemented as the AI ​​framework or AI application mentioned above, or as any other type of application framework or application; this embodiment does not impose any limitations. Relative to the guest operating system 1112, the application 1111 may reside in the user space of the secure container 111. It should be noted that the user space 11 of the secure container 111 may include other components besides the application 1111.

[0030] The guest operating system 1112 refers to the operating system running inside the secure container 111, which can provide fine-grained access control functions, encrypted data storage functions, and a trusted execution environment, etc., to ensure that the applications and data inside the container are protected. The access control function can restrict the processes inside the container from accessing resources based on security policies, the encrypted data storage function can protect the data inside the container based on encryption technology, and the trusted execution environment can be used to provide a higher level of security protection. The guest operating system 1112 includes drivers for various hardware resources contained in the secure container, such as drivers that pass through to various hardware resources in the secure container using passthrough technology.

[0031] In this embodiment, introducing GPU resources into the secure container scenario can solve the problem of the secure container's demand for computing resources. It should be noted that, in addition to GPU resources, similar resources such as NPU (Neural Processing Unit) and DPU (Data Processing Unit) can also be used as computing resources for the secure container. In this embodiment, GPU, NPU, DPU and other resources are collectively referred to as target resource objects. First, target resource objects refer to hardware computing resources. Second, target resource objects refer to computing resources that do not support virtualization or for which there is currently no mature and standardized virtualization solution. For example, TPU (Tensor Processing Unit, a simple matrix multiplication coprocessor directly connected to the host motherboard) does not have built-in scheduling functions and therefore does not support virtualization. For example, GPUs are mainly responsible for handling complex computing tasks and have a large built-in graphics card memory. Although some virtualization solutions have emerged, most of them face challenges in resource allocation in multi-tenant environments and security issues between different tenant information. They are still immature and not standardized. In contrast, there are relatively mature and standardized virtualization solutions for resource objects such as CPU, network card, and memory. These resource objects are referred to as non-target resource objects.

[0032] In the embodiments of this application, the secure container (or the cloud vendor to which the secure container belongs) has certain management of resource objects.Control requirements refer to the management and control related to secure containers. For example, creating, destroying, and migrating secure containers all involve the management and control of related resource objects. The implementation method of managing resource objects at the secure container level differs depending on whether the resource object supports virtualization technology or whether a mature virtualization technology exists. For resource objects that support virtualization technology and have mature virtualization technology, such as CPU, network interface card (NIC), and memory, the cloud vendor owning the secure container can implement the management and control logic for these resource objects in the container runtime component 112 within the secure container, without relying on the drivers for these resource objects. In other words, the management and control logic for resource objects at the secure container level does not need to be implemented in the resource object drivers. Thus, the cloud vendor owning the secure container can implement the management and control logic for these resource objects independently, without relying on the vendor of the resource objects. For resource objects that do not support virtualization technology or lack mature virtualization technology, such as GPUs and TPUs, managing these resource objects from the secure container level requires relying on the resource object's driver, i.e., calling the relevant API interfaces exposed by the driver. Thus, the cloud vendor to which the secure container belongs cannot implement the management logic for these resource objects themselves and must rely on the device vendor's implementation of the driver.

[0033] Based on the above, the various resource objects involved in the secure container can be divided into two categories: target resource objects and non-target resource objects. Non-target resource objects refer to resource objects that can implement management logic in the container runtime component 112 within the secure container based on virtualization technology. Managing non-target resource objects from the secure container level does not depend on the driver for non-target resource objects. Target resource objects refer to resource objects that cannot implement management logic in the container runtime component 112 within the secure container based on virtualization technology. Managing target resource objects from the secure container level depends on the driver for the target resource object.

[0034] Because the target resource objects themselves do not support virtualization or the virtualization scheme is immature, target resource objects cannot be directly applied to the secure container 111 based on virtualization methods like resources such as CPUs and memory. This raises the primary question of how to apply the target resource object to the secure container 111. Therefore, this application provides two implementation methods:

[0035] Implementation Method 1: Extend the functionality of the driver for the target resource object (e.g., GPU) by adding a functional module (referred to as the virtualization functional module) for virtualizing the target resource object. This virtualization functional module divides the target resource object (e.g., GPU) into multiple virtual computing resources (virtual GPUs), and provides the divided virtual computing resources to...The secure container 111 uses the target resource object's driver, which resides in the host operating system 12 and needs to provide an API (Application Programming Interface). This allows the secure container 111 to obtain relevant status information of the target resource object by calling this API, providing conditions for the migration of the secure container 111. However, since the target resource object's driver resides in the host operating system, this involves interaction with the host kernel. This interaction process can increase computing power consumption, leading to wasted computing power. Furthermore, since the driver is a proprietary solution of the target resource object vendor, cloud providers find it difficult to control the technical details and extend the driver's functionality.

[0036] Implementation Method 2: A passthrough technology is used to directly connect the target resource object to the secure container 111, forming an overall architecture of secure container 111 + passthrough device. In the passthrough technology approach, the target resource object can be referred to as the passthrough device of the secure container 111. The passthrough technology allows the secure container 111 to directly access the target resource object, rather than indirectly accessing it through the host operating system 12. Because passthrough technology allows the secure container 111 to directly access the target resource object, it can reduce the overhead of the host operating system 12 and improve I / O (Input / Output) performance; at the same time, it can reduce the number of times data is copied between the host operating system 12 and the secure container 111, reduce latency and improve throughput.

[0037] The embodiments of this application do not limit the passthrough technology used, such as SR-IOV (Single Root I / O Virtualization, a virtualization technology) or PCIPassthrough (Peripheral Component Interconnect Passthrough). SR-IOV can be used to enable physical devices such as network adapters and GPUs to directly provide virtual functions (VFs) for multiple virtual machines. Each virtual function can be configured independently and can directly access the hardware resources of the host machine. PCIPassthrough can be used to pass physical devices such as network adapters and GPUs to the secure container, so that the secure container can directly access these physical devices. These physical devices can be called passthrough devices.

[0038] In some preferred embodiments, device passthrough can also be achieved based on VFIO (Virtual Function I / O) technology. VFIO is a module that enables passthrough for PCI devices on the host machine.A device refers to an external device that is interconnected with the host machine's CPU via a PCI (Peripheral Component Interconnect) bus or a PCIe (Peripheral Component Interconnect Express) bus. The computer device illustrated in Figure 1b is shown as an example of using VFIO to pass through a target resource object to a secure container. The difference between the computer device shown in Figure 1b and the computer device shown in Figure 1a is that the host machine's operating system 12 includes not only KVM 121 but also VFIO 122 and drivers for various hardware resources on the host machine; VFIO 122 can directly pass the target resource object (e.g., GPU) to the secure container, which is beneficial for improving the efficiency of the secure container's access to the GPU. Furthermore, the computer device shown in Figure 1b is illustrated using an AI framework and AI application services developed and deployed based on the AI ​​framework as examples. The user space 1110 of the client machine also includes the runtime library of the target resource object; in Figure 1b, the GPU runtime library is illustrated as an example.

[0039] The AI ​​framework can be PyTorch or Tensorflow, etc. PyTorch is an open-source machine learning library for building and training deep learning models, and Tensorflow is an open-source machine learning framework for building and training various types of machine learning models. AI application services can be inference services for large language models, image recognition services, natural language processing services, search advertising recommendation services, and function computation services, etc., and this embodiment does not impose any limitations.

[0040] When the target resource object is implemented as a passthrough device of the security container 111, the driver for the target resource object is located in the guest operating system 1112. Taking the target resource object as a GPU as an example, the guest operating system 1112 includes a GPU driver. Correspondingly, for non-target resource objects that are not implemented as a passthrough device of the security container 111, their drivers are located in the host operating system 12. Of course, the host operating system 12 also includes virtualization-related functions such as KVM and VFIO.

[0041] Through the above-described architecture of secure container + passthrough device, when the secure container interacts with the target resource object (such as GPU), it can effectively reduce the overhead of the host operating system and improve I / O (Input / Output) performance; at the same time, it can reduce the number of times data is copied between the host operating system and the secure container, reduce latency and improve throughput.

[0042] The following will describe in detail the method of state transition of the secure container under the above-described passthrough architecture, taking the interaction process between any two nodes as an example. For ease of description, the terms "source node" and "target node" will be used to describe any...The two nodes are described in this application. This application does not limit the implementation of the source node and the target node. Taking the host machine dimension as an example, the source node can be implemented as the host machine that needs state migration, and the target node can be implemented as another host machine; taking the GPU dimension as an example, the source node can be implemented as the GPU on the host machine that needs state migration, and the target node can be implemented as another GPU. The two GPUs can be located on the same host machine or on different host machines, without limitation.

[0043] Figure 2a is a schematic diagram of the interaction process for state migration between the source node and the target node, used to migrate the workload in the first secure container on the source node. The first secure container refers to the secure container on the source node that needs state migration. The workload in the first secure container mainly refers to the state data of the resource objects involved in the first secure container. Migrating the workload in the first secure container mainly refers to migrating the state of the resource objects included in the first secure container. The resource objects involved in the first secure container may include target resource objects managed by a driver and non-target resource objects not managed by a driver. Migrating the workload in the first secure container includes migrating the state data of the target resource object in the first secure container and migrating the state data of non-target resource objects in the first secure container.

[0044] For example, taking the GPU as the target resource object, the workload in the first secure container may include the GPU's state data (e.g., the video memory list and its corresponding video memory content, other states in the GPU), CPU's state data (e.g., CPU register states), storage and network state data (e.g., user temporary storage data, network addresses, disk states, etc.), memory state data (e.g., kernel data and user data of the secure container), and metadata of the secure container (e.g., management data of the secure container, information of the container engine layer, etc.); wherein, the GPU's state data belongs to the state data of the target resource object, and the CPU's state data, storage and network state data, memory state data, and metadata of the secure container belong to the state data of non-target resource objects. Based on this, the state data of the GPU, the state data of the CPU, the state data of the network, the state data of the memory, and the metadata of the secure container can be migrated from the first secure container to the second secure container, thereby completing the migration process of the workload in the first secure container.

[0045] As shown in Figure 2a, the state migration between the source node and the target node can be achieved based on the following steps:

[0046] Step S1: During the operation of the first secure container, the operation of the target resource object in the first secure container is paused through the driver of the target resource object.

[0047] Step S2: Obtain the state data of the target resource object of the first secure container through the driver of the target resource object.

[0048] Step S3: Instruct the creation of a second secure container on the target node; the second secure container includes target resource objects managed by the driver and non-target resource objects not managed by the driver.

[0049] Step S4: Migrate the state data of the target resource object of the first secure container to the target resource object of the second secure container.

[0050] Step S5: Receive the state data of the target resource object of the first secure container and save the state data of the target resource object of the first secure container to the target resource object of the second secure container.

[0051] Step S6: Pause the operation of the first secure container through the container runtime component corresponding to the first secure container.

[0052] Step S7: Obtain the state data of the non-target resource objects of the first secure container through the container runtime component corresponding to the first secure container.

[0053] Step S8: Migrate the state data of the non-target resource objects of the first secure container to the non-target resource objects of the second secure container.

[0054] Step S9: Receive the status data of the non-target resource object of the first secure container, and save the status data of the non-target resource object of the first secure container in the non-target resource object of the second secure container, so as to migrate the workload in the first secure container to the second secure container.

[0055] In this embodiment, the target resource object refers to a resource object that the cloud vendor or secure container cannot directly manage, but needs to rely on the driver for management / control. It is generally a hardware resource object that is directly connected to the secure container, such as GPU, DPU or NPU. Correspondingly, the target resource object refers to a resource object that the cloud vendor or secure container can directly manage, without relying on the program for management / control. It can be a hardware resource object such as CPU, memory or network card; it can also be a virtual resource object virtualized by virtualization technology, such as virtual CPU, virtual memory or virtual network card; it can also be a software resource object, such as engine information and node information involved in the Pod metadata of the secure container. Among them, Pod is the smallest scheduling unit in Kubernetes, which can be understood as a specific secure container cluster. The secure container cluster can contain one or more secure containers. Kubernetes, often abbreviated as k8s, is an open-source container orchestration platform used for automatically deploying, scaling, and operating containerized applications. Engine information typically refers to configuration information related to the Pod's runtime environment, while node information refers to metadata related to the node to which the Pod is scheduled.

[0056] In this embodiment, the state data of the target resource object is accessed through the client operation of the first secure container.The driver program for the target resource object in the system acquires and saves the data. Acquiring and saving the state data of the target resource object through the driver program (page 7 / 27, CN 121636047 A) depends on the normal operation of the first secure container and requires that the state data of the target resource object within the first secure container is stable, i.e., unchanged. Therefore, during the operation of the first secure container, the execution of the target resource object can be paused through the driver program, and the state data of the target resource object can be acquired while it is paused, thus maintaining the consistency of the target resource object's state data. The state data of the target resource object describes its current state. If the target resource object has video memory, its state data may include a video memory list, video memory content, and other state data. Other state data refers to state data other than the video memory list and its corresponding video memory content, such as at least one of the following: register information, working status information, and configuration information within the target resource object. The video memory list refers to the list of video memory regions allocated to different applications on the target resource object. Among them, register information is used to describe the current state of the register, such as the register value, ID (Identity document), or type, etc. Configuration information is used to describe various parameters and settings of the target resource object, such as enabling / disabling graphics card functions, performance settings, or display mode, etc. Working status information can be used to describe the working status of the target resource object, including at least one of the following: the ID of the target resource object, power status, task allocation, etc.

[0057] In this embodiment, a second security container can be created on the target node to migrate the workload of the first security container, but the timing of creating the second security container is not limited. For example, after obtaining the status data of the target resource object of the first security container, it can be instructed to create a second security container on the target node. In the embodiment shown in Figure 2, this implementation method is used as an example for illustration, but it is not limited to this. For another example, the second security container can also be created on the target node before migrating the workload in the first security container. The creation time of the second security container is not limited, as long as the creation of the second security container is completed before the workload migration to the second security container. The second security container also needs to include both driver-managed target resource objects and driver-independent non-target resource objects. It should be noted that the number and types of target resource objects contained in the second security container need to match the number and types of target resource objects contained in the first security container; this matching refers to the similarity in the types of target resource objects.Similarly, the number of target resource objects contained in the second secure container must be greater than or equal to the number of target resource objects that the first secure container needs to migrate, so as to successfully migrate the state data of the target resource objects in the first secure container to the target resource objects in the second secure container. Here, the second secure container refers to a secure container created on the target node. The use of "first" and "second" to define the secure container is only used to distinguish between the secure container on the source node and the secure container on the target node, and does not restrict the configuration or type of the secure container. This application does not limit the specific implementation method of creating the second secure container. In some exemplary embodiments, Kata Containers, etc., can be used to create the second secure container. Kata Containers is a lightweight virtualization technology that combines the convenience of containers with the security isolation characteristics of virtual machines, and can be used to create second secure containers. Furthermore, the second secure container can be created by the container runtime component on the source node instructing the target node to create it, or it can be created by the management node instructing the target node to create it; this is not limited. Based on the creation of the second secure container on the target node, the state data of the target resource objects in the first secure container can be migrated to the target resource objects in the second secure container.

[0058] Through the above methods, the state data of the target resource object can be further migrated during the operation of the first secure container by creating a secure container on the target node. The migration process of the state data of non-target resource objects will be described below.

[0059] Considering that the secure container can manage non-target resource objects without relying on the driver of the non-target resource object, but can be managed through the container runtime component, unlike the previous method of obtaining and saving the state data of the target resource object in the first secure container through the driver of the target resource object, the state data of non-target resource objects in the first secure container can be obtained and saved through the container runtime component. To obtain and save the state data of non-target resource objects in the first secure container through the container runtime component, the state data of the non-target resource objects in the first secure container needs to be in a stable state, that is, unchanged. Based on this, the operation of the first secure container can be paused through the container runtime component. Here, pausing the operation of the first secure container mainly refers to pausing the operation of the guest operating system in the first secure container. The container runtime component can first obtain the ID or name of the first secure container, and then, based on this identification information and the container pause command provided by the container runtime component, pause the operation of the first secure container.

[0060] After pausing the operation of the first secure container, the non-target information of the first secure container can be obtained through the container runtime component.The target resource object's status data. The non-target resource object's status data describes its working state and may include at least one of the following: currently stored data, storage space information, runtime space information, internal register information, and configuration information. Register information describes the current state of registers, such as their value, ID, or type. Configuration information describes various parameters and settings of the non-target resource object, such as CPU function switches, performance settings, or operating modes. After obtaining the status data of the non-target resource objects in the first security container, this data can be migrated to the non-target resource objects in the second security container. It should be noted that the number of non-target resource objects in the second security container must be greater than or equal to the number of non-target resource objects to be migrated from the first security container to ensure successful migration of the status data from the non-target resource objects in the first security container to the non-target resource objects in the second security container.

[0061] In this embodiment, under the pass-through architecture between the target resource object and the secure container, the target resource object of the first secure container on the source node depends on driver management, while the non-target resource object does not depend on driver management. Considering the different characteristics of the target resource object and the non-target resource object, different layers of components can cooperate to obtain the state data of the target resource object and the non-target resource object in the first secure container through the driver of the target resource object and the container runtime component. This allows for a relatively accurate migration of the state data of the target resource object and the non-target resource object to the second secure container on the target node. In this way, the rapid migration of workloads in the secure container can be effectively completed through the cooperation of different layers of components, solving the problem of state migration of secure containers with target resource objects that cannot be directly managed by the secure container or its cloud provider.

[0062] The secure container migration method in this embodiment is applicable to various scenarios. For example, in scenarios such as application upgrades, resource optimization, or application failures, the workload of the secure container on the source node can be efficiently migrated to the secure container on the target node using the above method. Specifically, in application upgrade scenarios, a smooth transition between old and new versions is crucial. In resource optimization scenarios, migrating workloads can achieve a more even load distribution, avoid the formation of hotspot nodes, and improve the overall system stability and response speed. In application failure scenarios, when a node fails, services can be restored by migrating workloads to other unaffected nodes, ensuring data security.

[0063] In some optional embodiments, the guest operating system of the first secure container includes a driver for the target resource object, which is used to interact with the target resource object.

[0064] In the first secure container, applications in the guest operating system can run in user space. In other words, user space refers to the environment in which applications run within the secure container, which can be understood as the place where applications run in the guest operating system. In this embodiment, the user space of the first secure container may include the runtime library of the target resource object. The runtime library of the target resource object is built on the driver of the target resource object. The runtime library encapsulates the functions provided by the driver and can serve as a bridge for the application to interact with the driver of the target resource object. That is, the runtime library of the target resource object can be used for the application to interact with the driver of the target resource object. In other words, the application can initiate various requests to the driver of the target resource object through the runtime library of the target resource object. When the application initiates a request through the runtime library of the target resource object, the request can be transformed into a call to the driver of the target resource object.

[0065] Taking the GPU as an example, the runtime library of the GPU can be CUDA (Compute Unified Device Architecture). CUDA can encapsulate the functions provided by the GPU. When an application sends a request through CUDA, the request can be translated into a call to the GPU driver. That is, the application can use CUDA to call the GPU driver to interact with the GPU.

[0066] Based on this, step S1 in the foregoing embodiment, "pausing the operation of the target resource object of the first security container through the driver of the target resource object", can be implemented based on the following implementation method one or implementation method two:

[0067] Implementation method one: intercepting the access request initiated by the application in the first security container using the interception function in the runtime library to pause the operation of the target resource object of the first security container. The access request is initiated by the application to the driver of the target resource object through the library function in the runtime library.

[0068] This embodiment does not limit the way the interception function is obtained. In some exemplary embodiments, a function configuration interface can be provided to the user, and the interception function input by the user can be obtained in response to the user's configuration operation on the function configuration interface. Alternatively, a preset interception function can be obtained from the storage space corresponding to the first security container in a preset way. This embodiment does not impose any restrictions.

[0069] In this embodiment, interception functions can be added in advance in the runtime library, but the specific implementation of the interception functions is not limited. Optionally, interception functions can be added in the runtime library code based on hook technology to intercept functions that need to be intercepted in the runtime library. Hook technology is used to intercept function behavior without modifying the original code.

[0070] Specifically, the target function to be intercepted can be selected from the library functions in the runtime library. The library functions in the runtime library refer to the set of functions provided in the runtime library. After selecting the target function, a hook method can be selected, such as the LD_PRELOAD method or the function replacement method, and the selected hook method is used to add the intercepted function to the code of the runtime library. The LD_PRELOAD method can load a custom dynamic link library by setting the LD_PRELOAD environment variable. This dynamic link library contains the intercepted function that replaces the original library function. The function replacement method can replace the original library function with the intercepted function by modifying the pointer of the library function.

[0071] The application in the first security container can initiate access requests to the driver of the target resource object through the library functions in the runtime library. Through the intercepted function in the runtime library, the above-mentioned driver access behavior of the application in the first security container can be effectively intercepted, so that the driver of the target resource object cannot be triggered by the application in the first security container, thereby achieving the purpose of suspending the operation of the target resource object in the first security container.

[0072] In this way, on the one hand, the execution of the target resource object of the first security container can be paused by adding an interception function to the runtime library in the pre-processing stage without relying on the driver. Since the interception function can be written according to actual design requirements, this method has high controllability.

[0073] Implementation method two: Call the driver of the target resource object to send a pause command to the target resource object of the first security container to pause the execution of the target resource object of the first security container. Specifically, the identification information of the target resource object can be obtained through the driver. This identification information is used to identify the identity of the target resource object. Then, the current status of the target resource object can be queried through the driver based on the identification information to ensure that it can be paused. The pause command is sent to the target resource object through the driver. The process of sending the pause command can be implemented as a function call process, such as calling PauseDevice() or similar operations. Among them, PauseDevice() is a function used to pause or stop the operation of a device. By calling the driver in this way, the execution of the target resource object can be paused relatively quickly. Instruction manual, page 10 / 27, 14 CN 121636047 A

[0074] Through the above embodiments one and two, the operation of the target resource object of the first security container can be suspended relatively efficiently by adding an interception function in the runtime library or by calling the driver.

[0075] After suspending the operation of the target resource object of the first security container through the driver of the target resource object, step S2 can be continued to obtain the status data of the target resource object of the first security container.

[0076] This application embodiment does not limit the method of obtaining the state data of the target resource object of the first secure container. In some exemplary embodiments, it can be implemented based on the following steps S21-S22:

[0077] Step S21: Call the target application interface in the driver of the target resource object to obtain the video memory list of the target resource object of the first secure container and its corresponding video memory content. Here, the target application interface refers to the interface used to manage the video memory usage. Here, the video memory list refers to the list of video memory regions allocated to different applications on the target resource object, which may include information about the video memory segment used by each application, such as the starting address, size, type and other relevant information of the video memory segment. The starting address of the video memory segment refers to the starting physical address or virtual address of the video memory segment; the type of the video memory segment may be VRAM (a type of video memory segment) or WRAM (a type of video memory segment), etc. The video memory segment is used to store the video memory content, which will vary depending on the application running in the first secure container. In the case of running an AI framework and AI application in the first secure container, the video memory content may be various data related to the machine learning model, etc. Other relevant information may include the status of the video memory segment, such as idle or in use, and may also include the priority of the video memory segment, which is not limited in this embodiment.

[0078] Optionally, before obtaining the video memory list of the target resource object of the first security container, the video memory list of the target resource object may also be determined based on the following method: First, a proxy function may be added to the runtime library to proxy the access request initiated by the application in the first security container to the driver of the target resource object through the library function in the runtime library. Among them, the proxy function may be added to the runtime library based on hook technology. The specific implementation method of this content has been described in detail in the previous section "Adding an interception function to the runtime library", and will not be repeated here. In this way, when the application in the first security container initiates an access request to the driver of the target resource object, it may not be initiated directly through the library function in the runtime library, but the proxy function may initiate the access request on its behalf. The proxy function can determine the request type of the access request. The request type may include memory allocation request, initialization request, and mode setting request, etc. The memory allocation request may contain a list of video memory requested by the application from the target resource object. When the access request is a memory allocation request, the access request can be initiated and the list of video memory requested by the application can be read from it as the video memory list of the target resource object of the first security container.

[0079] Step S22: Call other application interfaces in the driver of the target resource object other than the target application interface to obtain other state data of the target resource object of the first security container. Other state data refers to data other than video memory.The status data is excluding the list and its corresponding video memory content. There may be one or more other application programming interfaces (APIs), depending on the type and quantity of other status data. One API can be used to obtain one or more other status data; this embodiment does not impose any limitations. Other status data may include: internal register information, working status information, and configuration information.

[0080] The following will use the implementation of other APIs based on the NVML library as an example. The NVML library is a library used for monitoring and managing target resource objects. Specifically, the API implemented based on the nvmlDeviceGetName() function in the driver of the target resource object can be called to obtain the name of the target resource object; the API implemented based on the nvmlDeviceGetUtilizationRates() function in the driver of the target resource object can also be called to obtain the runtime utilization rate of the target resource object; this embodiment does not impose any limitations.

[0081] In this way, the status data of the target resource object of the first security container can be obtained more accurately by calling the API of the driver of the target resource object. Then, step S3 can be executed to create a second secure container on the target node, and step S4 can be executed to migrate the state data of the target resource object of the first secure container to the target resource object of the second secure container.

[0082] This application embodiment does not limit the specific implementation of the state data migration method in step S4. In some exemplary embodiments, the non-target resource object of the first secure container may include a regular network card. Based on this, the state data of the target resource object of the first secure container can be copied to the target resource object of the second secure container through the regular network card based on TCP / IP (Transmission Control Protocol / Internet Protocol). In some exemplary embodiments, the non-target resource object of the first secure container may include an RDMA network card. Based on this, an RDMA operation can be issued through the RDMA network card, and then the state data of the target resource object of the first secure container can be copied to the target resource object of the second secure container through the RDMA operation. RDMA network cards support RDMA technology, enabling direct memory access between different nodes without CPU intervention, thereby improving data transmission efficiency and reducing latency. Therefore, RDMA operations issued by RDMA network cards can efficiently migrate the state data of a target resource object in the first security container to the target resource object in the second security container. Furthermore, when the target resource object is implemented as a GPU...In this case, the process of migrating the state data of the target resource object through the RDMA network card can be called GDR (GPU Direct RDMA), that is, the state data is directly transmitted between the GPUs of the first security container and the second security container through the RDMA network, without the need for memory copying through the CPU, which can realize high-speed transmission of GPU state data.

[0083] Correspondingly, one implementation of step S4, which migrates the state data of the target resource object of the first security container to the target resource object of the second security container, includes: instructing the creation of a first virtual machine in the second security container and binding the target resource object of the second security container to the first virtual machine, and migrating the state data of the target resource object of the first security container to the target resource object bound to the first virtual machine; and instructing the creation of a second virtual machine in the second security container and binding the non-target resource object of the second security container to the second virtual machine, and binding the target resource object bound to the first virtual machine to the second virtual machine, so as to migrate the state data of the target resource object of the first security container to the target resource object of the second security container. Correspondingly, in step S5, the state data of the target resource object of the first security container is received, and the state data of the target resource object of the first security container is stored in the target resource object of the second security container. Specifically, step S5 can be executed based on the following steps S51-52:

[0084] Step 51: According to the instruction information, create a first virtual machine in the second secure container, bind the target resource object of the second secure container to the first virtual machine, and migrate the state data of the target resource object of the first secure container to the target resource object bound to the first virtual machine. The instruction information comes from the container runtime component or management node on the source node, and is used to instruct the creation of a first virtual machine in the second secure container and the binding of the target resource object of the second secure container to the first virtual machine, and the migration of the state data of the target resource object of the first secure container to the target resource object bound to the first virtual machine.

[0085] The first virtual machine can be created using virtualization tools such as Kata Containers and QEMU (Quick EMUlator), and this embodiment does not impose any restrictions. QEMU is an open-source virtual machine monitor and emulator that allows multiple virtual machines to run simultaneously on a host machine and provides management and control capabilities for these virtual machines. After creating the first virtual machine, a mapping relationship can be established between the target resource object of the second secure container and the first virtual machine. Then, based on this mapping relationship, the state data of the target resource object of the first secure container can be sent to the target resource object bound to the first virtual machine. In this way, the target resource object bound to the first virtual machine in the second secure container can carry the state data of the target resource object of the first secure container.

[0086] Step 52: According to the instruction information, create a second virtual machine in the second secure container, bind the non-target resource object of the second secure container (page 12 / 27, CN 121636047 A) to the second virtual machine, and bind the target resource object bound to the first virtual machine to the second virtual machine, so as to migrate the state data of the target resource object of the first secure container to the target resource object of the second secure container. This instruction information comes from the container runtime component or management node on the source node, and is used to instruct the creation of a second virtual machine in the second secure container and the binding of the non-target resource object of the second secure container to the second virtual machine, and the rebinding of the target resource object bound to the first virtual machine to the second virtual machine, so as to migrate the state data of the target resource object of the first secure container to the target resource object of the second secure container.

[0087] The second virtual machine can be created using virtualization tools such as Kata Containers, QEMU, or Hypervisor; this embodiment does not impose any limitations. After creating the second virtual machine, a mapping relationship can be established between the non-target resource object of the second secure container and the second virtual machine. Furthermore, based on this mapping relationship, the state data of non-target resource objects in the first secure container can be migrated to the non-target resource objects bound to the second secure container and the second virtual machine. Additionally, considering that both the first and second virtual machines run in the second secure container, the target resource objects bound to the first virtual machine can be bound to the second virtual machine, and the first virtual machine can be destroyed or paused to run the second virtual machine in the second secure container. Since the target resource objects in the second secure container are bound to the second virtual machine, it means that the state data of the target resource objects in the first secure container are migrated to the target resource objects bound to the second virtual machine in the second secure container.

[0088] In this way, on the one hand, a mapping relationship can be established between the target resource objects and non-target resource objects of the second secure container and the second virtual machine, thereby ensuring the consistency of resources between the second virtual machine and the second secure container. That is, the second virtual machine can not only access the target resource objects of the second secure container, but also access the non-target resource objects of the second secure container. On the other hand, the creation and destruction of virtual machines can ensure that a virtual machine runs in the second secure container, thereby improving the isolation and security of the virtual machine and the correctness of the data.

[0089] In addition to the above embodiments, this application also provides an implementation method in which the state data of the target resource object of the first security container is stored in the target resource object of the second security container. Specifically, see steps S23, S24 and S30 in FIG2b. The security container migration method shown in FIG2b includes:

[0090] Step S21: During the operation of the first security container, the first security container is paused through the driver of the target resource object.The target resource object of the secure container runs.

[0091] Step S22: Obtain the state data of the target resource object of the first secure container through the driver of the target resource object.

[0092] Step S23: Instruct the creation of a third secure container on the target node; the third secure container includes target resource objects managed by the driver.

[0093] Step S24: Migrate the state data of the target resource object of the first secure container to the target resource object of the third secure container.

[0094] Step S25: Receive the state data of the target resource object of the first secure container and save the state data of the target resource object of the first secure container in the target resource object of the third secure container.

[0095] Step S26: Pause the operation of the first secure container through the container runtime component corresponding to the first secure container.

[0096] Step S27: Obtain the state data of the non-target resource objects of the first secure container through the container runtime component corresponding to the first secure container.

[0097] Step S28: Instruct the creation of a second secure container on the target node; the second secure container includes non-target resource objects not managed by the driver.

[0098] Step S29: Migrate the state data of the non-target resource object in the first security container to the non-target resource object in the specification of the second security container (page 13 / 27, CN 121636047 A).

[0099] Step S30: Transfer the target resource object of the third security container to the second security container to migrate the workload in the first security container to the second security container.

[0100] For details regarding steps S21-S22, S26-S27, S29, etc., please refer to the description of the corresponding steps in the embodiment shown in Figure 2a, which will not be repeated here.

[0101] In this embodiment, two security containers are created on the target node, namely a third security container and a second security container. The third security container includes a target resource object, and the second security container includes a non-target resource object. First, the state data of the target resource object in the first security container is stored in the target resource object of the third security container, and the state data of the non-target resource object in the first security container is migrated to the non-target resource object of the second security container. Then, the target resource object in the third security container is transferred to the second security container to migrate the workload in the first security container to the second security container. Transferring the target resource object in the third security container to the second security container means remounting the target resource object mounted on the third security container to the second security container and removing the mounting relationship between the target resource object and the third security container.

[0102] Furthermore, in this embodiment, the order in which the second and third security containers are created is not limited.The time frame can be pre-created or created in real time when state data migration is required; there is no limitation on this, as long as the creation of the security container is completed before using the corresponding security container. Similarly, the third security container can be created by the container runtime component on the source node instructing the target node to create it, or it can be created by the control node instructing the target node to create it; there is no limitation on this. Regardless of which object instructs the creation of the second and third security containers, the creation process of the second and third security containers, as well as the process of mounting related resource objects to the second or third security containers, are all completed by the container runtime component on the target node. These specific processes are not limited or concerned in this embodiment; some existing technologies can be referred to, or a self-developed solution can be adopted.

[0103] In this way, by creating a new security container on the target node, the state data of the target resource object of the first security container can be saved to the target resource object of the second security container more efficiently through the new security container.

[0104] Further optionally, as shown in FIG2b, after step S30, the following steps may be included:

[0105] Step S31: Stop the operation of the first secure container; specifically, the operation of the first secure container can be stopped by the container runtime component corresponding to the first secure container on the source node, where stopping means shutting down the first secure container.

[0106] Step S32: Stop the operation of the third secure container; specifically, the operation of the third secure container can be stopped by the container runtime component corresponding to the third secure container on the target node, where stopping means shutting down the third secure container.

[0107] Step S33: Restore the operation of the second secure container by the container runtime component corresponding to the second secure container in the target node, and restore the operation of the target resource object by the driver of the target resource object. The driver of the target resource object is located in the guest operating system of the second secure container. Specifically, the container runtime component can call the driver of the target resource object to restore the operation of the target resource object by the driver of the target resource object. For details on restoring the operation of the target resource object, please refer to the descriptions in subsequent steps S11 and S12, which will not be elaborated here.

[0108] After migrating the state data of the target resource object of the first security container to the target resource object of the second security container based on the above embodiments, step S6 can be executed to pause the operation of the first security container through the container runtime component. By pausing the operation of the first security container, the state data of the non-target resource objects of the first security container can be kept unchanged; in other words, pausing the operation of the first security container is intended to prevent the state data of the non-target resource objects from changing when step S7 is executed. The relevant content of this state data has been described in detail above, and will not be repeated here.

[0109] This application embodiment does not describe step S7 "obtaining the state data of the target resource object through the container runtime component corresponding to the first security container".Page 14 / 27 of the book, CN 121636047 A, defines the specific implementation of "state data of non-target resource objects of a secure container". In some exemplary embodiments, the container runtime component can obtain and save the state data of non-target resource objects of the first secure container based on the state saving tool in the user space of the first secure container. Alternatively, in some other exemplary embodiments, the user can manually save and restore the state data of non-target resource objects of the first secure container. For the container runtime component, it can respond to the user's save operation, obtain and save the state data of non-target resource objects of the first secure container.

[0110] The first secure container state saving tool is a secure container persistence tool implemented based on checkpoint-restore technology, which is a technology for real-time state saving and restoration of containers. This technology allows a container to save its runtime state data in one location (called a checkpoint) and restore that state data in another location (called a restore). In short, the state saving tool can be used to save the runtime state of a secure container and can be quickly restored to that runtime state in subsequent processes. This embodiment does not limit the specific implementation of the state saving tool. In some exemplary embodiments, the state saving tool can be implemented as a CRIU (Checkpoint / Restore In Userspace) tool. The CRIU tool is a tool for implementing process state snapshots and restoration, which can be used to save the state data of resource objects in user space and restore the state data again when needed.

[0111] Based on the above, the container runtime component can call a state saving tool, which can save the state data of the non-target resource objects of the first secure container to the memory of the source node. Specifically, the container runtime component can call the state saving tool by sending a call instruction to it. The state saving tool can take a state snapshot of the non-target resource objects of the first secure container to obtain the corresponding state data and save the state data to the memory of the source node.

[0112] In this way, the state data of the non-target resource objects of the first secure container can be obtained more efficiently based on the state saving tool.

[0113] After obtaining the state data of the non-target resource objects of the first secure container through the container runtime component, step S8 or step S28 can be executed to migrate the state data of the non-target resource objects of the first secure container to the non-target resource objects of the second secure container. In some exemplary embodiments, the non-target resource objects of the first secure container canIncluding ordinary network cards, based on this, the state data of the non-target resource objects of the first security container can be copied to the non-target resource objects of the second security container via TCP / IP using ordinary network cards. Optionally, if the non-target resource objects of the first and second security containers contain RDMA network cards, the state data of the non-target resource objects of the first security container can be migrated to the non-target resource objects of the second security container via RDMA operations. Since RDMA operations can directly access memory between different nodes without CPU intervention, it is beneficial to improve data transmission efficiency and reduce transmission latency. Therefore, the state data of the non-target resource objects of the first security container can be migrated to the non-target resource objects of the second security container efficiently and quickly.

[0114] In some optional embodiments, after the target node receives the state data of the target resource objects and non-target resource objects of the first security container and saves the state data to the second security container, that is, after completing the migration of the workload of the first security container, the working state of the second security container can be further restored based on the state data received by the second security container.

[0115] This application embodiment does not limit the specific implementation method of restoring the working state of the second secure container. In some exemplary embodiments, the working state of the second secure container can be restored based on the following steps S10-S13:

[0116] Step S10: Run the second secure container created in the aforementioned step S3 or step S27.

[0117] Step S11: Restore the working state of the non-target resource object according to the state data in the non-target resource object of the second secure container.

[0118] Step S12: Restore the working state of the target resource object according to the state data in the target resource object of the second secure container.

[0119] In some optional embodiments, the user space of the second secure container may include a state restoration tool, which corresponds to the state saving tool in the user space and is also implemented based on checkpoint-restore technology. Based on the above, the container runtime component can invoke the state recovery tool by sending an invocation command to it. The state recovery tool can provide the state data of the non-target resource object in the second security container to the non-target resource object, allowing the non-target resource object to restore its working state based on the state data. Alternatively, in some other exemplary embodiments, the user can manually restore the state data of the non-target resource object in the second security container. The container runtime component can respond to the user's restoration operation by adjusting the working state of the non-target resource object based on the state data of the non-target resource object in the second security container.

[0120] In some optional embodiments, the second secure container may include a driver for the target resource object, and the state data of the target resource object may include a video memory list and its corresponding video memory content and other state data, which will not be elaborated here. Based on this, the target application interface in the driver of the target resource object can be called to provide the video memory list and its corresponding video memory content in the target resource object of the second secure container to the target resource object of the second secure container. Here, the target application interface refers to the interface used to manage the video memory usage. The relevant content about the video memory list has been described in detail above, and will not be repeated here.

[0121] Afterwards, other application interfaces in the driver of the target resource object can be called to provide other state data in the target resource object of the second secure container to the target resource object of the second secure container to restore the working state of the target resource object. Here, there may be one or more other application interfaces, which can be determined according to the number of other state data types. One application interface can be used to obtain one or more other state data, which is not limited in this embodiment.

[0122] In this way, the target node can restore the working state of the second secure container more efficiently based on the received state data after completing the state data migration.

[0123] Based on the above embodiments, in addition to the source node and the target node, a management node may also be provided in this embodiment to manage and control the source node and the target node. The management node may issue a container migration task to the source node, which is used to instruct the workload of the first secure container on the source node to be migrated to the second secure container on the target node. Optionally, the management node may respond to a task issuance event and issue a container migration task to the source node. The task issuance event may be implemented as detecting a fault in the source node, detecting a fault in the first secure container on the source node, or detecting that the workload of the first secure container on the source node exceeds a preset range. The preset range can be set to any range according to actual design requirements, and this embodiment does not impose any restrictions.

[0124] In this embodiment, the management node may issue a container migration task to the source node in a timely manner, which is used to instruct the workload of the first secure container on the source node to be migrated to the second secure container on the target node.

[0125] Following the above embodiments, the secure container migration method provided in this application can be applied to intelligent computing service scenarios. The intelligent computing service can be deployed as an application in the user space of the first secure container in the above embodiments. Specifically, the user can deploy the first secure container carrying the intelligent computing service on the source node through a management node, and directly connect the GPU on the source node to the first secure container as the computing power resource for the intelligent computing service. In this scenario, the intelligent computing service in the first secure container runs on the GPU. During the operation of the intelligent computing service, the following situations may occur:When a user needs to upgrade the intelligent computing service, or when the intelligent computing service malfunctions, or when the source node fails, the intelligent computing service needs to be migrated, i.e., the workload of the first secure container needs to be migrated. As described on pages 16 / 27 of this specification (20 CN 121636047 A), the control node can determine the target node to which the migration needs to be performed and issue a state migration command to the source node, indicating that the first secure container on the source node needs to be migrated to the second secure container on the target node. Optionally, the control node can notify the target node to create the second secure container, or the source node can notify the target node to create the second secure container; the second secure container can be created in advance or in real time when state data migration is required. The source node executes the migration steps described in the above method embodiments according to the state migration instructions, migrating the intelligent computing service from the first secure container of the source node to the second secure container of the target node. Then, based on the various state data migrated, the states of GPU, CPU, network, memory, etc. in the second secure container are restored to ensure that the states of GPU, CPU, storage, and network before and after the migration remain consistent. The operation of the intelligent computing service can be restored in the second secure container, enabling the intelligent computing service to continue to provide services to customers.

[0126] The intelligent computing service scenario is only one application scenario of the secure container migration method provided in the embodiments of this application, and is not limited thereto. For example, the secure container migration method provided in the embodiments of this application can also be applied to the Function Compute service scenario. The Function Compute service can be deployed as an application in the user space of the first secure container as described in the above embodiments. Function Compute provides a way to execute code on demand. Users only need to write and upload function code, without having to worry about the maintenance and expansion of the underlying infrastructure. The migration process of the secure container in this Function Compute application scenario is the same as in the above embodiments, and will not be described again.

[0127] For example, the secure container migration method provided in this application embodiment can also be applied to the PAI (Platform for AI) scenario. The AI ​​platform service can be deployed as an application in the user space of the first secure container as described in the above embodiments. The AI ​​platform service is used for target users to build, train, deploy, and manage artificial intelligence and machine learning models. The migration process of the secure container in this PAI application scenario is the same as in the above embodiments, and will not be repeated here.

[0128] Following the above embodiments, this application embodiment also provides a secure container migration method for migrating workloads in a first secure container on a source node. The first secure container includes a target resource object and a non-target resource object, wherein the target resource object is a pass-through device of the first secure container. As shown in Figure 3, the method includes:

[0129] Step 31: During the operation of the first secure container, the operation of the target resource object of the first secure container is paused through the driver of the target resource object, and the status data of the target resource object of the first secure container is obtained.

[0130] Step 32: The status data of the target resource object of the first secure container is migrated to the target resource object of the second secure container, where the second secure container is a secure container on the target node.

[0131] Step 33: The operation of the first secure container is paused through the container runtime component corresponding to the first secure container, and the status data of the non-target resource objects of the first secure container is obtained.

[0132] Step 34: The status data of the non-target resource objects of the first secure container is migrated to the non-target resource objects of the second secure container, so as to migrate the workload in the first secure container to the second secure container.

[0133] Further optionally, the first secure container includes a guest operating system, the guest operating system includes a driver for the target resource object, and the user space of the first secure container includes a runtime library for the target resource object. Pausing the operation of the target resource object in the first secure container via the driver for the target resource object includes: intercepting access requests initiated by applications in the first secure container using interception functions in the runtime library to pause the operation of the target resource object in the first secure container; the access request is initiated by the application through library functions in the runtime library to the driver for the target resource object; or, calling the driver for the target resource object to send a pause command to the target resource object in the first secure container to pause the operation of the target resource object in the first secure container.

[0134] Further optionally, by using the driver of the target resource object, obtaining the status data of the target resource object of the first secure container (page 17 / 27, CN 121636047 A) includes: calling the target application interface in the driver of the target resource object to obtain the video memory list and its corresponding video memory content of the target resource object of the first secure container; calling other application interfaces in the driver of the target resource object besides the target application interface to obtain other status data of the target resource object of the first secure container, wherein the other status data refers to status data other than the video memory list and its corresponding video memory content.

[0135] Further optionally, the user space of the first secure container includes a runtime library of the target resource object, and the method further includes: using a proxy function in the runtime library to proxy the access request initiated by the application in the first secure container, and when the access request is a memory allocation request, obtaining the video memory list requested by the application as the video memory list of the target resource object of the first secure container; the access request is initiated by the application to the driver of the target resource object through the library function in the runtime library.

[0136] Optionally, the user space of the first secure container includes a state saving tool. Through the container runtime component corresponding to the first secure container, the first secure container is paused, and the state data of the non-target resource objects of the first secure container is obtained. This includes: the container runtime component pausing the first secure container to keep the state data of the non-target resource objects of the first secure container unchanged; and the container runtime component calling the state saving tool to obtain the state data of the non-target resource objects of the first secure container and saving it to the memory of the source node.

[0137] Further optionally, creating a second security container on the target node and migrating the state data of the target resource object of the first security container to the target resource object of the second security container includes: creating a third security container on the target node and migrating the state data of the target resource object of the first security container to the target resource object of the third security container; creating a second security container on the target node and transferring the target resource object of the third security container to the second security container to migrate the state data of the target resource object of the first security container to the target resource object of the second security container; or creating a first virtual machine in the second security container, binding the target resource object of the second security container to the first virtual machine, and migrating the state data of the target resource object of the first security container to the target resource object bound to the first virtual machine; creating a second virtual machine in the second security container, binding the non-target resource object of the second security container to the second virtual machine, and binding the target resource object bound to the first virtual machine to the second virtual machine to migrate the state data of the target resource object of the first security container to the target resource object of the second security container.

[0138] Further optionally, the non-target resource object of the first secure container includes an RDMA network card; migrating the state data of the target resource object of the first secure container to the target resource object of the second secure container includes: copying the state data of the target resource object of the first secure container to the target resource object of the second secure container through RDMA operation; migrating the state data of the non-target resource object of the first secure container to the non-target resource object of the second secure container includes: copying the state data of the non-target resource object of the first secure container to the non-target resource object of the second secure container through RDMA operation.

[0139] Further optionally, the method further includes: receiving a container migration task issued by the management node, the container migration task being used to instruct the migration of the workload of the first secure container on the source node to the second secure container on the target node.

[0140] The detailed implementation methods and beneficial effects of each step in the method of this embodiment have been described in detail in the foregoing embodiments, and will not be elaborated here.

[0141] Following the above embodiments, this application embodiment also provides a secure container migration method for migrating the state data of the non-target resource object of the first secure container on the source node.The workload of the first secure container is migrated, and a second secure container is created on the target node. The second secure container includes target resource objects that depend on driver management and non-target resource objects that do not depend on driver management. As shown in Figure 4, the method includes:

[0142] Step 42: Receive the status data of the target resource objects of the first secure container and save the status data of the target resource objects of the first secure container in the target resource objects of the second secure container;

[0143] Step 43: Receive the status data of the non-target resource objects of the first secure container and save the status data of the non-target resource objects of the first secure container in the non-target resource objects of the second secure container.

[0144] Further optionally, as shown in Figure 4, before step 42, the method further includes: Step 41: Create a second secure container on the target node. The second secure container includes target resource objects that depend on driver management and non-target resource objects that do not depend on driver management. It should be noted that this embodiment does not limit the timing of creating the second secure container, and Figure 4 is only an example.

[0145] Further optionally, the method further includes: running a second secure container; restoring the working state of the non-target resource object based on the state data in the non-target resource object of the second secure container; and restoring the working state of the target resource object based on the state data in the target resource object of the second secure container.

[0146] Further optionally, the user space of the second secure container includes: a state recovery tool, which restores the working state of the non-target resource object based on the state data in the non-target resource object of the second secure container, including: calling the state recovery tool to provide the state data in the non-target resource object of the second secure container to the non-target resource object to restore the working state of the non-target resource object.

[0147] Further optionally, the second secure container includes a driver for the target resource object, and the state data of the target resource object includes a video memory list and its corresponding video memory content and other state data; restoring the working state of the target resource object according to the state data in the target resource object of the second secure container includes: calling a target application interface in the driver of the target resource object to provide the video memory list and its corresponding video memory content in the target resource object of the second secure container to the target resource object of the second secure container; calling other application interfaces in the driver of the target resource object other than the target application interface to provide other state data in the target resource object of the second secure container to the target resource object of the second secure container to restore the working state of the target resource object, wherein the other state data refers to state data other than the video memory list and its corresponding video memory content.

[0148] Further optionally, the state data of the target resource object of the first secure container is stored in the second secure container.The target resource object includes: migrating the state data of the target resource object of the first security container to the target resource object of the third security container, wherein the third security container is a security container on the target node; when the second security container is created, transferring the target resource object of the third security container to the second security container to migrate the state data of the target resource object of the first security container to the target resource object of the second security container.

[0149] Further optionally, before creating the second security container on the target node, the method further includes: creating the third security container on the target node.

[0150] Further optionally, storing the state data of the target resource object of the first security container in the target resource object of the second security container includes: creating a first virtual machine in the second security container, binding the target resource object of the second security container to the first virtual machine, and migrating the state data of the target resource object of the first security container to the target resource object bound to the first virtual machine; creating a second virtual machine in the second security container, binding the non-target resource object of the second security container to the second virtual machine, and binding the target resource object bound to the first virtual machine to the second virtual machine to migrate the state data of the target resource object of the first security container to the target resource object of the second security container.

[0151] The detailed implementation methods and beneficial effects of each step in the method of this embodiment have been described in detail in the foregoing embodiments, and will not be elaborated here.

[0152] It should be noted that the execution subject of each step of the method provided in the above embodiments can be the same device, or the method can be executed by different devices. For example, the execution subject of steps 31 to 34 can be the first device; for example, the execution subject of steps 31 and 32 can be the first device, and the execution subject of steps 33 and 34 can be the second device; and so on.

[0153] In addition, in some processes described in the above embodiments and figures, multiple operations appear in a specific order. However, it should be clearly understood that these operations may not be executed in the order they appear in this document or may be executed in parallel. The sequence numbers of the operations, such as 31, 32, etc., are only used to distinguish the different operations, and the sequence number itself does not represent any execution order. In addition, these processes may include more or fewer operations, and these operations may be executed sequentially or in parallel. It should be noted that the descriptions such as "first" and "second" in this article are used to distinguish different messages, devices, modules, etc., and do not represent a chronological order, nor do they limit "first" and "second" to different types.

[0154] Figure 5 is a structural schematic diagram of a secure container migration device provided in another exemplary embodiment of this application, usingThe device is used to migrate workloads in a first secure container on a source node. The first secure container includes a target resource object and non-target resource objects, where the target resource object is a pass-through device of the first secure container. As shown in Figure 5, the device includes:

[0155] a first pause module 501, used to pause the operation of the target resource object of the first secure container during its operation by using the driver of the target resource object, and to obtain the status data of the target resource object of the first secure container; a first migration module 502, used to create a second secure container on the target node and migrate the status data of the target resource object of the first secure container to the target resource object of the second secure container; a second pause module 503, used to pause the operation of the first secure container by using the container runtime component corresponding to the first secure container, and to obtain the status data of the non-target resource object of the first secure container; and a second migration module 504, used to migrate the status data of the non-target resource object of the first secure container to the non-target resource object of the second secure container, so as to migrate the workload in the first secure container to the second secure container.

[0156] Further optionally, the first secure container includes a guest operating system, the guest operating system includes a driver for the target resource object, and the user space of the first secure container includes a runtime library for the target resource object. The first pause module 501 pauses the runtime of the target resource object in the first secure container through the driver for the target resource object, specifically by: intercepting access requests initiated by applications in the first secure container using interception functions in the runtime library to pause the operation of the target resource object in the first secure container; the access request is initiated by the application through library functions in the runtime library to the driver for the target resource object; or, calling the driver for the target resource object to send a pause command to the target resource object in the first secure container to pause the operation of the target resource object in the first secure container.

[0157] Further optionally, when the first pause module 501 obtains the status data of the target resource object in the first secure container through the driver for the target resource object, it is specifically used to: call the target application interface in the driver for the target resource object to obtain the video memory list and its corresponding video memory content of the target resource object in the first secure container; call other application interfaces in the driver for the target resource object other than the target application interface to obtain other status data of the target resource object in the first secure container, wherein the other status data refers to status data other than the video memory list and its corresponding video memory content.

[0158] Further optionally, the user space of the first secure container includes a runtime library for the target resource object, and the first pause module 501 is further configured to: use proxy functions in the runtime library to proxy the application in the first secure container that initiates the request.Access requests are made, and if the access request is a memory allocation request, the list of video memory requested by the application is obtained as the list of video memory of the target resource objects of the first secure container. The access request is initiated by the application through a library function in the runtime library to the driver of the target resource object.

[0159] Further optionally, the user space of the first secure container includes a state saving tool. When the second pause module 503 pauses the operation of the first secure container through the container runtime component corresponding to the first secure container and obtains the state data of the non-target resource objects in the first secure container specification (pages 20 / 27, CN 121636047 A), it is specifically used for: the container runtime component pausing the operation of the first secure container to keep the state data of the non-target resource objects of the first secure container unchanged; the container runtime component calling the state saving tool to obtain the state data of the non-target resource objects of the first secure container and saving it to the memory of the source node.

[0160] Further optionally, when the first migration module 502 creates a second secure container on the target node and migrates the state data of the target resource object of the first secure container to the target resource object of the second secure container, it is specifically used for: creating a third secure container on the target node and migrating the state data of the target resource object of the first secure container to the target resource object of the third secure container; creating a second secure container on the target node and transferring the target resource object of the third secure container to the second secure container, so as to migrate the state data of the target resource object of the first secure container to the target resource object of the second secure container; or creating a first virtual machine in the second secure container, binding the target resource object of the second secure container to the first virtual machine, and migrating the state data of the target resource object of the first secure container to the target resource object bound to the first virtual machine; creating a second virtual machine in the second secure container, binding the non-target resource object of the second secure container to the second virtual machine, and binding the target resource object bound to the first virtual machine to the second virtual machine, so as to migrate the state data of the target resource object of the first secure container to the target resource object of the second secure container.

[0161] Further optionally, the non-target resource object of the first security container includes an RDMA network card; when the first migration module 502 migrates the state data of the target resource object of the first security container to the target resource object of the second security container, it is specifically used to: copy the state data of the target resource object of the first security container to the target resource object of the second security container through RDMA operation; when the second migration module 504 migrates the state data of the non-target resource object of the first security container to the non-target resource object of the second security container, it is specifically used to: migrate the state data of the non-target resource object of the first security container to the non-target resource object of the second security container through RDMA operation.

[0162] Further optionally, the second migration module 504 is also used to: receive a container migration task issued by the control node, the container migration task being used to instruct the migration of the workload of the first secure container on the source node to the second secure container on the target node.

[0163] FIG6 is a schematic diagram of another secure container migration device provided in another exemplary embodiment of the present application, used to migrate the workload of the first secure container on the source node. The secure container migration device can be implemented in the target node, wherein a second secure container is created on the target node, the second secure container including a target resource object that depends on driver management and a non-target resource object that does not depend on driver management. As shown in FIG6, the device includes:

[0164] a first storage module 602, used to receive the status data of the target resource object of the first secure container and store the status data of the target resource object of the first secure container in the target resource object of the second secure container; a second storage module 603, used to receive the status data of the non-target resource object of the first secure container and store the status data of the non-target resource object of the first secure container in the non-target resource object of the second secure container.

[0165] Further optionally, as shown in FIG6, the device further includes: a creation module 601, used to create a second secure container on the target node.

[0166] Further optionally, the creation module 601 is also used to: run the second secure container; restore the working state of the non-target resource object according to the state data in the non-target resource object of the second secure container; and restore the working state of the target resource object according to the state data in the target resource object of the second secure container.

[0167] Further optionally, the user space of the second secure container includes: a state recovery tool. When the creation module 601 restores the working state of the non-target resource object according to the state data in the non-target resource object of the second secure container, it is specifically used to: call the state recovery tool to provide the state data in the non-target resource object of the second secure container to the non-target resource object to restore the working state of the non-target resource object. Specification page 21 / 27 25 CN 121636047 A

[0168] Further optionally, the second security container includes a driver for the target resource object, and the state data of the target resource object includes a video memory list and its corresponding video memory content and other state data; when the creation module 601 restores the working state of the target resource object according to the state data in the target resource object of the second security container, it is specifically used to: call the target application interface in the driver of the target resource object to provide the video memory list and its corresponding video memory content in the target resource object of the second security container to the target resource object of the second security container; call other application interfaces in the driver of the target resource object besides the target application interface to provide the video memory list and its corresponding video memory content in the target resource object of the second security container to the target resource object of the second security container.Other state data is provided to the target resource object of the second security container to restore the working state of the target resource object. The other state data refers to state data other than the video memory list and its corresponding video memory content.

[0169] Further optionally, the first saving module 602 saves the state data of the target resource object of the first security container in the target resource object of the second security container, including: migrating the state data of the target resource object of the first security container to the target resource object of the third security container, wherein the third security container is a security container on the target node; when the second security container is created, the target resource object of the third security container is transferred to the second security container to migrate the state data of the target resource object of the first security container to the target resource object of the second security container.

[0170] Further optionally, before creating the second security container on the target node, the creation module 601 is also used to: create the third security container on the target node.

[0171] Further optionally, when the first storage module 602 stores the state data of the target resource object of the first secure container in the target resource object of the second secure container, it is specifically used for: creating a first virtual machine in the second secure container, binding the target resource object of the second secure container to the first virtual machine, and migrating the state data of the target resource object of the first secure container to the target resource object bound to the first virtual machine; creating a second virtual machine in the second secure container, binding the non-target resource object of the second secure container to the second virtual machine, and rebinding the target resource object bound to the first virtual machine to the second virtual machine, so as to migrate the state data of the target resource object of the first secure container to the target resource object of the second secure container.

[0172] The internal functions and structure of the secure container migration device have been described above. As shown in FIG7, in practice, the secure container migration device can be implemented as a computer device, which can be implemented as the source node in the above embodiments, including: a memory 701, a processor 702, and a communication component 703.

[0173] The memory 701 is used to store computer programs and can be configured to store various other data to support operations on the computer device. Examples of this data include instructions for any application or method operating on a computer device, contact data, phone book data, messages, pictures, videos, etc. Optionally, memory 701 stores program code related to the first secure container and program code related to the container runtime component.

[0174] In some embodiments, processor 701, coupled to memory 702, is used to execute the computer program in memory 702 (here, the computer program mainly refers to the program code related to the container runtime component on the source node) for: during the operation of the first secure container, suspending the target resource of the first secure container through the driver of the target resource object.The source object runs and obtains the status data of the target resource object of the first secure container; it instructs the creation of a second secure container on the target node, and migrates the status data of the target resource object of the first secure container to the target resource object of the second secure container; through the container runtime component corresponding to the first secure container, it pauses the operation of the first secure container and obtains the status data of the non-target resource objects of the first secure container; it migrates the status data of the non-target resource objects of the first secure container to the non-target resource objects of the second secure container, so as to migrate the workload in the first secure container to the second secure container.

[0175] Further optionally, the first secure container includes a guest operating system, the guest operating system includes a driver for the target resource object, and the user space of the first secure container includes a runtime library for the target resource object. The processor 701, through the driver for the target resource object, suspends the runtime of the target resource object in the first secure container by: adding an interception function in the runtime library to intercept access requests initiated by the application in the first secure container to the driver for the target resource object through library functions in the runtime library, so as to suspend the runtime of the target resource object in the first secure container; or, calling the driver for the target resource object to send a pause command to the target resource object in the first secure container, so as to suspend the runtime of the target resource object in the first secure container.

[0176] Further optionally, when the processor 701 obtains the state data of the target resource object of the first secure container through the driver of the target resource object, it is specifically used to: call the target application interface in the driver of the target resource object to obtain the video memory list of the target resource object of the first secure container and its corresponding video memory content; call other application interfaces in the driver of the target resource object other than the target application interface to obtain other state data of the target resource object of the first secure container, wherein the other state data refers to state data other than the video memory list and its corresponding video memory content.

[0177] Further optionally, the user space of the first secure container includes a runtime library of the target resource object, and the processor 701 is further used to: add a proxy function in the runtime library to proxy the access request initiated by the application in the first secure container to the driver of the target resource object through the library function in the runtime library, and obtain the video memory list requested by the application as the video memory list of the target resource object of the first secure container when the access request is a memory allocation request.

[0178] Further optionally, the user space of the first secure container includes a state saving tool, and the processor 701 pauses the operation of the first secure container through the container runtime component corresponding to the first secure container and obtains the non-target state data of the first secure container.When the target resource object's state data is being processed, specifically: the container runtime component pauses the operation of the first secure container to keep the state data of the non-target resource objects in the first secure container unchanged; the container runtime component calls the state saving tool to save the state data of the non-target resource objects in the first secure container to the memory of the source node.

[0179] Further optionally, when the processor 701 creates a second secure container on the target node and migrates the state data of the target resource object of the first secure container to the target resource object of the second secure container, it is specifically used to: instruct the creation of a third secure container on the target node and migrate the state data of the target resource object of the first secure container to the target resource object of the third secure container; instruct the creation of a second secure container on the target node and transfer the target resource object of the third secure container to the second secure container to migrate the state data of the target resource object of the first secure container to the target resource object of the second secure container; or instruct the creation of a first virtual machine in the second secure container, bind the target resource object of the second secure container to the first virtual machine, and migrate the state data of the target resource object of the first secure container to the target resource object bound to the first virtual machine; instruct the creation of a second virtual machine in the second secure container, bind the non-target resource object of the second secure container to the second virtual machine, and bind the target resource object bound to the first virtual machine to the second virtual machine to migrate the state data of the target resource object of the first secure container to the target resource object of the second secure container.

[0180] Further optionally, the non-target resource object of the first secure container includes an RDMA network card; when the processor 701 migrates the state data of the target resource object of the first secure container to the target resource object of the second secure container, it is specifically used to: copy the state data of the target resource object of the first secure container to the target resource object of the second secure container through RDMA operation; when the processor 701 migrates the state data of the non-target resource object of the first secure container to the non-target resource object of the second secure container, it is specifically used to: migrate the state data of the non-target resource object of the first secure container to the non-target resource object of the second secure container through RDMA operation.

[0181] Further optionally, the processor 701 is also used to: receive a container migration task issued by the management node, the container migration task being used to instruct the migration of the workload of the first secure container on the source node to the second secure container on the target node.

[0182] In other embodiments, processor 701, coupled to memory 702, is configured to execute a computer program in memory 702 for: receiving state data of a target resource object in a first secure container, and storing the state data of the target resource object in a target resource object in a second secure container, the second secure container being on the target node.The processor 701 is further configured to: create a second secure container on the target node, the second secure container including a target resource object managed by a driver and a non-target resource object not managed by a driver.

[0183] Further optionally, the processor 701 is further configured to: run the second secure container; restore the working state of the non-target resource object according to the state data in the non-target resource object of the second secure container; restore the working state of the target resource object according to the state data in the target resource object of the second secure container.

[0185] Further optionally, the user space of the second secure container includes: a state restoration tool, and when the processor 701 restores the working state of the non-target resource object according to the state data in the non-target resource object of the second secure container, it is specifically configured to: call the state restoration tool to provide the state data in the non-target resource object of the second secure container to the non-target resource object to restore the working state of the non-target resource object.

[0186] Further optionally, the second secure container includes a driver for the target resource object, and the state data of the target resource object includes a video memory list and its corresponding video memory content and other state data; when the processor 701 restores the working state of the target resource object according to the state data in the target resource object of the second secure container, it is specifically used to: call the target application interface in the driver of the target resource object to provide the video memory list and its corresponding video memory content in the target resource object of the second secure container to the target resource object of the second secure container; call other application interfaces in the driver of the target resource object other than the target application interface to provide other state data in the target resource object of the second secure container to the target resource object of the second secure container to restore the working state of the target resource object, wherein the other state data refers to state data other than the video memory list and its corresponding video memory content.

[0187] Further optionally, when the processor 701 stores the state data of the target resource object of the first security container in the target resource object of the second security container, it is specifically used to: migrate the state data of the target resource object of the first security container to the target resource object of the third security container, wherein the third security container is a security container on the target node; and, if the second security container is created, transfer the target resource object of the third security container to the second security container to migrate the state data of the target resource object of the first security container to the target resource object of the second security container.

[0188] Further optionally, before creating the second security container on the target node, the processor 701 further includes: creating the third security container on the target node.

[0189] Further optionally, when the processor 701 saves the state data of the target resource object of the first security container in the target resource object of the second security container, it is specifically used for: creating a first virtual machine in the second security container, binding the target resource object of the second security container to the first virtual machine, and migrating the state data of the target resource object of the first security container to the target resource object bound to the first virtual machine; creating a second virtual machine in the second security container, binding the non-target resource object of the second security container to the second virtual machine, and binding the target resource object bound to the first virtual machine to the second virtual machine, so as to migrate the state data of the target resource object of the first security container to the target resource object of the second security container.

[0190] Further, as shown in FIG7, the computer device also includes: a display 704, a power supply component 705, an audio component 706, and other components. FIG7 only schematically shows some components and does not mean that the computer device only includes the components shown in FIG7. In addition, the components in the dashed box in FIG7 are optional components, not mandatory components, and the specific components may vary depending on the product form of the computer device. The computer device in this embodiment can be a terminal device such as a desktop computer, laptop computer, smartphone, or IoT device, or a server device such as a conventional server, cloud server, or server array. If the computer device in this embodiment is a terminal device such as a desktop computer, laptop computer, or smartphone, it may include the components within the dashed box in Figure 7; if the computer device in this embodiment is a server device such as a conventional server, cloud server, or server array, it may not include the components within the dashed box in Figure 7.

[0191] This application embodiment also provides a cloud computing system, including: a source node and a target node; the source node and the target node cooperate with each other to migrate the workload in the first secure container on the source node to the second secure container on the target node; wherein, the process of cooperation between the source node and the target node and the working process of each node during the cooperation process can be referred to the foregoing embodiment, and will not be repeated here.

[0192] Correspondingly, this application embodiment also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, can implement the steps in the above method embodiments.

[0193] Accordingly, embodiments of this application also provide a computer program product, including a computer program / instructions, which, when executed by a processor, implement the steps in the above method embodiments.

[0194] The above-mentioned memory can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random-access memory (SRAM), electrically erasable programmable read-only memory (EEPROM).Erasable Programmable Read Only Memory (EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Read-Only Memory (PROM), Read-Only Memory (ROM), Magnetic Storage, Flash Memory, Disk, or Optical Disk.

[0195] The above-described communication component is configured to facilitate wired or wireless communication between the device containing the communication component and other devices. The device containing the communication component can access wireless networks based on communication standards, such as WiFi, 2G, 3G, 4G / LTE, 5G, or combinations thereof. In one exemplary embodiment, the communication component receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, the communication component further includes a Near Field Communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on Radio Frequency Identification (RFID) technology, Infrared Data Association (IrDA) technology, Ultra Wide Band (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0196] The above-mentioned display includes a screen, which may include a Liquid Crystal Display (LCD) and a Touch Panel (TP). If the screen includes a Touch Panel, the screen can be implemented as a touch screen to receive input signals from the user. The Touch Panel includes one or more touch sensors to sense touch, swipe, and gestures on the Touch Panel. The touch sensors can sense not only the boundaries of the touch or swipe action, but also the duration and pressure associated with the touch or swipe operation.

[0197] The above-mentioned power supply component provides power to various components of the device in which the power supply component is located. The power supply component may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the device in which the power supply component is located. Instruction manual, pages 25 / 27, 29, CN 121636047 A

[0198] The above-described audio component can be configured to output and / or input audio signals. For example, the audio component includes a microphone (MIC) when the device containing the audio component is in an operating mode, such as call mode, recording mode, and voice mode.When the recognition mode is active, the microphone is configured to receive external audio signals. The received audio signals may be further stored in memory or transmitted via a communication component. In some embodiments, the audio component further includes a speaker for outputting audio signals.

[0199] Those skilled in the art will understand that embodiments of this application may be provided as methods, systems, or computer program products. Therefore, this application may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Moreover, this application may take the form of a computer program product implemented on one or more computer-readable storage media (including, but not limited to, disk storage, compact disc read-only memory (CD-ROM), optical storage, etc.) containing computer-usable program code.

[0200] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, may be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in one or more flowcharts and / or one or more block diagrams.

[0201] These computer program instructions may also be stored in a computer-readable storage medium capable of directing a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means that implement the functions specified in one or more flowcharts and / or one or more block diagrams.

[0202] These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, such that the instructions, which execute on the computer or other programmable apparatus, provide steps for implementing the functions specified in one or more flowcharts and / or one or more block diagrams.

[0203] In a typical configuration, a computing device includes one or more processors (Central Processing Unit, CPU), input / output interfaces, network interfaces, and memory.

[0204] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM.Memory is an example of computer-readable media.

[0205] Computer-readable media includes permanent and non-permanent, removable and non-removable media that can store information by any method or technology. Information can be computer-readable instructions, data structures, modules of programs or other data. Examples of computer storage media include, but are not limited to, phase-change random access memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, Digital Video Disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices or any other non-transferable medium that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves. Specification page 26 / 27 30 CN 121636047 A

[0206] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes the element.

[0207] The above are merely embodiments of this application and are not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principle of this application should be included within the scope of the claims of this application. Instruction manual, page 27 / 27, page 31, CN 121636047 A, Figure 1a; Instruction manual drawing, page 1 / 7, page 32, CN 121636047 A, Figure 1b; Instruction manual drawing, page 2 / 7, page 33, CN 121636047 A, Figure 2a; Instruction manual drawing, page 3 / 7, page 34, CN 121636047 A, Figure 2b; Instruction manual drawing, page 4 / 7, page 35, CN 121636047 A, Figure 3; Instruction manual drawing, page 5 / 7, page 36, CN121636047 A Figures 4, 5, 6, 6 / 7 pages of the drawings of the specification, 37 CN 121636047 A Figure 7, 7 / 7 pages of the drawings of the specification, 38 CN 121636047 A Abstract This application provides a secure container migration method, device, storage medium, and program product. The first secure container on the source node includes a target resource object and non-target resource objects. When the target resource object is implemented as a pass-through device for the secure container, different layers of components cooperate to address the different characteristics of the target and non-target resource objects. Specifically, the state data of the target and non-target resource objects in the first secure container is obtained through the driver program of the target resource object and the container runtime component. This allows for relatively accurate migration of the state data of the target and non-target resource objects to the second secure container on the target node. The cooperation of different layers of components effectively enables rapidmigration of workloads within the secure container, solving the problem of state migration for secure containers containing target resource objects that cannot be directly managed by the secure container or its cloud provider.

Claims

1. A secure container migration method, characterized by, A method for migrating a workload in a first secure container on a source node, the first secure container comprising target resource objects and non-target resource objects, the target resource objects being pass-through devices of the first secure container, the method comprising: suspending, by a driver of the target resource objects, running of the target resource objects of the first secure container and obtaining state data of the target resource objects of the first secure container during running of the first secure container; migrating the state data of the target resource objects of the first secure container to target resource objects of a second secure container, the second secure container being a secure container on a target node; suspending, by a container runtime component corresponding to the first secure container, running of the first secure container and obtaining state data of the non-target resource objects of the first secure container; migrating the state data of the non-target resource objects of the first secure container to non-target resource objects of the second secure container to migrate the workload in the first secure container to the second secure container.

2. The method of claim 1, wherein, The first secure container comprises a driver of the target resource objects in a guest operating system of the first secure container and a runtime library of the target resource objects in a user space of the first secure container, and suspending, by the driver of the target resource objects, running of the target resource objects of the first secure container comprises: intercepting, by an intercept function in the runtime library, an access request initiated by an application in the first secure container to suspend running of the target resource objects of the first secure container, the access request being initiated by the application to the driver of the target resource objects through a library function in the runtime library; or calling the driver of the target resource objects to send a suspension command to the target resource objects of the first secure container to suspend running of the target resource objects of the first secure container. Obtaining, by the driver of the target resource objects, the state data of the target resource objects of the first secure container comprises:

3. The method of claim 1, wherein, calling a target application programming interface in the driver of the target resource objects to obtain a list of video memory and corresponding video memory content of the target resource objects of the first secure container; and calling other application programming interfaces in the driver of the target resource objects other than the target application programming interface to obtain other state data of the target resource objects of the first secure container, the other state data being state data other than the list of video memory and the corresponding video memory content. The first secure container comprises the runtime library of the target resource objects in the user space of the first secure container, and the method further comprises:

4. The method of claim 1, wherein, using a proxy function in the runtime library to proxy an access request initiated by an application in the first secure container, and obtaining a list of video memory applied by the application as the list of video memory of the target resource objects of the first secure container in a case where the access request is a memory allocation request, the access request being initiated by the application to the driver of the target resource objects through a library function in the runtime library. ​ 5. The method of claim 1, wherein, The state saving tool is included in a user space of the first secure container, the first secure container is suspended by a container runtime component corresponding to the first secure container, and state data of a non-target resource object of the first secure container is acquired. The container runtime component suspends the first secure container to keep the state data of the non-target resource object of the first secure container unchanged. The container runtime component calls the state saving tool to acquire the state data of the non-target resource object of the first secure container and save the state data into the memory of the source node.

6. The method of claim 1, wherein, Indicating to create a second secure container on the target node, migrating the state data of the target resource object of the first secure container into a target resource object of the second secure container, including: Indicating to create a third secure container on the target node, migrating the state data of the target resource object of the first secure container into a target resource object of the third secure container; Indicating to create a second secure container on the target node, migrating the target resource object of the third secure container to the second secure container to migrate the state data of the target resource object of the first secure container into a target resource object of the second secure container; Or Indicating to create a first virtual machine in the second secure container, binding the target resource object of the second secure container with the first virtual machine, and migrating the state data of the target resource object of the first secure container into the target resource object bound with the first virtual machine; Indicating to create a second virtual machine in the second secure container, binding the non-target resource object of the second secure container with the second virtual machine, and binding the target resource object bound with the first virtual machine with the second virtual machine to migrate the state data of the target resource object of the first secure container into the target resource object of the second secure container.

7. The method according to any one of claims 1 to 6, characterized in that, The non-target resource object of the first secure container includes a remote direct data access network card; Migrating the state data of the target resource object of the first secure container into the target resource object of the second secure container includes copying the state data of the target resource object of the first secure container into the target resource object of the second secure container through a remote direct data access operation; Migrating the state data of the non-target resource object of the first secure container into the non-target resource object of the second secure container includes copying the state data of the non-target resource object of the first secure container into the non-target resource object of the second secure container through a remote direct data access operation.

8. The method according to any one of claims 1 to 6, characterized in that, Further comprising: Receiving a container migration task issued by a management node, the container migration task being used to indicate to migrate a workload of a first secure container on a source node to a second secure container on a target node.

9. A secure container migration method, characterized by, A method for migrating a workload of a first secure container on a source node, a second secure container is created on a target node, the second secure container includes a target resource object managed by a dependency driver and a non-target resource object not managed by the dependency driver; the method comprises: receiving state data of a target resource object of the first secure container, and saving the state data of the target resource object of the first secure container in a target resource object of the second secure container; receiving state data of a non-target resource object of the first secure container, and saving the state data of the non-target resource object of the first secure container in a non-target resource object of the second secure container.

10. The method of claim 9, wherein, Further comprising: running the second secure container; restoring a working state of the non-target resource object according to the state data in the non-target resource object of the second secure container; restoring a working state of the target resource object according to the state data in the target resource object of the second secure container.

11. The method of claim 10, wherein, the second secure container comprises a driver of the target resource object, and the state data of the target resource object comprises a video memory list and corresponding video memory content thereof and other state data; restoring a working state of the target resource object according to the state data in the target resource object of the second secure container, comprises: calling a target application program interface in the driver of the target resource object, and providing the video memory list and corresponding video memory content thereof in the target resource object of the second secure container to the target resource object of the second secure container; calling other application program interfaces in the driver of the target resource object except the target application program interface, and providing other state data in the target resource object of the second secure container to the target resource object of the second secure container to restore the working state of the target resource object, the other state data referring to state data except the video memory list and corresponding video memory content thereof.

12. The method according to any one of claims 9-11, characterized in that, saving the state data of the target resource object of the first secure container in the target resource object of the second secure container, comprises: migrating the state data of the target resource object of the first secure container to a target resource object of a third secure container, the third secure container being a secure container on the target node; in a case where the second secure container is created, transferring the target resource object of the third secure container to the second secure container to migrate the state data of the target resource object of the first secure container to the target resource object of the second secure container.

13. A computer device, comprising: The source node can be implemented, and the source node comprises a host computer, the host computer deploying a first secure container and a container runtime component, the first secure container comprising a target resource object and a non-target resource object, the target resource object being a pass-through device of the first secure container, wherein the container runtime component is configured to perform steps in the method of any one of claims 1-8.

14. A computer device, comprising: comprising: a memory and a processor; the memory is configured to store a computer program, and the processor is coupled to the memory and configured to execute the computer program in the memory to implement steps in the method of any one of claims 1-8 and claims 9-12.

15. A computer-readable storage medium, characterized in that, When the computer program is executed by a processor, the processor is enabled to implement the steps in the method of any of claims 1-8 and 9-12.

16. A computer program product, characterised in that, comprising computer program / instructions which, when executed by a processor, perform the steps in the method of any of claims 1-8 and 9-12.

17. A cloud computing system, characterized by comprising: a source node for performing the method of any of claims 1-8 and a target node for performing the method of any of claims 9-12; the source node and the target node cooperating with each other to migrate a workload in a first secure container on the source node to a second secure container on the target node.

Images