Container migration method and system, and computer-readable storage medium

Abstract

Disclosed in the present invention are a container migration method and system, and a computer-readable storage medium. The container migration system comprises a first computing device and a second computing device, wherein software and hardware resources of the first computing device are divided into an untrusted execution environment and a trusted execution environment (TEE), and software and hardware resources of the second computing device are divided into an untrusted execution environment and a TEE; the TEE of the first computing device comprises a first confidential container, and the TEE of the second computing device comprises a standby confidential container; and the first computing device is used for sending to the second computing device encrypted information for state information of the first confidential container, and the second computing device is used for decrypting the encrypted information to obtain the state information of the first confidential container, and implementing state synchronization between the standby confidential container and the first confidential container on the basis of the state information of the first confidential container. By means of the method, during the process of performing capacity expansion / system upgrade / load adjustment / fault repair on a computing device, a tenant service that is being processed by a confidential container running on the computing device cannot be interrupted.

Description

Container migration method, system, and computer-readable storage medium

[0001] This application claims priority to the Chinese patent application with application number 202311593767.5 filed with the State Intellectual Property Office of China on November 24, 2023, and priority to the Chinese patent application with the invention name “Container Migration Method, System and Computer-Readable Storage Medium”, all contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of virtualization technology, and in particular to a container migration method, system, and computer-readable storage medium. Background Art

[0003] A confidential container is a container created and run in a trusted execution environment (TEE). It is an important application scenario in container technology. TEE can protect the privacy and security of user data in confidential containers.

[0004] Currently, in the confidential container scenario, if the computing device (such as a server) used to implement the confidential container needs to be expanded / upgraded / load adjusted / fault repaired, the computing device will usually be restarted once or multiple times during the process of expanding / upgrading the system / load adjusting / fault repairing the computing device. This will cause the confidential container on the computing device to stop running, thereby causing the tenant business being processed by the confidential container to be interrupted, seriously affecting the tenant experience.

[0005] Summary of the Invention

[0006] The present application provides a container migration method, system and computer-readable storage medium, which can ensure that the tenant business being processed by the confidential container running on the computing device is not interrupted during the process of expanding the capacity / system upgrade / load adjustment / fault repair of the computing device, thereby improving the tenant experience.

[0007] In a first aspect, a container migration system is provided. The container migration system includes a first computing device and a second computing device. The software and hardware resources of the first computing device are divided into a first TEE and a first untrusted execution environment. The software and hardware resources of the second computing device are divided into a second TEE and a second untrusted execution environment. The first TEE includes a first container, and the second TEE includes a backup container.

[0008] The first computing device is used to send encrypted information of the state information of the first container to the second computing device;

[0009] The second computing device is used to receive and decrypt the encrypted information to obtain the status information of the first container;

[0010] The second computing device is further configured to synchronize the status of the standby container with the first container based on the status information of the first container.

[0011] The status information of the first container includes status information of tenant services running on the first container.

[0012] In the above solution, since the first computing device migrates the state information of the first container to the second computing device, the second computing device synchronizes the state of the backup container with the first container based on the state information of the migrated first container. Therefore, during the process of capacity expansion / system upgrade / load adjustment / fault repair on the first computing device, the second computing device can run the backup container to continue processing tenant business. Even if the first container is shut down due to restarting the first computing device, the tenant business will not be interrupted. In addition, since the backup container is located in the TEE of the second computing device, the TEE has higher security. The backup container continues to process tenant business, which can protect the security of tenant business data.

[0013] It can also be seen that, since the first computing device encrypts the state information of the first container and then transmits it to the second computing device, the above solution can also protect the security of the state information of the first container during transmission.

[0014] In some possible implementations, the state of the first container includes a memory state of the first container, or the state of the first container includes the memory state of the first container and a communication state between the first container and a virtual input and output (IO) device.

[0015] It can be understood that the status of the first container includes the memory status of the first container and the communication status between the first container and the virtual IO device. Compared with the first container status only including the memory status of the first container, when the status of the backup container and the first container are subsequently synchronized, the synchronized status is more complete.

[0016] In some possible implementations, the first TEE includes a first virtual machine, the second TEE includes a second virtual machine, the first container is located in the first virtual machine, and the backup container is located in the second virtual machine;

[0017] The processor of the first computing device is used to run the first virtual machine and encrypt the state information of the first container to obtain encrypted information;

[0018] The processor of the first computing device is configured to call a first communication interface in a first non-trusted execution environment and send encrypted information to a second communication interface, where the first communication interface is a communication interface of the first computing device and the second communication interface is a communication interface of the second computing device;

[0019] The processor of the second computing device is configured to obtain the encrypted information in the second untrusted execution environment;

[0020] The processor of the second computing device is configured to run a second virtual machine and decrypt the encrypted information to obtain the state information of the first container;

[0021] The processor of the second computing device is used to run the second virtual machine and synchronize the status of the standby container with the first container based on the status information of the first container.

[0022] By implementing the above implementation, since the state information of the first container is encrypted by the first virtual machine in the first TEE with higher security and decrypted by the second virtual machine in the second TEE with higher security, the encrypted information is transmitted in the first non-trusted execution environment with lower security and the second non-trusted execution environment with lower security. Therefore, the state information of the first container can be prevented from being stolen in the first non-trusted environment with lower security and the second non-trusted environment with lower security, and the state information security of the first container can be well protected.

[0023] In some possible implementations, the encrypted information is obtained by encrypting the state information of the first container using an encryption key, the encryption key is obtained through negotiation between the first virtual machine and the second virtual machine, and the encryption key is also used to decrypt the encrypted information.

[0024] In implementing the above implementation, since the encryption key is obtained through negotiation between the first virtual machine in the first TEE with higher security and the second virtual machine in the second TEE with higher security, the first computing device only needs to transmit the encrypted information to the second computing device, and there is no need to transmit the encryption key to the second computing device. Even if the encrypted information is stolen during the process of transmission from the first computing device to the second computing device, or in the first untrusted execution environment or the second untrusted execution environment, the thief cannot decrypt the stolen encrypted information because he cannot steal the encryption key. Therefore, the above implementation can further improve the security of the status information of the first container during transmission.

[0025] In some possible implementations, the processor of the first computing device is configured to execute the first virtual machine to obtain authentication information of the first virtual machine;

[0026] The processor of the first computing device is configured to call the first communication interface in the first non-trusted execution environment and send authentication information of the first virtual machine to the second communication interface;

[0027] The processor of the second computing device is configured to obtain authentication information of the first virtual machine in the second untrusted execution environment;

[0028] The processor of the second computing device is used to run the second virtual machine, and negotiate with the first virtual machine to obtain an encryption key when the first virtual machine is successfully authenticated based on the authentication information of the first virtual machine.

[0029] In the above implementation, since the first virtual machine negotiates with the second virtual machine to obtain the encryption key only after the second virtual machine is successfully authenticated based on the second virtual machine's authentication information, this can avoid migrating the state information of the first container to the second virtual machine when the second virtual machine is untrusted, thereby protecting the security of the state information of the first container.

[0030] In some possible implementations, the processor of the second computing device is configured to run the second virtual machine and obtain authentication information of the second virtual machine;

[0031] The processor of the second computing device is configured to call the second communication interface in the second non-trusted execution environment and send authentication information of the second virtual machine to the first communication interface;

[0032] The processor of the first computing device is configured to obtain authentication information of the second virtual machine in the first untrusted execution environment;

[0033] The processor of the first computing device is used to run the first virtual machine, and negotiate with the second virtual machine to obtain an encryption key when the second virtual machine is authenticated successfully based on the authentication information of the second virtual machine.

[0034] In the above implementation, since the second virtual machine obtains the encryption key through negotiation with the first virtual machine only after the first virtual machine has been authenticated based on the authentication information of the first virtual machine, this can avoid the situation where the state information of the first container is migrated to the second virtual machine when the first virtual machine is untrusted, thereby preventing security risks to the second computing device and the second virtual machine, thereby protecting the security of the second computing device and the second virtual machine.

[0035] In some possible implementations, the authentication information of the first virtual machine includes a current measurement value of a configuration of the first virtual machine and a historical measurement value of the configuration of the first virtual machine, and the authentication information of the second virtual machine includes a current measurement value of a configuration of the second virtual machine and a historical measurement value of the configuration of the second virtual machine;

[0036] The processor of the second computing device is configured to run the second virtual machine, and if it is determined that the current measurement value of the configuration of the first virtual machine is the same as the historical measurement value of the configuration of the first virtual machine, authenticate the first virtual machine successfully;

[0037] The processor of the first computing device is configured to run the first virtual machine, and when it is determined that a current measurement value of the configuration of the second virtual machine is the same as a historical measurement value of the configuration of the second virtual machine, the authentication of the second virtual machine is successful.

[0038] In some possible implementations, the processor of the second computing device is configured to run a second virtual machine to generate a message authentication code based on the encryption key and the first message;

[0039] The processor of the second computing device is configured to call the second communication interface in the second untrusted execution environment and send the first message and the message authentication code to the first communication interface;

[0040] The processor of the first computing device is configured to obtain a first message and a message authentication code in the first untrusted execution environment;

[0041] The processor of the first computing device is used to run the first virtual machine, and when it is determined that the encryption key is a valid key based on the first message and the message authentication code, encrypt the state information of the first container using the encryption key.

[0042] In the above implementation, the first virtual machine uses the encryption key to encrypt the state information of the first container only when it determines that the encryption key is a valid key based on the message authentication code from the second virtual machine and the first message. This avoids migrating the state information of the first container to the second virtual machine when the encryption key is invalid, thereby ensuring the accurate migration of the first container.

[0043] In some possible implementations, a processor of the first computing device is configured to execute a first virtual machine to generate a first private key and a first public key;

[0044] The processor of the first computing device is configured to call the first communication interface in the first untrusted execution environment and send the first public key to the second communication interface;

[0045] The processor of the second computing device is configured to obtain the first public key in the second untrusted execution environment;

[0046] The processor of the second computing device is configured to run a second virtual machine to generate a second private key and a second public key, and to generate an encryption key based on the second private key and the first public key;

[0047] The processor of the second computing device is configured to call the second communication interface in the second untrusted execution environment and send the second public key to the first communication interface;

[0048] The processor of the first computing device is configured to obtain a second public key in the first untrusted execution environment;

[0049] The processor of the first computing device is used to run the first virtual machine and generate an encryption key based on the first private key and the second public key.

[0050] As can be seen in the above implementation, the first and second virtual machines generate encryption keys based on public information (i.e., the other party's public key) and their respective private information (i.e., private key). Private information is not transmitted between the two. Even if the public information is stolen during transmission, the thief cannot steal the encryption key because the private information is not leaked. Therefore, the state information of the first container that is subsequently encrypted using the encryption key can be protected during transmission. In addition, the first and second virtual machines are both located in the TEE. Due to the high security of the TEE, the encryption key can be well protected.

[0051] In some possible implementations, the communication state information between the first container and the virtual IO device includes an identifier of the first container;

[0052] Before the processor of the first computing device is used to run the first virtual machine and encrypt the state information of the first container to obtain encrypted information, the processor of the first computing device is further used to:

[0053] Running the first virtual machine, and locating, in a virtual IO device driver of the first virtual machine based on the identifier of the first container, communication status information between the first container and the virtual IO device;

[0054] After the processor of the second computing device is used to run the second virtual machine and decrypt the encrypted information to obtain the state information of the first container, the processor of the second computing device is specifically used to:

[0055] Run the second virtual machine, load the memory status information of the first container into the memory of the standby container, and load the communication status information between the first container and the virtual IO device into the virtual IO device driver of the second virtual machine;

[0056] The second virtual machine is run, and based on the memory status information of the first container in the memory of the backup container and the communication status information between the first container and the virtual I / O device located in the virtual I / O device driver of the second virtual machine based on the identifier of the first container, the backup container is run, thereby achieving memory status synchronization between the backup container and the first container, and achieving communication status synchronization between the backup container and the first container.

[0057] Since the communication status information between the first container and the virtual IO device is located in the virtual IO device driver of the first virtual machine, and when the first virtual machine includes multiple confidential containers, the virtual IO device driver is shared by the multiple confidential containers. That is, the virtual IO device driver may include the communication status information between the multiple confidential containers and the virtual IO device. In order for the first virtual machine to accurately locate the communication status information between the first container and the virtual IO device from the virtual IO device driver, each confidential container in the first virtual machine can be set to carry the identity document (ID) of the confidential container in the communication request sent to the virtual IO device when communicating with the virtual IO device. Moreover, when the virtual IO device driver receives the communication request from the confidential container, it adds the identity document (ID) of the confidential container carried in the communication request together with the communication request to the virtual queue in the virtual IO device driver. Therefore, when the first virtual machine subsequently migrates the first container, the first virtual machine can accurately locate the communication request from the first container to the virtual IO device and the response information returned by the virtual IO device based on the communication request from the virtual queue in the virtual IO device driver based on the first container. The located information is the communication status information between the first container and the virtual IO device.

[0058] In addition, because the communication status information between the first container and the virtual IO device includes the identifier of the first container, when the second virtual machine runs the backup container, it can accurately locate the communication status information of the first container to the virtual IO device from the virtual IO device driver of the second virtual machine based on the identifier of the first container, thereby achieving synchronization of the communication status of the backup container and the first container based on the communication status information.

[0059] In a second aspect, a container migration method is provided, which is applied to a second computing device, wherein the software and hardware resources of the second computing device are divided into a second untrusted execution environment and a second TEE, and the second TEE includes a backup container. The method includes:

[0060] The second computing device obtains encrypted information of state information of a first container, where the first container is deployed in a first TEE, and the first TEE is a TEE of the first computing device;

[0061] The second computing device decrypts the encrypted information to obtain the state information of the first container;

[0062] The second computing device synchronizes the status of the backup container with the first container based on the status information of the first container.

[0063] In some possible implementations, the state of the first container includes a memory state of the first container, or the state of the first container includes the memory state of the first container and a communication state between the first container and a virtual IO device.

[0064] In some possible implementations, the second TEE includes a second virtual machine, and the backup container is deployed on the second virtual machine;

[0065] The second computing device obtains encrypted information of the state information of the first container, including:

[0066] The processor of the second computing device obtains, in the second untrusted execution environment, encrypted information of the state information of the first container;

[0067] The second computing device decrypts the encrypted information to obtain the state information of the first container, including:

[0068] The processor of the second computing device runs the second virtual machine and decrypts the encrypted information to obtain the state information of the first container;

[0069] The second computing device synchronizes the status of the standby container with the first container based on the status information of the first container, including:

[0070] The processor of the second computing device runs the second virtual machine and synchronizes the states of the backup container and the first container based on the state information of the first container.

[0071] In some possible implementations, the encrypted information is obtained by encrypting the status information of the first container using an encryption key, the encryption key is obtained through negotiation between the first virtual machine and the second virtual machine, and the encryption key is also used to decrypt the encrypted information. The first virtual machine is deployed in the first TEE, and the first container is deployed in the first virtual machine.

[0072] In some possible implementations, the method further includes:

[0073] The processor of the second computing device obtains authentication information of the first virtual machine in the second non-trusted execution environment;

[0074] The processor of the second computing device runs the second virtual machine, and when the first virtual machine is authenticated successfully based on the authentication information of the first virtual machine, negotiates with the first virtual machine to obtain the encryption key.

[0075] In some possible implementations, the method further includes:

[0076] The processor of the second computing device runs the second virtual machine and obtains authentication information of the second virtual machine;

[0077] The processor of the second computing device calls the second communication interface in the second non-trusted execution environment and sends authentication information of the second virtual machine to the first communication interface, where the second communication interface is the communication interface of the second computing device and the first communication interface is the communication interface of the first computing device.

[0078] In some possible implementations, the authentication information of the first virtual machine includes a current measurement value of a configuration of the first virtual machine and a historical measurement value of a configuration of the first virtual machine;

[0079] The method further comprises:

[0080] The processor of the second computing device runs the second virtual machine, and when it is determined that the current measurement value of the configuration of the first virtual machine is the same as the historical measurement value of the configuration of the first virtual machine, the authentication of the first virtual machine is successful.

[0081] In some possible implementations, the processor of the second computing device runs the second virtual machine to generate a message authentication code based on the encryption key and the first message;

[0082] The processor of the second computing device calls the second communication interface in the second non-trusted execution environment and sends the first message and the message authentication code to the first communication interface, where the second communication interface is the communication interface of the second computing device and the first communication interface is the communication interface of the first computing device.

[0083] In some possible implementations, the communication status information between the first container and the virtual IO device includes an identifier of the first container;

[0084] The processor of the second computing device runs the second virtual machine, and synchronizes the state of the standby container with the first container based on the state information of the first container, including:

[0085] The processor of the second computing device runs the second virtual machine, loads the memory state information of the first container into the memory of the backup container, and loads the communication state information between the first container and the virtual IO device into the virtual IO device driver of the second virtual machine;

[0086] The processor of the second computing device runs the second virtual machine and runs the backup container based on memory status information of the first container in the memory of the backup container and communication status information between the first container and the virtual IO device located in the virtual IO device driver of the second virtual machine based on the identifier of the first container, thereby achieving memory status synchronization between the backup container and the first container and achieving communication status synchronization between the backup container and the first container.

[0087] In a third aspect, a container migration method is provided, which is applied to a first computing device, wherein the software and hardware resources of the first computing device are divided into a first untrusted execution environment and a first TEE, and the first TEE includes a first container. The method includes:

[0088] The processor of the first computing device encrypts the state information of the first container in the first TEE to obtain encrypted information;

[0089] The processor of the first computing device calls the communication interface of the first computing device in the first non-trusted execution environment and sends the encrypted information to the second computing device. The encrypted information is used to migrate the state information of the first container to a backup container, and the backup container is deployed in a second TEE, which is the TEE of the second computing device.

[0090] In a fourth aspect, a container migration apparatus is provided, which is applied to a second computing device, wherein the software and hardware resources of the second computing device are divided into a second untrusted execution environment and a second TEE, wherein the second TEE includes a backup container, and the apparatus includes:

[0091] an acquisition module, configured to acquire encrypted information of state information of a first container, where the first container is deployed in a first TEE, and the first TEE is a TEE of the first computing device;

[0092] a decryption module, configured to decrypt the encrypted information to obtain the status information of the first container;

[0093] A synchronization module is used to synchronize the status of the standby container with the first container based on the status information of the first container.

[0094] In a fifth aspect, a container migration apparatus is provided, which is applied to a first computing device, wherein the software and hardware resources of the first computing device are divided into a first untrusted execution environment and a first TEE, wherein the first TEE includes a first container, and the apparatus includes:

[0095] an encryption module, configured to encrypt, in the first TEE, the state information of the first container to obtain encrypted information;

[0096] A sending module is used to call the communication interface of the first computing device in the first non-trusted execution environment and send the encrypted information to the second computing device, where the encrypted information is used to migrate the state information of the first container to a backup container, where the backup container is deployed in a second TEE, and the second TEE is the TEE of the second computing device.

[0097] Regarding the relevant beneficial effects and descriptions of the container migration method provided in the second / third aspects and any implementation of the second / third aspects, and the relevant beneficial effects and descriptions of the container migration device provided in the fourth / fifth aspects and any implementation of the fourth / fifth aspects, please refer to the relevant beneficial effects and descriptions of the container migration system provided in the aforementioned first aspect and any implementation of the first aspect, and no further details will be given here.

[0098] In a sixth aspect, a computing device is provided, comprising a processor and a memory; the processor is configured to execute instructions stored in the memory, so that the computing device implements the method provided by any one of the second to third aspects above, and any implementation of any one of the aspects.

[0099] In a seventh aspect, a container migration system is provided, comprising the container migration device described in the fourth aspect and the container migration device described in the fifth aspect.

[0100] In an eighth aspect, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores instructions, and the instructions are used to implement the method provided by any one of the second to third aspects above, and any implementation manner of any one of the aspects.

[0101] In the ninth aspect, a computer program product is provided, comprising a computer program. When the computer program is read and executed by a computing device, the computing device executes the method provided in any one of the second to third aspects above, and any implementation of any one of the aspects. BRIEF DESCRIPTION OF THE DRAWINGS

[0102] FIG1 is a schematic diagram of a VirtIO protocol architecture provided in an embodiment of the present application;

[0103] FIG2 is a schematic diagram of a VirtIO protocol communication model provided in an embodiment of the present application;

[0104] FIG3 is a schematic structural diagram of a container migration system provided in an embodiment of the present application;

[0105] FIG4 is an interactive schematic diagram of a container migration method provided in an embodiment of the present application;

[0106] FIG5 is a schematic diagram of a specific embodiment of a container migration method provided in an embodiment of the present application;

[0107] FIG6 is a schematic diagram of a process for obtaining an encryption key through negotiation according to an embodiment of the present application;

[0108] FIG7 is a schematic structural diagram of a container migration device provided in an embodiment of the present application;

[0109] FIG8 is a schematic structural diagram of another container migration device provided in an embodiment of the present application;

[0110] FIG9 is a schematic diagram of the structure of a computing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0111] The following describes the embodiments of the present invention in conjunction with the accompanying drawings. The terms used in the embodiments of the present invention are only used to explain the specific embodiments of the present invention, and are not intended to limit the present invention.

[0112] The terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequential order. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, and this is merely a way of distinguishing the objects of the same attributes when describing them in the embodiments of the present application. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, so that the process, method, system, product or equipment comprising a series of units need not be limited to those units, but may include other units that are not clearly listed or inherent to these processes, methods, products or equipment.

[0113] In order to facilitate a clear understanding of the technical solution provided by this application, some nouns and terms involved in the technical solution provided by this application are first explained.

[0114] (1) A trusted execution environment (TEE) is typically composed of hardware and software in a computing device. It can run on top of a normal operating system, providing a secure operating environment that prevents malware from accessing or tampering with the data and code in the TEE, ensuring the security and privacy of sensitive data and code. TEEs can be created in computing devices using technologies such as the ARM architecture-based TrustZone technology, AMD secure encrypted virtualization (SEV), and Intel Software Guard Extensions (SGX).

[0115] (2) Untrusted execution environment, which consists of the remaining software and hardware resources in the computing device except the software and hardware resources included in the TEE.

[0116] An untrusted execution environment (UE) doesn't mean the operating system (OS) or software running within it is malicious, but rather that it is less secure than a TEE. When the processor is operating in an UE, access to TEE resources (such as registers, memory, cache, and peripherals) is prohibited. If the processor attempts to access these resources, the system will crash. For example, TrustZone can configure sensitive memory as secure memory by configuring the trustzone address space controller (TZASC) and trustzone memory adapter (TZMA) registers. This secure memory is then inaccessible when the processor is operating in an UE. When the processor is operating in a TEE, it can access both TEE and UE resources.

[0117] (3) Confidential virtual machine (CVM), a product of the combination of confidential computing technology and cloud native technology, refers to a virtual machine located in TEE. Based on TEE, confidential virtual machine can achieve resource isolation, data encryption, and remote attestation, preventing cloud service providers and any high-authority third parties from stealing and tampering with the data in the confidential virtual machine, thereby effectively protecting tenant data and asset security.

[0118] (4) Ordinary virtual machine refers to a virtual machine located in a non-trusted execution environment.

[0119] (5) Confidential containers are the product of the combination of confidential computing technology and cloud native technology, and refer to containers located in TEE. Based on TEE, confidential containers can achieve resource isolation, data encryption, and remote attestation. They can prevent cloud service providers and any high-authority third parties from stealing and tampering with the data in confidential containers, thereby effectively protecting tenant data and asset security.

[0120] (6) Ordinary container refers to a container located in a non-trusted execution environment.

[0121] (7) A virtual input / output (IO) device, also known as a virtual device, is an IO device simulated by software and used to emulate the functions of a physical IO device in a computer system. It can provide interfaces and functions similar to those of a physical IO device, but is actually implemented in software.

[0122] The primary purpose of virtual IO devices is to virtualize and manage physical IO resources in a virtualized environment. In virtualization, when multiple virtual machines or containers share physical IO devices, virtual IO devices can help isolate and schedule resources, providing better performance and reliability.

[0123] The following are some common virtual IO devices:

[0124] Virtual disk: A virtual disk is a software-emulated disk device used to provide storage space for virtual machines or containers in a virtualized environment. A virtual disk divides the capacity of a physical disk into multiple logical volumes, providing independent disk space for each virtual machine or container.

[0125] Virtual Network Interface (VNIC): A virtual network interface (VNIC) is a software-emulated network interface device used to provide network connectivity for virtual machines or containers in a virtualized environment. A VNIC emulates the functionality of a physical network interface card, enabling communication between virtual machines or containers and the physical network.

[0126] A virtual serial port (VSP) is a software-emulated serial port device that simulates the functionality of a physical serial port in a virtualized environment. A VSP can be used to communicate between a virtual machine or container and external devices, such as debugging tools or external sensors.

[0127] A virtual universal serial bus device (VUS device), also known as a virtual USB device, is a software-emulated USB device that simulates the functionality of a physical USB device within a virtualized environment. A virtual USB device provides virtual machines or containers with access to and management of USB devices, such as USB storage devices or external input devices.

[0128] The emergence of virtual IO devices enables virtual machines or containers in a virtualized environment to better utilize and manage IO resources, providing more flexible and efficient IO functions.

[0129] [Corrected 11.10.2024 according to Rule 91] (8) Virtualized Input / Output (VirtIO) can be understood as a set of programs for virtualizing general input / output (I / O) devices. Specifically, as shown in Figure 1, the VirtIO protocol architecture includes a front-end driver and a virtual IO device (virtio device). The front-end driver can also be called a virtual IO device driver, and the virtual IO device can also be called a back-end device. The front-end driver includes drivers such as the virtual disk (virtio-blk) driver and the virtual network card (virtio-net) driver, which are located in the virtual machine. The virtual IO device is located in the hypervisor. The main function of the front-end driver is to discover the virtual IO device, accept requests from within the virtual machine, and communicate with the virtual IO device in the hypervisor according to the VirtIO protocol.

[0130] As shown in Figure 2, the VirtIO protocol uses a front-end and back-end communication model. A front-end driver runs in a virtual machine and exchanges data with virtual I / O devices in the hypervisor through virtual queues (virtqueues). Virtual queues are implemented using vrings, which are ring buffers shared between virtual machines and virtual I / O devices.

[0131] The vring uses three main components: the descriptor table, the available ring, and the used ring to achieve efficient data transmission. The following are their respective functions:

[0132] descriptor table:

[0133] Structure: It is an array, each element is a descriptor that describes the location and properties of a memory area. Each descriptor contains the physical address, length, and other control information of a memory area.

[0134] Purpose: The front-end driver and the virtual IO device share a reference to the buffer through the descriptor table. The front-end driver creates descriptors and adds them to the descriptor table, and the virtual IO device can read these descriptors to determine the location and size of the data to be read or written.

[0135] Available ring:

[0136] Structure: is a ring buffer containing a set of indices that correspond to descriptors in the Descriptor Table.

[0137] Purpose: Used to notify the virtual I / O device of new tasks. Specifically, the front-end driver adds a new index to the available ring, indicating the new task the virtual I / O device needs to process. The virtual I / O device polls this ring to check for new tasks. Updating the available ring is the front-end driver's way of notifying the virtual I / O device of new tasks.

[0138] used ring:

[0139] Structure: It is also a ring buffer containing a set of indexes corresponding to the descriptors in the descriptor table that have been used by the virtual IO device.

[0140] Purpose: Used to notify the front-end driver that the virtual I / O device has completed some tasks. Specifically, after completing a task, the virtual I / O device adds the index of the corresponding descriptor to the used ring. The front-end driver can periodically poll the used ring to check which tasks have been completed.

[0141] When a virtual machine is deployed with a container, if the container needs to perform IO operations, the virtual machine can allocate a corresponding virtual IO device to the container, and then the container can exchange data with the corresponding virtual IO device through the virtual queue in the front-end driver. When multiple containers are deployed on the same virtual machine, if multiple containers all need to perform IO operations, multiple containers all exchange data with their respective assigned virtual IO devices through the virtual queue in the front-end driver. In other words, multiple containers share the same front-end driver.

[0142] Next, the application scenarios involved in the embodiments of this application are introduced.

[0143] The embodiments of the present application relate to confidential container scenarios, and in particular to scenarios where it is necessary to expand the capacity / system upgrade / load adjust / fault repair of a computing device (such as a server) running a confidential container. In these scenarios, during the process of expanding the capacity / system upgrade / load adjust / fault repair of the computing device, the computing device is usually restarted once or multiple times, which will cause the confidential container on the computing device to stop running, thereby causing the tenant business being processed by the confidential container to be interrupted, seriously affecting the tenant experience.

[0144] To address the above-mentioned issues, the present application provides a container migration method and system, which migrates the confidential container on a computing device to another computing device before expanding / upgrading the system / adjusting the load / repairing the computing device running the confidential container. In this way, during the process of expanding / upgrading the system / adjusting the load / repairing the computing device, the migrated confidential container can be run on another computing device to continue processing tenant business, ensuring that tenant business is not interrupted and optimizing the tenant experience.

[0145] The following describes the container migration method and system provided by the present application in conjunction with the corresponding figures. Before introducing the container migration method and system provided by the present application, the concepts of container status, container migration, etc. involved in the embodiments of the present application are first introduced.

[0146] (1) The state of the container. If the container is not in the state of communicating with the virtual IO device, the state of the container only includes the memory state of the container. If the container is in the state of communicating with the virtual IO device, the state of the container includes the memory state of the container and the communication state between the container and the virtual IO device.

[0147] The memory state of a container refers to the memory data of the container, which can reflect the running state of the container, that is, the running state of the container process itself, including the business state of the container operation.

[0148] The communication status between a container and a virtual I / O device can reflect the communication status between the container and the virtual I / O device. This information may include read and write requests sent by the container to the virtual I / O device, as well as responses sent by the virtual I / O device to the container after processing the read and write requests from the container. For example, assuming that the virtual machine to which container A belongs implements I / O device virtualization based on the VirtIO protocol, the communication status information between container A and the virtual I / O device includes all information related to container A in the descriptor table, available ring, and used ring of the virtual queue in the front-end driver.

[0149] (2) Container migration can be understood as container state migration, that is, restoring the container state on another computing device by migrating the container state from the computing device to which the container belongs to another computing device.

[0150] First, please refer to Figure 3, which is a schematic diagram of the structure of a container migration system provided in an embodiment of the present application. As shown in Figure 3, the system includes a first computing device 100 and a second computing device 200. The first computing device 100 and the second computing device 200 can transmit data via a communication network using any communication mechanism or communication standard. The communication network can be a wide area network, a local area network, a point-to-point connection, or any combination thereof.

[0151] In FIG3 , the first computing device 100 can be regarded as a confidential container migration-out device (also referred to as a source host), and the second computing device 200 can be regarded as a confidential container migration-in device (also referred to as a target host).

[0152] The first computing device 100 and the second computing device 200 may be personal computers (PCs), tablet computers, physical servers, such as X86 servers or ARM servers, etc. The first computing device 100 and the second computing device 200 may belong to the same data center or different data centers.

[0153] As shown in Figure 3, the software and hardware resources of the first computing device 100 and the second computing device 200 are divided into a non-trusted execution environment and a TEE. In order to distinguish, in Figure 3 and the following embodiments, the non-trusted execution environment and the TEE in the first computing device 100 are respectively referred to as the first non-trusted execution environment and the first TEE, and the non-trusted execution environment and the TEE in the second computing device 200 are respectively referred to as the second non-trusted execution environment and the second TEE.

[0154] The first TEE may include one or more confidential virtual machines, each of which may include one or more confidential containers. In Figure 3, the first TEE includes a confidential virtual machine, and the confidential virtual machine includes two confidential containers as an example. Optionally, the first untrusted execution environment may include one or more ordinary virtual machines, each of which may include one or more ordinary containers, which is not shown in Figure 3.

[0155] The second TEE may include one or more confidential virtual machines, each of which may include one or more confidential containers. In Figure 3, the second TEE includes a confidential virtual machine, and the confidential virtual machine includes two confidential containers as an example. Optionally, the second non-trusted execution environment may include one or more ordinary virtual machines, each of which may include one or more ordinary containers, which is not shown in Figure 3.

[0156] The functions of the confidential VMs described above are the same as those of ordinary VMs. Any tasks that can be performed in ordinary VMs can also be performed in confidential VMs. Tenants can remotely log in to the confidential VMs and install, configure, and uninstall applications within the confidential VM operating system environment. The functions of the confidential containers described above are the same as those of ordinary containers. Any tasks that can be performed in ordinary containers can also be performed in confidential containers. Tenants can remotely operate confidential containers, such as starting and pausing them.

[0157] As shown in Figure 3, the first TEE includes a first virtual machine manager for managing the confidential virtual machine in the first TEE, the second TEE includes a second virtual machine manager for managing the confidential virtual machine in the second TEE, the first untrusted execution environment may also include a third virtual machine manager for managing the ordinary virtual machine in the first untrusted execution environment, and the second untrusted execution environment may also include a fourth virtual machine manager for managing the ordinary virtual machine in the second untrusted execution environment.

[0158] The first / second virtual machine manager can achieve logical isolation between different confidential virtual machines in the first / second TEE and manage confidential virtual machines, such as creating confidential virtual machines, simulating virtual hardware for confidential virtual machines according to the hardware layer (hardware simulation function), deleting confidential virtual machines, forwarding and / or processing network packets between all confidential virtual machines running on the first computing device 100 / second computing device 200 or forwarding network packets between the confidential virtual machines on the first computing device 100 / second computing device 200 and the external network (virtual switching function), processing input / output (I / O) generated by confidential virtual machines, etc.

[0159] The management operations performed by the first / second virtual machine manager on the confidential virtual machine are performed in collaboration with the third / fourth virtual machine manager. In other words, in order to protect the security of the confidential virtual machine, when the tenant needs to manage the confidential virtual machine in the first / second TEE, the third / fourth virtual machine manager obtains the tenant's management instructions and sends the management instructions to the first / second virtual machine manager, and the first / second virtual machine manager performs management operations according to the management instructions.

[0160] When the confidential virtual machine in the first / second TEE needs to communicate with the outside world, in order to protect the security of the confidential virtual machine, it communicates with the external network through the first / second virtual machine manager and the third / fourth virtual machine manager as a bridge. For example, the confidential virtual machine sends a message to the first / second virtual machine manager, which is forwarded to the third / fourth virtual machine manager, and then forwarded to the external network by the third / fourth virtual machine manager. The external network sends a message to the confidential virtual machine, which is forwarded to the first / second virtual machine manager by the third / fourth virtual machine manager, and then forwarded to the confidential virtual machine by the first / second virtual machine manager.

[0161] In the container migration system shown in FIG3 , in order to ensure that the first confidential container (referring to the confidential container to be migrated in the first computing device 100) remains secure after being migrated from the first computing device 100 to the second computing device 200, the first confidential container can be migrated to the TEE (i.e., the second TEE) of the second computing device 200. Specifically, a spare confidential container (referring to a container specifically used to implement the migration of other confidential containers (such as the first confidential container mentioned above) with empty memory) can be created in the second computing device 200. The first computing device 100 sends the status information of the first confidential container to the second computing device 200. As shown by the arrow in FIG3 , after receiving the status information of the first confidential container, the second computing device 200 restores the status of the first confidential container on the spare confidential container based on the status information of the first confidential container. In other words, the status of the spare confidential container is synchronized with the status of the first confidential container based on the status information of the first confidential container, thereby realizing the migration of the first confidential container.

[0162] To ensure the security of the first confidential container's status information during transmission from the first computing device 100 to the second computing device 200, the first computing device 100 may encrypt the first confidential container's status information before sending it to the second computing device 200. After receiving the encrypted information of the first confidential container's status information, the second computing device 200 decrypts the encrypted information to obtain the first confidential container's status information.

[0163] In a possible embodiment, in order to further improve the security of the status information of the first confidential container during the transmission from the first computing device 100 to the second computing device 200, the key used for encryption and decryption by the first computing device 100 and the second computing device 200 can be obtained through negotiation between the two parties. That is, the first computing device 100 and the second computing device 200 will not transmit the key, and the key will not be stolen during the transmission process.

[0164] In another possible embodiment, in order to further improve the security of the status information of the first confidential container during transmission from the first computing device 100 to the second computing device 200, the key used for encryption and decryption by the first computing device 100 and the second computing device 200 can be obtained through negotiation between the confidential virtual machine to which the first confidential container in the first computing device 100 (hereinafter referred to as the first confidential virtual machine) and the confidential virtual machine to which the backup confidential container in the second computing device 200 (hereinafter referred to as the second confidential virtual machine) belongs, and stored in the respective memories of the first confidential virtual machine and the second confidential virtual machine, and the encryption operation on the status information of the first confidential container is performed by the first confidential virtual machine, and the decryption operation on the encrypted information of the status information of the first confidential container is performed by the second confidential virtual machine. In other words, the key will not only not be used in the first computing device 100, but also will not be used in the second computing device 200. The data will not be transmitted between the device 100 and the second computing device 200, nor will it be transmitted in the untrusted execution environment in the first computing device 100 (i.e., the first untrusted execution environment) and the untrusted execution environment in the second computing device 200 (i.e., the second untrusted execution environment). Moreover, since the memory of the first confidential virtual machine belongs to the first TEE, the first untrusted execution environment has no right to access the key stored in the memory of the first confidential machine. Since the memory of the second confidential virtual machine belongs to the second TEE, the second untrusted execution environment also has no right to access the key stored in the memory of the second confidential virtual machine. Therefore, the key can be prevented from being obtained by malware in the first untrusted execution environment / the second untrusted execution environment, and the key can be prevented from being stolen by attackers when the first untrusted environment / the second untrusted environment is compromised, thereby better protecting the state information security of the first confidential container.

[0165] It should be understood that the container migration system shown in Figure 3 is merely an example provided by the present application and is not specifically limited in this application. For example, in a specific implementation, the container migration system shown in Figure 3 may further include a third computing device, a fourth computing device, and other computing devices, as well as a network device for forwarding communication data between computing devices.

[0166] To facilitate a clearer understanding of the process of confidential container migration by the container migration system shown in FIG3 , a detailed introduction is provided below in conjunction with an interactive schematic diagram of a container migration method provided in an embodiment of the present application shown in FIG4 .

[0167] S401: The first computing device 100 encrypts the status information of the first confidential container to obtain encrypted information.

[0168] S402 : The first computing device 100 sends encrypted information to the second computing device 200 , and correspondingly, the second computing device 200 receives the encrypted information sent by the first computing device 100 .

[0169] S403: The second computing device 200 decrypts the encrypted information to obtain the status information of the first confidential container.

[0170] S404: The second computing device 200 synchronizes the status of the backup confidential container with the first confidential container based on the status information of the first confidential container.

[0171] For the status of the first confidential container, please refer to the above description of the container status. For the sake of brevity, this description will not be repeated here.

[0172] Specifically, before performing the encryption operation, the first computing device 100 may first check whether the first confidential container is in a communication state with the virtual IO device. If it is determined that the first confidential container is in a communication state with the virtual IO device, the encrypted status information of the first confidential container includes the memory status information of the first confidential container and the communication status information between the first confidential container and the virtual IO device. Otherwise, the encrypted status information of the first confidential container only includes the memory status information of the first confidential container.

[0173] Optionally, if the first computing device 100 determines that the first confidential container is in a communication state with a virtual IO device, the encrypted status information of the first confidential container may also only include the memory status information of the first confidential container. This application does not make specific limitations on this.

[0174] In the case where the status information of the first confidential container only includes the memory status information of the first confidential container, in S404, the status synchronization between the backup confidential container and the first confidential container implemented by the second computing device 200 is memory status synchronization. In the case where the status information of the first confidential container includes the memory status information of the first confidential container and the communication status information between the first confidential container and the virtual IO device, in S404, the status synchronization between the backup confidential container and the first confidential container implemented by the second computing device 200 is memory status synchronization and communication status synchronization with the virtual IO device.

[0175] It can be understood that when the first computing device 100 determines that the first confidential container is in a communication state with the virtual IO device, the status information of the first confidential container encrypted by the first computing device 100 includes the memory status information of the first confidential container and the communication status information between the first confidential container and the virtual IO device. Compared with the state information of the first confidential container only including the memory status information of the first confidential container, when the status synchronization of the backup confidential container and the first confidential container is subsequently achieved, the synchronized state is more complete.

[0176] The algorithms used for encryption and decryption performed by the first computing device 100 and the second computing device 200 can be symmetric encryption algorithms, such as advanced encryption standard (AES), data encryption standard (DES), triple data encryption standard (3DES), etc., or can be asymmetric encryption algorithms, such as RSA, elliptic curve cryptography (ECC), digital signature algorithm (DSA), which is not specifically limited in this application.

[0177] It can be understood that since the status information of the above-mentioned first confidential container includes the status of the tenant business being processed by the first confidential container, after the second computing device 200 synchronizes the status of the backup confidential container with the first confidential container, the second computing device 200 can run the backup confidential container to continue processing the tenant business, and the tenant business is not affected by the expansion / system upgrade / load adjustment / fault repair of the first computing device 100.

[0178] The following describes a more specific embodiment of the container migration method provided by the present application, as shown in FIG4 . As shown in FIG5 , the embodiment includes the following steps:

[0179] S501: The processor of the first computing device 100 suspends the execution of the first confidential container.

[0180] In a possible embodiment, before S501, the processor of the first computing device 100 may also obtain the configuration information of the second computing device 200 (such as the central processing unit (CPU), memory, disk, network, etc.), check whether the configuration of the second computing device is the same as the configuration of the first computing device 100, and execute S501 if the configuration is the same. Otherwise, S501 may continue to be executed or may not be executed. If the configuration is the same, executing S501 can ensure that the migration of the first confidential container proceeds smoothly and that it can operate normally in the new environment. This helps to reduce problems and risks during the migration process and improve the success rate of container migration.

[0181] S502: The processor of the first computing device 100 and the processor of the second computing device 200 obtain an encryption key through negotiation.

[0182] For the specific implementation process of S502, please refer to the relevant description in FIG6 .

[0183] S503: The processor of the first computing device 100 runs the first confidential virtual machine, and uses the encryption key to encrypt the state information of the first confidential container to obtain encrypted information.

[0184] Taking the example of the first confidential container's status information including the first confidential container's memory status information and the communication status information between the first confidential container and the virtual IO device, S503 may specifically be: the processor of the first computing device 100 runs the first confidential virtual machine, obtains the first confidential container's memory status information and the communication status information between the first confidential container and the virtual IO device, and then encrypts the obtained information using an encryption key to obtain encrypted information. In a specific implementation, the memory status information of the first confidential container and the communication status information between the first confidential container and the virtual IO device may be encrypted and transmitted together or separately, and this application does not specifically limit this.

[0185] The following describes a process in which the processor of the first computing device 100 runs the first confidential virtual machine, obtains the memory status information of the first confidential container, and the communication status information between the first confidential container and the virtual IO device.

[0186] (1) Obtain memory status information of the first confidential container.

[0187] The processor of the first computing device 100 running the first confidential virtual machine can collect all context information of the first confidential container process through the process tracing (ptrace) mechanism, and then store this context information as image files according to functional classification. These image files are the memory status information of the first confidential container.

[0188] (2) Obtain communication status information between the first confidential container and the virtual IO device.

[0189] Since the communication status information between the first confidential container and the virtual IO device is located in the virtual IO device driver of the first confidential virtual machine, and when the first confidential virtual machine includes multiple confidential containers, the virtual IO device driver is shared by the multiple confidential containers. That is, the virtual IO device driver may include the communication status information between the multiple confidential containers and the virtual IO device. In order for the first confidential virtual machine to accurately locate the communication status information between the first confidential container and the virtual IO device from the virtual IO device driver, it can be set that when each confidential container in the first confidential virtual machine communicates with the virtual IO device, each confidential container in the first confidential virtual machine carries the identity document (ID) of the confidential container in the communication request sent to the virtual IO device. Moreover, when the virtual IO device driver receives the communication request from the confidential container, it adds the identity document (ID) of the confidential container carried in the communication request together with the communication request to the virtual queue in the virtual IO device driver. Therefore, when the first confidential virtual machine subsequently migrates the first confidential container, the first confidential virtual machine can accurately locate the communication request of the first confidential container to the virtual IO device and the response information returned by the virtual IO device based on the communication request from the virtual queue in the virtual IO device driver based on the identity document (ID). The located information is the communication status information between the first confidential container and the virtual IO device.

[0190] S504: The processor of the first computing device 100 calls the first communication interface in the first non-trusted execution environment and sends encrypted information to the second communication interface, where the first communication interface is the communication interface of the first computing device 100 and the second communication interface is the communication interface of the second computing device 200.

[0191] The communication interface may be a transceiver, or an input / output interface. Optionally, the transceiver may include a transmitter (transmitter) and a receiver (receiver). Optionally, the transceiver may be a transceiver circuit. Optionally, the input / output interface may be an input / output circuit.

[0192] S505: After receiving the encrypted information at the second communication interface, the processor of the second computing device 200 obtains the encrypted information in the second untrusted execution environment.

[0193] Specifically, after receiving the encrypted information, the second communication interface can store the encrypted information in the memory of the second non-trusted execution environment. Subsequently, the processor of the second computing device 200 can obtain the encrypted information from the memory of the second non-trusted execution environment.

[0194] S506: The processor of the second computing device 200 runs the second confidential virtual machine, uses the encryption key to decrypt the encrypted information, and obtains the status information of the first confidential container.

[0195] S507: The processor of the second computing device 200 runs the second confidential virtual machine, and synchronizes the status of the backup confidential container with the first confidential container based on the status information of the first confidential container.

[0196] It can be seen from the above embodiments that the status information of the first confidential container includes two situations: only the memory status information of the first confidential container, and the memory status information of the first confidential container and the communication status information between the first confidential container and the virtual IO device. The following is a detailed introduction to the process in which the processor of the second computing device 200 runs the second confidential virtual machine and realizes the status synchronization of the backup confidential container and the first confidential container based on the status information of the first confidential container in these two situations.

[0197] (1) The status information of the first confidential container only includes the memory status information of the first confidential container.

[0198] The processor of the second computing device 200 runs the second confidential virtual machine and loads the memory status information of the first confidential container into the memory of the backup confidential container, thereby achieving memory status synchronization between the backup confidential container and the first confidential container.

[0199] (2) The status information of the first confidential container includes the memory status information of the first confidential container and the communication status information between the first confidential container and the virtual IO device.

[0200] The processor of the second computing device 200 runs the second confidential virtual machine, loads the memory status information of the first confidential container into the memory of the backup confidential container, and loads the communication status information between the first confidential container and the virtual IO device into the virtual IO device driver of the second confidential virtual machine. Afterwards, based on the memory status information of the first confidential container in the backup confidential container and the communication status information between the first confidential container and the virtual IO device in the virtual IO device driver of the second confidential virtual machine, the backup confidential container is run, thereby achieving memory status synchronization between the backup confidential container and the first confidential container, and achieving communication status synchronization between the backup confidential container and the first confidential container.

[0201] It can be seen from the above embodiments that the communication status information between the first confidential container and the virtual IO device is originally located in the descriptor table, available ring and used ring in the virtual IO device driver of the first confidential virtual machine. In this application, the processor of the second computing device 200 running the second confidential virtual machine can add the communication status information between the first confidential container and the virtual IO device to the descriptor table, available ring and used ring in the virtual IO device driver of the second confidential virtual machine.

[0202] It can also be seen from the above embodiment that the communication status information between the first confidential container and the virtual IO device includes the identifier of the first confidential container. Therefore, after the processor of the second computing device 200 runs the second confidential virtual machine and loads the communication status information between the first confidential container and the virtual IO device into the virtual IO device driver of the second confidential virtual machine, when the backup confidential container is subsequently run, the communication status information between the first confidential container and the virtual IO device can be accurately located from the virtual IO device driver of the second confidential virtual machine based on the identifier of the first confidential container, thereby ensuring the accurate implementation of the migration process.

[0203] S508: The processor of the second computing device 200 runs the backup confidential container.

[0204] Optionally, the processor of the first computing device 100 may also encrypt the state information of the first confidential container using an encryption key in the first non-trusted execution environment.

[0205] Optionally, the processor of the second computing device 200 may also decrypt the encrypted information using the encryption key in the second non-trusted execution environment.

[0206] It can be understood that the processor of the first computing device 100 runs the first confidential virtual machine for encryption operations, and the processor of the second computing device 200 runs the second confidential virtual machine for decryption operations. In comparison, the processor of the first computing device 100 performs encryption operations in the first non-trusted execution environment, and the processor of the second computing device 200 performs decryption operations in the second non-trusted execution environment. Since the first confidential virtual machine is in the first TEE and the second confidential virtual machine is in the second TEE, the security of the first TEE is higher than that of the first non-trusted execution environment, and the security of the second TEE is higher than that of the second non-trusted execution environment. Therefore, the former implementation method can better protect the security of the status information of the first confidential container and reduce the risk of leakage of the status information of the first confidential container.

[0207] Next, the specific implementation process of step S502 is described with reference to the flowchart shown in FIG6 .

[0208] S610: The processor of the first computing device 100 obtains a first public key and a first private key.

[0209] The first public key and the first private key may be generated by the processor of the first computing device 100, or may be generated by another key management device and then sent to the first computing device 100. When the first public key and the first private key are generated by the processor of the first computing device 100, they may be generated by the processor of the first computing device 100 in the first untrusted execution environment, or in the first TEE, such as by running the first confidential virtual machine in the first TEE, which is not specifically limited in this application.

[0210] After the processor of the first computing device 100 obtains the first public key and the first private key, the first public key and the first private key can be stored in the memory of the first non-trusted execution environment, or in the memory of the first TEE. It can be understood that since the security of the first TEE is higher than the security of the first non-trusted execution environment, the first private key is stored in the memory of the first TEE. Compared with storing it in the memory of the first non-trusted execution environment, the former storage method is more secure.

[0211] S620: The processor of the second computing device 200 obtains a second public key and a second private key.

[0212] The second public key and the second private key may be generated by the processor of the second computing device 200, or may be generated by another key management device and then sent to the second computing device 200. When the second public key and the second private key are generated by the processor of the second computing device 200, they may be generated by the processor of the second computing device 200 in the second untrusted execution environment, or they may be generated in the second TEE, such as by running a second confidential virtual machine in the second TEE, which is not specifically limited in this application.

[0213] After the processor of the second computing device 200 obtains the second public key and the second private key, it can store the second public key and the second private key in the memory of the second non-trusted execution environment, or in the memory of the second TEE. It can be understood that since the security of the second TEE is higher than the security of the second non-trusted execution environment, the second private key is stored in the memory of the second TEE. Compared with storing it in the memory of the second non-trusted execution environment, the former storage method is more secure.

[0214] S630: The processor of the first computing device 100 calls the first communication interface in the first non-trusted execution environment and sends the first public key to the second communication interface.

[0215] S640: The processor of the second computing device 200 calls the second communication interface in the second non-trusted execution environment and sends the second public key to the first communication interface.

[0216] S650: After receiving the second public key at the first communication interface, the processor of the first computing device 100 obtains the second public key in the first untrusted execution environment.

[0217] Specifically, after receiving the second public key, the first communication interface can store the second public key in the memory of the first non-trusted execution environment. Subsequently, the processor of the first computing device 100 can obtain the second public key from the memory of the first non-trusted execution environment.

[0218] S660: After receiving the first public key at the second communication interface, the processor of the second computing device 200 obtains the first public key in the second non-trusted execution environment.

[0219] Specifically, after receiving the first public key, the second communication interface can store the first public key in the memory of the second non-trusted execution environment. Subsequently, the processor of the second computing device 200 can obtain the first public key from the memory of the second non-trusted execution environment.

[0220] S670: The processor of the first computing device 100 generates an encryption key according to the first private key and the second public key.

[0221] In one possible embodiment, the processor of the first computing device 100 generates an encryption key in the first TEE based on the first private key and the second public key, such as the first computing device 100 runs the first confidential virtual machine to generate an encryption key based on the first private key and the second public key.

[0222] In another possible embodiment, the processor of the first computing device 100 generates an encryption key according to the first private key and the second public key in the first non-trusted execution environment.

[0223] After generating the encryption key, the processor of the first computing device 100 may store the encryption key in the memory of the first non-trusted execution environment or the memory of the first TEE.

[0224] S680: The processor of the second computing device 200 generates an encryption key according to the second private key and the first public key.

[0225] In one possible embodiment, the processor of the second computing device 200 generates an encryption key in the second TEE based on the second private key and the first public key, such as the second computing device 200 runs a second confidential virtual machine to generate an encryption key based on the second private key and the first public key.

[0226] In another possible embodiment, the processor of the second computing device 200 generates an encryption key according to the second private key and the first public key in the second non-trusted execution environment.

[0227] After generating the encryption key, the processor of the second computing device 200 may store the encryption key in the memory of the second non-trusted execution environment or the memory of the second TEE.

[0228] It can be understood that the processor of the first computing device 100 generates an encryption key based on the first private key and the second public key in the first TEE, and the processor of the second computing device 200 generates an encryption key based on the second private key and the first public key in the second TEE, and the processor of the first computing device 100 stores the encryption key in the memory of the first TEE, and the processor of the second computing device 200 stores the encryption key in the memory of the second TEE. In comparison, the processor of the first computing device 100 generates an encryption key based on the first private key and the second public key in the first non-trusted execution environment, and the processor of the second computing device 200 stores the encryption key in the memory of the second TEE. The execution environment generates an encryption key based on the second private key and the first public key. The processor of the first computing device 100 stores the encryption key in the memory of the first non-trusted execution environment, and the processor of the second computing device 200 stores the encryption key in the memory of the second non-trusted execution environment. Since the security of the first TEE is higher than that of the first non-trusted execution environment, and the security of the second TEE is higher than that of the second non-trusted execution environment, the former implementation method can better protect the security of the encryption key, thereby better protecting the security of the status information of the first confidential container and reducing the risk of leakage of the status information of the first confidential container.

[0229] It can also be understood that if, during the key negotiation process shown in FIG6 , the operations performed by the processor of the first computing device 100 / second computing device 200 in the first TEE / second TEE are all performed by the processor of the first computing device 100 / second computing device 200 running the first confidential virtual machine / second confidential virtual machine, since the first confidential virtual machine is isolated from other confidential virtual machines in the first computing device 100, and the second confidential virtual machine is isolated from other confidential virtual machines in the second computing device 200, it is possible to achieve that the first private key is only known to the first confidential virtual machine, the second private key is only known to the second confidential virtual machine, and the encryption key is only known to the first confidential virtual machine and the second confidential virtual machine. Other confidential virtual machines in the first computing device 100 will not know the first private key and the encryption key, and other confidential virtual machines in the second computing device 200 will not know the second private key and the encryption key. Therefore, the risk of leakage of the status information of the first confidential container can be further reduced, and data security can be better protected.

[0230] The following describes the steps shown in FIG. 6 with reference to two specific examples.

[0231] Example 1: In S601, the processor of the first computing device 100 runs the first confidential virtual machine to generate a first private key that is a random number a, and a first public key that is A=g^a(mod p). In S602, the processor of the second computing device 200 runs the second confidential virtual machine to generate a second private key that is a random number b, and a second public key that is B=g^b(mod p), where g and p can be preset in the processor of the first computing device 100 and the processor of the second computing device 200, or can be pre-negotiated by the processor of the first computing device 100 and the processor of the second computing device 200. Then, in S603-S604, the processor of the first computing device 100 runs the first confidential virtual machine and the processor of the second computing device 2000 runs the second confidential virtual machine to exchange public keys. Next, in S605, the processor of the first computing device 100 runs the first confidential virtual machine and obtains the encryption key S=B^a(mod p) according to B, a and p. p), in S606, the processor of the second computing device 200 runs the second confidential virtual machine, and obtains the encryption key S=A^b(mod p) according to A, b and p.

[0232] Example 2: In S601, the processor of the first computing device 100 runs the first confidential virtual machine to generate a first private key that is a random number a, and a first public key that is A=a*Q(x,y). In S602, the processor of the second computing device 200 runs the second confidential virtual machine to generate a second private key that is a random number b, and a second public key that is B=b*Q(x,y), where Q(x,y) can be pre-negotiated by the processor of the first computing device 100 and the processor of the second computing device 200, such as a pre-negotiated base point G on a certain elliptic curve. Then, in S603-S604, the first computing device 100 and the second computing device 200 exchange public keys. Next, in S605, the processor of the first computing device 100 runs the first confidential virtual machine, and calculates the encryption key S=a*B=a*b*Q(x,y) based on the first private key a and the second public key B. In S606, the processor of the second computing device 200 can run the second confidential virtual machine, and calculate the encryption key S=b*A=b*a*Q(x,y) based on the second private key b and the first public key A.

[0233] It should be understood that the above-mentioned Example 1 and Example 2 are merely examples of implementation methods in which the processor of the first computing device 100 and the processor of the second computing device 200 negotiate to obtain an encryption key, and should not be regarded as specific limitations.

[0234] It can be seen that in Examples 1 and 2 above, the first confidential virtual machine and the second confidential virtual machine generate encryption keys based on public information (i.e., the other party's public key) and their respective private information (i.e., private key). Private information is not transmitted between the two. Even if the public information is stolen during transmission, the thief cannot steal the encryption key because the private information is not leaked. Therefore, the security of the state information of the first container subsequently encrypted with the encryption key can be protected during transmission. In addition, the first confidential virtual machine and the second confidential virtual machine are both located in the TEE. Due to the high security of the TEE, the encryption key can be well protected.

[0235] On the basis that the processor of the first computing device 100 runs the first confidential virtual machine and the processor of the second computing device 200 runs the second confidential virtual machine to negotiate and obtain the encryption key, in order to further protect the security of the state information of the first confidential container, in a possible embodiment, before the processor of the first computing device 100 runs the first confidential virtual machine and the processor of the second computing device 200 runs the second confidential virtual machine to negotiate the key, the processor of the first computing device 100 can run the first confidential virtual machine to send an authentication request to the second confidential virtual machine, requesting to obtain the authentication information of the second confidential virtual machine to authenticate the second confidential virtual machine. If the authentication is successful, the encryption key is negotiated with the second confidential virtual machine. Otherwise, no key negotiation is performed.

[0236] Specifically, the processor of the first computing device 100 can run the first confidential virtual machine to generate an authentication request, and then call the first communication interface in the first non-trusted execution environment to send an authentication request to the second communication interface. After receiving the authentication request, the second communication interface can run the second confidential virtual machine by the processor of the second computing device 200, obtain the authentication information of the second confidential virtual machine according to the authentication request, and then call the second communication interface in the second non-trusted execution environment to return the authentication information of the second confidential virtual machine to the first communication interface. After receiving the authentication information of the second confidential virtual machine, the first communication interface can store the authentication information of the second confidential virtual machine in the memory of the first non-trusted execution environment. Subsequently, the processor of the first computing device 100 can obtain the authentication information of the second confidential virtual machine from the memory of the first non-trusted execution environment, and run the first confidential virtual machine to authenticate the second confidential virtual machine according to the authentication information of the second confidential virtual machine.

[0237] The authentication information of the second confidential virtual machine may include the current measurement value of the configuration of the second confidential virtual machine and the historical measurement value of the configuration of the second confidential virtual machine. Optionally, the authentication information of the second confidential virtual machine may include the certificate of the second confidential virtual machine (which may also be a certificate chain), and the historical measurement value of the configuration of the second confidential virtual machine may be carried in the certificate of the second confidential virtual machine. Among them, the configuration of the second confidential virtual machine may include the virtual processor core type, the number of virtual processor cores, the memory address, the name and version of the APP installed on the second confidential virtual machine, and the image of the second confidential virtual machine; the measurement value of the configuration is used to measure whether the configuration has been tampered with, for example, comparing the current number of virtual processor cores with the historical number of virtual processor cores recorded in the certificate. If they are the same, it means that the number of virtual processor cores has not been tampered with. Otherwise, it means that the number of virtual processor cores has been tampered with (increased or decreased). For example, comparing the current hash value of the image of the second confidential virtual machine with the historical hash value of the image of the second confidential virtual machine recorded in the certificate. If they are the same, it means that the image has not been tampered with. Otherwise, it means that the image has been tampered with. In the present application, the configuration of the second confidential virtual machine is tampered with, which can be understood as the second confidential virtual machine is incomplete and is an untrusted virtual machine.

[0238] Optionally, the authentication information of the second confidential virtual machine may include the identity information of the second confidential virtual machine, which is used to uniquely identify and recognize the second confidential virtual machine. The identity information of the second confidential virtual machine may include the name of the second confidential virtual machine, a universally unique identifier (UUID), a media access control address (MAC address), operating system information (such as operating system type, version number), and virtual hardware information (such as the number of CPU cores, memory size, disk capacity, device configuration, etc.). It can be understood that the processor of the first computing device 100 runs the first confidential virtual machine, and the identity of the second confidential virtual machine can be authenticated through the identity information of the second confidential virtual machine. If the identity authentication is passed, the second confidential virtual machine is determined to be a trusted virtual machine. Otherwise, the second confidential virtual machine is determined to be an untrusted virtual machine. Optionally, the certificate of the second confidential virtual machine also includes the identity information provided by the second confidential virtual machine when applying for the certificate, the validity period of the certificate, the issuing authority of the certificate, the digital signature of the certificate, etc.

[0239] Specifically, the processor of the first computing device 100 can run the first confidential virtual machine to first authenticate the legitimacy and integrity of the certificate of the second confidential virtual machine, such as determining whether the certificate is valid based on the validity period of the certificate, determining whether the issuing organization of the certificate is a legal organization, and determining whether the certificate has been tampered with based on the digital signature of the certificate. When it is determined that the certificate is legal and complete, whether the second confidential virtual machine is a trusted virtual machine is determined by checking the current measurement value and certificate of the configuration of the second confidential virtual machine, and / or by checking whether the identity information of the second confidential virtual machine matches the identity information recorded in the certificate.

[0240] To protect the security of the second computing device 200 and the second confidential virtual machine in the second computing device 200, in one possible embodiment, before the processor of the second computing device 200 runs the second confidential virtual machine and the processor of the first computing device 100 runs the first confidential virtual machine to negotiate and obtain an encryption key, the processor of the first computing device 100 may also run the first confidential virtual machine to send authentication information of the first confidential virtual machine to the second confidential virtual machine, so that the second computing device 200 authenticates the first confidential virtual machine based on the authentication information. If the authentication is successful, the encryption key is negotiated with the first computing device 100; otherwise, no key negotiation is performed. Optionally, the authentication information of the first confidential virtual machine may be included in the authentication request sent by the first confidential virtual machine to the second confidential virtual machine as described above. In other words, the authentication request is also used to request the second confidential virtual machine to authenticate the first confidential virtual machine based on the authentication information of the first confidential virtual machine.

[0241] Specifically, the processor of the second computing device 200 can run the first confidential virtual machine to obtain the authentication information of the first confidential virtual machine, and generate an authentication request carrying the authentication information of the first confidential virtual machine, and then call the first communication interface in the first non-trusted execution environment to send the authentication request to the second communication interface. After receiving the authentication request, the second communication interface can store the authentication information of the first confidential virtual machine carried in the authentication request in the memory of the second non-trusted execution environment. Subsequently, the processor of the second computing device 200 can obtain the authentication information of the first confidential virtual machine from the memory of the second non-trusted execution environment, and run the second confidential virtual machine to authenticate the first confidential virtual machine based on the authentication information of the first confidential virtual machine.

[0242] The authentication information of the first confidential virtual machine may include the current measurement value of the configuration of the first confidential virtual machine and the historical measurement value of the configuration of the first confidential virtual machine. Optionally, the authentication information of the first confidential virtual machine may include the certificate of the first confidential virtual machine (or a certificate chain), and the historical measurement value of the configuration of the first confidential virtual machine may be carried in the certificate of the first confidential virtual machine. The configuration of the first confidential virtual machine may include the virtual processor core type, the number of virtual processor cores, the memory address, the name and version of the APP installed on the first confidential virtual machine, and the image of the first confidential virtual machine. The measurement value of the configuration is used to measure whether the configuration has been tampered with. For example, the current number of virtual processor cores is compared with the historical number of virtual processor cores recorded in the certificate. If they are the same, it indicates that the number of virtual processor cores has not been tampered with. Otherwise, it indicates that the number of virtual processor cores has been tampered with (increased or decreased). For another example, the current hash value of the image of the first confidential virtual machine is compared with the historical hash value of the image of the first confidential virtual machine recorded in the certificate. If they are the same, it indicates that the image has not been tampered with. Otherwise, it indicates that the image has been tampered with. In the present application, the configuration of the first confidential virtual machine is tampered with, which can be understood as the first confidential virtual machine is incomplete and is an untrusted virtual machine.

[0243] Optionally, the authentication information of the first confidential virtual machine may include the identity information of the first confidential virtual machine, which is used to uniquely identify and recognize the first confidential virtual machine. The identity information of the first confidential virtual machine may include the name of the first confidential virtual machine, a universally unique identifier (UUID), a media access control address (MAC address), operating system information (such as operating system type, version number), and virtual hardware information (such as the number of CPU cores, memory size, disk capacity, device configuration, etc.). It can be understood that the processor of the second computing device 200 runs the second confidential virtual machine and authenticates the identity of the first confidential virtual machine through the identity information of the first confidential virtual machine. If the identity authentication is passed, the first confidential virtual machine is determined to be a trusted virtual machine. Otherwise, the first confidential virtual machine is determined to be an untrusted virtual machine. Optionally, the certificate of the first confidential virtual machine also includes the identity information provided by the first confidential virtual machine when applying for the certificate, the validity period of the certificate, the issuing authority of the certificate, the digital signature of the certificate, etc.

[0244] Specifically, the processor of the second computing device 200 runs the second confidential virtual machine and can first authenticate the legitimacy and integrity of the certificate of the first confidential virtual machine, such as determining whether the certificate is valid based on the validity period of the certificate, determining whether the issuing organization of the certificate is a legal organization, and determining whether the certificate has been tampered with based on the digital signature of the certificate. When it is determined that the certificate is legal and complete, whether the first confidential virtual machine is a trusted virtual machine is determined by checking whether the current measurement value of the configuration of the first confidential virtual machine and the historical measurement value of the configuration of the first confidential virtual machine recorded in the certificate are consistent (identical), and / or whether the first confidential virtual machine is a trusted virtual machine is determined by checking whether the identity information of the first confidential virtual machine and the identity information recorded in the certificate match.

[0245] In a possible embodiment, after the processor of the first computing device 100 and the processor of the second computing device 200 negotiate to obtain the encryption key, and before the processor of the first computing device 100 uses the encryption key to encrypt the state information of the first confidential virtual machine, the processor of the first computing device 100 may also verify the validity of the encryption key, that is, verify whether the encryption key held by the processor of the first computing device 100 is the same as the encryption key held by the processor of the second computing device 200, so as to ensure the accurate migration process of the first confidential container.

[0246] Specifically, the processor of the first computing device 100 may verify the validity of the encryption key in the following manner:

[0247] After the processor of the first computing device 100 and the processor of the second computing device 200 negotiate to obtain the encryption key, the processor of the second computing device 200 can generate a message authentication code based on the encryption key and the first message in the second non-trusted execution environment / second TEE (in order to distinguish it from the message authentication code generated by the processor of the first computing device 100 below, the message authentication code generated by the processor of the second computing device 200 is referred to as the second message authentication code, and the message authentication code generated by the processor of the first computing device 100 is referred to as the first message authentication code), and call the second communication interface in the second non-trusted execution environment to send the second message to the first communication interface. Authentication code and the first message. After receiving the second message authentication code and the first message, the first communication interface can store the second message authentication code and the first message in the memory of the first non-trusted execution environment. Subsequently, the processor of the first computing device 100 can obtain the second message authentication code and the first message from the memory of the first non-trusted execution environment, and generate the first message authentication code in the first non-trusted execution environment / first TEE based on the encryption key held by itself and the first message, and then compare the first message authentication code and the second message authentication code. If it is determined that the two are the same, it is determined that the encryption key held by itself is valid. Otherwise, it is determined that the encryption key held by itself is invalid.

[0248] It can be understood that the processor of the second computing device 200 generates a second message authentication code in the second TEE, and the processor of the first computing device 100 generates a first message authentication code in the first TEE. In comparison, the processor of the second computing device 200 generates the second message authentication code in the second non-trusted execution environment, and the processor of the first computing device 100 generates the first message authentication code in the first non-trusted execution environment. Since the security of the first TEE is higher than the security of the first non-trusted execution environment, and the security of the second TEE is higher than the security of the second non-trusted execution environment, the former implementation method can better protect the security of the encryption key, thereby better protecting the security of the status information of the first confidential container, and reducing the risk of leakage of the status information of the first confidential container.

[0249] In a more specific embodiment, based on the encryption key obtained by negotiation between the processor of the first computing device 100 running the first confidential virtual machine and the processor of the second computing device 200 running the second confidential virtual machine, the processor of the second computing device 200 generates a second message authentication code in the second TEE, and generates a second message authentication code for the processor of the second computing device 200 running the second confidential virtual machine. The processor of the first computing device 100 generates a first message authentication code in the first TEE, and generates a first message authentication code for the processor of the first computing device 100 running the first confidential virtual machine, compared with the processor of the second computing device 200 running other software in the second TEE (such as running the second virtual machine manager). code, the processor of the first computing device 100 runs other software in the first TEE (such as running the first virtual machine manager) to generate a first message authentication code. Since the second confidential virtual machine is isolated from other confidential virtual machines in the second TEE, the second confidential virtual machine shares the second virtual machine manager with other confidential virtual machines in the second TEE, and the first confidential virtual machine is isolated from other confidential virtual machines in the first TEE, the first confidential virtual machine shares the first virtual machine manager with other confidential virtual machines in the first TEE. The message authentication code generation operation is performed by the second confidential virtual machine and the first confidential virtual machine, which can better protect the security of the encryption key, thereby better protecting the security of the state information of the first confidential container and reducing the risk of leakage of the state information of the first confidential container.

[0250] In a specific implementation, the processor of the second computing device 200 generates the second message authentication code based on its own encryption key and the first message, and the processor of the first computing device 100 generates the first message authentication code based on its own encryption key and the first message. The encryption key and the first message can be signed using the same signature algorithm, and the resulting signature information is the message authentication code. Signature algorithms include but are not limited to RSA, DSA, etc.

[0251] As can be seen from Figure 3, the first TEE is deployed with a first virtual machine manager, the second TEE is deployed with a second virtual machine manager, the first non-trusted execution environment is deployed with a third virtual machine manager, and the second non-trusted execution environment is deployed with a fourth virtual machine manager. In a possible embodiment, the above-mentioned operation of calling the first communication interface executed by the processor of the first computing device 100 in the first non-trusted execution environment can be executed by the processor of the first computing device 100 running the third virtual machine manager, and the above-mentioned operation of calling the second communication interface executed by the processor of the second computing device 200 in the second non-trusted execution environment can be executed by the processor of the second computing device 200 running the fourth virtual machine manager.

[0252] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0253] The above describes in detail the container migration method and system provided by the present application. Based on the same inventive concept, the following further introduces the container migration device and computing equipment provided by the present application.

[0254] It should be understood that the unit modules inside the container migration device can also be divided into multiple types. Each module can be a software module, a hardware module, or partly a software module and partly a hardware module. This application does not limit this.

[0255] Refer to Figure 7, which is a structural diagram of another container migration device 700 provided in an embodiment of the present application, which can be applied to the second computing device 200 shown in Figure 3. As shown in Figure 7, the device 700 includes: an acquisition module 710, a decryption module 720 and a synchronization module 730.

[0256] The following is an illustrative introduction to the functions of each module of the container migration device 700. It should be understood that the functions of each module described below are merely functions that the container migration device 700 may have in some embodiments of this application, and this application does not limit the functions of each module.

[0257] An acquisition module 710 is configured to obtain encrypted information of status information of a first confidential container, where the first confidential container is deployed in a first TEE, and the first TEE is a TEE of the first computing device 100;

[0258] a decryption module 720, configured to decrypt the encrypted information to obtain status information of the first confidential container;

[0259] The synchronization module 730 is configured to synchronize the status of the backup confidential container with the first confidential container based on the status information of the first confidential container.

[0260] In some possible embodiments, the state of the first confidential container includes the memory state of the first confidential container, or the state of the first confidential container includes the memory state of the first confidential container and the communication state between the first confidential container and the virtual IO device.

[0261] In some possible embodiments, the second TEE includes a second confidential virtual machine, the backup confidential container is deployed in the second confidential virtual machine, the acquisition module 710 is deployed in a second untrusted execution environment, and the decryption module 720 is deployed in the second confidential virtual machine.

[0262] In some possible embodiments, as shown in Figure 7, the device 700 also includes a key negotiation module 740, which is deployed on the second confidential virtual machine; the encrypted information is obtained by encrypting the status information of the first confidential container using an encryption key, and the encryption key is obtained through negotiation between the key negotiation module 740 and the first confidential virtual machine. The encryption key is also used to decrypt the encrypted information. The first confidential virtual machine is deployed on the first TEE, and the first confidential container is deployed on the first confidential virtual machine.

[0263] In some possible embodiments, as shown in FIG7 , the apparatus 700 further includes an authentication module 750 deployed in the second confidential virtual machine;

[0264] The acquisition module 710 is further configured to acquire authentication information of the first confidential virtual machine;

[0265] The key negotiation module 740 is configured to negotiate with the first confidential virtual machine to obtain an encryption key when the authentication module 750 successfully authenticates the first confidential virtual machine based on the authentication information of the first confidential virtual machine.

[0266] In some possible embodiments, as shown in FIG7 , the apparatus 700 further includes a sending module 760 , which is deployed in the second non-trusted execution environment;

[0267] The authentication module 750 is further configured to obtain authentication information of the second confidential virtual machine;

[0268] The above-mentioned sending module 760 is used to call the second communication interface to send the authentication information of the second confidential virtual machine to the first communication interface. The second communication interface is the communication interface of the second computing device 200, and the first communication interface is the communication interface of the first computing device 100.

[0269] In some possible embodiments, the authentication information of the first confidential virtual machine includes a current measurement value of a configuration of the first confidential virtual machine and a historical measurement value of a configuration of the first confidential virtual machine;

[0270] The authentication module 750 is configured to authenticate the first confidential virtual machine if it is determined that the current measurement value of the configuration of the first confidential virtual machine is the same as the historical measurement value of the configuration of the first confidential virtual machine.

[0271] In some possible embodiments, the authentication module 750 is further configured to generate a message authentication code based on the encryption key and the first message; the sending module 760 is further configured to call the second communication interface and send the first message and the message authentication code to the first communication interface.

[0272] In some possible embodiments, the communication state information between the first confidential container and the virtual IO device includes an identifier of the first confidential container;

[0273] The above-mentioned synchronization module 730 is specifically used to: load the memory status information of the first confidential container into the memory of the backup confidential container, and load the communication status information between the first confidential container and the virtual IO device into the virtual IO device driver of the second confidential virtual machine; based on the memory status information of the first confidential container in the memory of the backup confidential container, and the communication status information between the first confidential container and the virtual IO device located in the virtual IO device driver based on the identifier of the first confidential container, run the backup confidential container, thereby realizing memory status synchronization between the backup confidential container and the first confidential container, and realizing communication status synchronization between the backup confidential container and the first confidential container.

[0274] Refer to Figure 8, which is a structural diagram of a container migration device 800 provided in an embodiment of the present application, which can be applied to the first computing device 100 shown in Figure 3. As shown in Figure 8, the device 800 includes: a sending module 810 and an encryption module 820.

[0275] The following is an illustrative introduction to the functions of each module of the container migration device 800. It should be understood that the functions of each module described below are merely functions that the container migration device 800 may have in some embodiments of this application, and this application does not limit the functions of each module.

[0276] An encryption module 820 is configured to encrypt the state information of the first confidential container in the first TEE to obtain encrypted information;

[0277] The sending module 810 is used to call the first communication interface in the first non-trusted execution environment and send encrypted information to the second computing device 200. The encrypted information is used to migrate the status information of the first confidential container to the backup confidential container. The backup confidential container is deployed in the second TEE. The second TEE is the TEE of the second computing device 200, and the first communication interface is the communication interface of the first computing device 100.

[0278] In some possible embodiments, the state of the first confidential container includes the memory state of the first confidential container, or the state of the first confidential container includes the memory state of the first confidential container and the communication state between the first confidential container and the virtual IO device.

[0279] In some possible embodiments, the first TEE includes a first confidential virtual machine, and the first confidential container is located in the first confidential virtual machine; the above-mentioned encryption module 820 is deployed in the first confidential virtual machine, and the above-mentioned sending module 810 is deployed in the first untrusted execution environment.

[0280] In some possible embodiments, as shown in Figure 8, the above-mentioned device 800 also includes a key negotiation module 830, which is deployed on the first confidential virtual machine; the encrypted information is obtained by encrypting the status information of the first confidential container using an encryption key, and the encryption key is obtained through negotiation between the key negotiation module 830 and the second confidential virtual machine, and the encryption key is also used to decrypt the encrypted information.

[0281] In some possible embodiments, as shown in FIG8 , the apparatus 800 further includes an authentication module 840 deployed in the first confidential virtual machine;

[0282] The authentication module 840 is used to obtain authentication information of the first confidential virtual machine;

[0283] The sending module 810 is used to call the first communication interface to send the authentication information of the first confidential virtual machine to the second communication interface;

[0284] In some possible embodiments, as shown in FIG8 , the apparatus 800 further includes an acquisition module 850 deployed in the first non-trusted execution environment, and the acquisition module 850 is further configured to acquire authentication information of the second confidential virtual machine;

[0285] The key negotiation module 830 is configured to negotiate with the second confidential virtual machine to obtain an encryption key when the authentication module 840 successfully authenticates the second confidential virtual machine based on the authentication information of the second confidential virtual machine.

[0286] In some possible embodiments, the authentication information of the second confidential virtual machine includes a current measurement value of a configuration of the second confidential virtual machine and a historical measurement value of a configuration of the second confidential virtual machine;

[0287] The authentication module 840 is configured to authenticate the second confidential virtual machine if it is determined that the current measurement value of the configuration of the second confidential virtual machine is the same as the historical measurement value of the configuration of the second confidential virtual machine.

[0288] In some possible embodiments, the acquisition module 850 is configured to acquire the first message and a message authentication code from the second confidential virtual machine, where the message authentication code is generated by the second confidential virtual machine based on the encryption key and the first message;

[0289] The encryption module 820 is configured to encrypt the status information of the first confidential container using the encryption key when the authentication module 840 determines that the encryption key is a valid key based on the first message and the message authentication code.

[0290] In some possible embodiments, the communication status information between the first confidential container and the virtual IO device includes an identifier of the first confidential container; before the above-mentioned encryption module 820 is used to encrypt the status information of the first confidential container to obtain encrypted information, the encryption module 820 is also used to: locate the communication status information between the first confidential container and the virtual IO device in the virtual IO device driver of the first confidential virtual machine based on the identifier of the first confidential container.

[0291] Specifically, the specific implementation of various operations performed by the above-mentioned container migration device 700 and container migration device 800 can refer to the description in the relevant content of the above-mentioned container migration method embodiment. For the sake of brevity of the specification, it will not be repeated here.

[0292] 9, which is a schematic diagram of the structure of a computing device 900 provided in an embodiment of the present application, wherein the computing device 900 includes: a processor 910, a memory unit 920, a communication interface 930, a storage 940, an input device 950, and an output device 960, wherein the processor 910, the memory unit 920, the communication interface 930, the storage 940, the input device 950, and the output device 960 can be interconnected via a bus 970.

[0293] The processor 910 can read the program code (including instructions) stored in the memory unit 920 and execute the program code stored in the memory unit 920, so that the computing device 900 executes the steps performed by the first computing device 100 or the second computing device 200 in the container migration method provided by the above method embodiment.

[0294] The processor 910 can have a variety of specific implementation forms. For example, the processor 910 can be at least one CPU, as shown in Figure 9, including CPU0 and CPU1. The processor 910 can also be a graphics processing unit (GPU), etc. The processor 910 can also be a single-core processor or a multi-core processor. The processor 910 can be a combination of a CPU and a hardware chip. The above-mentioned hardware chip can be implemented as an application-specific integrated circuit (ASIC) or a programmable logic device (PLD). The above-mentioned PLD can be implemented as a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof. The processor 910 can also be implemented solely using a logic device with built-in processing logic, such as an FPGA or a digital signal processor (DSP).

[0295] The memory unit 920 is used to store the kernel, program codes, and program data generated when the processor 910 executes the program codes stored in the memory unit 920 .

[0296] When the computing device 900 is used to execute the steps performed by the second computing device 200 in the container migration method provided by the above-mentioned method embodiment, the program code stored in the memory unit 920 includes: code of the acquisition module 710, code of the decryption module 720, code of the synchronization module 730, code of the key negotiation module 740, code of the authentication module 750 and code of the sending module 760, etc., and the program data stored in the memory unit 920 includes: encryption key, status information of the first confidential container, encryption information, etc.

[0297] When the computing device 900 is used to execute the steps performed by the first computing device 100 in the container migration method provided by the above-mentioned method embodiment, the program code stored in the memory unit 920 includes: code of the sending module 810, code of the encryption module 820, code of the key negotiation module 830, code of the authentication module 840 and code of the acquisition module 850, etc., and the program data stored in the memory unit 920 includes: encryption key, status information of the first confidential container, encryption information, etc.

[0298] The communication interface 930 may be a wired interface (e.g., an Ethernet interface, a fiber optic interface, or other types of interfaces (e.g., an InfiniBand (IB) interface)) or a wireless interface (e.g., a cellular network interface or a wireless local area network interface) for communicating with other computing devices or apparatuses. When the communication interface 930 is a wired interface, the communication interface 930 may employ a protocol suite above the Transmission Control Protocol / Internet Protocol (TCP / IP), such as the Remote Function Call (RFC) protocol, the Simple Object Access Protocol (SOAP) protocol, the Simple Network Management Protocol (SNMP) protocol, the Common Object Request Broker Architecture (CORBA) protocol, and distributed protocols, etc.

[0299] The memory 940 may be a non-volatile memory, such as a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The memory 940 may also be a volatile memory, such as a random access memory (RAM), which is used as an external cache.

[0300] The input device 950 may include a mouse and a keyboard, etc. The user may input data or instructions to the computing device 900 through the input device 950, such as inputting a container migration instruction (the instruction instructing the migration of the first confidential container), etc.

[0301] The output device 960 may include a display, and the computing device 900 may provide data to the user through the display, such as displaying a migration completion notification to the user after the migration of the first confidential container is completed. The display may include a cathode ray tube (CRT), a plasma display panel (PDP), a liquid crystal display (LCD), and the like. Taking an LCD as an example, the liquid crystal display includes a liquid crystal panel and a backlight module, wherein the liquid crystal display panel includes a polarizing film, a glass substrate, a black matrix, a color filter, a protective film, a common electrode, a calibration layer, a liquid crystal layer (liquid crystal, spacer, sealant), a capacitor, a display electrode, a prism layer, and a light-scattering layer. The backlight module includes: an illumination light source, a reflector, a light guide plate, a diffuser, a brightness enhancement film (prism sheet), and a frame, and the like.

[0302] Bus 970 can be a PCIE or extended industry standard architecture (EISA) bus, etc. Bus 970 can be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, FIG9 shows only one thick line, but this does not mean that there is only one bus or only one type of bus.

[0303] It should be understood that the computing device 900 of the embodiment of the present application may correspond to the computing device including the container migration device 700 or the container migration device 800 in the embodiment of the present application, and may correspond to executing the corresponding subjects in the methods shown in Figures 4, 5, and 6 in the embodiment of the present application, and the operations and / or functions of each module in the computing device 900 are respectively for implementing the corresponding processes of the methods shown in Figures 4, 5, and 6. For the sake of brevity, they will not be repeated here.

[0304] It should be understood that the computing device 900 is only an example provided for an embodiment of the present application, and the computing device 900 may have more or fewer components than those shown in FIG. 9 , may combine two or more components, or may have different configuration implementations of the components.

[0305] An embodiment of the present application also provides a container migration system, which may include the above-mentioned container migration device 700 and container migration device 800.

[0306] An embodiment of the present application further provides a computer-readable storage medium, in which instructions are stored. When the instructions are executed, some or all of the steps of the container migration method described in the above embodiment can be implemented.

[0307] The embodiments of the present application also provide a computer program product. When the computer program product is read and executed by a computer, it can implement some or all of the steps of the container migration method described in the above method embodiment.

[0308] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0309] In the above embodiments, it can be implemented in whole or in part by software, hardware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website, computer, server or data center to another website, computer, server or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrations. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium, or a semiconductor medium.

[0310] The above is only a specific embodiment of the present application. Those skilled in the art may conceive of changes or substitutions based on the specific embodiment provided in this application, and all such changes or substitutions shall fall within the scope of protection of this application.

Claims

1. A container migration system, characterized in that: The system includes a first computing device and a second computing device, the software and hardware resources of the first computing device are divided into a first trusted execution environment TEE and a first untrusted execution environment, the software and hardware resources of the second computing device are divided into a second TEE and a second untrusted execution environment, the first TEE includes a first container, and the second TEE includes a backup container; The first computing device is used to send encrypted information of the state information of the first container to the second computing device; The second computing device is used to receive and decrypt the encrypted information to obtain the state information of the first container; The second computing device is further configured to synchronize the status of the backup container with the first container based on the status information of the first container.

2. The system according to claim 1, characterized in that The state of the first container includes a memory state of the first container, or the state of the first container includes a memory state of the first container and a communication state between the first container and a virtual input / output (IO) device.

3. The system according to claim 2, characterized in that The first TEE includes a first virtual machine, the second TEE includes a second virtual machine, the first container is located in the first virtual machine, and the backup container is located in the second virtual machine; The processor of the first computing device is used to run the first virtual machine and encrypt the state information of the first container to obtain the encrypted information; The processor of the first computing device is used to call a first communication interface in the first untrusted execution environment to send the encrypted information to a second communication interface, the first communication interface is the communication interface of the first computing device, and the second communication interface is the communication interface of the second computing device; The processor of the second computing device is used to obtain the encrypted information in the second untrusted execution environment; The processor of the second computing device is used to run the second virtual machine and decrypt the encrypted information to obtain the state information of the first container; The processor of the second computing device is used to run the second virtual machine and synchronize the status of the backup container with the first container based on the status information of the first container.

4. The system according to claim 3, characterized in that The encrypted information is obtained by encrypting the state information of the first container using an encryption key, the encryption key is obtained by the first virtual machine and the second virtual machine through negotiation, and the encryption key is also used to decrypt the encrypted information.

5. The system according to claim 4, characterized in that The processor of the first computing device is used to run the first virtual machine to obtain authentication information of the first virtual machine; The processor of the first computing device is used to call the first communication interface in the first untrusted execution environment and send the authentication information of the first virtual machine to the second communication interface; The processor of the second computing device is used to obtain authentication information of the first virtual machine in the second untrusted execution environment; The processor of the second computing device is used to run the second virtual machine, and negotiate with the first virtual machine to obtain the encryption key when the first virtual machine is authenticated based on the authentication information of the first virtual machine.

6. The system according to claim 4 or 5, characterized in that: The processor of the second computing device is used to run the second virtual machine and obtain authentication information of the second virtual machine; The processor of the second computing device is used to call the second communication interface in the second untrusted execution environment and send the authentication information of the second virtual machine to the first communication interface; The processor of the first computing device is used to obtain authentication information of the second virtual machine in the first untrusted execution environment; The processor of the first computing device is used to run the first virtual machine, and when the second virtual machine is authenticated based on the authentication information of the second virtual machine, negotiate with the second virtual machine to obtain the encryption key.

7. The system according to claim 6, characterized in that The authentication information of the first virtual machine includes a current measurement value of a configuration of the first virtual machine and a historical measurement value of the configuration of the first virtual machine, and the authentication information of the second virtual machine includes a current measurement value of a configuration of the second virtual machine and a historical measurement value of the configuration of the second virtual machine; The processor of the second computing device is used to run the second virtual machine, and if it is determined that the current measurement value of the configuration of the first virtual machine is the same as the historical measurement value of the configuration of the first virtual machine, authenticating that the first virtual machine passes; The processor of the first computing device is used to run the first virtual machine, and when it is determined that the current measurement value of the configuration of the second virtual machine is the same as the historical measurement value of the configuration of the second virtual machine, the second virtual machine is authenticated.

8. The system according to any one of claims 4 to 7, characterized in that: The processor of the second computing device is used to run the second virtual machine and generate a message authentication code based on the encryption key and the first message; The processor of the second computing device is used to call the second communication interface in the second untrusted execution environment, and send the first message and the message authentication code to the first communication interface; The processor of the first computing device is used to obtain the first message and the message authentication code in the first untrusted execution environment; The processor of the first computing device is used to run the first virtual machine, and when it is determined that the encryption key is a valid key based on the first message and the message authentication code, use the encryption key to encrypt the state information of the first container.

9. The system according to any one of claims 4 to 8, characterized in that: The processor of the first computing device is used to run the first virtual machine to generate a first private key and a first public key; The processor of the first computing device is used to call the first communication interface in the first untrusted execution environment and send the first public key to the second communication interface; The processor of the second computing device is used to obtain the first public key in the second untrusted execution environment; The processor of the second computing device is used to run the second virtual machine to generate a second private key and a second public key, and generate the encryption key based on the second private key and the first public key; The processor of the second computing device is used to call the second communication interface in the second untrusted execution environment and send the second public key to the first communication interface; The processor of the first computing device is used to obtain the second public key in the first untrusted execution environment; The processor of the first computing device is used to run the first virtual machine and generate the encryption key based on the first private key and the second public key.

10. The system according to any one of claims 3 to 9, characterized in that: The communication status information between the first container and the virtual IO device includes an identifier of the first container; Before the processor of the first computing device is used to run the first virtual machine and encrypt the state information of the first container to obtain the encrypted information, the processor of the first computing device is further used to: Running the first virtual machine, and locating communication status information between the first container and the virtual IO device in a virtual IO device driver of the first virtual machine based on the identifier of the first container; After the processor of the second computing device is used to run the second virtual machine and decrypt the encrypted information to obtain the state information of the first container, the processor of the second computing device is specifically used to: Running the second virtual machine, loading the memory status information of the first container into the memory of the standby container, and loading the communication status information between the first container and the virtual IO device into the virtual IO device driver of the second virtual machine; The second virtual machine is run, and based on the memory state information of the first container in the memory of the standby container and the communication state information between the first container and the virtual IO device located in the virtual IO device driver of the second virtual machine based on the identifier of the first container, the standby container is run, so as to synchronize the memory state of the standby container with the first container and synchronize the communication state of the standby container with the first container.

11. A container migration method, characterized in that: Applied to a second computing device, the software and hardware resources of the second computing device are divided into a second untrusted execution environment and a second TEE, the second TEE includes a backup container, and the method includes: The second computing device obtains encrypted information of state information of a first container, where the first container is deployed in a first TEE, and the first TEE is a TEE of the first computing device; The second computing device decrypts the encrypted information to obtain the state information of the first container; The second computing device synchronizes the status of the backup container with the first container based on the status information of the first container.

12. The method according to claim 11, characterized in that The state of the first container includes a memory state of the first container, or the state of the first container includes a memory state of the first container and a communication state between the first container and a virtual IO device.

13. The method according to claim 11 or 12, characterized in that: The second TEE includes a second virtual machine, and the backup container is deployed on the second virtual machine; The second computing device obtains encrypted information of the state information of the first container, including: The processor of the second computing device obtains, in the second untrusted execution environment, encrypted information of the state information of the first container; The second computing device decrypts the encrypted information to obtain the state information of the first container, including: The processor of the second computing device runs the second virtual machine and decrypts the encrypted information to obtain the state information of the first container; The second computing device synchronizes the status of the standby container with the first container based on the status information of the first container, including: The processor of the second computing device runs the second virtual machine and synchronizes the status of the backup container with the first container based on the status information of the first container.

14. The method according to claim 13, characterized in that The encrypted information is obtained by encrypting the status information of the first container using an encryption key, the encryption key is obtained by negotiation between the first virtual machine and the second virtual machine, and the encryption key is also used to decrypt the encrypted information. The first virtual machine is deployed in the first TEE, and the first container is deployed in the first virtual machine.

15. The method according to claim 14, characterized in that The method further comprises: The processor of the second computing device obtains authentication information of the first virtual machine in the second untrusted execution environment; The processor of the second computing device runs the second virtual machine, and when the first virtual machine is authenticated successfully based on the authentication information of the first virtual machine, negotiates with the first virtual machine to obtain the encryption key.

16. The method according to claim 14 or 15, characterized in that The method further comprises: The processor of the second computing device runs the second virtual machine to obtain authentication information of the second virtual machine; The processor of the second computing device calls a second communication interface in the second non-trusted execution environment to send authentication information of the second virtual machine to a first communication interface, where the second communication interface is the communication interface of the second computing device and the first communication interface is the communication interface of the first computing device.

17. The method according to claim 15 or 16, characterized in that The authentication information of the first virtual machine includes a current measurement value of a configuration of the first virtual machine and a historical measurement value of a configuration of the first virtual machine; The method further comprises: The processor of the second computing device runs the second virtual machine, and when it is determined that the current measurement value of the configuration of the first virtual machine is the same as the historical measurement value of the configuration of the first virtual machine, the first virtual machine is authenticated.

18. The method according to any one of claims 14 to 17, characterized in that The processor of the second computing device runs the second virtual machine to generate a message authentication code based on the encryption key and the first message; The processor of the second computing device calls a second communication interface in the second untrusted execution environment, and sends the first message and the message authentication code to a first communication interface, where the second communication interface is the communication interface of the second computing device, and the first communication interface is the communication interface of the first computing device.

19. The method according to any one of claims 13 to 18, characterized in that The communication status information between the first container and the virtual IO device includes an identifier of the first container; The processor of the second computing device runs the second virtual machine, and synchronizes the state of the standby container with the first container based on the state information of the first container, including: The processor of the second computing device runs the second virtual machine, loads the memory state information of the first container into the memory of the standby container, and loads the communication state information between the first container and the virtual IO device into the virtual IO device driver of the second virtual machine; The processor of the second computing device runs the second virtual machine, and runs the standby container based on the memory state information of the first container in the memory of the standby container and the communication state information between the first container and the virtual IO device located in the virtual IO device driver of the second virtual machine based on the identifier of the first container, so as to achieve memory state synchronization between the standby container and the first container, and achieve communication state synchronization between the standby container and the first container.

20. A container migration method, characterized in that: Applied to a first computing device, the software and hardware resources of the first computing device are divided into a first untrusted execution environment and a first TEE, the first TEE includes a first container, and the method includes: The processor of the first computing device encrypts the state information of the first container in the first TEE to obtain encrypted information; The processor of the first computing device calls the communication interface of the first computing device in the first non-trusted execution environment, and sends the encrypted information to the second computing device, where the encrypted information is used to migrate the state information of the first container to a backup container, and the backup container is deployed in a second TEE, which is the TEE of the second computing device.

21. A container migration device, characterized in that: Applied to a second computing device, the software and hardware resources of the second computing device are divided into a second untrusted execution environment and a second TEE, the second TEE includes a backup container, and the apparatus includes: An acquisition module, configured to acquire encrypted information of state information of a first container, where the first container is deployed in a first TEE, and the first TEE is a TEE of the first computing device; a decryption module, used for decrypting the encrypted information to obtain the state information of the first container; A synchronization module is used to synchronize the status of the standby container with the first container based on the status information of the first container.

22. A container migration device, characterized in that: Applied to a first computing device, the software and hardware resources of the first computing device are divided into a first untrusted execution environment and a first TEE, the first TEE includes a first container, and the apparatus includes: an encryption module, configured to encrypt the state information of the first container in the first TEE to obtain encrypted information; A sending module is used to call the communication interface of the first computing device in the first non-trusted execution environment, and send the encrypted information to the second computing device, wherein the encrypted information is used to migrate the state information of the first container to a backup container, and the backup container is deployed in a second TEE, and the second TEE is the TEE of the second computing device.

23. A computing device, characterized in that The computing device comprises a processor and a memory; the processor of the computing device is used to execute instructions stored in the memory of the computing device, so that the computing device executes the method according to any one of claims 11 to 20.

24. A computer-readable storage medium, characterized in that: The method comprises computer program instructions, and when the computer program instructions are executed by a computing device, the computing device performs the method according to any one of claims 11 to 20.

Images