Container migration method and system and computer readable storage medium

Abstract

The invention discloses a container migration method and system and a computer readable storage medium, the container migration system comprises a first computing device and a second computing device, software and hardware resources of the first computing device and software and hardware resources of the second computing device are both divided into a non-trusted execution environment (TEE) and a trusted execution environment (TEE), the TEE of the first computing device comprises a first confidential container, the TEE of the second computing device comprises a standby confidential container, the first computing device is used for sending encrypted information of state information of the first confidential container to the second computing device, and the second computing device is used for decrypting the encrypted information to obtain the state information of the first confidential container. And state synchronization of the standby confidential container and the first confidential container is realized based on the state information of the first confidential container. According to the method, in the process of capacity expansion / system upgrade / load adjustment / fault repair of the computing device, the tenant business which is being processed by the confidential container running on the computing device is not interrupted.

Description

Technical Field

[0001] This application relates to the field of virtualization technology, and in particular, to a container migration method, system, and computer-readable storage medium. Background Art

[0002] A confidential container refers to a container created and running in a trusted execution environment (TEE), which is an important application scenario in container technology. Based on the TEE, the privacy and security of user data in the confidential container can be protected.

[0003] Currently, in the confidential container scenario, if the computing device (such as a server) used to implement the confidential container needs to be expanded / system upgraded / load adjusted / fault repaired, during the process of expanding / system upgrading / load adjusting / fault repairing the computing device, the computing device is usually restarted once or multiple times, which will cause the confidential containers on the computing device to stop running, resulting in the interruption of the tenant services being processed by the confidential containers and seriously affecting the tenant experience. Summary of the Invention

[0004] This application provides a container migration method, system, and computer-readable storage medium, which can achieve that during the process of expanding / system upgrading / load adjusting / fault repairing the computing device, the tenant services being processed by the confidential containers running on the computing device are not interrupted, improving the tenant experience.

[0005] In a first aspect, a container migration system is provided. The container migration system includes a first computing device and a second computing device. The software and hardware resources of the first computing device are divided into a first TEE and a first non-trusted execution environment, and the software and hardware resources of the second computing device are divided into a second TEE and a second non-trusted execution environment. The first TEE includes a first container, and the second TEE includes a spare container;

[0006] The first computing device is configured to send encrypted information of the status information of the first container to the second computing device;

[0007] The second computing device is configured to receive and decrypt the encrypted information to obtain the status information of the first container;

[0008] The second computing device is further configured to synchronize the status of the spare container with the status of the first container based on the status information of the first container.

[0009] The above status information of the first container includes the status information of the tenant services running on the first container.

[0010] In the above solution, since the first computing device migrates the status information of the first container to the second computing device, and the second computing device realizes the status synchronization between the standby container and the first container based on the migrated status information of the first container, during the process of capacity expansion / system upgrade / load adjustment / fault repair of the first computing device, the second computing device can run the standby container to continue processing tenant services. Even if restarting the first computing device causes the first container to be closed, the tenant services will not be interrupted. In addition, since the standby container is located in the TEE of the second computing device and the security of the TEE is relatively high, having the standby container continue to process tenant services can protect the security of tenant service data.

[0011] It can also be seen that since the first computing device encrypts the status information of the first container and then transmits it to the second computing device, the above solution can also protect the security of the status information of the first container during the transmission process.

[0012] In some possible implementation manners, the status of the first container includes the memory status of the first container, or the status of the first container includes the memory status of the first container and the communication status between the first container and a virtual input output (IO) device.

[0013] It can be understood that the status of the first container includes the memory status of the first container and the communication status between the first container and the virtual IO device. Compared with the situation where the status of the first container only includes the memory status of the first container, when subsequently realizing the status synchronization between the standby container and the first container, the synchronized status is more complete.

[0014] In some possible implementation manners, the first TEE includes a first virtual machine, the second TEE includes a second virtual machine, the first container is located in the first virtual machine, and the standby container is located in the second virtual machine;

[0015] The processor of the first computing device is used to run the first virtual machine and encrypt the status information of the first container to obtain encrypted information;

[0016] The processor of the first computing device is used to call the first communication interface in the first non-trusted execution environment and send the encrypted information to the second communication interface. The first communication interface is the communication interface of the first computing device, and the second communication interface is the communication interface of the second computing device;

[0017] The processor of the second computing device is used to obtain the encrypted information in the second non-trusted execution environment;

[0018] The processor of the second computing device is used to run the second virtual machine and decrypt the encrypted information to obtain the status information of the first container;

[0019] The processor of the second computing device is used to run the second virtual machine and implement the status synchronization between the standby container and the first container based on the status information of the first container.

[0020] Implementing the above implementation method, since the status information of the first container is encrypted by the first virtual machine in the first TEE with higher security and decrypted by the second virtual machine in the second TEE with higher security, and the encrypted information is transmitted in the first non-trusted execution environment with lower security and the second non-trusted execution environment with lower security, the status information of the first container can be prevented from being stolen in the first non-trusted environment and the second non-trusted environment with lower security, and the security of the status information of the first container can be well protected.

[0021] In some possible implementation methods, the encrypted information is obtained by encrypting the status information of the first container using an encryption key, the encryption key is obtained by negotiation between the first virtual machine and the second virtual machine, and the encryption key is also used to decrypt the encrypted information.

[0022] Implementing the above implementation method, since the encryption key is obtained by negotiation between the first virtual machine in the first TEE with higher security and the second virtual machine in the second TEE with higher security, the first computing device only needs to transmit the encrypted information to the second computing device and does not need to transmit the encryption key to the second computing device. Even if the encrypted information is stolen during the transmission from the first computing device to the second computing device, or in the first non-trusted execution environment or in the second non-trusted execution environment, the thief cannot decrypt the stolen encrypted information because the encryption key cannot be obtained. Therefore, the above implementation method can further improve the security of the status information of the first container during transmission.

[0023] In some possible implementation methods, the processor of the first computing device is used to run the first virtual machine to obtain the authentication information of the first virtual machine;

[0024] The processor of the first computing device is used to call the first communication interface in the first non-trusted execution environment and send the authentication information of the first virtual machine to the second communication interface;

[0025] The processor of the second computing device is used to obtain the authentication information of the first virtual machine in the second non-trusted execution environment;

[0026] The processor of the second computing device is used to run the second virtual machine and, when the first virtual machine is authenticated to pass based on the authentication information of the first virtual machine, negotiate with the first virtual machine to obtain the encryption key.

[0027] Implementing the above implementation, since the first virtual machine negotiates with the second virtual machine to obtain an encryption key only when the second virtual machine passes the authentication based on the authentication information of the second virtual machine, this can avoid migrating the state information of the first container to the second virtual machine when the second virtual machine is an untrusted virtual machine, thereby protecting the security of the state information of the first container.

[0028] In some possible implementation manners, the processor of the second computing device is used to run the second virtual machine and obtain the authentication information of the second virtual machine;

[0029] The processor of the second computing device is used to call the second communication interface in the second non-trusted execution environment and send the authentication information of the second virtual machine to the first communication interface;

[0030] The processor of the first computing device is used to obtain the authentication information of the second virtual machine in the first non-trusted execution environment;

[0031] The processor of the first computing device is used to run the first virtual machine and negotiate with the second virtual machine to obtain an encryption key when the second virtual machine passes the authentication based on the authentication information of the second virtual machine.

[0032] Implementing the above implementation, since the second virtual machine negotiates with the first virtual machine to obtain an encryption key only when the first virtual machine passes the authentication based on the authentication information of the first virtual machine, this can avoid migrating the state information of the first container to the second virtual machine when the first virtual machine is an untrusted virtual machine, causing security risks to the second computing device and the second virtual machine, thereby protecting the security of the second computing device and the second virtual machine.

[0033] In some possible implementation manners, the authentication information of the first virtual machine includes the current measurement value of the configuration of the first virtual machine and the historical measurement value of the configuration of the first virtual machine, and the authentication information of the second virtual machine includes the current measurement value of the configuration of the second virtual machine and the historical measurement value of the configuration of the second virtual machine;

[0034] The processor of the second computing device is used to run the second virtual machine and authenticate that the first virtual machine passes when it is determined that the current measurement value of the configuration of the first virtual machine is the same as the historical measurement value of the configuration of the first virtual machine;

[0035] The processor of the first computing device is used to run the first virtual machine and authenticate that the second virtual machine passes when it is determined that the current measurement value of the configuration of the second virtual machine is the same as the historical measurement value of the configuration of the second virtual machine.

[0036] In some possible implementation manners, the processor of the second computing device is used to run the second virtual machine and generate a message authentication code based on the encryption key and the first message;

[0037] The processor of the second computing device is used to call the second communication interface in the second non-trusted execution environment and send a first message and a message authentication code to the first communication interface;

[0038] The processor of the first computing device is used to obtain the first message and the message authentication code in the first non-trusted execution environment;

[0039] The processor of the first computing device is used to run a first virtual machine, and when it is determined that the encryption key is a valid key based on the first message and the message authentication code, use the encryption key to encrypt the status information of the first container.

[0040] Implementing the above implementation method, since the first virtual machine encrypts the status information of the first container using the encryption key only when it is determined that the encryption key is a valid key according to the message authentication code from the second virtual machine and the first message, this can avoid migrating the status information of the first container to the second virtual machine when the encryption key is an invalid key, so as to ensure the accurate progress of the migration process of the first container.

[0041] In some possible implementation methods, the processor of the first computing device is used to run the first virtual machine to generate a first private key and a first public key;

[0042] The processor of the first computing device is used to call the first communication interface in the first non-trusted execution environment and send the first public key to the second communication interface;

[0043] The processor of the second computing device is used to obtain the first public key in the second non-trusted execution environment;

[0044] The processor of the second computing device is used to run the second virtual machine to generate a second private key and a second public key, and generate an encryption key based on the second private key and the first public key;

[0045] The processor of the second computing device is used to call the second communication interface in the second non-trusted execution environment and send the second public key to the first communication interface;

[0046] The processor of the first computing device is used to obtain the second public key in the first non-trusted execution environment;

[0047] The processor of the first computing device is used to run the first virtual machine and generate an encryption key based on the first private key and the second public key.

[0048] It can be seen that in the above implementation, the first virtual machine and the second virtual machine generate an encryption key based on public information (i.e., the public key of the other party) and the private information they each hold (i.e., the private key). The private information is not transmitted between the two. Even if the public information is stolen during the transmission process, since the private information is not leaked, the attacker cannot obtain the encryption key. Therefore, the security of the state information of the first container encrypted with the encryption key during the transmission process can be protected. In addition, both the first virtual machine and the second virtual machine are located in the TEE. Due to the high security of the TEE, the security of the encryption key can be well protected.

[0049] In some possible implementation manners, the communication state information between the first container and the virtual I / O device includes the identifier of the first container;

[0050] Before the processor of the first computing device runs the first virtual machine and encrypts the state information of the first container to obtain encrypted information, the processor of the first computing device is further configured to:

[0051] Run the first virtual machine, and locate the communication state information between the first container and the virtual I / O device in the virtual I / O device driver of the first virtual machine based on the identifier of the first container;

[0052] After the processor of the second computing device runs the second virtual machine and decrypts the encrypted information to obtain the state information of the first container, the processor of the second computing device is specifically configured to:

[0053] Run the second virtual machine, load the memory state information of the first container into the memory of the standby container, and load the communication state information between the first container and the virtual I / O device into the virtual I / O device driver of the second virtual machine;

[0054] Run the second virtual machine, and based on the memory state information of the first container in the memory of the standby container and the communication state information between the first container and the virtual I / O device located in the virtual I / O device driver of the second virtual machine based on the identifier of the first container, run the standby container, so as to implement the synchronization of the memory state between the standby container and the first container, and implement the synchronization of the communication state between the standby container and the first container.

[0055] Since the communication status information between the first container and the virtual I / O device is located in the virtual I / O device driver of the first virtual machine, and in the case where the first virtual machine includes multiple confidential containers, this virtual I / O device driver is shared by multiple confidential containers. That is to say, this virtual I / O device driver may include the communication status information of multiple confidential containers with the virtual I / O device. In order for the first virtual machine to accurately locate the communication status information between the first container and the virtual I / O device from the virtual I / O device driver, it can be set that when each confidential container in the first virtual machine communicates with the virtual I / O device, the communication request sent by each confidential container to the virtual I / O device carries the identity document (ID) of this confidential container. And when the virtual I / O device driver receives the communication request of the confidential container, it adds the ID of the confidential container carried in the communication request and the communication request to the virtual queue in the virtual I / O device driver. Thus, when the first virtual machine migrates the first container subsequently, the first virtual machine can accurately locate the communication request of the first container to the virtual I / O device and the response information returned by the virtual I / O device based on the communication request from the virtual queue in the virtual I / O device driver according to the ID of the first container. The information located is the communication status information between the first container and the virtual I / O device.

[0056] In addition, since the communication status information between the first container and the virtual I / O device includes the ID of the first container, when the second virtual machine runs the standby container, it can accurately locate the communication status information of the first container to the virtual I / O device from the virtual I / O device driver of the second virtual machine based on the ID of the first container, so as to realize the communication status synchronization between the standby container and the first container based on this communication status information.

[0057] In a second aspect, a container migration method is provided, which is applied to a second computing device. The software and hardware resources of the second computing device are divided into a second non-trusted execution environment and a second TEE, and the second TEE includes a standby container. The method includes:

[0058] The second computing device obtains the encrypted information of the status information of the first container, and the first container is deployed in a first TEE, and the first TEE is the TEE of the first computing device;

[0059] The second computing device decrypts the encrypted information to obtain the status information of the first container;

[0060] The second computing device realizes the status synchronization between the standby container and the first container based on the status information of the first container.

[0061] In some possible implementations, the state of the first container includes the memory state of the first container, or the state of the first container includes the memory state of the first container and the communication state between the first container and the virtual I / O device.

[0062] In some possible implementations, the second TEE includes a second virtual machine, and the standby container is deployed on the second virtual machine;

[0063] The second computing device obtains the encrypted information of the status information of the first container, including:

[0064] The processor of the second computing device obtains the encrypted information of the status information of the first container in the second non-trusted execution environment;

[0065] The second computing device decrypts the encrypted information to obtain the status information of the first container, including:

[0066] The processor of the second computing device runs the second virtual machine and decrypts the encrypted information to obtain the status information of the first container;

[0067] The second computing device synchronizes the state of the standby container with the state of the first container based on the status information of the first container, including:

[0068] The processor of the second computing device runs the second virtual machine and synchronizes the state of the standby container with the state of the first container based on the status information of the first container.

[0069] In some possible implementations, the encrypted information is obtained by encrypting the status information of the first container using an encryption key. The encryption key is obtained through negotiation between the first virtual machine and the second virtual machine. The encryption key is also used to decrypt the encrypted information. The first virtual machine is deployed on the first TEE, and the first container is deployed on the first virtual machine.

[0070] In some possible implementations, the method further includes:

[0071] The processor of the second computing device obtains the authentication information of the first virtual machine in the second non-trusted execution environment;

[0072] The processor of the second computing device runs the second virtual machine and negotiates with the first virtual machine to obtain the encryption key when the first virtual machine is authenticated based on the authentication information of the first virtual machine.

[0073] In some possible implementations, the method further includes:

[0074] The processor of the second computing device runs the second virtual machine and obtains the authentication information of the second virtual machine;

[0075] The processor of the second computing device calls a second communication interface in the second non-trusted execution environment and sends the authentication information of the second virtual machine to a first communication interface. The second communication interface is a communication interface of the second computing device, and the first communication interface is a communication interface of the first computing device.

[0076] In some possible implementation manners, the authentication information of the first virtual machine includes the current measurement value of the configuration of the first virtual machine and the historical measurement value of the configuration of the first virtual machine;

[0077] The method further includes:

[0078] The processor of the second computing device runs the second virtual machine and authenticates that the first virtual machine passes when it is determined that the current measurement value of the configuration of the first virtual machine is the same as the historical measurement value of the configuration of the first virtual machine.

[0079] In some possible implementation manners, the processor of the second computing device runs the second virtual machine and generates a message authentication code based on the encryption key and a first message;

[0080] The processor of the second computing device calls a second communication interface in the second non-trusted execution environment and sends the first message and the message authentication code to a first communication interface. The second communication interface is a communication interface of the second computing device, and the first communication interface is a communication interface of the first computing device.

[0081] In some possible implementation manners, the communication status information between the first container and the virtual IO device includes the identifier of the first container;

[0082] The processor of the second computing device runs the second virtual machine and realizes the status synchronization between the standby container and the first container based on the status information of the first container, including:

[0083] The processor of the second computing device runs the second virtual machine, loads the memory status information of the first container into the memory of the standby container, and loads the communication status information between the first container and the virtual IO device into the virtual IO device driver of the second virtual machine.

[0084] The processor of the second computing device runs the second virtual machine, runs the standby container based on the memory state information of the first container in the memory of the standby container and the communication state information of the first container and the virtual I / O device located in the virtual I / O device driver of the second virtual machine based on the identifier of the first container, so as to synchronize the memory state between the standby container and the first container and synchronize the communication state between the standby container and the first container.

[0085] In a third aspect, a container migration method is provided, which is applied to a first computing device. The software and hardware resources of the first computing device are divided into a first non-trusted execution environment and a first TEE. The first TEE includes a first container. The method includes:

[0086] The processor of the first computing device encrypts the state information of the first container in the first TEE to obtain encrypted information;

[0087] The processor of the first computing device calls the communication interface of the first computing device in the first non-trusted execution environment and sends the encrypted information to a second computing device. The encrypted information is used to migrate the state information of the first container to a standby container, and the standby container is deployed in a second TEE, and the second TEE is the TEE of the second computing device.

[0088] In a fourth aspect, a container migration device is provided, which is applied to a second computing device. The software and hardware resources of the second computing device are divided into a second non-trusted execution environment and a second TEE. The second TEE includes a standby container. The device includes:

[0089] An acquisition module, configured to acquire encrypted information of the state information of a first container, where the first container is deployed in a first TEE, and the first TEE is the TEE of the first computing device;

[0090] A decryption module, configured to decrypt the encrypted information to obtain the state information of the first container;

[0091] A synchronization module, configured to synchronize the state between the standby container and the first container based on the state information of the first container.

[0092] In a fifth aspect, a container migration device is provided, which is applied to a first computing device. The software and hardware resources of the first computing device are divided into a first non-trusted execution environment and a first TEE. The first TEE includes a first container. The device includes:

[0093] An encryption module, configured to encrypt the state information of the first container in the first TEE to obtain encrypted information;

[0094] A sending module, configured to call a communication interface of the first computing device in the first non-trusted execution environment to send the encrypted information to a second computing device, where the encrypted information is used to migrate status information of the first container to a standby container, and the standby container is deployed in a second TEE, and the second TEE is a TEE of the second computing device.

[0095] For the container migration method provided in the second aspect / the third aspect, and the related beneficial effects and descriptions of any implementation manner of the second aspect / the third aspect, and the container migration device provided in the fourth aspect / the fifth aspect, and the related beneficial effects and descriptions of any implementation manner of the fourth aspect / the fifth aspect, reference may be made to the container migration system provided in the foregoing first aspect and the related beneficial effects and descriptions of any implementation manner of the first aspect, which will not be elaborated herein.

[0096] In a sixth aspect, a computing device is provided, including a processor and a memory; the processor is configured to execute instructions stored in the memory, so that the computing device implements the method provided in any one of the second aspect to the third aspect, and any implementation manner of any one of the aspects.

[0097] In a seventh aspect, a container migration system is provided, including the container migration device described in the fourth aspect and the container migration device described in the fifth aspect.

[0098] In an eighth aspect, a computer-readable storage medium is provided, storing instructions for implementing the method provided in any one of the second aspect to the third aspect, and any implementation manner of any one of the aspects.

[0099] In a ninth aspect, a computer program product is provided, including a computer program, which, when read and executed by a computing device, causes the computing device to execute the method provided in any one of the second aspect to the third aspect, and any implementation manner of any one of the aspects. Description of the Drawings

[0100] Figure 1 is a schematic diagram of the VirtIO protocol architecture provided by an embodiment of the present application;

[0101] Figure 2 is a schematic diagram of the VirtIO protocol communication model provided by an embodiment of the present application;

[0102] Figure 3 is a schematic structural diagram of a container migration system provided by an embodiment of the present application;

[0103] Figure 4 is an interaction schematic diagram of a container migration method provided by an embodiment of the present application;

[0104] Figure 5 It is a schematic diagram of a specific embodiment of a container migration method provided by an embodiment of the present application;

[0105] Figure 6 It is a schematic diagram of a process of negotiating to obtain an encryption key provided by an embodiment of the present application;

[0106] Figure 7 It is a schematic diagram of the structure of a container migration device provided by an embodiment of the present application;

[0107] Figure 8 It is a schematic diagram of the structure of another container migration device provided by an embodiment of the present application;

[0108] Figure 9 It is a schematic diagram of the structure of a computing device provided by an embodiment of the present application. Detailed implementation manners

[0109] The embodiments of the present invention will be described below with reference to the accompanying drawings in the embodiments of the present invention. The terms used in the embodiments of the present invention are only used to explain the specific embodiments of the present invention, rather than to limit the present invention.

[0110] The terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and do not have to be used to describe a specific order or sequence. It should be understood that such terms can be interchanged under appropriate circumstances, which is only a way of distinguishing when describing objects with the same attributes in the embodiments of the present application. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, so that a process, method, system, product or device including a series of units does not have to be limited to those units, but may include other units not clearly listed or inherent to these processes, methods, products or devices.

[0111] For the convenience of clearly understanding the technical solutions provided by the present application, some nouns and terms related to the technical solutions provided by the present application will be explained first.

[0112] (1) A trusted execution environment (TEE) is usually composed of hardware and software in a computing device. It can run on top of a general operating system, providing a secure runtime environment that prevents malware from accessing or tampering with the data and code in the TEE, ensuring the security and privacy of sensitive data and code. The TEE can be created in a computing device through technologies such as the TrustZone technology based on the ARM architecture, AMD Secure Encrypted Virtualization (SEV), and Intel Software Guard Extensions (SGX).

[0113] (2) An untrusted execution environment is composed of the remaining software and hardware resources in a computing device other than those included in the TEE.

[0114] The untrusted execution environment does not mean that the operating system (OS) or software running in it is malicious, but its security is lower than that of the TEE. Because when the processor operates in the untrusted execution environment, the resources in the TEE (such as registers, memory, cache, peripherals, etc.) are prohibited from being accessed. Once the processor attempts to access these resources, the system will crash directly. For example, TrustZone can configure the TrustZone Address Space Controller (TZASC) register and the TrustZone Memory Adapter (TZMA) register to set sensitive memory as secure memory. When the processor operates in the untrusted execution environment, it cannot access this secure memory. When the processor operates in the TEE, it can access both the resources of the TEE and the resources of the untrusted execution environment.

[0115] (3) A confidential virtual machine (CVM) is the product of the combination of confidential computing technology and cloud-native technology, referring to the virtual machine located in the TEE. Based on the TEE, the confidential virtual machine can achieve resource isolation, data encryption, and remote attestation, preventing cloud service providers and any third party with high privileges from stealing and tampering with the data in the confidential virtual machine, thus effectively protecting the security of tenant data and assets.

[0116] (4) A general virtual machine refers to the virtual machine located in the untrusted execution environment.

[0117] (5) A confidential container is a product of the combination of confidential computing technology and cloud-native technology, referring to a container located in the TEE. Based on the TEE, the confidential container can achieve resource isolation, data encryption, and remote attestation, preventing cloud service providers and any third party with high privileges from stealing and tampering with the data in the confidential container, thus effectively protecting the security of tenant data and assets.

[0118] (6) A common container refers to a container located in an untrusted execution environment.

[0119] (7) A virtual input / output (IO) device, also known as a virtual device, is an IO device simulated by software, used to simulate the functions of a physical IO device in a computer system. It can provide interfaces and functions similar to those of a physical IO device, but is actually implemented through software.

[0120] The main purpose of a virtual IO device is to provide virtualization and management of physical IO resources in a virtualized environment. In virtualization technology, when multiple virtual machines or containers share physical IO devices, the virtual IO device can help achieve resource isolation and scheduling, and provide better performance and reliability.

[0121] The following are some common virtual IO devices:

[0122] Virtual disk: A virtual disk is a disk device simulated by software, used to provide storage space for virtual machines or containers in a virtualized environment. The virtual disk can divide the capacity of a physical disk into multiple logical volumes and provide independent disk space for each virtual machine or container.

[0123] Virtual network interface card (VNIC): A virtual network interface card is a network interface device simulated by software, used to provide network connections for virtual machines or containers in a virtualized environment. The virtual network interface card can simulate the functions of a physical network interface card to enable communication between virtual machines or containers and the physical network.

[0124] Virtual serial port: A virtual serial port is a serial port device simulated by software, used to simulate the functions of a physical serial port in a virtualized environment. The virtual serial port can be used for serial communication between virtual machines or containers and external devices, such as connecting to debugging tools or external sensors.

[0125] Virtual Universal Serial Bus Device (virtual universal serial bus device), abbreviated as virtual USB device, is a USB device simulated by software, used to simulate the functions of physical USB devices in a virtualized environment. Virtual USB devices can provide virtual machines or containers with access to and management of USB devices, such as connecting USB storage devices or external input devices, etc.

[0126] The emergence of virtual I / O devices enables virtual machines or containers in a virtualized environment to better utilize and manage I / O resources, providing more flexible and efficient I / O functions.

[0127] (8) Virtual Input / Output (VirtIO) can be understood as a set of programs for virtualizing general input / output (I / O) devices. Specifically, as Figure 1 shown, the VirtIO protocol architecture includes a front-end driver and a virtio device. Among them, the front-end driver can also be called the virtual I / O device driver, and the virtio device can also be called the back-end device. The front-end driver includes drivers such as the virtio-blk (virtual disk) driver and the virtio-net (virtual network card) driver, which are located in the virtual machine, and the virtio device is located in the hypervisor. The main function of the front-end driver is to discover the virtio device, accept requests from within the virtual machine, and communicate with the virtio device in the hypervisor according to the VirtIO protocol.

[0128] As Figure 2 shown, the VirtIO protocol uses a front-back communication model. The front-end driver runs in the virtual machine and exchanges data with the virtio device in the hypervisor through a virtqueue. The virtqueue is implemented through a vring, which is a circular buffer shared between the virtual machine and the virtio device.

[0129] The vring realizes efficient data transmission through three main components: a descriptor table, an available ring, and a used ring. The following are their respective functions:

[0130] descriptor table:

[0131] Structure: It is an array, and each element is a descriptor used to describe the location and attributes of a memory area. Each descriptor contains the physical address, length, and other control information of a block of memory.

[0132] Usage: The front - end driver and the virtual I / O device share references to buffers through a descriptor table. The front - end driver creates descriptors and adds them to the descriptor table, while the virtual I / O device can read these descriptors to determine the location and size of the data to be read or written.

[0133] available ring:

[0134] Structure: It is a circular buffer that contains a set of indices corresponding to the descriptors in the Descriptor Table.

[0135] Usage: It is used to notify the virtual I / O device that there are new tasks to be processed. Specifically, the front - end driver adds new indices to the available ring, indicating new tasks for the virtual I / O device to process. The virtual I / O device polls this ring to check if there are new tasks to execute. The update of the available ring is a way for the front - end driver to inform the virtual I / O device of new tasks to be processed.

[0136] used ring:

[0137] Structure: It is also a circular buffer that contains a set of indices corresponding to the descriptors in the descriptor table that have been used by the virtual I / O device.

[0138] Usage: It is used to notify the front - end driver that the virtual I / O device has completed some tasks. Specifically, after the virtual I / O device completes a task, it adds the index of the corresponding descriptor to the used ring. The front - end driver can poll the used ring regularly to check which tasks have been completed.

[0139] When a container is deployed in a virtual machine and the container needs to perform I / O operations, the virtual machine can allocate a corresponding virtual I / O device for the container. Then the container can interact with the corresponding virtual I / O device through the virtual queue in the front - end driver. When multiple containers are deployed in the same virtual machine and all of them need to perform I / O operations, multiple containers all interact with their respective allocated virtual I / O devices through the virtual queue in the front - end driver. That is to say, multiple containers share the same front - end driver.

[0140] Next, the application scenarios related to the embodiments of this application will be introduced.

[0141] Embodiments of this application relate to the scenario of confidential containers, especially scenarios such as expanding the capacity / system upgrading / load adjustment / fault repair of a computing device (such as a server) running confidential containers. In these scenarios, during the process of expanding the capacity / system upgrading / load adjustment / fault repair of the computing device, the computing device is usually restarted once or multiple times, which will cause the confidential containers on the computing device to stop running, resulting in the interruption of the tenant services being processed by the confidential containers and seriously affecting the tenant experience.

[0142] To address the above problems, this application provides a container migration method and system. Before expanding the capacity / system upgrading / load adjustment / fault repair of a computing device running confidential containers, the confidential containers on the computing device are migrated to another computing device, so as to enable the migrated confidential containers to continue processing tenant services on another computing device during the process of expanding the capacity / system upgrading / load adjustment / fault repair of the computing device, ensuring that tenant services are not interrupted and optimizing the tenant experience.

[0143] The container migration method and system provided by this application are introduced below in conjunction with the corresponding drawings. Before introducing the container migration method and system provided by this application, concepts such as the state of the container and container migration involved in the embodiments of this application are introduced first.

[0144] (1) The state of the container. If the container is not in a communication state with the virtual IO device, the state of the container only includes the memory state of the container. If the container is in a communication state with the virtual IO device, the state of the container includes the memory state of the container and the communication state between the container and the virtual IO device.

[0145] The memory state of the container refers to the memory data of the container, which can reflect the running state of the container, that is, the running state of the container process itself, including the business state of the container running, etc.

[0146] The communication state between the container and the virtual IO device can reflect what communication state the container and the virtual IO device are in, and can include the read / write requests sent by the container to the virtual IO device, and the response information sent by the virtual IO device to the container after processing the read / write requests from the container. For example, assuming that the virtual machine to which container A belongs implements IO device virtualization based on the above VirtIO protocol, the communication state information between container A and the virtual IO device includes all the information related to container A in the descriptor table, available ring, and used ring in the virtual queue in the above front-end driver.

[0147] (2) Container migration can be understood as the migration of the container state, that is, by migrating the container state from the computing device to which the container belongs to another computing device and restoring the container state on the other computing device.

[0148] First, please refer to Figure 3 , Figure 3 which is a schematic structural diagram of a container migration system provided by an embodiment of the present application. As Figure 3 shown, the system includes a first computing device 100 and a second computing device 200. The first computing device 100 and the second computing device 200 can transmit data through a communication network of any communication mechanism / communication standard. Among them, the communication network can be a wide area network, a local area network, a point-to-point connection, etc., or any combination thereof.

[0149] In Figure 3 , the first computing device 100 can be regarded as a confidential container migration-out device (which can also be called a source host), and the second computing device 200 can be regarded as a confidential container migration-in device (which can also be called a target host).

[0150] The first computing device 100 and the second computing device 200 can be a personal computer (PC), a tablet computer, a physical server, etc. The physical server can be an X86 server or an ARM server, etc. The first computing device 100 and the second computing device 200 can belong to the same data center or different data centers.

[0151] As Figure 3 shown, the software and hardware resources of the first computing device 100 and the second computing device 200 are divided into a non-trusted execution environment and a TEE. For the sake of distinction, in Figure 3 and the following embodiments, the non-trusted execution environment and the TEE in the first computing device 100 are correspondingly called the first non-trusted execution environment and the first TEE, and the non-trusted execution environment and the TEE in the second computing device 200 are correspondingly called the second non-trusted execution environment and the second TEE.

[0152] The first TEE may include one or more confidential virtual machines, and each confidential virtual machine may include one or more confidential containers. In Figure 3 , taking the first TEE including one confidential virtual machine and the confidential virtual machine including 2 confidential containers as an example, optionally, the first non-trusted execution environment may include one or more ordinary virtual machines, and each ordinary virtual machine may include one or more ordinary containers, Figure 3 which are not shown.

[0153] The second TEE may include one or more confidential virtual machines, and each confidential virtual machine may include one or more confidential containers. In Figure 3Among them, taking the second TEE including a confidential virtual machine, and the confidential virtual machine including 2 confidential containers as an example, optionally, the second untrusted execution environment may include one or more ordinary virtual machines, and each ordinary virtual machine may include one or more ordinary containers. Figure 3 Not shown.

[0154] The functions of the above-mentioned confidential virtual machine are the same as those of the ordinary virtual machine, and the work that can be completed in the ordinary virtual machine can be achieved in the confidential virtual machine. The tenant can remotely log in to the confidential virtual machine and operate the installation, setting, and uninstallation of applications in the operating system environment of the confidential virtual machine. The functions of the above-mentioned confidential containers are the same as those of the ordinary containers, and the work that can be completed in the ordinary containers can be achieved in the confidential containers. The tenant can remotely operate the confidential containers, such as starting the confidential containers, pausing the running of the confidential containers, etc.

[0155] As Figure 3 As shown, the first TEE includes a first virtual machine manager for managing the confidential virtual machines in the first TEE, the second TEE includes a second virtual machine manager for managing the confidential virtual machines in the second TEE, the first untrusted execution environment may also include a third virtual machine manager for managing the ordinary virtual machines in the first untrusted execution environment, and the second untrusted execution environment may also include a fourth virtual machine manager for managing the ordinary virtual machines in the second untrusted execution environment.

[0156] The first / second virtual machine manager can implement logical isolation between different confidential virtual machines in the first / second TEE and manage the confidential virtual machines. For example, creating confidential virtual machines, simulating virtual hardware for confidential virtual machines according to the hardware layer (hardware simulation function), deleting confidential virtual machines, forwarding and / or processing network packets between all confidential virtual machines running on the first computing device 100 / second computing device 200 or forwarding network packets between the confidential virtual machines on the first computing device 100 / second computing device 200 and the external network (virtual switching function), and processing input / output (I / O) generated by the confidential virtual machines, etc.

[0157] The management operations of the above-mentioned first / second virtual machine manager on the confidential virtual machines are executed in cooperation with the third / fourth virtual machine manager. In other words, in order to protect the security of the confidential virtual machines, when the tenant needs to manage the confidential virtual machines in the first / second TEE, it is the third / fourth virtual machine manager that obtains the tenant's management instructions and then sends the management instructions to the first / second virtual machine manager, and the first / second virtual machine manager performs management operations according to the management instructions.

[0158] When the confidential virtual machine in the first / second TEE needs to communicate with the outside world, to protect the security of the confidential virtual machine, the first / second virtual machine manager and the third / fourth virtual machine manager are used as bridges to communicate with the external network. For example, the confidential virtual machine sends a message to the first / second virtual machine manager, which forwards it to the third / fourth virtual machine manager, and then the third / fourth virtual machine manager forwards it to the external network. When the external network sends a message to the confidential virtual machine, the third / fourth virtual machine manager forwards it to the first / second virtual machine manager, and then the first / second virtual machine manager forwards it to the confidential virtual machine.

[0159] In Figure 3 In the container migration system shown, to ensure that the first confidential container (referring to the confidential container to be migrated in the first computing device 100) remains secure after being migrated from the first computing device 100 to the second computing device 200, the first confidential container can be migrated to the TEE (i.e., the second TEE) of the second computing device 200. Specifically, a standby confidential container (referring to a container dedicated to implementing the migration of other confidential containers (such as the above-mentioned first confidential container) and having an empty memory) can be created in the second computing device 200. The first computing device 100 sends the status information of the first confidential container to the second computing device 200, as Figure 3 shown by the arrow in, after receiving the status information of the first confidential container, the second computing device 200 restores the status of the first confidential container on the standby confidential container based on the status information of the first confidential container. In other words, the status synchronization between the standby confidential container and the first confidential container is achieved based on the status information of the first confidential container, thereby realizing the migration of the first confidential container.

[0160] To ensure the security of the status information of the first confidential container during the transmission from the first computing device 100 to the second computing device 200, before sending the status information of the first confidential container to the second computing device 200, the first computing device 100 can encrypt the status information of the first confidential container and then send it to the second computing device 200. After receiving the encrypted information of the status information of the first confidential container, the second computing device 200 can decrypt the encrypted information to obtain the status information of the first confidential container.

[0161] In a possible embodiment, to further enhance the security of the status information of the first confidential container during the transmission from the first computing device 100 to the second computing device 200, the keys used by the first computing device 100 and the second computing device 200 for encryption and decryption can be obtained through negotiation between the two. That is to say, the first computing device 100 and the second computing device 200 do not perform key transmission, and the situation where the key is stolen during the transmission process will not occur.

[0162] In another possible embodiment, in order to further enhance the security of the status information of the first confidential container during the transmission from the first computing device 100 to the second computing device 200, the keys used by the first computing device 100 and the second computing device 200 for encryption and decryption can be obtained through negotiation between the confidential virtual machine to which the first confidential container in the first computing device 100 belongs (hereinafter referred to as the first confidential virtual machine) and the confidential virtual machine to which the backup confidential container in the second computing device 200 belongs (hereinafter referred to as the second confidential virtual machine), and are stored in the respective memories of the first confidential virtual machine and the second confidential virtual machine. Moreover, the encryption operation on the status information of the first confidential container is executed by the first confidential virtual machine, and the decryption operation on the encrypted information of the status information of the first confidential container is executed by the second confidential virtual machine. That is to say, the key is not only not transmitted between the first computing device 100 and the second computing device 200, but also not transmitted in the non-trusted execution environment (i.e., the first non-trusted execution environment) in the first computing device 100 and the non-trusted execution environment (i.e., the second non-trusted execution environment) in the second computing device 200. Furthermore, since the memory of the first confidential virtual machine belongs to the first TEE, the first non-trusted execution environment has no right to access the key stored in the memory of the first confidential machine, and since the memory of the second confidential virtual machine belongs to the second TEE, the second non-trusted execution environment also has no right to access the key stored in the memory of the second confidential virtual machine. Therefore, it is possible to prevent the key from being obtained by malware in the first non-trusted execution environment / second non-trusted execution environment, and to prevent the key from being stolen by an attacker in the case where the first non-trusted environment / second non-trusted environment is compromised, thereby better protecting the security of the status information of the first confidential container.

[0163] It should be understood that Figure 3 The container migration system shown is only an example provided by the embodiments of the present application, and the present application is not specifically limited. For example, in specific implementation, Figure 3 The container migration system shown may further include more computing devices such as a third computing device and a fourth computing device, and may also include network devices for forwarding communication data between computing devices.

[0164] For the convenience of more clearly understanding Figure 3 the process of migrating the confidential container by the container migration system shown, the following will be introduced in detail in combination with Figure 4 the interaction diagram of a container migration method provided by the embodiments of the present application shown.

[0165] S401: The first computing device 100 encrypts the status information of the first confidential container to obtain encrypted information.

[0166] S402: The first computing device 100 sends encrypted information to the second computing device 200. Correspondingly, the second computing device 200 receives the encrypted information sent by the first computing device 100.

[0167] S403: The second computing device 200 decrypts the encrypted information to obtain the status information of the first confidential container.

[0168] S404: The second computing device 200 synchronizes the status of the standby confidential container with that of the first confidential container based on the status information of the first confidential container.

[0169] For the status of the first confidential container, please refer to the above introduction to the container status. For the sake of brevity of the specification, it will not be elaborated here.

[0170] Specifically, before performing the encryption operation, the first computing device 100 can first check whether the first confidential container is in a communication state with the virtual IO device. If it is determined that the first confidential container is in a communication state with the virtual IO device, the status information of the encrypted first confidential container includes the memory status information of the first confidential container and the communication status information between the first confidential container and the virtual IO device. Otherwise, the status information of the encrypted first confidential container only includes the memory status information of the first confidential container.

[0171] Optionally, if the first computing device 100 determines that the first confidential container is in a communication state with the virtual IO device, the status information of the encrypted first confidential container may also only include the memory status information of the first confidential container. This application does not make specific limitations on this.

[0172] In the case where the status information of the first confidential container only includes the memory status information of the first confidential container, in S404, the status synchronization of the standby confidential container and the first confidential container implemented by the second computing device 200 is memory status synchronization. In the case where the status information of the first confidential container includes the memory status information of the first confidential container and the communication status information between the first confidential container and the virtual IO device, in S404, the status synchronization of the standby confidential container and the first confidential container implemented by the second computing device 200 is memory status synchronization and communication status synchronization with the virtual IO device.

[0173] It can be understood that when the first computing device 100 determines that the first confidential container is in a communication state with the virtual IO device, the status information of the encrypted first confidential container by the first computing device 100 includes the memory status information of the first confidential container and the communication status information between the first confidential container and the virtual IO device. Compared with the case where the status information of the first confidential container only includes the memory status information of the first confidential container, the synchronized status is more complete when implementing the status synchronization of the standby confidential container and the first confidential container subsequently.

[0174] The algorithms used for encryption and decryption by the first computing device 100 and the second computing device 200 described above may be symmetric encryption algorithms, such as the Advanced Encryption Standard (AES), the Data Encryption Standard (DES), the Triple Data Encryption Standard (3DES), etc., or may be asymmetric encryption algorithms, such as RSA, Elliptic Curve Cryptography (ECC), the Digital Signature Algorithm (DSA). The present application does not make specific limitations.

[0175] It can be understood that since the status information of the first confidential container described above, including the tenant service status being processed by the first confidential container, after the second computing device 200 realizes the status synchronization between the standby confidential container and the first confidential container, the second computing device 200 can then run the standby confidential container to continue processing tenant services, and the tenant services are not affected by the expansion / system upgrade / load adjustment / fault repair of the first computing device 100.

[0176] The following introduces in combination with a more specific embodiment Figure 4 the container migration method provided by the present application as shown Figure 5 shown, and this embodiment includes the following steps:

[0177] S501: The processor of the first computing device 100 pauses running the first confidential container.

[0178] In a possible embodiment, before S501, the processor of the first computing device 100 may also obtain the configuration information of the second computing device 200 (such as the configuration of the central processing unit (CPU), memory, disk, network, etc.), check whether the configuration of the second computing device is the same as that of the first computing device 100, and if the configurations are the same, execute S501. Otherwise, S501 may be continued to be executed, or S501 may not be executed. If the configurations are the same, executing S501 can ensure the smooth progress of the migration of the first confidential container and enable it to run normally in the new environment. This helps to reduce problems and risks during the migration process and improve the success rate of container migration.

[0179] S502: The processors of the first computing device 100 and the second computing device 200 obtain an encryption key through a negotiation method.

[0180] For the specific implementation process of S502, please refer to Figure 6 the relevant description.

[0181] S503: The processor of the first computing device 100 runs the first confidential virtual machine, encrypts the status information of the first confidential container using an encryption key to obtain encrypted information.

[0182] Taking the status information of the first confidential container including the memory status information of the first confidential container and the communication status information between the first confidential container and the virtual IO device as an example, S503 can specifically be: The processor of the first computing device 100 runs the first confidential virtual machine, obtains the memory status information of the first confidential container and the communication status information between the first confidential container and the virtual IO device, and then encrypts the obtained information using the encryption key to obtain encrypted information. In a specific implementation, the memory status information of the first confidential container and the communication status information between the first confidential container and the virtual IO device can be encrypted and transmitted together or separately encrypted and transmitted. This application does not make specific limitations on this.

[0183] The following describes the process in which the processor of the first computing device 100 runs the first confidential virtual machine and obtains the memory status information of the first confidential container and the communication status information between the first confidential container and the virtual IO device.

[0184] (1) Obtain the memory status information of the first confidential container.

[0185] The processor of the first computing device 100 runs the first confidential virtual machine and can collect all the context information of the first confidential container process through the process tracing (ptrace) mechanism, and then stores these context information classified by function as individual image files, and these image files are the memory status information of the first confidential container.

[0186] (2) Obtain the communication status information between the first confidential container and the virtual IO device.

[0187] Since the communication status information between the first confidential container and the virtual I / O device is located in the virtual I / O device driver of the first confidential virtual machine, and in the case where the first confidential virtual machine includes multiple confidential containers, this virtual I / O device driver is shared by multiple confidential containers. That is to say, this virtual I / O device driver may include the communication status information of multiple confidential containers with the virtual I / O device. In order for the first confidential virtual machine to accurately locate the communication status information between the first confidential container and the virtual I / O device from the virtual I / O device driver, it can be set that when each confidential container in the first confidential virtual machine communicates with the virtual I / O device, the communication request sent by each confidential container to the virtual I / O device carries the identity document (ID) of this confidential container. And when the virtual I / O device driver receives the communication request of the confidential container, it adds the ID of the confidential container carried in the communication request and the communication request to the virtual queue in the virtual I / O device driver. Thus, when the first confidential virtual machine performs the migration of the first confidential container subsequently, the first confidential virtual machine can accurately locate the communication request of the first confidential container to the virtual I / O device and the response information returned by the virtual I / O device based on the communication request from the virtual queue in the virtual I / O device driver according to the ID of the first confidential container. The information located is the communication status information between the first confidential container and the virtual I / O device.

[0188] S504: The processor of the first computing device 100 calls the first communication interface in the first non-trusted execution environment to send the encrypted information to the second communication interface. The first communication interface is the communication interface of the first computing device 100, and the second communication interface is the communication interface of the second computing device 200.

[0189] The communication interface can be a transceiver, or an input / output interface. Optionally, the transceiver may include a transmitter and a receiver. Optionally, the transceiver can be a transceiver circuit. Optionally, the input / output interface can be an input / output circuit.

[0190] S505: After the second communication interface receives the encrypted information, the processor of the second computing device 200 obtains the encrypted information in the second non-trusted execution environment.

[0191] Specifically, after the second communication interface receives the encrypted information, it can store the encrypted information in the memory of the second non-trusted execution environment. Subsequently, the processor of the second computing device 200 can obtain the encrypted information from the memory of the second non-trusted execution environment.

[0192] S506: The processor of the second computing device 200 runs the second confidential virtual machine and decrypts the encrypted information using the encryption key to obtain the status information of the first confidential container.

[0193] S507: The processor of the second computing device 200 runs a second confidential virtual machine to synchronize the state of the standby confidential container with that of the first confidential container based on the state information of the first confidential container.

[0194] As can be seen from the above embodiments, the state information of the first confidential container may only include the memory state information of the first confidential container, or may include both the memory state information of the first confidential container and the communication state information between the first confidential container and the virtual IO device. In the following, the process in which the processor of the second computing device 200 runs the second confidential virtual machine to synchronize the state of the standby confidential container with that of the first confidential container based on the state information of the first confidential container will be introduced in detail for these two cases respectively.

[0195] (1) The state information of the first confidential container only includes the memory state information of the first confidential container.

[0196] The processor of the second computing device 200 runs the second confidential virtual machine, and loads the memory state information of the first confidential container into the memory of the standby confidential container, so as to synchronize the memory state of the standby confidential container with that of the first confidential container.

[0197] (2) The state information of the first confidential container includes the memory state information of the first confidential container and the communication state information between the first confidential container and the virtual IO device.

[0198] The processor of the second computing device 200 runs the second confidential virtual machine, loads the memory state information of the first confidential container into the memory of the standby confidential container, and loads the communication state information between the first confidential container and the virtual IO device into the virtual IO device driver of the second confidential virtual machine. Then, based on the memory state information of the first confidential container in the standby confidential container and the communication state information between the first confidential container and the virtual IO device in the virtual IO device driver of the second confidential virtual machine, the standby confidential container is run, so as to synchronize the memory state of the standby confidential container with that of the first confidential container, and to synchronize the communication state of the standby confidential container with that of the first confidential container.

[0199] As can be seen from the above embodiments, the communication state information between the first confidential container and the virtual IO device was originally located in the descriptor table, available ring, and used ring in the virtual IO device driver of the first confidential virtual machine. In this application, when the processor of the second computing device 200 runs the second confidential virtual machine, the communication state information between the first confidential container and the virtual IO device can be correspondingly added to the descriptor table, available ring, and used ring in the virtual IO device driver of the second confidential virtual machine.

[0200] As can also be seen from the above embodiments, the communication status information between the first confidential container and the virtual IO device includes the identifier of the first confidential container. Therefore, after the processor of the second computing device 200 runs the second confidential virtual machine and loads the communication status information between the first confidential container and the virtual IO device into the virtual IO device driver of the second confidential virtual machine, when running the standby confidential container subsequently, the communication status information between the first confidential container and the virtual IO device can be accurately located from the virtual IO device driver of the second confidential virtual machine based on the identifier of the first confidential container, ensuring the accurate progress of the migration process.

[0201] S508: The processor of the second computing device 200 runs the standby confidential container.

[0202] Optionally, the processor of the first computing device 100 may also encrypt the status information of the first confidential container using an encryption key in the first non-trusted execution environment.

[0203] Optionally, the processor of the second computing device 200 may also decrypt the encrypted information using an encryption key in the second non-trusted execution environment.

[0204] It can be understood that the processor of the first computing device 100 runs the first confidential virtual machine for encryption operations, and the processor of the second computing device 200 runs the second confidential virtual machine for decryption operations. Compared with the processor of the first computing device 100 performing encryption operations in the first non-trusted execution environment and the processor of the second computing device 200 performing decryption operations in the second non-trusted execution environment, since the first confidential virtual machine is in the first TEE and the second confidential virtual machine is in the second TEE, the security of the first TEE is higher than that of the first non-trusted execution environment, and the security of the second TEE is higher than that of the second non-trusted execution environment. Therefore, the former implementation method can better protect the security of the status information of the first confidential container and reduce the risk of leakage of the status information of the first confidential container.

[0205] Next, the specific implementation process of step S502 will be introduced in combination with Figure 6 the following process schematic diagram.

[0206] S610: The processor of the first computing device 100 obtains the first public key and the first private key.

[0207] The first public key and the first private key may be generated by the processor of the first computing device 100, or may be sent to the first computing device 100 by other key management devices after being generated. When the first public key and the first private key are generated by the processor of the first computing device 100, they may be generated by the processor of the first computing device 100 in the first non-trusted execution environment, or may be generated in the first TEE, such as running the first confidential virtual machine in the first TEE. The present application does not make specific limitations on this.

[0208] After the processor of the first computing device 100 obtains the first public key and the first private key, the first public key and the first private key can be stored in the memory of the first non-trusted execution environment, or can be stored in the memory of the first TEE. It can be understood that since the security of the first TEE is higher than that of the first non-trusted execution environment, compared with storing the first private key in the memory of the first non-trusted execution environment, the security of the former storage method is higher when the first private key is stored in the memory of the first TEE.

[0209] S620: The processor of the second computing device 200 obtains the second public key and the second private key.

[0210] The second public key and the second private key can be generated by the processor of the second computing device 200, or can be generated by other key management devices and sent to the second computing device 200. When the second public key and the second private key are generated by the processor of the second computing device 200, they can be generated by the processor of the second computing device 200 in the second non-trusted execution environment, or can be generated in the second TEE, such as running the second confidential virtual machine in the second TEE. The present application does not make specific limitations on this.

[0211] After the processor of the second computing device 200 obtains the second public key and the second private key, the second public key and the second private key can be stored in the memory of the second non-trusted execution environment, or can be stored in the memory of the second TEE. It can be understood that since the security of the second TEE is higher than that of the second non-trusted execution environment, compared with storing the second private key in the memory of the second non-trusted execution environment, the security of the former storage method is higher when the second private key is stored in the memory of the second TEE.

[0212] S630: The processor of the first computing device 100 calls the first communication interface in the first non-trusted execution environment and sends the first public key to the second communication interface.

[0213] S640: The processor of the second computing device 200 calls the second communication interface in the second non-trusted execution environment and sends the second public key to the first communication interface.

[0214] S650: After the first communication interface receives the second public key, the processor of the first computing device 100 obtains the second public key in the first non-trusted execution environment.

[0215] Specifically, after the first communication interface receives the second public key, the second public key can be stored in the memory of the first non-trusted execution environment. Subsequently, the processor of the first computing device 100 can obtain the second public key from the memory of the first non-trusted execution environment.

[0216] S660: After receiving the first public key at the second communication interface, the processor of the second computing device 200 obtains the first public key in the second non-trusted execution environment.

[0217] Specifically, after receiving the first public key, the second communication interface may store the first public key in the memory of the second non-trusted execution environment. Subsequently, the processor of the second computing device 200 may obtain the first public key from the memory of the second non-trusted execution environment.

[0218] S670: The processor of the first computing device 100 generates an encryption key based on the first private key and the second public key.

[0219] In a possible embodiment, the processor of the first computing device 100 generates an encryption key based on the first private key and the second public key in the first TEE. For example, the first computing device 100 runs a first confidential virtual machine to generate an encryption key based on the first private key and the second public key.

[0220] In another possible embodiment, the processor of the first computing device 100 generates an encryption key based on the first private key and the second public key in the first non-trusted execution environment.

[0221] After generating the encryption key, the processor of the first computing device 100 may store the encryption key in the memory of the first non-trusted execution environment or the memory of the first TEE.

[0222] S680: The processor of the second computing device 200 generates an encryption key based on the second private key and the first public key.

[0223] In a possible embodiment, the processor of the second computing device 200 generates an encryption key based on the second private key and the first public key in the second TEE. For example, the second computing device 200 runs a second confidential virtual machine to generate an encryption key based on the second private key and the first public key.

[0224] In another possible embodiment, the processor of the second computing device 200 generates an encryption key based on the second private key and the first public key in the second non-trusted execution environment.

[0225] After generating the encryption key, the processor of the second computing device 200 may store the encryption key in the memory of the second non-trusted execution environment or the memory of the second TEE.

[0226] It can be understood that the processor of the first computing device 100 generates an encryption key in the first TEE based on the first private key and the second public key, and the processor of the second computing device 200 generates an encryption key in the second TEE based on the second private key and the first public key. The processor of the first computing device 100 stores the encryption key in the memory of the first TEE, and the processor of the second computing device 200 stores the encryption key in the memory of the second TEE. In contrast, the processor of the first computing device 100 generates an encryption key in the first non-trusted execution environment based on the first private key and the second public key, and the processor of the second computing device 200 generates an encryption key in the second non-trusted execution environment based on the second private key and the first public key. The processor of the first computing device 100 stores the encryption key in the memory of the first non-trusted execution environment, and the processor of the second computing device 200 stores the encryption key in the memory of the second non-trusted execution environment. Since the security of the first TEE is higher than that of the first non-trusted execution environment, and the security of the second TEE is higher than that of the second non-trusted execution environment, the former implementation can better protect the security of the encryption key, thereby better protecting the security of the status information of the first confidential container and reducing the risk of leakage of the status information of the first confidential container.

[0227] It can also be understood that if Figure 6 In the key negotiation process shown, the operations performed by the processor of the first computing device 100 / second computing device 200 in the first TEE / second TEE are all executed by the processor of the first computing device 100 / second computing device 200 running the first confidential virtual machine / second confidential virtual machine. Since the first confidential virtual machine is isolated from other confidential virtual machines in the first computing device 100, and the second confidential virtual machine is isolated from other confidential virtual machines in the second computing device 200, it can be ensured that the first private key is only known to the first confidential virtual machine, the second private key is only known to the second confidential virtual machine, and the encryption key is only known to the first confidential virtual machine and the second confidential virtual machine. Other confidential virtual machines in the first computing device 100 will not know the first private key and the encryption key, and other confidential virtual machines in the second computing device 200 will not know the second private key and the encryption key. Therefore, the risk of leakage of the status information of the first confidential container can be further reduced, and the data security can be better protected.

[0228] The following introduces Figure 6 each step with two specific examples.

[0229] Example 1: In S601, the processor of the first computing device 100 runs the first confidential virtual machine to generate the first private key as a random number a, and the first public key is A = g^a (mod p). In S602, the processor of the second computing device 200 runs the second confidential virtual machine to generate the second private key as a random number b, and the second public key is B = g^b (mod p), where g and p can be preset in the processors of the first computing device 100 and the second computing device 200, or can be pre-negotiated by the processors of the first computing device 100 and the second computing device 200. Then, in S603 - S604, the processor of the first computing device 100 runs the first confidential virtual machine and the processor of the second computing device 2000 runs the second confidential virtual machine for public key exchange. Next, in S605, the processor of the first computing device 100 runs the first confidential virtual machine and calculates the encryption key S = B^a (mod p) according to B, a, and p. In S606, the processor of the second computing device 200 runs the second confidential virtual machine and calculates the encryption key S = A^b (mod p) according to A, b, and p.

[0230] Example 2: In S601, the processor of the first computing device 100 runs the first confidential virtual machine to generate the first private key as a random number a, and the first public key is A = a * Q(x, y). In S602, the processor of the second computing device 200 runs the second confidential virtual machine to generate the second private key as a random number b, and the second public key is B = b * Q(x, y), where Q(x, y) can be pre-negotiated by the processors of the first computing device 100 and the second computing device 200, such as the base point G on a certain pre-negotiated elliptic curve. Then, in S603 - S604, the first computing device 100 and the second computing device 200 conduct public key exchange. Next, in S605, the processor of the first computing device 100 runs the first confidential virtual machine and calculates the encryption key S = a * B = a * b * Q(x, y) according to the first private key a and the second public key B. In S606, the processor of the second computing device 200 can run the second confidential virtual machine and calculate the encryption key S = b * A = b * a * Q(x, y) according to the second private key b and the first public key A.

[0231] It should be understood that the above Example 1 and Example 2 are only examples of the implementation methods for the processors of the first computing device 100 and the second computing device 200 to negotiate and obtain the encryption key, and should not be regarded as specific limitations.

[0232] It can be seen that in the above Examples 1 and 2, the first confidential virtual machine and the second confidential virtual machine generate an encryption key based on public information (i.e., the public key of the other party) and the private information held by each (i.e., the private key). The private information is not transmitted between the two. Even if the public information is intercepted during transmission, since the private information is not leaked, the interceptor cannot obtain the encryption key. Therefore, the security of the state information of the first container encrypted with the encryption key during transmission can be protected. In addition, both the first confidential virtual machine and the second confidential virtual machine are located in the TEE. Due to the high security of the TEE, the encryption key can be well protected.

[0233] Based on the fact that the processor of the first computing device 100 running the first confidential virtual machine and the processor of the second computing device 200 running the second confidential virtual machine negotiate to obtain an encryption key, in a possible embodiment, in order to further protect the security of the state information of the first confidential container, before the processor of the first computing device 100 running the first confidential virtual machine and the processor of the second computing device 200 running the second confidential virtual machine perform key negotiation, the processor of the first computing device 100 can run the first confidential virtual machine to send an authentication request to the second confidential virtual machine, requesting to obtain the authentication information of the second confidential virtual machine to authenticate the second confidential virtual machine. If the authentication is passed, an encryption key is negotiated with the second confidential virtual machine; otherwise, key negotiation is not performed.

[0234] Specifically, the processor of the first computing device 100 can run the first confidential virtual machine to generate an authentication request, and then call the first communication interface in the first non-trusted execution environment to send the authentication request to the second communication interface. After receiving the authentication request, the second communication interface is run by the processor of the second computing device 200 to run the second confidential virtual machine to obtain the authentication information of the second confidential virtual machine according to the authentication request, and then call the second communication interface in the second non-trusted execution environment to return the authentication information of the second confidential virtual machine to the first communication interface. After receiving the authentication information of the second confidential virtual machine, the first communication interface can store the authentication information of the second confidential virtual machine in the memory of the first non-trusted execution environment. Subsequently, the processor of the first computing device 100 can obtain the authentication information of the second confidential virtual machine from the memory of the first non-trusted execution environment, and run the first confidential virtual machine to authenticate the second confidential virtual machine according to the authentication information of the second confidential virtual machine.

[0235] The authentication information of the second confidential virtual machine may include the current measurement value of the configuration of the second confidential virtual machine and the historical measurement value of the configuration of the second confidential virtual machine. Optionally, the authentication information of the second confidential virtual machine may include the certificate of the second confidential virtual machine (which may also be a certificate chain), and the historical measurement value of the configuration of the second confidential virtual machine may be carried in the certificate of the second confidential virtual machine. Among them, the configuration of the second confidential virtual machine may include the virtual processor core type, the number of virtual processor cores, the memory address, the name and version of the APP installed on the second confidential virtual machine, and the image of the second confidential virtual machine, etc.; the measurement value of the configuration is used to measure whether the configuration has been tampered with. For example, compare the current number of virtual processor cores with the historical number of virtual processor cores recorded in the certificate. If they are the same, it means that the number of virtual processor cores has not been tampered with. Otherwise, it means that the number of virtual processor cores has been tampered with (increased or decreased). Another example is to compare the current hash value of the image of the second confidential virtual machine with the historical hash value of the image of the second confidential virtual machine recorded in the certificate. If they are the same, it means that the image has not been tampered with. Otherwise, it means that the image has been tampered with. In this application, if the configuration of the second confidential virtual machine is tampered with, it can be understood that the second confidential virtual machine is incomplete and is an untrusted virtual machine.

[0236] Optionally, the authentication information of the second confidential virtual machine may include the identity information of the second confidential virtual machine, and this identity information is used to uniquely identify and recognize the second confidential virtual machine. The identity information of the second confidential virtual machine may include the name of the second confidential virtual machine, the universally unique identifier (UUID), the media access control address (MAC address), the operating system information (such as the operating system type, version number), and the virtual hardware information (such as the number of CPU cores, the memory size, the disk capacity, the device configuration, etc.). It can be understood that the processor of the first computing device 100 runs the first confidential virtual machine, and the identity of the second confidential virtual machine can be authenticated through the identity information of the second confidential virtual machine. If the identity authentication is passed, it is determined that the second confidential virtual machine is a trusted virtual machine. Otherwise, it is determined that the second confidential virtual machine is an untrusted virtual machine. Optionally, the certificate of the second confidential virtual machine also includes the identity information provided when the second confidential virtual machine applies for the certificate, the validity period of the certificate, the issuing authority of the certificate, the digital signature of the certificate, etc.

[0237] Specifically, the processor of the first computing device 100 may run the first confidential virtual machine to first authenticate the legality and integrity of the certificate of the second confidential virtual machine. For example, determine whether the certificate is valid according to the validity period of the certificate, determine whether the issuing authority of the certificate is a legal authority, and determine whether the certificate has been tampered with according to the digital signature of the certificate. When it is determined that the certificate is legal and complete, determine whether the second confidential virtual machine is a trusted virtual machine by checking the current measurement value of the configuration of the second confidential virtual machine and the certificate, and / or by checking whether the identity information of the second confidential virtual machine matches the identity information recorded in the certificate.

[0238] To protect the security of the second computing device 200 and the second confidential virtual machine in the second computing device 200, in a possible embodiment, before the processor of the second computing device 200 runs the second confidential virtual machine and the processor of the first computing device 100 runs the first confidential virtual machine to negotiate an encryption key, the processor of the first computing device 100 may also run the first confidential virtual machine to send the authentication information of the first confidential virtual machine to the second confidential virtual machine, so that the second computing device 200 authenticates the first confidential virtual machine according to the authentication information. If the authentication is passed, an encryption key is negotiated with the first computing device 100; otherwise, the key negotiation is not performed. Optionally, the authentication information of the first confidential virtual machine may be carried in the authentication request sent by the first confidential virtual machine to the second confidential virtual machine as described above. That is to say, the above authentication request is also used to request the second confidential virtual machine to authenticate the first confidential virtual machine based on the authentication information of the first confidential virtual machine.

[0239] Specifically, the processor of the second computing device 200 may run the first confidential virtual machine to obtain the authentication information of the first confidential virtual machine, generate an authentication request carrying the authentication information of the first confidential virtual machine, and then call the first communication interface in the first non-trusted execution environment to send the authentication request to the second communication interface. After receiving the authentication request, the second communication interface may store the authentication information of the first confidential virtual machine carried in the authentication request in the memory of the second non-trusted execution environment. Subsequently, the processor of the second computing device 200 may obtain the authentication information of the first confidential virtual machine from the memory of the second non-trusted execution environment and run the second confidential virtual machine to authenticate the first confidential virtual machine according to the authentication information of the first confidential virtual machine.

[0240] The authentication information of the first confidential virtual machine described above may include the current measurement value of the configuration of the first confidential virtual machine and the historical measurement value of the configuration of the first confidential virtual machine. Optionally, the authentication information of the first confidential virtual machine described above may include the certificate of the first confidential virtual machine (which may also be a certificate chain), and the historical measurement value of the configuration of the first confidential virtual machine may be carried in the certificate of the first confidential virtual machine. Among them, the configuration of the first confidential virtual machine may include the virtual processor core type, the number of virtual processor cores, the memory address, the name and version of the APP installed on the first confidential virtual machine, and the image of the first confidential virtual machine, etc. The measurement value of the configuration is used to measure whether the configuration has been tampered with. For example, compare the current number of virtual processor cores with the historical number of virtual processor cores recorded in the certificate. If they are the same, it means that the number of virtual processor cores has not been tampered with. Otherwise, it means that the number of virtual processor cores has been tampered with (increased or decreased). Another example is to compare the current hash value of the image of the first confidential virtual machine with the historical hash value of the image of the first confidential virtual machine recorded in the certificate. If they are the same, it means that the image has not been tampered with. Otherwise, it means that the image has been tampered with. In this application, if the configuration of the first confidential virtual machine is tampered with, it can be understood that the first confidential virtual machine is incomplete and is an untrusted virtual machine.

[0241] Optionally, the authentication information of the first confidential virtual machine described above may include the identity information of the first confidential virtual machine, and this identity information is used to uniquely identify and recognize the first confidential virtual machine. The identity information of the first confidential virtual machine may include the name of the first confidential virtual machine, the universally unique identifier (UUID), the media access control address (MAC address), the operating system information (such as the operating system type, version number), and the virtual hardware information (such as the number of CPU cores, the memory size, the disk capacity, the device configuration, etc.). It can be understood that the processor of the second computing device 200 runs the second confidential virtual machine, authenticates the identity of the first confidential virtual machine through the identity information of the first confidential virtual machine. In the case where the identity authentication is passed, it is determined that the first confidential virtual machine is a trusted virtual machine. Otherwise, it is determined that the first confidential virtual machine is an untrusted virtual machine. Optionally, the certificate of the first confidential virtual machine described above also includes the identity information provided when the first confidential virtual machine applies for the certificate, the validity period of the certificate, the issuing authority of the certificate, the digital signature of the certificate, etc.

[0242] Specifically, the processor of the second computing device 200 runs a second confidential virtual machine. It can first authenticate the legality and integrity of the certificate of the first confidential virtual machine. For example, it determines whether the certificate is valid according to the validity period of the certificate, determines whether the issuing authority of the certificate is a legal authority, and determines whether the certificate has been tampered with according to the digital signature of the certificate. When it is determined that the certificate is legal and complete, it determines whether the first confidential virtual machine is a trusted virtual machine by checking whether the current measurement value of the configuration of the first confidential virtual machine is the same as (identical to) the historical measurement value of the configuration of the first confidential virtual machine recorded in the certificate, and / or determines whether the first confidential virtual machine is a trusted virtual machine by checking whether the identity information of the first confidential virtual machine matches the identity information recorded in the certificate.

[0243] In a possible embodiment, after the processors of the first computing device 100 and the second computing device 200 negotiate to obtain an encryption key, and before the processor of the first computing device 100 encrypts the status information of the first confidential virtual machine using the encryption key, the processor of the first computing device 100 can also verify the validity of the encryption key, that is, verify whether the encryption key held by the processor of the first computing device 100 is the same as the encryption key held by the processor of the second computing device 200, to ensure the accurate progress of the migration process of the first confidential container.

[0244] Specifically, the processor of the first computing device 100 can verify the validity of the encryption key in the following manner:

[0245] After the processors of the first computing device 100 and the second computing device 200 negotiate to obtain an encryption key, the processor of the second computing device 200 can generate a message authentication code in the second non-trusted execution environment / second TEE according to the encryption key and the first message (in order to distinguish from the message authentication code generated by the processor of the first computing device 100 below, the message authentication code generated by the processor of the second computing device 200 is hereinafter referred to as the second message authentication code, and the message authentication code generated by the processor of the first computing device 100 is referred to as the first message authentication code), and call the second communication interface in the second non-trusted execution environment to send the second message authentication code and the first message to the first communication interface. After receiving the second message authentication code and the first message, the first communication interface can store the second message authentication code and the first message in the memory of the first non-trusted execution environment. Subsequently, the processor of the first computing device 100 can obtain the second message authentication code and the first message from the memory of the first non-trusted execution environment, and generate a first message authentication code in the first non-trusted execution environment / first TEE according to the encryption key held by itself and the first message, and then compare the first message authentication code and the second message authentication code. When it is determined that the two are the same, it determines that the encryption key held by itself is valid; otherwise, it determines that the encryption key held by itself is invalid.

[0246] It can be understood that the processor of the second computing device 200 generates a second message authentication code in the second TEE, and the processor of the first computing device 100 generates a first message authentication code in the first TEE. In contrast, the processor of the second computing device 200 generates a second message authentication code in the second non-trusted execution environment, and the processor of the first computing device 100 generates a first message authentication code in the first non-trusted execution environment. Since the security of the first TEE is higher than that of the first non-trusted execution environment, and the security of the second TEE is higher than that of the second non-trusted execution environment, the former implementation can better protect the security of the encryption key, thereby better protecting the security of the status information of the first confidential container and reducing the leakage risk of the status information of the first confidential container.

[0247] In a more specific embodiment, based on the fact that the processor of the first computing device 100 runs the first confidential virtual machine and the processor of the second computing device 200 runs the second confidential virtual machine to negotiate and obtain the encryption key, the processor of the second computing device 200 generates a second message authentication code in the second TEE, which is generated for the processor of the second computing device 200 to run the second confidential virtual machine. The processor of the first computing device 100 generates a first message authentication code in the first TEE, which is generated for the processor of the first computing device 100 to run the first confidential virtual machine. In contrast, when the processor of the second computing device 200 runs other software in the second TEE (such as running the second virtual machine manager) to generate the second message authentication code, and the processor of the first computing device 100 runs other software in the first TEE (such as running the first virtual machine manager) to generate the first message authentication code. Since the second confidential virtual machine is isolated from other confidential virtual machines in the second TEE, the second confidential virtual machine shares the second virtual machine manager with other confidential virtual machines in the second TEE, the first confidential virtual machine is isolated from other confidential virtual machines in the first TEE, and the first confidential virtual machine shares the first virtual machine manager with other confidential virtual machines in the first TEE. Generating the message authentication code by the second confidential virtual machine and the first confidential virtual machine can better protect the security of the encryption key, thereby better protecting the security of the status information of the first confidential container and reducing the leakage risk of the status information of the first confidential container.

[0248] In a specific implementation, the manner in which the processor of the second computing device 200 generates a second message authentication code based on the encryption key and the first message it holds, and the manner in which the processor of the first computing device 100 generates a first message authentication code based on the encryption key and the first message it holds, can be to sign the encryption key and the first message using the same signature algorithm, and the obtained signature information is the message authentication code. The signature algorithm includes but is not limited to RSA, DSA, etc.

[0249] ByFigure 3 It can be known that the first TEE deploys a first virtual machine manager, the second TEE deploys a second virtual machine manager, the first non-trusted execution environment deploys a third virtual machine manager, and the second non-trusted execution environment deploys a fourth virtual machine manager. In a possible embodiment, the operation of calling the first communication interface executed by the processor of the first computing device 100 in the first non-trusted execution environment can be executed by the processor of the first computing device 100 running the third virtual machine manager, and the operation of calling the second communication interface executed by the processor of the second computing device 200 in the second non-trusted execution environment can be executed by the processor of the second computing device 200 running the fourth virtual machine manager.

[0250] It should be understood that the magnitudes of the sequence numbers of the steps in the above embodiments do not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation to the implementation process of the embodiments of the present application.

[0251] The container migration method and system provided by the present application are elaborated in detail above. Based on the same inventive concept, the container migration device and computing device provided by the present application will be introduced below.

[0252] It should be understood that the unit modules inside the container migration device can also be divided in various ways. Each module can be a software module, a hardware module, or part of it can be a software module and part of it can be a hardware module. The present application does not limit it.

[0253] See Figure 7 , Figure 7 is a schematic structural diagram of another container migration device 700 provided by an embodiment of the present application, which can be applied to Figure 3 the second computing device 200 shown in Figure 7 As shown, the device 700 includes: an acquisition module 710, a decryption module 720, and a synchronization module 730.

[0254] The functions of each module of the container migration device 700 will be introduced exemplarily below. It should be understood that the functions of each module described by way of example below are only the functions that the container migration device 700 can have in some embodiments of the present application. The present application does not limit the functions of each module.

[0255] The acquisition module 710 is configured to acquire the encrypted information of the status information of the first confidential container. The first confidential container is deployed in the first TEE, and the first TEE is the TEE of the first computing device 100;

[0256] The decryption module 720 is configured to decrypt the encrypted information to obtain the status information of the first confidential container;

[0257] A synchronization module 730, configured to implement status synchronization between a standby confidential container and a first confidential container based on the status information of the first confidential container.

[0258] In some possible embodiments, the status of the first confidential container includes the memory status of the first confidential container, or the status of the first confidential container includes the memory status of the first confidential container and the communication status between the first confidential container and the virtual IO device.

[0259] In some possible embodiments, the second TEE includes a second confidential virtual machine, the standby confidential container is deployed in the second confidential virtual machine, the acquisition module 710 is deployed in the second non-trusted execution environment, and the decryption module 720 is deployed in the second confidential virtual machine.

[0260] In some possible embodiments, as Figure 7 shown, the apparatus 700 further includes a key negotiation module 740, deployed in the second confidential virtual machine; the encrypted information is obtained by encrypting the status information of the first confidential container using an encryption key, the encryption key is obtained through negotiation between the key negotiation module 740 and the first confidential virtual machine, and the encryption key is also used to decrypt the encrypted information. The first confidential virtual machine is deployed in the first TEE, and the first confidential container is deployed in the first confidential virtual machine.

[0261] In some possible embodiments, as Figure 7 shown, the apparatus 700 further includes an authentication module 750, deployed in the second confidential virtual machine;

[0262] The above-mentioned acquisition module 710 is further configured to acquire the authentication information of the first confidential virtual machine;

[0263] The above-mentioned key negotiation module 740 is configured to negotiate an encryption key with the first confidential virtual machine when the authentication module 750 authenticates that the first confidential virtual machine passes based on the authentication information of the first confidential virtual machine.

[0264] In some possible embodiments, as Figure 7 shown, the apparatus 700 further includes a sending module 760, deployed in the second non-trusted execution environment;

[0265] The above-mentioned authentication module 750 is further configured to acquire the authentication information of the second confidential virtual machine;

[0266] The above-mentioned sending module 760 is configured to call the second communication interface to send the authentication information of the second confidential virtual machine to the first communication interface. The second communication interface is the communication interface of the second computing device 200, and the first communication interface is the communication interface of the first computing device 100.

[0267] In some possible embodiments, the authentication information of the first confidential virtual machine includes the current measurement value of the configuration of the first confidential virtual machine and the historical measurement value of the configuration of the first confidential virtual machine;

[0268] The above authentication module 750 is used to authenticate that the first confidential virtual machine passes when it is determined that the current measurement value of the configuration of the first confidential virtual machine is the same as the historical measurement value of the configuration of the first confidential virtual machine.

[0269] In some possible embodiments, the above authentication module 750 is further used to generate a message authentication code based on an encryption key and a first message; the above sending module 760 is further used to call a second communication interface to send the first message and the message authentication code to the first communication interface.

[0270] In some possible embodiments, the communication status information between the first confidential container and the virtual IO device includes the identifier of the first confidential container;

[0271] The above synchronization module 730 is specifically used to: load the memory status information of the first confidential container into the memory of the standby confidential container, and load the communication status information between the first confidential container and the virtual IO device into the virtual IO device driver of the second confidential virtual machine; based on the memory status information of the first confidential container in the memory of the standby confidential container, and based on the communication status information of the first confidential container located in the virtual IO device driver according to the identifier of the first confidential container, run the standby confidential container, so as to realize the memory status synchronization between the standby confidential container and the first confidential container, and realize the communication status synchronization between the standby confidential container and the first confidential container.

[0272] See Figure 8 , Figure 8 FIG. is a schematic structural diagram of a container migration device 800 provided by an embodiment of the present application, which can be applied to Figure 3 the first computing device 100 shown in Figure 8 As shown, the device 800 includes: a sending module 810 and an encryption module 820.

[0273] Next, the functions of each module of the container migration device 800 will be introduced exemplarily. It should be understood that the functions of each module described by the following examples are only the functions that the container migration device 800 can have in some embodiments of the present application, and the present application does not limit the functions of each module.

[0274] The encryption module 820 is used to encrypt the status information of the first confidential container in the first TEE to obtain encrypted information;

[0275] A sending module 810, configured to call a first communication interface in a first non-trusted execution environment, and send encrypted information to a second computing device 200, where the encrypted information is used to migrate status information of a first confidential container to a standby confidential container, the standby confidential container is deployed in a second TEE, the second TEE is a TEE of the second computing device 200, and the first communication interface is a communication interface of the first computing device 100.

[0276] In some possible embodiments, the status of the first confidential container includes the memory status of the first confidential container, or the status of the first confidential container includes the memory status of the first confidential container and the communication status between the first confidential container and a virtual IO device.

[0277] In some possible embodiments, the first TEE includes a first confidential virtual machine, and the first confidential container is located in the first confidential virtual machine; the above-mentioned encryption module 820 is deployed in the first confidential virtual machine, and the above-mentioned sending module 810 is deployed in the first non-trusted execution environment.

[0278] In some possible embodiments, as Figure 8 shown, the above-mentioned device 800 further includes a key negotiation module 830, which is deployed in the first confidential virtual machine; the encrypted information is obtained by encrypting the status information of the first confidential container using an encryption key, the encryption key is obtained by negotiation between the key negotiation module 830 and a second confidential virtual machine, and the encryption key is also used to decrypt the encrypted information.

[0279] In some possible embodiments, as Figure 8 shown, the above-mentioned device 800 further includes an authentication module 840, which is deployed in the first confidential virtual machine;

[0280] The above-mentioned authentication module 840 is configured to obtain authentication information of the first confidential virtual machine;

[0281] The above-mentioned sending module 810 is configured to call the first communication interface and send the authentication information of the first confidential virtual machine to a second communication interface;

[0282] In some possible embodiments, as Figure 8 shown, the above-mentioned device 800 further includes an acquisition module 850, which is deployed in the first non-trusted execution environment, and the above-mentioned acquisition module 850 is further configured to acquire authentication information of the second confidential virtual machine;

[0283] The above-mentioned key negotiation module 830 is configured to negotiate an encryption key with the second confidential virtual machine when the authentication module 840 authenticates that the second confidential virtual machine passes based on the authentication information of the second confidential virtual machine.

[0284] In some possible embodiments, the authentication information of the second confidential virtual machine includes the current measurement value of the configuration of the second confidential virtual machine and the historical measurement value of the configuration of the second confidential virtual machine;

[0285] The above authentication module 840 is used to authenticate that the second confidential virtual machine passes when it is determined that the current metric value of the configuration of the second confidential virtual machine is the same as the historical metric value of the configuration of the second confidential virtual machine.

[0286] In some possible embodiments, the above acquisition module 850 is used to acquire a first message and a message authentication code from the second confidential virtual machine, and the message authentication code is generated by the second confidential virtual machine based on an encryption key and the first message;

[0287] The above encryption module 820 is used to encrypt the status information of the first confidential container with the encryption key when the above authentication module 840 determines that the encryption key is a valid key based on the first message and the message authentication code.

[0288] In some possible embodiments, the communication status information between the first confidential container and the virtual IO device includes the identifier of the first confidential container; before the above encryption module 820 encrypts the status information of the first confidential container to obtain encrypted information, the encryption module 820 is further used to: locate the communication status information between the first confidential container and the virtual IO device in the virtual IO device driver of the first confidential virtual machine based on the identifier of the first confidential container.

[0289] Specifically, for the specific implementation of the above container migration device 700 and container migration device 800 to perform various operations, reference may be made to the description in the relevant content of the above container migration method embodiment. For the sake of simplicity of the specification, it will not be elaborated here.

[0290] See Figure 9 , Figure 9 FIG. is a schematic structural diagram of a computing device 900 provided by an embodiment of the present application. The computing device 900 includes: a processor 910, a memory unit 920, a communication interface 930, a memory 940, an input device 950, and an output device 960. Among them, the processor 910, the memory unit 920, the communication interface 930, the memory 940, the input device 950, and the output device 960 can be interconnected through a bus 970.

[0291] Among them,

[0292] The processor 910 can read the program code (including instructions) stored in the memory unit 920, execute the program code stored in the memory unit 920, so that the computing device 900 executes the steps performed by the first computing device 100 or the second computing device 200 in the container migration method provided by the above method embodiment.

[0293] The processor 910 can have various specific implementation forms. For example, the processor 910 can be at least one CPU, such as Figure 9As shown, including CPU0 and CPU1, the processor 910 can also be a graphics processing unit (GPU), etc. The processor 910 can also be a single-core processor or a multi-core processor. The processor 910 can be a combination of a CPU and a hardware chip. The above hardware chip can be implemented by an application-specific integrated circuit (ASIC) or a programmable logic device (PLD). The above PLD can be implemented by a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof. The processor 910 can also be implemented by a logic device with built-in processing logic alone, such as an FPGA or a digital signal processor (DSP), etc.

[0294] The memory unit 920 is used to store kernels, program codes, and program data generated when the processor 910 executes the program codes stored in the memory unit 920.

[0295] When the computing device 900 is used to execute the steps performed by the second computing device 200 in the container migration method provided in the above method embodiments, the program codes stored in the memory unit 920 include: the code of the acquisition module 710, the code of the decryption module 720, the code of the synchronization module 730, the code of the key negotiation module 740, the code of the authentication module 750, the code of the sending module 760, etc. The program data stored in the memory unit 920 includes: encryption keys, status information of the first confidential container, encryption information, etc.

[0296] When the computing device 900 is used to execute the steps performed by the first computing device 100 in the container migration method provided in the above method embodiments, the program codes stored in the memory unit 920 include: the code of the sending module 810, the code of the encryption module 820, the code of the key negotiation module 830, the code of the authentication module 840, the code of the acquisition module 850, etc. The program data stored in the memory unit 920 includes: encryption keys, status information of the first confidential container, encryption information, etc.

[0297] The communication interface 930 can be a wired interface (such as an Ethernet interface, a fiber optic interface, other types of interfaces (e.g., an InfiniBand (IB) interface)), or a wireless interface (such as a cellular network interface or a wireless local area network interface) for communicating with other computing devices or apparatuses. When the communication interface 930 is a wired interface, the communication interface 930 can adopt a protocol family over the Transmission Control Protocol / Internet Protocol (TCP / IP), such as, for example, the Remote Function Call (RFC) protocol, the Simple Object Access Protocol (SOAP) protocol, the Simple Network Management Protocol (SNMP) protocol, the Common Object Request Broker Architecture (CORBA) protocol, and distributed protocols, etc.

[0298] The memory 940 can be a non-volatile memory, such as, for example, a Read-Only Memory (ROM), a Programmable ROM (PROM), an Erasable Programmable ROM (EPROM), an Electrically Erasable Programmable ROM (EEPROM), or a flash memory. The memory 940 can also be a volatile memory, and the volatile memory can be a Random Access Memory (RAM), which is used as an external cache.

[0299] The input device 950 can include a mouse, a keyboard, and so on. A user can input data or instructions to the computing device 900 through the input device 950, such as inputting a container migration instruction (the instruction indicates the migration of the first confidential container), and so on.

[0300] The output device 960 may include a display. The computing device 900 can provide data to the user through the display. For example, after the migration of the first confidential container is completed, a migration completion notification can be displayed to the user, etc. The display may include a cathode ray tube (CRT) display, a plasma display panel (PDP), a liquid crystal display (LCD), etc. Taking the LCD as an example, the liquid crystal display includes a liquid crystal panel and a backlight module. Among them, the liquid crystal display panel includes a polarizing film, a glass substrate, a black matrix, a color filter, a protective film, a common electrode, an alignment layer, a liquid crystal layer (liquid crystal, spacer, sealant), a capacitor, a display electrode, a prism layer, and a diffuser layer. The backlight module includes: a lighting source, a reflector, a light guide plate, a diffusion sheet, a brightness enhancement film (prism sheet), and a frame, etc.

[0301] The bus 970 can be a PCIE or an extended industry standard architecture (EISA) bus, etc. The above bus 970 can be divided into an address bus, a data bus, a control bus, etc. For the sake of convenience of representation, Figure 9 only a thick line is used to represent it in the figure, but it does not mean that there is only one bus or one type of bus.

[0302] It should be understood that the computing device 900 in the embodiments of the present application can correspond to the computing device including the container migration device 700 or the container migration device 800 in the embodiments of the present application, and can correspond to the execution of the Figure 4 、 Figure 5 、 Figure 6 corresponding main body in the method shown. And the operations and / or functions of each module in the computing device 900 are respectively for implementing Figure 4 、 Figure 5 、 Figure 6 the corresponding processes of the method shown. For the sake of brevity, they will not be elaborated here.

[0303] It should be understood that the computing device 900 is only an example provided in the embodiments of the present application. And the computing device 900 may have more or fewer components than Figure 9 the components shown, can combine two or more components, or can have different configurations of components to implement.

[0304] The embodiments of the present application also provide a container migration system, which may include the above-mentioned container migration device 700 and container migration device 800.

[0305] An embodiment of this application also provides a computer-readable storage medium. Instructions are stored in the computer-readable storage medium, and when the instructions are run, some or all of the steps of the container migration method described in the above embodiments can be implemented.

[0306] An embodiment of this application also provides a computer program product. When the computer program product is read and executed by a computer, some or all of the steps of the container migration method described in the above method embodiments can be implemented.

[0307] In the above embodiments, the descriptions of the respective embodiments have their own focuses. For parts not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0308] In the above embodiments, it can be implemented in whole or in part by software, hardware, or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions described in the embodiments of this application are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from a website, computer, server, or data center to another website, computer, server, or data center by wire (such as coaxial cable, optical fiber, digital subscriber line) or wirelessly (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer, or a data storage device such as a server or data center that includes one or more integrated available media. The available medium can be a magnetic medium (such as a floppy disk, hard disk, magnetic tape), an optical medium, or a semiconductor medium, etc.

[0309] The above is only the specific implementation manner of this application. Those skilled in the art of this technology can think of changes or substitutions according to the specific implementation manner provided by this application, and all should be covered within the protection scope of this application.

Claims

1. A container migration system, It is characterized in that The system includes a first computing device and a second computing device, the software and hardware resources of the first computing device are divided into a first trusted execution environment TEE and a first untrusted execution environment, the software and hardware resources of the second computing device are divided into a second TEE and a second untrusted execution environment, the first TEE includes a first container, and the second TEE includes a backup container; The first computing device is used to send encrypted information of the state information of the first container to the second computing device; The second computing device is used to receive and decrypt the encrypted information to obtain the state information of the first container; The second computing device is further configured to synchronize the status of the backup container with the first container based on the status information of the first container.

2. The system according to claim 1, It is characterized in that The state of the first container includes a memory state of the first container, or the state of the first container includes a memory state of the first container and a communication state between the first container and a virtual input / output (IO) device.

3. The system according to claim 2, It is characterized in that The first TEE includes a first virtual machine, the second TEE includes a second virtual machine, the first container is located in the first virtual machine, and the backup container is located in the second virtual machine; The processor of the first computing device is used to run the first virtual machine and encrypt the state information of the first container to obtain the encrypted information; The processor of the first computing device is used to call a first communication interface in the first untrusted execution environment to send the encrypted information to a second communication interface, the first communication interface is the communication interface of the first computing device, and the second communication interface is the communication interface of the second computing device; The processor of the second computing device is used to obtain the encrypted information in the second untrusted execution environment; The processor of the second computing device is used to run the second virtual machine and decrypt the encrypted information to obtain the state information of the first container; The processor of the second computing device is used to run the second virtual machine and synchronize the status of the backup container with the first container based on the status information of the first container.

4. The system according to claim 3, It is characterized in that The encrypted information is obtained by encrypting the state information of the first container using an encryption key, the encryption key is obtained by the first virtual machine and the second virtual machine through negotiation, and the encryption key is also used to decrypt the encrypted information.

5. The system according to claim 4, It is characterized in that The processor of the first computing device is used to run the first virtual machine to obtain authentication information of the first virtual machine; The processor of the first computing device is used to call the first communication interface in the first untrusted execution environment and send the authentication information of the first virtual machine to the second communication interface; The processor of the second computing device is used to obtain authentication information of the first virtual machine in the second untrusted execution environment; The processor of the second computing device is used to run the second virtual machine, and negotiate with the first virtual machine to obtain the encryption key when the first virtual machine is authenticated based on the authentication information of the first virtual machine.

6. The system according to claim 4 or 5, It is characterized in that The processor of the second computing device is used to run the second virtual machine and obtain authentication information of the second virtual machine; The processor of the second computing device is used to call the second communication interface in the second untrusted execution environment and send the authentication information of the second virtual machine to the first communication interface; The processor of the first computing device is used to obtain authentication information of the second virtual machine in the first untrusted execution environment; The processor of the first computing device is used to run the first virtual machine, and when the second virtual machine is authenticated based on the authentication information of the second virtual machine, negotiate with the second virtual machine to obtain the encryption key.

7. The system according to claim 6, It is characterized in that The authentication information of the first virtual machine includes a current measurement value of a configuration of the first virtual machine and a historical measurement value of the configuration of the first virtual machine, and the authentication information of the second virtual machine includes a current measurement value of a configuration of the second virtual machine and a historical measurement value of the configuration of the second virtual machine; The processor of the second computing device is used to run the second virtual machine, and if it is determined that the current measurement value of the configuration of the first virtual machine is the same as the historical measurement value of the configuration of the first virtual machine, authenticating that the first virtual machine passes; The processor of the first computing device is used to run the first virtual machine, and when it is determined that the current measurement value of the configuration of the second virtual machine is the same as the historical measurement value of the configuration of the second virtual machine, the second virtual machine is authenticated.

8. A system according to any one of claims 4 to 7, It is characterized in that The processor of the second computing device is used to run the second virtual machine and generate a message authentication code based on the encryption key and the first message; The processor of the second computing device is used to call the second communication interface in the second untrusted execution environment, and send the first message and the message authentication code to the first communication interface; The processor of the first computing device is used to obtain the first message and the message authentication code in the first untrusted execution environment; The processor of the first computing device is used to run the first virtual machine, and when it is determined that the encryption key is a valid key based on the first message and the message authentication code, use the encryption key to encrypt the state information of the first container.

9. A system according to any one of claims 4 to 8, It is characterized in that The processor of the first computing device is used to run the first virtual machine to generate a first private key and a first public key; The processor of the first computing device is used to call the first communication interface in the first untrusted execution environment and send the first public key to the second communication interface; The processor of the second computing device is used to obtain the first public key in the second untrusted execution environment; The processor of the second computing device is used to run the second virtual machine to generate a second private key and a second public key, and generate the encryption key based on the second private key and the first public key; The processor of the second computing device is used to call the second communication interface in the second untrusted execution environment and send the second public key to the first communication interface; The processor of the first computing device is used to obtain the second public key in the first untrusted execution environment; The processor of the first computing device is used to run the first virtual machine and generate the encryption key based on the first private key and the second public key.

10. The system according to any one of claims 3 to 9, It is characterized in that The communication status information between the first container and the virtual IO device includes an identifier of the first container; Before the processor of the first computing device is used to run the first virtual machine and encrypt the state information of the first container to obtain the encrypted information, the processor of the first computing device is further used to: Running the first virtual machine, and locating communication status information between the first container and the virtual IO device in a virtual IO device driver of the first virtual machine based on the identifier of the first container; After the processor of the second computing device is used to run the second virtual machine and decrypt the encrypted information to obtain the state information of the first container, the processor of the second computing device is specifically used to: Running the second virtual machine, loading the memory status information of the first container into the memory of the standby container, and loading the communication status information between the first container and the virtual IO device into the virtual IO device driver of the second virtual machine; The second virtual machine is run, and based on the memory state information of the first container in the memory of the standby container and the communication state information between the first container and the virtual IO device located in the virtual IO device driver of the second virtual machine based on the identifier of the first container, the standby container is run, so as to synchronize the memory state of the standby container with the first container and synchronize the communication state of the standby container with the first container.

11. A container migration method, It is characterized in that Applied to a second computing device, the software and hardware resources of the second computing device are divided into a second untrusted execution environment and a second TEE, the second TEE includes a backup container, and the method includes: The second computing device obtains encrypted information of state information of a first container, where the first container is deployed in a first TEE, and the first TEE is a TEE of the first computing device; The second computing device decrypts the encrypted information to obtain the state information of the first container; The second computing device synchronizes the status of the backup container with the first container based on the status information of the first container.

12. The method according to claim 11, It is characterized in that The state of the first container includes a memory state of the first container, or the state of the first container includes a memory state of the first container and a communication state between the first container and a virtual IO device.

13. The method according to claim 11 or 12, It is characterized in that The second TEE includes a second virtual machine, and the backup container is deployed on the second virtual machine; The second computing device obtains encrypted information of the state information of the first container, including: The processor of the second computing device obtains, in the second untrusted execution environment, encrypted information of the state information of the first container; The second computing device decrypts the encrypted information to obtain the state information of the first container, including: The processor of the second computing device runs the second virtual machine and decrypts the encrypted information to obtain the state information of the first container; The second computing device synchronizes the status of the standby container with the first container based on the status information of the first container, including: The processor of the second computing device runs the second virtual machine and synchronizes the status of the backup container with the first container based on the status information of the first container.

14. The method according to claim 13, It is characterized in that The encrypted information is obtained by encrypting the status information of the first container using an encryption key, the encryption key is obtained by negotiation between the first virtual machine and the second virtual machine, and the encryption key is also used to decrypt the encrypted information. The first virtual machine is deployed in the first TEE, and the first container is deployed in the first virtual machine.

15. The method according to claim 14, It is characterized in that The method further comprises: The processor of the second computing device obtains authentication information of the first virtual machine in the second untrusted execution environment; The processor of the second computing device runs the second virtual machine, and when the first virtual machine is authenticated successfully based on the authentication information of the first virtual machine, negotiates with the first virtual machine to obtain the encryption key.

16. The method according to claim 14 or 15, It is characterized in that The method further comprises: The processor of the second computing device runs the second virtual machine to obtain authentication information of the second virtual machine; The processor of the second computing device calls a second communication interface in the second non-trusted execution environment to send authentication information of the second virtual machine to a first communication interface, where the second communication interface is the communication interface of the second computing device and the first communication interface is the communication interface of the first computing device.

17. The method according to claim 15 or 16, It is characterized in that The authentication information of the first virtual machine includes a current measurement value of a configuration of the first virtual machine and a historical measurement value of a configuration of the first virtual machine; The method further comprises: The processor of the second computing device runs the second virtual machine, and when it is determined that the current measurement value of the configuration of the first virtual machine is the same as the historical measurement value of the configuration of the first virtual machine, the first virtual machine is authenticated.

18. The method according to any one of claims 14 to 17, It is characterized in that The processor of the second computing device runs the second virtual machine to generate a message authentication code based on the encryption key and the first message; The processor of the second computing device calls a second communication interface in the second untrusted execution environment to send the first message and the message authentication code to a first communication interface, where the second communication interface is the communication interface of the second computing device and the first communication interface is the communication interface of the first computing device.

19. The method according to any one of claims 13 to 18, It is characterized in that The communication status information between the first container and the virtual IO device includes an identifier of the first container; The processor of the second computing device runs the second virtual machine, and synchronizes the state of the standby container with the first container based on the state information of the first container, including: The processor of the second computing device runs the second virtual machine, loads the memory state information of the first container into the memory of the standby container, and loads the communication state information between the first container and the virtual IO device into the virtual IO device driver of the second virtual machine; The processor of the second computing device runs the second virtual machine, and runs the standby container based on the memory state information of the first container in the memory of the standby container and the communication state information between the first container and the virtual IO device located in the virtual IO device driver of the second virtual machine based on the identifier of the first container, so as to achieve memory state synchronization between the standby container and the first container, and achieve communication state synchronization between the standby container and the first container.

20. A container migration method, It is characterized in that Applied to a first computing device, the software and hardware resources of the first computing device are divided into a first untrusted execution environment and a first TEE, the first TEE includes a first container, and the method includes: The processor of the first computing device encrypts the state information of the first container in the first TEE to obtain encrypted information; The processor of the first computing device calls the communication interface of the first computing device in the first non-trusted execution environment, and sends the encrypted information to the second computing device, where the encrypted information is used to migrate the state information of the first container to a backup container, and the backup container is deployed in a second TEE, which is the TEE of the second computing device.

21. A container migration device, It is characterized in that Applied to a second computing device, the software and hardware resources of the second computing device are divided into a second untrusted execution environment and a second TEE, the second TEE includes a backup container, and the apparatus includes: An acquisition module, configured to acquire encrypted information of state information of a first container, where the first container is deployed in a first TEE, and the first TEE is a TEE of the first computing device; a decryption module, used for decrypting the encrypted information to obtain the state information of the first container; A synchronization module is used to synchronize the status of the standby container with the first container based on the status information of the first container.

22. A container migration device, It is characterized in that Applied to a first computing device, the software and hardware resources of the first computing device are divided into a first untrusted execution environment and a first TEE, the first TEE includes a first container, and the apparatus includes: an encryption module, configured to encrypt the state information of the first container in the first TEE to obtain encrypted information; A sending module is used to call the communication interface of the first computing device in the first non-trusted execution environment, and send the encrypted information to the second computing device, wherein the encrypted information is used to migrate the state information of the first container to a backup container, and the backup container is deployed in a second TEE, and the second TEE is the TEE of the second computing device.

23. A computing device, It is characterized in that The computing device comprises a processor and a memory; the processor of the computing device is used to execute instructions stored in the memory of the computing device, so that the computing device executes the method according to any one of claims 11 to 20.

24. A computer-readable storage medium, It is characterized in that The method comprises computer program instructions, and when the computer program instructions are executed by a computing device, the computing device performs the method according to any one of claims 11 to 20.

Images