A distributed caching method, system and acceleration method for large model inference service

Abstract

The application discloses a kind of distributed cache method, system and acceleration method for big model inference service, system includes confidential virtual machine cluster, zero-trust distributed cache system, FUSE client;Zero-trust distributed cache system includes data policy service, data engine plug-in, distributed file system, trusted access service;FUSE client is deployed in confidential container end, and based on remote proof and corresponding confidential container identity legality confirmation proof and trusted access service establish secure connection, to provide clear text data access service to corresponding confidential container.This scheme provides a distributed cache implementation scheme for confidential computing container cluster, while ensuring data security by using large-scale confidential container cluster, avoids the performance loss problem caused by repeated reading and encryption and decryption of the same data to external storage system, realizes the improvement of storage access efficiency in model training and inference and other scenarios.

Description

Technical Field

[0001] This invention belongs to the field of confidential computing containers, and in particular relates to a distributed caching method, system, and acceleration method for large model inference services. Background Technology

[0002] The explosive growth of large-scale models has presented unprecedented challenges to the underlying computing infrastructure. In training and inference scenarios, efficient data storage and transmission have become the core bottleneck restricting model iteration speed and resource utilization. Traditional storage systems often struggle to meet the stringent requirements of high throughput, large capacity, and ecosystem interoperability in large-scale model scenarios.

[0003] Meanwhile, to meet privacy and compliance requirements, confidential computing architectures based on Confidential Containers (CoCo) are gradually becoming one of the preferred solutions for large-scale multi-party joint training and sensitive data inference deployment due to their advantages in data security isolation and computational efficiency. However, as... Figure 1 As shown, to ensure data security, this technology requires obtaining a key through remote authentication each time data is loaded or written, and then encrypting and decrypting the data in a confidential environment. This further increases the latency of data loading and renders traditional data caching services unusable due to inherent security flaws. Summary of the Invention

[0004] The purpose of this invention is to provide a distributed caching method and system for confidential computing container clusters, addressing the aforementioned problems and achieving parallel compatibility between secure isolation and high data access efficiency in large-scale confidential container cluster application scenarios. Another objective of this invention is to provide an acceleration method for large model inference services, implemented using the aforementioned distributed caching system.

[0005] To achieve the above objectives, the present invention adopts the following technical solutions:

[0006] A distributed caching system for confidential computing container clusters includes:

[0007] A confidential virtual machine cluster consists of multiple confidential virtual machine nodes, which are built on a hardware trusted execution environment.

[0008] A zero-trust distributed caching system is loaded into the memory file system of a confidential virtual machine cluster to build a distributed caching cluster;

[0009] The zero-trust distributed caching system includes data policy services, data engine plugins, a distributed file system, and trusted access services.

[0010] The data policy service is used to establish an encrypted communication link between the distributed cache cluster and the cache management terminal based on hardware remote proof, and to receive the data access policy uploaded by the cache management terminal through the encrypted communication link.

[0011] The cache management terminal is used for users to set data access policies, which include external data source address, data source access authentication information, data decryption key, and a whitelist of confidential containers that are allowed to access the current data source.

[0012] The data engine plugin is used to obtain encrypted data from external data sources and decrypt it according to the data access strategy, cache the decrypted data to the distributed file system, and encrypt the data to be written and save it to external storage.

[0013] A distributed file system is used to caching decrypted data in fragments across multiple cache nodes;

[0014] Trusted access service is used to verify the legitimacy of the confidential container's identity through remote authentication and the data access policy.

[0015] The FUSE client, deployed on the confidential container, establishes a secure connection with the trusted access service based on remote authentication and the corresponding confidential container's identity legitimacy verification, in order to provide plaintext data access services to the corresponding confidential container.

[0016] In the aforementioned distributed caching system for confidential computing container clusters, the distributed file system provides metadata caching and data shard read / write caching capabilities, and utilizes encrypted memory and SSD encrypted disks to provide multi-level caching functionality.

[0017] In the aforementioned distributed caching system for confidential computing container clusters, the confidential virtual machine nodes are deployed as follows:

[0018] Build a memory file system that runs on a confidential virtual machine;

[0019] All components of the zero-trust distributed caching system, including the distributed caching component, key management component, and initialization service component, are deployed to the aforementioned in-memory file system;

[0020] Based on the aforementioned memory file system, construct images of all confidential virtual machine nodes of the distributed caching system;

[0021] The confidential virtual machine node is deployed using the aforementioned confidential virtual machine node image.

[0022] In the aforementioned distributed caching system for confidential computing container clusters, the distributed caching components include the data policy management service, data engine plugin, distributed file system, and trusted access service.

[0023] The key management component is used to manage system keys, including inter-node communication keys and data storage keys.

[0024] In the aforementioned distributed caching system for confidential computing container clusters, the key management component is used to derive keys based on hardware root trust.

[0025] Alternatively, a temporary key can be generated based on encrypted memory, and the temporary key can be synchronized to all trusted nodes in the cluster via the Raft protocol.

[0026] In the aforementioned distributed caching system for confidential computing container clusters, the initialization service component is used to:

[0027] When the current node starts, it attempts to establish a connection with an existing distributed cache cluster.

[0028] Upon successful connection, submit a hardware remote proof report to the distributed cache cluster;

[0029] Does the full component metric in the distributed cache cluster verification report conform to the preset strategy?

[0030] After successful verification, receive the communication certificate issued by the distributed caching cluster;

[0031] Based on the aforementioned communication certificate, establish communication with the existing distributed cache cluster, join the cluster, and synchronize the key and data fragmentation information.

[0032] In the aforementioned distributed caching system for confidential computing container clusters, this system also includes a zero-trust confidential virtual machine management system, including the aforementioned confidential virtual machine node image and confidential virtual machine controller.

[0033] Furthermore, the confidential virtual machine controller is compatible with Kubernetes in the following ways:

[0034] By using Kubernetes' CRD extension mechanism, confidential virtual machines are abstracted into native Kubernetes resources, enabling the reuse of the Kubernetes scheduler to achieve hybrid deployment of confidential containers and cache nodes;

[0035] The confidential virtual machine is connected to the Kubernetes CNI network, so that the cache cluster and the container cluster are in the same overlay virtual network layer.

[0036] A distributed caching method for confidential computing container clusters, implemented based on the aforementioned distributed caching system for confidential computing container clusters, includes,

[0037] Build a confidential virtual machine cluster based on a hardware trusted execution environment;

[0038] The zero-trust distributed caching system runs on the aforementioned confidential virtual machine cluster after being verified by hardware metrics in a hardware trusted execution environment.

[0039] An encrypted communication link with the cache management terminal is established based on hardware remote authentication;

[0040] The data access policy uploaded by the cache management terminal is received through the encrypted communication link. The data access policy includes the external data source address, data source access authentication information, data decryption key, and a whitelist of confidential containers that are allowed to access the current data source.

[0041] Based on the aforementioned data access strategy, encrypted data is obtained from and decrypted from external data sources.

[0042] The decrypted data is fragmented and cached across multiple cache nodes;

[0043] The identity of the confidential container is confirmed through remote authentication and the aforementioned data access policy;

[0044] A secure connection is established between the FUSE client deployed on the confidential container and the trusted access service based on remote authentication and corresponding confidential container identity legitimacy verification, so as to provide the corresponding confidential container with plaintext data access service cached on each cache node.

[0045] In the aforementioned distributed caching method for confidential computing container clusters, this method also includes,

[0046] The first node starts up, responding to the cluster initialization request and establishing a new distributed cache cluster;

[0047] The new node starts up and attempts to establish a connection with the existing distributed cache cluster.

[0048] Upon successful connection, submit a hardware remote proof report to the distributed cache cluster;

[0049] Does the full component metric in the distributed cache cluster verification report conform to the preset strategy?

[0050] After successful verification, receive the communication certificate issued by the distributed caching cluster;

[0051] Based on the aforementioned communication certificate, establish communication with the existing distributed cache cluster, join the cluster, and synchronize the key and data fragmentation information.

[0052] An acceleration method for large model inference services, implemented using the aforementioned distributed caching system for confidential computing container clusters, includes:

[0053] The encrypted model file is obtained from the external data source address according to the data access policy, and after decryption, it is stored in fragments to multiple cache nodes of the distributed cache cluster.

[0054] After identity verification and remote authentication, the confidential container establishes a secure connection with the distributed cache cluster.

[0055] The confidential container reads decrypted model file fragments from a nearby cache node;

[0056] Based on Kubernetes' ability to schedule confidential virtual machines, a cache node expansion command is triggered when expansion is needed. The new cache node joins the cluster based on cluster-level remote authentication, synchronizing the key and data shard information.

[0057] The advantages of this invention are:

[0058] This solution provides a distributed caching implementation for confidential computing container clusters. While ensuring data security by utilizing large-scale confidential container clusters, it avoids the performance loss caused by repeatedly reading and decrypting the same data from external storage systems. It improves storage access efficiency in scenarios such as model training and inference, and becomes an important part of building a large-scale confidential container service infrastructure.

[0059] All nodes in the distributed cache cluster are mutually verified, which ensures the security of the entire cluster. At the same time, users of the cluster can confirm the security status of the entire cluster with one verification, which greatly facilitates the trusted configuration and trusted mounting of the subsequent cache system and realizes a zero-trust node dynamic expansion mechanism.

[0060] Design a FUSE client on the container side and automatically establish a trusted connection based on remote proof, so that confidential containers can efficiently access cached data without modification, achieving seamless data access. Attached Figure Description

[0061] Figure 1 A trusted architecture based on confidential containers for existing technologies;

[0062] Figure 2 This is an overall architecture diagram of the distributed caching system for confidential computing container clusters of this invention;

[0063] Figure 3 This is a diagram of the zero-trust confidential virtual machine cluster management framework in the distributed caching system for confidential computing container clusters of this invention;

[0064] Figure 4 This is a diagram of the zero-trust distributed caching system architecture in the distributed caching system for confidential computing container clusters of this invention;

[0065] Figure 5 The diagram illustrates the principle of accelerating model file reading by deploying large-scale distributed inference using the system of this invention;

[0066] Figure 6 This diagram illustrates the principle of accelerating model file reading for deploying large-scale distributed training using the system of this invention. Detailed Implementation

[0067] like Figure 2 As shown, the distributed caching system for confidential computing container clusters provided by this invention mainly includes two parts: a zero-trust confidential virtual machine management system and a zero-trust distributed caching system.

[0068] The Zero Trust Confidential Virtual Machine Management System consists of three main parts: confidential virtual machine node images, a Kubernetes-compatible confidential virtual machine controller (CVM-controller), and a distributed cache cluster creation client. On the one hand, it provides highly reliable basic software for the creation of zero trust distributed cache systems, and on the other hand, it provides a highly secure and elastic operating environment that can be dynamically scaled.

[0069] The zero-trust distributed caching system includes functional components such as a data engine plugin, a distributed file system, and a trusted access service compatible with FUSE (Filesystem in Userspace). Leveraging the core capabilities of the aforementioned confidential virtual machine management framework, and by providing trusted configuration of the cache cluster based on remote proof, transparent encryption / decryption access to external storage, trusted data mounting based on remote proof, and flexible distributed dynamic scaling, it provides users with the same high concurrency, high throughput, and high efficiency experience as ordinary distributed caching clusters without compromising the high-security threat model of confidential computing. This reduces the number of external data accesses and encryption / decryption operations, improving the read and write efficiency of the confidential container cluster.

[0070] like Figure 3 As shown, this solution utilizes confidential computing to construct a zero-trust cluster execution environment, enabling the operation of components in the zero-trust distributed caching system to be trusted without requiring trust from any human entity, including system administrators and host maintenance personnel, while still ensuring data security. To achieve this goal, this solution includes the following components:

[0071] 1) High-Security Cache Cluster Node Design: A confidential virtual machine is used as the underlying runtime environment for the nodes, implementing basic capabilities such as virtual machine memory encryption and CPU instruction isolation. Through a combination of technologies including a streamlined virtual machine operating system, kernel hardening, secure boot, and forced data erasure, a minimal environment supporting the operation of a distributed storage system is constructed, mitigating the security risks associated with excessively large confidential virtual machine TCBs. A memory file system is built and mounted within the confidential virtual machine, deploying all components of the zero-trust distributed cache system, including distributed cache components, key management components, initialization service components, hardened kernel, and firmware, to the memory file system. Based on this memory file system, images of all confidential virtual machine nodes of the distributed cache system are built, allowing the system to load all components of the zero-trust distributed cache system into the confidential virtual machine's memory file system and run them during system startup. This ensures that all programs are encrypted in memory and that all components can be directly measured by confidential computing hardware. A transparent encrypted persistent local disk is provided, providing a secure storage foundation for designing multi-level caching mechanisms. A secure node image of the distributed system is built based on this node for the deployment of the zero-trust distributed cache system.

[0072] 2) Kubernetes-compatible confidential virtual machine scheduling and management implementation: By extending the Kubernetes (K8s) CRD mechanism, confidential virtual machines are abstracted as native K8s resource types. The K8s scheduler is reused to achieve hybrid deployment of confidential containers and confidential virtual machines, such as... Figure 3 As shown, this enables the hybrid scheduling and deployment of confidential containers and confidential virtual machines, as well as hybrid elastic scaling, within the same Kubernetes cluster.

[0073] Hybrid scheduling allows distributed cache nodes to be scheduled closer to the read end, reducing network latency, and can also be scheduled to nodes with idle compute and storage resources, improving resource utilization. Simultaneously, the confidential virtual machine will be connected to the Kubernetes CNI network, placing the distributed cache cluster and the confidential container cluster on the same overlay network, further reducing network latency and improving the performance of the distributed cache system.

[0074] 3) Design of a trusted node extension protocol based on remote proof: such as Figure 3As shown, each node has an initialization service. When a new node starts up, it actively connects to an existing distributed cache cluster through its built-in initialization service. If the connection is successful, the new node submits an integrity measurement report generated by its hardware trusted execution environment to the cluster for remote verification. Since this scheme provides full component measurement capabilities in the node design, the cluster can determine whether the node is a legitimate distributed cache node based on the measurement values ​​in the hardware report. If the node is proven legitimate, the new node will obtain a certificate for legitimate communication issued by the cluster. The new node then establishes an encrypted channel with the cluster using this certificate, becoming a member of the distributed cache cluster. If the node is the first node, it waits for the cluster initialization request to establish a new distributed cluster.

[0075] Through the node extension protocol described above, all nodes in the distributed cache cluster are mutually verified, which not only ensures the security of the entire cluster, but also allows users of the cluster to confirm the security status of the entire cluster with a single verification, providing great convenience for the subsequent trusted configuration and trusted mounting of the cache system.

[0076] 4) Zero-Trust Key Management Service Implementation: Each node has a key management component, which mainly provides keys for communication between nodes in the distributed caching system and for data encryption on disk. To ensure the zero-trust requirement of the distributed caching system, i.e., to prevent the key from being known by any human entity throughout its entire lifecycle, this solution designs two management mechanisms: sealing key and temporary key.

[0077] On confidential virtual machine platforms with a Sealing Key mechanism (such as CSV), all keys are derived from the sealing key. Since the sealing key is derived from the hardware root of trust and various metrics, it cannot be obtained manually, thus ensuring key security.

[0078] Of course, Intel TDX and AMD SEV-SNP do not currently support sealing keys. This solution adopts a temporary key management method, that is, all keys are temporarily generated in memory and not saved to disk. By designing based on the Raft protocol to achieve multi-replica consistency, the temporary keys are synchronized to all trusted nodes in the cluster, thereby ensuring that keys can be recovered through a majority of nodes if lost.

[0079] The zero-trust distributed caching system runs on a confidential virtual machine cluster, and its system integrity is verified by confidential computing hardware metrics. For end users, the system presents itself as a highly trusted distributed file system, providing highly available, scalable, and high-throughput file read / write services. To ensure data security without compromising the threat model of confidential containers, this solution incorporates security enhancements in internal implementations such as external data synchronization and confidential container access.

[0080] Specifically, such as Figure 4 As shown, the distributed caching components of the zero-trust distributed caching system include data policy service, data engine plugin, distributed file system, and trusted access service.

[0081] 1) Data Policy Service: Used to establish an encrypted communication link between the distributed cache cluster and the cache management terminal based on hardware remote proof, and to receive data access policies uploaded by the cache management terminal through the encrypted communication link.

[0082] The cache management interface is used for users to set and manage data access policies. The data access policy includes the data source address for data access, data source access authentication information, data decryption key, and a whitelist of confidential containers that are allowed to access the data source.

[0083] 3) Data Engine Plugin: This includes plugins for connecting to various external storage systems, featuring transparent encryption and decryption of multi-source data, and achieving data synchronization with external storage based on user-configured data policies. In read scenarios, the plugin acquires encrypted data, decrypts it, and then stores it in shards to a distributed cache cluster; in write scenarios, it encrypts the corresponding data and transmits it to the external storage source.

[0084] 4) Distributed File System: This solution employs a multi-level caching distributed file system, providing metadata caching and data shard read / write caching capabilities, and utilizing memory and SSDs to provide multi-level caching functionality. For data read caching, read data is sharded and cached across multiple nodes, with automatic adjustment of shard distribution based on business load. A well-designed cache data replacement algorithm optimizes cache hit rate and read performance. For data write caching, a multi-replica write cache mechanism is designed to achieve high-performance asynchronous / synchronous write-back capabilities.

[0085] 5) Trusted access service for confidential containers: Unlike traditional distributed file systems, this solution enforces user-defined data access policies through trusted access. It remotely verifies that the metric value of the confidential container that needs to read data is in the policy whitelist before allowing the confidential container application to establish a connection with the distributed file system to read data.

[0086] In addition, this system includes a FUSE client deployed on the confidential container. This implementation provides a POSIX-compliant FUSE client, which automatically establishes a trusted connection between the confidential container and the distributed service based on remote authentication. The FUSE client provides local metadata and data caching capabilities, thereby reducing network requests and further improving data retrieval efficiency.

[0087] In this caching system, encrypted data stored externally is decrypted and cached in high-security memory built from a confidential virtual machine. Through trusted mounting based on remote authentication, confidential containers can securely and efficiently access the decrypted data directly. Furthermore, when multiple containers need to access data simultaneously, the system avoids the significant overhead of repeatedly fetching data from inefficient external storage and manually decrypting it. This achieves data decryption and caching only once, efficiently supplying multiple application containers simultaneously, resulting in a significant leap in overall performance.

[0088] Application scenario examples

[0089] Accelerated reading of large model inference model files, such as... Figure 5 As shown

[0090] When deploying large-model distributed inference, a zero-trust distributed caching system can be used to pre-warm and synchronize encrypted large model files from an external object storage system. These files are decrypted and saved to the caching system in advance. The caching system then fragments the model files and distributes them to nodes in the cluster to provide high-throughput data read capabilities. When a large number of containers of the distributed inference service start up, they can directly read the decrypted model data from multiple cache nodes in the distributed caching cluster. Leveraging the high throughput of the caching service and the proximity advantage of being on the same overlay network, each inference service container can enjoy efficient model file access.

[0091] When large-model distributed inference services face peak access and require rapid scaling, the system leverages Kubernetes' scheduling capabilities for confidential virtual machines and implements dynamic data sharding for rapid scaling of the caching system based on standard Kubernetes scheduling. This quickly provides the caching system with high-throughput service capabilities for the corresponding model files and supports rapid scaling of inference container instances.

[0092] Large model training checkpoint read / write acceleration, such as Figure 6 As shown

[0093] During large model training, temporary data (checkpoints) needs to be saved periodically to prevent training node failures and to quickly resume training. The data saved at one time during large model training can reach hundreds of gigabytes. During the saving and reading of checkpoints, the GPU is usually idle. Therefore, improving the efficiency of checkpoint reading and writing is crucial to improving GPU utilization.

[0094] like Figure 6As shown, the distributed caching system designed using this solution can synchronously and quickly write checkpoint data to the memory file system (RamFS) in the confidential computing node. This ensures high throughput and low latency almost identical to those in a normal environment, thereby quickly completing data write recovery training. The data is then gradually cached in blocks on encrypted SSDs, referred to as encrypted disks, and finally uploaded concurrently to external object storage for persistent storage. This fully utilizes the storage performance of all parties and securely and efficiently completes data storage in a confidential environment.

[0095] When it is necessary to restore data using checkpoints, loading speed can be improved by using a loading process similar to loading an inference model.

[0096] As can be seen from the above, this solution utilizes a confidential virtual machine to build a highly secure distributed caching system. Encrypted data in external storage is decrypted and temporarily stored in this system. With the efficient memory encryption and security isolation capabilities of the confidential virtual machine, the decrypted data can still be protected at the confidential virtual machine level. Furthermore, confidential containers that need to use the data can directly obtain plaintext data from the caching system, thereby avoiding the performance loss caused by repeatedly reading and decrypting the same data from the external storage system in a large-scale confidential container cluster.

[0097] The specific embodiments described herein are merely illustrative of the spirit of the invention. Those skilled in the art to which this invention pertains may make various modifications or additions to the described specific embodiments or use similar methods to substitute them, without departing from the spirit of the invention or exceeding the scope defined by the appended claims.

[0098] Although this paper frequently uses terms such as confidential container cluster, confidential virtual machine cluster, zero-trust confidential virtual machine management system, confidential virtual machine node image, CVM controller, distributed cache cluster creation client, zero-trust distributed cache system, distributed cache component, data policy management service, data engine plugin, distributed file system, trusted access service, key management component, and initialization service component, the possibility of using other terms is not excluded. The use of these terms is merely for the convenience of describing and explaining the essence of this invention; interpreting them as any additional limitation would contradict the spirit of this invention.

Claims

1. A distributed caching system for confidential computing container clusters, characterized in that, include: A confidential virtual machine cluster consists of multiple confidential virtual machine nodes, which are built on a hardware trusted execution environment. A zero-trust distributed caching system is loaded into the memory file system of a confidential virtual machine cluster to build a distributed caching cluster; The zero-trust distributed caching system includes data policy services, data engine plugins, a distributed file system, and trusted access services. The data policy service is used to establish an encrypted communication link between the distributed cache cluster and the cache management terminal based on hardware remote proof, and to receive the data access policy uploaded by the cache management terminal through the encrypted communication link. The cache management terminal is used for users to set data access policies, which include external data source address, data source access authentication information, data decryption key, and a whitelist of confidential containers that are allowed to access the current data source. The data engine plugin is used to obtain encrypted data from external data sources and decrypt it according to the data access strategy, cache the decrypted data to the distributed file system, and encrypt the data to be written and save it to external storage. A distributed file system is used to caching decrypted data in fragments across multiple cache nodes; Trusted access service is used to verify the legitimacy of the confidential container's identity through remote authentication and the data access policy. The FUSE client, deployed on the confidential container, establishes a secure connection with the trusted access service based on remote authentication and the corresponding confidential container's identity legitimacy verification, in order to provide plaintext data access services to the corresponding confidential container.

2. The distributed caching system for confidential computing container clusters according to claim 1, characterized in that, The distributed file system provides metadata caching and data shard read / write caching capabilities, and utilizes encrypted memory and SSD encrypted disks to provide multi-level caching functionality.

3. The distributed caching system for confidential computing container clusters according to claim 2, characterized in that, Deploy the confidential virtual machine node as follows: Build a memory file system that runs on a confidential virtual machine; All components of the zero-trust distributed caching system, including the distributed caching component, key management component, and initialization service component, are deployed to the aforementioned in-memory file system; Based on the aforementioned memory file system, construct images of all confidential virtual machine nodes of the distributed caching system; The confidential virtual machine node is deployed using the aforementioned confidential virtual machine node image.

4. The distributed caching system for confidential computing container clusters according to claim 3, characterized in that, The distributed caching component includes the data policy management service, data engine plugin, distributed file system, and trusted access service. The key management component is used to manage system keys, including inter-node communication keys and data storage keys.

5. The distributed caching system for confidential computing container clusters according to claim 4, characterized in that, The key management component is used to derive keys based on hardware trust roots; Alternatively, a temporary key can be generated based on encrypted memory, and the temporary key can be synchronized to all trusted nodes in the cluster via the Raft protocol.

6. The distributed caching system for confidential computing container clusters according to claim 3, characterized in that, The initialization service component is used for, When the current node starts, it attempts to establish a connection with an existing distributed cache cluster. Upon successful connection, submit a hardware remote proof report to the distributed cache cluster; Does the full component metric in the distributed cache cluster verification report conform to the preset strategy? After successful verification, receive the communication certificate issued by the distributed caching cluster; Based on the aforementioned communication certificate, establish communication with the existing distributed cache cluster, join the cluster, and synchronize the key and data fragmentation information.

7. The distributed caching system for confidential computing container clusters according to claim 3, characterized in that, This system also includes a zero-trust confidential virtual machine management system, including the aforementioned confidential virtual machine node image and confidential virtual machine controller; Furthermore, the confidential virtual machine controller is compatible with Kubernetes in the following ways: By using Kubernetes' CRD extension mechanism, confidential virtual machines are abstracted into native Kubernetes resources, enabling the reuse of the Kubernetes scheduler to achieve hybrid deployment of confidential containers and cache nodes; The confidential virtual machine is connected to the Kubernetes CNI network, so that the cache cluster and the container cluster are in the same overlay virtual network layer.

8. A distributed caching method for confidential computing container clusters, characterized in that, The implementation of the distributed caching system for confidential computing container clusters based on any one of claims 1 to 7 includes, Build a confidential virtual machine cluster based on a hardware trusted execution environment; The zero-trust distributed caching system runs on the aforementioned confidential virtual machine cluster after being verified by hardware metrics in a hardware trusted execution environment. An encrypted communication link with the cache management terminal is established based on hardware remote authentication; The data access policy uploaded by the cache management terminal is received through the encrypted communication link. The data access policy includes the external data source address, data source access authentication information, data decryption key, and a whitelist of confidential containers that are allowed to access the current data source. Based on the aforementioned data access strategy, encrypted data is obtained from and decrypted from external data sources. The decrypted data is fragmented and cached across multiple cache nodes; The identity of the confidential container is confirmed through remote authentication and the aforementioned data access policy; A secure connection is established between the FUSE client deployed on the confidential container and the trusted access service based on remote authentication and corresponding confidential container identity legitimacy verification, so as to provide the corresponding confidential container with plaintext data access service cached on each cache node.

9. The distributed caching method for confidential computing container clusters according to claim 8, characterized in that, This method also includes, The first node starts up, responding to the cluster initialization request and establishing a new distributed cache cluster; The new node starts up and attempts to establish a connection with the existing distributed cache cluster. Upon successful connection, submit a hardware remote proof report to the distributed cache cluster; Does the full component metric in the distributed cache cluster verification report conform to the preset strategy? After successful verification, receive the communication certificate issued by the distributed caching cluster; Based on the aforementioned communication certificate, establish communication with the existing distributed cache cluster, join the cluster, and synchronize the key and data fragmentation information.

10. An acceleration method for large model inference services, characterized in that, Implemented using the distributed caching system for confidential computing container clusters as described in any one of claims 1 to 7, comprising: The encrypted model file is obtained from the external data source address according to the data access policy, and after decryption, it is stored in fragments to multiple cache nodes of the distributed cache cluster. After identity verification and remote authentication, the confidential container establishes a secure connection with the distributed cache cluster. The confidential container reads decrypted model file fragments from a nearby cache node; Based on Kubernetes' ability to schedule confidential virtual machines, a cache node expansion command is triggered when expansion is needed. The new cache node joins the cluster based on cluster-level remote authentication, synchronizing the key and data shard information.

Images