Distributed caching method and system and acceleration method for large model reasoning service

Abstract

The invention discloses a distributed caching method and system and an acceleration method for large model reasoning service. The system comprises a confidential virtual machine cluster, a zero-trust distributed caching system and a FUSE client. The zero-trust distributed cache system comprises a data strategy service, a data engine plug-in, a distributed file system and a trusted access service. And the FUSE client is deployed at the confidential container end, and establishes secure connection with the trusted access service based on the remote proof and the identity legality confirmation proof of the corresponding confidential container so as to provide a plaintext data access service for the corresponding confidential container. According to the scheme, a distributed cache implementation scheme is provided for the confidential computing container cluster, and the problem of performance loss caused by repeatedly reading, encrypting and decrypting the same data from an external storage system is avoided while the data security is guaranteed by using the large-scale confidential container cluster; and the storage access efficiency in scenes such as model training and reasoning is improved.

Description

Technical Field

[0001] The present invention belongs to the field of confidential computing containers, and in particular relates to a distributed caching method, system, and acceleration method for large-model inference services. Background Art

[0002] The current explosive growth of large models poses unprecedented challenges to the underlying computing infrastructure. In both training and inference scenarios, efficient data access and transmission have become a core bottleneck restricting model iteration speed and resource utilization. Traditional storage systems often struggle to meet the stringent requirements of large models for high data throughput, large capacity, and ecosystem interoperability.

[0003] At the same time, in order to meet privacy computing and compliance requirements, the confidential computing architecture based on Confidential Containers (CoCo) has gradually become one of the preferred solutions for large-scale model multi-party joint training and sensitive data reasoning deployment due to its advantages in data security isolation and computing efficiency. Figure 1 As shown, to ensure data security, this technology requires remote attestation to obtain a key each time data is loaded or written, and then encrypts and decrypts the data in a confidential environment. This not only increases data loading latency, but also makes traditional data caching services unsuitable due to security logic flaws. Summary of the Invention

[0004] To address the aforementioned issues, the present invention aims to provide a distributed caching method and system for confidential computing container clusters, achieving the parallel compatibility of secure isolation and high data access efficiency in large-scale confidential container cluster applications. Another object of the present invention is to provide an acceleration method for large-model inference services, implemented using the aforementioned distributed caching system.

[0005] In order to achieve the above object, the present invention adopts the following technical solutions: A distributed caching system for confidential computing container clusters, including: A confidential virtual machine cluster, consisting of multiple confidential virtual machine nodes, each of which is built based on a hardware trusted execution environment; A zero-trust distributed cache system is loaded into the memory file system of the confidential virtual machine cluster to build a distributed cache cluster; The zero-trust distributed cache system includes data policy services, data engine plug-ins, distributed file systems, and trusted access services; Data policy service, used to establish an encrypted communication link between the distributed cache cluster and the cache management end based on hardware remote attestation, and receive data access policies uploaded by the cache management end through the encrypted communication link; The cache management terminal is used for users to set data access policies, which include external data source addresses, data source access authentication information, data decryption keys, and a whitelist of confidential containers that are allowed to access the current data source; A data engine plug-in is used to obtain encrypted data from an external data source and decrypt it according to the data access strategy, cache the decrypted data in the distributed file system, and encrypt the data to be written and save it in external storage; Distributed file system, used to cache decrypted data fragments to multiple cache nodes; Trusted access service, used to confirm the legitimacy of the confidential container identity through remote attestation and the data access policy; The FUSE client is deployed on the confidential container side and establishes a secure connection with the trusted access service based on remote attestation and the corresponding confidential container identity legitimacy confirmation to provide plaintext data access services to the corresponding confidential container.

[0006] In the above-mentioned distributed caching system for confidential computing container clusters, the distributed file system provides metadata caching and data shard read and write caching capabilities, and uses encrypted memory and SSD encrypted disks to provide multi-level caching functions.

[0007] In the above-mentioned distributed cache system for confidential computing container clusters, the confidential virtual machine nodes are deployed in the following manner: Build an in-memory file system running on a confidential virtual machine; All components of the zero-trust distributed cache system, including the distributed cache component, the key management component, and the initialization service component, are deployed to the memory file system; Building all confidential virtual machine node images of the distributed cache system based on the memory file system; Deploy the confidential virtual machine node using the confidential virtual machine node image.

[0008] In the above-mentioned distributed cache system for confidential computing container clusters, the distributed cache components include the data policy management service, data engine plug-in, distributed file system, and trusted access service; The key management component is used to manage system keys including inter-node communication keys and data storage keys.

[0009] In the above-mentioned distributed caching system for confidential computing container clusters, the key management component is used to derive keys based on the hardware trust root; Alternatively, generate a temporary key based on encrypted memory and synchronize the temporary key to all trusted nodes in the cluster through the Raft protocol.

[0010] In the above-mentioned distributed cache system for confidential computing container clusters, the initialization service component is used to: When the current node starts, it attempts to establish a connection with an existing distributed cache cluster; In response to a successful connection, submitting a hardware remote attestation report to the distributed cache cluster; Whether the measurement values ​​of all components in the distributed cache cluster verification report comply with the preset strategy; After verification, receive the communication certificate issued by the distributed cache cluster; Based on the communication certificate, establish communication with the existing distributed cache cluster and join the cluster, synchronize the key and data sharding information.

[0011] In the above-mentioned distributed cache system for confidential computing container clusters, the system also includes a zero-trust confidential virtual machine management system, including the confidential virtual machine node image and confidential virtual machine controller; The confidential VM controller is compatible with Kubernetes in the following ways: Confidential VMs are abstracted into Kubernetes native resources through the Kubernetes CRD extension mechanism, allowing the Kubernetes scheduler to be reused to achieve hybrid deployment of confidential containers and cache nodes. The confidential VM is connected to the Kubernetes CNI network, so that the cache cluster and container cluster are in the same overlay virtual network layer.

[0012] A distributed caching method for confidential computing container clusters is implemented based on the distributed caching system for confidential computing container clusters, including: Build confidential virtual machine clusters based on hardware trusted execution environments; The zero-trust distributed cache system runs on the confidential virtual machine cluster after hardware measurement verification of the hardware trusted execution environment; Establish an encrypted communication link with the cache management end based on hardware remote attestation; Receive a data access policy uploaded by the cache management terminal through the encrypted communication link, wherein the data access policy includes an external data source address, data source access authentication information, a data decryption key, and a whitelist of confidential containers allowed to access the current data source; Obtaining encrypted data from an external data source based on the data access strategy and decrypting the data; Cache the decrypted data into shards on multiple cache nodes; Confirm the legitimacy of the confidential container identity through remote attestation and the data access policy; Based on remote attestation and the corresponding confidential container identity legitimacy confirmation proof, a secure connection is established between the FUSE client deployed on the confidential container side and the trusted access service to provide the corresponding confidential container with plaintext data access services cached in each cache node.

[0013] In the above-mentioned distributed caching method for confidential computing container clusters, the method further includes: The first node starts up and responds to the cluster initialization request to establish a new distributed cache cluster; The new node starts and attempts to establish a connection with the existing distributed cache cluster; In response to a successful connection, submitting a hardware remote attestation report to the distributed cache cluster; Whether the measurement values ​​of all components in the distributed cache cluster verification report comply with the preset strategy; After verification, receive the communication certificate issued by the distributed cache cluster; Based on the communication certificate, establish communication with the existing distributed cache cluster and join the cluster, synchronize the key and data sharding information.

[0014] A method for accelerating large-model inference services, implemented using the distributed cache system for confidential computing container clusters, includes: Obtain the encrypted model file from the external data source address according to the data access strategy, decrypt it and store it in shards on multiple cache nodes in the distributed cache cluster; After identity legitimacy confirmation and remote attestation, the confidential container establishes a secure connection with the distributed cache cluster; The confidential container reads the decrypted model file shard from the neighboring cache node; Based on Kubernetes's scheduling capabilities for confidential virtual machines, cache node expansion instructions are triggered when capacity expansion is required. New cache nodes join the cluster based on cluster-level remote attestation and synchronize keys and data sharding information.

[0015] The advantages of the present invention are: This solution provides a distributed cache implementation for confidential computing container clusters. While utilizing large-scale confidential container clusters to ensure data security, it also avoids the performance loss caused by repeatedly reading and encrypting the same data from external storage systems. This improves storage access efficiency in scenarios such as model training and inference, becoming a key component in building a large-scale confidential container service infrastructure. All nodes in the distributed cache cluster are mutually certified, ensuring the security of the entire cluster. Cluster users can also confirm the security status of the entire cluster with a single certification, greatly facilitating the trusted configuration and trusted mounting of subsequent cache systems and enabling a dynamic zero-trust node expansion mechanism. A FUSE client is designed on the container side, and a trusted connection is automatically established based on remote attestation, allowing confidential containers to efficiently access cached data without modification, achieving seamless data access. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 Provide a trusted architecture based on confidential containers for existing technologies; Figure 2 This is a diagram of the overall architecture of the distributed cache system for confidential computing container clusters of the present invention; Figure 3 This is a zero-trust confidential virtual machine cluster management framework diagram in the distributed cache system for confidential computing container clusters of the present invention; Figure 4 This is a diagram of the zero-trust distributed cache system architecture in the distributed cache system for confidential computing container clusters of the present invention; Figure 5 Schematic diagram of accelerating model file reading by deploying large model distributed reasoning using the system of the present invention; Figure 6 This is a schematic diagram of the model file reading acceleration principle for deploying large-model distributed training using the system of the present invention. DETAILED DESCRIPTION

[0017] like Figure 2 As shown, the distributed cache system for confidential computing container clusters provided by the present invention mainly includes two parts: a zero-trust confidential virtual machine management system and a zero-trust distributed cache system.

[0018] The zero-trust confidential virtual machine management system consists of three parts: confidential virtual machine node image, k8s-compatible confidential virtual machine controller (CVM-controller) and distributed cache cluster creation client. On the one hand, it provides highly reliable basic software for the creation of zero-trust distributed cache system, and on the other hand, it provides a highly secure and elastic operating environment that can be dynamically expanded and contracted.

[0019] The zero-trust distributed cache system includes functional components such as a data engine plug-in, a distributed file system, and a FUSE (Filesystem in Userspace)-compatible trusted access service. Leveraging the core capabilities of the aforementioned confidential virtual machine management framework, and providing remote attestation-based trusted configuration of cache clusters, transparent encryption and decryption access to external storage, trusted data mounting based on remote attestation, and flexible distributed dynamic capacity expansion, it provides users with the same high concurrency, high throughput, and high-efficiency experience as a standard distributed cache cluster, without compromising the high-security threat model of confidential computing. This reduces external data access and encryption and decryption times, improving the efficiency of reading and writing data within the confidential container cluster.

[0020] like Figure 3 As shown in the figure, this solution uses confidential computing to build a zero-trust cluster execution environment, allowing the operation of zero-trust distributed cache system components without trusting any human entities, including system administrators, host operation and maintenance personnel, while also ensuring data security. To achieve this goal, this solution includes the following parts: 1) High-Security Cache Cluster Node Design: Confidential VMs are used as the node's base operating environment, implementing fundamental capabilities such as VM memory encryption and CPU instruction isolation. By streamlining the VM operating system, implementing kernel hardening, secure boot, and mandatory data erasure, a minimal environment is constructed to support the operation of the distributed storage system, mitigating the security risks associated with the excessively large TCB of confidential VMs. An in-memory file system is built and mounted within the confidential VM. All components of the zero-trust distributed cache system, including the distributed cache component, key management component, initialization service component, hardened kernel, and firmware, are deployed to the in-memory file system. Based on the in-memory file system, all confidential VM node images for the distributed cache system are constructed. Upon system startup, the confidential VM node image loads all components of the zero-trust distributed cache system into the confidential VM's in-memory file system and runs them. This ensures that all programs are encrypted in memory and that all components are directly measurable by the confidential computing hardware. Transparent encrypted persistent local disks are provided, providing a secure storage foundation for the design of a multi-level caching mechanism. Secure node images of the distributed system are constructed based on this node for zero-trust distributed cache system deployment.

[0021] 2) Implementation of confidential VM scheduling and management compatible with Kubernetes: By extending the CRD mechanism of Kubernetes (K8s), confidential VMs are abstracted as K8s native resource types, and the K8s scheduler is reused to achieve mixed deployment of confidential containers and confidential VMs, such as Figure 3 As shown in , hybrid scheduling and deployment of confidential containers and confidential virtual machines, as well as hybrid elastic scaling, can be achieved in the same k8s cluster.

[0022] Hybrid scheduling allows distributed cache nodes to be scheduled closer to the read end, reducing network latency. It also allows distributed cache nodes to be scheduled to nodes with idle compute and storage resources, improving resource utilization. Furthermore, confidential VMs are connected to the Kubernetes CNI network, allowing the distributed cache cluster and confidential container cluster to reside on the same overlay network, further reducing network latency and improving distributed cache system performance.

[0023] 3) Design of trusted node extension protocol based on remote attestation: Figure 3As shown, each node has an initialization service. When a new node starts, it actively connects to the existing distributed cache cluster through its built-in initialization service. If the connection is successful, the new node submits the integrity measurement report generated by its hardware trusted execution environment to the cluster for remote attestation by the existing cluster. Because this solution provides full component measurement capabilities in the node design, the cluster can determine whether the node is a legitimate distributed cache node based on the measurement values ​​in the hardware report. If the node is legitimate, the new node will obtain a certificate for legitimate communication issued by the cluster. The new node uses the certificate to establish an encrypted channel with the cluster and become a member of the distributed cache cluster. If the node is the first node, it waits for the cluster initialization request and establishes a new distributed cluster.

[0024] Through the above-mentioned node extension protocol, all nodes in the distributed cache cluster are mutually certified, which not only ensures the security of the entire cluster, but also allows users of the cluster to confirm the security status of the entire cluster through a single certification, providing great convenience for the subsequent trusted configuration and trusted mounting of the cache system.

[0025] 4) Zero-Trust Key Management Service Implementation: Each node has a key management component that provides keys for inter-node communication and data encryption in the distributed cache system. To ensure the zero-trust requirements of the distributed cache system, which prevents keys from being known to any human entity throughout their lifecycle, this solution designs two management mechanisms: sealing keys and temporary keys. On confidential virtual machine platforms with a sealing key mechanism (such as CSV), all keys are derived from the sealing key. Since the sealing key is derived from the hardware trust root and various measurement values, it cannot be obtained manually, thus ensuring key security.

[0026] Of course, Intel TDX and AMD SEV-SNP currently do not support sealing keys. This solution uses a temporary key management method, that is, all keys are temporarily generated in memory instead of being stored on disk. The design is based on the Raft protocol to achieve multi-copy consistency and synchronize temporary keys to all trusted nodes in the cluster, ensuring that the key can be recovered by the majority of nodes if it is lost.

[0027] The zero-trust distributed caching system runs on a cluster of confidential virtual machines, its system integrity verified by confidential computing hardware metrics. To end users, the system appears as a highly trusted distributed file system, providing highly available, scalable, and high-throughput file read and write services. To ensure data security while maintaining the threat model of confidential containers, this solution incorporates security enhancements in internal implementations such as external data synchronization and confidential container access.

[0028] Specifically, if Figure 4 As shown in the figure, the distributed cache components of the zero-trust distributed cache system include data policy services, data engine plug-ins, distributed file systems, and trusted access services.

[0029] 1) Data Policy Service: Used to establish an encrypted communication link between the distributed cache cluster and the cache management end based on hardware remote attestation, and receive data access policies uploaded by the cache management end through the encrypted communication link.

[0030] The cache management end is used for users to set data access policies and manage the data access policies set by users. The data access policy includes the data source address for data access, data source access authentication information, data decryption key and the confidential container whitelist allowed to access the data source.

[0031] 3) Data Engine Plugin: This plugin includes plugins for connecting to various external storage systems. It features transparent encryption and decryption of multi-source data and synchronizes data with external storage based on user-configured data policies. In read scenarios, the plugin retrieves encrypted data, decrypts it, and then stores it in shards in a distributed cache cluster. In write scenarios, the plugin encrypts the data and transmits it to the external storage source.

[0032] 4) Distributed File System: This solution utilizes a multi-level cache distributed file system, providing metadata caching and data shard read / write caching capabilities. It also leverages memory and SSDs to provide multi-level caching. For data read caching, read data shards are cached across multiple nodes, with automatic shard distribution adjusted based on business load. A well-designed cache data replacement algorithm optimizes cache hit rate and read performance. For data write caching, a multi-replica write cache mechanism is designed to achieve high-performance asynchronous / synchronous write-back capabilities.

[0033] 5) Trusted access service for confidential containers: Unlike traditional distributed file systems, this solution enforces user-defined data access policies through trusted access. It uses remote attestation to confirm that the measurement value of the confidential container that needs to read data is in the policy whitelist before allowing the confidential container application to establish a connection with the distributed file system to read data.

[0034] The system also includes a FUSE client deployed on the confidential container side. This implementation provides a POSIX-compatible FUSE client that automatically establishes a trusted connection between the confidential container and the distributed service based on remote attestation. The FUSE client provides local metadata and data caching capabilities, reducing network requests and further improving data access efficiency.

[0035] This caching system decrypts and caches encrypted data from external storage in highly secure memory built by confidential virtual machines. Through trusted mounting based on remote attestation, confidential containers can securely and efficiently access decrypted data directly. Furthermore, when a large number of different containers need to access data simultaneously, the system avoids the significant overhead of repeatedly fetching data from inefficient external storage and decrypting it themselves. Instead, data is decrypted and cached once, efficiently serving multiple application containers simultaneously, resulting in a significant improvement in overall performance.

[0036] Application scenario examples Accelerate the reading of large model inference model files, such as Figure 5 shown When deploying large-model distributed inference, a zero-trust distributed cache system can be used to pre-sync encrypted large model files from external object storage systems, decrypting them and pre-storing them in the cache system. The cache system then shards the model files and distributes them across the cluster's nodes, providing high-throughput data read capabilities. When launching large batches of distributed inference service containers, the decrypted model data can be directly read from multiple cache nodes in the distributed cache cluster. Leveraging the cache service's high throughput and proximity to the same overlay network, each inference service container can access the model file efficiently.

[0037] When large-model distributed inference services face access peaks and need to be rapidly expanded, relying on K8s's scheduling capabilities for confidential virtual machines, the cache system's dynamic data sharding can be rapidly expanded based on standard K8s scheduling, quickly providing the cache system with high-throughput service capabilities for the corresponding model files and supporting the rapid expansion of inference container instances.

[0038] Large model training checkpoint reading and writing acceleration, such as Figure 6 shown When training large models, to prevent training node failures, temporary data (checkpoints) need to be periodically saved for rapid training recovery. During large model training, the amount of data saved at a time can reach hundreds of GB, and the GPU is usually idle during checkpoint saving and reading. Therefore, improving the efficiency of checkpoint reading and writing is crucial to improving GPU utilization.

[0039] like Figure 6As shown, using the distributed cache system designed by the present scheme, checkpoint data can be written into the memory file system (RamFS) in the confidential computing node in a synchronous and fast manner, which can guarantee almost consistent high throughput and low latency with the general environment, thereby quickly completing the data write recovery training, gradually caching the data to the encrypted SSD hard disk, referred to as encrypted disk, and finally persisting in the external object storage through concurrent uploading, which can fully utilize the storage performance of each party and safely and efficiently complete the data saving in the confidential environment.

[0040] When it is necessary to recover data by using checkpoint, the loading speed can be improved by using a loading process similar to the inference model loading.

[0041] As can be seen from the above, the present scheme builds a distributed cache system with high security characteristics by using a confidential virtual machine, temporarily stores the encrypted data in the external storage in the system after decryption, and takes advantage of the efficient memory encryption capability and security isolation capability of the confidential virtual machine. On the one hand, the decrypted data can still be protected at the level of the confidential virtual machine, and on the other hand, the confidential containers that need to use the data can directly obtain the plaintext data from the cache system, thereby avoiding the performance loss problem caused by repeated reading and encryption and decryption of the same data from the external storage system in a large-scale confidential container cluster.

[0042] The specific embodiments described herein are merely illustrative of the spirit of the present application. Those skilled in the art of the present application can make various modifications or supplements to the described specific embodiments or use similar ways to replace them, without deviating from the spirit of the present application or exceeding the scope defined by the appended claims.

[0043] Although the terms such as confidential container cluster, confidential virtual machine cluster, zero-trust confidential virtual machine management system, confidential virtual machine node image, CVM-controller, distributed cache cluster creation client, zero-trust distributed cache system, distributed cache component, data policy management service, data engine plug-in, distributed file system, trusted access service, secret key management component, and initialization service component are used more frequently herein, the possibility of using other terms is not excluded. The use of these terms is only to facilitate the description and explanation of the essence of the present application; any interpretation of them as any kind of additional limitation is contrary to the spirit of the present application.

Claims

1. A distributed caching system for confidential computing container clusters, characterized by: include: A confidential virtual machine cluster, consisting of multiple confidential virtual machine nodes, each of which is built based on a hardware trusted execution environment; A zero-trust distributed cache system is loaded into the memory file system of the confidential virtual machine cluster to build a distributed cache cluster; The zero-trust distributed cache system includes data policy services, data engine plug-ins, distributed file systems, and trusted access services; Data policy service, used to establish an encrypted communication link between the distributed cache cluster and the cache management terminal based on hardware remote attestation, and receive data access policies uploaded by the cache management terminal through the encrypted communication link; The cache management terminal is used for users to set data access policies, which include external data source addresses, data source access authentication information, data decryption keys, and a whitelist of confidential containers that are allowed to access the current data source; A data engine plug-in is used to obtain encrypted data from an external data source and decrypt it according to the data access strategy, cache the decrypted data in the distributed file system, and encrypt the data to be written and save it in external storage; Distributed file system, used to cache decrypted data fragments to multiple cache nodes; Trusted access service, used to confirm the legitimacy of the confidential container identity through remote attestation and the data access policy; The FUSE client is deployed on the confidential container side and establishes a secure connection with the trusted access service based on remote attestation and the corresponding confidential container identity legitimacy confirmation to provide plaintext data access services to the corresponding confidential container.

2. The distributed cache system for confidential computing container clusters according to claim 1, characterized in that The distributed file system provides metadata caching and data shard read and write caching capabilities, and uses encrypted memory and SSD encrypted disks to provide multi-level caching functions.

3. The distributed cache system for confidential computing container clusters according to claim 2, characterized in that: Deploy the confidential VM node as follows: Build an in-memory file system running on a confidential virtual machine; All components of the zero-trust distributed cache system, including the distributed cache component, the key management component, and the initialization service component, are deployed to the memory file system; Building all confidential virtual machine node images of the distributed cache system based on the memory file system; Deploy the confidential virtual machine node using the confidential virtual machine node image.

4. The distributed cache system for confidential computing container clusters according to claim 3, characterized in that The distributed cache component includes the data policy management service, data engine plug-in, distributed file system, and trusted access service; The key management component is used to manage system keys including inter-node communication keys and data storage keys.

5. The distributed cache system for confidential computing container clusters according to claim 4, characterized in that: The key management component is used to derive keys based on the hardware root of trust; Alternatively, generate a temporary key based on encrypted memory and synchronize the temporary key to all trusted nodes in the cluster through the Raft protocol.

6. The distributed cache system for confidential computing container clusters according to claim 3, characterized in that The initialization service component is used to: When the current node starts, it attempts to establish a connection with an existing distributed cache cluster; In response to a successful connection, submitting a hardware remote attestation report to the distributed cache cluster; Whether the measurement values ​​of all components in the distributed cache cluster verification report comply with the preset strategy; After verification, receive the communication certificate issued by the distributed cache cluster; Based on the communication certificate, establish communication with the existing distributed cache cluster and join the cluster, synchronize the key and data sharding information.

7. The distributed cache system for confidential computing container clusters according to claim 3, characterized in that The system also includes a zero-trust confidential virtual machine management system, including the confidential virtual machine node image and confidential virtual machine controller; The confidential VM controller is compatible with Kubernetes in the following ways: Confidential VMs are abstracted into Kubernetes native resources through the Kubernetes CRD extension mechanism, allowing the Kubernetes scheduler to be reused to achieve hybrid deployment of confidential containers and cache nodes. The confidential VM is connected to the Kubernetes CNI network, so that the cache cluster and container cluster are in the same overlay virtual network layer.

8. A distributed caching method for confidential computing container clusters, characterized in that: The distributed cache system for confidential computing container clusters according to any one of claims 1 to 7 is implemented, including: Build confidential virtual machine clusters based on hardware trusted execution environments; The zero-trust distributed cache system runs on the confidential virtual machine cluster after hardware measurement verification of the hardware trusted execution environment; Establish an encrypted communication link with the cache management end based on hardware remote attestation; Receive a data access policy uploaded by the cache management terminal through the encrypted communication link, wherein the data access policy includes an external data source address, data source access authentication information, a data decryption key, and a whitelist of confidential containers allowed to access the current data source; Obtaining encrypted data from an external data source based on the data access strategy and decrypting the data; Cache the decrypted data into shards on multiple cache nodes; Confirm the legitimacy of the confidential container identity through remote attestation and the data access policy; Based on remote attestation and the corresponding confidential container identity legitimacy confirmation proof, a secure connection is established between the FUSE client deployed on the confidential container side and the trusted access service to provide the corresponding confidential container with plaintext data access services cached in each cache node.

9. The distributed caching method for confidential computing container clusters according to claim 8, characterized in that: The method further comprises, The first node starts up and responds to the cluster initialization request to establish a new distributed cache cluster; The new node starts and attempts to establish a connection with the existing distributed cache cluster; In response to a successful connection, submitting a hardware remote attestation report to the distributed cache cluster; Whether the measurement values ​​of all components in the distributed cache cluster verification report comply with the preset strategy; After verification, receive the communication certificate issued by the distributed cache cluster; Based on the communication certificate, establish communication with the existing distributed cache cluster and join the cluster, synchronize the key and data sharding information.

10. An acceleration method for large model inference service, characterized in that: The distributed cache system for confidential computing container clusters according to any one of claims 1 to 7 is implemented, comprising: Obtain the encrypted model file from the external data source address according to the data access strategy, decrypt it and store it in shards on multiple cache nodes in the distributed cache cluster; After identity legitimacy confirmation and remote attestation, the confidential container establishes a secure connection with the distributed cache cluster; The confidential container reads the decrypted model file shard from the neighboring cache node; Based on Kubernetes's scheduling capabilities for confidential virtual machines, cache node expansion instructions are triggered when capacity expansion is required. New cache nodes join the cluster based on cluster-level remote attestation and synchronize keys and data sharding information.

Images