Secure communication systems and methods for GPU acceleration in trusted execution environments

Abstract

A system and method for secure execution of proprietary algorithms and secure communication between a secure enclave and a GPU, particularly suited for protecting sensitive data in Large Language Model (LLM) marketplaces. The system employs a trusted environment with an SGX enclave, a library operating system (LibOS), and a trusted initialization component to manage encrypted communication. A novel re-encryption process, performed in a separate execution context, further enhances security. The method involves establishing a virtualized connection, intercepting communications, encrypting data, and requesting re-encryption before GPU processing. A chained attestation strategy, with the GPU, trusted initialization component, and SGX enclave attesting to each other, ensures system integrity. This architecture protects models and data from unauthorized access, enabling secure GPU-accelerated computing while maintaining confidentiality and trustworthiness, promoting secure data processing in LLM marketplaces and various computing environments.

Description

[0001] SECURE COMMUNICATION SYSTEMS AND METHODS FOR GPU ACCELERATION IN TRUSTED EXECUTION ENVIRONMENTS

[0002] CROSS-REFERENCE TO RELATED APPLICATIONS

[0003] This application claims the benefit of priority from U.S. Provisional Patent Application No. 63 / 660,606, filed June 17, 2024 the contents of which are incorporated by reference in their entirety.

[0004] FIELD OF THE DISCLOSURE

[0005] The disclosure herein relates to systems and methods for secure multi-party computation. More specifically, the disclosure pertains to methods and platforms which use virtualization and re-encryption for GPU acceleration in trusted execution environments by securing communication with secure platforms such as Intel Software Guard Extensions (SGX) enabled environments.

[0006] BACKGROUND

[0007] The increasing demand for high-performance computing has led to the widespread adoption of Graphics Processing Units (GPUs) for accelerating computationally intensive tasks. GPUs offer significant performance advantages in various domains, including machine learning, scientific simulations, and data analytics.

[0008] However, the use of GPUs in cloud environments and other shared computing platforms raises significant security concerns. Sensitive data and proprietary algorithms, such as Large Language Models (LLMs) used for natural language processing or other models used to make predictions based upon input data, are often processed on these platforms, making them vulnerable to unauthorized access, modification, or theft.

[0009] Traditional security measures, such as software-based encryption and access controls, may not be sufficient to protect against sophisticated attacks, particularly those targeting the hardware or firmware level. Moreover, the reliance on the underlying cloud infrastructure and its vast number of software components introduces additional trust issues. Currently available LLM solutions are not truly confidential as they often require a user to trust the LLM provider with their sensitive information.

[0010] To address these concerns, Trusted Execution Environments (TEEs) have emerged as a promising solution. TEEs, such as Intel SGX, provide a secure, isolated environment for executing code and protecting sensitive data. SGX enclaves offer confidential execution of user-level applications, ensuring that the code and data within the enclave are protected from unauthorized access, even from privileged software or hardware. However, SGX enclaves cannot directly communicate with peripheral devices but rely on the kernel and its device drivers to do the actual communication.

[0011] While SGX provides a strong foundation for security, enabling secure communication between SGX enclaves and GPUs presents a unique set of challenges. Existing solutions often rely on complex software stacks and introduce significant performance overhead. Furthermore, they may not fully address the need for end-to-end data protection and attestation.

[0012] Therefore, there is a need for a more efficient and robust system and method for securing communications between SGX enclaves and GPUs, providing a secure platform for executing models and protecting sensitive data in GPU-accelerated computing platforms.

[0013] SUMMARY OF THE EMBODIMENTS

[0014] According to one aspect of the presently disclosed subject matter, a system is introduced for secure communication between a secure enclave and a GPU, addressing the need for enhanced security in computing environments where sensitive data is processed. The system achieves this by employing a combination of virtualization and re-encryption techniques to protect data transmitted between the secure enclave and the GPU. The system includes a device driver of the GPU. A library operating system (LibOS) is configured to intercept communications between the secure enclave and the GPU device driver. An encryption module within the LibOS encrypts outgoing communications from the secure enclave using a first cryptographic key. To further enhance security, a re-encryption request module within the device driver requests a reencryption of the encrypted communications using a second cryptographic key before transmission to the GPU. This re-encryption is performed by a separate execution context.

[0015] In some embodiments, the encryption module within the LibOS uses a symmetric encryption algorithm to provide efficient encryption of the data. The re-encryption request module within the device driver transmits a re-encryption request to the separate execution context via a virtual serial port, providing a secure and reliable communication channel.

[0016] The separate execution context may be a virtual machine, providing an isolated environment for performing the cryptographic operations. In some cases, the virtual machine is running in root mode, providing enhanced privileges for accessing system resources.

[0017] In one particular embodiment, the LibOS is Gramine, a lightweight library operating system designed for secure execution environments. The system may also include an attestation module configured to verify the integrity of the secure enclave, ensuring that it is operating in a trusted state.

[0018] Where appropriate, the secure enclave may be an Intel SGX enclave, providing a hardware-based security boundary for protecting sensitive code and data. The GPU supports confidential computing, enabling secure processing of data within the GPU itself.

[0019] According to another aspect of the invention, a comprehensive system is introduced for secure execution of proprietary algorithms, addressing the need for enhanced security and confidentiality in computing environments where sensitive models and data are processed. The system leverages a combination of hardware-based security features, including SGX enclaves and attestation mechanisms, to protect proprietary algorithms and sensitive data throughout the execution process.

[0020] The system for secure execution may comprise an SGX enclave configured to host an inference module for executing the proprietary algorithms. This SGX enclave provides a secure, isolated environment, ensuring that the proprietary algorithms and data are protected from unauthorized access or modification. A trusted initialization component is configured to run before the operating system, communicate with a GPU, and fetch a cryptographic key from the GPU, establishing a secure foundation for the system. In some embodiments, the trusted initialization component comprises a micro-hypervisor.

[0021] Accordingly, a library operating system (LibOS) within the SGX enclave may be configured to intercept communication between the inference module and the GPU and encrypt a payload prior to transmitting it to a driver. The LibOS is configured to limit the possible behaviors of the executing inference module, further enhancing security.

[0022] A driver is configured to request the trusted initialization component to re-encrypt the payload before transmitting it to the GPU, ensuring that the data is protected throughout the communication pipeline.

[0023] The system may further include an attestation module configured to perform a chained attestation process, comprising the GPU attesting to the trusted initialization component, the trusted initialization component attesting to the SGX enclave, and the SGX enclave attesting to an external entity. In some embodiments, the attestation module uses the DMTF SPDM protocol for GPU attestation. The SGX enclave attests to the external entity using a remotely verifiable quote.

[0024] In one particular embodiment, the GPU is an Nvidia GPU with Hopper architecture, providing hardware-based support for confidential computing. The proprietary algorithms may be Large Language Models (LLMs), enabling secure processing of sensitive natural language data. The external entity may be a customer or model provider, facilitating a secure marketplace for proprietary algorithms. In some embodiments, the driver is modified to skip the SPDM protocol and key generation, streamlining the initialization process and reducing the attack surface.

[0025] According to still another aspect of the presently disclosed subject matter, a method is taught for secure communication between a secure enclave, such as an SGX enclave, and a GPU. The method may include: establishing a virtualized connection between the SGX enclave and a device driver of the GPU; intercepting communication between the SGX enclave and the GPU's device driver using a library operating system (LibOS); encrypting outgoing communications from the SGX enclave within the LibOS using a first cryptographic key; requesting, by the device driver, a re-encryption of the encrypted communications using a second cryptographic key before transmitting to the GPU; and performing the re-encryption in a separate execution context.

[0026] To secure data processing in a Large Language Model (LLM) marketplace, the method further comprises: establishing a secure execution environment using at least one secure enclave to protect an inference module and sensitive data from unauthorized access; receiving a pre-trained model from an external entity; validating the pre-trained model; encrypting the model and storing it securely within the at least one secure enclave; processing customer prompts using the inference module within the secure enclave, ensuring that the prompts and responses remain confidential; establishing trusted initialization component for managing communication between the secure enclave and a GPU; utilizing the GPU to accelerate inference process while maintaining data encryption during transmission between the secure enclave and the GPU; implementing a chained attestation strategy, wherein each component of the system attests to the integrity and security of its successor.

[0027] This chained attestation strategy comprises: the GPU attesting to the trusted initialization component; the trusted initialization component attesting to the secure enclave; and the secure enclave attesting to the external entity.

[0028] The establishment of a trusted initialization component for managing communication between the secure enclave and a GPU comprises providing a hypervisor, and the hypervisor managing key exchanges and re-encryption requests between the SGX enclave and the GPU.

[0029] In some implementations, establishing a virtualized connection comprises providing a hypervisor, and the hypervisor managing key exchanges and re-encryption requests between the SGX enclave and the GPU. The method may further comprise generating the first cryptographic key using a Diffie-Hellman protocol during the initialization of the SGX enclave. The device driver may obtain the second cryptographic key using an authentication process with the GPU, ensuring that the GPU is equipped with confidential computing capabilities and / or is an authentic Nvidia device.

[0030] The method may further include verifying that the GPU is authentic before establishing the virtualized connection, possibly using a security protocol such as the DMTF SPDM protocol. In some embodiments, the separate execution context comprises a secondary SGX enclave, which possesses the first cryptographic key and second cryptographic key.

[0031] To enhance security and auditability, the method may further comprise a step of logging encryption and re-encryption operations performed by the LibOS and the device driver for auditability and traceability of secure communications. The method may involve identifying sensitive communication types (e.g., data payloads) which require encryption, with the library operating system (LibOS) intercepting and encrypting these sensitive communications, while allowing non-sensitive communications (e.g., metadata) to pass unencrypted.

[0032] To further enhance security and prevent unauthorized access, the method includes dynamically updating the first and second cryptographic keys based on criteria such as usage patterns and / or time intervals. The method may also comprise an initialization process comprising verifying cryptographic signatures of software components loaded into the SGX enclave and the GPU, ensuring authenticity before execution.

[0033] To improve performance, the method may further comprise optimizing the encryption and decryption processes to reduce computational overhead, thereby enhancing the performance of secure communications between the SGX enclave and the GPU. The secure communication architecture can be extended to additional peripheral devices beyond the GPU, using similar virtualization and encryption techniques to maintain data confidentiality and integrity across multiple hardware components.

[0034] The method may also include an error handling and recovery mechanism that detects communication failures or security breaches and initiates corrective actions to maintain data integrity and secure operations. The method can support a multi-tenant environment where data from different users or applications is securely isolated and managed within the SGX enclave, ensuring that tenant data is not accessible to unauthorized parties.

[0035] Finally, the method may further comprise performing remote attestation of the secure communication setup, allowing external entities to verify the integrity and security configurations of the SGX enclave and GPU interactions.

[0036] By combining these features, the present invention provides a robust, efficient, and highly secure system and method for executing proprietary algorithms, communicating between a secure enclave and a GPU, and securing data processing in an LLM marketplace, enabling the secure processing of sensitive data and the protection of valuable intellectual property in a variety of computing environments.

[0037] BRIEF DESCRIPTION OF THE FIGURES

[0038] For a better understanding of the embodiments and to show how it may be carried into effect, reference will now be made, purely by way of example, to the accompanying drawings.

[0039] With specific reference now to the drawings in detail, it is stressed that the particulars shown are by way of example and for purposes of illustrative discussion of selected embodiments only, and are presented in the cause of providing what is believed to be the most useful and readily understood description of the principles and conceptual aspects. In this regard, no attempt is made to show structural details in more detail than is necessary for a fundamental understanding; the description taken with the drawings making apparent to those skilled in the art how the various selected embodiments may be put into practice. In the accompanying drawings:

[0040] Fig. 1 is block diagram illustrating a system for secure execution of proprietary algorithms, highlighting the key components and their interactions within a trusted environment;

[0041] Fig. 2 is a block diagram illustrating a system for secure communication between a secure enclave and a GPU, detailing the re-encryption process and the virtualized connection;

[0042] Fig. 3 is a flowchart illustrating a method for secure communication between an SGX enclave and a GPU, emphasizing the steps involved in encryption and re-encryption;

[0043] Fig. 4 is a flowchart illustrating a method for securing data processing in a large language model (LLM) marketplace, outlining the key steps for model validation, secure execution, and attestation, and

[0044] Fig. 5 is a flowchart illustrating a chained attestation strategy, detailing the process for verifying the integrity and security of the system components.

[0045] DETAILED DESCRIPTION

[0046] Aspects of the present disclosure relate to system and methods for secure multi-party computation. More specifically, the disclosure pertains to methods and platforms which use virtualization and reencryption for GPU acceleration in trusted execution environments by securing communication with secure platforms such as Intel Software Guard Extensions (SGX) enabled environments. A comprehensive system and method may be provided for secure execution of proprietary algorithms as well as secure communication between secure enclaves and a GPU. The system addresses the need for enhanced security and confidentiality in computing environments where sensitive models and data are processed. The system and method leverage a combination of hardware-based security features, virtualization, re-encryption, and attestation mechanisms to protect proprietary algorithms and sensitive data throughout the execution and communication process. The invention is particularly well-suited for securing data processing in a Large Language Model (LLM) marketplace.

[0047] In various embodiments of the disclosure, one or more tasks as described herein may be performed by a data processor, such as a computing platform or distributed computing system for executing a plurality of instructions. Optionally, the data processor includes or accesses a volatile memory for storing instructions, data or the like. Additionally, or alternatively, the data processor may access a non-volatile storage, for example, a magnetic hard-disk, flash-drive, removable media or the like, for storing instructions and / or data.

[0048] It is particularly noted that the systems and methods of the disclosure herein may not be limited in its application to the details of construction and the arrangement of the components or methods set forth in the description or illustrated in the drawings and examples. The systems and methods of the disclosure may be capable of other embodiments, or of being practiced and carried out in various ways and technologies.

[0049] Alternative methods and materials similar or equivalent to those described herein may be used in the practice or testing of embodiments of the disclosure. Nevertheless, particular methods and materials are described herein for illustrative purposes only. The materials, methods, and examples are not intended to be necessarily limiting.

[0050] Reference is now made to the block diagram of Fig. 1 which shows a system 100 for secure execution of propriety algorithms provided by an external entity 150. The system includes a trusted environment 120 and an attestation module 130. The trusted environment 120 is configured and operable to leverage hardware-based security features to protect sensitive information, while explicitly excluding the Operating System (OS) 111 and other applications 113 from the trusted environment 120. The attestation module is configured and operable to establish trust in the system by verifying the integrity and authenticity of its constituent parts.

[0051] The trusted environment 120 may serve as a security boundary, isolating sensitive components and data from the untrusted operating system and other potentially malicious applications. The trusted environment 120 encapsulates a secure enclave 122 and an encrypted communication channel 127. The secure enclave 122 contains a library 124, an encryption module 126, an inference module 128, at least a section of a device driver 112 extracted from the kernel 110 via the encrypted communication channel 127, and a trusted initialization component 123. It is noted that the trusted environment 120 creates a protected space where these critical elements can operate without fear of interference or compromise. This isolation ensures that sensitive data and proprietary algorithms remain confidential and protected from unauthorized access, modification, or theft, thereby enabling the secure execution of models and the preservation of data integrity.

[0052] It is noted that the secure enclave 122 may be an element, such as Intel SGX for example, which provides a hardware-protected, isolated execution environment within the computing platform. Accordingly, the secure enclave 122 may allow applications to create secure regions of memory, where sensitive code, such as a propriety algorithm or model from an external entity 150, may be executed and private data may be processed with a high degree of confidentiality and integrity. It is a particular feature of the secure enclave that even privileged software, such as the operating system 111 , a hypervisor or other apps 113, cannot directly access the contents of an enclave, protecting against a wide range of software-based attacks. By leveraging hardware-based memory encryption and integrity checks, the secure enclave 122 may ensure that sensitive computations and data remain protected, enabling the secure execution of models and the safeguarding of valuable intellectual property.

[0053] The Inference Module 128 is a computational element within the secure system 100, responsible for executing the propriety algorithm or model and generating results based on input data. Operating within the protected confines of the secure enclave 122, the inference module 128 is shielded from potential attacks and unauthorized access, ensuring the confidentiality and integrity of both the model and the sensitive data it processes. By leveraging the secure environment provided by the secure enclave 122, the Inference Module 128 can perform its computations with a high degree of assurance, enabling the reliable and trustworthy delivery of results.

[0054] Within the secure enclave 122, the library 124 serves as an intermediary, providing essential functions for the secure operation of the Inference Module 128. The library 124 facilitates communication between the Inference Module and the outside world, particularly with the Device Driver. The encryption module 126 within the library 124 encrypts all outgoing communications from the secure enclave 122, ensuring that any data transmitted outside the secure boundary remains protected from unauthorized access. Thus the library 124 and the encryption module 126 enable the secure and reliable operation of the Inference Module within the protected environment of the secure enclave 122.

[0055] The trusted initialization component 123 is a critical element that operates to manage encrypted communication. The trusted initialization component 123 operates before the operating system 111 takes control, establishing a secure foundation for the entire system 100. Its primary function is to provide essential security services, such as key management and cryptographic operations, in a trusted manner. To achieve this, the trusted initialization component 123 may communicate directly with the GPU 125, fetching a cryptographic key that is used for encrypting data. Additionally, it may receive re-encryption requests from the device driver 112, enabling the system 100 to perform cryptographic operations on data transmitted between the secure enclave 122 and the GPU 125. By performing these functions before the operating system 111 loads, the trusted initialization component 123 ensures that the system 100 starts in a secure state and that sensitive data remains protected throughout the entire execution process.

[0056] The device driver 112 is an element, which normally runs within the kernel 110 outside the trusted environment 120. It is a particular feature of the secure system 100 that at least a section of the device driver 112 may be executed within the secure enclave 122.

[0057] The device driver 112 receives encrypted data from the encryption module 126 within the enclave's library 124. Before transmitting this data to the GPU 125, the device driver 112 initiates a re-encryption process by sending a re-encryption request to the trusted initialization component 123. This ensures that the data is further protected by a different cryptographic key. By orchestrating this re-encryption process and enabling communication with the GPU 125, the device driver plays a vital role in the overall function and security of the system. Accordingly, the secure communication between the enclave 122 and the GPU 125 allows the execution of propriety algorithms to be encapsulated by the trusted environment 120

[0058] The attestation module 130 performs a chained attestation process, where each component attests to the trustworthiness of the next, creating a chain of trust that extends from the hardware level up to the external entity 150. The attestation process may involve verifying the cryptographic measurements and configurations of each component, ensuring that they have not been tampered with or compromised. By successfully completing the attestation process, the Attestation Module 130 provides assurance to the External Entity 150 that the system is operating in a secure and trustworthy manner, thereby enabling the secure execution of models and the protection of sensitive data. Reference is now made to Fig. 2 which represents a system 200 for providing secure communication between a secure enclave 222 and a GPU 225. The system for secure communication 200 depicted in Fig. 2 provides a possible example of a specific implementation of a secure data pathway that can be utilized within the broader system 100 for secure execution of proprietary algorithms, as illustrated in Fig. 1. While Fig. 1 outlines the general architecture for protecting models and data, Fig. 2 details a particular pathway for ensuring secure communication between the enclave 222 and the GPU 225. The reencryption mechanism and virtualized connection described in Fig. 2 can be integrated into the system in Fig. 1 to enhance the security of data as it is transmitted to the GPU for processing, ensuring that the proprietary algorithms and sensitive data remain protected throughout the entire execution pipeline.

[0059] The system 200 for secure communication leverages a combination of virtualization and reencryption techniques to protect data transmitted between the enclave 222, such as an SGX, and a GPU 225 and thereby addresses the challenge of enabling secure and confidential communication in a computing environment where sensitive data is processed.

[0060] The system 200 includes an execution context 230 used to perform cryptographic operations and a separate a virtual connection 250 between an encryption module 226 of a library 224 within the secure enclave 222 and a device driver 212 within the kernel 210.

[0061] The Library 224 such as a library operating system (LibOS) may reside within an SGX Enclave 222 for example to provide essential functions for the secure operation of the system. The virtual connection 250 may rely on virtualization to connect connects the enclave 222 to a device driver 212 such as Nvidia's device driver residing in the kernel 210.

[0062] In operation, the Inference Module 228 within the SGX Enclave 222 generates data that needs to be processed by the GPU 225. The Encryption Module 226 encrypts this data before it leaves the SGX Enclave 222. The encrypted data is then transmitted to the Device Driver 212 via the Virtual Connection.

[0063] Upon receiving the encrypted data, the Device Driver 212 initiates a re-encryption process. It sends a Re-encryption Request to the Execution Context 230. The Execution Context 230 is a separate execution environment that possesses the cryptographic keys necessary to perform the re-encryption. The reencrypted data is then sent back to the Device Driver 212.

[0064] Finally, the Device Driver 212 transmits the re-encrypted data to the GPU 225 for processing.

[0065] By using a combination of virtualization and re-encryption, the system ensures that sensitive data remains protected throughout the communication pipeline, from the SGX Enclave 222 to the GPU 225. The Trusted Computing Base (TCB) of this solution includes only the code executing inside the enclaves, which allows the use a full-fledged hypervisor like KVM.

[0066] This architecture can be extended for other devices. Thus, integrating these ideas into cloud infrastructure may facilitate the introduction of additional confidential devices to the SGX world.

[0067] Reference is now made to the flowchart of Fig. 3 which illustrates a method (300) for secure communication between an SGX enclave and a GPU. The method may be used to protect sensitive data during transmission. The method includes the following steps:

[0068] Step 301 : Establish a Virtual Connection Between Enclave and GPU: The method begins by establishing a virtualized connection between the SGX enclave and the GPU. Simple architecture may be used, which relies on virtualization to connect Nvidia's device driver with an SGX enclave. This virtualization allows the secure enclave to communicate with the GPU's device driver without directly exposing sensitive data to the underlying operating system.

[0069] Step 302: Intercept Communication Between Enclave and GPU: The communication between the SGX enclave and the GPU is intercepted by a library operating system (LibOS). A library OS, like Gramine, may act as an intermediary between the library and the driver and may be modified to encrypt all outgoing communication using its internal key. This interception allows the LibOS to perform security-related operations on the data before it is transmitted to the GPU.

[0070] Step 303: Use First Key to Encrypt Outgoing Communication from SGX: The LibOS then uses a first cryptographic key to encrypt all outgoing communication from the SGX enclave. This encryption ensures that the data is protected from unauthorized access during transmission.

[0071] Step 304: Use Second Key to Re-encrypt of Encrypted Communication: Before the encrypted communication is transmitted to the GPU, a second key may be used to re-encrypt the encrypted communication. Accordingly, the driver may be modified to request a re-encryption before transmitting the packets to the GPU.

[0072] Step 305: Perform the Re-encryption in Execution Context: The re-encryption is typically performed in a separate execution context, such as another virtual machine or a root mode environment. This separation of the re-encryption process from the main execution environment enhances security by limiting the exposure of the cryptographic keys.

[0073] Step 306: Transmit Re-encrypted Communication to GPU: The re-encrypted communication is transmitted to the GPU for processing. This ensures that the data remains protected throughout the entire communication pipeline, from the SGX enclave to the GPU.

[0074] Reference is now made to the flowchart of Fig. 4 which illustrates a method (400) for securing data processing in a large language model (LLM) marketplace, ensuring the confidentiality and integrity of both the model and the user's data.

[0075] While the method illustrated in the flowchart of Fig. 3 details the secure communication mechanisms between the SGX enclave and the GPU, the method illustrated in the flowchart of Fig. 4 broadens the scope to illustrate the overall process for securing data processing within a Large Language Model (LLM) marketplace. This process encompasses not only secure communication but also model validation, encryption, and attestation, ensuring end-to-end security for both the model provider and the user. Such a process for securing data processing may include the following steps:

[0076] Step 401 : Establish a Secure Execution Environment: The method begins by establishing a secure execution environment using at least one secure enclave, such as Intel SGX. In order to protect the application and the model it operates on, these may be wrapped by an SGX enclave, whose memory is encrypted and inaccessible from other applications, the operating system, and even hardware devices. This secure enclave provides a protected environment for executing the inference module and storing sensitive data.

[0077] Step 402: Receive Pre-trained Model from External Entity: A pre-trained model is received from an external entity, such as a model provider. This model contains the proprietary algorithms and knowledge that will be used for processing user prompts.

[0078] Step 403: Validate Pre-trained Model: Before using the pre-trained model, it may be validated to ensure its integrity and authenticity. This validation process may involve verifying cryptographic signatures or performing other security checks to prevent the use of malicious or corrupted models.

[0079] Step 404: Encrypt the Model within a Secure Enclave: The validated model is then encrypted and stored securely within the secure enclave. This encryption protects the model from unauthorized access or modification.

[0080] Step 405: Process Customer Prompts Using the Inference Module: Customer prompts are processed using the inference module within the secure enclave. The inference module uses the pre-trained model to generate responses to the customer prompts. Step 406: Ensure Prompts and Responses Remain Confidential: The method ensures that the prompts and responses remain confidential throughout the entire process. This may involve encrypting the prompts and responses before they are transmitted outside the secure enclave.

[0081] Step 407: Establish Trusted Initialization Component: A trusted initialization component, such as a micro-hypervisor or the like, is established to manage communication between the secure enclave and a GPU. A micro-hypervisor may be leveraged and slight modifications may be introduced to various open- source software components, e.g., the Nvidia driver and library OS. This trusted initialization component runs before the operating system and provides a secure foundation for the system.

[0082] Step 408: Manage Communication Between Secure Enclave and GPU: The trusted initialization component manages communication between the secure enclave and the GPU, ensuring that all data is transmitted securely.

[0083] Step 409: Use GPU to Accelerate Inference Enclave: Inference processes may benefit from hardware acceleration provided by GPU. Accordingly, the GPU is used to accelerate the inference process, improving the performance of the system.

[0084] Step 410: Maintain Data Encryption Between Secure Enclave and GPU: Data encryption is maintained during transmission between the secure enclave and the GPU, protecting sensitive information from unauthorized access.

[0085] Step 411 : Implement Attestation Strategy: An attestation strategy is implemented to verify the integrity and security of the system components. This attestation strategy ensures that all components are operating in a trusted state.

[0086] The secure data processing method outlined in Fig. 4 relies on a robust attestation process to verify the integrity and security of the system components. Reference is now made to the flowchart of Fig. 5 which illustrates a method for a possible chained attestation strategy (500) used to verify the integrity and security of the system components.

[0087] Fig. 5 provides a detailed illustration of the chained attestation strategy, highlighting the specific steps involved in establishing trust between the GPU, the trusted initialization component, the secure enclave, and the external entity."

[0088] Step 501 : GPU Attests to Trusted Initialization Component: The GPU attests to the trusted initialization component, verifying its identity and integrity.

[0089] Step 502: Trusted Initialization Component Attests to Secure Enclave: The trusted initialization component attests to the secure enclave, verifying its identity and integrity.

[0090] Step 503: Secure Enclave Attests to External Entity: The secure enclave attests to the external entity, such as a customer or model provider, verifying the overall integrity and security of the system.

[0091] This chained attestation strategy ensures that all components are operating in a trusted state, providing assurance to the external entity that the system is secure.

[0092] Technical and scientific terms used herein should have the same meaning as commonly understood by one of ordinary skill in the art to which the disclosure pertains. Nevertheless, it is expected that during the life of a patent maturing from this application many relevant systems and methods will be developed. Accordingly, the scope of the terms such as computing unit, network, display, memory, server and the like are intended to include all such new technologies a priori.

[0093] As used herein the term “about” refers to at least ± 10 %.

[0094] The terms "comprises", "comprising", "includes", "including", “having” and their conjugates mean "including but not limited to" and indicate that the components listed are included, but not generally to the exclusion of other components. Such terms encompass the terms "consisting of' and "consisting essentially of". The phrase "consisting essentially of' means that the composition or method may include additional ingredients and / or steps, but only if the additional ingredients and / or steps do not materially alter the basic and novel characteristics of the claimed composition or method.

[0095] As used herein, the singular form "a", "an" and "the" may include plural references unless the context clearly dictates otherwise. For example, the term "a compound" or "at least one compound" may include a plurality of compounds, including mixtures thereof.

[0096] The word “exemplary” is used herein to mean “serving as an example, instance or illustration”. Any embodiment described as “exemplary” is not necessarily to be construed as preferred or advantageous over other embodiments or to exclude the incorporation of features from other embodiments.

[0097] The word “optionally” is used herein to mean “is provided in some embodiments and not provided in other embodiments”. Any particular embodiment of the disclosure may include a plurality of “optional” features unless such features conflict.

[0098] Whenever a numerical range is indicated herein, it is meant to include any cited numeral (fractional or integral) within the indicated range. The phrases “ranging / ranges between” a first indicate number and a second indicate number and “ranging / ranges from” a first indicate number “to” a second indicate number are used herein interchangeably and are meant to include the first and second indicated numbers and all the fractional and integral numerals therebetween. It should be understood, therefore, that the description in range format is merely for convenience and brevity and should not be construed as an inflexible limitation on the scope of the disclosure. Accordingly, the description of a range should be considered to have specifically disclosed all the possible sub-ranges as well as individual numerical values within that range. For example, description of a range such as from 1 to 6 should be considered to have specifically disclosed sub-ranges such as from 1 to 3, from 1 to 4, from 1 to 5, from 2 to 4, from 2 to 6, from 3 to 6 etc., as well as individual numbers within that range, for example, 1 , 2, 3, 4, 5, and 6 as well as non-integral intermediate values. This applies regardless of the breadth of the range.

[0099] It is appreciated that certain features of the disclosure, which are, for clarity, described in the context of separate embodiments, may also be provided in combination in a single embodiment. Conversely, various features of the disclosure, which are, for brevity, described in the context of a single embodiment, may also be provided separately or in any suitable sub-combination or as suitable in any other described embodiment of the disclosure. Certain features described in the context of various embodiments are not to be considered essential features of those embodiments unless the embodiment is inoperative without those elements.

[0100] Although the disclosure has been described in conjunction with specific embodiments thereof, it is evident that many alternatives, modifications and variations will be apparent to those skilled in the art. Accordingly, it is intended to embrace all such alternatives, modifications and variations that fall within the spirit and broad scope of the appended claims.

[0101] All publications, patents and patent applications mentioned in this specification are herein incorporated in their entirety by reference into the specification, to the same extent as if each individual publication, patent or patent application was specifically and individually indicated to be incorporated herein by reference. In addition, citation or identification of any reference in this application shall not be construed as an admission that such reference is available as prior art to the present disclosure. To the extent that section headings are used, they should not be construed as necessarily limiting.

[0102] The scope of the disclosed subject matter is defined by the appended claims and includes both combinations and sub combinations of the various features described hereinabove as well as variations and modifications thereof, which would occur to persons skilled in the art upon reading the foregoing description.

Claims

CLAIMS1 . A system for secure communication between a secure enclave and a GPU, comprising: a device driver of the GPU; a library operating system (LibOS) configured to intercept communications between the secure enclave and the GPU device driver; an encryption module within the LibOS that encrypts outgoing communications from the secure enclave using a first cryptographic key; a re-encryption request module within the device driver that requests a re-encryption of the encrypted communications using a second cryptographic key before transmission to the GPU; and a separate execution context configured to perform the re-encryption of the communications.

2. The system of claim 1 , wherein the encryption module within the LibOS uses a symmetric encryption algorithm.

3. The system of claim 1 , wherein the re-encryption request module within the device driver transmits a re-encryption request to the separate execution context via a virtual serial port.

4. The system of claim 1 , wherein the separate execution context is a virtual machine.

5. The system of claim 4, wherein the virtual machine is running in root mode.

6. The system of claim 1 , wherein the LibOS is Gramine.

7. The system of claim 1 , further comprising an attestation module configured to verify the integrity of the secure enclave.

8. The system of claim 1 , wherein the secure enclave is an Intel SGX enclave.

9. The system of claim 1 , wherein the GPU supports confidential computing.

10. A system for secure execution of propriety algorithms, comprising: an SGX enclave configured to host an inference module for executing the propriety algorithms; a trusted initialization component configured to: run before the operating system, communicate with a GPU, and fetch a cryptographic key from the GPU; a library operating system (LibOS) within the SGX enclave configured to: intercept communication between the inference module and the GPU, and encrypt a payload prior to transmitting it to a driver; a driver configured to request the trusted initialization component to re-encrypt the payload before transmitting it to the GPU; an attestation module configured to perform a chained attestation process, comprising: the GPU attesting to the trusted initialization component the trusted initialization component attesting to the SGX enclave the SGX enclave attesting to an external entity.11 . The system of claim 11 , wherein the trusted initialization component comprises a microhypervisor.

12. The system of claim 11 , wherein the LibOS is configured to limit the possible behaviors of the executing inference module.

13. The system of claim 11 , wherein the attestation module uses the DMTF SPDM protocol for GPU attestation.

14. The system of claim 11 , wherein the SGX enclave attests to the external entity using a remotely verifiable quote.

15. The system of claim 11 , wherein the GPU is an Nvidia GPU with Hopper architecture.

16. The system of claim 11 , wherein the propriety algorithms are Large Language Models (LLMs).

17. The system of claim 11 , wherein the external entity is a customer or model provider.

18. The system of claim 11 , wherein the driver is modified to skip the SPDM protocol and key generation.

19. A method for securing data processing in a large language model (LLM) marketplace, comprising: establishing a secure execution environment using at least one secure enclave to protect an inference module and sensitive data from unauthorized access; receiving a pre-trained model from an external entity; validating the pre-trained model; encrypting the model and storing it securely within the at least one secure enclave; processing customer prompts using the inference module within the secure enclave, ensuring that the prompts and responses remain confidential; establishing trusted initialization component for managing communication between the secure enclave and a GPU; utilizing the GPU to accelerate inference process while maintaining data encryption during transmission between the secure enclave and the GPU; implementing a chained attestation strategy, wherein each component of the system attests to the integrity and security of its successor.

20. The method of claim 19 wherein the step of implementing a chained attestation strategy comprises: the GPU attesting to the trusted initialization component; the trusted initialization component attesting to the secure enclave; and the secure enclave attesting to the external entity.

21. The method of claim 19 wherein the step of establishing trusted initialization component for managing communication between the secure enclave and a GPU comprises providing a hypervisor, and the hypervisor managing key exchanges and re-encryption requests between the SGX enclave and the GPU.

22. A method for secure communication between an SGX enclave and a GPU, comprising: establishing a virtualized connection between the SGX enclave and a device driver of the GPU; intercepting communication between the SGX enclave and the GPU's device driver using a library operating system (LibOS); encrypting outgoing communications from the SGX enclave within the LibOS using a first cryptographic key; requesting, by the device driver, a re-encryption of the encrypted communications using a second cryptographic key before transmitting to the GPU; and performing the re-encryption in a separate execution context.

23. The method of claim 22, wherein the step of establishing a virtualized connection comprises providing a hypervisor, and the hypervisor managing key exchanges and re-encryption requests between the SGX enclave and the GPU.

24. The method of claim 22, further generating the first cryptographic key using a Diffie-Hellman protocol during the initialization of the SGX enclave.

25. The method of claim 22, further comprising the device driver obtaining the second cryptographic key using an authentication process with the GPU.

26. The method of claim 25 wherein the authentication process comprises ensuring that the GPU is equipped with confidential computing capabilities.

27. The method of claim 25 wherein the authentication process comprises ensuring that the GPU is an authentic Nvidia device.

28. The method of claim 22, further comprising verifying that the GPU is authentic before establishing the virtualized connection.

29. The method of claim 28, wherein the step of verifying that the GPU is authentic comprises using a security protocol.

30. The method of claim 28, wherein the step of verifying that the GPU is authentic comprises using a DMTF SPDM protocol.31 . The method of claim 22, wherein the separate execution context comprises a secondary SGX enclave.

32. The method of claim 31 wherein the secondary SGX enclave possesses the first cryptographic key and second cryptographic key.

33. The method of claim 22, further comprising a step of logging encryption and re-encryption operations performed by the LibOS and the device driver for auditability and traceability of secure communications.

34. The method of claim 22, further comprising: identifying sensitive communication types which require encryption; the library operating system (LibOS) intercepting and encrypting the sensitive communications.

35. The method of claim 34, further comprising allowing non-sensitive communications to pass unencrypted.

36. The method of claim 34 wherein the sensitive communication types comprise data payloads.

37. The method of claim 36 wherein the non-sensitive communications comprise metadata.

38. The method of claim 22, further comprising dynamically updating the first and second cryptographic keys , to enhance security and prevent unauthorized access.

39. The method of claim 38 wherein the updating is based on criteria selected from a group including usage patterns, time intervals and combinations thereof.

40. The method of claim 22, wherein further comprising an initialization process comprising: verifying cryptographic signatures of software components loaded into the SGX enclave, comprising; verifying cryptographic signatures of software components loaded into the GPU, and ensuring authenticity before execution.

41. The method of claim 22, further comprising optimizing the encryption and decryption processes to reduce computational overhead, thereby enhancing the performance of secure communications between the SGX enclave and the GPU.

42. The method of claim 22, wherein the secure communication architecture is extended to additional peripheral devices beyond the GPU, using similar virtualization and encryption techniques to maintain data confidentiality and integrity across multiple hardware components.

43. The method of claim 22, further comprising an error handling and recovery mechanism that detects communication failures or security breaches and initiates corrective actions to maintain data integrity and secure operations.

44. The method of claim 22, further comprising supporting a multi-tenant environment where data from different users or applications is securely isolated and managed within the SGX enclave, ensuring that tenant data is not accessible to unauthorized parties.

45. The method of claim 22, further comprising performing remote attestation of the secure communication setup, allowing external entities to verify the integrity and security configurations of the SGX enclave and GPU interactions.

Images