Method for transmitting data in heterogeneous computing system and heterogeneous computing system

Abstract

The invention provides a method for transmitting data in a heterogeneous computing system and the heterogeneous computing system. The heterogeneous computing system comprises a host CPU, a target hardware module and a heterogeneous computing card, the target hardware module and the heterogeneous computing card are packaged into an integral structure in an anti-disassembly mode, the target hardware module is connected to the CPU through a first bus and connected to the heterogeneous computing card through a second bus, and a system kernel-level target software module runs on the host CPU. The data transmission method comprises the steps that in a first time period, a target software module transmits a first ciphertext corresponding to a first data piece to a target hardware module through a first bus, and meanwhile, a second data piece is encrypted; the first data piece and the second data piece belong to a first data block, and the first data block is from a user mode application. And in a second time period following the first time period, the target hardware module decrypts the first ciphertext, and transmits the decrypted and recovered first data piece to the heterogeneous computing card through the second bus.

Description

Technical Field

[0001] This specification relates to the field of computer technology, and more particularly to a method for transmitting data in a heterogeneous computing system and a heterogeneous computing system. Background Technology

[0002] Currently, with the rapid development of e-commerce, large-scale model inference, and cloud computing technologies, high-value data such as personal financial data, transaction records, and identity information are growing exponentially and are being computed and stored in a highly centralized manner on various servers and cloud platforms. At the same time, security incidents such as data breaches, tampering, forgery, and man-in-the-middle attacks occur frequently, leading to user privacy leaks, financial losses, and damage to corporate reputation, becoming a key bottleneck restricting the healthy development of the digital economy. Traditional encryption technologies, access controls, and firewalls alone are insufficient to cope with the complex and ever-changing attack surfaces. In particular, data still faces the risk of being illegally read, stolen, executed, or tampered with during computation and internal transmission within cloud platforms. Therefore, a technical solution is urgently needed to improve the security of data during computation and transmission. Summary of the Invention

[0003] This specification describes one or more embodiments of a method for transmitting data in a heterogeneous computing system and a heterogeneous computing system, which can avoid bus attacks on the bus between the heterogeneous computing card and the CPU, and improve the security and efficiency of data transmission in the heterogeneous computing system.

[0004] According to a first aspect, a method for transmitting data in a heterogeneous computing system is provided, the heterogeneous computing system comprising a host CPU, a target hardware module and a heterogeneous computing card packaged as a single unit in a non-removable manner, wherein the target hardware module is connected to the CPU via a first bus and to the heterogeneous computing card via a second bus, and the host CPU runs a target software module at the system kernel level, the method comprising:

[0005] During the first time period, the target software module transmits the first ciphertext corresponding to the first data slice to the target hardware module through the first bus, and simultaneously encrypts the second data slice; the first data slice and the second data slice both belong to the first data block, and the first data block comes from the user-space application;

[0006] In the second time period following the first time period, the target hardware module decrypts the first ciphertext and transmits the decrypted and recovered first data fragment to the heterogeneous computing card through the second bus.

[0007] According to a second aspect, a method for transmitting data in a heterogeneous computing system is provided, the heterogeneous computing system comprising a host CPU, a target hardware module and a heterogeneous computing card packaged as a single unit in a non-removable manner, wherein the target hardware module is connected to the CPU via a first bus and to the heterogeneous computing card via a second bus, and the host CPU runs a target software module at the system kernel level, the method comprising:

[0008] During the third time period, the target hardware module transmits the third ciphertext corresponding to the third data slice to the host CPU through the first bus, and encrypts the fourth data slice at the same time; the third data slice and the fourth data slice both belong to the second data block, and the second data block comes from the heterogeneous computing card;

[0009] In the fourth time period following the third time period, the target software module decrypts the third ciphertext and provides the decrypted and recovered third data fragment to the user-space application.

[0010] According to a third aspect, a heterogeneous computing system is provided, comprising: a host CPU, and a target hardware module and a heterogeneous computing card packaged as a single unit in a non-removable manner, wherein the target hardware module is connected to the CPU via a first bus and to the heterogeneous computing card via a second bus, and the host CPU runs a target software module at the system kernel level, wherein:

[0011] The target software module is used to transmit the first ciphertext corresponding to the first data slice to the target hardware module through the first bus during a first time period, and simultaneously encrypt the second data slice; the first data slice and the second data slice both belong to the first data block, and the first data block comes from the user-space application.

[0012] The target hardware module is used to decrypt the first ciphertext in a second time period following the first time period, and transmit the decrypted and recovered first data fragment to the heterogeneous computing card through a second bus.

[0013] According to a fourth aspect, a heterogeneous computing system is provided, comprising: a host CPU, and a target hardware module and a heterogeneous computing card packaged as a single unit in a non-removable manner, wherein the target hardware module is connected to the CPU via a first bus and to the heterogeneous computing card via a second bus, and the host CPU runs a target software module at the system kernel level, wherein:

[0014] The target hardware module is used to transmit the third ciphertext corresponding to the third data slice to the host CPU through the first bus during the third time period, and at the same time encrypt the fourth data slice; the third data slice and the fourth data slice both belong to the second data block, and the second data block comes from the heterogeneous computing card.

[0015] The target software module is used to decrypt the third ciphertext in a fourth time period following the third time period, and provide the decrypted and recovered third data fragment to the user-space application.

[0016] According to a fifth aspect, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed in a computer, causes the computer to perform the method described in the first or second aspect.

[0017] In the embodiments described in this specification, by providing an encryption / decryption software module (target software module) on the CPU side and an encryption / decryption hardware module (target hardware module) on the heterogeneous computing device side, the security of data transmission between the CPU and the heterogeneous computing card is ensured. Through the cooperation of the encryption / decryption software module and the encryption / decryption hardware module, the data transmitted on the exposed bus between the CPU and the heterogeneous computing device is always encrypted, thereby preventing bus attacks. Furthermore, the target software module and the target hardware module employ streaming encryption, transmission, and decryption processing, improving the efficiency of encrypted transmission on the bus without increasing additional data segmentation costs. Attached Figure Description

[0018] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This illustrates a typical large-scale model reasoning scenario;

[0020] Figure 2 This diagram illustrates a bus attack in a heterogeneous computing system.

[0021] Figure 3 This demonstrates the security enhancement solutions offered by different accelerator card providers;

[0022] Figure 4 A schematic structural diagram of a heterogeneous computing system according to one embodiment is shown;

[0023] Figure 5 This diagram illustrates the transfer of data from the host CPU to the heterogeneous computing card.

[0024] Figure 6 This diagram illustrates the transfer of data from a heterogeneous computing card to the host CPU.

[0025] Figure 7 A schematic diagram of H2D data transmission in one embodiment is shown;

[0026] Figure 8 A schematic diagram of D2H data transmission in one embodiment is shown;

[0027] Figure 9 This diagram illustrates H2D data transmission in a multi-engine core embodiment.

[0028] Figure 10 This diagram illustrates D2H data transmission in a multi-engine core embodiment. Detailed Implementation

[0029] The solution provided in this specification will now be described with reference to the accompanying drawings.

[0030] As mentioned earlier, in the context of big data and cloud platforms, data security protection has become a focus of attention for all parties. Figure 1 This illustrates a typical large-model inference scenario. In this scenario, the large-model provider transmits the large model to the server provider, thereby deploying the large model to the cloud. Users access the large-model service in the cloud via the network. Specifically, users can transmit prompts to the cloud via the network. The server provider, based on the large model file and the user's prompts, utilizes its powerful computing resources to perform model inference and computation, returning the inference results to the user. To enhance computing performance, the server typically employs a heterogeneous computing system of "CPU + accelerator card" for data processing, where the accelerator card can be dedicated hardware suitable for specific types of computing tasks. In the model inference scenario, the accelerator card is typically a GPU.

[0031] In computational scenarios such as large model inference, users (including large model providers and users) expect server providers to offer cross-domain data control to prevent attackers from stealing or tampering with data. In other words, they expect server providers to ensure data security during the computation process.

[0032] To enhance data security, some server providers employ Trusted Execution Environment (TEE) solutions to ensure the security of data computation processes. Currently, several CPU-based TEE solutions exist, collectively known as CPU TEE solutions. CPU TEE solutions primarily utilize hardware-level isolation and encryption mechanisms to construct a secure execution domain logically isolated from the general computing environment within the processor, ensuring the confidentiality and integrity of sensitive code and data. Typical implementations, such as Intel SGX, AMD SEV, and ARM TrustZone, all introduce secure memory access control, real-time encryption engines, and trusted boot mechanisms into the CPU through instruction set extensions and microarchitectural enhancements. Applications can invoke specific instructions to create protected secure enclaves with independent address spaces and context states, preventing unauthorized software (including operating systems and virtual machine monitors) from accessing their internal data. Simultaneously, the CPU constructs a trusted boot chain through a hardware root key, supporting remote authentication and ensuring that the code loaded into the enclave has not been tampered with.

[0033] However, for heterogeneous computing systems employing a "CPU + accelerator card" architecture, simply providing a TEE environment within the CPU is insufficient to ensure the overall security of the computing environment. For example... Figure 2 As shown, in heterogeneous computing systems, the CPU and heterogeneous computing cards (i.e., accelerator cards) are typically connected via a bus, on which plaintext data is transmitted. Even if a Trusted Execution Environment (TEE) is implemented in the CPU, and even if some heterogeneous accelerator card manufacturers implement a TEE within their heterogeneous accelerator cards, attackers can still steal user data by attacking the bus.

[0034] To prevent data leaks caused by bus attacks, some accelerator card manufacturers have proposed security enhancement solutions for their products, avoiding bus attacks by transmitting encrypted data on the bus. Figure 3This section illustrates security enhancement schemes from different accelerator card providers. It can be seen that to achieve encrypted transmission on the bus, modifications are needed to several components related to the accelerator card, including the user-space software stack and the accelerator card's hardware. Specifically, the user-space software stack needs modification to include software related to encrypted transmission; the accelerator card hardware also needs improvement to incorporate a dedicated encryption / decryption module. If an accelerator card from vendor A is used, data generated by upper-layer applications (e.g., model inference applications) needs to be encrypted by the encrypted transmission software in vendor A's accelerator card software stack, transmitted encrypted via vendor A's KMD (Kernel Mode Driver) on a bus such as PCIe, and then decrypted in vendor A's built-in encryption / decryption module before accelerated computation. If an accelerator card from vendor B is used, data generated by upper-layer applications needs to be encrypted by the encrypted transmission software in vendor B's accelerator card software stack, transmitted encrypted via vendor B's KMD on the bus, and then decrypted in vendor B's built-in encryption / decryption module. Therefore, each accelerator card manufacturer would need to make modifications to the entire data pipeline to achieve secure data transmission. Consequently, the aforementioned solutions are very costly to implement and have poor versatility.

[0035] Considering the data security requirements of the aforementioned heterogeneous computing systems and the shortcomings of related technologies, this specification proposes a universal security enhancement scheme adapted to various heterogeneous computing cards, capable of extending the Trusted Execution Environment to heterogeneous computing systems. Furthermore, it proposes a scheme to improve data processing and transmission efficiency within the aforementioned universal heterogeneous computing systems.

[0036] Figure 4 A schematic structural diagram of a heterogeneous computing system according to one embodiment is shown. As shown, in terms of hardware structure, the heterogeneous computing system includes a host CPU 20, a heterogeneous computing card 120, and a target hardware module 110 between the two. The target hardware module 110 and the heterogeneous computing card 120 are packaged as a single unit using a non-removable method to form a heterogeneous computing device 10. The target hardware module 110 has a first interface 111 and a second interface 112, wherein the first interface 111 is used to connect to the CPU via a first bus B1, and the second interface 112 is connected to the heterogeneous computing card 120 via a second bus B2. The first bus B1 and the second bus B2 are of the same type, i.e., conform to the same bus protocol, for example, both being PCIe buses. In the case of a PCIe bus, the first bus B1 and the second bus B2 can reside in the same bus domain without the need for bus address space translation. The target hardware module 110 is connected between the CPU and the heterogeneous computing card via the bus, performing encryption and decryption transmission operations between the two.

[0037] Specifically, the heterogeneous computing card 120 can be any computing hardware / chip / circuit different from the CPU; for example, it can be a GPU, FPGA, homomorphic computing chip, etc. Furthermore, the heterogeneous computing card 120 can be from any manufacturer, as long as it has a standard interface that supports the aforementioned bus protocol. Figure 4 In the process, the heterogeneous computing card 120 can be connected to the second interface 112 of the target hardware device 110 via bus B2 through the fourth interface 121.

[0038] The target hardware module 110 is used for data encryption / decryption and secure transmission between the CPU 20 and the heterogeneous computing card 120. The target hardware module 110 can be implemented as an FPGA, ASIC, IP core, etc., and may include dedicated circuit modules for data encryption / decryption. The target hardware module 110 and the heterogeneous computing card 120 are fixedly packaged into a non-removable integrated structure, wherein the second interface 112 and the second bus B2 are encapsulated within this non-removable integrated structure, while the first interface 111 and the first bus B1 are exposed. The first interface 111 can be connected to the third interface 211 on the CPU side via the bus B1.

[0039] The aforementioned non-removable integrated structure can be implemented in various forms. In one embodiment, the target hardware module 110 and the heterogeneous computing card 120 can be formed as a single chip, for example, as two hardware parts on a single complete chip. To make it more universally applicable to various accelerator cards, in one embodiment, after the target hardware module 110 and the heterogeneous computing card 120 are connected via the second bus B2, they are fixed together by an anti-tamper device to form an integrated structure. This anti-tamper device can be a one-piece molded device that, once disassembled, will completely destroy the entire heterogeneous computing device. Alternatively, the anti-tamper device can have a disassembly detector that, once disassembly occurs, can generate data records and report them to the verification party, making the target hardware module and even the entire heterogeneous computing device unable to pass the startup verification and thus unusable.

[0040] At the software level, a dedicated encryption / decryption software module, hereinafter referred to as target software module 210, is deployed on the host side. Specifically, the host operating system (OS) runs on the host CPU, and target software module 210 is OS kernel-level program code, running on the CPU as part of the operating system kernel layer. To ensure end-to-end data security, preferably, a Trusted Execution Environment (TEE) is provided on the CPU, and target software module 210 runs in the CPU TEE environment. It can be understood that the host side includes other components besides the CPU, such as main memory, and various user-mode applications also run on top of the operating system.

[0041] The target software module 210 on the CPU side works in conjunction with the target hardware module 110 in the heterogeneous computing device to achieve secure data transmission between the CPU and the heterogeneous computing card, preventing bus attacks. Specifically, data sent from the CPU 20 to the heterogeneous computing card 120 can be encrypted by the target software module 210 and transmitted to the heterogeneous computing device 10 via the first bus B1. In the heterogeneous computing device 10, the target hardware module 110 decrypts the ciphertext and sends the plaintext to the heterogeneous computing card 120 via the second bus B2.

[0042] For data sent from the heterogeneous computing card 120 to the CPU 20, within the heterogeneous computing device 10, the target hardware module 110 obtains the plaintext data via the second bus B2 and encrypts it. Then, the target hardware module 110 transmits the ciphertext to the CPU 20 via the first bus B1. In the CPU 20, the target software module 210 decrypts the ciphertext and sends the resulting plaintext to the upper-layer application.

[0043] Throughout the process described above, encrypted data is always transmitted on the first bus B1, thus preventing bus attacks. Plaintext is transmitted on the second bus B2; however, since the second bus B2 is encapsulated within a non-removable heterogeneous computing device, there is no risk of data leakage. Furthermore, due to the plaintext transmission on the second bus B2, the heterogeneous computing card 120 can normally receive plaintext data through its bus interface. Therefore, the heterogeneous computing card 120 is unaware of the entire process and requires no hardware or software modifications. This extends the trusted execution environment to the entire heterogeneous computing system.

[0044] The following description Figure 4 The specific working process of the heterogeneous computing system shown.

[0045] Figure 5 This diagram illustrates the transfer of data from the host CPU to a heterogeneous computing card. This data transfer direction can be referred to as H2D (Host to Device). Figure 5 As shown, firstly, in step (1), the heterogeneous computing card software stack sends an H2D request to the heterogeneous computing card. This request is passed through the heterogeneous computing card KMD to the heterogeneous computing card hardware. Next, in step (2), the heterogeneous computing card initiates an H2D request to the bus (i.e., the aforementioned second bus). Specifically, the heterogeneous computing card DMA (Direct Memory Access) hardware can initiate an H2D request to the PCIe bus. In step (3), the target hardware module captures or intercepts the H2D request sent by the heterogeneous computing card and sends the H2D request to the target software module.

[0046] Next, in step (4), the target software module at the system kernel level responds to the above request by reading plaintext data from the user-space application on the host side, encrypting it, and saving it to the encrypted data space on the host side. This encrypted data space can be a memory region specifically allocated for the target software module.

[0047] Subsequently, in step (5), the target hardware module initiates a PCIe read request (e.g., MRd TLP for memory data reading) via the first bus to read data from the encrypted data space. This step can also be understood as the target software module transmitting encrypted data to the target hardware module via the first bus. After decrypting the encrypted data, the target hardware module transmits the decrypted plaintext to the heterogeneous computing card via the second bus, saving it to the plaintext data space on the heterogeneous computing card side.

[0048] Although the H2D request was initiated by the host-side user in the above example, in other scenarios, data transfer requests in the H2D direction can also be initiated by the heterogeneous computing card. Its data processing procedure is similar to... Figure 5 similar.

[0049] Figure 6 This diagram illustrates the transfer of data from a heterogeneous computing card to the host CPU. This data transfer direction can be referred to as D2H (Device to Host). Figure 6 As shown, firstly, in step (1), the heterogeneous computing card software stack sends a D2H request to the heterogeneous computing card. This request is transmitted to the heterogeneous computing card hardware via the heterogeneous computing card KMD. Next, in step (2), the heterogeneous computing card initiates a D2H request to the bus (i.e., the aforementioned second bus). In step (3), the target hardware module captures or intercepts the D2H request sent by the heterogeneous computing card. If it is determined to be a D2H request, the target hardware module obtains the plaintext data from the heterogeneous computing card side via the second bus and encrypts it. Then, the target hardware module transmits the encrypted data to the host side via the first bus. Specifically, the target hardware module can write the encrypted data into the aforementioned encrypted data space on the host side via a PCIe write request.

[0050] Furthermore, in step (4), the target hardware module sends the address and D2H request in the encrypted data space to the target software module via the first bus. Next, in step (5), the target software module reads the ciphertext data from the encrypted data space, decrypts it to obtain plaintext data, and stores the plaintext data in a specific address in the memory space belonging to the user-mode application, thereby providing it to the host-side user-mode application.

[0051] Although the D2H request in the above example was initiated by the host-side user, in other scenarios, data transfer requests in the D2H direction can also be initiated by the heterogeneous computing card. Its data processing procedure is similar to... Figure 6 similar.

[0052] As can be seen, in Figure 5 and Figure 6 During this process, the encryption and decryption processes executed by the target software module on the host side are imperceptible to user-space applications (including the heterogeneous computing card's software stack) and the CPU's bus interface, requiring no modification to the heterogeneous computing card's software. On the heterogeneous computing device side, since plaintext transmission with the heterogeneous computing card still occurs via the original type of bus, the heterogeneous computing card is also unaware of this process, and no hardware modification is required. Thus, through the cooperation of the target software module and the target hardware module, secure data transmission between the host and the heterogeneous computing device is achieved without any modification to the heterogeneous computing card.

[0053] Furthermore, in one implementation, the host-side CPU supports Single Instruction Multiple Data (SIMD) parallel computing mechanism and is equipped with a hardware encryption / decryption engine. The target software module can utilize the host-side SIMD mechanism and hardware encryption / decryption engine to achieve high-speed data read / write and high-speed encryption / decryption. Moreover, the target software module does not use any explicit memory migration (CPU memcpy) operations during plaintext / ciphertext data transfer and encryption / decryption, achieving zero software copying during host-side encryption / decryption, minimizing the additional costs associated with encryption / decryption operations, and reducing performance loss during encryption / decryption.

[0054] Furthermore, to improve the performance of heterogeneous computing systems, we consider optimizing and accelerating the encryption and decryption transmission process through streaming processing.

[0055] Figure 7 A schematic diagram of H2D data transmission in one embodiment is shown. Figure 7 As shown, in conventional dense-state transmission, for example Figure 3 In the encrypted transmission between vendor A and vendor B, when H2D transmission is required, the heterogeneous computing card software stack at the user-space layer encrypts the entire plaintext data block, and then executes the H2D encrypted transmission using the memcpy operation. Decryption can only be performed after the heterogeneous computing card receives the entire encrypted data block. In some scenarios, the amount of plaintext data is very large, such as parameter matrices or tensors in a large-scale model, resulting in considerable waiting time in the above serial execution method. If data segmentation and encryption are required, the heterogeneous computing card software stack resides at the user-space application layer, and application-level data segmentation requires corresponding multi-stream task scheduling, increasing the complexity and cost of data processing. Furthermore, the heterogeneous computing card software stack also requires corresponding adaptation.

[0056] According to the embodiments in this specification, based on Figure 4 The heterogeneous computing system architecture shown can conveniently and cost-effectively implement streaming encryption / decryption and streaming transmission, thereby improving encrypted transmission efficiency. According to this embodiment, during H2D transmission, the target software module in kernel mode streams plaintext data slices, encrypts the data slices, and informs the target hardware module. The target hardware module streams the data slices via a first bus (e.g., by moving data to memory using PCIe read requests), streams and decrypts them, and then stores them in the plaintext data space on the heterogeneous computing card side via a second bus.

[0057] Specifically, when data needs to be transferred from the host side to the heterogeneous computing card, the system obtains the data block to be transferred (hereinafter referred to as the first data block) from the user-space application and stores it in a buffer (e.g., a DMA cache) for bus transfer. The aforementioned user-space application can be various applications, such as large model applications. The first data block can be a batch of user data, such as parameter matrices and tensors required for large model inference, user input text, etc.

[0058] like Figure 7 As shown in the lower half, during time period T0, the target software module reads data slice 1 from the buffer used for bus transmission and encrypts it using encryption / decryption parameters. The size of data slice 1 and subsequent data slices read is determined by the size limit of the data packets to be transmitted via the bus protocol. For example, in the case of a PCIe bus, the data packets transmitted via the first bus are PCIe format TLPs, which have a header and a payload. The size of each data slice is determined based on the maximum acceptable length of the payload, i.e., ensuring that the size of the encrypted data does not exceed the maximum acceptable length of the payload.

[0059] During time interval T1 following T0, data slice 1 is encrypted, and the target software module transmits the ciphertext 1 corresponding to data slice 1 to the target hardware module via the first bus. Specifically, in the case of a PCIe bus, the target software module can transmit a first data packet conforming to the PCIe protocol, the payload of which includes ciphertext 1. Simultaneously, the target software module continues to read data slice 2 from the aforementioned buffer and encrypts data slice 2 using encryption parameters.

[0060] During time interval T2 following T1, the transmission of ciphertext 1 is completed. The target hardware module decrypts ciphertext 1 corresponding to data slice 1 and transmits the recovered data slice 1 to the heterogeneous computing card via the second bus.

[0061] Meanwhile, during time period T2, the encryption of data slice 2 is complete. The target software module can transmit the ciphertext 2 corresponding to data slice 2 to the target hardware module via the first bus. Specifically, a second data packet conforming to the PCIe protocol can be transmitted, the payload of which includes ciphertext 2. At the same time, the target software module continues to read data slice 3 from the aforementioned buffer and encrypts data slice 3.

[0062] The above process continues until all n data slices in the first data block have been transmitted to the heterogeneous computing card. The correlation and continuity of the data slices originating from the same data block can be indicated using the bus protocol specification. For example, in the case of a PCIe bus, n PCIe data packets containing ciphertext corresponding to n data slices have correlated transaction descriptors, which include one of the following: requester ID, tag, or address. For example, the data packets may have the same tag, or have consecutive addresses, etc. These transaction descriptors can be transparently transmitted to the heterogeneous computing card, which then reconstructs the n data slices into the first data block. This is a function already available to external devices under the PCIe bus mechanism.

[0063] Unlike conventional schemes, in the above process, the encryption operation of the target software module on the (i+1)th data slice, the transmission operation of the ith data slice on the first bus, and the decryption operation of the target hardware module on the (i-1)th data slice can be executed synchronously and in parallel, forming a pipelined process. The decryption operation can begin as soon as the transmission of a data slice is completed, without waiting for the encryption and transmission of all data blocks. Figure 7 As can be seen from the comparison in the first half, this streaming processing greatly speeds up the efficiency of H2D data encryption and decryption transmission and shortens the time.

[0064] Furthermore, as mentioned earlier, implementing streaming encryption based on existing heterogeneous computing card software stacks requires data segmentation at the application layer, introducing more workload for adapting to heterogeneous computing card software stacks, and adding more stream scheduling tasks after data packet segmentation. In addition, differences in alignment after segmentation may introduce more data link layer packets, leading to a decrease in overall memory migration performance. However, in the above embodiment, data segmentation and streaming encryption are performed through the target software module at the kernel layer. It should be understood that when the kernel layer transmits data via the bus, it inherently needs to segment the data into packets of the size required by the bus protocol. Therefore, the method in the above embodiment, which segments data blocks according to bus protocol packets before encryption, does not introduce additional segmentation operations or additional data link layer packets. Furthermore, the streaming processing in the above embodiment does not require modification of the heterogeneous computing card software stack and does not introduce redundant scheduling tasks at the application layer. Thus, streaming encrypted transmission after data segmentation is achieved in an extremely "lightweight" manner.

[0065] Figure 8 A schematic diagram of D2H data transmission in one embodiment is shown. (See diagram below.) Figure 8 As shown, in conventional dense-state transmission, for example Figure 3 In the encrypted transmission between vendor A and vendor B, when D2H transmission is required, the heterogeneous accelerator card hardware itself encrypts the entire plaintext data block, and then performs the D2H encrypted transmission using the memcpy operation. The data can only be decrypted after the heterogeneous computing card software stack receives the entire encrypted data block. This process is time-consuming.

[0066] According to the embodiments in this specification, such as Figure 8 As shown in the lower part, during D2H transmission, the target hardware module (via the second bus) streams the plaintext data from the heterogeneous computing card, encrypts it, and then streams it to the host-side encrypted data space via the first bus. The target software module streams the encrypted data, decrypts it, and saves it to the host-side plaintext data space for use by user-space applications.

[0067] Specifically, when data needs to be transferred from the heterogeneous computing card to the host side, the heterogeneous computing card inherently divides the data block to be transferred (hereinafter referred to as the second data block) into data slices for transmission via data packets of the bus protocol. This second data block can be the computation result after computation by the heterogeneous computing card, intermediate results of large model inference, final results, etc.

[0068] like Figure 8 As shown, during time period S0, the target hardware module obtains data slice 1 from the heterogeneous computing card via the second bus and encrypts data slice 1. Specifically, in the case of a PCIe bus, the target hardware module receives a data packet conforming to the PCIe protocol transmitted by the heterogeneous computing card via the second bus, and extracts data slice 1 from the payload portion of the data packet. Then, it encrypts data slice 1 using encryption parameters.

[0069] During time S1 following S0, data slice 1 is encrypted, and the target hardware module transmits the ciphertext 1 corresponding to data slice 1 to the host CPU via the first bus. Simultaneously, the target hardware module continues to acquire data slice 2 and encrypts it. Specifically, in the case of a PCIe bus, the target hardware module can transmit a third data packet conforming to the PCIe protocol via the first bus, the payload of which includes ciphertext 1. At the same time, the target hardware module receives a fourth data packet conforming to the PCIe protocol transmitted by the heterogeneous computing card via the second bus, extracts data slice 2 from the payload of the fourth data packet, and encrypts it using encryption parameters.

[0070] During the time interval S2 following S1, the transmission of ciphertext 1 is completed. The target software module decrypts ciphertext 1 and provides the recovered data fragment 1 to the user-space application.

[0071] Meanwhile, during this time period S2, the encryption of data slice 2 is complete. The target hardware module can transmit the ciphertext 2 corresponding to data slice 2 to the host CPU via the first bus. Specifically, ciphertext 2 can be included in the payload of a data packet conforming to the PCIe protocol and transmitted to the host CPU. At the same time, the target hardware module continues to obtain data slice 3 from the heterogeneous computing card and encrypts data slice 3.

[0072] The above process continues until all n data slices in the second data block have been transmitted to the host CPU. The correlation and continuity of the data slices originating from the same data block can be indicated using the bus protocol specification. For example, in the case of a PCIe bus, n PCIe data packets containing ciphertext corresponding to n data slices have correlated transaction descriptors, which include one of the following: requester ID, tag, or address.

[0073] In the above process, the encryption operation of the target hardware module on the (i+1)th data slice, the transmission operation of the ith data slice on the first bus, and the decryption operation of the target software module on the (i-1)th data slice can be executed synchronously and in parallel, forming a pipelined processing. The decryption operation can begin as soon as the transmission of a data slice is completed, without waiting for the encryption and transmission of all data blocks. Figure 8 As can be seen from the comparison in the first half, this streaming processing greatly speeds up the efficiency of D2H data encryption and decryption transmission and shortens the time.

[0074] Furthermore, similar to the aforementioned analysis, in the above embodiments, data blocks are segmented according to bus protocol data packets and transmitted in encrypted streaming mode without introducing additional segmentation operations or additional data link layer packets. Moreover, the streaming processing in the above embodiments does not require modification of heterogeneous computing card hardware and does not introduce redundant scheduling tasks at the application layer, thus facilitating the secure streaming transmission of data.

[0075] Furthermore, the inventors discovered that PCIe transmission bandwidth is typically much greater than the encryption / decryption bandwidth of a single encryption / decryption engine on the host side. For example, in Figure 7In this context, the time required to transmit the i-th data segment via the first bus is generally significantly less than the time required for the target software module to encrypt the (i+1)-th data segment. This forces the bus transmission to wait for the encryption operation. Therefore, multiple encryption / decryption engines can be used in parallel to increase the encryption / decryption bandwidth and improve the overall performance of encrypted transmission. Specifically, the host CPU can use a multi-core CPU, with each CPU core acting as an encryption / decryption engine. The CPU core acting as the encryption / decryption engine will be referred to as an engine core. Thus, the host CPU can include m engine cores. The number of engine cores depends on the hardware configuration.

[0076] Figure 9 This diagram illustrates H2D data transmission in a multi-engine core embodiment. Figure 9 The overall architecture of midstream encryption / decryption and transmission Figure 7 The second half is similar. The difference is that after the target software module reads any data slice i from the cache, it divides it into m sub-data slices, distributes them to m engine cores, and uses the m engine cores to encrypt the m sub-data slices in parallel. During transmission, the ciphertexts of the m sub-data slices are merged into the ciphertext i corresponding to data slice i, and transmitted through the first bus. The processing on the target hardware module side remains unchanged.

[0077] Figure 10 This diagram illustrates D2H data transmission in a multi-engine core embodiment. Figure 10 The overall architecture of midstream encryption / decryption and transmission Figure 8 The second half is similar. The difference is that after the target software module receives and obtains the ciphertext i corresponding to any data slice i via the first bus, it divides it into m sub-ciphertext slices, distributes them to m engine cores, and uses the m engine cores to decrypt the m sub-ciphertext slices in parallel. After decryption, the m sub-data slices are combined into data slice i. Finally, the entire data block is recovered and provided to the user-space application. The processing on the target hardware module side remains unchanged.

[0078] In the above process, by using multiple engine cores to perform encryption / decryption in parallel, the encryption and decryption time of the target software module can be shortened, the waiting time for bus transmission can be reduced, and the overall efficiency can be further improved.

[0079] In summary, the embodiments described in this specification ensure secure data transmission between the CPU and the heterogeneous computing card by providing an encryption / decryption software module (target software module) on the CPU side and an encryption / decryption hardware module (target hardware module) on the heterogeneous computing device side. Through the cooperation of the encryption / decryption software module and the encryption / decryption hardware module, encrypted data is always transmitted on the exposed bus between the CPU and the heterogeneous computing device, thereby preventing bus attacks. Furthermore, the target software module and the target hardware module employ streaming encryption, transmission, and decryption processing, improving the efficiency of encrypted transmission on the bus without increasing additional data segmentation costs.

[0080] According to another embodiment, a heterogeneous computing system is also provided, including: a host CPU, and a target hardware module and a heterogeneous computing card packaged as a single unit in a non-removable manner, wherein the target hardware module is connected to the CPU via a first bus and to the heterogeneous computing card via a second bus, and the host CPU runs a target software module at the system kernel level, wherein:

[0081] The target software module is used to transmit the first ciphertext corresponding to the first data slice to the target hardware module through the first bus during a first time period, and simultaneously encrypt the second data slice; the first data slice and the second data slice both belong to the first data block, and the first data block comes from the user-space application.

[0082] The target hardware module is used to decrypt the first ciphertext in a second time period following the first time period, and transmit the decrypted and recovered first data fragment to the heterogeneous computing card through a second bus.

[0083] According to another embodiment, a heterogeneous computing system is also provided, including: a host CPU, and a target hardware module and a heterogeneous computing card packaged as a single unit in a non-removable manner, wherein the target hardware module is connected to the CPU via a first bus and to the heterogeneous computing card via a second bus, and the host CPU runs a target software module at the system kernel level, wherein:

[0084] The target hardware module is used to transmit the third ciphertext corresponding to the third data slice to the host CPU through the first bus during the third time period, and at the same time encrypt the fourth data slice; the third data slice and the fourth data slice both belong to the second data block, and the second data block comes from the heterogeneous computing card.

[0085] The target software module is used to decrypt the third ciphertext in a fourth time period following the third time period, and provide the decrypted and recovered third data fragment to the user-space application.

[0086] This specification also provides a computer-readable non-volatile storage medium storing a computer program that, when executed by a processor, can be used to perform one or more steps of one or more methods described or illustrated herein, or to provide the functionality described or illustrated herein. In this document, a computer-readable non-volatile storage medium or medium may include one or more semiconductor or other integrated circuits (ICs) (e.g., field-programmable gate arrays (FPGAs) or application-specific integrated circuits (ASICs)), hard disk drives (HDDs), hybrid hard disk drives (HHDs), optical disks, optical disk drives (ODDs), magneto-optical disks, magneto-optical drives, floppy disks, floppy disk drives (FDDs), magnetic tape, solid-state drives (SSDs), RAM drives, secure digital cards or drives, any other suitable computer-readable non-volatile storage medium, or any suitable combination thereof where appropriate. A computer-readable non-volatile storage medium may be volatile, non-volatile, or a combination of volatile and non-volatile.

[0087] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0088] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0089] The above description is merely an embodiment of this specification and is not intended to limit this specification. Various modifications and variations can be made to this specification by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this specification should be included within the scope of the claims of this application.

Claims

1. A method for transmitting data in a heterogeneous computing system, the heterogeneous computing system comprising a host CPU, a target hardware module and a heterogeneous computing card encapsulated as a whole structure in a manner of anti-disassembly, wherein the target hardware module is connected to the CPU through a first bus and connected to the heterogeneous computing card through a second bus, and a target software module at a system kernel level is running on the host CPU, the method comprising: in a first time period, the target software module transmits first ciphertext corresponding to a first data piece to the target hardware module through the first bus, while encrypting a second data piece; the first data piece and the second data piece belong to a first data block, and the first data block is from a user application; in a second time period following the first time period, the target hardware module decrypts the first ciphertext, and transmits the first data piece recovered by decryption to the heterogeneous computing card through the second bus. The first bus and the second bus are both PCIe buses. The first ciphertext is transmitted in a first data packet conforming to a PCIe protocol, and a payload part of the first data packet includes the first ciphertext.

2. The method of claim 1, wherein, The second data piece is encrypted by: reading a data piece conforming to a preset size from a buffer for bus transmission as the second data piece, wherein the preset size is a payload size of each data packet under a PCIe protocol; and 3. The method of claim 2, wherein, encrypting the second data piece using an encryption parameter. 4.The method of claim 1, further comprising, in the second time period, the target software module transmitting second ciphertext corresponding to the second data piece to the target hardware module through the first bus, while encrypting other data pieces. The first bus and the second bus are both PCIe buses. The first ciphertext is included in a first data packet conforming to a PCIe protocol, and the second ciphertext is included in a second data packet conforming to a PCIe protocol, and transaction descriptors of the first data packet and the second data packet have relevance, and the transaction descriptors include one of the following: a requester ID, a tag, and an address.

5. The method of claim 4, wherein, The host CPU comprises m engine cores; and the second data piece is encrypted by: encrypting m second sub-data pieces in parallel using the m engine cores; and the m second sub-data pieces constitute the second data piece.

6. The method of claim 1, wherein, The first bus and the second bus are both PCIe buses; and the method further comprises: reading the second data piece according to a payload size of each data packet under a PCIe bus protocol; 7. The method of claim 6, wherein, splitting the second data piece into m sub-data pieces and distributing the m sub-data pieces to the m engine cores. 8.A method for transmitting data in a heterogeneous computing system, the heterogeneous computing system comprising a host CPU, a target hardware module and a heterogeneous computing card encapsulated as a whole structure in a manner of anti-disassembly, wherein the target hardware module is connected to the CPU through a first bus and connected to the heterogeneous computing card through a second bus, and a target software module at a system kernel level is running on the host CPU, the method comprising: ​ ​ In a third time period, the target hardware module transmits third ciphertext corresponding to a third data piece to the host CPU through the first bus, while encrypting a fourth data piece; the third data piece and the fourth data piece belong to a second data block, and the second data block comes from a heterogeneous computing card; In a fourth time period following the third time period, the target software module decrypts the third ciphertext, and provides the third data piece recovered by decryption to a user application.

9. The method of claim 8, wherein, The first bus and the second bus are both PCIe buses; The third ciphertext corresponding to the third data piece includes a third data packet conforming to a PCIe protocol, and a payload part of the third data packet includes the third ciphertext.

10. The method of claim 9, wherein, Encrypting the fourth data piece includes: Receiving a fourth data packet conforming to a PCIe protocol transmitted by the heterogeneous computing card through the second bus, and extracting the fourth data piece from a payload part of the fourth data packet; Encrypting the fourth data piece using an encryption parameter.

11. The method of claim 8, further comprising, in the fourth time period, the target hardware module transmitting fourth ciphertext corresponding to the fourth data piece to the host CPU through the first bus, while encrypting a fifth data piece.

12. The method of claim 11, wherein, The first bus and the second bus are both PCIe buses; The third ciphertext is included in a third data packet conforming to a PCIe protocol, and the fourth ciphertext is included in a fifth data packet conforming to a PCIe protocol; transaction descriptors of the third data packet and the fifth data packet have relevance; and the transaction descriptors include one of the following: a requester ID, a tag, and an address.

13. The method of claim 8, wherein, The host CPU includes m engine cores; Decrypting the third ciphertext by the target software module includes: Decrypting m third sub-ciphertext pieces in parallel using the m engine cores; The m third sub-ciphertext pieces constitute the third ciphertext.

14. The method of claim 13, wherein, The first bus and the second bus are both PCIe buses; and the method further includes: Obtaining the third ciphertext according to a payload of a PCIe data packet transmitted through the first bus; Splitting the third ciphertext into m third sub-ciphertext pieces, and distributing the m third sub-ciphertext pieces to the m engine cores.

15. A heterogeneous computing system comprising: A host CPU, and a target hardware module and a heterogeneous computing card packaged as an integral structure in a tamper-proof manner, wherein the target hardware module is connected to the CPU through a first bus, and connected to the heterogeneous computing card through a second bus; and a target software module running at a system kernel level on the host CPU, wherein: The target software module is configured to, in a first time period, transmit first ciphertext corresponding to a first data piece to the target hardware module through the first bus, while encrypting a second data piece; the first data piece and the second data piece belong to a first data block, and the first data block comes from a user application; The target hardware module is configured to, in a second time period following the first time period, decrypt the first ciphertext, and transmit the first data piece recovered by decryption to the heterogeneous computing card through the second bus.

16. A heterogeneous computing system comprising: A host CPU, a target hardware module and a heterogeneous computing card packaged as an integral structure in a tamper-proof manner, wherein the target hardware module is connected to the CPU through a first bus and connected to the heterogeneous computing card through a second bus, and a target software module at a system kernel level is run on the host CPU, wherein: the target hardware module is configured to transmit third ciphertext corresponding to a third data piece to the host CPU through the first bus and encrypt a fourth data piece in a third time period; the third data piece and the fourth data piece belong to a second data block, and the second data block comes from the heterogeneous computing card; the target software module is configured to decrypt the third ciphertext in a fourth time period succeeding the third time period and provide the third data piece recovered by decryption to a user application. 17.A computer readable storage medium having stored thereon a computer program, which, when executed in a computer, causes the computer to perform the method of any one of claims 1-14.

Images